How to properly use sed in for loop? I want to loop through an array and each loop, open a file template.txt, do spring replacement and output result to a file. For example:
values=( "val_1982" "val_1985" "val_1987" )
for i in "${values[#]}"
do
sed -e "s/\${VAL_ID}/$i/" \
-e "s/\${ENTRY}/"local"/" template.txt > outputfile_$i.txt
done
But I get:
bash: template.txt: command not found
bash: template.txt: command not found
bash: template.txt: command not found
The output files are created but all empty.
I replaced with this and it worked:
for i in val_1982 val_1985 val_1987
do
sed -e "s/\${VAL_ID}/$i/" \
-e "s/\${ENTRY}/"local"/" template.txt > outputfile_$i.txt
done
Related
I need to use sed instead of echo and append to the end of a file: list.txt
list.txt has a list of directories:
/desktop/test1/file1
/desktop/test2/file1
I need to append another directory with slashes to this list.txt at the end of the file if that directory already doesn't exist in the list. Such as: /desktop/file1
The result should be:
/desktop/test1/file1
/desktop/test2/file1
/desktop/file1
I've tried using this script but am running into syntax errors with the a command which I've been seeing could be a mac issue?:
#!/bin/bash
if ! grep -q "/desktop/file1" user/admin/Desktop/list.txt; then
sed -i -e '$a/desktop/file1' user/admin/Desktop/list.txt
fi
The a command is used as a\ in standard sed and should be followed by a newline and the text to be written. The form you use is a GNU extension. So a standard sed command to do the job could be:
sed -i '' '$a\
/desktop/file1' user/admin/Desktop/list.txt
or
sed -i '' '$a\'$'\n''/desktop/file1' user/admin/Desktop/list.txt
using ANSI-C quoting in bash.
I have these files
NotRequired.txt (having lines which need to be remove)
Need2CleanSED.txt (big file , need to clean)
Need2CleanGRP.txt (big file , need to clean)
content:
more NotRequired.txt
[abc-xyz_pqr-pe2_123]
[lon-abc-tkt_1202]
[wat-7600-1_414]
[indo-pak_isu-5_761]
I am reading above file and want to remove lines from Need2Clean???.txt, trying via SED and GREP but no success.
myFile="NotRequired.txt"
while IFS= read -r HKline
do
sed -i '/$HKline/d' Need2CleanSED.txt
done < "$myFile"
myFile="NotRequired.txt"
while IFS= read -r HKline
do
grep -vE \"$HKline\" Need2CleanGRP.txt > Need2CleanGRP.txt
done < "$myFile"
Looks as if the Variable and characters [] making some problem.
What you're doing is extremely inefficient and error prone. Just do this:
grep -vF -f NotRequired.txt Need2CleanGRP.txt > tmp &&
mv tmp Need2CleanGRP.txt
Thanks to grep -F the above treats each line of NotRequired.txt as a string rather than a regexp so you don't have to worry about escaping RE metachars like [ and you don't need to wrap it in a shell loop - that one command will remove all undesirable lines in one execution of grep.
Never do command file > file btw as the shell might decide to execute the > file first and so empty file before command gets a chance to read it! Always do command file > tmp && mv tmp file instead.
Your assumption is correct. The [...] construct looks for any characters in that set, so you have to preface ("escape") them with \. The easiest way is to do that in your original file:
sed -i -e 's:\[:\\[:' -e 's:\]:\\]:' "${myFile}"
If you don't like that, you can probably put the sed command in where you're directing the file in:
done < replace.txt|sed -e 's:\[:\\[:' -e 's:\]:\\]:'
Finally, you can use sed on each HKline variable:
HKline=$( echo $HKline | sed -e 's:\[:\\[:' -e 's:\]:\\]:' )
try gnu sed:
sed -Ez 's/\n/\|/g;s!\[!\\[!g;s!\]!\\]!g; s!(.*).!/\1/d!' NotRequired.txt| sed -Ef - Need2CleanSED.txt
Two sed process are chained into one by shell pipe
NotRequired.txt is 'slurped' by sed -z all at once and substituted its \n and [ meta-char with | and \[ respectively of which the 2nd process uses it as regex script for the input file, ie. Need2CleanSED.txt. 1st process output;
/\[abc-xyz_pqr-pe2_123\]|\[lon-abc-tkt_1202\]|\[wat-7600-1_414\]|\[indo-pak_isu-5_761\]/d
add -u ie. unbuffered, option to evade from batch process, sort of direct i/o
suppose a file that contains:
a=1234
b=5678
c=word
i am writing a bash script that will change a's value. i use following command:
echo `sed "s/\(a=\).*/\1 new/" file` >file
it gives:
a=new b=5678 c=word
i also tried following (using anchors) still getting same format:
echo `sed "s/\(a=\).*[\n]$/\1 new/" file` >file
echo `sed "s/\(a=\).*'\n'$/\1 new/" file` >file
where i want:
a=new
b=5678
c=word
how to do that?
Use the -i argument to edit the file in place:
sed -i 's/\(a=\).*/\1new/' file
You could a following sed too once.
sed -i '/^a=/s/=.*/=new/g' Input_file
I am trying to use sed in bash script as follows:
#!/bin/bash
for i in `seq 1 10`;
do
j=$(($i-1))
OLD="-option_something something/string1_${j}.txt"
NEW="-option_somehting something/string1_${i}.txt"
sed -e "s/$OLD/$NEW/g" file_to_edit.txt
# sed -e "s/$OLD/$NEW/g" file_to_edit.txt > file_to_edit.txt.tmp && mv file_to_edit.txt.tmp file_to_edit.txt
done
But I keep getting following error:
sed: -e expression #1, char 71: unknown option tos'`
I tried the commented line as well, but it does not work too.
It works fine on command line. I do not know what is the problem in script.
Any suggestions? Thanks.
You have a / in the value of OLD and NEW, which is the same character you're using as the delimiter in your sed expression. So the final expression ends up looking like:
sed -e "s/-option_something something/string1_${j}.txt/-option_somehting something/string1_${i}.txt/g"
Do you see all the / in there? Consider instead:
sed -e "s|$OLD|$NEW|g" file_to_edit.txt
You can use any character as the delimiter for sed's s command.
I am trying to insert a line in the beginning of a file using sed.
I tried below commands :
sed -i '1s/^/LINE TO INSERT\n/' test.txt
sed: illegal option -- i --> Error thrown
sed '1i/^/LINE TO INSERT\n/' test.txt
sed: Function 1i/^/LINE TO INSERT\n/ cannot be parsed. --> Error thrown
Both the ways came out to be failed.
Any possible solution to it ? I am using ksh script on HP-UX.
Thanks.
How about good old ed?
printf '%s\n' 1i 'LINE TO INSERT' . w | ed -s file
printf is used to send each command to ed on a separate line.
Alternatively, if you're terrified of ed like me, you can just use a temporary file, as suggested in the comments:
echo 'LINE TO INSERT' > tmp && cat tmp test > new && mv new test && rm tmp
I think you have a typo: you're missing the closing apostrophe from your 1st command. Otherwise it's fine. I.e.:
You have this: sed -i '1s/^/... test.txt
But you need this: sed -i '1s/^/...' test.txt
Putting all together: sed -i '1s/^/LINE TO INSERT\n/' test.txt
Update: if -i is not supported, then you can use a temporary file:
sed '1s/^/LINE TO INSERT\n/' test.txt > /tmp/test.txt.tmp
mv /tmp/test.txt.tmp test.txt