I am trying to insert a line in the beginning of a file using sed.
I tried below commands :
sed -i '1s/^/LINE TO INSERT\n/' test.txt
sed: illegal option -- i --> Error thrown
sed '1i/^/LINE TO INSERT\n/' test.txt
sed: Function 1i/^/LINE TO INSERT\n/ cannot be parsed. --> Error thrown
Both the ways came out to be failed.
Any possible solution to it ? I am using ksh script on HP-UX.
Thanks.
How about good old ed?
printf '%s\n' 1i 'LINE TO INSERT' . w | ed -s file
printf is used to send each command to ed on a separate line.
Alternatively, if you're terrified of ed like me, you can just use a temporary file, as suggested in the comments:
echo 'LINE TO INSERT' > tmp && cat tmp test > new && mv new test && rm tmp
I think you have a typo: you're missing the closing apostrophe from your 1st command. Otherwise it's fine. I.e.:
You have this: sed -i '1s/^/... test.txt
But you need this: sed -i '1s/^/...' test.txt
Putting all together: sed -i '1s/^/LINE TO INSERT\n/' test.txt
Update: if -i is not supported, then you can use a temporary file:
sed '1s/^/LINE TO INSERT\n/' test.txt > /tmp/test.txt.tmp
mv /tmp/test.txt.tmp test.txt
Related
I need to write a shell script to insert a parameter string after every big letter in a file.
$parameter="4"
Example input.txt
AppLe
House
Example output.txt
A4ppL4e
H4ouse
I've tried to use
sed '/[A-Z]/i\$1\'
Can anyone help me?
THX
With GNU/BSD/busybox sed which support the -i option:
param=4
sed -i'' -e 's/\([[:upper:]]\)/\1'"$param"'/g' input.txt
This replaces each uppercase letter inside the captured group \(...\) globally with the first captured group \1 and the value of variable param in-place.
With standard sed you need a temporary file or sponge from the moreutils package:
param=4
sed 's/\([[:upper:]]\)/\1'"$param"'/g' input.txt > temp && mv temp input.txt
param=4
sed 's/\([[:upper:]]\)/\1'"$param"'/g' input.txt | sponge input.txt
Use a file editor like ed to edit files:
printf "%s\n" 'g/[[:upper:]]/s/\([[:upper:]]\)/\1'"$param"'/g' w | ed -s input.txt
or if you like heredocs better
ed -s input.txt <<EOF
g/[[:upper:]]/s/\([[:upper:]]\)/\1${param}/g
w
EOF
I have these files
NotRequired.txt (having lines which need to be remove)
Need2CleanSED.txt (big file , need to clean)
Need2CleanGRP.txt (big file , need to clean)
content:
more NotRequired.txt
[abc-xyz_pqr-pe2_123]
[lon-abc-tkt_1202]
[wat-7600-1_414]
[indo-pak_isu-5_761]
I am reading above file and want to remove lines from Need2Clean???.txt, trying via SED and GREP but no success.
myFile="NotRequired.txt"
while IFS= read -r HKline
do
sed -i '/$HKline/d' Need2CleanSED.txt
done < "$myFile"
myFile="NotRequired.txt"
while IFS= read -r HKline
do
grep -vE \"$HKline\" Need2CleanGRP.txt > Need2CleanGRP.txt
done < "$myFile"
Looks as if the Variable and characters [] making some problem.
What you're doing is extremely inefficient and error prone. Just do this:
grep -vF -f NotRequired.txt Need2CleanGRP.txt > tmp &&
mv tmp Need2CleanGRP.txt
Thanks to grep -F the above treats each line of NotRequired.txt as a string rather than a regexp so you don't have to worry about escaping RE metachars like [ and you don't need to wrap it in a shell loop - that one command will remove all undesirable lines in one execution of grep.
Never do command file > file btw as the shell might decide to execute the > file first and so empty file before command gets a chance to read it! Always do command file > tmp && mv tmp file instead.
Your assumption is correct. The [...] construct looks for any characters in that set, so you have to preface ("escape") them with \. The easiest way is to do that in your original file:
sed -i -e 's:\[:\\[:' -e 's:\]:\\]:' "${myFile}"
If you don't like that, you can probably put the sed command in where you're directing the file in:
done < replace.txt|sed -e 's:\[:\\[:' -e 's:\]:\\]:'
Finally, you can use sed on each HKline variable:
HKline=$( echo $HKline | sed -e 's:\[:\\[:' -e 's:\]:\\]:' )
try gnu sed:
sed -Ez 's/\n/\|/g;s!\[!\\[!g;s!\]!\\]!g; s!(.*).!/\1/d!' NotRequired.txt| sed -Ef - Need2CleanSED.txt
Two sed process are chained into one by shell pipe
NotRequired.txt is 'slurped' by sed -z all at once and substituted its \n and [ meta-char with | and \[ respectively of which the 2nd process uses it as regex script for the input file, ie. Need2CleanSED.txt. 1st process output;
/\[abc-xyz_pqr-pe2_123\]|\[lon-abc-tkt_1202\]|\[wat-7600-1_414\]|\[indo-pak_isu-5_761\]/d
add -u ie. unbuffered, option to evade from batch process, sort of direct i/o
I've writted a sed script to replace all ^^ with NULL. It seems though that sed is only catching a pair, but not including the second in that pair as it continues to search.
echo "^^^^" | sed 's/\^\^/\^NULL\^/g'
produces
^NULL^^NULL^
when it should produce
^NULL^NULL^NULL^
Try with a loop to apply your command again to modified pattern space:
echo "^^^^" | sed ':a;s/\^\^/\^NULL\^/;t a;'
To edit a file in place on OSX, try the -i flag and multiline command:
sed -i '' ':a
s/\^\^/\^NULL\^/
t a' file
With GNU sed:
sed -i ':a;s/\^\^/\^NULL\^/;t a;' file
or simply redirect the command to a temporary file before renaming it:
sed ':a;s/\^\^/\^NULL\^/;t a;' file > tmp && mv tmp file
I really like SLePort solution, but since it is not working for you, you can try with (tested on Linux, not Mac):
echo "^^^^" | sed 's/\^\^/\^NULL\^/g; s//\^NULL\^/g'
It is doing the same as the former solution, but explicitly, not looping with tags.
You can omit the pattern in the second command and sed will use the previous pattern.
I am on RHEL 7.3. Line 12 of myfile.txt looks like:
image: /currentimage/myimage
I need a bash script to change it to:
image: /newimage/otherimage
I tried doing it this way:
sed -i '12s/image: /currentimage/myimage/image: /newimage/otherimage/' ./myfile
But it fails with:
sed: unknown option to `s'
You are using / as your sed separator, AND it is used in your paths. Try using | as your separator instead.
sed -i '12s|image: /currentimage/myimage|image: /newimage/otherimage|' ./myfile
Additionally you can escape each / in the file path like so \/ .
If you can use Awk, it is pretty simple to move on the 12th line using the NR variable which represents the line being processed,
awk 'NR==12{$2="/newimage/otherimage/"}1' myfile > tmp && mv tmp myfile
The part > tmp && mv tmp myfile is equivalent to to the -i option in sed for an indirect way to do modify the file in-place.
So, disclaimer: I am pretty new to using bash and zsh, so there is a chance the answer is really simple. Nonetheless. I checked previous postings and couldn't find anything. (edit: I have tried this in both bash and zsh shells- same problem.)
I have a directory with many files and am trying to remove the first line from each file.
So say the directory contains: file1.txt file2.txt file3.txt ... etc.
I am using the sed command (non-GNU):
sed -i -e "1d" *.txt
For some reason, this is only removing the first line of the first file. I thought that the *.txt would affect all files matching the pattern in directory. Strangely, it is creating the file duplicates with -e appended, but both the duplicate and original are the same.
I tried this with other commands (e.g. ls *.txt) and it works fine. Is there something about sed I am missing?
Thank you in advance.
Different versions of sed in differing operating systems support various parameters.
OpenBSD (5.4) sed
The -i flag is unavailable. You can use the following /bin/sh syntax:
for i in *.txt
do
f=`mktemp -p .`
sed -e "1d" "${i}" > "${f}" && mv -- "${f}" "${i}"
done
FreeBSD (11-CURRENT) sed
The -i flag requires an extension, even if it's empty. Thus must be written as sed -i "" -e "1d" *.txt
GNU sed
This looks to see if the argument following -i is another option (or possibly a command). If so, it assumes an in-place modification. If it appears to be a file extension such as ".bak", it will rename the original with the ".bak" and then modify it into the original file's name.
There might be other variations on other platforms, but those are the three I have at hand.
use it without -e !
for one file use:
sed -i '1d' filename
for all files use :
sed -i '1d' *.txt
or
files=/path/to/files/*.extension ; for var in $files ; do sed -i '1d' $var ; done
.for me i use ubuntu and debian based systems , this method is working for me 100% , but for other platformes i'm not sure , so this is other method :
replace first line with emty pattern , and remove empty lines , (double commands):
for files in $(ls /path/to/files/*.txt); do sed -i "s/$(head -1 "$files")//g" "$files" ; sed -i '/^$/d' "$files" ; done
Note: if your files contain splash '/' , then it will give error , so in this case sed command should look like this ( sed -i "s[$(head -1 "$files")[[g" )
hope that's what you're looking for :)
The issue here is that the line number isn't reset when sed opens a new file, so 1 only matches the first line of the first file.
One solution is to use a shell loop, calling sed once for each file. Gumnos' answer shows how to do this in the most widely compatible way, although if you have a version of sed supporting the -i flag, you could do this instead:
for i in *.txt; do
sed -i.bak '1d' "$i"
done
It is possible to avoid creating the backup file by passing an empty suffix but personally, I don't think it's such a bad thing. One day you'll be grateful for it!
It appears that you're not working with GNU tools but if you were, I would recommend using GNU awk for this task. The variable FNR is useful here, as it keeps track of the record number for each file individually, allowing you to do this:
gawk -i inplace 'FNR>1' *.txt
Using the inplace extension, this allows you to remove the first line from each of your files, by only printing the lines where FNR is greater than 1.
Testing it out:
$ seq 5 > file1
$ seq 5 > file2
$ gawk -i inplace 'FNR>1' file1 file2
$ cat file1
2
3
4
5
$ cat file2
2
3
4
5
The last argument you are passing to the Sed is the problem
try something like this.
var=(`find *txt`)
for file in "${var[#]}"
do
sed -i -e 1d $file
done
This did the trick for me.