I would like to run a find and replace on an HTML file through the command line.
My command looks something like this:
sed -e s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html > index.html
When I run this and look at the file afterward, it is empty. It deleted the contents of my file.
When I run this after restoring the file again:
sed -e s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html
The stdout is the contents of the file, and the find and replace has been executed.
Why is this happening?
When the shell sees > index.html in the command line it opens the file index.html for writing, wiping off all its previous contents.
To fix this you need to pass the -i option to sed to make the changes inline and create a backup of the original file before it does the changes in-place:
sed -i.bak s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html
Without the .bak the command will fail on some platforms, such as Mac OSX.
An alternative, useful, pattern is:
sed -e 'script script' index.html > index.html.tmp && mv index.html.tmp index.html
That has much the same effect, without using the -i option, and additionally means that, if the sed script fails for some reason, the input file isn't clobbered. Further, if the edit is successful, there's no backup file left lying around. This sort of idiom can be useful in Makefiles.
Quite a lot of seds have the -i option, but not all of them; the posix sed is one which doesn't. If you're aiming for portability, therefore, it's best avoided.
sed -i 's/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g' index.html
This does a global in-place substitution on the file index.html. Quoting the string prevents problems with whitespace in the query and replacement.
use sed's -i option, e.g.
sed -i bak -e s/STRING_TO_REPLACE/REPLACE_WITH/g index.html
To change multiple files (and saving a backup of each as *.bak):
perl -p -i -e "s/\|/x/g" *
will take all files in directory and replace | with x
this is called a “Perl pie” (easy as a pie)
You should try using the option -i for in-place editing.
Warning: this is a dangerous method! It abuses the i/o buffers in linux and with specific options of buffering it manages to work on small files. It is an interesting curiosity. But don't use it for a real situation!
Besides the -i option of sed
you can use the tee utility.
From man:
tee - read from standard input and write to standard output and files
So, the solution would be:
sed s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html | tee | tee index.html
-- here the tee is repeated to make sure that the pipeline is buffered. Then all commands in the pipeline are blocked until they get some input to work on. Each command in the pipeline starts when the upstream commands have written 1 buffer of bytes (the size is defined somewhere) to the input of the command. So the last command tee index.html, which opens the file for writing and therefore empties it, runs after the upstream pipeline has finished and the output is in the buffer within the pipeline.
Most likely the following won't work:
sed s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html | tee index.html
-- it will run both commands of the pipeline at the same time without any blocking. (Without blocking the pipeline should pass the bytes line by line instead of buffer by buffer. Same as when you run cat | sed s/bar/GGG/. Without blocking it's more interactive and usually pipelines of just 2 commands run without buffering and blocking. Longer pipelines are buffered.) The tee index.html will open the file for writing and it will be emptied. However, if you turn the buffering always on, the second version will work too.
sed -i.bak "s#https.*\.com#$pub_url#g" MyHTMLFile.html
If you have a link to be added, try this. Search for the URL as above (starting with https and ending with.com here) and replace it with a URL string. I have used a variable $pub_url here. s here means search and g means global replacement.
It works !
The problem with the command
sed 'code' file > file
is that file is truncated by the shell before sed actually gets to process it. As a result, you get an empty file.
The sed way to do this is to use -i to edit in place, as other answers suggested. However, this is not always what you want. -i will create a temporary file that will then be used to replace the original file. This is problematic if your original file was a link (the link will be replaced by a regular file). If you need to preserve links, you can use a temporary variable to store the output of sed before writing it back to the file, like this:
tmp=$(sed 'code' file); echo -n "$tmp" > file
Better yet, use printf instead of echo since echo is likely to process \\ as \ in some shells (e.g. dash):
tmp=$(sed 'code' file); printf "%s" "$tmp" > file
And the ed answer:
printf "%s\n" '1,$s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g' w q | ed index.html
To reiterate what codaddict answered, the shell handles the redirection first, wiping out the "input.html" file, and then the shell invokes the "sed" command passing it a now empty file.
I was searching for the option where I can define the line range and found the answer. For example I want to change host1 to host2 from line 36-57.
sed '36,57 s/host1/host2/g' myfile.txt > myfile1.txt
You can use gi option as well to ignore the character case.
sed '30,40 s/version/story/gi' myfile.txt > myfile1.txt
With all due respect to the above correct answers, it's always a good idea to "dry run" scripts like that, so that you don't corrupt your file and have to start again from scratch.
Just get your script to spill the output to the command line instead of writing it to the file, for example, like that:
sed -e s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html
OR
less index.html | sed -e s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g
This way you can see and check the output of the command without getting your file truncated.
in my script i edit a file by using "sed":
sed -i -e /NumberOfEntries=*/d "$playlist_name" > /dev/null # deletes "NumberOfEntries"-line
sed -i -e '/^$/d' "$playlist_name" > /dev/null # deletes all empty lines
This works just fine but while execution (and shortly after) there are one or two temporary files named something like "sedJjHEt2" (the string after sed changes everytime i run the script). The file is marked with a lock and red X-Symbol.
Is there a way to turn this off or am I missing something that I need to adjust? Or is this because of the "-i" Option?
Thanks in advance!
Because of the "-i" Option.
once you use "-i", sed will create a temporary, and then replace the original file with the temporary file.
I'm trying to write a bash function to delete a garbage string (including the quotation marks and parentheses) from a text file and resave the file with the same filename.
I got it to work when I save it as a new file...
function fixfile()
{
sed 's/(\"garbagestringhere\")//g' /Users/peter/.emacs.d/recent-addresses > /Users/peter/.emacs.d/recent-addresses-fixed;
}
...but when I save it with the same filename as before, it wipes the file:
function fixfile()
{
sed 's/(\"garbagestringhere\")//g' /Users/peter/.emacs.d/recent-addresses > /Users/peter/.emacs.d/recent-addresses;
}
How do I fix this?
For GNU sed man sed will reveal the -i option:
-i[SUFFIX], --in-place[=SUFFIX]
edit files in place (makes backup if SUFFIX supplied)
If you are on OSx (or BSD) you can employ (see this reference answer)
sed -i '' 's/whatever//g' file
// ^ note the space
Even if in your particular case I suggest the use of sed -i,
in a more general approach you can consider the use of sponge
from moreutils.
sed '...' file | grep '...' | awk '...' | sponge file
Sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before opening the output file. This allows constricting pipelines that read from and write to the same file.
If no output file is specified, sponge outputs to stdout
(Git from here, or from here...)
Alternatively you can use a temporary file that you delete before you exit from your function.
Temporary file with a random (or pseudorandom) namefile.
I have a template file I want to copy and then edit from a script, inserting content at specific template points. For example, my template file might be something like,
...
rm -rf SomeDirectory
make install
#{INSERT-CONTENT-HERE}
do-something-else
...
In another script, I want to add content at "#{INSERT-CONTENT-HERE}" within a loop, i.e.
for i in c; do
# Write content to the template file copy at the correct point.
done
I think sed is the right tool, but I'm not familiar enough to know the syntax, and the man page isn't helping.
An example:
echo "Line #{INSERT-CONTENT-HERE}" | sed 's/#{INSERT-CONTENT-HERE}/---/'
To modify a file:
sed -i 's/#{INSERT-CONTENT-HERE}/---#{INSERT-CONTENT-HERE}/' filename
where -i means in-place edit so be warned
if you do:
sed -i.bak 's/#{INSERT-CONTENT-HERE}/---/' filename
it should back up original as filename.bak
also to make multiple substitutions at each line use the g flag:
sed -i.bak 's/#{INSERT-CONTENT-HERE}/---/g' filename
You can copy the output of all the commands into a temporary file and then copy the contents of that entire file into the template file:
TEMPFILE=`mktemp` && (
for i in c
echo "SomeTextBasedOn $i" >> $TEMPFILE
done
sed -i '/{INSERT-CONTENT-HERE}/r '$TEMPFILE targetfile
rm $TEMPFILE
)
This question already has answers here:
How to delete from a text file, all lines that contain a specific string?
(21 answers)
Closed 5 years ago.
I want to delete a line containing a specific string from the file. How can I do this without using awk? I tried to use sed but I could not achieve it.
This should do it:
sed -e s/deletethis//g -i *
sed -e "s/deletethis//g" -i.backup *
sed -e "s/deletethis//g" -i .backup *
it will replace all occurrences of "deletethis" with "" (nothing) in all files (*), editing them in place.
In the second form the pattern can be edited a little safer, and it makes backups of any modified files, by suffixing them with ".backup".
The third form is the way some versions of sed like it. (e.g. Mac OS X)
man sed for more information.
sed -i '/pattern/d' file
Use 'd' to delete a line. This works at least with GNU-Sed.
If your Sed doesn't have the option, to change a file in place, maybe you can use an intermediate file, to store the modification:
sed '/pattern/d' file > tmpfile && mv tmpfile file
Writing directly to the source doesn't work: sed '/pattern/d' FILE > FILE so make a copy before trying out, if you doubt it. The redirection to a new file FILE will open a new file FILE before reading from it, so reading from it will result in an empty input.
Try the vim-way:
ex -s +"g/foo/d" -cwq file.txt