Imagine the following data stored in file data.txt
1, StringString, AnotherString 545
I want to replace "StringString" with "Strung" with the following code
sed -ir 's/String+/Strung/g' data.txt
But it won't work. This works though:
sed -ri 's/String+/Strung/g' data.txt
I don't see any reason why the order of option flags would matter. Is it a bug or is there an explanation?
Please note that I'm not looking for a workaround but rather why the order of -ir and -ri matters.
Sidenotes: The switch -i "edits the file in place" while -r allows "extended regular expression" (allowing the + operator). I'm running sed 4.2.1 Dec. 2010 on Ubuntu 12.10.
When doing -ir you are specifying that "r" should be the suffix for the backup file.
You should be able to do -i -r if you need them in that order
Did you check sed --help or man sed?
-i[SUFFIX], --in-place[=SUFFIX]
edit files in place (makes backup if extension supplied).
The default operation mode is to break symbolic and hard links.
This can be changed with --follow-symlinks and --copy.
I'm trying to undergo pdbqt-flexible files merge into one pdb using following script:
http://prosciens.com/prosciens/oldproscienssarl/files/flexrigidpdbqt2pdb_template.sh
Problematic fragment:
Let's merge the files
First we clean up the model PDB
grep ATOM ${FLEXPDBQT}_${MODEL}.pdb > ${FLEXPDBQT}_${MODEL}.pdb.tmp
Next we create a list of residues
cut -c 18-27 ${FLEXPDBQT}_${MODEL}.pdb.tmp > residuelistraw.tmp`
cat residuelistraw.tmp | uniq > residuelist.tmp
Then we split the model file into residues
while read r
do
rns= echo $r | sed 's/ //g'
egrep "[ \t]$r[ \t]" ${FLEXPDBQT}_${MODEL}.pdb.tmp > $rns.pdb.tmp
sed -i 's/'$FLEXPDBQT'_'$MODEL'.pdb.tmp://' $rns.pdb.tmp
Currently it fails at #3 step yielding following error:
/flexrigidpdbqt2pdb_template.sh: line 133: $ rns.pdb.tmp: ambiguous redirect
sed: -e expression # 1, character 9: unknown option for the `s' command
I tried fix the error using some sed substitution:
rns=`echo "${r/ /}"`
echo $rns
egrep "[ \t]$r[ \t]" ${FLEXPDBQT}_${MODEL}.pdb.tmp > $rns.pdb.tmp
sed -i 's/'$FLEXPDBQT'_'$MODEL'.pdb.tmp://' $rns.pdb.tmp
But so far nothing changed.
My sed version is 4.4
On the "ambiguous redirection" error
An "ambiguous redirection" error doesn't come from sed at all, it comes from your shell.
An "ambiguous redirection" error means your shell can't start the command you gave it at all, because it wasn't able to perform the redirections requested as part of that command's environment.
In that case, the variable rns is empty.
That's because rns= echo $r | sed 's/ //g' doesn't assign rns to be the output of sed at all. Instead, it assigns a transient environment variable named rns to be an empty string, only for the duration of execution of echo $r (the output of which is sent to sed, and from there to the script's stdout).
Instead, use:
rns=${r//[[:space:]]/}
...or, less efficiently:
rns=$(sed -e 's/ //g' <<<"$r")
To avoid the same error in cases where the variable isn't empty, be sure you quote!
That is, instead of running ... >$file, always run ... >"$file", to ensure that unwanted string-splitting or glob expansion can't make an otherwise-valid redirection unworkable. (This doesn't happen with all shell versions, but that means that failing to quote causes your code's behavior to be unpredictable unless you know which shell release it's going to be run with!).
On the "unknown option" error
If you use / as the sigil separating parts of your sed command, then you must not have any /s in the data being substituted. If you cannot guarantee this, use a different sigil instead of /; for example, s#/foo/bar#/baz/qux# works properly.
It looks like that the essential part to repair that script was changing rns as you proposed to:
rns=${r//[[:space:]]/}
and subsequent quoting of the output:
sed -i 's/'$FLEXPDBQT'_'$MODEL'.pdb.tmp://' "$rns.pdb.tmp"
However, further problems come out with next step, not previously shown:
sed '/'"HN ${r}"'/ r '${rns}'.pdb.tmp' ${RIGIDPDBQT}_${LIGNAME}_${MODEL}_apo.pdb > "${RIGIDPDBQT}_${LIGNAME}_${MODEL}_apo.pdb.tmp"
It yields no output (and no error), propably due to another problem with sed (which I really can't understand).
It is quite hard to give here a working example - a lot of preformatted txt files. Despite that, the problem about which I asked is solved, thank you!
For some mysterious reason, some elements in my CSV data appear as s/stWgvN52??f2& ?" instead of stWgvN522tw0JtZZnyXj, which messes up the file because I have ; set as the CSV delimiter.
I attempted to replace the defective string using sed as follows:
$ sed -i 's/stWgvN52??f2& ?"/stWgvN522tw0JtZZnyXj/g' file.csv
but I get the following error:
sed: 1: "access_logs_2014-04.csv": command a expects \ followed by text
What is the reason?
When you use the -i option, you have to specify the extension of the backup file that gets made. Some versions of sed expect the extension directly appended to the -i option, so what you wrote would work. But other versions (like the version on OS X) require it to be a separate option, so you have to write:
sed -i '' 's/stWgvN52??f2& ?"/stWgvN522tw0JtZZnyXj/g' file.csv
to specify that you don't want a backup file.
I am trying to search and replace the text of a single line with sed characters inside that line. The line is a file path thus it has /*. instead of it. I have searched and found n\ and opposing \ but non of these seem to work. Whatever I am doing wrong it's simple, but I can't see the tree through the forest.
data to edit:
Include conf.d/*.conf
I am trying to edit the
conf.d/*.conf
conf.d/vhosts/*.conf
Include is else where in the file, but this is the only occurrence where it's the first word. this is what I have so far:
sed -i "/^Include/ s/\conf.d/*.conf\/ */\conf.d/vhosts/*.conf\/" /etc/httpd/conf/httpd.conf
I've alsow tried
sed -i "/^Include/ s/[conf.d/*.conf]/[conf.d/vhosts/*.conf]/g" /etc/httpd/conf/httpd.conf
This is the standard search error message:
Error Message: sed: -e expression #1, char 29: unknown option to `s'
I know it's something simple, but I can't seem to figure it out. I have been stuck on this for the past hour. If I can get past this, then I can edit the documentroot and other elements with sed like characters.
sed version GNU sed version 4.2.1
sed -i "s|conf.d/\*.conf|conf.d/vhosts/\*.conf|g" /path/to/file
Changed Separator to |
Escaped asterisk
I've got a makefile (developed for gmake on Linux) that I'm attempting to port to MacOS, but it seems like sed doesn't want to cooperate. What I do is use GCC to autogenerate dependency files, and then tweak them a bit using sed. The relevant portion of the makefile:
$(OBJ_DIR)/%.d: $(SRC_DIR)/%.cpp
$(CPPC) -MM -MD $< -o $#
sed -i 's|\(.*\)\.o:|$(OBJ_DIR)/\1.o $(OBJ_DIR)/\1.d $(TEST_OBJ_DIR)/\1_utest.o:|' $#
While this runs with no trouble under GNU/Linux, I get errors like the following when attempting to build on MacOS:
sed: 1: "test/obj/equipmentConta ...": undefined label 'est/obj/equipmentContainer_utest.d'
sed: 1: "test/obj/dice_utest.d": undefined label 'est/obj/dice_utest.d'
sed: 1: "test/obj/color-string_u ...": undefined label 'est/obj/color-string_utest.d'
It would seem like sed is chopping off a character, but I can't see the solution.
OS X sed handles the -i argument differently to the Linux version.
You can generate a command that might "work" for both by adding -e in this way:
# vv
sed -i -e 's|\(.*\)\.o:|$(OBJ_DIR)/\1.o $(OBJ_DIR)/\1.d $(TEST_OBJ_DIR)/\1_utest.o:|' $#
OS X sed -i interprets the next thing after the -i as a file extension for a backup copy of the in-place edit. (The Linux version only does this if there is no space between the -i and the extension.) Obviously a side affect of using this is that you will get a backup file with -e as an extension, which you may not want. Please refer to other answers to this question for more details, and cleaner approaches that can be used instead.
The behaviour you see is because OS X sed consumes the s||| as the extension (!) then interprets the next argument as a command - in this case it begins with t, which sed recognizes as a branch-to-label command expecting the target label as an argument - hence the error you see.
If you create a file test you can reproduce the error:
$ sed -i 's|x|y|' test
sed: 1: "test": undefined label 'est'
Actually, doing
sed -i -e "s/blah/blah/" files
doesn't do what you expect in MacOS either. Instead it creates backup files with -e extension.
The proper command for MacOS is
sed -i "" -e "s/blah/blah/" files
On Linux, remove the space between -i and "" (see related answer)
sed -i"" -e "s/blah/blah/" files
The currently accepted answer is flawed in two very important ways.
With BSD sed (the OSX version), the -e option is interpreted as
a file extension and therefore creates a backup file with a -e
extension.
Testing for the darwin kernel as suggested is not a reliable
approach to a cross platform solution since GNU or BSD sed could
be present on any number of systems.
A much more reliable test would be to simply test for the --version option which is only found in the GNU version of sed.
sed --version >/dev/null 2>&1
Once the correct version of sed is determined, we can then execute the command in its proper syntax.
GNU sed syntax for -i option:
sed -i -- "$#"
BSD sed syntax for -i option:
sed -i "" "$#"
Finally put it all together in a cross platform function to execute an in place edit sed commend:
sedi () {
sed --version >/dev/null 2>&1 && sed -i -- "$#" || sed -i "" "$#"
}
Example usage:
sedi 's/old/new/g' 'some_file.txt'
This solution has been tested on OSX, Ubuntu, Freebsd, Cygwin, CentOS, Red Hat Enterprise, & Msys.
martin clayton's helpful answer provides a good explanation of the problem[1], but his solution - as he states - has a potentially unwanted side effect.
Here are side-effect-free solutions:
Caveat: Solving the -i syntax problem alone, as below, may not be enough, because there are many other differences between GNU sed and BSD/macOS sed (for a comprehensive discussion, see this answer of mine).
Workaround with -i: Create a backup file temporarily, then clean it up:
With a non-empty suffix (backup-file filename extension) option-argument (a value that is not the empty string), you can use -i in a way that works with both BSD/macOS sed and GNU sed, by directly appending the suffix to the -i option.
This can be utilized to create a backup file temporarily that you can clean up right away:
sed -i.bak 's/foo/bar/' file && rm file.bak
Obviously, if you do want to keep the backup, simply omit the && rm file.bak part.
Workaround that is POSIX-compliant, using a temporary file and mv:
If only a single file is to be edited in-place, the -i option can be bypassed to avoid the incompatibility.
If you restrict your sed script and other options to POSIX-compliant features, the following is a fully portable solution (note that -i is not POSIX-compliant).
sed 's/foo/bar' file > /tmp/file.$$ && mv /tmp/file.$$ file
This command simply writes the modifications to a temporary file and, if the sed command succeeds (&&), replaces the original file with the temporary one.
If you do want to keep the original file as a backup, add another mv command that renames the original first.
Caveat: Fundamentally, this is what -i does too, except that it tries to preserve permissions and extended attributes (macOS) of the original file; however, if the original file is a symlink, both this solution and -i will replace the symlink with a regular file.
See the bottom half of this answer of mine for details on how -i works.
[1] For a more in-depth explanation, see this answer of mine.
This isn't quite an answer to the question, but one can get linux-equivalent behavior through
brew install gnu-sed
# Add to .bashrc / .zshrc
export PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"
(previously there was a --with-default-names option to brew install gnu-sed but that has recently been removed)
I came across this issue as well and thought of the following solution:
darwin=false;
case "`uname`" in
Darwin*) darwin=true ;;
esac
if $darwin; then
sedi="/usr/bin/sed -i ''"
else
sedi="sed -i"
fi
$sedi 's/foo/bar/' /home/foobar/bar
Works for me ;-), YMMV
I work in a multi-OS team where ppl build on Windows, Linux and OS X. Some OS X users complained because they got another error - they had the GNU port of sed installed so I had to specify the full path.
I've corrected the solution posted by #thecarpy:
Here's a proper cross-platform solution for sed -i:
sedi() {
case $(uname) in
Darwin*) sedi=('-i' '') ;;
*) sedi='-i' ;;
esac
LC_ALL=C sed "${sedi[#]}" "$#"
}
I avoid using sed -i when writing scripts and i came up with simple solution:
printf '%s' "$(sed 's/foo/bar' file)" > file
much compatible and is POSIX-compliant. It is doing pretty much the same as sed -i, but this one does not create temp files, it directly redirect the changes to file.
As a noob idk what's the cons of doing this, the only matters is "It works"