I would like to find and replace a string in multiple files using bash command. I am using sed which I am not really familiar with.
My variables:
$FILE = (/home/user/file1.txt, /home/user/file2.txt)
$REL = 5.0
My code:
for f in ${FILES[#]}; do sed -i "$f" "s/__ver__ =*/__ver__=$REL/g";
output:
sed: -e expression #1, char 2: unknown command: `/'
sed: -e expression #1, char 2: unknown command: `/'
What is wrong with my expression?
1) The filename should be specified as the last argument for sed expression:
2) bash's for loop should ended with done keyword
for f in ${FILES[#]}; do sed -i "s/__ver__ =*/__ver__=$REL/" "$f"; done
If your files have similar naming format you can avoid for loop:
sed -i 's/__ver__ =*/__ver__=$REL/' /home/user/file[2].txt
Related
I can replace text with the following command:
for f in $(ls some/dir); do sed 's/XXX/YYY/g' template.$f.txt; done
But when I try to use in-place replacement I fail:
sed: 1: "template.test00.types ...": undefined label 'emplate.test00.types.txt'
What am I doing wrong?
EDIT: Here is a complete minimal example:
for f in $(ls some/dir); do echo $f && cat -n some/dir/$f; done
a.txt
1 path: XXX/725
b.txt
1 path: XXX/615
c.txt
1 path: XXX/931
Here is how sed works correctly without actual replacement:
for f in $(ls some/dir); do sed 's/XXX/YYY/g' some/dir/$f; done
path: YYY/725
path: YYY/615
path: YYY/931
And here is the failure:
for f in $(ls some/dir); do sed -i 's/XXX/YYY/g' some/dir/$f; done
sed: 1: "some/dir/a.txt": unterminated substitute pattern
sed: 1: "some/dir/b.txt": unterminated substitute pattern
sed: 1: "some/dir/c.txt": unterminated substitute pattern
I am using the sed command on Ubuntu to replace content.
This initial command comes from here.
sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"
However, as you can see, I have a slash in the replacement. The slash causes the command to throw:
sed: -e expression #1, char 9: unknown option to `s'
Moreover, my replacement is stored in a variable.
So the following will not work because of the slash:
sed -i "$ s/$/ $1/" "$DIR./result/doc.md"
As stated here and in duplicate, I should use another delimiter. If I try with #:
sed -i "$ s#$# $1#" "$DIR./result/doc.md"
It gives the error:
sed: -e expression #1, char 42: unterminated `s' command
My question is:
How can I use a variable in this command as well as other delimiter than / ?
Don't use sed here; perl and awk allow more robust approaches.
sed doesn't allow variables to be passed out-of-band from code, so they always need to be escaped. Use a language without that limitation, and you have code that always works, no matter what characters your data contains.
The Short Answer: Using perl
The below is taken from BashFAQ #21:
inplace_replace() {
local search=$1; shift; local replace=$1; shift
in="$search" out="$replace" perl -pi -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' "$#"
}
inplace_replace '#' "replacement" "$DIR/result/doc.md"
The Longer Answer: Using awk
...or, using awk to do a streaming replacement, and a shell function to make that file replacement instead:
# usage as in: echo "in should instead be out" | gsub_literal "in" "out"
gsub_literal() {
local search=$1 replace=$2
awk -v s="${search//\\/\\\\}" -v r="${rep//\\/\\\\}" 'BEGIN {l=length(s)} {o="";while (i=index($0, s)) {o=o substr($0,1,i-1) r; $0=substr($0,i+l)} print o $0}'
}
# usage as in: inplace_replace "in" "out" /path/to/file1 /path/to/file2 ...
inplace_replace() {
local search=$1 replace=$2 retval=0; shift; shift
for file; do
tempfile=$(mktemp "$file.XXXXXX") || { retval |= $?; continue; }
if gsub_literal "$search" "$replace" <"$file" >"$tempfile"; then
mv -- "$tempfile" "$file" || (( retval |= $? ))
else
rm -f -- "$tempfile" || (( retval |= $? ))
fi
done
}
TL;DR:
Try:
sed -i '$ s#$# '"$1"'#' "$DIR./result/doc.md"
Long version:
Let's start with your original code:
sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"
And let's compare it to the code you referenced:
sed -i '$ s/$/abc/' file.txt
We can see that they don't exactly match up. I see that you've correctly made this substitution:
file.txt --> "$DIR./result/doc.md"
That looks fine (although I do have my doubts about the . after $DIR ). However, the other substitution doesn't look great:
abc --> /replacement
You actually introduced another delimeter /. However, if we replace the delimiters with '#' we get this:
sed -i '$ s#$# /replacement#' "$DIR./result/doc.md"
I think that the above is perfectly valid in sed/bash. The $# will not be replaced by the shell because it is single quoted. The $DIR variable will be interpolated by the shell because it is double quoted.
Looking at one of your attempts:
sed -i "$ s#$# $1#" "$DIR./result/doc.md"
You will have problems due to the shell interpolation of $# in the double quotes. Let's correct that by replacing with single quotes (but leaving $1 unquoted):
sed -i '$ s#$# '"$1"'#' "$DIR./result/doc.md"
Notice the '"$1"'. I had to surround $1 with '' to basically unescape the surrounding single quotes. But then I surrounded the $1 with double quotes so we could protect the string from white spaces.
Use shell parameter expansion to add escapes to the slashes in the variable:
$ cat file
foo
bar
baz
$ set -- ' /repl'
$ sed "s/$/$1/" file
sed: 1: "s/$/ /repl/": bad flag in substitute command: 'r'
$ sed "s/$/${1//\//\\\/}/" file
foo /repl
bar /repl
baz /repl
That is a monstrosity of leaning toothpicks, but it serves to transform this:
sed "s/$/ /repl/"
into
sed "s/$/ \/repl/"
The same technique can be used for whatever you choose as the sed s/// delimiter.
I wrote this code:
cat /etc/passwd | cut -d : -f1 | sed -n "${FT_LINE1}, ${FT_LINE2} p"
Output:
sed: -e expression #1, char 1: unknown command: `,'
But I have a problem with variables $FT_LINE1, $FT_LINE2.
When I use constants instead of a variables, this code works correctly
cat /etc/passwd | cut -d : -f1 | sed -n "3, 5 p"
I tried to use these constructions:
sed -n -e "${FT_LINE1}, ${FT_LINE2} p"
sed -n "{$FT_LINE1}, {$FT_LINE2} p"
sed -n "${FT_LINE1},${FT_LINE2} p"
sed -n "${FT_LINE1}, ${FT_LINE2}" p
sed -n "$FT_LINE1, $FT_LINE2" p
but the error remained.
As noted in melpomene and PesaThe's comments, sed address ranges can't be blank, both shell variables ${FT_LINE1}, and ${FT_LINE2}, must be set to some appropriate value.
This simplest way to reproduce the error is:
sed ,
Which outputs:
sed: -e expression #1, char 1: unknown command: `,'
Because , is not a sed command, it's just a delimiter that separates range addresses.
It might help to look at some other related errors. Let's add a starting address of 1:
sed 1,
Output:
sed: -e expression #1, char 2: unexpected `,'
Which seems unhelpful, since it should be expecting an address after the ,. Now let's add a second address of 1:
sed 1,1
Output:
sed: -e expression #1, char 3: missing command
A little better, but really it's char 4 that's missing a command, or rather there's a missing command after char 3.
Now let's add a command, and a bit of input and it works:
echo foo | sed 1,1p
Output:
foo
I'm new in bash, and I'm trying to delete a line in a file I'm creating.
So without further ado :
if [[ $(ls -1 | grep 'fichiers.toCheck' | wc -l) -eq 0 ]]; then
touch fichiers.toCheck
fi
find . -name '*.mp4' > fichiers.toCheck
while read p; do
echo $p
sed -i "$p/d" ./fichiers.toCheck
done <fichiers.toCheck
Console is giving me this:
sed: 1: "./fichiers.toCheck": invalid command code .
I'm suspecting sed interprets the "/" in the line as an argument (the line is something like "./nosound.mp4".
What's your guess?
edit 2 = the correct syntax was with -i.bak
sed -i.bak "s#$p##" fichiers.toCheck
edit = so here's my experiments :
1
while read p; do
echo $p
sed -i "/$p/d" fichiers.toCheck
done <fichiers.toCheck
And I get :
sed: 1: "fichiers.toCheck": invalid command code f
2
sed -i "#$p#d" fichiers.toCheck
and same error :
sed: 1: "fichiers.toCheck": invalid command code f
Your sed syntax is wrong, to delete a line containing a pattern from a bash variable. Also being using FreeBSD native sed in OS X use the -i.bak for in-place edits.
sed -i.bak "/$p/d" fichiers.toCheck
If you suspect your variable contains / change the sed separator to # and use the traditional pattern s/<pattern>/<replacement>/ style with the replacement part set to empty, i.e.
sed -i.bak "s#$p##" fichiers.toCheck
I have a script like this:
#!/bin/bash
xx="-e \"s|a|b|g\""
sed -i $xx file
But sed breaks with message:
sed: -e expression #1, char 1: unknown command: `"'
Using set -x I can see that the command is being expanded to sed -i -e '"s|a|b|g"' file, so I guess the double quotes are why it is not working.
How to fix this?
I'm not sure exactly why you want to do what you're doing but I think that this might help:
$ cat file
a a
$ xx=( -e 's|a|b|g' -e 's|b|c|g' )
$ sed "${xx[#]}" file
c c
Use an array to store each argument to sed. Use "${xx[#]}" to safely expand the array, passing each element as a single argument.
You can build up the array like this:
$ xx=()
$ xx+=( -e 's|a| b c |g' )
$ xx+=( -e 's|c| d |g' )
$ sed "${xx[#]}" file
b d b d
You could try expanding the strings using eval, but it is not often recommended by bash aficionados.
#!/bin/bash
xx="-e 's|b|a|g'"
eval sed -i "$xx" file
You can see it getting expanded when using eval, below is the snippet from set +x
++ xx='-e '\''s|a|b|g'\'''
++ eval sed -i '-e '\''s|a|b|g'\''' file
+++ sed -i -e 's|a|b|g' file
To see it in action:-
$ cat file
line1/script
alaalaala++
line1/script
line2/script
line3/script
alaalaala--
line1/script
$ ./script.sh ; cat file
line1/script
blbblbblb++
line1/script
line2/script
line3/script
blbblbblb--
line1/script