If I run these commands from a script:
#my.sh
PWD=bla
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
xxx
bla
it is fine.
But, if I run:
#my.sh
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
$ sed: -e expression #1, char 8: Unknown option to `s'
I read in tutorials that to substitute environment variables from shell you need to stop, and 'out quote' the $varname part so that it is not substituted directly, which is what I did, and which works only if the variable is defined immediately before.
How can I get sed to recognize a $var as an environment variable as it is defined in the shell?
Your two examples look identical, which makes problems hard to diagnose. Potential problems:
You may need double quotes, as in sed 's/xxx/'"$PWD"'/'
$PWD may contain a slash, in which case you need to find a character not contained in $PWD to use as a delimiter.
To nail both issues at once, perhaps
sed 's#xxx#'"$PWD"'#'
In addition to Norman Ramsey's answer, I'd like to add that you can double-quote the entire string (which may make the statement more readable and less error prone).
So if you want to search for 'foo' and replace it with the content of $BAR, you can enclose the sed command in double-quotes.
sed 's/foo/$BAR/g'
sed "s/foo/$BAR/g"
In the first, $BAR will not expand correctly while in the second $BAR will expand correctly.
Another easy alternative:
Since $PWD will usually contain a slash /, use | instead of / for the sed statement:
sed -e "s|xxx|$PWD|"
You can use other characters besides "/" in substitution:
sed "s#$1#$2#g" -i FILE
一. bad way: change delimiter
sed 's/xxx/'"$PWD"'/'
sed 's:xxx:'"$PWD"':'
sed 's#xxx#'"$PWD"'#'
maybe those not the final answer,
you can not known what character will occur in $PWD, / : OR #.
if delimiter char in $PWD, they will break the expression
the good way is replace(escape) the special character in $PWD.
二. good way: escape delimiter
for example:
try to replace URL as $url (has : / in content)
x.com:80/aa/bb/aa.js
in string $tmp
URL
A. use / as delimiter
escape / as \/ in var (before use in sed expression)
## step 1: try escape
echo ${url//\//\\/}
x.com:80\/aa\/bb\/aa.js #escape fine
echo ${url//\//\/}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//\//\/}"
x.com:80\/aa\/bb\/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s/URL/${url//\//\\/}/"
URL
echo $tmp | sed "s/URL/${url//\//\/}/"
URL
OR
B. use : as delimiter (more readable than /)
escape : as \: in var (before use in sed expression)
## step 1: try escape
echo ${url//:/\:}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//:/\:}"
x.com\:80/aa/bb/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s:URL:${url//:/\:}:g"
x.com:80/aa/bb/aa.js
With your question edit, I see your problem. Let's say the current directory is /home/yourname ... in this case, your command below:
sed 's/xxx/'$PWD'/'
will be expanded to
sed `s/xxx//home/yourname//
which is not valid. You need to put a \ character in front of each / in your $PWD if you want to do this.
Actually, the simplest thing (in GNU sed, at least) is to use a different separator for the sed substitution (s) command. So, instead of s/pattern/'$mypath'/ being expanded to s/pattern//my/path/, which will of course confuse the s command, use s!pattern!'$mypath'!, which will be expanded to s!pattern!/my/path!. I’ve used the bang (!) character (or use anything you like) which avoids the usual, but-by-no-means-your-only-choice forward slash as the separator.
Dealing with VARIABLES within sed
[root#gislab00207 ldom]# echo domainname: None > /tmp/1.txt
[root#gislab00207 ldom]# cat /tmp/1.txt
domainname: None
[root#gislab00207 ldom]# echo ${DOMAIN_NAME}
dcsw-79-98vm.us.oracle.com
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: ${DOMAIN_NAME}/g'
--- Below is the result -- very funny.
domainname: ${DOMAIN_NAME}
--- You need to single quote your variable like this ...
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: '${DOMAIN_NAME}'/g'
--- The right result is below
domainname: dcsw-79-98vm.us.oracle.com
VAR=8675309
echo "abcde:jhdfj$jhbsfiy/.hghi$jh:12345:dgve::" |\
sed 's/:[0-9]*:/:'$VAR':/1'
where VAR contains what you want to replace the field with
I had similar problem, I had a list and I have to build a SQL script based on template (that contained #INPUT# as element to replace):
for i in LIST
do
awk "sub(/\#INPUT\#/,\"${i}\");" template.sql >> output
done
If your replacement string may contain other sed control characters, then a two-step substitution (first escaping the replacement string) may be what you want:
PWD='/a\1&b$_' # these are problematic for sed
PWD_ESC=$(printf '%s\n' "$PWD" | sed -e 's/[\/&]/\\&/g')
echo 'xxx' | sed "s/xxx/$PWD_ESC/" # now this works as expected
for me to replace some text against the value of an environment variable in a file with sed works only with quota as the following:
sed -i 's/original_value/'"$MY_ENVIRNONMENT_VARIABLE"'/g' myfile.txt
BUT when the value of MY_ENVIRONMENT_VARIABLE contains a URL (ie https://andreas.gr) then the above was not working.
THEN use different delimiter:
sed -i "s|original_value|$MY_ENVIRNONMENT_VARIABLE|g" myfile.txt
Tool: Git Bash for Windows
Problem: Trying to insert the text "\connect central" at the top of each file in a directory.
Code:
for f in $DIR/*.sql; do
sed -i "1i \\\connect central" $f
done
This does try to edit inline and insert my text, but three backslashes (like I've read everywhere) doesn't create the single backslash like I'm expected. Instead I get:
I've also tried some variants along the lines of:
for f in $DIR/*.sql; do
sed -i -e "1i `\\\connect central`" $f
done
but that throws an error of sed: -e expression #1, char 3: expected \ aftera', c',i'`
Use single quotes instead of double quotes. Backslash is an escape character inside double quotes, so you need to double it to pass it through to the sed command literally. It has no special meaning inside single quotes.
sed -i '1i \\\connect central' "$f"
To do it with double quotes (which you might need if there's variable content in the string you're inserting), you have to double all the backslashes:
sed -i "1i \\\\\\connect central" "$f"
For more information, see Difference between single and double quotes in Bash
This question already has answers here:
sed substitution with Bash variables
(6 answers)
Closed 5 years ago.
I have a variable in a config, which I would like to replace with a value.
ROOT="rROOT"
I would like to replace that with
ROOT="$root"
So with the value of $root (Important are the quotation marks).
So far I have tried it that way
sed -i s/'ROOT=rROOT'/'ROOT="$root"'/g $directory/settings.conf
The result is that
ROOT="$root"
But this is stored in the variable $root (It is always a path)
root: /
How can I replace rROOT with the value of $root?
Sed Version: (GNU sed) 4.2.2
Ok, don't like to ruin my scripts for testing:
cat tageslog.sh | sed "s/alt=185094/alt=\${root}/g"
Use double quotes, but mask the dollar sign, so it doesn't get lost and root interpreted while calling sed.
Use ${root} instead of "$root".
sed "s/ROOT=.rRoot./ROOT=\${root}/g" $directory/settings.conf
if this works, use the -i switch:
sed -i "s/ROOT=.rRoot./ROOT=\${root}/g" $directory/settings.conf
You have different problems:
Testing with echo
I think you tested your command with
echo ROOT="rROOT" | sed s/'ROOT=rROOT'/'ROOT="$root"'/g
The double quotes won't appear in the output of the echo, so you will end up with a command working for ROOT=rROOT.
When the inputfile has qouble quotes, you do not have to insert them.
Testing with the double quotes is possible with
echo 'ROOT="rROOT"' | sed s/'ROOT=rROOT'/'ROOT="$root"'/g
Place the double quotes outside the single quotes;
You can test this with echo:
echo 'Showing "$PWD" in double quotes is "'$PWD'"'
echo 'With additional spaces the last part is " '$PWD' " '
Root vaiable has slashes that will confuse sed.
The root variable is replaced before sed tries to understand the command.
You can use another character, like #, in the sed command:
sed 's#before#after#'
When your input has double quotes:
echo 'ROOT="rROOT"' | sed 's#ROOT="rROOT"#ROOT="'$root'"#g'
# or by remembering strings
echo 'ROOT="rROOT"' | sed -r 's#(ROOT=")rROOT(")#\1'$root'\2#g'
Input without double quotes
echo 'ROOT=rROOT' | sed 's#ROOT=rROOT#ROOT="'$root'"#g'
# or by remembering strings
echo 'ROOT=rROOT' | sed -r 's#(ROOT=)rROOT#\1"'$root'"#g'
I'm using the Git Bash shell on Windows, and trying to replace a string like this in an XML file using sed:
<customTag>C:\path\to\2016a.0</customTag>
To a string like this:
<customTag>C:\path\to\2017b.0</customTag>
I can do the replacement directly like this:
$ cat test.txt
<customTag>C:\path\to\2016a.0</customTag>
$ sed -i 's^<customTag>C:\\path\\to\\2016a.0</customTag>^<customTag>C:\\path\\to\\2017b.0</customTag>^g' test.txt
$ cat test.txt
<customTag>C:\path\to\2017b.0</customTag>
But if I need to pass in variables for those strings, the replacement doesn't work.
$ cat test.txt
<customTag>C:\path\to\2016a.0</customTag>
$ export OLD_VER=2016a.0
$ export NEW_VER=2017b.0
$ sed -i 's^<customTag>C:\\path\\to\\${OLD_VER}</customTag>^<customTag>C:\\path\\to\\${NEW_VER}</customTag>^g' test.txt
$ cat test.txt
<customTag>C:\path\to\2016a.0</customTag>
Or if I use double quotes around the sed expression, I get "Invalid back reference", presumably because it thinks the 2 in the year is a reference.
$ sed -i "s^<customTag>C:\\path\\to\\${OLD_VER}</customTag>^<customTag>C:\\path\\to\\${NEW_VER}</customTag>^g" test.txt
sed: -e expression #1, char 87: Invalid back reference
What's the correct way to escape or quote this, or would I be better off using something like awk?
Keep the single quotes on the ends, and add single quotes around each variable. The single quotes prevent the shell from collapsing your double backslashes. The extra single quotes leave the variable refs outside of quotes.
Or (don't laugh) consider using forward slashes. Windows recognizes both kinds of slash as path separators; it's only the DOS command shell that does not.
I want to pass this command in my script:
sed -n -e "/Next</a></p>/,/Next</a></p>/ p" file.txt
This command (should) extract all text between the two matched patterns, which are both Next</a></p> in my case. However when I run my script I keep getting errors. I've tried:
sed -n -e "/Next\<\/a\>\<\/p\>/,/Next<\/a\>\<\/p>/ p" file.txt with no luck.
I believe the generic pattern for this command is this:
sed -n -e "/pattern1/,/pattern2/ p" file.txt
I can't get it working for Next</a></p> though and I'm guessing it has something to do with the special characters I am encasing. Is there any way to pass Next</a></p> in the sed command? Thanks in advance guys! This community is awesome!
You don't need to use / as a regular expression delimiter. Using a different character will make quoting issues slightly easier. The syntax is
\cregexc
where c can be any character (other than \) that you don't use in the regex. In this case, : might be a good choice:
sed -n -e '\:Next</a></p>:,\:Next</a></p>: p' file.txt
Note that I changed " to ' because inside double quotes, \ will be interpreted by bash as an escape character, whereas inside single quotes \ is just treated as a regular character. Consequently, you could have written the version with escaped slashes like this:
sed -n -e '/Next<\/a><\/p>/,/Next<\/a><\/p>/ p' file.txt
but I think the version with colons is (slightly) easier to read.
You need to escape the forward slashes inside the regular expressions with a \, since the forward slashes serve as delimiters for the regexes
sed -n -e '/Next<\/a><\/p>/,/Next<\/a><\/p>/p' file.txt