If I run these commands from a script:
#my.sh
PWD=bla
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
xxx
bla
it is fine.
But, if I run:
#my.sh
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
$ sed: -e expression #1, char 8: Unknown option to `s'
I read in tutorials that to substitute environment variables from shell you need to stop, and 'out quote' the $varname part so that it is not substituted directly, which is what I did, and which works only if the variable is defined immediately before.
How can I get sed to recognize a $var as an environment variable as it is defined in the shell?
Your two examples look identical, which makes problems hard to diagnose. Potential problems:
You may need double quotes, as in sed 's/xxx/'"$PWD"'/'
$PWD may contain a slash, in which case you need to find a character not contained in $PWD to use as a delimiter.
To nail both issues at once, perhaps
sed 's#xxx#'"$PWD"'#'
In addition to Norman Ramsey's answer, I'd like to add that you can double-quote the entire string (which may make the statement more readable and less error prone).
So if you want to search for 'foo' and replace it with the content of $BAR, you can enclose the sed command in double-quotes.
sed 's/foo/$BAR/g'
sed "s/foo/$BAR/g"
In the first, $BAR will not expand correctly while in the second $BAR will expand correctly.
Another easy alternative:
Since $PWD will usually contain a slash /, use | instead of / for the sed statement:
sed -e "s|xxx|$PWD|"
You can use other characters besides "/" in substitution:
sed "s#$1#$2#g" -i FILE
一. bad way: change delimiter
sed 's/xxx/'"$PWD"'/'
sed 's:xxx:'"$PWD"':'
sed 's#xxx#'"$PWD"'#'
maybe those not the final answer,
you can not known what character will occur in $PWD, / : OR #.
if delimiter char in $PWD, they will break the expression
the good way is replace(escape) the special character in $PWD.
二. good way: escape delimiter
for example:
try to replace URL as $url (has : / in content)
x.com:80/aa/bb/aa.js
in string $tmp
URL
A. use / as delimiter
escape / as \/ in var (before use in sed expression)
## step 1: try escape
echo ${url//\//\\/}
x.com:80\/aa\/bb\/aa.js #escape fine
echo ${url//\//\/}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//\//\/}"
x.com:80\/aa\/bb\/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s/URL/${url//\//\\/}/"
URL
echo $tmp | sed "s/URL/${url//\//\/}/"
URL
OR
B. use : as delimiter (more readable than /)
escape : as \: in var (before use in sed expression)
## step 1: try escape
echo ${url//:/\:}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//:/\:}"
x.com\:80/aa/bb/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s:URL:${url//:/\:}:g"
x.com:80/aa/bb/aa.js
With your question edit, I see your problem. Let's say the current directory is /home/yourname ... in this case, your command below:
sed 's/xxx/'$PWD'/'
will be expanded to
sed `s/xxx//home/yourname//
which is not valid. You need to put a \ character in front of each / in your $PWD if you want to do this.
Actually, the simplest thing (in GNU sed, at least) is to use a different separator for the sed substitution (s) command. So, instead of s/pattern/'$mypath'/ being expanded to s/pattern//my/path/, which will of course confuse the s command, use s!pattern!'$mypath'!, which will be expanded to s!pattern!/my/path!. I’ve used the bang (!) character (or use anything you like) which avoids the usual, but-by-no-means-your-only-choice forward slash as the separator.
Dealing with VARIABLES within sed
[root#gislab00207 ldom]# echo domainname: None > /tmp/1.txt
[root#gislab00207 ldom]# cat /tmp/1.txt
domainname: None
[root#gislab00207 ldom]# echo ${DOMAIN_NAME}
dcsw-79-98vm.us.oracle.com
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: ${DOMAIN_NAME}/g'
--- Below is the result -- very funny.
domainname: ${DOMAIN_NAME}
--- You need to single quote your variable like this ...
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: '${DOMAIN_NAME}'/g'
--- The right result is below
domainname: dcsw-79-98vm.us.oracle.com
VAR=8675309
echo "abcde:jhdfj$jhbsfiy/.hghi$jh:12345:dgve::" |\
sed 's/:[0-9]*:/:'$VAR':/1'
where VAR contains what you want to replace the field with
I had similar problem, I had a list and I have to build a SQL script based on template (that contained #INPUT# as element to replace):
for i in LIST
do
awk "sub(/\#INPUT\#/,\"${i}\");" template.sql >> output
done
If your replacement string may contain other sed control characters, then a two-step substitution (first escaping the replacement string) may be what you want:
PWD='/a\1&b$_' # these are problematic for sed
PWD_ESC=$(printf '%s\n' "$PWD" | sed -e 's/[\/&]/\\&/g')
echo 'xxx' | sed "s/xxx/$PWD_ESC/" # now this works as expected
for me to replace some text against the value of an environment variable in a file with sed works only with quota as the following:
sed -i 's/original_value/'"$MY_ENVIRNONMENT_VARIABLE"'/g' myfile.txt
BUT when the value of MY_ENVIRONMENT_VARIABLE contains a URL (ie https://andreas.gr) then the above was not working.
THEN use different delimiter:
sed -i "s|original_value|$MY_ENVIRNONMENT_VARIABLE|g" myfile.txt
Related
Cannot seem to find an answer to this one online...
I have a string variable (externally sourced) with new lines "\n" encoded as strings.
I want to replace those strings with actual new line carriage returns. The code below can achieve this...
echo $EXT_DESCR | sed 's/\\n/\n/g'
But when I try to store the result of this into it's own variable, it converts them back to strings
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
How can this be achieved, or what I'm I doing wrong?
Here's my code I've been testing to try get the right results
EXT_DESCR="This is a text\nWith a new line"
echo $EXT_DESCR | sed 's/\\n/\n/g'
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
echo ""
echo "$NEW_DESCR"
No need for sed, using parameter expansion:
$ foo='1\n2\n3'; echo "${foo//'\n'/$'\n'}"
1
2
3
With bash 4.4 or newer, you can use the E operator in ${parameter#operator}:
$ foo='1\n2\n3'; echo "${foo#E}"
1
2
3
Other answers contain alternative solutions. (I especially like the parameter expansion one.)
Here's what's wrong with your attempt:
In
echo $EXT_DESCR | sed 's/\\n/\n/g'
the sed command is in single quotes, so sed gets s/\\n/\n/g as is.
In
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
the whole command is in backticks, so a round of backslash processing is applied. That leads to sed getting the code s/\n/\n/g, which does nothing.
A possible fix for this code:
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\\\n/\\n/g'`
By doubling up the backslashes, we end up with the right command in sed.
Or (easier):
NEW_DESCR=$(echo $EXT_DESCR | sed 's/\\n/\n/g')
Instead of backticks use $( ), which has less esoteric escaping rules.
Note: Don't use ALL_UPPERCASE for your shell variables. UPPERCASE is (informally) reserved for system variables such as HOME and special built-in variables such as IFS or RANDOM.
Depending on what exactly you need it for:
echo -e $EXT_DESCR
might be all you need.
From echo man page:
-e
enable interpretation of backslash escapes
This printf would do the job by interpreting all escaped constructs:
printf -v NEW_DESCR "%b" "$EXT_DESCR"
-v option will store output in a variable so no need to use command substitution here.
Problem with your approach is use of old back-ticks. You could do:
NEW_DESCR=$(echo "$EXT_DESCR" | sed 's/\\n/\n/g')
Assuming you're using gnu sed as BSD sed won't work with this approach.
File.txt
/aaa/bbb/ccc/ddd
/aaa/bbb/ccc/mmm
/aaa/eee/ccc/ddd
if my $(pwd) is /aaa/bbb/ccc
the it should delete only first two
I have tried like sed /^$(pwd)/d but not worked
The problem here is that you are using $(pwd), which tries to execute a command pwd. This result contains slashes, so that the final command is something like:
sed /^/aaa/bbb/ccc/d
Which sed cannot handle and returns an error:
sed: -e expression #1, char 4: extra characters after command
You should instead use another delimiter. For example, _:
sed "\_${PWD}_d"
As 123 comments below, you need to escape the first delimiter if it is not a substitution. I also enclose the var within ${ } to prevent the variable to be considered PWD_ instead of PWD.
You can use awk for a nicer approach:
$ awk -v patt="$PWD" '!($0 ~ patt)' file
/aaa/eee/ccc/ddd
Note $PWD is the same as executing pwd.
grep can also do the job:
grep -v "$(pwd)" file
Just to precise the answer of fedorqui...
In your question there is another problem because you variable $pwd contain special sed symbols (/).
So the sed will not be glad...
Some solution for example could be find here : Replace a string in shell script using a variable
So you could use additional variable to correct this problem.
This work perfectly for your example (I just replace echo $(pwd) by 'echo /aaa/bbb/ccc').
pwd_bis=$( echo $(pwd) | sed 's/[\/]/\\\0/g' )
sed "/^${pwd_bis}/d" File.txt
I have redirected some string into one parameter for ex: ab=jyoti,priya, pranit
I have one file : Name.txt which contains -
jyoti
prathmesh
John
Kelvin
pranit
I want to delete the records from the Name.txt file which are contain in ab parameter.
Please suggest if this can be done ?
If ab is a shell variable, you can easily turn it into an extended regular expression, and use it with grep -E:
grep -E -x -v "${ab//,/|}" Name.txt
The string substitution ${ab//,/|} returns the value of $ab with every , substituted with a | which turns it into an extended regular expression, suitable for passing as an argument to grep -E.
The -v option says to remove matching lines.
The -x option specifies that the match needs to cover the whole input line, so that a short substring will not cause an entire longer line to be removed. Without it, ab=prat would cause pratmesh to be removed.
If you really require a sed solution, the transformation should be fairly trivial. grep -E -v -x 'aaa|bbb|ccc' is equivalent to sed '/^\(aaa\|bbb\|ccc)$/d' (with some dialects disliking the backslashes, and others requiring them).
To do an in-place edit (modify Name.txt without a temporary file), try this:
sed -i "/^\(${ab//,/\|}\)\$/d" Name.txt
This is not entirely robust against strings containing whitespace or other shell metacharacters, but if you just need
Try with
sed -e 's/\bjyoti\b//g;s/\bpriya\b//g' < Name.txt
(using \b assuming you need word boundaries)
this will do it:
for param in `echo $ab | sed -e 's/[ ]+//g' -e 's/,/ /g'` ; do res=`sed -e "s/$param//g" < name.txt`; echo $res > name.txt; done
echo $res
First of all i apologise in case this has been answered before but i couldn't solve my problem.
I need to search a pattern and then replace it with a line of text comprising of both text and variable.Btw i am using bash..
say
$var = "stacko.ver/rulz=" **Note: $var contain double quotes & = & a dot and /**
i want to so the follow
1.Search for ;te.xt = Note: The value to be search contain ; & = and a dot
2.Replace it with
textnum=$var
Of course $var should be replaced with its actual value
My attempts
sed -i "s/;te.xt =/textnum=$var/" file
sed -i "s/;te.xt =/textnum="$var"/" file
sed -i "s/";te.xt ="/"textnum=$var"/" file
None of these actually worked , either sed giving me an error or the value of $var not shown in file
Thanks for the help
Regards
Quoting doesn't help since this is a sed issue, not a bash issue. Just pick a sed s-expression delimiter that doesn't appear in your text:
sed -i "s|;te.xt =|textnum=$var|" file
You can pick any delimiter for s that doesn't appear in your input. sed -e 'streetlight' is a perfectly valid sed command.
I can see the error:
$ var="stacko.ver/rulz="
$ data="foo ;te.xt = bar"
$ sed "s/;te.xt =/textnum=$var/" <<< "$data"
sed: -e expression #1, char 31: unknown option to `s'
The problem is that $var contains a slash, so sed's s/// command is breaking. You need to pick a character that does not appear in $var
$ sed "s#;te.xt =#textnum=$var#" <<< "$data"
foo textnum=stacko.ver/rulz= bar
This can be hard -- what if slash and hash are in $var? Using bash, you can use ANSI-C quoting to use a control character that is unlikely to appear in your data, e.g.
$ sed $'s\037;te.xt =\037textnum=$var\037' <<< "$data"
foo textnum=stacko.ver/rulz= bar
Please see these simple commands:
$ echo $tmp
UY\U[_
$ echo "a" | sed "s|a|${tmp}|g"
UY[_
The \U is eaten. Other backslashes won't survive either.
How can I make the above command work as expected?
If it's only backslash that is "eaten" by sed and escaping just that is enough, then try:
echo "a" | sed "s|a|${tmp//\\/\\\\}|g"
Confusing enough for you? \\ represents a single \ since it needs to be escaped in the shell too.
The inital // is similar to the g modifier in s/foo/bar/g, if you only want the first occurring pattern to be replaced, skip it.
The docs about ${parameter/pattern/string} is available here: http://www.gnu.org/s/bash/manual/bash.html#Shell-Parameter-Expansion
Edit: Depending on what you want to do, you might be better of not using sed for this actually.
$ tmp="UY\U[_"
$ in="a"
$ echo ${in//a/$tmp}
UY\U[_
You could reparse $tmp itself through sed
echo "a" | sed "s|a|$(echo ${tmp} | sed 's|\\|\\\\|g')|g"