File.txt
/aaa/bbb/ccc/ddd
/aaa/bbb/ccc/mmm
/aaa/eee/ccc/ddd
if my $(pwd) is /aaa/bbb/ccc
the it should delete only first two
I have tried like sed /^$(pwd)/d but not worked
The problem here is that you are using $(pwd), which tries to execute a command pwd. This result contains slashes, so that the final command is something like:
sed /^/aaa/bbb/ccc/d
Which sed cannot handle and returns an error:
sed: -e expression #1, char 4: extra characters after command
You should instead use another delimiter. For example, _:
sed "\_${PWD}_d"
As 123 comments below, you need to escape the first delimiter if it is not a substitution. I also enclose the var within ${ } to prevent the variable to be considered PWD_ instead of PWD.
You can use awk for a nicer approach:
$ awk -v patt="$PWD" '!($0 ~ patt)' file
/aaa/eee/ccc/ddd
Note $PWD is the same as executing pwd.
grep can also do the job:
grep -v "$(pwd)" file
Just to precise the answer of fedorqui...
In your question there is another problem because you variable $pwd contain special sed symbols (/).
So the sed will not be glad...
Some solution for example could be find here : Replace a string in shell script using a variable
So you could use additional variable to correct this problem.
This work perfectly for your example (I just replace echo $(pwd) by 'echo /aaa/bbb/ccc').
pwd_bis=$( echo $(pwd) | sed 's/[\/]/\\\0/g' )
sed "/^${pwd_bis}/d" File.txt
Related
I have a shell script that contains the following line
PROC_ID=$(cat myfile.sh | grep running)
which, after you echo out the value would be 1234 or something like that.
What I want to do is find and replace instances of this line with a literal value
I want to replace it with PROC_ID=1234 instead of having the function call.
I've tried doing this in another shell script using sed but I can't get it to work
STR_TO_USE="PROC_ID=${1}"
STR_TO_REP='PROC_ID=$(cat myfile.sh | grep running)'
sed -i "s/$STR_TO_REP/$STR_TO_USE/g" sample.sh
but it complains stating sed: 1: "sample.sh": unterminated substitute pattern
How can I achieve this?
EDIT:
sample.sh should contain beforehand
#!/bin/bash
....
PROC_ID=$(cat myfile.sh | grep running)
echo $PROC_ID
....
After, it should contain
#!/bin/bash
....
PROC_ID=1234
echo $PROC_ID
....
The script I'm using as described above will be taking the in an arg from the command line, hence STR_TO_USE="PROC_ID=${1}"
Simply:
sed /^PROC_ID=/s/=.*/=1234/
Translation:
At line begining by PROC_ID=
replace = to end of line by =1234.
or more accurate
sed '/^[ \o11]*PROC_ID=.*myfile.*running/s/=.*/=1234/'
could be enough
([ \o11]* mean some spaces and or tabs could even prepand)
Well, first, I want to point out something obvious. This: $(cat myfile.sh | grep running) will at the very least NOT only contain the string 1234 but will certainly also contain the string running. But since you aren't asking for help with that, I'll leave it alone.
All you need in your above sed, is first to backslash the $.
STR_TO_REP='PROC_ID=\$(cat myfile.sh | grep running)'
This allows the sed command to be terminated.
If I run these commands from a script:
#my.sh
PWD=bla
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
xxx
bla
it is fine.
But, if I run:
#my.sh
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
$ sed: -e expression #1, char 8: Unknown option to `s'
I read in tutorials that to substitute environment variables from shell you need to stop, and 'out quote' the $varname part so that it is not substituted directly, which is what I did, and which works only if the variable is defined immediately before.
How can I get sed to recognize a $var as an environment variable as it is defined in the shell?
Your two examples look identical, which makes problems hard to diagnose. Potential problems:
You may need double quotes, as in sed 's/xxx/'"$PWD"'/'
$PWD may contain a slash, in which case you need to find a character not contained in $PWD to use as a delimiter.
To nail both issues at once, perhaps
sed 's#xxx#'"$PWD"'#'
In addition to Norman Ramsey's answer, I'd like to add that you can double-quote the entire string (which may make the statement more readable and less error prone).
So if you want to search for 'foo' and replace it with the content of $BAR, you can enclose the sed command in double-quotes.
sed 's/foo/$BAR/g'
sed "s/foo/$BAR/g"
In the first, $BAR will not expand correctly while in the second $BAR will expand correctly.
Another easy alternative:
Since $PWD will usually contain a slash /, use | instead of / for the sed statement:
sed -e "s|xxx|$PWD|"
You can use other characters besides "/" in substitution:
sed "s#$1#$2#g" -i FILE
一. bad way: change delimiter
sed 's/xxx/'"$PWD"'/'
sed 's:xxx:'"$PWD"':'
sed 's#xxx#'"$PWD"'#'
maybe those not the final answer,
you can not known what character will occur in $PWD, / : OR #.
if delimiter char in $PWD, they will break the expression
the good way is replace(escape) the special character in $PWD.
二. good way: escape delimiter
for example:
try to replace URL as $url (has : / in content)
x.com:80/aa/bb/aa.js
in string $tmp
URL
A. use / as delimiter
escape / as \/ in var (before use in sed expression)
## step 1: try escape
echo ${url//\//\\/}
x.com:80\/aa\/bb\/aa.js #escape fine
echo ${url//\//\/}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//\//\/}"
x.com:80\/aa\/bb\/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s/URL/${url//\//\\/}/"
URL
echo $tmp | sed "s/URL/${url//\//\/}/"
URL
OR
B. use : as delimiter (more readable than /)
escape : as \: in var (before use in sed expression)
## step 1: try escape
echo ${url//:/\:}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//:/\:}"
x.com\:80/aa/bb/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s:URL:${url//:/\:}:g"
x.com:80/aa/bb/aa.js
With your question edit, I see your problem. Let's say the current directory is /home/yourname ... in this case, your command below:
sed 's/xxx/'$PWD'/'
will be expanded to
sed `s/xxx//home/yourname//
which is not valid. You need to put a \ character in front of each / in your $PWD if you want to do this.
Actually, the simplest thing (in GNU sed, at least) is to use a different separator for the sed substitution (s) command. So, instead of s/pattern/'$mypath'/ being expanded to s/pattern//my/path/, which will of course confuse the s command, use s!pattern!'$mypath'!, which will be expanded to s!pattern!/my/path!. I’ve used the bang (!) character (or use anything you like) which avoids the usual, but-by-no-means-your-only-choice forward slash as the separator.
Dealing with VARIABLES within sed
[root#gislab00207 ldom]# echo domainname: None > /tmp/1.txt
[root#gislab00207 ldom]# cat /tmp/1.txt
domainname: None
[root#gislab00207 ldom]# echo ${DOMAIN_NAME}
dcsw-79-98vm.us.oracle.com
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: ${DOMAIN_NAME}/g'
--- Below is the result -- very funny.
domainname: ${DOMAIN_NAME}
--- You need to single quote your variable like this ...
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: '${DOMAIN_NAME}'/g'
--- The right result is below
domainname: dcsw-79-98vm.us.oracle.com
VAR=8675309
echo "abcde:jhdfj$jhbsfiy/.hghi$jh:12345:dgve::" |\
sed 's/:[0-9]*:/:'$VAR':/1'
where VAR contains what you want to replace the field with
I had similar problem, I had a list and I have to build a SQL script based on template (that contained #INPUT# as element to replace):
for i in LIST
do
awk "sub(/\#INPUT\#/,\"${i}\");" template.sql >> output
done
If your replacement string may contain other sed control characters, then a two-step substitution (first escaping the replacement string) may be what you want:
PWD='/a\1&b$_' # these are problematic for sed
PWD_ESC=$(printf '%s\n' "$PWD" | sed -e 's/[\/&]/\\&/g')
echo 'xxx' | sed "s/xxx/$PWD_ESC/" # now this works as expected
for me to replace some text against the value of an environment variable in a file with sed works only with quota as the following:
sed -i 's/original_value/'"$MY_ENVIRNONMENT_VARIABLE"'/g' myfile.txt
BUT when the value of MY_ENVIRONMENT_VARIABLE contains a URL (ie https://andreas.gr) then the above was not working.
THEN use different delimiter:
sed -i "s|original_value|$MY_ENVIRNONMENT_VARIABLE|g" myfile.txt
I'm using the Git Bash shell on Windows, and trying to replace a string like this in an XML file using sed:
<customTag>C:\path\to\2016a.0</customTag>
To a string like this:
<customTag>C:\path\to\2017b.0</customTag>
I can do the replacement directly like this:
$ cat test.txt
<customTag>C:\path\to\2016a.0</customTag>
$ sed -i 's^<customTag>C:\\path\\to\\2016a.0</customTag>^<customTag>C:\\path\\to\\2017b.0</customTag>^g' test.txt
$ cat test.txt
<customTag>C:\path\to\2017b.0</customTag>
But if I need to pass in variables for those strings, the replacement doesn't work.
$ cat test.txt
<customTag>C:\path\to\2016a.0</customTag>
$ export OLD_VER=2016a.0
$ export NEW_VER=2017b.0
$ sed -i 's^<customTag>C:\\path\\to\\${OLD_VER}</customTag>^<customTag>C:\\path\\to\\${NEW_VER}</customTag>^g' test.txt
$ cat test.txt
<customTag>C:\path\to\2016a.0</customTag>
Or if I use double quotes around the sed expression, I get "Invalid back reference", presumably because it thinks the 2 in the year is a reference.
$ sed -i "s^<customTag>C:\\path\\to\\${OLD_VER}</customTag>^<customTag>C:\\path\\to\\${NEW_VER}</customTag>^g" test.txt
sed: -e expression #1, char 87: Invalid back reference
What's the correct way to escape or quote this, or would I be better off using something like awk?
Keep the single quotes on the ends, and add single quotes around each variable. The single quotes prevent the shell from collapsing your double backslashes. The extra single quotes leave the variable refs outside of quotes.
Or (don't laugh) consider using forward slashes. Windows recognizes both kinds of slash as path separators; it's only the DOS command shell that does not.
I'm using sed to replace a character in a file with a variable. This variable is basically reading the contents of a file or a webpage which contains multiple hash-like strings like below, which are randomly generated:
define('AUTH_KEY', 'CVo|BO;Qt1B|+GE}+h2/yU7h=5`/wRV{>%h.b_;s%S8-p|>qpf]|/Vf#`&[g~*:&');
define('SECURE_AUTH_KEY', '{G2-<^jWRd7]2,?]6hhM^*asg.2C.+k=gf33-m+ZK_{Mt|q*<ELF4|gPjyxtTh!)');
define('LOGGED_IN_KEY', 'jSNo9Z;5d]tzZoh-QQ`{M-&~y??$R({:*m`0={67=+mF?L.e+R{;)+4}qCAAHz=C');
define('NONCE_KEY', '19Vt4=%8j/Z-&~ni0S<]9)J^~sy9dh|h9M_RX2#K0]F9+.v+[BP1d&B&}-FTKIJ,');
define('AUTH_SALT', 'jr7f?T|#Cbo]XVAo}N^ilkvD>dC-rr]5{al64|?_Hz }JG$yEi:_aU )Olp YAD+');
define('SECURE_AUTH_SALT', 'hm#Z%O!X_mr?lM|>>~r-?F%wi R($}|9R[):4^NTsj+gS[qnv}7|+0<9e-$DJjju');
define('LOGGED_IN_SALT', 'tyPHBOCkXZh_4H;G|^.&|^#JPB/f;{}y_Orj!6AH?#wovx+KKtTZ A[HMS9SZJ|N');
define('NONCE_SALT', 'Eb-/t 5D-vPV9I--8F<[^lcherGv.g+|7p6;+xP|5g6P}tup1K.vuHAQ=uWZ#}H^');
The variable is defined like so:
KEY=$(cat keyfile)
I used the following sed syntax:
sed -i "s/stringtoreplace/$KEY/" /path/to/file
I've also tried different variations, using single-quotes, quotes around the variables
sed -i "s/stringtoreplace/"$KEY"/" /path/to/file
sed -i "s/stringtoreplace/"${KEY}"/" /path/to/file
sed -i "s/stringtoreplace/'$KEY'/" /path/to/file
I think I "brute-forced" every way possible to put quotes and I don't know how to escape randomly generated characters like those. I keep getting the following error:
unterminated s command
Any suggestions on how to replace the string with the weird-hash-like variable?
sed is an excellent tool for simple substitutions on a single line but for anything else you should use awk:
awk -v key="$KEY" '{sub(/stringtoreplace/,key)}1' file
That will work no matter what characters "$KEY" contains, except for "&".
One possibility is to use bash instead of sed. That makes the substitution easy, but you'll have to emulate the -i option.
Something like this:
TMPFILE=$(mktemp)
KEY=$(cat keyfile)
while IFS= read -r LINE; do
echo "${LINE//stringtoreplace/$KEY}"
done </path/to/file >$TMPFILE
mv $TMPFILE /path/to/file
Try this:
KEY=$(cat keyfile | sed -e 's/[]\/()$*.^|[]/\\&/g')
sed -i "s/stringtoreplace/$KEY/" /path/to/file
The problem is that the $KEY variable contains slashes, which are the delimiters for your s command.
sed can do more than search and replace:
sed '/stringtoreplace/{
d
r keyfile
}' /path/to/file
I'm assuming "stringtoreplace" occurs by itself on a line.
I have variable which has value "abcd.txt".
I want to store everything before the ".txt" in a second variable, replacing the ".txt" with ".log"
I have no problem echoing the desired value:
a="abcd.txt"
echo $a | sed 's/.txt/.log/'
But how do I get the value "abcd.log" into the second variable?
You can use command substitution as:
new_filename=$(echo "$a" | sed 's/.txt/.log/')
or the less recommended backtick way:
new_filename=`echo "$a" | sed 's/.txt/.log/'`
You can use backticks to assign the output of a command to a variable:
logfile=`echo $a | sed 's/.txt/.log/'`
That's assuming you're using Bash.
Alternatively, for this particular problem Bash has pattern matching constructs itself:
stem=$(textfile%%.txt)
logfile=$(stem).log
or
logfile=$(textfile/%.txt/.log)
The % in the last example will ensure only the last .txt is replaced.
The simplest way is
logfile="${a/\.txt/\.log}"
If it should be allowed that the filename in $a has more than one occurrence of .txt in it, use the following solution. Its more safe. It only changes the last occurrence of .txt
logfile="${a%%\.txt}.log"
if you have Bash/ksh
$ var="abcd.txt"
$ echo ${var%.txt}.log
abcd.log
$ variable=${var%.txt}.log