I want to replace the version in my code using git rev-parse HEAD with template string %VERSION% in a source file.
For simplicity I will use date as version command in this question.
Given test.txt
$ echo "This is test-%VERSION%." > test.txt
$ cat test.txt
This is test-%VERSION%.
Expect
This is test-Sat Dec 2 16:48:59 +07 2017.
These are failed try
$ echo "This is test-%VERSION%." > test.txt
$ sed -i 's/%VERSION/`date`/' test.txt && cat test.txt
This is test-`date`%.
$ echo "This is test-%VERSION%." > test.txt
$ DD=`date` sed -i 's/%VERSION/$DD/' test.txt && cat test.txt
This is test-$DD%.
$ echo "This is test-%VERSION%." > test.txt
$ DD=`date` sed -i "s/%VERSION/$DD/" test.txt && cat test.txt
This is test-%.
Do I really need to use xargs ?
You can embed $(...) within double-quotes, but not in single-quotes:
sed -i "s/%VERSION%/$(date)/" test.txt && cat test.txt
(Same as `...` but you shouldn't use that obsolete syntax, $(...) is better.)
Btw, for testing purposes, it's better to use sed without -i,
so the original file is not modified:
sed "s/%VERSION%/$(date)/" test.txt
As a side note, and this is a completely different discussion,
but worth mentioning here.
This may look like it should work but it doesn't, and you may wonder why:
DD=$(date) sed -i "s/%VERSION%/$DD/" test.txt && cat test.txt
Why it doesn't work?
Because the $DD embedded in the "..." is evaluated at the time the command is executed.
At that time the value of DD is not set to the output of $(date).
In the "..." it will have whatever value it had before executing the command.
For the sed process, the value DD with the output of $(date) is visible,
but sed doesn't use that, because why would it.
The "..." passed to sed is evaluated by the shell, not by sed.
Use double-quotes to do the substitution and avoid using the outdated `` construct but rather use the $(..) syntax for Command substitution
sed -i "s/%VERSION%/$(date)/" file
Also another way if you just want to use the single quotes, would be to wrap the substitution part in double-quotes and then single-quote on top of it, something like sed 's/%VERSION%/'"$(date)"'/' file which is less efficient than simply double-quoting the entire substitution string.
Related
I want to replace a line in a file with multiple lines. I know I can use \n in the sed replace, but that is rather ugly. I was hoping to HEARDOCs.
So I can do this to replace the line with multiple lines:
$ cat sedtest
DINGO=bingo
$ sed -i -e "s/^DINGO.*$/# added by $(whoami) on $(date)\nDINGO=howdy/" sedtest
$ cat sedtest
# added by user on Sun Feb 3 08:55:44 EST 2019
DINGO=howdy
In the command I want to put the replacement in new lines so it's easier to read/understand. So far I have been using HEREDOCs when I want to add new lines to a file:
CAT << EOF | sudo tee -a file1 file2 file3
line one
line two
line three
EOF
And this has worked well for appending/adding. Is it possible to do something similar but instead use the output as the replacement in sed or is there some other way to do what I'm looking for?
Is this what you're trying to do?
$ awk 'NR==FNR{new=(NR>1?new ORS:"") $0;next} $0=="DINGO=bingo"{$0=new} 1' - file <<!
# added by $(whoami) on $(date)
DINGO=howdy
!
# added by user on Sun, Feb 3, 2019 8:50:41 AM
DINGO=howdy
Note that the above is using literal string operations so it'll work for any characters in the old or new strings unlike your sed script which would fail given /s or any ERE regexp character or capture groups or backreferences or... in the input (see Is it possible to escape regex metacharacters reliably with sed for details).
This might work for you (GNU sed):
cat <<! | sed -i -e '/^DINGO/r /dev/stdin' -e '//d' file
# added by $(whoami) on $(date)
DINGO=howdy
!
This replaces lines starting DINGO with the here-document which is piped to stdin as file within the sed command.
An alternative:
cat <<! | sed -i -e '/^DINGO/e cat /dev/stdin' -e 's/bingo/howdy' file
# added by $(whoami) on $(date)
!
N.B. In the alternative solution the here-doc will only be read once!
I have file called "text_file1.txt" and the content in the file is
"subject= /C=US/O=AAA/OU=QA/OU=12345/OU=TESTAPP/"
Now what i want to achieve is to the content to be like below:
"subject= /C=US/O=AAA/$$$QA/###12345/###TESTAPP/"
when i execute the below piece of code:
#! /bin/ksh
OU1="QA"
OU2=12345
OU3="TESTAPP"
`sed -i "s/OU=$OU1/$$$\${OU1}/g" text_file1.txt`
`sed -i "s/OU=$OU2/###\${OU2}/g" text_file1.txt`
`sed -i "s/OU=$OU3/###\${OU3}/g" text_file1.txt`
content=`cat text_file1.txt`
echo "content:$content"
i get the output like this:
content:subject= /C=US/O=Wells Fargo/2865528655{OU1}/###12345/###TESTAPP/CN=03032015_CUST_2131_Unix_CLBLABB34C02.wellsfargo.com
only this command "sed -i "s/OU=$OU1/$$$\${OU1}/g" text_file1.txt" is not working as expected.Can anyone please suggest some idea on this?
Thanks in advance.
Two things play into this:
You have to escape $ (i.e., use \$) in doubly-quoted shell strings if you want a literal $, and
\ does not retain its literal meaning when it comes before a $ inside backticks (that is to say, inside backticks, \$ becomes just $).
When you write
`sed -i "s/OU=$OU1/$$$\${OU1}/g" text_file1.txt`
because the command is in backticks, you spawn a subshell with the command
sed -i "s/OU=$OU1/$$$${OU1}/g" text_file1.txt
Since $$$$ is inside a doubly-quoted string, variable expansion takes place, and it is expanded as two occurrences of $$ (the process ID of the shell that's doing the expansion). This means that the code sed sees is ultimately
s/OU=QA/1234512345{OU1}/g
...if the process ID of the spawned subshell is 12345.
In this particular case, you don't need the command substitution (the backticks), so you could write
sed -i "s/OU=$OU1/\$\$\$${OU1}/g" text_file1.txt
However, using shell variables in sed code is always a problem. Consider, if you will, what would happen if OU1 had the value /; e rm -Rf * # (hint: GNU sed has an e instruction that runs shell commands). For this reason, I would always prefer awk to do substitutions that involve shell variables:
cp text_file1.txt text_file1.txt~
awk -v OU1="$OU1" '{ gsub("OU=" OU1, "$$$" OU1) } 1' text_file1.txt~ > text_file1.txt
This avoids code injection problems by not treating OU1 as code.
If you have GNU awk 4.1 or later,
awk -v OU1="$OU1" -i inplace '{ gsub("OU=" OU1, "$$$" OU1) } 1' text_file1.txt
can do the whole thing without a (visible) temporary file.
Does this help as a start?
echo ''
OU1="QA"
echo "subject= /C=US/O=AAA/OU=${OU1}/OU=12345/OU=TESTAPP/" \
| sed -e "s|/OU=${OU1}/|/OU=\$\$\$${OU1}/|g"
The result is:
subject= /C=US/O=AAA/OU=$$$QA/OU=12345/OU=TESTAPP/
(You are mixing up the use of $ signs .)
You must be careful when putting $ inside double quotes.
sed -i "s/OU=$OU1/"'$$$'"${OU1}/g" text_file1.txt
Example:
$ OU1="QA"
$ echo 'OU=QA' | sed "s/OU=$OU1/"'$$$'"${OU1}/g"
$$$QA
I have abc.sh:
exec $ROOT/Subsystem/xyz.sh
On a Unix box, if I print echo $HOME then I get /HOME/COM/FILE.
I want to replace $ROOT with $HOME using sed.
Expected Output:
exec /HOME/COM/FILE/Subsystem/xyz.sh
I tried, but I'm not getting the expected output:
sed 's/$ROOT/"${HOME}"/g' abc.sh > abc.sh.1
Addition:
If I have abc.sh
exec $ROOT/Subsystem/xyz.sh $ROOT/ystem/xyz1.sh
then with
sed "s|\$INSTALLROOT/|${INSTALLROOT}|" abc.sh
it is only replacing first $ROOT, i.e., output is coming as
exec /HOME/COM/FILE/Subsystem/xyz.sh $ROOT/ystem/xyz1.sh
Say:
sed "s|\$ROOT|${HOME}|" abc.sh
Note:
Use double quotes so that the shell would expand variables.
Use a separator different than / since the replacement contains /
Escape the $ in the pattern since you don't want to expand it.
EDIT: In order to replace all occurrences of $ROOT, say
sed "s|\$ROOT|${HOME}|g" abc.sh
This might work for you:
sed 's|$ROOT|'"${HOME}"'|g' abc.sh > abc.sh.1
This may also can help
input="inputtext"
output="outputtext"
sed "s/$input/${output}/" inputfile > outputfile
The safe for a special chars workaround from https://www.baeldung.com/linux/sed-substitution-variables with improvement for \ char:
#!/bin/bash
to="/foo\\bar#baz"
echo "str %FROM% str" | sed "s#%FROM%#$(echo ${to//\\/\\\\} | sed 's/#/\\#/g')#g"
I have a bash script which checks for a string pattern in file and delete entire line i same file but somehow its not deleting the line and no throwing any error .same command from command prompt deletes from file .
#array has patterns
for k in "${patternarr[#]}
do
sed -i '/$k/d' file.txt
done
sed version is >4
when this loop completes i want all lines matching string pattern in array to be deleted from file.txt
when i run sed -i '/pataern/d file.txt from command prompt then it works fine but not inside bash
Thanks in advance
Here:
sed -i '/$k/d' file.txt
The sed script is singly-quoted, which prevents shell variable expansion. It will (probably) work with
sed -i "/$k/d" file.txt
I say "probably" because what it will do depends on the contents of $k, which is just substituted into the sed code and interpreted as such. If $k contains slashes, it will break. If it comes from an untrustworthy source, you open yourself up to code injection (particularly with GNU sed, which can be made to execute shell commands).
Consider k=^/ s/^/rm -Rf \//e; #.
It is generally a bad idea to substitute shell variables into sed code (or any other code). A better way would be with GNU awk:
awk -i inplace -v pattern="$k" '!($0 ~ pattern)' file.txt
Or to just use grep -v and a temporary file.
first of all, you got an unclosed double quote around ${patternarr[#]} in your for statement.
Then your problem is that you use single quotes in the sed argument, making your shell not evaluate the $k within the quotes:
% declare -a patternarr=(foo bar fu foobar)
% for k in ${patternarr[#]}; do echo sed -i '/$k/d' file.txt; done
sed -i /$k/d file.txt
sed -i /$k/d file.txt
sed -i /$k/d file.txt
sed -i /$k/d file.txt
if you replace them with double quotes, here it goes:
% for k in ${patternarr[#]}; do echo sed -i "/$k/d" file.txt; done
sed -i /foo/d file.txt
sed -i /bar/d file.txt
sed -i /fu/d file.txt
sed -i /foobar/d file.txt
Any time you write a loop in shell just to manipulate text you have the wrong approach. This is probably closer to what you really should be doing (no surrounding loop required):
awk -v ks="${patternarr[#]}" 'BEGIN{gsub(/ /,")|(",ks); ks="("ks")} $0 !~ ks' file.txt
but there may be even better approaches still (e.g. only checking 1 field instead of the whole line, or using word boundaries, or string comparison or....) if you show us some sample input and expected output.
You need to use double quotes to interpolate shell variables inside the sed command, like:
for k in ${patternarr[#]}; do
sed -i "/$k/d" file.txt
done
I have a large number of words in a text file to replace.
This script is working up until the sed command where I get:
sed: 1: "*.js": invalid command code *
PS... Bash isn't one of my strong points - this doesn't need to be pretty or efficient
cd '/Users/xxxxxx/Sites/xxxxxx'
echo `pwd`;
for line in `cat myFile.txt`
do
export IFS=":"
i=0
list=()
for word in $line; do
list[$i]=$word
i=$[i+1]
done
echo ${list[0]}
echo ${list[1]}
sed -i "s/{$list[0]}/{$list[1]}/g" *.js
done
You're running BSD sed (under OS X), therefore the -i flag requires an argument specifying what you want the suffix to be.
Also, no files match the glob *.js.
This looks like a simple typo:
sed -i "s/{$list[0]}/{$list[1]}/g" *.js
Should be:
sed -i "s/${list[0]}/${list[1]}/g" *.js
(just like the echo lines above)
So myFile.txt contains a list of from:to substitutions, and you are looping over each of those. Why don't you create a sed script from this file instead?
cd '/Users/xxxxxx/Sites/xxxxxx'
sed -e 's/^/s:/' -e 's/$/:/' myFile.txt |
# Output from first sed script is a sed script!
# It contains substitutions like this:
# s:from:to:
# s:other:substitute:
sed -f - -i~ *.js
Your sed might not like the -f - which means sed should read its script from standard input. If that is the case, perhaps you can create a temporary script like this instead;
sed -e 's/^/s:/' -e 's/$/:/' myFile.txt >script.sed
sed -f script.sed -i~ *.js
Another approach, if you don't feel very confident with sed and think you are going to forget in a week what the meaning of that voodoo symbols is, could be using IFS in a more efficient way:
IFS=":"
cat myFile.txt | while read PATTERN REPLACEMENT # You feed the while loop with stdout lines and read fields separated by ":"
do
sed -i "s/${PATTERN}/${REPLACEMENT}/g"
done
The only pitfall I can see (it may be more) is that if whether PATTERN or REPLACEMENT contain a slash (/) they are going to destroy your sed expression.
You can change the sed separator with a non-printable character and you should be safe.
Anyway, if you know whats on your myFile.txt you can just use any.