This question already has answers here:
Environment variable substitution in sed
(12 answers)
sed substitution with Bash variables
(6 answers)
Closed 10 months ago.
I can't seem to use a variable in a sed command, for example:
sed "24s/.*/"$ct_tname"/" file1.sas > file2.sas
I want $ct_tname the variable, not literally $ct_tname, which is what I keep getting.
Anybody know how to get this to work?
The problem is actually more complex and I omitted some information.
ct_fname="%let outputfile="/user/ct_"$1".csv";"
Here, $1 is the argument passed in at the start of my bash script (sed is being run inside a bash script).
This doesn't run successfully, but it does run if I replace ct_fname with
ct_fname="%let table=ct_$1;"
Is there a way to get the first ct_fname to be passed successfully?
you need to use double quotes (") instead of single quotes (').
single quotes pass their content literally, without translating variables (expansion).
try
sed "24s/.*/\"$ct_tname\"/" file1.sas > file2.sas
btw, if you're going to be editing a file (that is if file2.sas is a temporary file), you should be using ed instead.
In my case, i just remplaced single quotes by the double ones:
for a in $(cat ext.cnf); do sed -n "/$a$/p" file1 >> file2; done
For now, it's working well...
The problem is that when $ct_fname is substituted, sed sees extra / separators, so
sed "24s/.*/"$ct_tname"/" file1.sas > file2.sas
becomes
sed "24s/.*/"%let outputfile=/user/ct_ARGUMENT1.csv;"/" file1.sas > file2.sas
and you'll get a sed error because there are 5 / instead of the expected 3.
Instead, change your sed separators to an unused character like | or :, and either single or double quotes will work just fine:
sed '24s|.*|'$ct_tname'|' file1.sas > file2.sas
sed "24s|.*|"$ct_tname"|" file1.sas > file2.sas
Shell variables are not expanded inside single quotes. Try this instead:
sed "24s/.*/\"$ct_tname\"/" file1.sas > file2.sas
You need to use double (") quotes, with single (') quotes the value of the variable doesn't get replaced. Since you have double quotes in your replacement text, you need to escape them:
sed "24s/.*/\"$ct_tname\"/" file1.sas > file2.sas
Other answers focus on the use of escaped double quotes in their examples. Note that this is not always what you want :
$ FOO="auie"; echo foo123bar|sed "s/123/\"$FOO\"/"
foo"auie"bar
$ FOO="auie"; echo foo123bar|sed "s/123/$FOO/"
fooauiebar
$ FOO="auie"; echo fooauiebar|sed "s/\"$FOO\"/123/"
fooauiebar
$ FOO="auie"; echo fooauiebar|sed "s/$FOO/123/"
foo123bar
Related
I am trying to change the values in a text file using sed in a Bash script with the line,
sed 's/draw($prev_number;n_)/draw($number;n_)/g' file.txt > tmp
This will be in a for loop. Why is it not working?
Variables inside ' don't get substituted in Bash. To get string substitution (or interpolation, if you're familiar with Perl) you would need to change it to use double quotes " instead of the single quotes:
# Enclose the entire expression in double quotes
$ sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
# Or, concatenate strings with only variables inside double quotes
# This would restrict expansion to the relevant portion
# and prevent accidental expansion for !, backticks, etc.
$ sed 's/draw('"$prev_number"';n_)/draw('"$number"';n_)/g' file.txt > tmp
# A variable cannot contain arbitrary characters
# See link in the further reading section for details
$ a='foo
bar'
$ echo 'baz' | sed 's/baz/'"$a"'/g'
sed: -e expression #1, char 9: unterminated `s' command
Further Reading:
Difference between single and double quotes in Bash
Is it possible to escape regex metacharacters reliably with sed
Using different delimiters for sed substitute command
Unless you need it in a different file you can use the -i flag to change the file in place
Variables within single quotes are not expanded, but within double quotes they are. Use double quotes in this case.
sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
You could also make it work with eval, but don’t do that!!
This may help:
sed "s/draw($prev_number;n_)/draw($number;n_)/g"
You can use variables like below. Like here, I wanted to replace hostname i.e., a system variable in the file. I am looking for string look.me and replacing that whole line with look.me=<system_name>
sed -i "s/.*look.me.*/look.me=`hostname`/"
You can also store your system value in another variable and can use that variable for substitution.
host_var=`hostname`
sed -i "s/.*look.me.*/look.me=$host_var/"
Input file:
look.me=demonic
Output of file (assuming my system name is prod-cfm-frontend-1-usa-central-1):
look.me=prod-cfm-frontend-1-usa-central-1
I needed to input github tags from my release within github actions. So that on release it will automatically package up and push code to artifactory.
Here is how I did it. :)
- name: Invoke build
run: |
# Gets the Tag number from the release
TAGNUMBER=$(echo $GITHUB_REF | cut -d / -f 3)
# Setups a string to be used by sed
FINDANDREPLACE='s/${GITHUBACTIONSTAG}/'$(echo $TAGNUMBER)/
# Updates the setup.cfg file within version number
sed -i $FINDANDREPLACE setup.cfg
# Installs prerequisites and pushes
pip install -r requirements-dev.txt
invoke build
Retrospectively I wish I did this in python with tests. However it was fun todo some bash.
Another variant, using printf:
SED_EXPR="$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)"
sed "${SED_EXPR}" file.txt
or in one line:
sed "$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)" file.txt
Using printf to build the replacement expression should be safe against all kinds of weird things, which is why I like this variant.
this is perhaps one of the most discussed topics here. I tried almost all the commands and other tweaks found here, but something doesn't seems to be doing well.
i would want to replace all the double quotes in my file with whitespace/blank
I'm seeing the below error when i tried to execute this command.
sed "s/"/ \''/g' x_orbit.txt > new.tx
sed: -e expression #1, char 3: unterminated `s' command
You're close. Just use single quotes, so the shell doesn't try to expand the metacharacters in your sed command:
sed 's/"/ /g' x_orbit.txt > new.txt
You could try tr for example:
tr '"' ' ' < x_orbit.txt > new.txt
The script you provided:
sed "s/"/ \''/g' x_orbit.txt > new.tx
means:
sed # invoke sed to execute the following script:
" # enclose the script in double quotes rather than single so the shell can
# interpret it (e.g. to expand variables like $HOME) before sed gets to
# interpret the result of that expansion
s/ # replace what follows until the next /
" # exit the double quotes so the shell can now not only expand variables
# but can now do globbing and file name expansion on wildcards like foo*
/ # end the definition of the regexp you want to replace so it is null since
# after the shell expansion there was no text for sed to read between
# this / and the previous one (the 2 regexp delimiters)
\' # provide a blank then an escaped single quote for the shell to interpret for some reason
'/g' # enclose the /g in single quotes as all scripts should be quoted by default.
That is so far off the correct syntax it's kinda shocking which is why I dissected it above to try to help you understand what you wrote so you'll see why it doesn't work. Where did you get the idea to write it that way (or to put it another way - what did you think each character in that script meant? I'm asking as it indicates a fundamental misunderstanding of how quoting and escaping works in shell so it'd be good if we could help correct that misunderstanding rather than just correct that script.
When you use any script or string in shell, simply always enclose it in single quotes:
sed 'script' file
var='string'
unless you NEED to use double quotes to let a variable expand and then use double quotes unless you NEED to use no quotes to let globbing and file name expansion happen.
An awk version:
awk '{gsub(/"/," ")}1' file
gsub is used for the replace
1 is always true, so line is printed
I want to issue this command from the bash script
sed -e $beginning,$s/pattern/$variable/ file
but any possible combination of quotes gives me an error, only one that works:
sed -e "$beginning,$"'s/pattern/$variable/' file
also not good, because it do not dereferences the variable.
Does my approach can be implemented with sed?
Feel free to switch the quotes up. The shell can keep things straight.
sed -e "$beginning"',$s/pattern/'"$variable"'/' file
You can try this:
$ sed -e "$beginning,$ s/pattern/$variable/" file
Example
file.txt:
one
two
three
Try:
$ beginning=1
$ variable=ONE
$ sed -e "$beginning,$ s/one/$variable/" file.txt
Output:
ONE
two
three
There are two types of quotes:
Single quotes preserve their contents (> is the prompt):
> var=blah
> echo '$var'
$var
Double quotes allow for parameter expansion:
> var=blah
> echo "$var"
blah
And two types of $ sign:
One to tell the shell that what follows is the name of a parameter to be expanded
One that stands for "last line" in sed.
You have to combine these so
The shell doesn't think sed's $ has anything to do with a parameter
The shell parameters still get expanded (can't be within single quotes)
The whole sed command is quoted.
One possibility would be
sed "$beginning,\$s/pattern/$variable/" file
The whole command is in double quotes, i.e., parameters get expanded ($beginning and $variable). To make sure the shell doesn't try to expand $s, which doesn't exist, the "end of line" $ is escaped so the shell doesn't try anything funny.
Other options are
Double quoting everything but adding a space between $ and s (see Ren's answer)
Mixing quoting types as needed (see Ignacio's answer)
Methods that don't work
sed '$beginning,$s/pattern/$variable/' file
Everything in single quotes: the shell parameters are not expanded (doesn't follow rule 2 above). $beginning is not a valid address, and pattern would be literally replaced by $variable.
sed "$beginning,$s/pattern/$variable/" file
Everything in double qoutes: the parameters are expanded, including $s, which isn't supposed to (doesn't follow rule 1 above).
the following form worked for me from within script
sed $beg,$ -e s/pattern/$variable/ file
the same form will also work if executed from the shell
This question already has answers here:
sed substitution with Bash variables
(6 answers)
Closed 8 years ago.
I want to search for a string enclosed in double quotes given by a positional parameter. This is what I've done:
#!/bin/bash
FOO=$0
sed -i 's/"${FOO}"/bar/m' file.txt
Neither "$FOO" nor \"a${FOO}\"/ work.
I am aware that for variables to be expanded, you have to enclose the whole regex in double quotes (sed substitution with bash variables). The thing is that the string I am searching for is also enclosed in double quotes, that is why I enclosed the whole regex in single quotes. I also tried to enclose it in double quotes and escaping the double quotes, but it didn't work.
An explanation of the logic behind single and double quotes escaping would be really useful.
Use double quotes to quote the sed commands as
sed -i "s/\"${FOO}\"/bar/m" file.txt
Test:
$ echo $b
hello
$ echo \"hello\" | sed "s/\"$b\"/asdf/"
asdf
This question already has answers here:
sed substitution with Bash variables
(6 answers)
Closed 6 years ago.
I execute the following bash script:
#!/bin/bash
version=$1
echo $version
sed 's/\${version.number}/$version/' template.txt > readme.txt
I'm expecting to replace all instances of ${version.number} with the contents of the variable "version". Instead the literal text $version is being inserted.
What do I need to do to make sed use the current value of $version instead?
sed "s/\${version.number}/$version/" template.txt > readme.txt
Only double quotes do dollar-sign replacement. That also means single quotes don't require the dollar sign to be escaped.
You could also simply unquote the variables
sed 's/'${version.number}'/'$version'/' template.txt > readme.txt
It's a bit old, but it might still be helpful...
For me it worked using a double escape as Philip said (and escaping parenthesis, if you use them...):
#!/bin/bash
LIVE_DB_NAME='wordpress';
STAGING='staging';
sed -r "s/define\('DB_NAME', '[a-zA-Z0-9]+'\);/define('DB_NAME', '\\${LIVE_DB_NAME}');/" ${STAGING}/wp-config.php >tmp1;