I'm trying to use the send command with a sed which uses a variable.
Having trouble to escape correctly.
send "sed "1i//$VAR" /file > /tmp/out\r"
If I use the sed command separately (which adds the $VAR text as first line of file), it works:
sed "1i//$VAR" /file > /tmp/out
But I can't figure out how to escape within the send command.
Inside double quotes, single quotes lose their special meaning, so you probably need:
send "sed '1i//$VAR' /file > /tmp/out\r"
On the local machine, the $VAR is placed into the command. On the remote machine, the sed command is enclosed in single quotes, protecting it from further abuse.
This should work:
send "sed '1i//$VAR' /file > /tmp/out\r"
Only the type of the outermost quotes matters for determining whether variables are interpolated.
Related
in jenkins-pipeline, I'm trying to use SED to append the following line to the end of a file.
sh "sed -i '\$ s/\$/ public_file=\\/var\\/lib\\/jenkins\\/workspace\\/test-project\\ ansible_ssh_common_args='-o StrictHostKeyChecking=no' /' file.txt"
but I can't figure out how to escape the below line in my Jenkins file to make it work.
ansible_ssh_common_args='-o StrictHostKeyChecking=no'
I've already tried the following, which works in katacoda playground, but not in jenkins pipeline.
'"'"'-o StrictHostKeyChecking=no'"'"' /' file.txt
use " to wrap sed command, then you can use ' in command directly without to escape it.
use #, but / as delimiter for sed s command, then you no need to escape the / appeared in file path to make the whole commend more concise and readable.
sed -i "\$ s#\$# public_file=/var/lib/jenkins/workspace/test-project ansible_ssh_common_args='-o StrictHostKeyChecking=no' #" file.txt
In bash, you cannot escape single quotes within single quotes.
See the bash manual page:
Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
You can use double quotes instead; then escape them as needed. You can then use single quotes within them without further escaping.
I am trying to change the values in a text file using sed in a Bash script with the line,
sed 's/draw($prev_number;n_)/draw($number;n_)/g' file.txt > tmp
This will be in a for loop. Why is it not working?
Variables inside ' don't get substituted in Bash. To get string substitution (or interpolation, if you're familiar with Perl) you would need to change it to use double quotes " instead of the single quotes:
# Enclose the entire expression in double quotes
$ sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
# Or, concatenate strings with only variables inside double quotes
# This would restrict expansion to the relevant portion
# and prevent accidental expansion for !, backticks, etc.
$ sed 's/draw('"$prev_number"';n_)/draw('"$number"';n_)/g' file.txt > tmp
# A variable cannot contain arbitrary characters
# See link in the further reading section for details
$ a='foo
bar'
$ echo 'baz' | sed 's/baz/'"$a"'/g'
sed: -e expression #1, char 9: unterminated `s' command
Further Reading:
Difference between single and double quotes in Bash
Is it possible to escape regex metacharacters reliably with sed
Using different delimiters for sed substitute command
Unless you need it in a different file you can use the -i flag to change the file in place
Variables within single quotes are not expanded, but within double quotes they are. Use double quotes in this case.
sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
You could also make it work with eval, but don’t do that!!
This may help:
sed "s/draw($prev_number;n_)/draw($number;n_)/g"
You can use variables like below. Like here, I wanted to replace hostname i.e., a system variable in the file. I am looking for string look.me and replacing that whole line with look.me=<system_name>
sed -i "s/.*look.me.*/look.me=`hostname`/"
You can also store your system value in another variable and can use that variable for substitution.
host_var=`hostname`
sed -i "s/.*look.me.*/look.me=$host_var/"
Input file:
look.me=demonic
Output of file (assuming my system name is prod-cfm-frontend-1-usa-central-1):
look.me=prod-cfm-frontend-1-usa-central-1
I needed to input github tags from my release within github actions. So that on release it will automatically package up and push code to artifactory.
Here is how I did it. :)
- name: Invoke build
run: |
# Gets the Tag number from the release
TAGNUMBER=$(echo $GITHUB_REF | cut -d / -f 3)
# Setups a string to be used by sed
FINDANDREPLACE='s/${GITHUBACTIONSTAG}/'$(echo $TAGNUMBER)/
# Updates the setup.cfg file within version number
sed -i $FINDANDREPLACE setup.cfg
# Installs prerequisites and pushes
pip install -r requirements-dev.txt
invoke build
Retrospectively I wish I did this in python with tests. However it was fun todo some bash.
Another variant, using printf:
SED_EXPR="$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)"
sed "${SED_EXPR}" file.txt
or in one line:
sed "$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)" file.txt
Using printf to build the replacement expression should be safe against all kinds of weird things, which is why I like this variant.
I want to issue this command from the bash script
sed -e $beginning,$s/pattern/$variable/ file
but any possible combination of quotes gives me an error, only one that works:
sed -e "$beginning,$"'s/pattern/$variable/' file
also not good, because it do not dereferences the variable.
Does my approach can be implemented with sed?
Feel free to switch the quotes up. The shell can keep things straight.
sed -e "$beginning"',$s/pattern/'"$variable"'/' file
You can try this:
$ sed -e "$beginning,$ s/pattern/$variable/" file
Example
file.txt:
one
two
three
Try:
$ beginning=1
$ variable=ONE
$ sed -e "$beginning,$ s/one/$variable/" file.txt
Output:
ONE
two
three
There are two types of quotes:
Single quotes preserve their contents (> is the prompt):
> var=blah
> echo '$var'
$var
Double quotes allow for parameter expansion:
> var=blah
> echo "$var"
blah
And two types of $ sign:
One to tell the shell that what follows is the name of a parameter to be expanded
One that stands for "last line" in sed.
You have to combine these so
The shell doesn't think sed's $ has anything to do with a parameter
The shell parameters still get expanded (can't be within single quotes)
The whole sed command is quoted.
One possibility would be
sed "$beginning,\$s/pattern/$variable/" file
The whole command is in double quotes, i.e., parameters get expanded ($beginning and $variable). To make sure the shell doesn't try to expand $s, which doesn't exist, the "end of line" $ is escaped so the shell doesn't try anything funny.
Other options are
Double quoting everything but adding a space between $ and s (see Ren's answer)
Mixing quoting types as needed (see Ignacio's answer)
Methods that don't work
sed '$beginning,$s/pattern/$variable/' file
Everything in single quotes: the shell parameters are not expanded (doesn't follow rule 2 above). $beginning is not a valid address, and pattern would be literally replaced by $variable.
sed "$beginning,$s/pattern/$variable/" file
Everything in double qoutes: the parameters are expanded, including $s, which isn't supposed to (doesn't follow rule 1 above).
the following form worked for me from within script
sed $beg,$ -e s/pattern/$variable/ file
the same form will also work if executed from the shell
I want to add some text on a line:
sudo sed -i '5imytext 16/16' /file
Now I've added mytext 16/16 on line 5 of the file, but I actually want to add the text 'mytext' 16/16 (mytext between single quotes).
I tried
sudo sed -i '5i'mytext' 16/16' /file
but it didn't work. Can someone help me?
The single quotes that you're trying to use in your insertion string are interfering with the ones around the sed command.
The simplest thing to do is to use different quotes around your sed command:
"5i'mytext' 16/16"
Normally it's best to use single quotes around a sed command but it would be more tricky in this case:
'5i'"'"'mytext'"'"' 16/16'
Basically, you need to put the single quotes inside double quotes somehow and in this case there's no reason not to double quote the whole command.
As suggested by 123 in the comments, an alternative would be to put your sed command into a script file:
5i'mytext' 16/16
Then use the -f switch to sed:
sed -f script
This avoids the need to use two kinds of quotes.
Use double quote in these cases. Because:
Single quote can't have single quote inside it. ('\'' won't work)
Double quote can have both single quote and double quote inside it. ("'\"" will work)
Example:
sudo sed -i "5i'mytext' 16/16" /file
You could use double quotes around your sed command, but that won't help you if you also need to insert double quotes. An alternative would be to use: \x27
Example: echo a|sed 's/a/\x27/' ➡ '
I have written a bash script which calls a sed command (amongst other things) on a file to complete a find/replace of 2 different strings.
The trouble is, after running the script, I check the files and nothing has been updated. However, if I run the commands that are being produced (I echo them as output anyway) then they work.
For example, inside the script I have:
echo "/usr/local/bin/sed -i -e 's/${String1}/${String1R}/g;s/\/${String2}\//\/${String2R}\//g' ${ROOT_DIR}/data/file.sql"
/usr/local/bin/sed -i -e 's/${String1}/${String1R}/g;s/\/${String2}\//\/${TString2R}\//g' ${ROOT_DIR}/data/file.sql
Running the script does not change file.sql; however, if I run the command that is printed to console e.g. /usr/local/bin/sed -i -e 's/file_name1/file_name2/g;s//path_substring1///path_substring2//g' /path/to/file/file.sql it works perfectly!
Use double quotes instead of single quotes. Single quotes would prevent variable expansion.
/usr/local/bin/sed -i -e "s/${String1}/${String1R}/g;s/\/${String2}\//\/${TString2R}\//g" ${ROOT_DIR}/data/file.sql
Moreover, it seems that your variables are path strings which might contain forward slashes, i.e. /. In that event use a different separator:
"s|${String1}|${String1R}|g"
Using a different separator would obviate the need of escaping / in the pattern and replacement.