I am using sed to replace url's in a file, everything works fine just a hiccup when the url contains a '\'
exmaple url: http**://www.example.com/simi/icr
# variables
ICR_KEY=somekey
ICR_KEY_VAL="http\://www.example.com/simi/icr"
sed "s!${ICR_KEY}=.*!${ICR_KEY}=${ICR_KEY_VAL}!" properties > tmp
This replaces the URL, but the output does not contain the backslash from the variable value.
Both bash and sed interpret the backslash as escape character. Use single quotes to prevent this for bash, and double the backslash for sed:
ICR_KEY_VAL='http\\://www.example.com/simi/icr'
Related
in jenkins-pipeline, I'm trying to use SED to append the following line to the end of a file.
sh "sed -i '\$ s/\$/ public_file=\\/var\\/lib\\/jenkins\\/workspace\\/test-project\\ ansible_ssh_common_args='-o StrictHostKeyChecking=no' /' file.txt"
but I can't figure out how to escape the below line in my Jenkins file to make it work.
ansible_ssh_common_args='-o StrictHostKeyChecking=no'
I've already tried the following, which works in katacoda playground, but not in jenkins pipeline.
'"'"'-o StrictHostKeyChecking=no'"'"' /' file.txt
use " to wrap sed command, then you can use ' in command directly without to escape it.
use #, but / as delimiter for sed s command, then you no need to escape the / appeared in file path to make the whole commend more concise and readable.
sed -i "\$ s#\$# public_file=/var/lib/jenkins/workspace/test-project ansible_ssh_common_args='-o StrictHostKeyChecking=no' #" file.txt
In bash, you cannot escape single quotes within single quotes.
See the bash manual page:
Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
You can use double quotes instead; then escape them as needed. You can then use single quotes within them without further escaping.
this is perhaps one of the most discussed topics here. I tried almost all the commands and other tweaks found here, but something doesn't seems to be doing well.
i would want to replace all the double quotes in my file with whitespace/blank
I'm seeing the below error when i tried to execute this command.
sed "s/"/ \''/g' x_orbit.txt > new.tx
sed: -e expression #1, char 3: unterminated `s' command
You're close. Just use single quotes, so the shell doesn't try to expand the metacharacters in your sed command:
sed 's/"/ /g' x_orbit.txt > new.txt
You could try tr for example:
tr '"' ' ' < x_orbit.txt > new.txt
The script you provided:
sed "s/"/ \''/g' x_orbit.txt > new.tx
means:
sed # invoke sed to execute the following script:
" # enclose the script in double quotes rather than single so the shell can
# interpret it (e.g. to expand variables like $HOME) before sed gets to
# interpret the result of that expansion
s/ # replace what follows until the next /
" # exit the double quotes so the shell can now not only expand variables
# but can now do globbing and file name expansion on wildcards like foo*
/ # end the definition of the regexp you want to replace so it is null since
# after the shell expansion there was no text for sed to read between
# this / and the previous one (the 2 regexp delimiters)
\' # provide a blank then an escaped single quote for the shell to interpret for some reason
'/g' # enclose the /g in single quotes as all scripts should be quoted by default.
That is so far off the correct syntax it's kinda shocking which is why I dissected it above to try to help you understand what you wrote so you'll see why it doesn't work. Where did you get the idea to write it that way (or to put it another way - what did you think each character in that script meant? I'm asking as it indicates a fundamental misunderstanding of how quoting and escaping works in shell so it'd be good if we could help correct that misunderstanding rather than just correct that script.
When you use any script or string in shell, simply always enclose it in single quotes:
sed 'script' file
var='string'
unless you NEED to use double quotes to let a variable expand and then use double quotes unless you NEED to use no quotes to let globbing and file name expansion happen.
An awk version:
awk '{gsub(/"/," ")}1' file
gsub is used for the replace
1 is always true, so line is printed
I am on Mac OS X and using sed for an in-place replacement.
Essentially I have this:
#!/bin/sh -e
PREFIX="$1"
sed -i bak -e 's|OCAMLDIR|"${PREFIX}"|g' ocamloptrev
Where PREFIX is a path, hence I'm using the |.
Unfortunately, the variable in the file path is not getting evaluated as I expected, I end up with:
OCAMLC="${PREFIX}"/bin/ocamlopt
How can I get the right evaluation of ${PREFIX} into the sed command?
Try this:
#!/bin/sh -e
PREFIX="$1"
sed -i bak -e 's|OCAMLDIR|'"${PREFIX}"'|g' ocamloptrev
What you're basically doing, is "exiting"/getting outside the single-quoted string, entering into a double-quoted string, interpreting the variable inside double-quotes, and then entering the single quotes again.
With this simple example, we could also just use double-quotes, which allow variables to be interpreted:
#!/bin/sh -e
PREFIX="$1"
sed -i bak -e "s|OCAMLDIR|${PREFIX}|g" ocamloptrev
If you try to use double-quotes ("") inside single-quotes, they don't get interpreted either. This part of the Bash manual explains this in more detail.
3.1.2.2 Single Quotes
Enclosing characters in single quotes (‘'’) preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
3.1.2.3 Double Quotes
Enclosing characters in double quotes (‘"’) preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes (see Shell Expansions). ...
Shell variables are not expanded inside single quotes (there are no metacharacters within single quotes, not even backslashes), so you need to use something like this, with the double quotes around ${PREFIX} ensuring that spaces etc in the value are handled correctly:
sed -i bak -e 's|OCAMLDIR|'"${PREFIX}"'|g' ocamloptrev
Or you could even use:
sed -i bak -e "s|OCAMLDIR|${PREFIX}|g" ocamloptrev
The latter is safe because the material inside the double quotes does not contain shell metacharacters (dollar signs, backslashes and back-quotes are the main danger signs). If there were dodgy characters in the rest of the string, the first version is safer to use.
Personally, I'd use .bak rather than just bak as the suffix.
I have file with comments like this:
\$max_servers = 2;
\#\## BLOCKED ANYWHERE
I'm trying to
Replace all instances of \$ with $.
Replace all instances of \#\## with ###.
I wonder how I can go about doing that via sed or awk
What I have tried so far without much success using vi or vim
%s/^\//gc
%s/^#/\\/###/gc
Thank you
Another option to replace all [#$] in one pass is to use the following regular expression. The following is VI syntax:
:%s/\\\([$#]\)/\1/g
Replace the characters in the brackets [] with whatever you need if its more than just # and $.
The first \\ is a backslash - escaped since its inside a regular expression
The expression between the \( and \) is saved and later used in the replacement as \1.
Escaping backslash will work
#echo "\#\##"| sed "s/\\\\#\\\\##/###/g"
###
# echo "\\$"| sed "s/\\\\\\$/$/g"
$
In order to replace a backslash, you have to double it up, so it can quote itself much the way other special characters must be quoted. You can use sed instead of vim to help automate the process a bit:
$ sed -e 's/^\\\$/$/' -e 's/^\\#\\##/###/' $file > $new_file
Note that you have to put a backslash in front of dollar signs since they are used to mark an end of line in regular expressions. That's why I have \\\$ in my first expression. One backslash to quote the backslash and another backslash to quote the dollar sign.
By the way, these same sed expressions will also work inside Vim depending upon your Vim settings.
You escape special characters with the backslash. So for example, to replace everything with \$, you would do
%s/\\\$/$/g
sed 's|^\\\([$#]\)\\\{0,1\}|\1|' YourFile
work for your sample bu will also remove the 2 \ in \$\ ...,
I need use sed into bash script, for add lines after any line numer of script with some pair of values (below work)
sed -i.bak '14i\some_text=some_text' file
But I need on script bash (sh) for expand variables (below not work)
sed -i.bak '$number_linei\$var1=$var2' $var3
Just use double quotes instead of single quotes. You'll also need to use {} to delimit the number_line variable correctly and escape the \, too.
sed -i.bak "${number_line}i\\$var1=$var2" $var3
I'd personally prefer to see all of the variables use the {}, ending up with something like:
sed -i.bak "${number_line}i\\${var1}=${var2}" ${var3}
Change single quotes to double quotes:
man bash:
Enclosing characters in single quotes preserves the literal value of
each character within the quotes.
Enclosing characters in double quotes preserves the literal value of
all characters within the quotes, with the exception of $, `, \, and,
when history expansion is enabled, !. The characters $ and ` retain
their special meaning within double quotes.