This question already has answers here:
How to escape single quote in sed?
(9 answers)
Closed 2 years ago.
I want to replace a set of characters with ' using sed.
This post suggest:
With single quotes around the argument (sed 's/…/…/'), use '\'' to put a single quote in the replacement text.
So, I tried following:
echo 'abcd' | sed 's/[abcd]/\'/g'
But it simply ends up expecting more input:
anir#DESKTOP-4856511:~$ echo 'abcd' | sed 's/[abcd]/\'/g'
>
>
>
> ^C
When I copy pasted echo 'abcd' | sed 's/[abcd]/\'/g' in .sh file and ran, it gave me following error:
anir#DESKTOP-4856511:~/Mahesha999/delete$ ./trysed.sh
./trysed.sh: line 1: unexpected EOF while looking for matching `''
./trysed.sh: line 2: syntax error: unexpected end of file
What the right way to do this? Is it impossible to escape single quote inside single quoted string (and I have to use double quotes only as explained here)?
As the post says:
With single quotes around the argument (sed 's/…/…/'), use '\'' to put a single quote in the replacement text.
So, using your example, you would do:
echo 'abcd' | sed 's/[abcd]/'\''/g'
If you want to replace with just one single quote:
echo 'abcd' | sed 's/[abcd][abcd]*/'\''/g'
The shell does not allow single quotes inside a single quoted string. What the code above does is create three strings (which are not separated by anything):
single-quoted string: 's/[abcd]/'
unquoted string containing just an escaped single-quote: \'
single-quoted string: '/g'
The shell then expands them and because they are not separated, they effectively become joined into a single string.
sed sees: s/[abcd]/'/g
Related
This question already has answers here:
Escape a string for a sed replace pattern
(17 answers)
Closed 1 year ago.
I have a script that passes a variable into a sed command like this:
sed "s-\t-&${SUBDIRECTORY}/-"
But if the variable contains - (dash), then the sed command throws an error.
So this script:
VARIABLE="test-variable"
sed "s-\t-&${VARIABLE}/-"
Results in this error:
sed: 1: "s-\t-&test-variable/-": bad flag in substitute command: 'v'
I have not been able to find any answers to this issue; it works fine without the -.
How can I fix this?
Use a shell parameter expansion that escapes each instance of -:
sed "s-\t-&${VARIABLE//-/\\-}/-"
In the Bash manual, under Shell Parameter Expansion:
${parameter/pattern/string}
The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string. [...] If pattern begins with /, all matches of pattern are replaced with string. Normally only the first match is replaced. [...]
Proper escaping is a fairly difficult problem in the shell, but you could do something like:
$ variable="test-variable"
$ printf '\t\n' | v="$variable" perl -pe 's-\t-$ENV{v}-'
test-variable
I have read here Difference between single and double quotes in Bash that single quotes will not allow any kind of interpolation. However I came across a situation where single quotes were actually the only way to interpolate my variable' value.
For the sake of an example I created a file named 'test_file' containing
1
2
3
4
5
6
7
8
9
10
Than in the same directory I created a script whose purpose was to extract parts of this file, I called it test.sh and this is its content:
#!/bin/bash
BEGIN=3
END=9
cat test_file | sed -n ''$BEGIN', '$END'p'
Now I tried different setups of the last line like:
cat test_file | sed -n '${BEGIN}, ${END}p'
cat test_file | sed -n '"${BEGIN}", "${END}"p'
cat test_file | sed -n '"$BEGIN", "$END"p'
But none of them worked.
My question is: why does it work that way?
Quotes of both flavors are a shortcut: they cause the enclosed characters to be treated as if they were each escaped by \. The difference between single and double quotes, then, is that double quotes do not escape $ (or following characters which form part of an expansion), allowing parameter and command substitutions to occur (with the results being treated as double-quoted strings as well).
Your first consists of an empty string '', followed by an unquoted expansion, then the literal string ,, the unquoted expansion, and a literal p.
Your second is a single literal sting.
Your third adds literal double quotes around the literal strings ${BEGIN} and ${END}.
The fourth is similar to the third, but drops the literal { and }.
The correct string is
sed -n "$BEGIN, ${END}p" # sed -n "3, 9p"
this is perhaps one of the most discussed topics here. I tried almost all the commands and other tweaks found here, but something doesn't seems to be doing well.
i would want to replace all the double quotes in my file with whitespace/blank
I'm seeing the below error when i tried to execute this command.
sed "s/"/ \''/g' x_orbit.txt > new.tx
sed: -e expression #1, char 3: unterminated `s' command
You're close. Just use single quotes, so the shell doesn't try to expand the metacharacters in your sed command:
sed 's/"/ /g' x_orbit.txt > new.txt
You could try tr for example:
tr '"' ' ' < x_orbit.txt > new.txt
The script you provided:
sed "s/"/ \''/g' x_orbit.txt > new.tx
means:
sed # invoke sed to execute the following script:
" # enclose the script in double quotes rather than single so the shell can
# interpret it (e.g. to expand variables like $HOME) before sed gets to
# interpret the result of that expansion
s/ # replace what follows until the next /
" # exit the double quotes so the shell can now not only expand variables
# but can now do globbing and file name expansion on wildcards like foo*
/ # end the definition of the regexp you want to replace so it is null since
# after the shell expansion there was no text for sed to read between
# this / and the previous one (the 2 regexp delimiters)
\' # provide a blank then an escaped single quote for the shell to interpret for some reason
'/g' # enclose the /g in single quotes as all scripts should be quoted by default.
That is so far off the correct syntax it's kinda shocking which is why I dissected it above to try to help you understand what you wrote so you'll see why it doesn't work. Where did you get the idea to write it that way (or to put it another way - what did you think each character in that script meant? I'm asking as it indicates a fundamental misunderstanding of how quoting and escaping works in shell so it'd be good if we could help correct that misunderstanding rather than just correct that script.
When you use any script or string in shell, simply always enclose it in single quotes:
sed 'script' file
var='string'
unless you NEED to use double quotes to let a variable expand and then use double quotes unless you NEED to use no quotes to let globbing and file name expansion happen.
An awk version:
awk '{gsub(/"/," ")}1' file
gsub is used for the replace
1 is always true, so line is printed
This question already has answers here:
Is it possible to escape regex metacharacters reliably with sed
(4 answers)
Closed 5 years ago.
I have replaced urls in the past using sed with no problem before. However, this url imparticular is giving me trouble. It has quite a few ampersands and I need to replace them. How would I go about doing that?
sed -i.bak "s#<string>https://www.url1toreplace.com?blah=1234585474738743874386328764287364238746283764287346872364fN&blah=Y&blah=%2Fwebapp%2Fwcs%2Fblahblah%2Fblah%2Fen%2Fblahahah%3Fblah%3e212e123152%26cm_mmc%3DBLAH-_-BLAH-_-Null-_-Null</string>#<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>#g" path/to/xml/file
My problem is that it's not fully replacing the url. How do I escape the ampersands so I can successfully replace www.url1toreplace.com with www.urltoreplace.com and everything that follows?
In the replacement text, you need to escape &.
For example, without the escape, the whole of the original match is substituted in for each &:
$ echo '&' | sed 's#&#a & b & c#'
a & b & c
With the escape, \&, & is treated as an ordinary character:
$ echo '&' | sed 's#&#a \& b \& c#'
a & b & c
Your example
Let's take this test file:
$ cat file
<string>https://www.url1toreplace.com?blah=1234585474738743874386328764287364238746283764287346872364fN&blah=Y&blah=%2Fwebapp%2Fwcs%2Fblahblah%2Fblah%2Fen%2Fblahahah%3Fblah%3e212e123152%26cm_mmc%3DBLAH-_-BLAH-_-Null-_-Null</string>
And run the original command:
$ sed "s#<string>https://www.url1toreplace.com?blah=1234585474738743874386328764287364238746283764287346872364fN&blah=Y&blah=%2Fwebapp%2Fwcs%2Fblahblah%2Fblah%2Fen%2Fblahahah%3Fblah%3e212e123152%26cm_mmc%3DBLAH-_-BLAH-_-Null-_-Null</string>#<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132\&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>#g" file
<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>
The above command fails. If we escape the &, however, we get:
$ sed 's#<string>https://www.url1toreplace.com?blah=1234585474738743874386328764287364238746283764287346872364fN&blah=Y&blah=%2Fwebapp%2Fwcs%2Fblahblah%2Fblah%2Fen%2Fblahahah%3Fblah%3e212e123152%26cm_mmc%3DBLAH-_-BLAH-_-Null-_-Null</string>#<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132\&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>#g' file
<string>https://www.urltoreplace.com/blah/blah/blah/blah/en/blah?blah=129i312093132&cm_mmc=BLAH-_-BLAH-_-Null-_-Null</string>
This succeeds: the & in the replacement string successfully appears in the output.
Sample data file:
$ cat xfile
<string>https://www.old.home.com?x=123&y=abc&z=ABC_mmc%3D</string>
Desired output:
<string>https://www.new.home.biz?A=XYZ&B=123&C=987_jjj%2XD</string>
As John1024's already pointed out, if a sed replacement string contains &'s, the &'s have to be escaped (\&) (because & has a special meaning to sed).
Hmmmm, but that could be a major pain in the keister if ya gotta go through and (manually?) change all sed replacement patterns from & to \&. But this replacement can be automated with a few minor assumptions ...
Assumptions:
search and replace patterns can be stored in variables before and after, respectively (actually, only the after variable is needed for this idea to work, but for this example I'll use before and after variables)
before and after contain normal strings w/out any special escapes
your version of bash supports character replacement via the ${var// /} construct
Apply escapes to the after variable on the fly:
$ before='old.home.com?x=123&y=abc&z=ABC_mmc%3D'
$ after='new.home.biz?A=XYZ&B=123&C=987_jjj%2XD'
$ sed "s#${before}#${after//\&/\\\&}#g" xfile
<string>https://www.new.home.biz?A=XYZ&B=123&C=987_jjj%2XD</string>
${after//\&/\\\&} : in the after variable, replace all occurrences of & with \&
This eliminates the need to go through and manually escape all occurrences of & in the replacement string.
File Content(file.txt):
table=$table_name
data=$data_name
Shell Script:
name=kush_123
cat file.txt | grep 'table' | sed "s\table_name\$name\g"
Expected output:
table=$kush_123
This gives error
unterminated s command
if the name variable has _ in it.
If you really want to use a backslash as a delimiter, it needs to be escaped itself so that the double-quoted string preserves it before passing to sed:
sed "s\\table_name\\$name\\g"
Otherwise, sed receives the string stable_name$nameg as its script. (\t, \$, and \g expand to t, $, and g, respectively). In this case, the letter t (as it immediately follows the s) is used as the delimiter, and the error results because there aren't enough ts in the result to provide a complete command.
Of course, if you try this, sed should complain that a backslash cannot be used as the delimiter for the s command. Use a different character:
sed "s/table_name/$name/g"
In general, building such scripts dynamically is fragile, because it assumes you know the value of $name doesn't contain your chosen delimiter.
On more investigation i found out the variable name has trailing spaces as the variable value was passed from python code. I trimmed extra spaces and it worked. Thanks :)