In the below shell script I try to print A2D(Vlog-Ams-#Cross) with special characters escaped. For example replace ( with \( but sed won't have any effect.
#! /bin/sh
parameter="A2D(Vlog-Ams-#Cross)"
echo $parameter
parameterz=`echo "$parameter" | sed 's/(/\\(/g'`
echo $parameterz
The output is
A2D(Vlog-Ams-#Cross)
A2D(Vlog-Ams-#Cross)
If I do the same on my c-shell terminal, it works fine.
Any ideas?
You use backslashs within a backtick command and that's tricky. If the sed command didn't occur within backticks, it would work correctly. When the shell looks for the closing backtick, however, it removes one level of backslash quoting, so you get
sed 's/(/\(/g'
and that's a no-op. If your shell permits it, use $(...) instead of backticks; in this way you avoid these quoting problems.
In your replacement \\( the first \ escapes the second \. But you must escape the (, too:
$ echo 'A2D(Vlog-Ams-#Cross)' | sed -e 's/(/\\\(/g' -e 's/)/\\\)/g'
A2D\(Vlog-Ams-#Cross\)
Related
I am trying to find and replace a specific text content using the sed command and to run it via a shell script.
Below is the sample script that I am using:
fp=/asd/filename.txt
fd="sed -i -E 's ($2).* $2:$3 g' ${fp}"
eval $fd
and executing the same by passing the arguments:
./test.sh update asd asdfgh
But if the argument string contains $ , it breaks the commands and it is replacing with wrong values, like
./test.sh update asd $apr1$HnIF6bOt$9m3NzAwr.aG1Yp.t.bpIS1.
How can I make sure that the values inside the variables are not expanded because of the $?
Updated
sh file test.sh
set -xv
fp="/asd/filename.txt"
sed -iE "s/(${2//'$'/'\$'}).*/${2//'$'/'\$'}:${3//'$'/'\$'}/g" "$fp"
text file filename.txt
hello:world
Outputs
1)
./test.sh update hello WORLD
sed -iE "s/(${2//'$'/'\$'}).*/${2//'$'/'\$'}:${3//'$'/'\$'}/g" "$fp"
++ sed -iE 's/(hello).*/hello:WORLD/g' /asd/filename.txt
2)
./test.sh update hello '$apr1$hosgaxyv$D0KXp5dCyZ2BUYCS9BmHu1'
sed -iE "s/(${2//'$'/'\$'}).*/${2//'$'/'\$'}:${3//'$'/'\$'}/g" "$fp"
++ sed -iE 's/(hello).*/hello:'\''$'\''apr1'\''$'\''hosgaxyv'\''$'\''D0KXp5dCyZ2BUYCS9BmHu1/g' /asd/filename.txt
In both the case , its not replacing the content
You don't need eval here at all:
fp=/asd/filename.txt
sed -i -E "s/(${2//'$'/'\$'}).*/\1:${3//'$'/'\$'}/g" "$fp"
The whole sed command is in double quotes so variables can expand.
I've replaced the blank as the s separator with / (doesn't really matter in the example).
I've used \1 to reference the first capture group instead of repeating the variable in the substitution.
Most importantly, I've used ${2//'$'/'\$'} instead of $2 (and similar for $3). This escapes every $ sign as \$; this is required because of the double quoting, or the $ get eaten by the shell before sed gets to see them.
When you call your script, you must escape any $ in the input, or the shell tries to expand them as variable names:
./test.sh update asd '$apr1$HnIF6bOt$9m3NzAwr.aG1Yp.t.bpIS1.'
Put the command-line arguments that are filenames in single quotes:
./test.sh update 'asd' '$apr1$HnIF6bOt$9m3NzAwr.aG1Yp.t.bpIS1'
must protect all the script arguments with quotes if having space and special shell char, and escape it if it's a dollar $, and -Ei instead of -iE even better drop it first for test, may add it later if being really sure
I admit i won't understant your regex so let's just get in the gist of solution, no need eval;
fp=/asd/filename.txt
sed -Ei "s/($2).*/$2:$3/g" $fp
./test.sh update asd '\$apr1\$HnIF6bOt\$9m3NzAwr.aG1Yp.t.bpIS1.'
I'm trying to use sed as root to alter the default PS1.
The first 2 lines were just to make sure I had the syntax structured right and to see if the quotation marks made a difference,
running them consecutively allows me to change a comment near the top of the file and then change it back.
Opening the file in nano confirms the changes are effective, which should rule out 'write permissions'.
sed -i 's/If not running interactively,/stringtoreplaceitwith/' /etc/skel/.bashrc
sed -i "s/stringtoreplaceitwith/If not running interactively,/" /etc/skel/.bashrc
sed -i "s/\[\033[01;32m\]\u#\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ /Replace PS1/" /etc/skel/.bashrc
I'm not sure if it's something else about the string's structure,
but for some reason, it's not finding what I'd like to substitute.
\[\033[01;32m\]\u#\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$
(with a trailing blank).
Your problem is due to quoting and escaping. Let's reduce the string you're trying to substitute to just this:
\w\[\033[00m\]\$
(with a trailing blank) – this contains all the difficulties, but is less verbose.
To match this, we have to
escape all the backslashes: \w becomes \\w etc.
escape [ and ], because they're special to sed: [ becomes \[, and \[ becomes \\\[ (escaped backslash, escaped [)
escape $: \$ becomes \\\$
Then, we have to use single quotes around our sed command so the shell doesn't modify the string:
$ sed 's/\\w\\\[\\033\[00m\\\]\\\$ /Replace PS1/' <<< '\w\[\033[00m\]\$ '
Replace PS1
It can also work with double quotes, but then we have to add another round of escaping for the shell: \[ becomes \\\[ (escaped for sed) becomes \\\\\\[ (escaped for the shell), and so on.
$ sed "s/\\\\w\\\\\\[\\\\033\\[00m\\\\\\]\\\\\\$ /Replace PS1/" <<< '\w\[\033[00m\]\$ '
Replace PS1
Double quoting only makes sense if you're using variables, though, for obvious reasons.
Finally, for your actual string:
$ echo '\[\033[01;32m\]\u#\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ ' \
| sed 's/\\\[\\033\[01;32m\\\]\\u#\\h\\\[\\033\[00m\\\]:\\\[\\033\[01;34m\\\]\\w\\\[\\033\[00m\\\]\\\$ /Replace PS1/'
Replace PS1
I have just recently got back into learning bash. Currently working on a project of mine and when using sed I've run into an issue, I've tried looking around the web for help but haven't had any joy. I suspect as I may not be using the correct terminology so I can't find what I'm looking for. ANYHOW.
So in my script I'm trying to assign the output of date to a variable. Here's the line from my script.
origdate=$(date)
When I call it the output looks like this:
Wed Oct 5 19:40:45 BST 2016
Part of my script then generates a file and writes information to it, part of which I am trying to use sed to find lines and replace parts of it. This is the first I've been playing around with sed, I've used it successfully so far for my needs. However I'm getting stuck when I try this:
sed -i '/origdate=empty/c\'$origdate'' $sd/pingcheck-email-$job.txt
When I run the script and it gets to this line, this is the error I'm getting:
sed: can't read Oct: No such file or directory
sed: can't read 5: No such file or directory
sed: can't read 19:52:56: No such file or directory
sed: can't read BST: No such file or directory
sed: can't read 2016: No such file or directory
I suspect it's something to do with the spaces in the date (variable), my question is: how can I work around this? Can I get sed to 'ignore' the spaces? or should I just use cut to cut the field for the date, and set that to a variable and the same thing again to set the time to another variable?
Even if someone could kindly point me in the right direction that'd be great!
Thanks in advance!
double quote the variable
sed -i '/origdate=empty/c\'"$origdate"'' $sd/pingcheck-email-$job.txt
or alternatively, the whole script
sed -i "/origdate=empty/c\$origdate" $sd/pingcheck-email-$job.txt
The problem is not with sed but rather with how bash word splits on your date given your command.
Bash
In bash, word splitting is performed on the command line so that text is broken up into a list of arguments. To illustrate, I'm going to run a simple script that outputs the first argument only.
bash -c 'echo $1' ignored_0 foo bar
Think of bash -c 'echo $1' ignored_0 as the command (sed in your case) and foo bar as the arguments. In this case, foo bar is split into two arguments, foo and bar.
To pass foo bar in as the first parameter, you need to have the text in either single or double quotes. See the GNU manual on quoting.
bash -c 'echo $1' ignored_0 'foo bar'
bash -c 'echo $1' ignored_0 "foo bar"
Parameter expansion does not occur when the variable is inside a single quote.
var="foo bar"
bash -c 'echo $1' ignored_0 '$var'
bash -c 'echo $1' ignored_0 "$var"
NOTE: In the command `bash -c 'echo $1', I do not want $1 to expand before being passed as an argument to bash because that's part of the code I want to execute.
Parameter expansion occurs when variables are outside of quotes, but word splitting will apply after the parameter is expanded. From the bash man page in the Word Splitting section:
The shell scans the results of parameter expansion, command
substitution, and arithmetic expansion that did not occur within
double quotes for word splitting.
From the GNU bash manual on Word Splitting:
The shell scans the results of parameter expansion, command
substitution, and arithmetic expansion that did not occur within
double quotes for word splitting.
var="foo bar"
bash -c 'echo $1' ignored_0 $var
The last step in Shell Expansions in Quote Removal where unquoted quote characters are removed before being passed to commands. The following command shows that ''"" has no effect on the arguments passed.
bash -c 'echo $1' ignored_0 foo''""
Application
In your example, the trailing '' after $origdate is extraneous. The important part is that $origdate is not quoted so word splitting applies to the expanded variable.
When -e is not passed to the sed command, sed expects the expression to be in one argument, or word from bash. When you run your command, your expression is /origdate=empty/c\Wed and the rest of the date is considered to be files for the expression to be applied to.
The simple fix is to put double quotes around the string for which you want to prevent word splitting. I've modified the command so that anyone can run this example without having the files on their system.
In this example, the \ must be escaped so that it is not considered an escape character for $.
echo "origdate=empty" | sed "/origdate=empty/c\\$origdate"
You can also change the type of quotes you are using without affecting word splitting like so.
echo "origdate=empty" | sed '/origdate=empty/c\'"$origdate"
You need escape by double slash
\ / \%
I would like to do that:
i="1"; echo -e '#!/usr/bin/env bash\nmyprogram -i "input_${i}.txt"'
and pipe it to a job scheduler.
However, this doesn't replace the variable i by its value. Instead, I obtain this:
#!/usr/bin/env bash
myprogram -i "input_${i}.txt"
I played a bit with option -e of echo and with single-/double-quote but could not make it work. For instance, I get this:
i="1"; echo -e "#!/usr/bin/env bash\nmyprogram -i \"input_${i}.txt\""
-bash: !/usr/bin/env: event not found
My bash version is 4.1.2.
Try this:
i="1"; echo -e '#!/usr/bin/env bash\nmyprogram -i '"\"input_${i}.txt\""
You can echo single- and double-quoted strings at the same time.
Try also escaping the exclamation mark:
\! should be okay, and will not be read as an "event" by bash.
i="1"; echo -e '#!/usr/bin/env bash\nmyprogram -i "input_'"${i}"'.txt"'
Basically, use single quotes until you need to interpolate, then close the single quotes, open the double quotes, add the interpolation, close the double quotes, reopen single quotes, and finish the string. In the shell, quotation marks don't delimit a word; they just change the interpretation of the part of a word falling between them.
I want to escape dots from an IP address in Unix shell scripts (bash or ksh) so that I can match the exact address in a grep command.
echo $ip_addr | sed "s/\./\\\./g"
works (outputs 1\.2\.3\.4), but
ip_addr_escaped=`echo $ip_addr | sed "s/\./\\\./g"`
echo $ip_addr_escaped
Doesn't (outputs 1.2.3.4)
How can I correctly escape the address?
Edit:
It looks like
ip_addr_escaped=`echo $ip_addr | sed "s/\./\\\\\\\./g"`
works, but that's clearly awful!
bash parameter expansion supports pattern substitution, which will look (slightly) cleaner and doesn't require a call to sed:
echo ${ip_addr//./\\.}
Yeah, processing of backslashes is one of the strange quoting-related behaviors of backticks `...`. Rather than fighting with it, it's better to just use $(...), which the same except that its quoting rules are smarter and more intuitive. So:
ip_addr_escaped=$(echo $ip_addr | sed "s/\./\\\./g")
echo $ip_addr_escaped
But if the above is really your exact code — you have a parameter named ip_addr, and you want to replace . with \. — then you can use Bash's built-in ${parameter/pattern/string} notation:
ip_addr_escaped=${ip_addr//./\\.}
Or rather:
grep "${ip_addr//./\\.}" [FILE...]
Replace the double quotes with single quotes.