I want to escape dots from an IP address in Unix shell scripts (bash or ksh) so that I can match the exact address in a grep command.
echo $ip_addr | sed "s/\./\\\./g"
works (outputs 1\.2\.3\.4), but
ip_addr_escaped=`echo $ip_addr | sed "s/\./\\\./g"`
echo $ip_addr_escaped
Doesn't (outputs 1.2.3.4)
How can I correctly escape the address?
Edit:
It looks like
ip_addr_escaped=`echo $ip_addr | sed "s/\./\\\\\\\./g"`
works, but that's clearly awful!
bash parameter expansion supports pattern substitution, which will look (slightly) cleaner and doesn't require a call to sed:
echo ${ip_addr//./\\.}
Yeah, processing of backslashes is one of the strange quoting-related behaviors of backticks `...`. Rather than fighting with it, it's better to just use $(...), which the same except that its quoting rules are smarter and more intuitive. So:
ip_addr_escaped=$(echo $ip_addr | sed "s/\./\\\./g")
echo $ip_addr_escaped
But if the above is really your exact code — you have a parameter named ip_addr, and you want to replace . with \. — then you can use Bash's built-in ${parameter/pattern/string} notation:
ip_addr_escaped=${ip_addr//./\\.}
Or rather:
grep "${ip_addr//./\\.}" [FILE...]
Replace the double quotes with single quotes.
Related
I want to issue this command from the bash script
sed -e $beginning,$s/pattern/$variable/ file
but any possible combination of quotes gives me an error, only one that works:
sed -e "$beginning,$"'s/pattern/$variable/' file
also not good, because it do not dereferences the variable.
Does my approach can be implemented with sed?
Feel free to switch the quotes up. The shell can keep things straight.
sed -e "$beginning"',$s/pattern/'"$variable"'/' file
You can try this:
$ sed -e "$beginning,$ s/pattern/$variable/" file
Example
file.txt:
one
two
three
Try:
$ beginning=1
$ variable=ONE
$ sed -e "$beginning,$ s/one/$variable/" file.txt
Output:
ONE
two
three
There are two types of quotes:
Single quotes preserve their contents (> is the prompt):
> var=blah
> echo '$var'
$var
Double quotes allow for parameter expansion:
> var=blah
> echo "$var"
blah
And two types of $ sign:
One to tell the shell that what follows is the name of a parameter to be expanded
One that stands for "last line" in sed.
You have to combine these so
The shell doesn't think sed's $ has anything to do with a parameter
The shell parameters still get expanded (can't be within single quotes)
The whole sed command is quoted.
One possibility would be
sed "$beginning,\$s/pattern/$variable/" file
The whole command is in double quotes, i.e., parameters get expanded ($beginning and $variable). To make sure the shell doesn't try to expand $s, which doesn't exist, the "end of line" $ is escaped so the shell doesn't try anything funny.
Other options are
Double quoting everything but adding a space between $ and s (see Ren's answer)
Mixing quoting types as needed (see Ignacio's answer)
Methods that don't work
sed '$beginning,$s/pattern/$variable/' file
Everything in single quotes: the shell parameters are not expanded (doesn't follow rule 2 above). $beginning is not a valid address, and pattern would be literally replaced by $variable.
sed "$beginning,$s/pattern/$variable/" file
Everything in double qoutes: the parameters are expanded, including $s, which isn't supposed to (doesn't follow rule 1 above).
the following form worked for me from within script
sed $beg,$ -e s/pattern/$variable/ file
the same form will also work if executed from the shell
EDIT: the command substitution is not necessary for the surprising behavior, although it is the most common use case. The same question applies to just echo "'!b'"
b=a
# Enable history substitution.
# This option is on by default on interactive shells.
set -H
echo '!b'
# Output: '!b'
# OK. Escaped by single quotes.
echo $(echo '!b')
# Output: '!b'
# OK. Escaped by single quotes.
echo "$(echo '$b')"
# Output: '$b'
# OK. Escaped by single quotes.
echo "$(echo '!b')"
# Output: history expands
# BAD!! WHY??
In the last example, what is the best way to escape the !?
Why was it not escaped even if I used single quotes, while echo "$(echo '$b')" was? What is the difference between ! and $?
Why was does echo $(echo '!b') (no quotes) work? (pointed by #MBlanc).
I would prefer to do this without:
set +H as I would need set -H afterwards to maintain shell state
backslash escapes, because I need one for every ! and it has to be outside the quotes:
echo "$(echo '\!a')"
# '\!a'.
# No good.
echo "$(echo 'a '\!' b '\!' c')"
# a ! b ! c
# Good, but verbose.
echo $(echo '!b') (no quotes), because the command could return spaces.
Version:
bash --version | head -n1
# GNU bash, version 4.2.25(1)-release (i686-pc-linux-gnu)
In your last example,
echo "$(echo '!b')"
the exclamation point is not single-quoted. Because history expansion occurs so early in the parsing process, the single quotes are just part of the double-quoted string; the parser hasn't recognized the command substitution yet to establish a new context where the single quotes would be quoting operators.
To fix, you'll have to temporarily turn off history expansion:
set +H
echo "$(echo '!b')"
set -H
This was repeatedly reported as a bug, most recently against bash 4.3 in 2014, for behavior going back to bash 3.
There was some discussion whether this constituted a bug or expected but perhaps undesirable behavior; it seems the consensus has been that, however you want to characterize the behavior, it shouldn't be allowed to continue.
It's fixed in bash 4.4, echo "$(echo '!b')" doesn't expand, echo "'!b'" does, which I regard as proper behavior because the single quotes are shell syntax markers in the first example and not in the second.
If History Expansion is enabled, you can only echo the ! character if it is put in single quotes, escaped or if followed by a whitespace character, carriage return, or =.
From man bash:
Only backslash (\) and single quotes can quote the history
expansion character.
Several characters inhibit history expansion if found immediately fol-
lowing the history expansion character, even if it is unquoted: space,
tab, newline, carriage return, and =.
I believe the key word here is “Only”. The examples provided in the question only consider the outer most quotes being double quotes.
Sometimes you need to make a small addition to a big command pipe
The OP's "Good, but verbose" example is actually pretty awesome for many cases.
Please forgive the contrived example. The whole reason I need such a solution is that I have a lot of distracting, nested code. But, it boils down to: I must do a !d in sed within a double quoted bash command expansion.
This works
$ ifconfig | sed '/inet/!d'
inet 127.0.0.1 netmask 0xff000000
…
This does not
$ echo "$(ifconfig | sed '/inet/!d')"
-bash: !d': event not found
This is a simplest compromise
$ echo "$(ifconfig | sed '/inet/'\!'d')"
inet 127.0.0.1 netmask 0xff000000
…
Using the compromise allows me to insert a few characters into the existing code and produce a Pull Request that anyone can understand… even though resulting code is more difficult to understand. If I did a complete refactor, the code reviewers would have a much more challenging time verifying it. And of course this bash has no unit tests.
In the below shell script I try to print A2D(Vlog-Ams-#Cross) with special characters escaped. For example replace ( with \( but sed won't have any effect.
#! /bin/sh
parameter="A2D(Vlog-Ams-#Cross)"
echo $parameter
parameterz=`echo "$parameter" | sed 's/(/\\(/g'`
echo $parameterz
The output is
A2D(Vlog-Ams-#Cross)
A2D(Vlog-Ams-#Cross)
If I do the same on my c-shell terminal, it works fine.
Any ideas?
You use backslashs within a backtick command and that's tricky. If the sed command didn't occur within backticks, it would work correctly. When the shell looks for the closing backtick, however, it removes one level of backslash quoting, so you get
sed 's/(/\(/g'
and that's a no-op. If your shell permits it, use $(...) instead of backticks; in this way you avoid these quoting problems.
In your replacement \\( the first \ escapes the second \. But you must escape the (, too:
$ echo 'A2D(Vlog-Ams-#Cross)' | sed -e 's/(/\\\(/g' -e 's/)/\\\)/g'
A2D\(Vlog-Ams-#Cross\)
I have a file that is taking in a path as an argument:
./<filename> /path/to/file...
What I want to do is replace the /path/to/... part with /another/file/...
I was trying to sed the argument in the following manner:
CUR_PATH=$1
OLD_PATH="\/path\/to\/"
NEW_PATH="\/another\/file\/"
sed "s/$OLD_PATH/$NEW_PATH/" $CUR_PATH
But this isn't working because of the fact that sed is trying to actually modify the file at CUR_PATH and not the actual statement of CUR_PATH. How do I fix this? Thanks.
Another possibility is to use a here string:
CUR_PATH=$1
OLD_PATH="/path/to/"
NEW_PATH="/another/file/"
sed "s|$OLD_PATH|$NEW_PATH|" <<< $CUR_PATH
Also note that you can vary the delimiters for the substitution in sed, so that you don't have to escape the slashes in your path variables.
You don't need sed. bash a built-in substitution for variables. You can use:
NEW_PATH=${OLD_PATH/\/path\/to\//\/another\/file\/}
Note the backslashing of the /, because the expression is ${variable/old/new}.
You can use bash's substitution as Diego suggests, but for this particular case it is probably cleaner to do:
NEW_PATH="/another/file/${OLD_PATH##*/}"
which will replace the entire leading path of OLD_PATH with the string "/another/file/". Note that the double quotes are only necessary if OLD_PATH may contain whitespace.
If you do want to use sed, you can simply echo OLD_PATH into a pipe. And, when using sed for manipulating filenames, it is convenient to use a different separator. For example:
NEW_PATH=$( echo $OLD_PATH | sed s#/path/to/my#/another/file# )
I'm having difficulty getting this bash script to perform the formatting of an input.
It's pretty straight-forward, but when it executes the line that starts with 'newstring=', it doesn't perform the sed operation, it only prints my input (up until the first white-space) then prints my sed command directly after. What am I doing wrong?
#! /bin/bash
##format paths/strings with spaces to escape the spaces with a forward-slash'\'
##then use 'open' to open finder at current-set directory (based on path)
oldstring="$1"
newstring="$oldstring | sed 's/ /\\ /g')"
cd $newstring
open .
You should simply do:
cd "$1"
open .
This avoids running sub-processes and deals with various problems that the sed script doesn't (such as names containing $ symbols, or other shell metacharacters). Generally, if a variable (or positional parameter such as $1) is a file name that could contain spaces, use it surrounded by double quotes every time.
Try putting the command in backquotes like
newstring=`echo "$oldstring" | sed 's/ /\\ /g')`
#Jonathan Leffler's is the correct solution, since adding escapes doesn't actually do what you want but double-quoting does. However, I'll take this opportunity to point out that there's a better way to add escapes using bash's built-in substitution capability instead of sed:
newstring="${oldstring/ /\\ }"
So there you have it, a better way to implement the wrong solution. Personally, I voted for Jonathan's.