Let's say, you have a Bash alias like:
alias rxvt='urxvt'
which works fine.
However:
alias rxvt='urxvt -fg '#111111' -bg '#111111''
won't work, and neither will:
alias rxvt='urxvt -fg \'#111111\' -bg \'#111111\''
So how do you end up matching up opening and closing quotes inside a string once you have escaped quotes?
alias rxvt='urxvt -fg'\''#111111'\'' -bg '\''#111111'\''
seems ungainly although it would represent the same string if you're allowed to concatenate them like that.
If you really want to use single quotes in the outermost layer, remember that you can glue both kinds of quotation. Example:
alias rxvt='urxvt -fg '"'"'#111111'"'"' -bg '"'"'#111111'"'"
# ^^^^^ ^^^^^ ^^^^^ ^^^^
# 12345 12345 12345 1234
Explanation of how '"'"' is interpreted as just ':
' End first quotation which uses single quotes.
" Start second quotation, using double-quotes.
' Quoted character.
" End second quotation, using double-quotes.
' Start third quotation, using single quotes.
If you do not place any whitespaces between (1) and (2), or between (4) and (5), the shell will interpret that string as a one long word.
I always just replace each embedded single quote with the sequence: '\'' (that is: quote backslash quote quote) which closes the string, appends an escaped single quote and reopens the string.
I often whip up a "quotify" function in my Perl scripts to do this for me. The steps would be:
s/'/'\\''/g # Handle each embedded quote
$_ = qq['$_']; # Surround result with single quotes.
This pretty much takes care of all cases.
Life gets more fun when you introduce eval into your shell-scripts. You essentially have to re-quotify everything again!
For example, create a Perl script called quotify containing the above statements:
#!/usr/bin/perl -pl
s/'/'\\''/g;
$_ = qq['$_'];
then use it to generate a correctly-quoted string:
$ quotify
urxvt -fg '#111111' -bg '#111111'
result:
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
which can then be copy/pasted into the alias command:
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
(If you need to insert the command into an eval, run the quotify again:
$ quotify
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
result:
'alias rxvt='\''urxvt -fg '\''\'\'''\''#111111'\''\'\'''\'' -bg '\''\'\'''\''#111111'\''\'\'''\'''\'''
which can be copy/pasted into an eval:
eval 'alias rxvt='\''urxvt -fg '\''\'\'''\''#111111'\''\'\'''\'' -bg '\''\'\'''\''#111111'\''\'\'''\'''\'''
Since Bash 2.04 syntax $'string' allows a limit set of escapes.
Since Bash 4.4, $'string' also allows the full set of C-style escapes, making the behavior differ slightly in $'string' in previous versions. (Previously the $('string') form could be used.)
Simple example in Bash 2.04 and newer:
$> echo $'aa\'bb'
aa'bb
$> alias myvar=$'aa\'bb'
$> alias myvar
alias myvar='aa'\''bb'
In your case:
$> alias rxvt=$'urxvt -fg \'#111111\' -bg \'#111111\''
$> alias rxvt
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
Common escaping sequences works as expected:
\' single quote
\" double quote
\\ backslash
\n new line
\t horizontal tab
\r carriage return
Below is copy+pasted related documentation from man bash (version 4.4):
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard. Backslash escape sequences, if present, are decoded as follows:
\a alert (bell)
\b backspace
\e
\E an escape character
\f form feed
\n new line
\r carriage return
\t horizontal tab
\v vertical tab
\\ backslash
\' single quote
\" double quote
\? question mark
\nnn the eight-bit character whose value is the octal
value nnn (one to three digits)
\xHH the eight-bit character whose value is the hexadecimal
value HH (one or two hex digits)
\uHHHH the Unicode (ISO/IEC 10646) character whose value is
the hexadecimal value HHHH (one to four hex digits)
\UHHHHHHHH the Unicode (ISO/IEC 10646) character whose value
is the hexadecimal value HHHHHHHH (one to eight
hex digits)
\cx a control-x character
The expanded result is single-quoted, as if the dollar sign had not been present.
See Quotes and escaping: ANSI C like strings on bash-hackers.org wiki for more details. Also note that "Bash Changes" file (overview here) mentions a lot for changes and bug fixes related to the $'string' quoting mechanism.
According to unix.stackexchange.com How to use a special character as a normal one? it should work (with some variations) in bash, zsh, mksh, ksh93 and FreeBSD and busybox sh.
I don't see the entry on his blog (link pls?) but according to the gnu reference manual:
Enclosing characters in single quotes
(‘'’) preserves the literal value of
each character within the quotes. A
single quote may not occur between
single quotes, even when preceded by a
backslash.
so bash won't understand:
alias x='y \'z '
however, you can do this if you surround with double quotes:
alias x="echo \'y "
> x
> 'y
I can confirm that using '\'' for a single quote inside a single-quoted string does work in Bash, and it can be explained in the same way as the "gluing" argument from earlier in the thread. Suppose we have a quoted string: 'A '\''B'\'' C' (all quotes here are single quotes). If it is passed to echo, it prints the following: A 'B' C.
In each '\'' the first quote closes the current single-quoted string, the following \' glues a single quote to the previous string (\' is a way to specify a single quote without starting a quoted string), and the last quote opens another single-quoted string.
Both versions are working, either with concatenation by using the escaped single quote character (\'), or with concatenation by enclosing the single quote character within double quotes ("'").
The author of the question did not notice that there was an extra single quote (') at the end of his last escaping attempt:
alias rxvt='urxvt -fg'\''#111111'\'' -bg '\''#111111'\''
│ │┊┊| │┊┊│ │┊┊│ │┊┊│
└─STRING──┘┊┊└─STRIN─┘┊┊└─STR─┘┊┊└─STRIN─┘┊┊│
┊┊ ┊┊ ┊┊ ┊┊│
┊┊ ┊┊ ┊┊ ┊┊│
└┴─────────┴┴───┰───┴┴─────────┴┘│
All escaped single quotes │
│
?
As you can see in the previous nice piece of ASCII/Unicode art, the last escaped single quote (\') is followed by an unnecessary single quote ('). Using a syntax-highlighter like the one present in Notepad++ can prove very helpful.
The same is true for another example like the following one:
alias rc='sed '"'"':a;N;$!ba;s/\n/, /g'"'"
alias rc='sed '\'':a;N;$!ba;s/\n/, /g'\'
These two beautiful instances of aliases show in a very intricate and obfuscated way how a file can be lined down. That is, from a file with a lot of lines you get only one line with commas and spaces between the contents of the previous lines. In order to make sense of the previous comment, the following is an example:
$ cat Little_Commas.TXT
201737194
201802699
201835214
$ rc Little_Commas.TXT
201737194, 201802699, 201835214
Simple example of escaping quotes in shell:
$ echo 'abc'\''abc'
abc'abc
$ echo "abc"\""abc"
abc"abc
It's done by finishing already opened one ('), placing escaped one (\'), then opening another one ('). This syntax works for all commands. It's very similar approach to the 1st answer.
I'm not specifically addressing the quoting issue because, well, sometimes, it's just reasonable to consider an alternative approach.
rxvt() { urxvt -fg "#${1:-000000}" -bg "#${2:-FFFFFF}"; }
which you can then call as:
rxvt 123456 654321
the idea being that you can now alias this without concern for quotes:
alias rxvt='rxvt 123456 654321'
or, if you need to include the # in all calls for some reason:
rxvt() { urxvt -fg "${1:-#000000}" -bg "${2:-#FFFFFF}"; }
which you can then call as:
rxvt '#123456' '#654321'
then, of course, an alias is:
alias rxvt="rxvt '#123456' '#654321'"
(oops, i guess i kind of did address the quoting :)
How to escape single quotes (') and double quotes (") with hex and octal chars
If using something like echo, I've had some really complicated and really weird and hard-to-escape (think: very nested) cases where the only thing I could get to work was using octal or hex codes!
Here are some basic examples just to demonstrate how it works:
1. Single quote example, where ' is escaped with hex \x27 or octal \047 (its corresponding ASCII code):
hex \x27
echo -e "Let\x27s get coding!"
# OR
echo -e 'Let\x27s get coding!'
Result:
Let's get coding!
octal \047
echo -e "Let\047s get coding!"
# OR
echo -e 'Let\047s get coding!'
Result:
Let's get coding!
2. Double quote example, where " is escaped with hex \x22 or octal \042 (its corresponding ASCII code).
Note: bash is nuts! Sometimes even the ! char has special meaning, and must either be removed from within the double quotes and then escaped "like this"\! or put entirely within single quotes 'like this!', rather than within double quotes.
# 1. hex; also escape `!` by removing it from within the double quotes
# and escaping it with `\!`
$ echo -e "She said, \x22Let\x27s get coding"\!"\x22"
She said, "Let's get coding!"
# OR put it all within single quotes:
$ echo -e 'She said, \x22Let\x27s get coding!\x22'
She said, "Let's get coding!"
# 2. octal; also escape `!` by removing it from within the double quotes
$ echo -e "She said, \042Let\047s get coding"\!"\042"
She said, "Let's get coding!"
# OR put it all within single quotes:
$ echo -e 'She said, \042Let\047s get coding!\042'
She said, "Let's get coding!"
# 3. mixed hex and octal, just for fun
# also escape `!` by removing it from within the double quotes when it is followed by
# another escape sequence
$ echo -e "She said, \x22Let\047s get coding! It\x27s waaay past time to begin"\!"\042"
She said, "Let's get coding! It's waaay past time to begin!"
# OR put it all within single quotes:
$ echo -e 'She said, \x22Let\047s get coding! It\x27s waaay past time to begin!\042'
She said, "Let's get coding! It's waaay past time to begin!"
Note that if you don't properly escape !, when needed, as I've shown two ways to do above, you'll get some weird errors, like this:
$ echo -e "She said, \x22Let\047s get coding! It\x27s waaay past time to begin!\042"
bash: !\042: event not found
OR:
$ echo -e "She said, \x22Let\x27s get coding!\x22"
bash: !\x22: event not found
One more alternative: this allows mixed expansion and non-expansion all within the same bash string
Here is another demo of an alternative escaping technique.
First, read the main answer by #liori to see how the 2nd form below works. Now, read these two alternative ways of escaping characters. Both examples below are identical in their output:
CMD="gs_set_title"
# 1. 1st technique: escape the $ symbol with a backslash (\) so it doesn't
# run and expand the command following it
echo "$CMD '\$(basename \"\$(pwd)\")'"
# 2. 2nd technique (does the same thing in a different way): escape the
# $ symbol using single quotes around it, and the single quote (') symbol
# using double quotes around it
echo "$CMD ""'"'$(basename "$(pwd)")'"'"
Sample output:
gs_set_title '$(basename "$(pwd)")'
gs_set_title '$(basename "$(pwd)")'
Note: for my gs_set_title bash function, which I have in my ~/.bash_aliases file somewhere around here, see my other answer here.
References:
https://en.wikipedia.org/wiki/ASCII#Printable_characters
https://serverfault.com/questions/208265/what-is-bash-event-not-found/208266#208266
See also my other answer here: How do I write non-ASCII characters using echo?.
I just use shell codes.. e.g. \x27 or \\x22 as applicable. No hassle, ever really.
Since one cannot put single quotes within single quoted strings, the simplest and most readable option is to use a HEREDOC string
command=$(cat <<'COMMAND'
urxvt -fg '#111111' -bg '#111111'
COMMAND
)
alias rxvt=$command
In the code above, the HEREDOC is sent to the cat command and the output of that is assigned to a variable via the command substitution notation $(..)
Putting a single quote around the HEREDOC is needed since it is within a $()
IMHO the real answer is that you can't escape single-quotes within single-quoted strings.
Its impossible.
If we presume we are using bash.
From bash manual...
Enclosing characters in single quotes preserves the literal value of each
character within the quotes. A single quote may not occur
between single quotes, even when preceded by a backslash.
You need to use one of the other string escape mechanisms " or \
There is nothing magic about alias that demands it use single quotes.
Both the following work in bash.
alias rxvt="urxvt -fg '#111111' -bg '#111111'"
alias rxvt=urxvt\ -fg\ \'#111111\'\ -bg\ \'#111111\'
The latter is using \ to escape the space character.
There is also nothing magic about #111111 that requires single quotes.
The following options achieves the same result the other two options, in that the rxvt alias works as expected.
alias rxvt='urxvt -fg "#111111" -bg "#111111"'
alias rxvt="urxvt -fg \"#111111\" -bg \"#111111\""
You can also escape the troublesome # directly
alias rxvt="urxvt -fg \#111111 -bg \#111111"
A minimal answer is needed so that people can get going without spending a lot of time as I had to sifting through people waxing eloquent.
There is no way to escape single quotes or anything else within single quotes.
The following is, perhaps surprisingly, a complete command:
$ echo '\'
whose output is:
\
Backslashes, surprisingly to even long-time users of bash, have no meaning inside single quotes. Nor does anything else.
Most of these answers hit on the specific case you're asking about. There is a general approach that a friend and I have developed that allows for arbitrary quoting in case you need to quote bash commands through multiple layers of shell expansion, e.g., through ssh, su -c, bash -c, etc. There is one core primitive you need, here in native bash:
quote_args() {
local sq="'"
local dq='"'
local space=""
local arg
for arg; do
echo -n "$space'${arg//$sq/$sq$dq$sq$dq$sq}'"
space=" "
done
}
This does exactly what it says: it shell-quotes each argument individually (after bash expansion, of course):
$ quote_args foo bar
'foo' 'bar'
$ quote_args arg1 'arg2 arg2a' arg3
'arg1' 'arg2 arg2a' 'arg3'
$ quote_args dq'"'
'dq"'
$ quote_args dq'"' sq"'"
'dq"' 'sq'"'"''
$ quote_args "*"
'*'
$ quote_args /b*
'/bin' '/boot'
It does the obvious thing for one layer of expansion:
$ bash -c "$(quote_args echo a'"'b"'"c arg2)"
a"b'c arg2
(Note that the double quotes around $(quote_args ...) are necessary to make the result into a single argument to bash -c.) And it can be used more generally to quote properly through multiple layers of expansion:
$ bash -c "$(quote_args bash -c "$(quote_args echo a'"'b"'"c arg2)")"
a"b'c arg2
The above example:
shell-quotes each argument to the inner quote_args individually and then combines the resulting output into a single argument with the inner double quotes.
shell-quotes bash, -c, and the already once-quoted result from step 1, and then combines the result into a single argument with the outer double quotes.
sends that mess as the argument to the outer bash -c.
That's the idea in a nutshell. You can do some pretty complicated stuff with this, but you have to be careful about order of evaluation and about which substrings are quoted. For instance, the following do the wrong things (for some definition of "wrong"):
$ (cd /tmp; bash -c "$(quote_args cd /; pwd 1>&2)")
/tmp
$ (cd /tmp; bash -c "$(quote_args cd /; [ -e *sbin ] && echo success 1>&2 || echo failure 1>&2)")
failure
In the first example, bash immediately expands quote_args cd /; pwd 1>&2 into two separate commands, quote_args cd / and pwd 1>&2, so the CWD is still /tmp when the pwd command is executed. The second example illustrates a similar problem for globbing. Indeed, the same basic problem occurs with all bash expansions. The problem here is that a command substitution isn't a function call: it's literally evaluating one bash script and using its output as part of another bash script.
If you try to simply escape the shell operators, you'll fail because the resulting string passed to bash -c is just a sequence of individually-quoted strings that aren't then interpreted as operators, which is easy to see if you echo the string that would have been passed to bash:
$ (cd /tmp; echo "$(quote_args cd /\; pwd 1\>\&2)")
'cd' '/;' 'pwd' '1>&2'
$ (cd /tmp; echo "$(quote_args cd /\; \[ -e \*sbin \] \&\& echo success 1\>\&2 \|\| echo failure 1\>\&2)")
'cd' '/;' '[' '-e' '*sbin' ']' '&&' 'echo' 'success' '1>&2' '||' 'echo' 'failure' '1>&2'
The problem here is that you're over-quoting. What you need is for the operators to be unquoted as input to the enclosing bash -c, which means they need to be outside the $(quote_args ...) command substitution.
Consequently, what you need to do in the most general sense is to shell-quote each word of the command not intended to be expanded at the time of command substitution separately, and not apply any extra quoting to the shell operators:
$ (cd /tmp; echo "$(quote_args cd /); $(quote_args pwd) 1>&2")
'cd' '/'; 'pwd' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")
/
$ (cd /tmp; echo "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
'cd' '/'; [ -e *'sbin' ] && 'echo' 'success' 1>&2 || 'echo' 'failure' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
success
Once you've done this, the entire string is fair game for further quoting to arbitrary levels of evaluation:
$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")"
/
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")"
/
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")")"
/
$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")"
success
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *sbin ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")"
success
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")")"
success
etc.
These examples may seem overwrought given that words like success, sbin, and pwd don't need to be shell-quoted, but the key point to remember when writing a script taking arbitrary input is that you want to quote everything you're not absolutely sure doesn't need quoting, because you never know when a user will throw in a Robert'; rm -rf /.
To better understand what is going on under the covers, you can play around with two small helper functions:
debug_args() {
for (( I=1; $I <= $#; I++ )); do
echo -n "$I:<${!I}> " 1>&2
done
echo 1>&2
}
debug_args_and_run() {
debug_args "$#"
"$#"
}
that will enumerate each argument to a command before executing it:
$ debug_args_and_run echo a'"'b"'"c arg2
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)"
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'>
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''>
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'>
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'bash'"'"' '"'"'-c'"'"' '"'"''"'"'"'"'"'"'"'"'debug_args_and_run'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'echo'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'a"b'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'c'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'arg2'"'"'"'"'"'"'"'"''"'"''>
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''>
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'>
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
In the given example, simply used double quotes instead of single quotes as outer escape mechanism:
alias rxvt="urxvt -fg '#111111' -bg '#111111'"
This approach is suited for many cases where you just want to pass a fixed string to a command: Just check how the shell will interpret the double-quoted string through an echo, and escape characters with backslash if necessary.
In the example, you'd see that double quotes are sufficient to protect the string:
$ echo "urxvt -fg '#111111' -bg '#111111'"
urxvt -fg '#111111' -bg '#111111'
Here is an elaboration on The One True Answer referenced above:
Sometimes I will be downloading using rsync over ssh and have to escape a filename with a ' in it TWICE! (OMG!) Once for bash and once for ssh. The same principle of alternating quotation delimiters is at work here.
For example, let's say we want to get: Louis Theroux's LA Stories ...
First you enclose Louis Theroux in single quotes for bash and double quotes for ssh:
'"Louis Theroux"'
Then you use single quotes to escape a double quote '"'
The use double quotes to escape the apostrophe "'"
Then repeat #2, using single quotes to escape a double quote '"'
Then enclose LA Stories in single quotes for bash and double quotes for ssh: '"LA Stories"'
And behold! You wind up with this:
rsync -ave ssh '"Louis Theroux"''"'"'"'"''"s LA Stories"'
which is an awful lot of work for one little ' -- but there you go
Obviously, it would be easier simply to surround with double quotes, but where's the challenge in that? Here is the answer using only single quotes. I'm using a variable instead of alias so that's it's easier to print for proof, but it's the same as using alias.
$ rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'
$ echo $rxvt
urxvt -fg '#111111' -bg '#111111'
Explanation
The key is that you can close the single quote and re-open it as many times as you want. For example foo='a''b' is the same as foo='ab'. So you can close the single quote, throw in a literal single quote \', then reopen the next single quote.
Breakdown diagram
This diagram makes it clear by using brackets to show where the single quotes are opened and closed. Quotes are not "nested" like parentheses can be. You can also pay attention to the color highlighting, which is correctly applied. The quoted strings are maroon, whereas the \' is black.
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\' # original
[^^^^^^^^^^] ^[^^^^^^^] ^[^^^^^] ^[^^^^^^^] ^ # show open/close quotes
urxvt -fg ' #111111 ' -bg ' #111111 ' # literal characters remaining
(This is essentially the same answer as Adrian's, but I feel this explains it better. Also his answer has 2 superfluous single quotes at the end.)
in addition to #JasonWoof perfect answer i want to show how i solved related problem
in my case encoding single quotes with '\'' will not always be sufficient, for example if a string must quoted with single quotes, but the total count of quotes results in odd amount
#!/bin/bash
# no closing quote
string='alecxs\'solution'
# this works for string
string="alecxs'solution"
string=alecxs\'solution
string='alecxs'\''solution'
let's assume string is a file name and we need to save quoted file names in a list (like stat -c%N ./* > list)
echo "'$string'" > "$string"
cat "$string"
but processing this list will fail (depending on how many quotes the string does contain in total)
while read file
do
ls -l "$file"
eval ls -l "$file"
done < "$string"
workaround: encode quotes with string manipulation
string="${string//$'\047'/\'\$\'\\\\047\'\'}"
# result
echo "$string"
now it works because quotes are always balanced
echo "'$string'" > list
while read file
do
ls -l "$file"
eval ls -l "$file"
done < list
Hope this helps when facing similar problem
Another way to fix the problem of too many layers of nested quotation:
You are trying to cram too much into too tiny a space, so use a bash function.
The problem is you are trying to have too many levels of nesting, and the basic alias technology is not powerful enough to accommodate. Use a bash function like this to make it so the single, double quotes back ticks and passed in parameters are all handled normally as we would expect:
lets_do_some_stuff() {
tmp=$1 #keep a passed in parameter.
run_your_program $# #use all your passed parameters.
echo -e '\n-------------' #use your single quotes.
echo `date` #use your back ticks.
echo -e "\n-------------" #use your double quotes.
}
alias foobarbaz=lets_do_some_stuff
Then you can use your $1 and $2 variables and single, double quotes and back ticks without worrying about the alias function wrecking their integrity.
This program prints:
el#defiant ~/code $ foobarbaz alien Dyson ring detected #grid 10385
alien Dyson ring detected #grid 10385
-------------
Mon Oct 26 20:30:14 EDT 2015
-------------
shell_escape () {
printf '%s' "'${1//\'/\'\\\'\'}'"
}
Implementation explanation:
double quotes so we can easily output wrapping single quotes and use the ${...} syntax
bash's search and replace looks like: ${varname//search/replacement}
we're replacing ' with '\''
'\'' encodes a single ' like so:
' ends the single quoting
\' encodes a ' (the backslash is needed because we're not inside quotes)
' starts up single quoting again
bash automatically concatenates strings with no white space between
there's a \ before every \ and ' because that's the escaping rules for ${...//.../...} .
string="That's "'##$*&^`(##'
echo "original: $string"
echo "encoded: $(shell_escape "$string")"
echo "expanded: $(bash -c "echo $(shell_escape "$string")")"
P.S. Always encode to single quoted strings because they are way simpler than double quoted strings.
Here are my two cents -- in the case if one wants to be sh-portable, not just bash-specific ( the solution is not too efficient, though, as it starts an external program -- sed ):
put this in quote.sh ( or just quote ) somewhere on your PATH :
# this works with standard input (stdin)
quote() {
echo -n "'" ;
sed 's/\(['"'"']['"'"']*\)/'"'"'"\1"'"'"'/g' ;
echo -n "'"
}
case "$1" in
-) quote ;;
*) echo "usage: cat ... | quote - # single-quotes input for Bourne shell" 2>&1 ;;
esac
An example:
$ echo -n "G'day, mate!" | ./quote.sh -
'G'"'"'day, mate!'
And, of course, that converts back:
$ echo 'G'"'"'day, mate!'
G'day, mate!
Explanation: basically we have to enclose the input with quotes ', and then also replace any single quote within with this micro-monster: '"'"' ( end the opening quote with a pairing ', escape the found single quote by wrapping it with double quotes -- "'", and then finally issue a new opening single quote ', or in pseudo-notation : ' + "'" + ' == '"'"' )
One standard way to do that would be to use sed with the following substitution command:
s/\(['][']*\)/'"\1"'/g
One small problem, though, is that in order to use that in shell one needs to escape all these single quote characters in the sed expression itself -- what leads to something like
sed 's/\(['"'"']['"'"']*\)/'"'"'"\1"'"'"'/g'
( and one good way to build this result is to feed the original expression s/\(['][']*\)/'"\1"'/g to Kyle Rose' or George V. Reilly's scripts ).
Finally, it kind of makes sense to expect the input to come from stdin -- since passing it through command-line arguments could be already too much trouble.
( Oh, and may be we want to add a small help message so that the script does not hang when someone just runs it as ./quote.sh --help wondering what it does. )
If you're generating the shell string within Python 2 or Python 3, the following may help to quote the arguments:
#!/usr/bin/env python
from __future__ import print_function
try: # py3
from shlex import quote as shlex_quote
except ImportError: # py2
from pipes import quote as shlex_quote
s = """foo ain't "bad" so there!"""
print(s)
print(" ".join([shlex_quote(t) for t in s.split()]))
This will output:
foo ain't "bad" so there!
foo 'ain'"'"'t' '"bad"' so 'there!'
If you have GNU Parallel installed you can use its internal quoting:
$ parallel --shellquote
L's 12" record
<Ctrl-D>
'L'"'"'s 12" record'
$ echo 'L'"'"'s 12" record'
L's 12" record
From version 20190222 you can even --shellquote multiple times:
$ parallel --shellquote --shellquote --shellquote
L's 12" record
<Ctrl-D>
'"'"'"'"'"'"'L'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'s 12" record'"'"'"'"'"'"'
$ eval eval echo '"'"'"'"'"'"'L'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'s 12" record'"'"'"'"'"'"'
L's 12" record
It will quote the string in all supported shells (not only bash).
This function:
quote ()
{
local quoted=${1//\'/\'\\\'\'};
printf "'%s'" "$quoted"
}
allows quoting of ' inside '. Use as this:
$ quote "urxvt -fg '#111111' -bg '#111111'"
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
If the line to quote gets more complex, like double quotes mixed with single quotes, it may become quite tricky to get the string to quote inside a variable. When such cases show up, write the exact line that you need to quote inside an script (similar to this).
#!/bin/bash
quote ()
{
local quoted=${1//\'/\'\\\'\'};
printf "'%s'" "$quoted"
}
while read line; do
quote "$line"
done <<-\_lines_to_quote_
urxvt -fg '#111111' -bg '#111111'
Louis Theroux's LA Stories
'single quote phrase' "double quote phrase"
_lines_to_quote_
Will output:
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
'Louis Theroux'\''s LA Stories'
''\''single quote phrase'\'' "double quote phrase"'
All correctly quoted strings inside single quotes.
Here is another solution. This function will take a single argument and appropriately quote it using the single-quote character, just as the voted answer above explains:
single_quote() {
local quoted="'"
local i=0
while [ $i -lt ${#1} ]; do
local ch="${1:i:1}"
if [[ "$ch" != "'" ]]; then
quoted="$quoted$ch"
else
local single_quotes="'"
local j=1
while [ $j -lt ${#1} ] && [[ "${1:i+j:1}" == "'" ]]; do
single_quotes="$single_quotes'"
((j++))
done
quoted="$quoted'\"$single_quotes\"'"
((i+=j-1))
fi
((i++))
done
echo "$quoted'"
}
So, you can use it this way:
single_quote "1 2 '3'"
'1 2 '"'"'3'"'"''
x="this text is quoted: 'hello'"
eval "echo $(single_quote "$x")"
this text is quoted: 'hello'
Related
Let's say, you have a Bash alias like:
alias rxvt='urxvt'
which works fine.
However:
alias rxvt='urxvt -fg '#111111' -bg '#111111''
won't work, and neither will:
alias rxvt='urxvt -fg \'#111111\' -bg \'#111111\''
So how do you end up matching up opening and closing quotes inside a string once you have escaped quotes?
alias rxvt='urxvt -fg'\''#111111'\'' -bg '\''#111111'\''
seems ungainly although it would represent the same string if you're allowed to concatenate them like that.
If you really want to use single quotes in the outermost layer, remember that you can glue both kinds of quotation. Example:
alias rxvt='urxvt -fg '"'"'#111111'"'"' -bg '"'"'#111111'"'"
# ^^^^^ ^^^^^ ^^^^^ ^^^^
# 12345 12345 12345 1234
Explanation of how '"'"' is interpreted as just ':
' End first quotation which uses single quotes.
" Start second quotation, using double-quotes.
' Quoted character.
" End second quotation, using double-quotes.
' Start third quotation, using single quotes.
If you do not place any whitespaces between (1) and (2), or between (4) and (5), the shell will interpret that string as a one long word.
I always just replace each embedded single quote with the sequence: '\'' (that is: quote backslash quote quote) which closes the string, appends an escaped single quote and reopens the string.
I often whip up a "quotify" function in my Perl scripts to do this for me. The steps would be:
s/'/'\\''/g # Handle each embedded quote
$_ = qq['$_']; # Surround result with single quotes.
This pretty much takes care of all cases.
Life gets more fun when you introduce eval into your shell-scripts. You essentially have to re-quotify everything again!
For example, create a Perl script called quotify containing the above statements:
#!/usr/bin/perl -pl
s/'/'\\''/g;
$_ = qq['$_'];
then use it to generate a correctly-quoted string:
$ quotify
urxvt -fg '#111111' -bg '#111111'
result:
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
which can then be copy/pasted into the alias command:
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
(If you need to insert the command into an eval, run the quotify again:
$ quotify
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
result:
'alias rxvt='\''urxvt -fg '\''\'\'''\''#111111'\''\'\'''\'' -bg '\''\'\'''\''#111111'\''\'\'''\'''\'''
which can be copy/pasted into an eval:
eval 'alias rxvt='\''urxvt -fg '\''\'\'''\''#111111'\''\'\'''\'' -bg '\''\'\'''\''#111111'\''\'\'''\'''\'''
Since Bash 2.04 syntax $'string' allows a limit set of escapes.
Since Bash 4.4, $'string' also allows the full set of C-style escapes, making the behavior differ slightly in $'string' in previous versions. (Previously the $('string') form could be used.)
Simple example in Bash 2.04 and newer:
$> echo $'aa\'bb'
aa'bb
$> alias myvar=$'aa\'bb'
$> alias myvar
alias myvar='aa'\''bb'
In your case:
$> alias rxvt=$'urxvt -fg \'#111111\' -bg \'#111111\''
$> alias rxvt
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
Common escaping sequences works as expected:
\' single quote
\" double quote
\\ backslash
\n new line
\t horizontal tab
\r carriage return
Below is copy+pasted related documentation from man bash (version 4.4):
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard. Backslash escape sequences, if present, are decoded as follows:
\a alert (bell)
\b backspace
\e
\E an escape character
\f form feed
\n new line
\r carriage return
\t horizontal tab
\v vertical tab
\\ backslash
\' single quote
\" double quote
\? question mark
\nnn the eight-bit character whose value is the octal
value nnn (one to three digits)
\xHH the eight-bit character whose value is the hexadecimal
value HH (one or two hex digits)
\uHHHH the Unicode (ISO/IEC 10646) character whose value is
the hexadecimal value HHHH (one to four hex digits)
\UHHHHHHHH the Unicode (ISO/IEC 10646) character whose value
is the hexadecimal value HHHHHHHH (one to eight
hex digits)
\cx a control-x character
The expanded result is single-quoted, as if the dollar sign had not been present.
See Quotes and escaping: ANSI C like strings on bash-hackers.org wiki for more details. Also note that "Bash Changes" file (overview here) mentions a lot for changes and bug fixes related to the $'string' quoting mechanism.
According to unix.stackexchange.com How to use a special character as a normal one? it should work (with some variations) in bash, zsh, mksh, ksh93 and FreeBSD and busybox sh.
I don't see the entry on his blog (link pls?) but according to the gnu reference manual:
Enclosing characters in single quotes
(‘'’) preserves the literal value of
each character within the quotes. A
single quote may not occur between
single quotes, even when preceded by a
backslash.
so bash won't understand:
alias x='y \'z '
however, you can do this if you surround with double quotes:
alias x="echo \'y "
> x
> 'y
I can confirm that using '\'' for a single quote inside a single-quoted string does work in Bash, and it can be explained in the same way as the "gluing" argument from earlier in the thread. Suppose we have a quoted string: 'A '\''B'\'' C' (all quotes here are single quotes). If it is passed to echo, it prints the following: A 'B' C.
In each '\'' the first quote closes the current single-quoted string, the following \' glues a single quote to the previous string (\' is a way to specify a single quote without starting a quoted string), and the last quote opens another single-quoted string.
Both versions are working, either with concatenation by using the escaped single quote character (\'), or with concatenation by enclosing the single quote character within double quotes ("'").
The author of the question did not notice that there was an extra single quote (') at the end of his last escaping attempt:
alias rxvt='urxvt -fg'\''#111111'\'' -bg '\''#111111'\''
│ │┊┊| │┊┊│ │┊┊│ │┊┊│
└─STRING──┘┊┊└─STRIN─┘┊┊└─STR─┘┊┊└─STRIN─┘┊┊│
┊┊ ┊┊ ┊┊ ┊┊│
┊┊ ┊┊ ┊┊ ┊┊│
└┴─────────┴┴───┰───┴┴─────────┴┘│
All escaped single quotes │
│
?
As you can see in the previous nice piece of ASCII/Unicode art, the last escaped single quote (\') is followed by an unnecessary single quote ('). Using a syntax-highlighter like the one present in Notepad++ can prove very helpful.
The same is true for another example like the following one:
alias rc='sed '"'"':a;N;$!ba;s/\n/, /g'"'"
alias rc='sed '\'':a;N;$!ba;s/\n/, /g'\'
These two beautiful instances of aliases show in a very intricate and obfuscated way how a file can be lined down. That is, from a file with a lot of lines you get only one line with commas and spaces between the contents of the previous lines. In order to make sense of the previous comment, the following is an example:
$ cat Little_Commas.TXT
201737194
201802699
201835214
$ rc Little_Commas.TXT
201737194, 201802699, 201835214
Simple example of escaping quotes in shell:
$ echo 'abc'\''abc'
abc'abc
$ echo "abc"\""abc"
abc"abc
It's done by finishing already opened one ('), placing escaped one (\'), then opening another one ('). This syntax works for all commands. It's very similar approach to the 1st answer.
I'm not specifically addressing the quoting issue because, well, sometimes, it's just reasonable to consider an alternative approach.
rxvt() { urxvt -fg "#${1:-000000}" -bg "#${2:-FFFFFF}"; }
which you can then call as:
rxvt 123456 654321
the idea being that you can now alias this without concern for quotes:
alias rxvt='rxvt 123456 654321'
or, if you need to include the # in all calls for some reason:
rxvt() { urxvt -fg "${1:-#000000}" -bg "${2:-#FFFFFF}"; }
which you can then call as:
rxvt '#123456' '#654321'
then, of course, an alias is:
alias rxvt="rxvt '#123456' '#654321'"
(oops, i guess i kind of did address the quoting :)
How to escape single quotes (') and double quotes (") with hex and octal chars
If using something like echo, I've had some really complicated and really weird and hard-to-escape (think: very nested) cases where the only thing I could get to work was using octal or hex codes!
Here are some basic examples just to demonstrate how it works:
1. Single quote example, where ' is escaped with hex \x27 or octal \047 (its corresponding ASCII code):
hex \x27
echo -e "Let\x27s get coding!"
# OR
echo -e 'Let\x27s get coding!'
Result:
Let's get coding!
octal \047
echo -e "Let\047s get coding!"
# OR
echo -e 'Let\047s get coding!'
Result:
Let's get coding!
2. Double quote example, where " is escaped with hex \x22 or octal \042 (its corresponding ASCII code).
Note: bash is nuts! Sometimes even the ! char has special meaning, and must either be removed from within the double quotes and then escaped "like this"\! or put entirely within single quotes 'like this!', rather than within double quotes.
# 1. hex; also escape `!` by removing it from within the double quotes
# and escaping it with `\!`
$ echo -e "She said, \x22Let\x27s get coding"\!"\x22"
She said, "Let's get coding!"
# OR put it all within single quotes:
$ echo -e 'She said, \x22Let\x27s get coding!\x22'
She said, "Let's get coding!"
# 2. octal; also escape `!` by removing it from within the double quotes
$ echo -e "She said, \042Let\047s get coding"\!"\042"
She said, "Let's get coding!"
# OR put it all within single quotes:
$ echo -e 'She said, \042Let\047s get coding!\042'
She said, "Let's get coding!"
# 3. mixed hex and octal, just for fun
# also escape `!` by removing it from within the double quotes when it is followed by
# another escape sequence
$ echo -e "She said, \x22Let\047s get coding! It\x27s waaay past time to begin"\!"\042"
She said, "Let's get coding! It's waaay past time to begin!"
# OR put it all within single quotes:
$ echo -e 'She said, \x22Let\047s get coding! It\x27s waaay past time to begin!\042'
She said, "Let's get coding! It's waaay past time to begin!"
Note that if you don't properly escape !, when needed, as I've shown two ways to do above, you'll get some weird errors, like this:
$ echo -e "She said, \x22Let\047s get coding! It\x27s waaay past time to begin!\042"
bash: !\042: event not found
OR:
$ echo -e "She said, \x22Let\x27s get coding!\x22"
bash: !\x22: event not found
One more alternative: this allows mixed expansion and non-expansion all within the same bash string
Here is another demo of an alternative escaping technique.
First, read the main answer by #liori to see how the 2nd form below works. Now, read these two alternative ways of escaping characters. Both examples below are identical in their output:
CMD="gs_set_title"
# 1. 1st technique: escape the $ symbol with a backslash (\) so it doesn't
# run and expand the command following it
echo "$CMD '\$(basename \"\$(pwd)\")'"
# 2. 2nd technique (does the same thing in a different way): escape the
# $ symbol using single quotes around it, and the single quote (') symbol
# using double quotes around it
echo "$CMD ""'"'$(basename "$(pwd)")'"'"
Sample output:
gs_set_title '$(basename "$(pwd)")'
gs_set_title '$(basename "$(pwd)")'
Note: for my gs_set_title bash function, which I have in my ~/.bash_aliases file somewhere around here, see my other answer here.
References:
https://en.wikipedia.org/wiki/ASCII#Printable_characters
https://serverfault.com/questions/208265/what-is-bash-event-not-found/208266#208266
See also my other answer here: How do I write non-ASCII characters using echo?.
I just use shell codes.. e.g. \x27 or \\x22 as applicable. No hassle, ever really.
Since one cannot put single quotes within single quoted strings, the simplest and most readable option is to use a HEREDOC string
command=$(cat <<'COMMAND'
urxvt -fg '#111111' -bg '#111111'
COMMAND
)
alias rxvt=$command
In the code above, the HEREDOC is sent to the cat command and the output of that is assigned to a variable via the command substitution notation $(..)
Putting a single quote around the HEREDOC is needed since it is within a $()
IMHO the real answer is that you can't escape single-quotes within single-quoted strings.
Its impossible.
If we presume we are using bash.
From bash manual...
Enclosing characters in single quotes preserves the literal value of each
character within the quotes. A single quote may not occur
between single quotes, even when preceded by a backslash.
You need to use one of the other string escape mechanisms " or \
There is nothing magic about alias that demands it use single quotes.
Both the following work in bash.
alias rxvt="urxvt -fg '#111111' -bg '#111111'"
alias rxvt=urxvt\ -fg\ \'#111111\'\ -bg\ \'#111111\'
The latter is using \ to escape the space character.
There is also nothing magic about #111111 that requires single quotes.
The following options achieves the same result the other two options, in that the rxvt alias works as expected.
alias rxvt='urxvt -fg "#111111" -bg "#111111"'
alias rxvt="urxvt -fg \"#111111\" -bg \"#111111\""
You can also escape the troublesome # directly
alias rxvt="urxvt -fg \#111111 -bg \#111111"
A minimal answer is needed so that people can get going without spending a lot of time as I had to sifting through people waxing eloquent.
There is no way to escape single quotes or anything else within single quotes.
The following is, perhaps surprisingly, a complete command:
$ echo '\'
whose output is:
\
Backslashes, surprisingly to even long-time users of bash, have no meaning inside single quotes. Nor does anything else.
Most of these answers hit on the specific case you're asking about. There is a general approach that a friend and I have developed that allows for arbitrary quoting in case you need to quote bash commands through multiple layers of shell expansion, e.g., through ssh, su -c, bash -c, etc. There is one core primitive you need, here in native bash:
quote_args() {
local sq="'"
local dq='"'
local space=""
local arg
for arg; do
echo -n "$space'${arg//$sq/$sq$dq$sq$dq$sq}'"
space=" "
done
}
This does exactly what it says: it shell-quotes each argument individually (after bash expansion, of course):
$ quote_args foo bar
'foo' 'bar'
$ quote_args arg1 'arg2 arg2a' arg3
'arg1' 'arg2 arg2a' 'arg3'
$ quote_args dq'"'
'dq"'
$ quote_args dq'"' sq"'"
'dq"' 'sq'"'"''
$ quote_args "*"
'*'
$ quote_args /b*
'/bin' '/boot'
It does the obvious thing for one layer of expansion:
$ bash -c "$(quote_args echo a'"'b"'"c arg2)"
a"b'c arg2
(Note that the double quotes around $(quote_args ...) are necessary to make the result into a single argument to bash -c.) And it can be used more generally to quote properly through multiple layers of expansion:
$ bash -c "$(quote_args bash -c "$(quote_args echo a'"'b"'"c arg2)")"
a"b'c arg2
The above example:
shell-quotes each argument to the inner quote_args individually and then combines the resulting output into a single argument with the inner double quotes.
shell-quotes bash, -c, and the already once-quoted result from step 1, and then combines the result into a single argument with the outer double quotes.
sends that mess as the argument to the outer bash -c.
That's the idea in a nutshell. You can do some pretty complicated stuff with this, but you have to be careful about order of evaluation and about which substrings are quoted. For instance, the following do the wrong things (for some definition of "wrong"):
$ (cd /tmp; bash -c "$(quote_args cd /; pwd 1>&2)")
/tmp
$ (cd /tmp; bash -c "$(quote_args cd /; [ -e *sbin ] && echo success 1>&2 || echo failure 1>&2)")
failure
In the first example, bash immediately expands quote_args cd /; pwd 1>&2 into two separate commands, quote_args cd / and pwd 1>&2, so the CWD is still /tmp when the pwd command is executed. The second example illustrates a similar problem for globbing. Indeed, the same basic problem occurs with all bash expansions. The problem here is that a command substitution isn't a function call: it's literally evaluating one bash script and using its output as part of another bash script.
If you try to simply escape the shell operators, you'll fail because the resulting string passed to bash -c is just a sequence of individually-quoted strings that aren't then interpreted as operators, which is easy to see if you echo the string that would have been passed to bash:
$ (cd /tmp; echo "$(quote_args cd /\; pwd 1\>\&2)")
'cd' '/;' 'pwd' '1>&2'
$ (cd /tmp; echo "$(quote_args cd /\; \[ -e \*sbin \] \&\& echo success 1\>\&2 \|\| echo failure 1\>\&2)")
'cd' '/;' '[' '-e' '*sbin' ']' '&&' 'echo' 'success' '1>&2' '||' 'echo' 'failure' '1>&2'
The problem here is that you're over-quoting. What you need is for the operators to be unquoted as input to the enclosing bash -c, which means they need to be outside the $(quote_args ...) command substitution.
Consequently, what you need to do in the most general sense is to shell-quote each word of the command not intended to be expanded at the time of command substitution separately, and not apply any extra quoting to the shell operators:
$ (cd /tmp; echo "$(quote_args cd /); $(quote_args pwd) 1>&2")
'cd' '/'; 'pwd' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")
/
$ (cd /tmp; echo "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
'cd' '/'; [ -e *'sbin' ] && 'echo' 'success' 1>&2 || 'echo' 'failure' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
success
Once you've done this, the entire string is fair game for further quoting to arbitrary levels of evaluation:
$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")"
/
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")"
/
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")")"
/
$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")"
success
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *sbin ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")"
success
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")")"
success
etc.
These examples may seem overwrought given that words like success, sbin, and pwd don't need to be shell-quoted, but the key point to remember when writing a script taking arbitrary input is that you want to quote everything you're not absolutely sure doesn't need quoting, because you never know when a user will throw in a Robert'; rm -rf /.
To better understand what is going on under the covers, you can play around with two small helper functions:
debug_args() {
for (( I=1; $I <= $#; I++ )); do
echo -n "$I:<${!I}> " 1>&2
done
echo 1>&2
}
debug_args_and_run() {
debug_args "$#"
"$#"
}
that will enumerate each argument to a command before executing it:
$ debug_args_and_run echo a'"'b"'"c arg2
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)"
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'>
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''>
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'>
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'bash'"'"' '"'"'-c'"'"' '"'"''"'"'"'"'"'"'"'"'debug_args_and_run'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'echo'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'a"b'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'c'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'arg2'"'"'"'"'"'"'"'"''"'"''>
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''>
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'>
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
In the given example, simply used double quotes instead of single quotes as outer escape mechanism:
alias rxvt="urxvt -fg '#111111' -bg '#111111'"
This approach is suited for many cases where you just want to pass a fixed string to a command: Just check how the shell will interpret the double-quoted string through an echo, and escape characters with backslash if necessary.
In the example, you'd see that double quotes are sufficient to protect the string:
$ echo "urxvt -fg '#111111' -bg '#111111'"
urxvt -fg '#111111' -bg '#111111'
Here is an elaboration on The One True Answer referenced above:
Sometimes I will be downloading using rsync over ssh and have to escape a filename with a ' in it TWICE! (OMG!) Once for bash and once for ssh. The same principle of alternating quotation delimiters is at work here.
For example, let's say we want to get: Louis Theroux's LA Stories ...
First you enclose Louis Theroux in single quotes for bash and double quotes for ssh:
'"Louis Theroux"'
Then you use single quotes to escape a double quote '"'
The use double quotes to escape the apostrophe "'"
Then repeat #2, using single quotes to escape a double quote '"'
Then enclose LA Stories in single quotes for bash and double quotes for ssh: '"LA Stories"'
And behold! You wind up with this:
rsync -ave ssh '"Louis Theroux"''"'"'"'"''"s LA Stories"'
which is an awful lot of work for one little ' -- but there you go
Obviously, it would be easier simply to surround with double quotes, but where's the challenge in that? Here is the answer using only single quotes. I'm using a variable instead of alias so that's it's easier to print for proof, but it's the same as using alias.
$ rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'
$ echo $rxvt
urxvt -fg '#111111' -bg '#111111'
Explanation
The key is that you can close the single quote and re-open it as many times as you want. For example foo='a''b' is the same as foo='ab'. So you can close the single quote, throw in a literal single quote \', then reopen the next single quote.
Breakdown diagram
This diagram makes it clear by using brackets to show where the single quotes are opened and closed. Quotes are not "nested" like parentheses can be. You can also pay attention to the color highlighting, which is correctly applied. The quoted strings are maroon, whereas the \' is black.
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\' # original
[^^^^^^^^^^] ^[^^^^^^^] ^[^^^^^] ^[^^^^^^^] ^ # show open/close quotes
urxvt -fg ' #111111 ' -bg ' #111111 ' # literal characters remaining
(This is essentially the same answer as Adrian's, but I feel this explains it better. Also his answer has 2 superfluous single quotes at the end.)
in addition to #JasonWoof perfect answer i want to show how i solved related problem
in my case encoding single quotes with '\'' will not always be sufficient, for example if a string must quoted with single quotes, but the total count of quotes results in odd amount
#!/bin/bash
# no closing quote
string='alecxs\'solution'
# this works for string
string="alecxs'solution"
string=alecxs\'solution
string='alecxs'\''solution'
let's assume string is a file name and we need to save quoted file names in a list (like stat -c%N ./* > list)
echo "'$string'" > "$string"
cat "$string"
but processing this list will fail (depending on how many quotes the string does contain in total)
while read file
do
ls -l "$file"
eval ls -l "$file"
done < "$string"
workaround: encode quotes with string manipulation
string="${string//$'\047'/\'\$\'\\\\047\'\'}"
# result
echo "$string"
now it works because quotes are always balanced
echo "'$string'" > list
while read file
do
ls -l "$file"
eval ls -l "$file"
done < list
Hope this helps when facing similar problem
Another way to fix the problem of too many layers of nested quotation:
You are trying to cram too much into too tiny a space, so use a bash function.
The problem is you are trying to have too many levels of nesting, and the basic alias technology is not powerful enough to accommodate. Use a bash function like this to make it so the single, double quotes back ticks and passed in parameters are all handled normally as we would expect:
lets_do_some_stuff() {
tmp=$1 #keep a passed in parameter.
run_your_program $# #use all your passed parameters.
echo -e '\n-------------' #use your single quotes.
echo `date` #use your back ticks.
echo -e "\n-------------" #use your double quotes.
}
alias foobarbaz=lets_do_some_stuff
Then you can use your $1 and $2 variables and single, double quotes and back ticks without worrying about the alias function wrecking their integrity.
This program prints:
el#defiant ~/code $ foobarbaz alien Dyson ring detected #grid 10385
alien Dyson ring detected #grid 10385
-------------
Mon Oct 26 20:30:14 EDT 2015
-------------
shell_escape () {
printf '%s' "'${1//\'/\'\\\'\'}'"
}
Implementation explanation:
double quotes so we can easily output wrapping single quotes and use the ${...} syntax
bash's search and replace looks like: ${varname//search/replacement}
we're replacing ' with '\''
'\'' encodes a single ' like so:
' ends the single quoting
\' encodes a ' (the backslash is needed because we're not inside quotes)
' starts up single quoting again
bash automatically concatenates strings with no white space between
there's a \ before every \ and ' because that's the escaping rules for ${...//.../...} .
string="That's "'##$*&^`(##'
echo "original: $string"
echo "encoded: $(shell_escape "$string")"
echo "expanded: $(bash -c "echo $(shell_escape "$string")")"
P.S. Always encode to single quoted strings because they are way simpler than double quoted strings.
Here are my two cents -- in the case if one wants to be sh-portable, not just bash-specific ( the solution is not too efficient, though, as it starts an external program -- sed ):
put this in quote.sh ( or just quote ) somewhere on your PATH :
# this works with standard input (stdin)
quote() {
echo -n "'" ;
sed 's/\(['"'"']['"'"']*\)/'"'"'"\1"'"'"'/g' ;
echo -n "'"
}
case "$1" in
-) quote ;;
*) echo "usage: cat ... | quote - # single-quotes input for Bourne shell" 2>&1 ;;
esac
An example:
$ echo -n "G'day, mate!" | ./quote.sh -
'G'"'"'day, mate!'
And, of course, that converts back:
$ echo 'G'"'"'day, mate!'
G'day, mate!
Explanation: basically we have to enclose the input with quotes ', and then also replace any single quote within with this micro-monster: '"'"' ( end the opening quote with a pairing ', escape the found single quote by wrapping it with double quotes -- "'", and then finally issue a new opening single quote ', or in pseudo-notation : ' + "'" + ' == '"'"' )
One standard way to do that would be to use sed with the following substitution command:
s/\(['][']*\)/'"\1"'/g
One small problem, though, is that in order to use that in shell one needs to escape all these single quote characters in the sed expression itself -- what leads to something like
sed 's/\(['"'"']['"'"']*\)/'"'"'"\1"'"'"'/g'
( and one good way to build this result is to feed the original expression s/\(['][']*\)/'"\1"'/g to Kyle Rose' or George V. Reilly's scripts ).
Finally, it kind of makes sense to expect the input to come from stdin -- since passing it through command-line arguments could be already too much trouble.
( Oh, and may be we want to add a small help message so that the script does not hang when someone just runs it as ./quote.sh --help wondering what it does. )
If you're generating the shell string within Python 2 or Python 3, the following may help to quote the arguments:
#!/usr/bin/env python
from __future__ import print_function
try: # py3
from shlex import quote as shlex_quote
except ImportError: # py2
from pipes import quote as shlex_quote
s = """foo ain't "bad" so there!"""
print(s)
print(" ".join([shlex_quote(t) for t in s.split()]))
This will output:
foo ain't "bad" so there!
foo 'ain'"'"'t' '"bad"' so 'there!'
If you have GNU Parallel installed you can use its internal quoting:
$ parallel --shellquote
L's 12" record
<Ctrl-D>
'L'"'"'s 12" record'
$ echo 'L'"'"'s 12" record'
L's 12" record
From version 20190222 you can even --shellquote multiple times:
$ parallel --shellquote --shellquote --shellquote
L's 12" record
<Ctrl-D>
'"'"'"'"'"'"'L'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'s 12" record'"'"'"'"'"'"'
$ eval eval echo '"'"'"'"'"'"'L'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'s 12" record'"'"'"'"'"'"'
L's 12" record
It will quote the string in all supported shells (not only bash).
This function:
quote ()
{
local quoted=${1//\'/\'\\\'\'};
printf "'%s'" "$quoted"
}
allows quoting of ' inside '. Use as this:
$ quote "urxvt -fg '#111111' -bg '#111111'"
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
If the line to quote gets more complex, like double quotes mixed with single quotes, it may become quite tricky to get the string to quote inside a variable. When such cases show up, write the exact line that you need to quote inside an script (similar to this).
#!/bin/bash
quote ()
{
local quoted=${1//\'/\'\\\'\'};
printf "'%s'" "$quoted"
}
while read line; do
quote "$line"
done <<-\_lines_to_quote_
urxvt -fg '#111111' -bg '#111111'
Louis Theroux's LA Stories
'single quote phrase' "double quote phrase"
_lines_to_quote_
Will output:
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
'Louis Theroux'\''s LA Stories'
''\''single quote phrase'\'' "double quote phrase"'
All correctly quoted strings inside single quotes.
Here is another solution. This function will take a single argument and appropriately quote it using the single-quote character, just as the voted answer above explains:
single_quote() {
local quoted="'"
local i=0
while [ $i -lt ${#1} ]; do
local ch="${1:i:1}"
if [[ "$ch" != "'" ]]; then
quoted="$quoted$ch"
else
local single_quotes="'"
local j=1
while [ $j -lt ${#1} ] && [[ "${1:i+j:1}" == "'" ]]; do
single_quotes="$single_quotes'"
((j++))
done
quoted="$quoted'\"$single_quotes\"'"
((i+=j-1))
fi
((i++))
done
echo "$quoted'"
}
So, you can use it this way:
single_quote "1 2 '3'"
'1 2 '"'"'3'"'"''
x="this text is quoted: 'hello'"
eval "echo $(single_quote "$x")"
this text is quoted: 'hello'
I am writing a BASH script and I am using the bash command. Which one of the following is correct (or are both incorrect)?
bash $pbs_dir/${module_name}.${target_ID}.${instance_ID}.pbs
or
bash \$pbs_dir/\${module_name}.\${target_ID}.\${instance_ID}.pbs
\$ will be expanded to literal $, so there is a big difference:
$ a="hello"
$ echo $a
hello
$ echo \$a
$a
Also note that you almost always want to double quote your parameter expansions to avoid word splitting and pathname expansion:
echo "$a"
So you properly want to use the following:
bash "$pbs_dir/${module_name}.${target_ID}.${instance_ID}.pbs"
In bash :
Using double quotes
echo "*" #Globbing is never done
echo "$variable" # Variable expansion is always done
echo "$(command)" # command expansion is always done
Using single quotes
echo '*' #Globbing is never done
echo '$variable' # Variable expansion is never done
echo '$(command)' # command expansion is never done
Using no quotes
echo * #Globbing always done.
echo $variable; # Variable expansion is always done
echo $(command) # command expansion is always done
will this apply for all commands?
It looks like you are looking for exceptions, and I'd guess you have some in mind. I'm going to make the assumption that set -f/set -o noglob are being excluded from this case?
When you use the dd command, globbing will not occure, even if unquoted.
$ ls *.txt
blah.txt file1.txt file2.txt file.txt logfile.txt
$ dd if=*.txt of=glob.txt
dd: failed to open ‘*.txt’: No such file or directory
Rebuttal and false positives
Here are some examples that are odd, but follow expansion
variable='2010-09-08 12:34:56'
echo "$variable" | xargs date +%s -d
date: extra operand '12:34:56'
The extra operand shows that variable expansion is happening, you are losing the quotes in the pipe.
$ date +%s -d 2010-09-08 12:34:56
date: extra operand ‘12:34:56’
This also happens if you create a script to echo $1 and then expand your quoted variable while passing. It expands, and works as expected. So, the issue is not with xargs, but with your expansion before the pipe which is normal.
Eval... evals whole purpose is to do expansion of its args prior to running a command. Expansion also happens with bash -c, except it takes one argument. So, again, this is not an expansion issue, but a command usage issue.
cmd='printf "%s\n" "$(date -d "$variable" +%c)"'
bash -c $cmd
works the same as the expanded version
$ bash -c printf "%s\n" "$(date -d "$variable" +%c)"
printf: usage: printf [-v var] format [arguments]
I really enjoyed Hauri's $'...' and $"..." information--however, those are not the samething we are talking about. They are in fact behaving as the bash man page says they should. $'' is as different from '' as (()) is from $(())
I got excited about this one, so...
$ ls
mclark.txt qt sign_in.txt skel.bash
$ zip m*t.zip *t
$ ls *.zip
m*t.zip
However, this isn't right either-- the splat expands, but upon no match zip uses it as a literal. I found a few commands that did this, but if there was a match (I added a my.zip later) it uses the matched expansion (an error was thrown, b/c my.zip was a text file for testing purposes).
There are multiple forces in place. In general you can assume that single quotes is to hide the contents from bash expansion. Double quotes is to group values which might have white space so that bash sees them as one logical unit but also disable globbing. There are many caveats though...
Enclosing characters in double quotes preserves the literal value of
all characters within the quotes, with the exception of $, ', \, and,
when history expansion is enabled, !. The characters $ and ' retain
their special meaning within double quotes. The backslash retains
its special meaning only when followed by one of the following
characters: $, ', ", \, or . A double quote may be quoted
within double quotes by preceding it with a backslash. If enabled,
history expansion will be performed unless an ! appearing in double
quotes is escaped using a backslash. The backslash pre-ceding the !
is not removed.
See section QUOTING from man bash
This example below, will either confuse you further or make it clearer.
$ echo "*test*"
*test*
$ echo '*test*'
*test*
$ msg=$(echo '*test*')
$ echo '$msg'
$msg
$ echo "$msg"
*test*
$ echo $msg
speedtest test1 test2 test3 test4 test5 testdir
note that if there were no matches it would print *test* not empty line as commented by Hastur.
some other interesting tidbits
note that this doesn't work
$ echo 'single quotes don\'t escape'
but this works
$ echo "\"double quotes\" escape"
but you can use one in other without escaping
$ echo '"' "'"
Short answer: Yes
This asumptions are basicaly true, alway!
variable='2010-09-08 12:34:56'
vname=variable
date -d "$variable" +%s
1283942096
date -d "${!vname}" +%s
1283942096
date -d $variable +%s
date: extra operand '+%s'
Try 'date --help' for more information.
date -d '$variable' +%s
date: invalid date '$variable'
date -d ${!vname} +%s
date: extra operand '+%s'
Try 'date --help' for more information.
But
Some commands like xargs work precisely about expansion and parameter distribution.
echo "$variable" | xargs date +%s -d
date: extra operand '12:34:56'
Try 'date --help' for more information.
You have to use -0 arg to xargs:
echo "$variable" | xargs -0 date +%s -d
1283942096
Builtin commands could use args differently, especialy eval:
cmd='printf "%s\n" $(date -d "$variable" +%c)'
eval $cmd
Wed
Sep
8
12:34:56
2010
cmd='printf "%s\n" "$(date -d "$variable" +%c)"'
eval "$cmd"
Wed Sep 8 12:34:56 2010
eval $cmd
Wed Sep 8 12:34:56 2010
bash -c "$cmd"
Mon May 16 00:00:00 2016
bash -c $cmd
printf: usage: printf [-v var] format [arguments]
Syntax of funny thing under bash are not limited to "..", '..', and ${}
$'...' let you print special characters, but don't expand variables:
echo $'This\tis\ta string containing ${variable}'
This is a string containing ${variable}
Backticks: For compatibility, backtick are always supported. If not very readable, you may see this in some scripts:
echo `date +%c -d "${!vname}"`
Wed Sep 8 12:34:56 2010
Syntaxe $"..." could be used for localization:
export TEXTDOMAIN=bash
export LANG=fr_CH.utf-8
echo $"Running"
En cours d'exécution
If nothing matches *.xtx while a.txt is a file
mv a.txt *.xtx
will get you an unexpected result too.
The same applies for other things like cp and even this treats it as quoted:
$ ls *.xtx
/bin/ls: cannot access *.xtx: No such file or directory
$ echo "A" > *.xtx
$ ls *.xtx
*.xtx
$
While they should all return error just like if there were more the one file you would get "ambiguous redirect".
will this apply for all commands?
Yes.
From Bash reference manuals:
3.1.2.2 Single Quotes
Enclosing characters in single quotes (') preserves the literal
value of each character within the quotes. A single quote may not
occur between single quotes, even when preceded by a backslash.
and
3.1.2.3 Double Quotes
Enclosing characters in double quotes (") preserves the literal
value of all characters within the quotes, with the exception of $,
`, \, and, when history expansion is enabled, !. The characters
$ and ` retain their special meaning within double quotes (see
Shell Expansions). The backslash retains its special meaning only
when followed by one of the following characters: $, `, ", \,
or newline. Within double quotes, backslashes that are followed by one
of these characters are removed. Backslashes preceding characters
without a special meaning are left unmodified. A double quote may be
quoted within double quotes by preceding it with a backslash. If
enabled, history expansion will be performed unless an ! appearing
in double quotes is escaped using a backslash. The backslash preceding
the ! is not removed.
The special parameters * and # have special meaning when in double
quotes (see Shell Parameter Expansion).
Probably the shell reference manuals and the shell man pages contain the intended behavior ... but the result might not always be what was originally intended.
Reading the "QUOTING" section of the man pages is also intresting. This is a section from the bash man page concerning the single and double quotes: (which is pretty much the same content as the reference manual)
Enclosing characters in single quotes preserves the literal value of
each character within the quotes. A single quote may not occur
between single quotes, even when preceded by a backslash.
Enclosing characters in double quotes preserves the literal value of
all characters within the quotes, with the exception of $, `, \, and,
when history expansion is enabled, !. The characters $ and `
retain their special meaning within double quotes. The backslash
retains its special meaning only when followed by one of the following
characters: $, `, ", \, or . A double quote may be
quoted within double quotes by preceding it with a backslash. If
enabled, history expan- sion will be performed unless an !
appearing in double quotes is escaped using a backslash. The
backslash preceding the ! is not removed.
Let's say, you have a Bash alias like:
alias rxvt='urxvt'
which works fine.
However:
alias rxvt='urxvt -fg '#111111' -bg '#111111''
won't work, and neither will:
alias rxvt='urxvt -fg \'#111111\' -bg \'#111111\''
So how do you end up matching up opening and closing quotes inside a string once you have escaped quotes?
alias rxvt='urxvt -fg'\''#111111'\'' -bg '\''#111111'\''
seems ungainly although it would represent the same string if you're allowed to concatenate them like that.
If you really want to use single quotes in the outermost layer, remember that you can glue both kinds of quotation. Example:
alias rxvt='urxvt -fg '"'"'#111111'"'"' -bg '"'"'#111111'"'"
# ^^^^^ ^^^^^ ^^^^^ ^^^^
# 12345 12345 12345 1234
Explanation of how '"'"' is interpreted as just ':
' End first quotation which uses single quotes.
" Start second quotation, using double-quotes.
' Quoted character.
" End second quotation, using double-quotes.
' Start third quotation, using single quotes.
If you do not place any whitespaces between (1) and (2), or between (4) and (5), the shell will interpret that string as a one long word.
I always just replace each embedded single quote with the sequence: '\'' (that is: quote backslash quote quote) which closes the string, appends an escaped single quote and reopens the string.
I often whip up a "quotify" function in my Perl scripts to do this for me. The steps would be:
s/'/'\\''/g # Handle each embedded quote
$_ = qq['$_']; # Surround result with single quotes.
This pretty much takes care of all cases.
Life gets more fun when you introduce eval into your shell-scripts. You essentially have to re-quotify everything again!
For example, create a Perl script called quotify containing the above statements:
#!/usr/bin/perl -pl
s/'/'\\''/g;
$_ = qq['$_'];
then use it to generate a correctly-quoted string:
$ quotify
urxvt -fg '#111111' -bg '#111111'
result:
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
which can then be copy/pasted into the alias command:
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
(If you need to insert the command into an eval, run the quotify again:
$ quotify
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
result:
'alias rxvt='\''urxvt -fg '\''\'\'''\''#111111'\''\'\'''\'' -bg '\''\'\'''\''#111111'\''\'\'''\'''\'''
which can be copy/pasted into an eval:
eval 'alias rxvt='\''urxvt -fg '\''\'\'''\''#111111'\''\'\'''\'' -bg '\''\'\'''\''#111111'\''\'\'''\'''\'''
Since Bash 2.04 syntax $'string' allows a limit set of escapes.
Since Bash 4.4, $'string' also allows the full set of C-style escapes, making the behavior differ slightly in $'string' in previous versions. (Previously the $('string') form could be used.)
Simple example in Bash 2.04 and newer:
$> echo $'aa\'bb'
aa'bb
$> alias myvar=$'aa\'bb'
$> alias myvar
alias myvar='aa'\''bb'
In your case:
$> alias rxvt=$'urxvt -fg \'#111111\' -bg \'#111111\''
$> alias rxvt
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
Common escaping sequences works as expected:
\' single quote
\" double quote
\\ backslash
\n new line
\t horizontal tab
\r carriage return
Below is copy+pasted related documentation from man bash (version 4.4):
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard. Backslash escape sequences, if present, are decoded as follows:
\a alert (bell)
\b backspace
\e
\E an escape character
\f form feed
\n new line
\r carriage return
\t horizontal tab
\v vertical tab
\\ backslash
\' single quote
\" double quote
\? question mark
\nnn the eight-bit character whose value is the octal
value nnn (one to three digits)
\xHH the eight-bit character whose value is the hexadecimal
value HH (one or two hex digits)
\uHHHH the Unicode (ISO/IEC 10646) character whose value is
the hexadecimal value HHHH (one to four hex digits)
\UHHHHHHHH the Unicode (ISO/IEC 10646) character whose value
is the hexadecimal value HHHHHHHH (one to eight
hex digits)
\cx a control-x character
The expanded result is single-quoted, as if the dollar sign had not been present.
See Quotes and escaping: ANSI C like strings on bash-hackers.org wiki for more details. Also note that "Bash Changes" file (overview here) mentions a lot for changes and bug fixes related to the $'string' quoting mechanism.
According to unix.stackexchange.com How to use a special character as a normal one? it should work (with some variations) in bash, zsh, mksh, ksh93 and FreeBSD and busybox sh.
I don't see the entry on his blog (link pls?) but according to the gnu reference manual:
Enclosing characters in single quotes
(‘'’) preserves the literal value of
each character within the quotes. A
single quote may not occur between
single quotes, even when preceded by a
backslash.
so bash won't understand:
alias x='y \'z '
however, you can do this if you surround with double quotes:
alias x="echo \'y "
> x
> 'y
I can confirm that using '\'' for a single quote inside a single-quoted string does work in Bash, and it can be explained in the same way as the "gluing" argument from earlier in the thread. Suppose we have a quoted string: 'A '\''B'\'' C' (all quotes here are single quotes). If it is passed to echo, it prints the following: A 'B' C.
In each '\'' the first quote closes the current single-quoted string, the following \' glues a single quote to the previous string (\' is a way to specify a single quote without starting a quoted string), and the last quote opens another single-quoted string.
Both versions are working, either with concatenation by using the escaped single quote character (\'), or with concatenation by enclosing the single quote character within double quotes ("'").
The author of the question did not notice that there was an extra single quote (') at the end of his last escaping attempt:
alias rxvt='urxvt -fg'\''#111111'\'' -bg '\''#111111'\''
│ │┊┊| │┊┊│ │┊┊│ │┊┊│
└─STRING──┘┊┊└─STRIN─┘┊┊└─STR─┘┊┊└─STRIN─┘┊┊│
┊┊ ┊┊ ┊┊ ┊┊│
┊┊ ┊┊ ┊┊ ┊┊│
└┴─────────┴┴───┰───┴┴─────────┴┘│
All escaped single quotes │
│
?
As you can see in the previous nice piece of ASCII/Unicode art, the last escaped single quote (\') is followed by an unnecessary single quote ('). Using a syntax-highlighter like the one present in Notepad++ can prove very helpful.
The same is true for another example like the following one:
alias rc='sed '"'"':a;N;$!ba;s/\n/, /g'"'"
alias rc='sed '\'':a;N;$!ba;s/\n/, /g'\'
These two beautiful instances of aliases show in a very intricate and obfuscated way how a file can be lined down. That is, from a file with a lot of lines you get only one line with commas and spaces between the contents of the previous lines. In order to make sense of the previous comment, the following is an example:
$ cat Little_Commas.TXT
201737194
201802699
201835214
$ rc Little_Commas.TXT
201737194, 201802699, 201835214
Simple example of escaping quotes in shell:
$ echo 'abc'\''abc'
abc'abc
$ echo "abc"\""abc"
abc"abc
It's done by finishing already opened one ('), placing escaped one (\'), then opening another one ('). This syntax works for all commands. It's very similar approach to the 1st answer.
I'm not specifically addressing the quoting issue because, well, sometimes, it's just reasonable to consider an alternative approach.
rxvt() { urxvt -fg "#${1:-000000}" -bg "#${2:-FFFFFF}"; }
which you can then call as:
rxvt 123456 654321
the idea being that you can now alias this without concern for quotes:
alias rxvt='rxvt 123456 654321'
or, if you need to include the # in all calls for some reason:
rxvt() { urxvt -fg "${1:-#000000}" -bg "${2:-#FFFFFF}"; }
which you can then call as:
rxvt '#123456' '#654321'
then, of course, an alias is:
alias rxvt="rxvt '#123456' '#654321'"
(oops, i guess i kind of did address the quoting :)
How to escape single quotes (') and double quotes (") with hex and octal chars
If using something like echo, I've had some really complicated and really weird and hard-to-escape (think: very nested) cases where the only thing I could get to work was using octal or hex codes!
Here are some basic examples just to demonstrate how it works:
1. Single quote example, where ' is escaped with hex \x27 or octal \047 (its corresponding ASCII code):
hex \x27
echo -e "Let\x27s get coding!"
# OR
echo -e 'Let\x27s get coding!'
Result:
Let's get coding!
octal \047
echo -e "Let\047s get coding!"
# OR
echo -e 'Let\047s get coding!'
Result:
Let's get coding!
2. Double quote example, where " is escaped with hex \x22 or octal \042 (its corresponding ASCII code).
Note: bash is nuts! Sometimes even the ! char has special meaning, and must either be removed from within the double quotes and then escaped "like this"\! or put entirely within single quotes 'like this!', rather than within double quotes.
# 1. hex; also escape `!` by removing it from within the double quotes
# and escaping it with `\!`
$ echo -e "She said, \x22Let\x27s get coding"\!"\x22"
She said, "Let's get coding!"
# OR put it all within single quotes:
$ echo -e 'She said, \x22Let\x27s get coding!\x22'
She said, "Let's get coding!"
# 2. octal; also escape `!` by removing it from within the double quotes
$ echo -e "She said, \042Let\047s get coding"\!"\042"
She said, "Let's get coding!"
# OR put it all within single quotes:
$ echo -e 'She said, \042Let\047s get coding!\042'
She said, "Let's get coding!"
# 3. mixed hex and octal, just for fun
# also escape `!` by removing it from within the double quotes when it is followed by
# another escape sequence
$ echo -e "She said, \x22Let\047s get coding! It\x27s waaay past time to begin"\!"\042"
She said, "Let's get coding! It's waaay past time to begin!"
# OR put it all within single quotes:
$ echo -e 'She said, \x22Let\047s get coding! It\x27s waaay past time to begin!\042'
She said, "Let's get coding! It's waaay past time to begin!"
Note that if you don't properly escape !, when needed, as I've shown two ways to do above, you'll get some weird errors, like this:
$ echo -e "She said, \x22Let\047s get coding! It\x27s waaay past time to begin!\042"
bash: !\042: event not found
OR:
$ echo -e "She said, \x22Let\x27s get coding!\x22"
bash: !\x22: event not found
One more alternative: this allows mixed expansion and non-expansion all within the same bash string
Here is another demo of an alternative escaping technique.
First, read the main answer by #liori to see how the 2nd form below works. Now, read these two alternative ways of escaping characters. Both examples below are identical in their output:
CMD="gs_set_title"
# 1. 1st technique: escape the $ symbol with a backslash (\) so it doesn't
# run and expand the command following it
echo "$CMD '\$(basename \"\$(pwd)\")'"
# 2. 2nd technique (does the same thing in a different way): escape the
# $ symbol using single quotes around it, and the single quote (') symbol
# using double quotes around it
echo "$CMD ""'"'$(basename "$(pwd)")'"'"
Sample output:
gs_set_title '$(basename "$(pwd)")'
gs_set_title '$(basename "$(pwd)")'
Note: for my gs_set_title bash function, which I have in my ~/.bash_aliases file somewhere around here, see my other answer here.
References:
https://en.wikipedia.org/wiki/ASCII#Printable_characters
https://serverfault.com/questions/208265/what-is-bash-event-not-found/208266#208266
See also my other answer here: How do I write non-ASCII characters using echo?.
I just use shell codes.. e.g. \x27 or \\x22 as applicable. No hassle, ever really.
Since one cannot put single quotes within single quoted strings, the simplest and most readable option is to use a HEREDOC string
command=$(cat <<'COMMAND'
urxvt -fg '#111111' -bg '#111111'
COMMAND
)
alias rxvt=$command
In the code above, the HEREDOC is sent to the cat command and the output of that is assigned to a variable via the command substitution notation $(..)
Putting a single quote around the HEREDOC is needed since it is within a $()
A minimal answer is needed so that people can get going without spending a lot of time as I had to sifting through people waxing eloquent.
There is no way to escape single quotes or anything else within single quotes.
The following is, perhaps surprisingly, a complete command:
$ echo '\'
whose output is:
\
Backslashes, surprisingly to even long-time users of bash, have no meaning inside single quotes. Nor does anything else.
IMHO the real answer is that you can't escape single-quotes within single-quoted strings.
Its impossible.
If we presume we are using bash.
From bash manual...
Enclosing characters in single quotes preserves the literal value of each
character within the quotes. A single quote may not occur
between single quotes, even when preceded by a backslash.
You need to use one of the other string escape mechanisms " or \
There is nothing magic about alias that demands it use single quotes.
Both the following work in bash.
alias rxvt="urxvt -fg '#111111' -bg '#111111'"
alias rxvt=urxvt\ -fg\ \'#111111\'\ -bg\ \'#111111\'
The latter is using \ to escape the space character.
There is also nothing magic about #111111 that requires single quotes.
The following options achieves the same result the other two options, in that the rxvt alias works as expected.
alias rxvt='urxvt -fg "#111111" -bg "#111111"'
alias rxvt="urxvt -fg \"#111111\" -bg \"#111111\""
You can also escape the troublesome # directly
alias rxvt="urxvt -fg \#111111 -bg \#111111"
Most of these answers hit on the specific case you're asking about. There is a general approach that a friend and I have developed that allows for arbitrary quoting in case you need to quote bash commands through multiple layers of shell expansion, e.g., through ssh, su -c, bash -c, etc. There is one core primitive you need, here in native bash:
quote_args() {
local sq="'"
local dq='"'
local space=""
local arg
for arg; do
echo -n "$space'${arg//$sq/$sq$dq$sq$dq$sq}'"
space=" "
done
}
This does exactly what it says: it shell-quotes each argument individually (after bash expansion, of course):
$ quote_args foo bar
'foo' 'bar'
$ quote_args arg1 'arg2 arg2a' arg3
'arg1' 'arg2 arg2a' 'arg3'
$ quote_args dq'"'
'dq"'
$ quote_args dq'"' sq"'"
'dq"' 'sq'"'"''
$ quote_args "*"
'*'
$ quote_args /b*
'/bin' '/boot'
It does the obvious thing for one layer of expansion:
$ bash -c "$(quote_args echo a'"'b"'"c arg2)"
a"b'c arg2
(Note that the double quotes around $(quote_args ...) are necessary to make the result into a single argument to bash -c.) And it can be used more generally to quote properly through multiple layers of expansion:
$ bash -c "$(quote_args bash -c "$(quote_args echo a'"'b"'"c arg2)")"
a"b'c arg2
The above example:
shell-quotes each argument to the inner quote_args individually and then combines the resulting output into a single argument with the inner double quotes.
shell-quotes bash, -c, and the already once-quoted result from step 1, and then combines the result into a single argument with the outer double quotes.
sends that mess as the argument to the outer bash -c.
That's the idea in a nutshell. You can do some pretty complicated stuff with this, but you have to be careful about order of evaluation and about which substrings are quoted. For instance, the following do the wrong things (for some definition of "wrong"):
$ (cd /tmp; bash -c "$(quote_args cd /; pwd 1>&2)")
/tmp
$ (cd /tmp; bash -c "$(quote_args cd /; [ -e *sbin ] && echo success 1>&2 || echo failure 1>&2)")
failure
In the first example, bash immediately expands quote_args cd /; pwd 1>&2 into two separate commands, quote_args cd / and pwd 1>&2, so the CWD is still /tmp when the pwd command is executed. The second example illustrates a similar problem for globbing. Indeed, the same basic problem occurs with all bash expansions. The problem here is that a command substitution isn't a function call: it's literally evaluating one bash script and using its output as part of another bash script.
If you try to simply escape the shell operators, you'll fail because the resulting string passed to bash -c is just a sequence of individually-quoted strings that aren't then interpreted as operators, which is easy to see if you echo the string that would have been passed to bash:
$ (cd /tmp; echo "$(quote_args cd /\; pwd 1\>\&2)")
'cd' '/;' 'pwd' '1>&2'
$ (cd /tmp; echo "$(quote_args cd /\; \[ -e \*sbin \] \&\& echo success 1\>\&2 \|\| echo failure 1\>\&2)")
'cd' '/;' '[' '-e' '*sbin' ']' '&&' 'echo' 'success' '1>&2' '||' 'echo' 'failure' '1>&2'
The problem here is that you're over-quoting. What you need is for the operators to be unquoted as input to the enclosing bash -c, which means they need to be outside the $(quote_args ...) command substitution.
Consequently, what you need to do in the most general sense is to shell-quote each word of the command not intended to be expanded at the time of command substitution separately, and not apply any extra quoting to the shell operators:
$ (cd /tmp; echo "$(quote_args cd /); $(quote_args pwd) 1>&2")
'cd' '/'; 'pwd' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")
/
$ (cd /tmp; echo "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
'cd' '/'; [ -e *'sbin' ] && 'echo' 'success' 1>&2 || 'echo' 'failure' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
success
Once you've done this, the entire string is fair game for further quoting to arbitrary levels of evaluation:
$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")"
/
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")"
/
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")")"
/
$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")"
success
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *sbin ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")"
success
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")")"
success
etc.
These examples may seem overwrought given that words like success, sbin, and pwd don't need to be shell-quoted, but the key point to remember when writing a script taking arbitrary input is that you want to quote everything you're not absolutely sure doesn't need quoting, because you never know when a user will throw in a Robert'; rm -rf /.
To better understand what is going on under the covers, you can play around with two small helper functions:
debug_args() {
for (( I=1; $I <= $#; I++ )); do
echo -n "$I:<${!I}> " 1>&2
done
echo 1>&2
}
debug_args_and_run() {
debug_args "$#"
"$#"
}
that will enumerate each argument to a command before executing it:
$ debug_args_and_run echo a'"'b"'"c arg2
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)"
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'>
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''>
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'>
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'bash'"'"' '"'"'-c'"'"' '"'"''"'"'"'"'"'"'"'"'debug_args_and_run'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'echo'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'a"b'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'c'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'arg2'"'"'"'"'"'"'"'"''"'"''>
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''>
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'>
1:<echo> 2:<a"b'c> 3:<arg2>
a"b'c arg2
In the given example, simply used double quotes instead of single quotes as outer escape mechanism:
alias rxvt="urxvt -fg '#111111' -bg '#111111'"
This approach is suited for many cases where you just want to pass a fixed string to a command: Just check how the shell will interpret the double-quoted string through an echo, and escape characters with backslash if necessary.
In the example, you'd see that double quotes are sufficient to protect the string:
$ echo "urxvt -fg '#111111' -bg '#111111'"
urxvt -fg '#111111' -bg '#111111'
Here is an elaboration on The One True Answer referenced above:
Sometimes I will be downloading using rsync over ssh and have to escape a filename with a ' in it TWICE! (OMG!) Once for bash and once for ssh. The same principle of alternating quotation delimiters is at work here.
For example, let's say we want to get: Louis Theroux's LA Stories ...
First you enclose Louis Theroux in single quotes for bash and double quotes for ssh:
'"Louis Theroux"'
Then you use single quotes to escape a double quote '"'
The use double quotes to escape the apostrophe "'"
Then repeat #2, using single quotes to escape a double quote '"'
Then enclose LA Stories in single quotes for bash and double quotes for ssh: '"LA Stories"'
And behold! You wind up with this:
rsync -ave ssh '"Louis Theroux"''"'"'"'"''"s LA Stories"'
which is an awful lot of work for one little ' -- but there you go
Obviously, it would be easier simply to surround with double quotes, but where's the challenge in that? Here is the answer using only single quotes. I'm using a variable instead of alias so that's it's easier to print for proof, but it's the same as using alias.
$ rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'
$ echo $rxvt
urxvt -fg '#111111' -bg '#111111'
Explanation
The key is that you can close the single quote and re-open it as many times as you want. For example foo='a''b' is the same as foo='ab'. So you can close the single quote, throw in a literal single quote \', then reopen the next single quote.
Breakdown diagram
This diagram makes it clear by using brackets to show where the single quotes are opened and closed. Quotes are not "nested" like parentheses can be. You can also pay attention to the color highlighting, which is correctly applied. The quoted strings are maroon, whereas the \' is black.
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\' # original
[^^^^^^^^^^] ^[^^^^^^^] ^[^^^^^] ^[^^^^^^^] ^ # show open/close quotes
urxvt -fg ' #111111 ' -bg ' #111111 ' # literal characters remaining
(This is essentially the same answer as Adrian's, but I feel this explains it better. Also his answer has 2 superfluous single quotes at the end.)
in addition to #JasonWoof perfect answer i want to show how i solved related problem
in my case encoding single quotes with '\'' will not always be sufficient, for example if a string must quoted with single quotes, but the total count of quotes results in odd amount
#!/bin/bash
# no closing quote
string='alecxs\'solution'
# this works for string
string="alecxs'solution"
string=alecxs\'solution
string='alecxs'\''solution'
let's assume string is a file name and we need to save quoted file names in a list (like stat -c%N ./* > list)
echo "'$string'" > "$string"
cat "$string"
but processing this list will fail (depending on how many quotes the string does contain in total)
while read file
do
ls -l "$file"
eval ls -l "$file"
done < "$string"
workaround: encode quotes with string manipulation
string="${string//$'\047'/\'\$\'\\\\047\'\'}"
# result
echo "$string"
now it works because quotes are always balanced
echo "'$string'" > list
while read file
do
ls -l "$file"
eval ls -l "$file"
done < list
Hope this helps when facing similar problem
Another way to fix the problem of too many layers of nested quotation:
You are trying to cram too much into too tiny a space, so use a bash function.
The problem is you are trying to have too many levels of nesting, and the basic alias technology is not powerful enough to accommodate. Use a bash function like this to make it so the single, double quotes back ticks and passed in parameters are all handled normally as we would expect:
lets_do_some_stuff() {
tmp=$1 #keep a passed in parameter.
run_your_program $# #use all your passed parameters.
echo -e '\n-------------' #use your single quotes.
echo `date` #use your back ticks.
echo -e "\n-------------" #use your double quotes.
}
alias foobarbaz=lets_do_some_stuff
Then you can use your $1 and $2 variables and single, double quotes and back ticks without worrying about the alias function wrecking their integrity.
This program prints:
el#defiant ~/code $ foobarbaz alien Dyson ring detected #grid 10385
alien Dyson ring detected #grid 10385
-------------
Mon Oct 26 20:30:14 EDT 2015
-------------
shell_escape () {
printf '%s' "'${1//\'/\'\\\'\'}'"
}
Implementation explanation:
double quotes so we can easily output wrapping single quotes and use the ${...} syntax
bash's search and replace looks like: ${varname//search/replacement}
we're replacing ' with '\''
'\'' encodes a single ' like so:
' ends the single quoting
\' encodes a ' (the backslash is needed because we're not inside quotes)
' starts up single quoting again
bash automatically concatenates strings with no white space between
there's a \ before every \ and ' because that's the escaping rules for ${...//.../...} .
string="That's "'##$*&^`(##'
echo "original: $string"
echo "encoded: $(shell_escape "$string")"
echo "expanded: $(bash -c "echo $(shell_escape "$string")")"
P.S. Always encode to single quoted strings because they are way simpler than double quoted strings.
Here are my two cents -- in the case if one wants to be sh-portable, not just bash-specific ( the solution is not too efficient, though, as it starts an external program -- sed ):
put this in quote.sh ( or just quote ) somewhere on your PATH :
# this works with standard input (stdin)
quote() {
echo -n "'" ;
sed 's/\(['"'"']['"'"']*\)/'"'"'"\1"'"'"'/g' ;
echo -n "'"
}
case "$1" in
-) quote ;;
*) echo "usage: cat ... | quote - # single-quotes input for Bourne shell" 2>&1 ;;
esac
An example:
$ echo -n "G'day, mate!" | ./quote.sh -
'G'"'"'day, mate!'
And, of course, that converts back:
$ echo 'G'"'"'day, mate!'
G'day, mate!
Explanation: basically we have to enclose the input with quotes ', and then also replace any single quote within with this micro-monster: '"'"' ( end the opening quote with a pairing ', escape the found single quote by wrapping it with double quotes -- "'", and then finally issue a new opening single quote ', or in pseudo-notation : ' + "'" + ' == '"'"' )
One standard way to do that would be to use sed with the following substitution command:
s/\(['][']*\)/'"\1"'/g
One small problem, though, is that in order to use that in shell one needs to escape all these single quote characters in the sed expression itself -- what leads to something like
sed 's/\(['"'"']['"'"']*\)/'"'"'"\1"'"'"'/g'
( and one good way to build this result is to feed the original expression s/\(['][']*\)/'"\1"'/g to Kyle Rose' or George V. Reilly's scripts ).
Finally, it kind of makes sense to expect the input to come from stdin -- since passing it through command-line arguments could be already too much trouble.
( Oh, and may be we want to add a small help message so that the script does not hang when someone just runs it as ./quote.sh --help wondering what it does. )
If you're generating the shell string within Python 2 or Python 3, the following may help to quote the arguments:
#!/usr/bin/env python
from __future__ import print_function
try: # py3
from shlex import quote as shlex_quote
except ImportError: # py2
from pipes import quote as shlex_quote
s = """foo ain't "bad" so there!"""
print(s)
print(" ".join([shlex_quote(t) for t in s.split()]))
This will output:
foo ain't "bad" so there!
foo 'ain'"'"'t' '"bad"' so 'there!'
If you have GNU Parallel installed you can use its internal quoting:
$ parallel --shellquote
L's 12" record
<Ctrl-D>
'L'"'"'s 12" record'
$ echo 'L'"'"'s 12" record'
L's 12" record
From version 20190222 you can even --shellquote multiple times:
$ parallel --shellquote --shellquote --shellquote
L's 12" record
<Ctrl-D>
'"'"'"'"'"'"'L'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'s 12" record'"'"'"'"'"'"'
$ eval eval echo '"'"'"'"'"'"'L'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'s 12" record'"'"'"'"'"'"'
L's 12" record
It will quote the string in all supported shells (not only bash).
This function:
quote ()
{
local quoted=${1//\'/\'\\\'\'};
printf "'%s'" "$quoted"
}
allows quoting of ' inside '. Use as this:
$ quote "urxvt -fg '#111111' -bg '#111111'"
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
If the line to quote gets more complex, like double quotes mixed with single quotes, it may become quite tricky to get the string to quote inside a variable. When such cases show up, write the exact line that you need to quote inside an script (similar to this).
#!/bin/bash
quote ()
{
local quoted=${1//\'/\'\\\'\'};
printf "'%s'" "$quoted"
}
while read line; do
quote "$line"
done <<-\_lines_to_quote_
urxvt -fg '#111111' -bg '#111111'
Louis Theroux's LA Stories
'single quote phrase' "double quote phrase"
_lines_to_quote_
Will output:
'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''
'Louis Theroux'\''s LA Stories'
''\''single quote phrase'\'' "double quote phrase"'
All correctly quoted strings inside single quotes.
Here is another solution. This function will take a single argument and appropriately quote it using the single-quote character, just as the voted answer above explains:
single_quote() {
local quoted="'"
local i=0
while [ $i -lt ${#1} ]; do
local ch="${1:i:1}"
if [[ "$ch" != "'" ]]; then
quoted="$quoted$ch"
else
local single_quotes="'"
local j=1
while [ $j -lt ${#1} ] && [[ "${1:i+j:1}" == "'" ]]; do
single_quotes="$single_quotes'"
((j++))
done
quoted="$quoted'\"$single_quotes\"'"
((i+=j-1))
fi
((i++))
done
echo "$quoted'"
}
So, you can use it this way:
single_quote "1 2 '3'"
'1 2 '"'"'3'"'"''
x="this text is quoted: 'hello'"
eval "echo $(single_quote "$x")"
this text is quoted: 'hello'
Going over the POSIX standard, I came across another rather technical/pointless question. It states:
Within the backquoted style of command substitution, <backslash> shall retain its literal meaning, except when followed by: '$' , '`' , or <backslash>.
It's easy to see why '`' and '\' lose their literal meanings: nested command substitution demands a "different" backquote inside the command substitution, which in turn forces '\' to lose its literal meaning. So, for instance, the following different behavior seems reasonable:
$ echo $(echo \\\\)
\\
$ echo `echo \\\\`
\
But what about '$'? I.e., what's the point or, more concretely, a possible benefit of the following difference?
$ echo $(echo \$\$)
$$
$ echo `echo \$\$`
4735
As '$' by itself is not ruled out inside backquotes, it looks like you would use either '$' or '\\\$' all the time, but never the middle case '\$'.
To recap,
$ echo `echo $$` # PID, OK
4735
$ echo `echo \\\$\\\$` # literal "$$", OK
$$
$ echo `echo \$\$` # What's the point?
4735
PS: I know this question is rather technical... I myself go for the more modern $(...) substitution all the time, but I'm still curious.
By adding a \, you make the inner subshell expand it instead of the outer shell. A good example would be to actually force the starting of a new shell, like this:
$ echo $$
4988
$ echo `sh -c 'echo $$'`
4988
$ echo `sh -c 'echo \$\$'`
4990
$ echo `sh -c 'echo \\\$\\\$'`
$$
Basic Answer
Consider the following command, which finds the base directory where gcc was installed:
gcc_base=$(dirname $(dirname $(which gcc)))
With the $(...) notation, there is no problem with the parsing; it is trivial and is one of the primary reason why the notation is recommended. The equivalent command using back-ticks is:
gcc_base=`dirname \`dirname \\\`which gcc\\\`\``
When the shell first parses this command, it encounters the first backtick, and has to find the matching close backtick. That's when the quoted section comes into effect:
Within the backquoted style of command substitution, shall retain its literal meaning, except when followed by: '$' , '`' , or .
gcc_base=`dirname \`dirname \\\`which gcc\\\`\``
^ ^ ^ ^ ^ ^
1 2 3 4 5 6
backslash-backtick - special rule
backslash-backslash - special rule
backslash-backtick - special rule
backslash-backslash - special rule
backslash-backtick - special rule
backslash-backtick - special rule
So, the unescaped backtick at the end marks the end of the outermost backtick command. The sub-shell that processes that command sees:
dirname `dirname \`which gcc\``
The backslash-back escapes are given the special treatment again, and the sub-sub-shell sees:
dirname `which gcc`
The sub-sub-sub-shell gets to see which gcc and evaluates it (e.g. /usr/gcc/v4.6.1/bin/gcc).
The sub-sub-shell evaluates dirname /usr/gcc/v4.6.1/bin/gcc and produces /usr/gcc/v4.6.1/bin.
The sub-shell evaluates dirname /usr/gcc/v4.6.1/bin and produces /usr/gcc/v4.6.1.
The shell assigns /usr/gcc/v4.6.1 to gcc_base.
In this example, the backslashes were only followed by the special characters - backslash, backtick, dollar. A more complex example would have, for example, \" sequences in the command, and then the special rule would not apply; the \" would simply be copied through unchanged and passed to the relevant sub-shell(s).
Extraordinarily Complex Stuff
For example, suppose you had a command with a blank in its name (heaven forbid; and this shows why!) such as totally amazing (two blanks; it is a more stringent test than a single blank). Then you could write:
$ cmd="totally amazing"
$ echo "$cmd"
totally amazing
$ which "$cmd"
/Users/jleffler/bin/totally amazing
$ dirname $(which "$cmd")
usage: dirname path
$ # Oops!
$ dirname "$(which \"\$cmd\")"
"$cmd": not found
.
$ # Oops!
$ dirname "$(which \"$cmd\")"
"totally: not found
amazing": not found
.
$ dirname "$(eval which \"$cmd\")"
totally amazing: not found
.
$ dirname "$(eval which \"\$cmd\")"
/Users/jleffler/bin
$ # Ouch, but at least that worked!
$ # But how to extend that to the next level?
$ dirname "$(eval dirname \"\$\(eval which \\\"\\\$cmd\\\"\)\")"
/Users/jleffler
$
OK - well, that's the "easy" one! Do you need a better reason to avoid spaces in command names or path names? I've also demonstrated to my own satisfaction that it works correctly with pathnames that contain spaces.
So, can we compress the learning cycle for backticks? Yes...
$ cat x3.sh
cmd="totally amazing"
which "$cmd"
dirname "`which \"$cmd\"`"
dirname "`dirname \"\`which \\"\$cmd\\\"\`\"`"
$ sh -x x3.sh
+ cmd='totally amazing'
+ which 'totally amazing'
/Users/jleffler/bin/totally amazing
++ which 'totally amazing'
+ dirname '/Users/jleffler/bin/totally amazing'
/Users/jleffler/bin
+++ which 'totally amazing'
++ dirname '/Users/jleffler/bin/totally amazing'
+ dirname /Users/jleffler/bin
/Users/jleffler
$
That is still a ghastly, daunting, non-intuitive set of escape sequences. It's actually shorter than the version for $(...) notation, and doesn't use any eval commands (which always complicate things).
This probably has to do with the strange way the Bourne shell parses substitutions (the real Korn shell is slightly similar but most other shells do not exhibit the strange behaviour at all).
Basically, the Bourne shell's parser does not interpret substitutions ($ and `) inside double-quotes, or parameter substitution ($) anywhere. This is only done at expansion time. Also, in many cases unmatched quotes (single-quotes, double-quotes or backquotes) are not an error; the closing quote is assumed at the end.
One consequence is that if a parameter substitution with a word containing spaces like ${v+a b} occurs outside double-quotes, it is not parsed correctly and will cause an expansion error when executed. The space needs to be quoted. Other shells do not have this problem.
Another consequence is that double-quotes inside backquotes inside double-quotes do not work reliably. For example,
v=0; echo "`v=1; echo " $v "`echo b"
will print
1 echo b
in most shells (one command substitution), but
0 b
in the Bourne shell and the real Korn shell (ksh93) (two command substitutions).
(Ways to avoid the above issue are to assign the substitution to a variable first, so double-quotes are not necessary, or to use new-style command substitution.)
The real Korn shell (ksh93) attempts to preserve much of the strange Bourne shell behaviour but does parse substitutions at parse time. Thus, ${v+a b} is accepted but the above example has "strange" behaviour. A further strange thing is that something like
echo "`${v+pwd"
is accepted (the result is like with the missing closing brace). And where does the opening brace in the error message from
echo "`${v+pwd`"
come from?
The below session shows an obscure case where $ and \$ differ in a non-obvious way:
$ echo ${.sh.version}
Version JM 93u 2011-02-08
$ v=0; echo "`v=1; echo "${v+p q}"`echo b"
p qecho b
$ v=0; echo "`v=1; echo "\${v+p q}"`echo b"
p{ q}b
Basically, a backslash is an escape character. You put it before another character to represent something special. An 'n','t','$' and '\'are these special characters.
"\n" --> newline
"\t" --> tab (indent)
"\$" --> $ (because a $ before a word in shell denotes a variable)
"\\" --> \
The backslash before characters is only interpreted the above way when it is inside quotes.
If you want to find more info or other escape chars go here