In bash :
Using double quotes
echo "*" #Globbing is never done
echo "$variable" # Variable expansion is always done
echo "$(command)" # command expansion is always done
Using single quotes
echo '*' #Globbing is never done
echo '$variable' # Variable expansion is never done
echo '$(command)' # command expansion is never done
Using no quotes
echo * #Globbing always done.
echo $variable; # Variable expansion is always done
echo $(command) # command expansion is always done
will this apply for all commands?
It looks like you are looking for exceptions, and I'd guess you have some in mind. I'm going to make the assumption that set -f/set -o noglob are being excluded from this case?
When you use the dd command, globbing will not occure, even if unquoted.
$ ls *.txt
blah.txt file1.txt file2.txt file.txt logfile.txt
$ dd if=*.txt of=glob.txt
dd: failed to open ‘*.txt’: No such file or directory
Rebuttal and false positives
Here are some examples that are odd, but follow expansion
variable='2010-09-08 12:34:56'
echo "$variable" | xargs date +%s -d
date: extra operand '12:34:56'
The extra operand shows that variable expansion is happening, you are losing the quotes in the pipe.
$ date +%s -d 2010-09-08 12:34:56
date: extra operand ‘12:34:56’
This also happens if you create a script to echo $1 and then expand your quoted variable while passing. It expands, and works as expected. So, the issue is not with xargs, but with your expansion before the pipe which is normal.
Eval... evals whole purpose is to do expansion of its args prior to running a command. Expansion also happens with bash -c, except it takes one argument. So, again, this is not an expansion issue, but a command usage issue.
cmd='printf "%s\n" "$(date -d "$variable" +%c)"'
bash -c $cmd
works the same as the expanded version
$ bash -c printf "%s\n" "$(date -d "$variable" +%c)"
printf: usage: printf [-v var] format [arguments]
I really enjoyed Hauri's $'...' and $"..." information--however, those are not the samething we are talking about. They are in fact behaving as the bash man page says they should. $'' is as different from '' as (()) is from $(())
I got excited about this one, so...
$ ls
mclark.txt qt sign_in.txt skel.bash
$ zip m*t.zip *t
$ ls *.zip
m*t.zip
However, this isn't right either-- the splat expands, but upon no match zip uses it as a literal. I found a few commands that did this, but if there was a match (I added a my.zip later) it uses the matched expansion (an error was thrown, b/c my.zip was a text file for testing purposes).
There are multiple forces in place. In general you can assume that single quotes is to hide the contents from bash expansion. Double quotes is to group values which might have white space so that bash sees them as one logical unit but also disable globbing. There are many caveats though...
Enclosing characters in double quotes preserves the literal value of
all characters within the quotes, with the exception of $, ', \, and,
when history expansion is enabled, !. The characters $ and ' retain
their special meaning within double quotes. The backslash retains
its special meaning only when followed by one of the following
characters: $, ', ", \, or . A double quote may be quoted
within double quotes by preceding it with a backslash. If enabled,
history expansion will be performed unless an ! appearing in double
quotes is escaped using a backslash. The backslash pre-ceding the !
is not removed.
See section QUOTING from man bash
This example below, will either confuse you further or make it clearer.
$ echo "*test*"
*test*
$ echo '*test*'
*test*
$ msg=$(echo '*test*')
$ echo '$msg'
$msg
$ echo "$msg"
*test*
$ echo $msg
speedtest test1 test2 test3 test4 test5 testdir
note that if there were no matches it would print *test* not empty line as commented by Hastur.
some other interesting tidbits
note that this doesn't work
$ echo 'single quotes don\'t escape'
but this works
$ echo "\"double quotes\" escape"
but you can use one in other without escaping
$ echo '"' "'"
Short answer: Yes
This asumptions are basicaly true, alway!
variable='2010-09-08 12:34:56'
vname=variable
date -d "$variable" +%s
1283942096
date -d "${!vname}" +%s
1283942096
date -d $variable +%s
date: extra operand '+%s'
Try 'date --help' for more information.
date -d '$variable' +%s
date: invalid date '$variable'
date -d ${!vname} +%s
date: extra operand '+%s'
Try 'date --help' for more information.
But
Some commands like xargs work precisely about expansion and parameter distribution.
echo "$variable" | xargs date +%s -d
date: extra operand '12:34:56'
Try 'date --help' for more information.
You have to use -0 arg to xargs:
echo "$variable" | xargs -0 date +%s -d
1283942096
Builtin commands could use args differently, especialy eval:
cmd='printf "%s\n" $(date -d "$variable" +%c)'
eval $cmd
Wed
Sep
8
12:34:56
2010
cmd='printf "%s\n" "$(date -d "$variable" +%c)"'
eval "$cmd"
Wed Sep 8 12:34:56 2010
eval $cmd
Wed Sep 8 12:34:56 2010
bash -c "$cmd"
Mon May 16 00:00:00 2016
bash -c $cmd
printf: usage: printf [-v var] format [arguments]
Syntax of funny thing under bash are not limited to "..", '..', and ${}
$'...' let you print special characters, but don't expand variables:
echo $'This\tis\ta string containing ${variable}'
This is a string containing ${variable}
Backticks: For compatibility, backtick are always supported. If not very readable, you may see this in some scripts:
echo `date +%c -d "${!vname}"`
Wed Sep 8 12:34:56 2010
Syntaxe $"..." could be used for localization:
export TEXTDOMAIN=bash
export LANG=fr_CH.utf-8
echo $"Running"
En cours d'exécution
If nothing matches *.xtx while a.txt is a file
mv a.txt *.xtx
will get you an unexpected result too.
The same applies for other things like cp and even this treats it as quoted:
$ ls *.xtx
/bin/ls: cannot access *.xtx: No such file or directory
$ echo "A" > *.xtx
$ ls *.xtx
*.xtx
$
While they should all return error just like if there were more the one file you would get "ambiguous redirect".
will this apply for all commands?
Yes.
From Bash reference manuals:
3.1.2.2 Single Quotes
Enclosing characters in single quotes (') preserves the literal
value of each character within the quotes. A single quote may not
occur between single quotes, even when preceded by a backslash.
and
3.1.2.3 Double Quotes
Enclosing characters in double quotes (") preserves the literal
value of all characters within the quotes, with the exception of $,
`, \, and, when history expansion is enabled, !. The characters
$ and ` retain their special meaning within double quotes (see
Shell Expansions). The backslash retains its special meaning only
when followed by one of the following characters: $, `, ", \,
or newline. Within double quotes, backslashes that are followed by one
of these characters are removed. Backslashes preceding characters
without a special meaning are left unmodified. A double quote may be
quoted within double quotes by preceding it with a backslash. If
enabled, history expansion will be performed unless an ! appearing
in double quotes is escaped using a backslash. The backslash preceding
the ! is not removed.
The special parameters * and # have special meaning when in double
quotes (see Shell Parameter Expansion).
Probably the shell reference manuals and the shell man pages contain the intended behavior ... but the result might not always be what was originally intended.
Reading the "QUOTING" section of the man pages is also intresting. This is a section from the bash man page concerning the single and double quotes: (which is pretty much the same content as the reference manual)
Enclosing characters in single quotes preserves the literal value of
each character within the quotes. A single quote may not occur
between single quotes, even when preceded by a backslash.
Enclosing characters in double quotes preserves the literal value of
all characters within the quotes, with the exception of $, `, \, and,
when history expansion is enabled, !. The characters $ and `
retain their special meaning within double quotes. The backslash
retains its special meaning only when followed by one of the following
characters: $, `, ", \, or . A double quote may be
quoted within double quotes by preceding it with a backslash. If
enabled, history expan- sion will be performed unless an !
appearing in double quotes is escaped using a backslash. The
backslash preceding the ! is not removed.
Related
I am running these two commands in Git bash.
Why they behave differently? Aren't they supposed to do the same thing or am I missing something?
git diff > D:\Patches\afterWGComment.txt
creates file PatchesafterWGComment.txt in D:/
git diff > D:/Patches/afterWGComment.txt
correctly creates file afterWGComment.txt in D:/Patches/
Note that D:/Patches/ folder is present before running the above commands.
Bash treats backslash as an escape character, meaning that the symbol following it is interpreted literally, and the backslash itself is dropped.
$ echo $HOME
/home/user
$ echo \$HOME
$HOME
Under Windows, where backslash serves as a path separator, this causes some inconvenience. Fortunately, inside single quotes a backslash character loses its special meaning and is treated literally (as any other character, except a single quote):
$ echo '\$HOME'
\$HOME
Therefore, if you are going to copy&paste a Windows path into Git bash, put it inside single quotes:
git diff > 'D:\Patches\afterWGComment.txt'
Backslash is an escape character used to escape meta characters. This means you need to escape the escape:
D:\\Patches\\afterWGComment.txt
Alternative you can put your string in single quotes, which will make all characters literal:
'D\Patches\afterWGComment.txt'
Some meta characters: *, ~, $, !, ...
Well the Backslash (\) in Linux generally means a escape character. So in your case the backslash is escaping strings. Try with a cd "D:\Patches\afterWGComment.txt" and you can see the difference.
The back slash has a very long history in Unix (and therefore in Linux) of meanning: quote next character.
There are three ways to quote in the shell (where you type commands):
The backquote (\)
Single quotes (')
Double quotes (")
In the order from stronger to softer. For example, a $ is an special character in the shell, this will print the value of a variable:
$ a=Hello
$ echo $a
Hello
But this will not:
$ echo \$a
$a
$ echo '$a'
$a
$ echo "$a"
Hello
In most cases, a backslash will make the next character "not special", and usually will convert to the same character:
$ echo \a
a
Windows decided to use \ to mean as the same as the character / means in Unix file paths.
To write a path in any Unix like shell with backslashes, you need to quote them:
$ echo \\
\
$ echo '\'
\
$ echo "\\"
\
For the example you present, just quote the path:
$ echo "Hello" > D:\\Patches\\afterWGComment.txt
That will create the file afterWGComment.txt that contains the word Hello.
Or:
$ echo "Hello" > 'D:\Patches\afterWGComment.txt'
$ echo "Hello" > "D:\\Patches\\afterWGComment.txt"
$ echo "Hello" > "D:/Patches/afterWGComment.txt"
Quoting is not simple, it has adquired a long list of details since the 1660's.
echo some string when use single quotes it's printed successfully
➜ ~ echo 'LOGIN_IDENTITY=sdf!121sdf$78sd!8'
LOGIN_IDENTITY=sdf!121sdf$78sd!8
but when use double quotes, it cannot be printed successfully and changed to another strange commands
➜ ~ echo "LOGIN_IDENTITY=sdf!121sdf$78sd!8"
➜ ~ echo "LOGIN_IDENTITY=sdffind . -name 'application.properties' | xargs grep 'login'sdf$78sdawk '{print "\""$0"\""}' a"
So what's wrong with double quotes?
It is because of Parameter Expansion in shell, when you have an ! character within the double-quotes, it tried to expand it to have a value.
Following excerpt from man bash page, history-expansion subsection:
History expansions are introduced by the appearance of the history expansion
character, which is ‘!’ by default. Only ‘\’ and ‘'’ may be used to escape the > history expansion character, but the history expansion character is also
treated as quoted if it immediately precedes the closing double quote in a
double-quoted string.
You can avoid the expansion either by using single-quotes(')
$ echo "LOGIN_IDENTITY=sdf!121sdf$78sd!8"
-bash: !121: event not found
change the above assignment to
$ echo "LOGIN_IDENTITY=sdf"'!'"121sdf$78sd"'!'"8"
LOGIN_IDENTITY=sdf!121sdf8sd!8
Notice the single-quote around the ! character.
I'm almost certain the code I have here worked before. Here's a simplified version and what it produces:
a="atext"
b="btext"
var=$'${a}\n${b}\n'
printf "var=$var"
Which produces output:
var=${a}
${b}
The real code outputs var to file, but the variable expansions aren't happening for some reason.
If this can't work, can you suggest a nice alternative way, and why one uses $' '? Thanks.
GNU bash, version 4.3.42
$'' is a quoting type used to allow backslash escape sequences to describe literal strings with nonprintable characters and other such oddities. Thus, $'\n' evaluates to a single character -- a newline -- whereas '\n' and "\n" both evaluate to two characters, the first being a backslash and the second being an n.
If you want to have the exact behavior of your original code -- putting a literal newline between the results of two different expansions -- you can switch quote types partway through a string:
a="atext"
b="btext"
var="$a"$'\n'"$b"
printf '%s' "var=$var"
That is, right next to each other, with no spaces between:
"$a"
$'\n'
"$b"
This gives you $a and $b expanded, with a literal newline between them.
Why does this matter? Try the following:
$ a=atext
$ b=btext
$ var1="$a\n$b" # Assign with literal "\" and "n" characters
$ printf "$var1" # Here, printf changes the "\n" into the newline
atext
btext
$ printf '%s' "$var1" # ...but this form shows that the "\n" are really there
atext\nbtext
$ var2="$a"$'\n'"$b" # now, we put a single newline in the string
$ printf '%s' "$var2" # and now even accurate use of printf shows that newline
atext
btext
Just replace the single quotes with double quotes.
$ cat test
a="atext"
b="btext"
var=$"${a}\n${b}\n"
printf "var=$var"
$ sh test
var=atext
btext
For variable expansion you either need to use double quotes or no quotes. Single quotes negate expansion.
Should I double quote or escape with \ special characters like ',
$ echo "'"
'
$ echo \'
'
Here is apparently doesn't matter, but are there situations where there is a difference, except for $, `` or`, when I know there is a difference.
Thanks,
Eric J.
You can use either backslashes, single quotes, or (on occasion) double quotes.
Single quotes suppress the replacement of environment variables, and all special character expansions. However, a single quote character cannot be inside single quotes -- even when preceded by a backslash. You can include double quotes:
$ echo -e 'The variable is called "$FOO".'
The variable is called "$FOO".
Double quotes hide the glob expansion characters from the shell (* and ?), but it will interpolate shell variables. If you use echo -e or set shopt -s xpg_echo, the double quotes will allow the interpolation of backslash-escaped character sequences such as \a, and \t. To escape those, you have to backslash-escape the backslash:
$ echo -e "The \\t character sequence represents a tab character."
The \t character sequence represents a tab character."
The backslash character will prevent the expansion of special characters including double quotes and the $ sign:
$ echo -e "The variable is called \"\$FOO\"."
The variable is called "$FOO".
So, which one to choose? Which everyone looks the best. For example, in the preceding echo command, I would have been better off using single quotes and that way I wouldn't have the confusing array of backslashes one right after another.
On the other hand:
$ echo -e "The value of \$FOO is '$FOO'."
The value of FOO is 'bar'.
is probably better than trying something like this:
$ echo -e 'The value of $FOO is '"'$FOO'."
Readability should be the key.
From bash manpage
Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !.
With this in mind, how is it that echo -ne "\n" produces a newline? Wouldn't the shell expand "\n" before it ever gets passed to echo?
I thought it might work because echo is a builtin and so the shell is smart enough to do the right thing. However, even calling the external /usr/bin/echo -ne "\n" works.
What's even more curious is that regardless if I double-quote or single-quote \n, the following two commands show that bash is passing \\n as the argument:
$ strace /usr/bin/echo "\n" 2>&1 | head -n1
execve("/usr/bin/echo", ["/usr/bin/echo", "\\n"], [/* 33 vars */]) = 0
$ strace /usr/bin/echo '\n' 2>&1 | head -n1
execve("/usr/bin/echo", ["/usr/bin/echo", "\\n"], [/* 33 vars */]) = 0
What's going on here?
Read on:
The backslash retains its special meaning only when followed by one of the following characters: $, `, ", \, or <newline>.
Since it’s just followed by a letter, it does not retain its special meaning and is passed literally to the program.