The question is basically, how does bash deal with double quotes and brackets?
I couldn't come up with a better example, so apologies about that, but imagine something like this:
"$(echo "${foo}")"
What happens there? Is ${foo} technically out of the quotation scope? Is bash smart enough to also consider that it is inside of the parentheses and work inside-out? Would \"${foo}\" work better?
I've tried Googling this, but no luck. Please note that this is different from just nested quoting questions, as it includes expansion.
Thanks!
Each command substitution establishes a new quoting context, so the correct way to avoid word splitting inside nested command substitutions is to use double quotes.
In this example, white space is preserved:
$ echo "$(echo "$(echo 'foo bar')")" # outer echo sees 'foo bar'
foo bar
However, missing any of the double quotes means that the string is split by the shell:
$ echo $(echo "$(echo 'foo bar')") # outer echo sees 'foo' 'bar'
foo bar
$ echo "$(echo $(echo 'foo bar'))" # middle echo sees 'foo' 'bar'
foo bar
echo outputs each argument, separated by a spaces, which is where the single space between "foo" and "bar" comes from.
The example you gave does save you from word splitting completely:
"$(echo "${foo}")"
while,
"$(echo ${foo})"
would save you from word splitting on the output but not on the variable $foo.
$() opens a subshell and you need to quote your vars in this subshell too. Consider the following example:
"$(echo "$(echo "$foo")")"
None of the pair of quotes above is optional, if you miss any of them, you will be subject to word splitting.
Consider another example:
"$foo var in same shell, $(cmd) - this output is also in the same shell"
In above, $foo and the output of cmd are safe, but if the cmd includes another variable inside $(), then that variable will need to be quoted.
What is foo? Is it a value? try
"$(echo "$\{foo\}")"
Related
I know there is a duplicate for this question already at: How to trim whitespace from a Bash variable?.
I read all the answers there but I have a question about another solution in my mind and I want to know if this works.
This is the solution I think works.
a=$(printf "%s" $a)
Here is a demonstration.
$ a=" foo "
$ a=$(printf "%s" $a)
$ echo "$a"
foo
Is there any scenario in which this solution may fail?
If there is such a scenario in which this solution may fail, can we modify this solution to handle that scenario without compromising the simplicity of the solution too much?
If the variable a is set with something like "-e", "-n" in the begining, depending on how you process later your result, a user might crash your script:
-e option allows echo to interpret things backslashed.
Even in the case you only want to display the variable a, -n would screw your layout.
You could think about using regex to check if your variable starts with '-' and is followed by one of the available echo options (-n, -e, -E, --help, --version).
It fails when the input contains spaces between non-whitespace characters.
$ a=" foo bar "
$ a=$(printf "%s" $a)
$ echo "$a"
foobar
The expected output was the following instead.
foo bar
You could use Bash's builtin pattern substitution.
Note: Bash pattern substitution uses 'Pathname Expansion' (glob) pattern matching, not regular expressions. My solution requires enabling the optional shell behaviour extglob (shopt -s extglob).
$shopt -s extglob
$ a=" foo bar "
$ echo "Remove trailing spaces: '${a/%*([[:space:]])}'"
Remove trailing spaces: ' foo bar'
$ echo "Remove leading spaces: '${a/#*([[:space:]])}'"
Remove leading spaces: 'foo bar '
$ echo "Remove all spaces anywhere: '${a//[[:space:]]}'"
Remove all spaces anywhere: 'foobar'
For reference, refer to the 'Parameter Expansion' (Pattern substitution) and 'Pathname Expansion' subsections of the EXPANSION section of the Bash man page.
The following command throws an error, as expected, because a=foo happens to be one command and bar happens to be anohter due to the whitespace in between.
$ a=foo bar
-bash: bar: command not found
Now consider the following.
$ x="foo bar"
$ ls $x
ls: cannot access foo: No such file or directory
ls: cannot access bar: No such file or directory
The above output clearly shows that when we write $x without double-quotes, the value in $x is split into two arguments.
So I would expect the following command to throw an error as well.
$ a=$x
$ echo "$a"
foo bar
I was expecting that a=$x would be executed as if it were a=foo bar and it would throw bar: command not found error again. But you can see in the above output, that there was no error. It appears that a=$x was executed as if it were a="foo bar".
I don't understand the rules here. What rules of shell require that when a=$x is executed, the entire value in $x should be considered to be a single argument?
Quoting and parsing are subjects with many pitfalls. What you're seeing is because the command line string is tokenised (step 3) before expansions (step 5) happen.
The $variable on RHS of the assingnment will work without quoting in bash.
I'm not sure with other shells. Therefore, if you will use quoting, nothing can go wrong (mostly). So, simply use default the quoted "$variable" syntax, you will save for yourself many debugging hours.
Only remember, in the double quotes aren't expanded
the globbing characters, like *, ? and ~,
nor the brace expansion like echo "{a..b}{b..c}" prints {a..b}{b..c} instead of ab ac bb bc
Command substitutions, variable expansions works inside of double quotes.
Whitespace is used by the shell to separate words. Your first example
$ a=foo bar
consists of two words, a=foo and bar. The shell interprets this as a command bar preceded by an assignment which causes a to have the value foo in bar's environment.
The second example
x="foo bar"
ls $x
is a simple assignment to x. The quotes prevent the shell from interpreting this as the command bar with augmented environment x=foo as in the previous example. When $x is expanded without quotes, it presents two separate words foo and bar as arguments to the ls command. That is, parameter expansions undergo word-splitting in this situation.
The third example
a=$x
demonstrates a special case in bash. Parameter expansions on the right-hand of an assignment do not undergo word splitting, so this is equivalent to a="$x". a is assigned the exact value of $x.
From the bash man page under the heading Parameters (quoted is the 3.2 version, but applies to later versions as well)(emphasis mine):
A variable may be assigned to by a statement of the form
name=[value]
If value is not given, the variable is assigned the null string.
All values undergo tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, and quote
removal (see EXPANSION below).
You'll note that two actions, word-splitting and pathname expansion, are missing from the list of things applied to value.
I think below example will help you. I have set the debugger so we can see what is going on.
$ a='foo $100 bar'
a='foo $100 bar'
+ a='foo $100 bar'
$ x=$a
x=$a
+ x='foo $100 bar'
$ a="foo $100 bar"
a="foo $100 bar"
+ a='foo 00 bar'
$ x=$a
x=$a
+ x='foo 00 bar'
value of variable a is being assigned to X with in single quotes. So it will not give error.
Below is your example which explains in itself:
$ x="foo bar"
x="foo bar"
+ x='foo bar'
$ a=$x
a=$x
+ a='foo bar'
$ echo "$a"
echo "$a"
+ echo 'foo bar'
foo bar
Is there a way to tell bash/zsh to not parse the quotes at all but give them to a shell function verbatim?
$ argtest abc def "ghi jkl" $'mno\tpqr' $'stu\nvwx'
abc
def
"ghi jkl"
$'mno\tpqr'
$'stu\nvwx'
You might be thinking why I don't just do
argtest abc def '"ghi jkl"' "$'mno\tpqr'" "$'stu\nvwx'"
But the argtest function I'm trying to create tries to wrap around other commands which can have noglob prefixes. So I need a way of being able to tell apart * and '*' .
In zsh, you can use the q parameter expansion flag, but it's messy. One q escapes individual characters as necessary; two expands the text in single quotes; three in double quotes. (The notation ${:-stuff} simply expands to the text following :-; it's a wrapper that allows you to create anonymous parameters.)
$ echo "foo bar"
foo bar
$ echo ${(qq):-"foo bar"}
'foo bar'
$ echo ${(qqq):-"foo bar"}
"foo bar"
$ argtest () {
function> echo "$1"
function> }
$ argtest "foo bar"
foo bar
$ argtest ${(qqq):-"foo bar"}
"foo bar"
No, there is no way to disable the shell's quote handling.
If you want this for your own convenience, reading arguments from a here document or similar might be acceptable.
But if you want your users to be able to write quotes at the shell prompt and have them preserved, there is no way to do that (short of writing your own shell).
In bash, you can access $BASH_COMMAND to see the literal, pre-parsing command being executed. Thus, while you can't prevent the shell from parsing an argument list, you can see its pre-parsed state.
However -- this gives you only the entire argument; you need to do string-splitting yourself, if going this route. As such, I would describe this as an ill-advised course of action.
I have looked at this question, but it does not cover my use case.
Suppose I have the variable foo which holds the four-character literal \x60.
I want to perform ANSI C Quoting on the contents of this variable and store it into another variable bar.
I tried the following, but none of them achieved the desired effect.
bar=$'$foo'
echo $bar
bar=$"$foo"
echo $bar
Output:
$foo
\x61
Desired output (actual value of \x61):
a
How might I achieve this in the general case, including non-printable characters? Note that in this case a was used just as an example to make it easier to test whether the method worked.
By far the simplest solution, if you are using bash:
printf %b "$foo"
Or, to save it in another variable name bar:
printf -v bar %b "$foo"
From help printf:
In addition to the standard format specifications described in printf(1)
and printf(3), printf interprets:
%b expand backslash escape sequences in the corresponding argument
%q quote the argument in a way that can be reused as shell input
%(fmt)T output the date-time string resulting from using FMT as a format
string for strftime(3)
There are edge cases, though:
\c terminates output, backslashes in \', \", and \? are not removed,
and octal escapes beginning with \0 may contain up to four digits
The following works:
eval bar=\$\'$x\'
The command bar=$'\x61' has to be constructed first, then eval evaluates the newly built command.
I just found out that I can do this. Edited based on comments.
bar=$( echo -ne "$foo" )
The best method I know is
y=$(printf $(echo "$foo"|sed 's/%/%%/g'))
As mentioned in the comments, this trims trailing newlines from $foo. To overcome this:
moo=$(echo "${foo}:end"|sed 's/%/%%/g')
moo=$(printf "$moo")
moo=${moo%:end}
# the escaped string is in $moo
echo "+++${moo}---"
Since bash 4.4 there is a variable expansion to do exactly that:
$ foo='\x61'; echo "$foo" "${foo#E}"
\x61 a
To set another variable use:
$ printf -v bar "${foo#E}"; echo "$bar"
a
Sample of conversion via shell. Problem, the code is octal using \0nnn and hexdecimal (not on all shell) using \xnn (where n are [hexa]digit)
foo="\65"
print "$( echo "$foo" | sed 's/\\/&0/' )"
5
with awk, you could certainly convert it directly
I want to something like this in bash:
alias foo='bar="$(echo hello world | grep \"hello world\")"; echo $bar;'; foo
Expected output: hello world
Ouput: grep: world": No such file or directory
The outer quotes have to be single quotes, with double quotes $bar would be empty.
The next quotes have to be double quotes, with single quotes $() wouldn't expand.
The inner quotes could be both type of quotes, but single quotes doesn't allow single quotes inside of them.
How to I achieve this?
The stuff inside $() represents a subshell, so you are allowed to place un-escaped double quotes inside
alias foo='bar="$(echo testing hello world | grep "hello world")"; echo "$bar"'
It's a bit unclear what "something like this" means, but the simplest way to achieve what seems to be the point here is a simple function:
foo() {
echo 'hello world' | grep 'hello world'
}
foo
There's no need for an intermediate variable assignment (it will be lost anyway).
Functions are generally preferred over aliases because of more flexibility (parameter handling) and readability (multiple lines; less escaping).
Always use the simplest solution which could possibly work.
Escape the spaces
alias foo='bar="$(echo hello world | grep hello\ world)"; echo $bar;'
The double quotes around $() are not necessary:
alias foo='bar=$(echo hello world | grep "hello world"); echo $bar;'
foo
# Output:
hello world