Is there a way to tell bash/zsh to not parse the quotes at all but give them to a shell function verbatim?
$ argtest abc def "ghi jkl" $'mno\tpqr' $'stu\nvwx'
abc
def
"ghi jkl"
$'mno\tpqr'
$'stu\nvwx'
You might be thinking why I don't just do
argtest abc def '"ghi jkl"' "$'mno\tpqr'" "$'stu\nvwx'"
But the argtest function I'm trying to create tries to wrap around other commands which can have noglob prefixes. So I need a way of being able to tell apart * and '*' .
In zsh, you can use the q parameter expansion flag, but it's messy. One q escapes individual characters as necessary; two expands the text in single quotes; three in double quotes. (The notation ${:-stuff} simply expands to the text following :-; it's a wrapper that allows you to create anonymous parameters.)
$ echo "foo bar"
foo bar
$ echo ${(qq):-"foo bar"}
'foo bar'
$ echo ${(qqq):-"foo bar"}
"foo bar"
$ argtest () {
function> echo "$1"
function> }
$ argtest "foo bar"
foo bar
$ argtest ${(qqq):-"foo bar"}
"foo bar"
No, there is no way to disable the shell's quote handling.
If you want this for your own convenience, reading arguments from a here document or similar might be acceptable.
But if you want your users to be able to write quotes at the shell prompt and have them preserved, there is no way to do that (short of writing your own shell).
In bash, you can access $BASH_COMMAND to see the literal, pre-parsing command being executed. Thus, while you can't prevent the shell from parsing an argument list, you can see its pre-parsed state.
However -- this gives you only the entire argument; you need to do string-splitting yourself, if going this route. As such, I would describe this as an ill-advised course of action.
Related
The question is basically, how does bash deal with double quotes and brackets?
I couldn't come up with a better example, so apologies about that, but imagine something like this:
"$(echo "${foo}")"
What happens there? Is ${foo} technically out of the quotation scope? Is bash smart enough to also consider that it is inside of the parentheses and work inside-out? Would \"${foo}\" work better?
I've tried Googling this, but no luck. Please note that this is different from just nested quoting questions, as it includes expansion.
Thanks!
Each command substitution establishes a new quoting context, so the correct way to avoid word splitting inside nested command substitutions is to use double quotes.
In this example, white space is preserved:
$ echo "$(echo "$(echo 'foo bar')")" # outer echo sees 'foo bar'
foo bar
However, missing any of the double quotes means that the string is split by the shell:
$ echo $(echo "$(echo 'foo bar')") # outer echo sees 'foo' 'bar'
foo bar
$ echo "$(echo $(echo 'foo bar'))" # middle echo sees 'foo' 'bar'
foo bar
echo outputs each argument, separated by a spaces, which is where the single space between "foo" and "bar" comes from.
The example you gave does save you from word splitting completely:
"$(echo "${foo}")"
while,
"$(echo ${foo})"
would save you from word splitting on the output but not on the variable $foo.
$() opens a subshell and you need to quote your vars in this subshell too. Consider the following example:
"$(echo "$(echo "$foo")")"
None of the pair of quotes above is optional, if you miss any of them, you will be subject to word splitting.
Consider another example:
"$foo var in same shell, $(cmd) - this output is also in the same shell"
In above, $foo and the output of cmd are safe, but if the cmd includes another variable inside $(), then that variable will need to be quoted.
What is foo? Is it a value? try
"$(echo "$\{foo\}")"
I know there is a duplicate for this question already at: How to trim whitespace from a Bash variable?.
I read all the answers there but I have a question about another solution in my mind and I want to know if this works.
This is the solution I think works.
a=$(printf "%s" $a)
Here is a demonstration.
$ a=" foo "
$ a=$(printf "%s" $a)
$ echo "$a"
foo
Is there any scenario in which this solution may fail?
If there is such a scenario in which this solution may fail, can we modify this solution to handle that scenario without compromising the simplicity of the solution too much?
If the variable a is set with something like "-e", "-n" in the begining, depending on how you process later your result, a user might crash your script:
-e option allows echo to interpret things backslashed.
Even in the case you only want to display the variable a, -n would screw your layout.
You could think about using regex to check if your variable starts with '-' and is followed by one of the available echo options (-n, -e, -E, --help, --version).
It fails when the input contains spaces between non-whitespace characters.
$ a=" foo bar "
$ a=$(printf "%s" $a)
$ echo "$a"
foobar
The expected output was the following instead.
foo bar
You could use Bash's builtin pattern substitution.
Note: Bash pattern substitution uses 'Pathname Expansion' (glob) pattern matching, not regular expressions. My solution requires enabling the optional shell behaviour extglob (shopt -s extglob).
$shopt -s extglob
$ a=" foo bar "
$ echo "Remove trailing spaces: '${a/%*([[:space:]])}'"
Remove trailing spaces: ' foo bar'
$ echo "Remove leading spaces: '${a/#*([[:space:]])}'"
Remove leading spaces: 'foo bar '
$ echo "Remove all spaces anywhere: '${a//[[:space:]]}'"
Remove all spaces anywhere: 'foobar'
For reference, refer to the 'Parameter Expansion' (Pattern substitution) and 'Pathname Expansion' subsections of the EXPANSION section of the Bash man page.
The following command throws an error, as expected, because a=foo happens to be one command and bar happens to be anohter due to the whitespace in between.
$ a=foo bar
-bash: bar: command not found
Now consider the following.
$ x="foo bar"
$ ls $x
ls: cannot access foo: No such file or directory
ls: cannot access bar: No such file or directory
The above output clearly shows that when we write $x without double-quotes, the value in $x is split into two arguments.
So I would expect the following command to throw an error as well.
$ a=$x
$ echo "$a"
foo bar
I was expecting that a=$x would be executed as if it were a=foo bar and it would throw bar: command not found error again. But you can see in the above output, that there was no error. It appears that a=$x was executed as if it were a="foo bar".
I don't understand the rules here. What rules of shell require that when a=$x is executed, the entire value in $x should be considered to be a single argument?
Quoting and parsing are subjects with many pitfalls. What you're seeing is because the command line string is tokenised (step 3) before expansions (step 5) happen.
The $variable on RHS of the assingnment will work without quoting in bash.
I'm not sure with other shells. Therefore, if you will use quoting, nothing can go wrong (mostly). So, simply use default the quoted "$variable" syntax, you will save for yourself many debugging hours.
Only remember, in the double quotes aren't expanded
the globbing characters, like *, ? and ~,
nor the brace expansion like echo "{a..b}{b..c}" prints {a..b}{b..c} instead of ab ac bb bc
Command substitutions, variable expansions works inside of double quotes.
Whitespace is used by the shell to separate words. Your first example
$ a=foo bar
consists of two words, a=foo and bar. The shell interprets this as a command bar preceded by an assignment which causes a to have the value foo in bar's environment.
The second example
x="foo bar"
ls $x
is a simple assignment to x. The quotes prevent the shell from interpreting this as the command bar with augmented environment x=foo as in the previous example. When $x is expanded without quotes, it presents two separate words foo and bar as arguments to the ls command. That is, parameter expansions undergo word-splitting in this situation.
The third example
a=$x
demonstrates a special case in bash. Parameter expansions on the right-hand of an assignment do not undergo word splitting, so this is equivalent to a="$x". a is assigned the exact value of $x.
From the bash man page under the heading Parameters (quoted is the 3.2 version, but applies to later versions as well)(emphasis mine):
A variable may be assigned to by a statement of the form
name=[value]
If value is not given, the variable is assigned the null string.
All values undergo tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, and quote
removal (see EXPANSION below).
You'll note that two actions, word-splitting and pathname expansion, are missing from the list of things applied to value.
I think below example will help you. I have set the debugger so we can see what is going on.
$ a='foo $100 bar'
a='foo $100 bar'
+ a='foo $100 bar'
$ x=$a
x=$a
+ x='foo $100 bar'
$ a="foo $100 bar"
a="foo $100 bar"
+ a='foo 00 bar'
$ x=$a
x=$a
+ x='foo 00 bar'
value of variable a is being assigned to X with in single quotes. So it will not give error.
Below is your example which explains in itself:
$ x="foo bar"
x="foo bar"
+ x='foo bar'
$ a=$x
a=$x
+ a='foo bar'
$ echo "$a"
echo "$a"
+ echo 'foo bar'
foo bar
I have a string:
{2013/05/01},{2013/05/02},{2013/05/03}
I want to append a { at the beginning and a } at the end. The output should be:
{{2013/05/01},{2013/05/02},{2013/05/03}}
However, in my shell script when I concatenate the curly braces to the beginning and end of the string, the output is as follows:
{2013/05/01} {2013/05/02} {2013/05/03}
Why does this happen? How can I achieve my result? Am sure there is a simple solution to this but I am a unix newbie, thus would appreciate some help.
Test script:
#!/usr/bin/ksh
valid_data_range="{2013/05/01},{2013/05/02},{2013/05/03}"
finalDates="{"$valid_data_range"}"
print $finalDates
The problem is that when you have a list in braces outside quotes, the shell performs Brace Expansion (bash manual, but ksh will be similar). Since the 'outside quotes' bit is important, it also tells you how to avoid the problem — enclose the string in quotes when printing:
#!/usr/bin/ksh
valid_data_range="{2013/05/01},{2013/05/02},{2013/05/03}"
finalDates="{$valid_data_range}"
print "$finalDates"
(The print command is specific to ksh and is not present in bash. The change in the assignment line is more cosmetic than functional.)
Also, the brace expansion would not occur in bash; it only occurs when the braces are written directly. This bilingual script (ksh and bash):
valid_data_range="{2013/05/01},{2013/05/02},{2013/05/03}"
finalDates="{$valid_data_range}"
printf "%s\n" "$finalDates"
printf "%s\n" $finalDates
produces:
ksh
{{2013/05/01},{2013/05/02},{2013/05/03}}
{2013/05/01}
{2013/05/02}
{2013/05/03}
bash (also zsh)
{{2013/05/01},{2013/05/02},{2013/05/03}}
{{2013/05/01},{2013/05/02},{2013/05/03}}
Thus, when you need to use the variable $finalDates, ensure it is inside double quotes:
other_command "$finalDates"
if [ "$finalDates" = "$otherString" ]
then : whatever
else : something
fi
Etc — using your preferred layout for whatever you don't like about mine.
You can say:
finalDates=$'{'"$valid_data_range"$'}'
The problem is that the shell is performing brace expansion. This allows you to generate a series of similar strings:
$ echo {a,b,c}
a b c
That's not very impressive, but consider
$ echo a{b,c,d}e
abc ace ade
In order to suppress brace expansion, you can use the set command to turn it off temporarily
$ set +B
$ echo a{b,c,d}e
a{b,c,d}e
$ set -B
$ echo a{b,c,d}e
abe ace ade
Let's say there is one script s1, and I need to pass argument $1 with value foo bar, with space in it. This can be done
./s1 "foo bar"
however, when I want to run the above command in another script (say s2), how should I put it? If I put it as above, foo bar will be interpreted as two arguments (to s1) rather than one.
You can try quoting $1:
./s2 "$1"
Use single quotes.
./script 'this is a line'
To consider variable substitutions use double quotes
./script "this is a line"
How about:
./s1 foo\ bar
Would that work?