How to understand quoting variables deeply [duplicate] - bash

This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 6 years ago.
I test the following bash file test.sh. But how to explain the output. Sometimes I would remember quoting string variables means preserving whitespace in a single variable, but how to understand it instead of remembering it.
list="one two three"
for a in $list
do
echo "$a"
done
for a in "$list"
do
echo "$a"
done
output:
one
two
three
one two three

Continuing from the comment and explanation of word-splitting and the Internal Field Separator, here is a short example that should help:
#!/bin/bash
list="one two three"
printf "\nDefault IFS (space, tab, newline):\n\n"
for a in $list
do
echo "$a"
done
for a in "$list"
do
echo "$a"
done
printf "\nIFS breaking only on newline:\n\n"
IFS=$'\n'
for a in $list
do
echo "$a"
done
for a in "$list"
do
echo "$a"
done
(if you continue in a script and don't exit after setting IFS to a new value and need to restore the current (or default) IFS, either save the current (e.g. curifs="$IFS" and restore when done with your block, IFS="$curifs") or just reset to the default (e.g. IFS=$' \t\n'). you can also, just run your block with the new IFS in a subshell, e.g. (IFS=$',\n'; #do stuff), or in a while block, e.g. while IFS=$'\n' read -r line; do ... )
Example Use/Output
$ bash ifs.sh
Default IFS (space, tab, newline):
one
two
three
one two three
IFS breaking only on newline:
one two three
one two three

Related

Print with space [duplicate]

This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 1 year ago.
How to save a line with spaces without line breaks in a file?
Such as this:
Value2="Server=server1.windows.net,1433;Database=DB;Persist Security Info=False;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;"
using:
printf '%s\n' $Value2 >> "./print.csv"
At present it gets saved as:
Server=server1.windows.net,1433;Database=DB;Persist
Security
Info=False;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection
Timeout=30;
The variable $Value2 isn't quoted, and has two separate spaces, so the printf command sees three strings, not one -- so printf prints three strings on three lines. To make printf see just one string, put the variable in quotes, like this:
printf '%s\n' "${Value2}"
Output:
Server=server1.windows.net,1433;Database=DB;Persist Security Info=False;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;
Or, as proof use wc to count the lines:
printf '%s\n' "${Value2}" | wc -l
Output:
1

bash: IFS "stuck" after temporarily changing it for array building

I've encountered a strange problem after temporarily changing IFS for the purpose of array building:
$ echo "1 2 3" |while read myVar1 myVar2; do echo "myVar1: ${myVar1}"; echo "myVar2: ${myVar2}"; done
myVar1: 1
myVar2: 2 3
$ IFS=':' myPaths=( ${PATH} ) # this works: I have /home/morgwai/bin on ${myPaths[0]} , /usr/local/sbin on ${myPaths[1]} and so on
$ echo "1 2 3" |while read myVar1 myVar2; do echo "myVar1: ${myVar1}"; echo "myVar2: ${myVar2}"; done
myVar1: 1 2 3
myVar2:
$ echo $IFS
$ echo "1:2:3" |while read myVar1 myVar2; do echo "myVar1: ${myVar1}"; echo "myVar2: ${myVar2}"; done ;
myVar1: 1
myVar2: 2:3
Normally when I change IFS temporarily for any other command than array building (for example IFS=',' echo whatever) its value is changed only during the execution of that, however here it seems as if IFS got permanently changed to a colon (although echo $IFS doesn't show this, which is even more strange...).
Is this a bug or somehow an expected behavior that I don't understand?
I'm using bash version 4.4.18 if it matters...
Note: I know that I can build the same array using IFS=':' read -a myPaths <<< ${PATH} and then IFS gets reverted to the default value normally, but that's not the point: I'm trying to understand what actually happens in the example I gave above.
Thanks!
You're just setting variables, not setting a variable followed by executing a command (ie, the way you build array is a pure variable assignment, not a command, hence both assignments become permanent).
The issue with an IFS of : not showing up in echo $IFS is caused by shell parameter expansion and word splitting.
Consider:
$ IFS=:
$ echo $IFS
$ echo "$IFS"
:
When a parameter expansion is not quoted, it undergoes word splitting afterwords. From the manual:
The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.
and
The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words using these characters as field terminators. If IFS is unset, or its value is exactly <space><tab><newline>, the default, then sequences of <space>, <tab>, and <newline> at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words.
So when IFS is a colon, splitting a word consisting of just a colon results in a (single) empty word. Always quote your variables to prevent unexpected gotchas like this.

why is [ for i in ${ARR[#]}; do echo ${i} ] printing space-delimited output? [duplicate]

This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 5 years ago.
I have read input from file using:
declare -a ARR
readarray -t ARR < <(cat /etc/passwd | tr "\n" "\n" )
This prints it fine, but I want to access each line:
printf '%s\n' "${ARR[#]}
This is splitting the input on spaces:
for i in ${ARR[#]}; do
echo ${i}
done
Does echo requires a particular option to print it correctly?
Based on this answer, changing it to echo "${i}" should fix it, but it doesn't.
This has the same problem:
printf "${i} \n"
To avoid splitting on spaces, replace:
for i in ${ARR[#]}; do
With:
for i in "${ARR[#]}"; do
Without the double-quotes, the shell does word-splitting.
It would also be best to replace echo ${i} with echo "${i}". Without the double-quotes the ${i} will be subject to word-splitting (which will cause multiple spaces to be collapsed to one blank) and pathname expansion (which may cause words with shell-active characters to be replaced with filenames from the current directory).

bash - split string into array WITH empty values

pattern="::a::b::"
oldIFS=$IFS
IFS="::"
read -r -a extractees <<< $pattern
IFS=$oldIFS
this results in
{"a","b"}
however, I need to maintain the indices, so I want
{"","a","b",""}
(for comparison, if I wanted {"a","b"}, I would have written "a::b".
Why? because these elements are later split again (on a different delimiter) and the empty "" values should result in an empty list then.
How do I achieve this?
No field separator can be longer than 1 character, unfortunately, so '::' → ':'.
Aside of that, globbing should be explicitly turned off to prevent potential filename expansion in an unquoted variable.
set -f # disable globbing
pattern=":a:b c:"
oldIFS=$IFS
IFS=":"
extractees=($pattern)
IFS=$oldIFS
echo "'${extractees[0]}'"
echo "'${extractees[1]}'"
echo "'${extractees[2]}'"
echo "'${extractees[3]}'"

What does the Bash operator <<< (i.e. triple less than sign) mean?

What does the triple-less-than-sign bash operator, <<<, mean, as inside the following code block?
LINE="7.6.5.4"
IFS=. read -a ARRAY <<< "$LINE"
echo "$IFS"
echo "${ARRAY[#]}"
Also, why does $IFS remain to be a space, not a period?
It redirects the string to stdin of the command.
Variables assigned directly before the command in this way only take effect for the command process; the shell remains untouched.
From man bash
Here Strings
A variant of here documents, the format is:
<<<word
The word is expanded and supplied to the command on its standard input.
The . on the IFS line is equivalent to source in bash.
Update: More from man bash (Thanks gsklee, sehe)
IFS The Internal Field Separator that is used for word splitting
after expansion and to split lines into words with the read
builtin command. The default value is "<space><tab><new‐line>".
yet more from man bash
The environment for any simple command or function may be augmented
temporarily by prefixing it with parameter assignments, as described
above in PARAMETERS. These assignment statements affect only the environment seen by that command.
The reason that IFS is not being set is that bash isn't seeing that as a separate command... you need to put a line feed or a semicolon after the command in order to terminate it:
$ cat /tmp/ifs.sh
LINE="7.6.5.4"
IFS='.' read -a ARRAY <<< "$LINE"
echo "$IFS"
echo "${ARRAY[#]}"
$ bash /tmp/ifs.sh
7 6 5 4
but
$ cat /tmp/ifs.sh
LINE="7.6.5.4"
IFS='.'; read -a ARRAY <<< "$LINE"
echo "$IFS"
echo "${ARRAY[#]}"
$ bash /tmp/ifs.sh
.
7 6 5 4
I'm not sure why doing it the first way wasn't a syntax error though.

Resources