This is probably a stupid question but i am trying to understand why version 1 of the code below works and version 2 doesn't:
version 1:
$ VAR=toto:fofo:bar
$ IFS=:
$ set $VAR
$ echo $1
toto
version 2:
$ IFS=:
$ set toto:fofo:bar
$ echo $1
toto fofo bar
I don't understand why in the first version ':' are interpreted as a separator but in the second one they are not interpreted at all as if they are only interpreted if they are the output of a variable substitution ?
You're right. Word splitting only applies to the result of unquoted parameter expansions and command substitutions. It does not affect shell parsing or grammar.
Here's man bash with emphasis:
IFS
The Internal Field Separator that is used for word splitting after expansion [...]
Related
I've encountered a strange problem after temporarily changing IFS for the purpose of array building:
$ echo "1 2 3" |while read myVar1 myVar2; do echo "myVar1: ${myVar1}"; echo "myVar2: ${myVar2}"; done
myVar1: 1
myVar2: 2 3
$ IFS=':' myPaths=( ${PATH} ) # this works: I have /home/morgwai/bin on ${myPaths[0]} , /usr/local/sbin on ${myPaths[1]} and so on
$ echo "1 2 3" |while read myVar1 myVar2; do echo "myVar1: ${myVar1}"; echo "myVar2: ${myVar2}"; done
myVar1: 1 2 3
myVar2:
$ echo $IFS
$ echo "1:2:3" |while read myVar1 myVar2; do echo "myVar1: ${myVar1}"; echo "myVar2: ${myVar2}"; done ;
myVar1: 1
myVar2: 2:3
Normally when I change IFS temporarily for any other command than array building (for example IFS=',' echo whatever) its value is changed only during the execution of that, however here it seems as if IFS got permanently changed to a colon (although echo $IFS doesn't show this, which is even more strange...).
Is this a bug or somehow an expected behavior that I don't understand?
I'm using bash version 4.4.18 if it matters...
Note: I know that I can build the same array using IFS=':' read -a myPaths <<< ${PATH} and then IFS gets reverted to the default value normally, but that's not the point: I'm trying to understand what actually happens in the example I gave above.
Thanks!
You're just setting variables, not setting a variable followed by executing a command (ie, the way you build array is a pure variable assignment, not a command, hence both assignments become permanent).
The issue with an IFS of : not showing up in echo $IFS is caused by shell parameter expansion and word splitting.
Consider:
$ IFS=:
$ echo $IFS
$ echo "$IFS"
:
When a parameter expansion is not quoted, it undergoes word splitting afterwords. From the manual:
The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.
and
The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words using these characters as field terminators. If IFS is unset, or its value is exactly <space><tab><newline>, the default, then sequences of <space>, <tab>, and <newline> at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words.
So when IFS is a colon, splitting a word consisting of just a colon results in a (single) empty word. Always quote your variables to prevent unexpected gotchas like this.
I got a problem with storing some lines in a bash shell variable.
There is a file that contains a string like this.
$ vim a.txt
------we-are-in-vim------
first line
second line
-------end-of-file-------
This file has two empty lines that placed on the end of it.
When I cat this file, I can see that a blank is printed!
$ cat a.txt
first line
second line
$
Well. now, I can easily imagine that I can put this in a bash variable.
Let's try!
$ VAR=`cat a.txt`
$ echo "${VAR}"
first line
second line
$
Ok. I did not rap the cat command's output with double quotation! ;)
$ VAR="`cat a.txt`"
$ echo "${VAR}"
first line
second line
$
Ok. let's try with printf built-in variable assignment feature!
$ printf -v VAR "`cat a.txt`"
$ echo "${VAR}"
first line
second line
$
....Ok let's try with the mapfile command!
$ mapfile < a.txt VAR
$ printf '%s' "${VAR[#]}"
first line
second line
$
The mapfile command worked, but this is exactly the same with cat!
$ VAR2=`printf '%s' "${VAR[#]}"`
$ echo "${VAR2}"
first line
second line
$
I have already tried 'changing IFS to nothing' in bash,
But the result is exactly the same!
How can I assign the string that has two empty lines on the end of it, to a bash variable?
The usual wisdom is to append a character and remove it after:
$ a="`cat a.txt; echo x`"; echo "${a%x}"
first line
second line
$
That's a work around for the POSIX (most shells) specified variable expansion.
POSIX require that trailing newlines should be removed.
removing sequences of one or more characters at the end of the substitution
The alternative, if reading from a file (instead of executing some command), is to do the reading directly with the shell. If the shell's read accepts the option -d (since bash 2.04):
$ IFS='' read -d '' VAR < a.txt
$ echo "$VAR"
Or read with the command readarray (aka mapfile, since bash 4.0 alpha) as you already found.
It is because that is how command-substitution in bash works!
See this man bash except under COMMAND SUBSTITUTION
Command substitution allows the output of a command to replace the command name. There are two forms:
$(command)
`command`
[..] Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command with any trailing newlines deleted. [..]
On my GNU bash, version 4.3.42(1)-release I am doing some tests to answer a question. The idea is to split a :-separated string and and each of its elements into an array.
For this, I try to set the IFS to : in the scope of the command, so that the split is automatic and IFS remains untouched:
$ myvar="a:b:c"
$ IFS=: d=($myvar)
$ printf "%s\n" ${d[#]}
a
b
c
And apparently IFS remains the same:
$ echo $IFS
# empty
The BASH reference says that:
If IFS is unset, the parameters are separated by spaces. If IFS is
null, the parameters are joined without intervening separators.
However, then I notice that the IFS is kind of broken, so that echo $myvar returns a b c instead of a:b:c.
Unsetting the value solves it:
$ unset IFS
$ echo $myvar
a:b:c
But I wonder: what is causing this? Isn't IFS=: command changing IFS just in the scope of the command being executed?
I see in Setting IFS for a single statement that this indeed works:
$ IFS=: eval 'd=($myvar)'
$ echo $myvar
a:b:c
But I don't understand why it does and IFS=: d=($myvar) does not.
I was going to comment on this when I saw you use it, but it didn't occur to me until just now what the problem was. The line
IFS=: d=($myvar)
doesn't temporarily set IFS; it simply sets two variables in the current shell. (Simple commands can be prefixed with local environment settings, but an assignment statement itself is not a simple command.)
When you write
echo $IFS
IFS expands to :, but because : is the first character of IFS, it is removed during word splitting. Using
echo "$IFS"
would show that IFS is still set to :.
IFS behaves fine with read on the same line:
myvar="a:b:c"
IFS=: read -ra d <<< "$myvar"
printf "%s\n" "${d[#]}"
a
b
c
Check value of IFS:
declare -p IFS
-bash: declare: IFS: not found
so clearly IFS has not been tampered in current shell.
Now check original input:
echo $myvar
a:b:c
or:
echo "$myvar"
a:b:c
In bash, you can set set a variable that is valid for a single statement only if that statement is not itself a variable assignment. For example:
$ foo=one bar=two
$ echo $foo
one
To make the second assignment part of a statement, you need ... some other statement. As you've noticed, eval works. In addition, read should work:
$ foo="one:two:three"
$ IFS=: read -r -a bar <<< "$foo"
$ declare -p bar
declare -a bar='([0]="one" [1]="two" [2]="three")'
I've been using bash for about 3 mounth.
I'm understanding the language step by step but I have a question.
The real significate of $ in bash is the same of C?
I mean the $ not $1, $0, $# etc etc.
Only the $.
The $ is used to perform parameter expansion. For a variable named foo, the expression $foo expands to the value of the variable.
$ foo=3
$ echo "foo"
foo
$ echo "$foo"
3
$ is also used as the default/generic prompt, but there it is simply used as a distinctive character; it has no actual meaning, and could be replaced without causing any change in functionality.
What does the triple-less-than-sign bash operator, <<<, mean, as inside the following code block?
LINE="7.6.5.4"
IFS=. read -a ARRAY <<< "$LINE"
echo "$IFS"
echo "${ARRAY[#]}"
Also, why does $IFS remain to be a space, not a period?
It redirects the string to stdin of the command.
Variables assigned directly before the command in this way only take effect for the command process; the shell remains untouched.
From man bash
Here Strings
A variant of here documents, the format is:
<<<word
The word is expanded and supplied to the command on its standard input.
The . on the IFS line is equivalent to source in bash.
Update: More from man bash (Thanks gsklee, sehe)
IFS The Internal Field Separator that is used for word splitting
after expansion and to split lines into words with the read
builtin command. The default value is "<space><tab><new‐line>".
yet more from man bash
The environment for any simple command or function may be augmented
temporarily by prefixing it with parameter assignments, as described
above in PARAMETERS. These assignment statements affect only the environment seen by that command.
The reason that IFS is not being set is that bash isn't seeing that as a separate command... you need to put a line feed or a semicolon after the command in order to terminate it:
$ cat /tmp/ifs.sh
LINE="7.6.5.4"
IFS='.' read -a ARRAY <<< "$LINE"
echo "$IFS"
echo "${ARRAY[#]}"
$ bash /tmp/ifs.sh
7 6 5 4
but
$ cat /tmp/ifs.sh
LINE="7.6.5.4"
IFS='.'; read -a ARRAY <<< "$LINE"
echo "$IFS"
echo "${ARRAY[#]}"
$ bash /tmp/ifs.sh
.
7 6 5 4
I'm not sure why doing it the first way wasn't a syntax error though.