Looking forward to understand the behavior of read -d when it comes along with IFS variable
$ cat f1
a:b:c
d:e:f
$ while IFS= read -d: ; do echo $REPLY; done < f1
a
b
c d
e
$ while IFS=: read; do echo $REPLY; done < f1
a:b:c
d:e:f
IFS is used when you're reading several variables with read:
$ echo foo:bar:baz | (IFS=: read FOO BAR BAZ; echo $FOO; echo $BAR; echo $BAZ)
foo
bar
baz
Whereas, the -d option specifies what your line separator for read is; read won't read beyond a single line:
$ echo foo:bar:baz%baz:qux:quux% | while IFS=: read -d% FOO BAR BAZ; do echo ---; echo $FOO; echo $BAR; echo $BAZ; done
---
foo
bar
baz
---
baz
qux
quux
IFS is inter field separator.
You say to shell which symbols is used to split fields.
It is used in two directions, not only when reading.
One special case that you use here is IFS=. You use IFS= here to correctly handle input starting with spaces.
You can compare:
echo " a" | IFS= read a
echo " a" | read a
That is important when you handle files and they can contain leading spaces in their names.
Please compare:
$ echo " a" | ( IFS= read a; echo .$a. )
. a.
$ echo " a" | ( read a; echo .$a. )
.a.
UPDATE. As you probably already noted,
this construction
$ echo a | read a
doesn't work. Because shell creates a subshell for the '''read''', and you can see the value of $a only inside it.
You can also use while, what is more often:
$ echo a | while read a; do echo $a; done
Related
Here is my shell script.
printf "foo" | (read -r a; echo $a)
printf "bar" | while read -r a
do
echo $a
done
Here is the output.
$ sh foo.sh
foo
My question is: Why is foo printed but bar not printed? When read -r a is able to read a string that is not newline-terminated why is it unable to do so when it used as a condition of a while loop?
I'm trying to split key value pairs (around an = sign) which I then use to edit a config file, using bash. But I need an alternative to the <<< syntax for IFS.
The below works on my host system, but when i log in to my ubuntu virtual machine through ssh I have the wrong bash version. Whatever I try, <<< fails. (I am definitely calling the right version of bash at the top of the file, using #!/bin/bash (and I've tried #!/bin/sh etc too)).
I know I can use IFS as follows on my host mac os x system:
var="word=hello"
IFS='=' read -a array <<<"$var"
echo ${array[0]} ${array[1]]}
#alternative -for calling through e.g. sh file.sh param=value
for var in "$#"
do
IFS='=' read -a array <<<"$var"
echo ${array[0]} ${array[1]]}
done
#alternative
IFS='=' read -ra array <<< "a=b"
declare -p array
echo ${array[0]} ${array[1]}
But this doesn't work on my vm.
I also know that I can should be able to switch the <<< syntax through backticks, $() or echo "$var" | ... but I can't get it to work - as follows:
#Fails
IFS='=' read -ra myarray -d '' <"$var"
echo ${array[0]} ${array[1]]}
#Fails
echo "$var" | IFS='=' read -a array
echo ${array[0]} ${array[1]]}
#fails
echo "a=b" | IFS='=' read -a array
declare -p array
echo ${array[0]} ${array[1]}
Grateful for any pointers as I'm really new to bash.
Your first failed attempt is because < and <<< are different operators. < opens the named file.
The second fails because read only sets the value of array in the subshell started by the pipe; that shell exits after the completion of the pipe, and array disappears with it.
The third fails for the same reason as the second; the declare that follows doesn't make any difference.
Your attempts have been confounded because you have to use the variable in the same sub-shell as read.
$ echo 'foo=bar' | { IFS='=' read -a array; echo ${array[0]}; }
foo
And if you want your variable durable (ie, outside the sub-shell scope):
$ var=$(echo 'foo=bar' | { IFS='=' read -a array; echo ${array[0]}; })
$ echo $var
foo
Clearly, it isn't pretty.
Update: If -a is missing, that suggests you're out of the land of arrays. You can try parameter substitution:
str='foo=bar'
var=${str%=*}
val=${str#*=}
And if that doesn't work, fall back to good ole cut:
str='foo=bar'
var=$(echo $str | cut -f 1 -d =)
val=$(echo $str | cut -f 2 -d =)
It seems everybody else wants to remove any additional whitespace, however I have the opposite problem.
I have a file, call it some_file.txt that looks like
a b c d
and some more
and I'm reading it line-by-line with sed,
num_lines=$(cat some_file.txt | wc -l)
for i in $(seq 1 $num_lines); do
echo $(sed "${i}q;d" $file)
string=$(sed "${i}q;d" $file)
echo $string
done
I would expect the number of whitespace characters to stay the same, however the output I get is
a b c d
a b c d
and some more
and some more
So it seems that the problem is with sed removing the extra whitespace between chars, anyway to fix this?
Have a look at this example:
$ echo Hello World
Hello World
$ echo "Hello World"
Hello World
sed is not your problem, your problem is that bash removes the whitespaces when passing the output of sed into echo.
You just need to surround whatever echo is supposed to print with double quotation marks. So instead of
echo $(sed "${i}q;d" $file)
echo $string
You write
echo "$(sed "${i}q;d" $file)"
echo "$string"
The new script should look like this:
#!/usr/bin/env bash
file=some_file.txt
num_lines=$(cat some_file.txt | wc -l)
for i in $(seq 1 $num_lines); do
echo "$(sed "${i}q;d" $file)"
string=$(sed "${i}q;d" $file)
echo "$string"
done
prints the correct output:
a b c d
a b c d
and some more
and some more
However, if you just want to go through your file line by line, I strongly recommend something like this:
while IFS= read -r line; do
echo "$line"
done < some_file.txt
Question from the comments: What to do if you only want 33 lines starting from line x. One possible solution is this:
#!/usr/bin/env bash
declare -i s=$1
declare -i e=${s}+32
sed -n "${s},${e}p" $file | while IFS= read -r line; do
echo "$line"
done
(Note that I would probably include some validation of $1 in there as well.)
I declare s and e as integer variables, then even bash can do some simple arithmetic on them and calculate the actual last line to print.
I'm trying to use \n to output two seperate lines using echo and store that in a variable:
VAR=$( echo -e "foo\nbar" )
But the output I get is:
$ echo $VAR
foo bar
It works fine by itself:
$ echo -e "foo\nbar"
foo
bar
What am I doing wrong?
You need to wrap $VAR in double quotes:
echo "$VAR"
Otherwise, word splitting occurs. This means that "foo" and "bar" are treated as two separate arguments to echo and the newline between them is lost.
You can use set -x to enable debug mode and see what happens:
$ set -x
$ echo $VAR
+ echo foo bar
foo bar
$ echo "$VAR"
+ echo 'foo
bar'
foo
bar
I have this script:
#!/usr/bin/env bash
read -p "Provide arguments: " -a arr <<< "foo \"bar baz\" buz"
for i in ${arr[#]}
do
echo $i
done
which incorrectly outputs:
foo
"bar
baz"
buz
How can I make it interpret user input so that parameters within quotation marks would make a single array element? Like this:
foo
bar baz
buz
EDIT:
To be clear: I don't want the user to input each element in separate line, so read-ing in loop is not what I'm looking for.
You're better off supplying user input using a different delimiter, e.g. ;.
OLDIFS=$IFS
IFS=$';'
read -p "Provide arguments: " -a var
for i in "${!var[#]}"
do
echo Argument $i: "${var[i]}"
done
IFS=$OLDIFS
Upon execution:
Provide arguments: foo;bar baz;buz
Argument 0: foo
Argument 1: bar baz
Argument 2: buz
Plus a modification to trim the variables:
echo Argument $i: $(echo "${var[i]}" | sed -e 's/^ *//g' -e 's/ *$//g')