When I try to use the read command in Bash like this:
echo hello | read str
echo $str
Nothing echoed, while I think str should contain the string hello. Can anybody please help me understand this behavior?
The read in your script command is fine. However, you execute it in the pipeline, which means it is in a subshell, therefore, the variables it reads to are not visible in the parent shell. You can either
move the rest of the script in the subshell, too:
echo hello | { read str
echo $str
}
or use command substitution to get the value of the variable out of the subshell
str=$(echo hello)
echo $str
or a slightly more complicated example (Grabbing the 2nd element of ls)
str=$(ls | { read a; read a; echo $a; })
echo $str
Other bash alternatives that do not involve a subshell:
read str <<END # here-doc
hello
END
read str <<< "hello" # here-string
read str < <(echo hello) # process substitution
Typical usage might look like:
i=0
echo -e "hello1\nhello2\nhello3" | while read str ; do
echo "$((++i)): $str"
done
and output
1: hello1
2: hello2
3: hello3
The value disappears since the read command is run in a separate subshell: Bash FAQ 24
To put my two cents here: on KSH, reading as is to a variable will work, because according to the IBM AIX documentation, KSH's read does affects the current shell environment:
The setting of shell variables by the read command affects the current shell execution environment.
This just resulted in me spending a good few minutes figuring out why a one-liner ending with read that I've used a zillion times before on AIX didn't work on Linux... it's because KSH does saves to the current environment and BASH doesn't!
I really only use read with "while" and a do loop:
echo "This is NOT a test." | while read -r a b c theRest; do
echo "$a" "$b" "$theRest"; done
This is a test.
For what it's worth, I have seen the recommendation to always use -r with the read command in bash.
You don't need echo to use read
read -p "Guess a Number" NUMBER
Another alternative altogether is to use the printf function.
printf -v str 'hello'
Moreover, this construct, combined with the use of single quotes where appropriate, helps to avoid the multi-escape problems of subshells and other forms of interpolative quoting.
Do you need the pipe?
echo -ne "$MENU"
read NUMBER
Related
running following command on zsh
zsh$ echo Hello | read str ; echo "str is: "$str
str is: Hello
whereas in bash, it doesn't work
bash$ echo Hello | read str ; echo "str is: "$str
str is:
This thread mentions read command runs in subshell so current session has no clue about it. I'm not able to find why it works in zsh.
The read command has the same behaviour in both shells, but bash runs the read in a subshell whereas zsh does not.
Some shells don't need to use a subshell for the read in your example (in general, the last command in a pipeline).
To avoid having to switch to another shell interpreter, you may have the read read from something other than a pipe:
read str <<<'hello'
printf 'str is %s\n' "$str"
Or, if all you want is to output the string, output it in the same subshell:
echo 'hello' | { read str && printf 'str is %s\n' "$str"; }
I'm new to UNIX and have this really simple problem:
I have a text-file (input.txt) containing a string in each line. It looks like this:
House
Monkey
Car
And inside my shell script I need to read this input file line by line to get to a variable like this:
things="House,Monkey,Car"
I know this sounds easy, but I just couldnt find any simple solution for this. My closest attempt so far:
#!/bin/sh
things=""
addToString() {
things="${things},$1"
}
while read line; do addToString $line ;done <input.txt
echo $things
But this won't work. Regarding to my google research I thought the while loop would create a new sub shell, but this I was wrong there (see the comment section). Nevertheless the variable "things" was still not available in the echo later on. (I cannot just write the echo inside the while loop, because I need to work with that string later on)
Could you please help me out here? Any help will be appreciated, thank you!
What you proposed works fine! I've only made two changes here: Adding missing quotes, and handling the empty-string case.
things=""
addToString() {
if [ -n "$things" ]; then
things="${things},$1"
else
things="$1"
fi
}
while read -r line; do addToString "$line"; done <input.txt
echo "$things"
If you were piping into while read, this would create a subshell, and that would eat your variables. You aren't piping -- you're doing a <input.txt redirection. No subshell, code works without changes.
That said, there are better ways to read lists of items into shell variables. On any version of bash after 3.0:
IFS=$'\n' read -r -d '' -a things <input.txt # read into an array
printf -v things_str '%s,' "${things[#]}" # write array to a comma-separated string
echo "${things_str%,}" # print that string w/o trailing comma
...on bash 4, that first line can be:
readarray -t things <input.txt # read into an array
This is not a shell solution, but the truth is that solutions in pure shell are often excessively long and verbose. So e.g. to do string processing it is better to use special tools that are part of the “default” Unix environment.
sed ':b;N;$!bb;s/\n/,/g' < input.txt
If you want to omit empty lines, then:
sed ':b;N;$!bb;s/\n\n*/,/g' < input.txt
Speaking about your solution, it should work, but you should really always use quotes where applicable. E.g. this works for me:
things=""
while read line; do things="$things,$line"; done < input.txt
echo "$things"
(Of course, there is an issue with this code, as it outputs a leading comma. If you want to skip empty lines, just add an if check.)
This might/might not work, depending on the shell you are using. On my Ubuntu 14.04/x64, it works with both bash and dash.
To make it more reliable and independent from the shell's behavior, you can try to put the whole block into a subshell explicitly, using the (). For example:
(
things=""
addToString() {
things="${things},$1"
}
while read line; do addToString $line ;done
echo $things
) < input.txt
P.S. You can use something like this to avoid the initial comma. Without bash extensions (using short-circuit logical operators instead of the if for shortness):
test -z "$things" && things="$1" || things="${things},${1}"
Or with bash extensions:
things="${things}${things:+,}${1}"
P.P.S. How I would have done it:
tr '\n' ',' < input.txt | sed 's!,$!\n!'
You can do this too:
#!/bin/bash
while read -r i
do
[[ $things == "" ]] && things="$i" || things="$things","$i"
done < <(grep . input.txt)
echo "$things"
Output:
House,Monkey,Car
N.B:
Used grep to tackle with empty lines and the probability of not having a new line at the end of file. (Normal while read will fail to read the last line if there is no newline at the end of file.)
When we use shell expansion, it gives all the expanded word in one line. For example:
#!/bin/bash
data="Hello\ {World,Rafi}"
eval echo $data
This produces the following output:
Hello World Hello Rafi
Is it possible to output each line on a separate line like this?
Hello World
Hello Rafi
If I understand you right, you want to generate multiple words using brace expansion ({...}), then print each word on a separate line.
If you don't absolutely have to store "Hello\ {World,Rafi}" in a variable, you can do this with printf shell-builtin
printf "%s\n" "Hello "{Rafi,World}
Some explanation:
The format string (here: %s\n) is reused until all the arguments to printf is used up (Reference).
%s\n consumes 1 argument
"Hello "{Rafi,World} returns 2 words/arguments i.e. "Hello Rafi" and "Hello World"
So, this printf command is equivalent to
printf "%s\n%s\n" "Hello Rafi" "Hello World"
except you don't have to type all that up.
#!/bin/bash
data="Hello\ {World'\n',Rafi'\n',Kamal'\n'}"
eval echo -e "$data"
echo -e will evaluate newline characters.
Same as Antarus'a answer, except that echo has "-n". From http://unixhelp.ed.ac.uk/CGI/man-cgi?echo
-n do not output the trailing newline
#!/bin/bash
data="Hello\ {World,Rafi}'\n'"
eval echo -n -e "$data"
Actually, your problem is not the expansion but the echo command. Depending on your system, you might get what you want by
#!/bin/bash
data="Hello\ {World\\\\n,Rafi}"
eval echo -e "$data"
It is different solution but a very clean one.
#!/bin/bash
names="World Rafi"
for name in $names
do
echo Hello $name
done
Instead of using eval (which is dangerous and really not a good practice — see my comment in your post), another strategy would be to use an array. The following will do exactly what you want, in a clean and safe way:
data=( "Hello "{World,Rafi} )
printf "%s\n" "${data[#]}"
How do I print a newline? This merely prints \n:
$ echo -e "Hello,\nWorld!"
Hello,\nWorld!
Use printf instead:
printf "hello\nworld\n"
printf behaves more consistently across different environments than echo.
Make sure you are in Bash.
$ echo $0
bash
All these four ways work for me:
echo -e "Hello\nworld"
echo -e 'Hello\nworld'
echo Hello$'\n'world
echo Hello ; echo world
echo $'hello\nworld'
prints
hello
world
$'' strings use ANSI C Quoting:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
You could always do echo "".
For example,
echo "Hello,"
echo ""
echo "World!"
On the off chance that someone finds themselves beating their head against the wall trying to figure out why a coworker's script won't print newlines, look out for this:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
echo $(GET_RECORDS);
As in the above, the actual running of the method may itself be wrapped in an echo which supersedes any echos that may be in the method itself. Obviously, I watered this down for brevity. It was not so easy to spot!
You can then inform your comrades that a better way to execute functions would be like so:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
GET_RECORDS;
Simply type
echo
to get a new line
POSIX 7 on echo
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html
-e is not defined and backslashes are implementation defined:
If the first operand is -n, or if any of the operands contain a <backslash> character, the results are implementation-defined.
unless you have an optional XSI extension.
So I recommend that you should use printf instead, which is well specified:
format operand shall be used as the format string described in XBD File Format Notation [...]
the File Format Notation:
\n <newline> Move the printing position to the start of the next line.
Also keep in mind that Ubuntu 15.10 and most distros implement echo both as:
a Bash built-in: help echo
a standalone executable: which echo
which can lead to some confusion.
str='hello\nworld'
$ echo | sed "i$str"
hello
world
You can also do:
echo "hello
world"
This works both inside a script and from the command line.
On the command line, press Shift+Enter to do the line break inside the string.
This works for me on my macOS and my Ubuntu 18.04 (Bionic Beaver) system.
For only the question asked (not special characters etc) changing only double quotes to single quotes.
echo -e 'Hello,\nWorld!'
Results in:
Hello,
World!
There is a new parameter expansion added in Bash 4.4 that interprets escape sequences:
${parameter#operator} - E operator
The expansion is a string that is the value of parameter with
backslash escape sequences expanded as with the $'…' quoting
mechanism.
$ foo='hello\nworld'
$ echo "${foo#E}"
hello
world
I just use echo without any arguments:
echo "Hello"
echo
echo "World"
To print a new line with echo, use:
echo
or
echo -e '\n'
This could better be done as
x="\n"
echo -ne $x
-e option will interpret backslahes for the escape sequence
-n option will remove the trailing newline in the output
PS: the command echo has an effect of always including a trailing newline in the output so -n is required to turn that thing off (and make it less confusing)
My script:
echo "WARNINGS: $warningsFound WARNINGS FOUND:\n$warningStrings
Output:
WARNING : 2 WARNINGS FOUND:\nWarning, found the following local orphaned signature file:
On my Bash script I was getting mad as you until I've just tried:
echo "WARNING : $warningsFound WARNINGS FOUND:
$warningStrings"
Just hit Enter where you want to insert that jump. The output now is:
WARNING : 2 WARNINGS FOUND:
Warning, found the following local orphaned signature file:
If you're writing scripts and will be echoing newlines as part of other messages several times, a nice cross-platform solution is to put a literal newline in a variable like so:
newline='
'
echo "first line${newline}second line"
echo "Error: example error message n${newline}${usage}" >&2 #requires usage to be defined
If the previous answers don't work, and there is a need to get a return value from their function:
function foo()
{
local v="Dimi";
local s="";
.....
s+="Some message here $v $1\n"
.....
echo $s
}
r=$(foo "my message");
echo -e $r;
Only this trick worked on a Linux system I was working on with this Bash version:
GNU bash, version 2.2.25(1)-release (x86_64-redhat-linux-gnu)
You could also use echo with braces,
$ (echo hello; echo world)
hello
world
This got me there....
outstuff=RESOURCE_GROUP=[$RESOURCE_GROUP]\\nAKS_CLUSTER_NAME=[$AKS_CLUSTER_NAME]\\nREGION_NAME=[$REGION_NAME]\\nVERSION=[$VERSION]\\nSUBNET-ID=[$SUBNET_ID]
printf $outstuff
Yields:
RESOURCE_GROUP=[akswork-rg]
AKS_CLUSTER_NAME=[aksworkshop-804]
REGION_NAME=[eastus]
VERSION=[1.16.7]
SUBNET-ID=[/subscriptions/{subidhere}/resourceGroups/makeakswork-rg/providers/Microsoft.Network/virtualNetworks/aks-vnet/subnets/aks-subnet]
Sometimes you can pass multiple strings separated by a space and it will be interpreted as \n.
For example when using a shell script for multi-line notifcations:
#!/bin/bash
notify-send 'notification success' 'another line' 'time now '`date +"%s"`
With jq:
$ jq -nr '"Hello,\nWorld"'
Hello,
World
Additional solution:
In cases, you have to echo a multiline of the long contents (such as code/ configurations)
For example:
A Bash script to generate codes/ configurations
echo -e,
printf might have some limitation
You can use some special char as a placeholder as a line break (such as ~) and replace it after the file was created using tr:
echo ${content} | tr '~' '\n' > $targetFile
It needs to invoke another program (tr) which should be fine, IMO.
May God never give you the bane of working on Solaris.
Now, I am trying to run this simple shell script:
#!/bin/sh
input="a
b
c"
data="123"
while read eachline
do
data="$data$eachline"
done << EOF
$(echo "$input")
EOF
echo "$data"
exit 0
On RHEL(BASH) I receive output as expected i.e "123abc", however, on Solaris I receive just "123".
After fair bit of googling, I realized that Solaris is forking a process for code inside the while loop and hence the variable's($data) value is not reflected on the outside of while loop.
Any hope to make this code compatible on both platforms would be greatly appreciated.
And oh yes, using a temp file for redirection would not be a very elegant solution :| .
Do you have a bash executable on the Solaris box ? I note you're referring to bash on RHEL, but your shell is set to #!/bin/sh (i.e. the vanilla Bourne shell).
How about this (please note backticks):
#!/bin/sh
input="a
b
c"
data="123"
z=`while read eachline
do
data="$data$eachline"
done << EOF
$(echo "$input")
EOF
echo $data
`
echo $z
exit 0
Your script works fine on Mac OS X 10.5, which is certified Single Unix Specification version 3.
The behaviour of while in /bin/sh on Solaris is very strange. Can you provide the link where you found the forking issue?
A possible solution may be to use another shell, e.g., /bin/ksh.
Edit
The examples provided by your links don't show an issue with while but a normal behaviour of any shell. The basic construct of all of them is:
first_command | second_command_updating_variable
The fact that second_command_updating_variable is a while loop is not important. What is important is that the second command of the pipe is executed in a subshell and cannot modify variables of its parent shell.
This code below solved my problem. I guess the key here was to echo the variable from within the subshell and ofcourse, not use the "here" document.
I have tested this code on CSH/BASH/KSH.
#!/bin/sh
input="a
b
c"
printf "$input" | {
data="123"
while read eachline
do
data="$data$eachline"
done
echo "$data"
}
exit 0
My heartfelt thanks to all who participated in this discussion.
Why execute a subshell in the here document?
while read eachline
do
data="$data$eachline"
done << EOF
$(echo "$input")
EOF
You could just interpolate:
while read eachline
do
data="$data$eachline"
done << EOF
$input
EOF
Or echo:
echo "$input" | while read eachline
do
data="$data$eachline"
done
Actually, when I tested these, they all worked on Solaris just fine for me, so I'm not sure why you had trouble.