Is it possible to retain trailing newlines when storing the output of a command in a variable?
$ echo "foo
" # newline appears after foo
foo
$ MYVAR="$(echo "foo
")"
$ echo "${MYVAR}" # newline is stripped
foo
$
You can add a sentinel to the end of the stream:
$ my_var=$'foo\n\n'
$ captured=$( echo -n "$my_var"; echo -n "x" )
which you then remove:
$ captured=${captured%x}
$ echo "$captured"
Related
-x means xtrace and -v means verbose.
When researching, I find that bash -x "expands commands" while -v prints out the entire command to stdout before execution.
The only difference I see when testing this is that bash -v <scriptname> will display comments of each command as well.
Is this really the only difference? Can someone elaborate?
From the manual:
-v Print shell input lines as they are read.
-x Print commands and their arguments as they are executed.
You can see this distinction when looking at how piped commands are handled:
$ set -v
$ echo "Hello World" | sed 's/World/Earth/'
echo "Hello World" | sed 's/World/Earth/'
Hello Earth
versus:
$ set -x
$ echo "Hello World" | sed 's/World/Earth/'
+ sed s/World/Earth/
+ echo 'Hello World'
Hello Earth
Also, it appears that xtrace (-x) uses the $PS4 variable, while verbose (-v) does not.
The outputs are very similar for simple commands, but it seems that set -v (verbose) simply repeats the input line, while set -x (xtrace) shows the command(s) that are run after variable expansions and word splitting, etc. Considering the examples below, it seems to me that it's worth using both when debugging shell scripts: verbose helps navigate the script and show the intention of the commands, while xtrace shows in more detail what is actually being run. I'm going to use set -vx when debugging scripts from now on. The differences become more obvious in examples with variables, command lists, functions, etc:
$ set +vx # both disabled (default for most people)
$ foo=(1 2 '3 three')
$ printf 'foo is %s\n' "${foo[#]}" && echo ''
foo is 1
foo is 2
foo is 3 three
$ set -v # verbose
$ printf 'foo is %s\n' "${foo[#]}" && echo ''
printf 'foo is %s\n' "${foo[#]}" && echo ''
foo is 1
foo is 2
foo is 3 three
$ set +v
set +v
$ set -x # xtrace
$ printf 'foo is %s\n' "${foo[#]}" && echo ''
+ printf 'foo is %s\n' 1 2 '3 three'
foo is 1
foo is 2
foo is 3 three
+ echo ''
$ set +x
+ set +x
$ set -vx # both verbose and xtrace
$ printf 'foo is %s\n' "${foo[#]}" && echo ''
printf 'foo is %s\n' "${foo[#]}" && echo '' # from verbose, input line simply repeated
+ printf 'foo is %s\n' 1 2 '3 three' # from xtrace, command list split, variables substituted
foo is 1
foo is 2
foo is 3 three
+ echo ''
# word splitting behaviour is made more obvious by xtrace
# (note the array is unquoted now)
$ printf 'foo is %s\n' ${foo[#]} && echo ''
printf 'foo is %s\n' ${foo[#]} && echo ''
+ printf 'foo is %s\n' 1 2 3 three # missing single quotes due to word splitting
foo is 1
foo is 2
foo is 3
foo is three # !
+ echo ''
# function with both verbose and xtrace set
$ function bar { printf '%s\n' "${foo[#]}"; }
function bar { printf '%s\n' "${foo[#]}"; } # verbose repeats definition, nothing output by xtrace
$ bar
bar # verbose repeats function call
+ bar # xtrace shows commands run
+ printf '%s\n' 1 2 '3 three'
1
2
3 three
Aside: other shells
The above examples are from bash. In my testing, the output of posix dash for both options is very similar, although of course no arrays are possible in that shell. zsh expands the xtrace output slightly by prepending with a line number, but otherwise behaves very similarly (although here, its different word splitting behaviour is apparent):
$ zsh
% foo=(1 2 '3 three')
% set -vx
% printf 'foo is %s\n' ${foo[#]} && echo ''
printf 'foo is %s\n' ${foo[#]} && echo '' # verbose repeats input command
+zsh:14> printf 'foo is %s\n' 1 2 '3 three' # xtrace prints expanded command with line number
foo is 1
foo is 2
foo is 3 three
+zsh:14> echo ''
I'm trying to use \n to output two seperate lines using echo and store that in a variable:
VAR=$( echo -e "foo\nbar" )
But the output I get is:
$ echo $VAR
foo bar
It works fine by itself:
$ echo -e "foo\nbar"
foo
bar
What am I doing wrong?
You need to wrap $VAR in double quotes:
echo "$VAR"
Otherwise, word splitting occurs. This means that "foo" and "bar" are treated as two separate arguments to echo and the newline between them is lost.
You can use set -x to enable debug mode and see what happens:
$ set -x
$ echo $VAR
+ echo foo bar
foo bar
$ echo "$VAR"
+ echo 'foo
bar'
foo
bar
I have this script:
#!/usr/bin/env bash
read -p "Provide arguments: " -a arr <<< "foo \"bar baz\" buz"
for i in ${arr[#]}
do
echo $i
done
which incorrectly outputs:
foo
"bar
baz"
buz
How can I make it interpret user input so that parameters within quotation marks would make a single array element? Like this:
foo
bar baz
buz
EDIT:
To be clear: I don't want the user to input each element in separate line, so read-ing in loop is not what I'm looking for.
You're better off supplying user input using a different delimiter, e.g. ;.
OLDIFS=$IFS
IFS=$';'
read -p "Provide arguments: " -a var
for i in "${!var[#]}"
do
echo Argument $i: "${var[i]}"
done
IFS=$OLDIFS
Upon execution:
Provide arguments: foo;bar baz;buz
Argument 0: foo
Argument 1: bar baz
Argument 2: buz
Plus a modification to trim the variables:
echo Argument $i: $(echo "${var[i]}" | sed -e 's/^ *//g' -e 's/ *$//g')
Redirection to a file is very usefull to append a string as a new line to a file, like
echo "foo" >> file.txt
echo "bar" >> file.txt
Result:
foo
bar
But is it also possible to redirect a string to the same line in the file ?
Example:
echo "foo" <redirection-command-for-same-line> file.txt
echo "bar" <redirection-command-for-same-line> file.txt
Result:
foobar
The newline is added by echo, not by the redirection. Just pass the -n switch to echo to suppress it:
echo -n "foo" >> file.txt
echo -n "bar" >> file.txt
-n do not output the trailing newline
An alternate way to echo results to one line would be to simply assign the results to variables. Example:
j=$(echo foo)
i=$(echo bar)
echo $j$i
foobar
echo $i $j
bar foo
This is particularly useful when you have more complex functions, maybe a complex 'awk' statement to pull out a particular cell in a row, then pair it with another set.
Working with printf in a bash script, adding no spaces after "\n" does not create a newline, whereas adding a space creates a newline, e. g.:
No space after "\n"
NewLine=`printf "\n"`
echo -e "Firstline${NewLine}Lastline"
Result:
FirstlineLastline
Space after "\n "
NewLine=`printf "\n "`
echo -e "Firstline${NewLine}Lastline"
Result:
Firstline
Lastline
Question: Why doesn't 1. create the following result:
Firstline
Lastline
I know that this specific issue could have been worked around using other techniques, but I want to focus on why 1. does not work.
Edited:
When using echo instead of printf, I get the expected result, but why does printf work differently?
NewLine=`echo "\n"`
echo -e "Firstline${NewLine}Lastline"
Result:
Firstline
Lastline
The backtick operator removes trailing new lines. See 3.4.5. Command substitution at http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_04.html
Note on edited question
Compare:
[alvaro#localhost ~]$ printf "\n"
[alvaro#localhost ~]$ echo "\n"
\n
[alvaro#localhost ~]$ echo -e "\n"
[alvaro#localhost ~]$
The echo command doesn't treat \n as a newline unless you tell him to do so:
NAME
echo - display a line of text
[...]
-e enable interpretation of backslash escapes
POSIX 7 specifies this behaviour here:
[...] with the standard output of the command, removing sequences of one or more characters at the end of the substitution
Maybe people will come here with the same problem I had:
echoing \n inside a code wrapped in backsticks. A little tip:
printf "astring\n"
# and
printf "%s\n" "astring"
# both have the same effect.
# So... I prefer the less typing one
The short answer is:
# Escape \n correctly !
# Using just: printf "$myvar\n" causes this effect inside the backsticks:
printf "banana
"
# So... you must try \\n that will give you the desired
printf "banana\n"
# Or even \\\\n if this string is being send to another place
# before echoing,
buffer="${buffer}\\\\n printf \"$othervar\\\\n\""
One common problem is that if you do inside the code:
echo 'Tomato is nice'
when surrounded with backsticks will produce the error
command Tomato not found.
The workaround is to add another echo -e or printf
printed=0
function mecho(){
#First time you need an "echo" in order bash relaxes.
if [[ $printed == 0 ]]; then
printf "echo -e $1\\\\n"
printed=1
else
echo -e "\r\n\r$1\\\\n"
fi
}
Now you can debug your code doing in prompt just:
(prompt)$ `mySuperFunction "arg1" "etc"`
The output will be nicely
mydebug: a value
otherdebug: whathever appended using myecho
a third string
and debuging internally with
mecho "a string to be hacktyped"
$ printf -v NewLine "\n"
$ echo -e "Firstline${NewLine}Lastline"
Firstline
Lastline
$ echo "Firstline${NewLine}Lastline"
Firstline
Lastline
It looks like BASH is removing trailing newlines.
e.g.
NewLine=`printf " \n\n\n"`
echo -e "Firstline${NewLine}Lastline"
Firstline Lastline
NewLine=`printf " \n\n\n "`
echo -e "Firstline${NewLine}Lastline"
Firstline
Lastline
Your edited echo version is putting a literal backslash-n into the variable $NewLine which then gets interpreted by your echo -e. If you did this instead:
NewLine=$(echo -e "\n")
echo -e "Firstline${NewLine}Lastline"
your result would be the same as in case #1. To make that one work that way, you'd have to escape the backslash and put the whole thing in single quotes:
NewLine=$(printf '\\n')
echo -e "Firstline${NewLine}Lastline"
or double escape it:
NewLine=$(printf "\\\n")
Of course, you could just use printf directly or you can set your NewLine value like this:
printf "Firstline\nLastline\n"
or
NewLine=$'\n'
echo "Firstline${NewLine}Lastline" # no need for -e
For people coming here wondering how to use newlines in arguments to printf, use %b instead of %s:
$> printf "a%sa" "\n"
a\na
$> printf "a%ba" "\n"
a
a
From the manual:
%b expand backslash escape sequences in the corresponding argument
We do not need "echo" or "printf" for creating the NewLine variable:
NewLine="
"
printf "%q\n" "${NewLine}"
echo "Firstline${NewLine}Lastline"
Bash delete all trailing newlines in commands substitution.
To save trailing newlines, assign printf output to the variable with printf -v VAR
instead of
NewLine=`printf "\n"`
echo -e "Firstline${NewLine}Lastline"
#FirstlineLastline
use
printf -v NewLine '\n'
echo -e "Firstline${NewLine}Lastline"
#Firstline
#Lastline
Explanation
According to bash man
3.5.4 Command Substitution
$(command)
or
`command`
Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting.
So, after adding any trailing newlines, bash will delete them.
var=$(printf '%s\n%s\n\n\n' 'foo' 'bar')
echo "$var"
output:
foo
bar
According to help printf
printf [-v var] format [arguments]
If the -v option is supplied, the output is placed into the value of the shell variable VAR rather than being sent to the standard output.
In this case, for safe copying of formatted text to the variable, use the [-v var] option:
printf -v var '%s\n%s\n\n\n' 'foo' 'bar'
echo "$var"
output:
foo
bar
Works ok if you add "\r"
$ nl=`printf "\n\r"` && echo "1${nl}2"
1
2