I'm not understanding bash's echo command. Can you explain why echo $a just outputs a newline in the following?
MacbookAir1:so1 palfvin$ a='[]'
MacbookAir1:so1 palfvin$ echo $a
MacbookAir1:so1 palfvin$ echo "$a"
[]
MacbookAir1:so1 palfvin$ echo '[]'
[]
MacbookAir1:so1 palfvin$
I'd particularly appreciate a reference to some documentation explaining this.
The bash version is GNU bash, version 3.2.51(1)-release (x86_64-apple-darwin13)
Update: Output of examination of IFS after resetting per comment thread, is as follows:
MacbookAir1:~ palfvin$ echo "$IFS" | od -c
0000000 \t \n \n
0000004
MacbookAir1:~ palfvin$ echo "$IFS" | od -h
0000000 0920 0a0a
0000004
Having the nullglob shell option set can cause this. [] is a glob pattern that matches any of the characters between [ and ], which is to say nothing. Normally, if the shell finds a pattern that doesn't match anything, it just leaves it alone; but with nullglob it's removed :
$ a='[]'
$ echo $a
[]
$ shopt -s nullglob
$ echo $a
$
From the Filename Expansion section of the bash manual:
After word splitting, unless the -f option has been set (see The Set
Builtin), Bash scans each word for the characters ‘*’, ‘?’, and ‘[’.
If one of these characters appears, then the word is regarded as a
pattern, and replaced with an alphabetically sorted list of file names
matching the pattern. If no matching file names are found, and the
shell option nullglob is disabled, the word is left unchanged. If the
nullglob option is set, and no matches are found, the word is removed.
What's your $IFS set to?
When I set it to IFS='[]', then I can reproduce your result (on a MacBook Pro running Mavericks 10.9.1 and using Bash 3.2.51(1)). When IFS is set to spaces, the problem does not reproduce:
$ echo []
[]
$ x=[]
$ echo $x
[]
$ IFS='[]'
$ echo $x
$
Incidentally, bash -v invokes verbose mode; it echoes the shell script as it processes it. You might still be in a sub-shell, which shouldn't matter much until you try to exit from it, when you'll find yourself in a long-forgotten parent shell instead of having a closed window.
Related
I am using a bash script that expects command line arguments preceded by a dash followed by a file name:
./script -abc file_name
In the script, I try to print out all of the arguments to the script using:
echo "$#"
which displays:
-abc file_name
which is expected. Everything prints fine except for when I use the letter "n" by itself, i.e.:
./script -n file_name
The resulting echo statement inside the script only prints out the file_name argument and not the "-n" argument at all. This behavior only occurs with the letter "n" by itself. When I try "-nx" or 'n' with any other letter(s) the echo statement prints out fine...
Why does this echo statement not work with "-n" by itself?
The -n or -e are valid options for the echo command.
Therefore it's consumed by the echo command and only the remaining text is shown.
That's a known problem of echo and to avoid this you can use printf.
printf "%s\n" "$*"
Some interesting behaviour here:
when
$ set -- -n foo
then you have observed that you don't see the "-n" argument, nor the trailing newline:
$ echo "$#" | od -c
0000000 f o o
0000003
but using the "$*" parameter expansion does give the expected results:
$ echo "$*" | od -c
0000000 - n f o o \n
0000007
Without reading the source code, I suspect that when echo is given only a single argument, it knows there are no options to be parsed.
Nevertheless, use printf so you have complete control over your output.
I know there is a duplicate for this question already at: How to trim whitespace from a Bash variable?.
I read all the answers there but I have a question about another solution in my mind and I want to know if this works.
This is the solution I think works.
a=$(printf "%s" $a)
Here is a demonstration.
$ a=" foo "
$ a=$(printf "%s" $a)
$ echo "$a"
foo
Is there any scenario in which this solution may fail?
If there is such a scenario in which this solution may fail, can we modify this solution to handle that scenario without compromising the simplicity of the solution too much?
If the variable a is set with something like "-e", "-n" in the begining, depending on how you process later your result, a user might crash your script:
-e option allows echo to interpret things backslashed.
Even in the case you only want to display the variable a, -n would screw your layout.
You could think about using regex to check if your variable starts with '-' and is followed by one of the available echo options (-n, -e, -E, --help, --version).
It fails when the input contains spaces between non-whitespace characters.
$ a=" foo bar "
$ a=$(printf "%s" $a)
$ echo "$a"
foobar
The expected output was the following instead.
foo bar
You could use Bash's builtin pattern substitution.
Note: Bash pattern substitution uses 'Pathname Expansion' (glob) pattern matching, not regular expressions. My solution requires enabling the optional shell behaviour extglob (shopt -s extglob).
$shopt -s extglob
$ a=" foo bar "
$ echo "Remove trailing spaces: '${a/%*([[:space:]])}'"
Remove trailing spaces: ' foo bar'
$ echo "Remove leading spaces: '${a/#*([[:space:]])}'"
Remove leading spaces: 'foo bar '
$ echo "Remove all spaces anywhere: '${a//[[:space:]]}'"
Remove all spaces anywhere: 'foobar'
For reference, refer to the 'Parameter Expansion' (Pattern substitution) and 'Pathname Expansion' subsections of the EXPANSION section of the Bash man page.
Echoing without quotes... 1 line. Fine.
$ echo $(ls -1dmb /bin/*) > test
$ wc -l test
1 test
Echoing with quotes... 396 lines. Bad.
$ echo "$(ls -1dmb /bin/*)" > test
$ wc -l test
396 test
The problem comes when using echo for writing a file and expanding a long variable.
Why does this happen? How to fix it?
ls is detecting that your stdout is not a terminal.
check the output of ls -1dmb /bin/* | cat vs ls -1dmb /bin/*. It's ls, who is splitting the output.
Similarly, for ls --color=auto case, color option is used, based on whether the stdout is terminal or not.
When quotes are used, echo is provided with a single arguments, which has embedded newlines, spaces, which are echoed as-is to file.
When quotes are not used, echo is provided multiple arguments, which are split by the IFS. Thus echo prints all of them in a single line.
But, don't skip these quotes...
How to fix it:
I think, the splitting always occurs at the end of some file name & never in between a filename. So one of these 2 options may work for you:
ls -1dmb /bin/* | tr '\n' ' ' >test
ls -1dmb /bin/* | tr -d '\n' >test
#anishsane correctly answers the topic question (that ls is doing the wrapping and ways to remove them) and covers the quoting issue as well but the quoting issue is responsible for the line count difference and not ls.
The issue here is entirely one of quoting and how command lines, echo, and command substitution all work.
The output from "$(ls ...)" is a single string with embedded newlines protected from the shell via the quotes. Hand that value to echo and echo spits it back out literally (with the newlines).
The output from $(ls ...) is a string that is unprotected from the shell and thus undergoes word splitting and whitespace normalization. Command substitution cannot terminate your command line early (you wouldn't want echo $(ls -1) in a directory with two files to run echo first_file; second_file would you?) the newlines are left as word separators between the arguments to echo. The shell then word splits the result on whitespace (including newlines) and gives echo a list of arguments at which point echo happily executes echo first_file second_file ... which, as you can guess, only outputs a single line of output.
Try this to see what I mean:
$ c() {
printf 'argc: %s\n' "$#";
printf 'argv: %s\n' "$#"
}
$ ls *
a.sh b.sh temp
$ ls -1dmb *
a.sh, b.sh, temp
$ c "$(ls -1dmb *)"
argc: 1
argv: a.sh, b.sh, temp
$ c $(ls -1dmb *)
argc: 3
argv: a.sh,
argv: b.sh,
argv: temp
What's the difference between
echo -n "
> "
and
echo -n ""
The first one produces a newline whereas the second doesn't.
I'm running GNU bash, version 4.2.45(1)-release (x86_64-pc-linux-gnu)
Edit : I get that the input gets a newline here. I should have been clearer with the question. Consider the following script input.sh
#!/bin/bash
echo -n $1
The following doesn't produce a newline.
./input.sh "
> "
The string
"
> "
has a newline in it as far as bash is concerned. The -n flag just means that echo will not print an extra newline at the end of your output. It will still reproduce your input, including newlines.
Expanding on #chepner's comment. Consider this:
$ set -- "
"
$ echo -n "$1" | od -c
0000000 \n
0000001
$ echo -n $1 | od -c
0000000
When you leave the variable unquoted, any leading or trailing sequences of whitespace are removed by shell. So bash discards your newline when you don't quote $1. This happens before "echo" is invoked, so "echo -n" is given no arguments.
From the Word Splitting section in the manual:
If IFS is unset, or its value is exactly <space><tab><newline>, the default, then sequences of <space>, <tab>, and <newline> at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words.
What does the triple-less-than-sign bash operator, <<<, mean, as inside the following code block?
LINE="7.6.5.4"
IFS=. read -a ARRAY <<< "$LINE"
echo "$IFS"
echo "${ARRAY[#]}"
Also, why does $IFS remain to be a space, not a period?
It redirects the string to stdin of the command.
Variables assigned directly before the command in this way only take effect for the command process; the shell remains untouched.
From man bash
Here Strings
A variant of here documents, the format is:
<<<word
The word is expanded and supplied to the command on its standard input.
The . on the IFS line is equivalent to source in bash.
Update: More from man bash (Thanks gsklee, sehe)
IFS The Internal Field Separator that is used for word splitting
after expansion and to split lines into words with the read
builtin command. The default value is "<space><tab><new‐line>".
yet more from man bash
The environment for any simple command or function may be augmented
temporarily by prefixing it with parameter assignments, as described
above in PARAMETERS. These assignment statements affect only the environment seen by that command.
The reason that IFS is not being set is that bash isn't seeing that as a separate command... you need to put a line feed or a semicolon after the command in order to terminate it:
$ cat /tmp/ifs.sh
LINE="7.6.5.4"
IFS='.' read -a ARRAY <<< "$LINE"
echo "$IFS"
echo "${ARRAY[#]}"
$ bash /tmp/ifs.sh
7 6 5 4
but
$ cat /tmp/ifs.sh
LINE="7.6.5.4"
IFS='.'; read -a ARRAY <<< "$LINE"
echo "$IFS"
echo "${ARRAY[#]}"
$ bash /tmp/ifs.sh
.
7 6 5 4
I'm not sure why doing it the first way wasn't a syntax error though.