I've noticed that to print the value of the $IFS variable in the shell I have to do something like:
$ printf '%s\nYour IFS is: %q' 'Hello' "$IFS"
Hello
Your IFS is: $' \t\n'
My question is why do I need to pass the IFS in that special way? For example, why (or why wouldn't) it be possible to do:
$ echo $IFS -- some parameter that prints special characters?
$ printf "$IFS" or $ printf '$IFS' -- why wouldn't either of these work?
Why $ printf "%q" $IFS and $ printf "%q" '$IFS' don't show this properly but $ printf "%q" "$IFS" does?
$ echo $IFS -- some parameter that prints special characters?
echo doesn't have such a parameter
$ printf "$IFS" or $ printf '$IFS' -- why wouldn't either of these work?
The first does interpolation of the string just like echo does and prints the IFS string just like it is - which by default is a bunch of whitespaces.
The second does not do interpolation and obviously prints $IFS.
printf "%q" $IFS
The variable has already been expanded into whitespaces that are eaten up by the shell so nothing is passed as a second parameter to printf and therefore %q has no input to work with.
printf "%q" '$IFS'
The string $IFS is passed as a parameter to %q which just adds escape characters to it.
You are running into three basic problems:
Unquoted variable references (as in echo $IFS and printf "%q" $IFS) undergo word splitting. Basically, any whitespace in the variable's value is treated as a separator between "words" (which get passed as arguments to the command). But "whitespace" is defined as the characters in $IFS (that's what it's for!). Therefore, the entire variable's value gets treated as just spacing, not actual content, and effectively vanishes!
This is one of the many examples of why you should put double-quotes around variable references.
Single-quoted strings (as in printf '$IFS' and printf "%q" '$IFS') don't undergo variable expansion at all. In these cases, $IFS is just a literal string, not a variable reference.
Finally, the default characters in $IFS aren't particularly visible on screen. printf "$IFS" actually does print them correctly (unless you put "%" or "\" in IFS for some reason), but you can't really see them. By default, the first character in $IFS is a space, so when that prints the cursor moves to the second column, but nothing visible appears. The second character is a tab, so the cursor moves even further over, but again nothing is actually visible. Then the last character is a newline, so the cursor moves to the beginning of the next line... and again, nothing visible appears.
This is why printf %q is needed -- the %q format converts the not-really-visible characters in $IFS into a visible, readable representation.
If you don't quote the variable, word-splitting is done after the variable is expanded. Word splitting treats all the characters in IFS as word delimiters; multiple of them in a row between words are collapsed into a single space, and if the output is entirely delimiters, nothing at all is output.
You need to use quotes around the variable to output it literally.
Then you need to use the %q format operator to output it as escape sequences so you can see what the individual characters are. Otherwise you wouldn't be able to tell that the second character is a TAB, you would just see a bunch of spaces on the screen. And the newline would just go to the next line.
Related
cat file.json gives me what I want to capture inside of $JSON:
{
key: "value\nwith\nnewline\nchars"
}
I can't do JSON=$(cat file.json) though because then the newline characters are translated and I get after echo $JSON or echo -e $JSON.
{
key: "value
with
newline
chars"
}.
How can I preserve the newline characters inside of $JSON?
Capture using command substitution doesn't perform the translation you're worried about here, but using echo (or misusing printf by substituting into the format string rather than a separate parameter) will.
To emit a variable with backslash sequences intact, use:
printf '%s\n' "$JSON"
This avoids behavior that echo can have (either explicitly with bash's noncompliant extension for echo -e, or implicitly when the xpg_echo flag is enabled in bash, or as default out-of-the-box behavior with other, POSIX+XSI-compatible /bin/sh implementations) wherein escape sequences are replaced by echo, even if the variable passed as an argument had a multi-character backslash sequence.
Given the following code in bash:
filename=${1}
content1=${#:2}
content2="${#:2}"
content3=${*:2}
content4="${*:2}"
echo "${content1}" > "${filename}"
echo "${content2}" >> "${filename}"
echo "${content3}" >> "${filename}"
echo "${content4}" >> "${filename}"
What is the difference between the "contents" ? When I will see the difference? What is the better way to save the content that I get and why?
In an assignment, the right-hand side doesn't have to be quoted to prevent word splitting and globbing, unless it contains a blank.
These two are the same:
myvar=$var
myvar="$var"
but these are not:
myvar='has space'
myvar=has space
The last one tries to run a command space with the environment variable myvar set to value has.
This means that content1 is the same as content2, and content3 is the same as content4, as they only differ in RHS quoting.
The differences thus boil down to the difference between $# and $*; the fact that subarrays are used doesn't matter. Quoting from the manual:
(for $*):
When the expansion occurs within double quotes, it expands to a single word with the value of each parameter separated by the first character of the IFS special variable.
(for $#):
When the expansion occurs within double quotes, each parameter expands to a separate word.
Since you're using quotes (as you almost always should when using $# or $*), the difference is that "${*:2}" is a single string, separated by the first character of IFS, and "${#:2}" expands to separate words, blank separated.
Example:
$ set -- par1 par2 par3 # Set $1, $2, and $3
$ printf '<%s>\n' "${#:2}" # Separate words
<par2>
<par3>
$ printf '<%s>\n' "${*:2}" # Single word, blank separated (default IFS)
<par2 par3>
$ IFS=, # Change IFS
$ printf '<%s>\n' "${*:2}" # Single word, comma separated (new IFS)
<par2,par3>
As for when to use $# vs. $*: as a rule of thumb, you almost always want "$#" and almost never the unquoted version of either, as the latter is subject to word splitting and globbing.
"$*" is useful if you want to join array elements into a single string as in the example, but the most common case, I guess, is iterating over positional parameters in a script or function, and that's what "$#" is for. See also this Bash Pitfall.
The only difference is that using $* will cause the arguments to be concatenated with the first character of IFS, while $# will cause the arguments to be concatenated with a single space (regardless of the value of IFS):
$ set a b c
$ IFS=-
$ c1="$*"
$ c2="$#"
$ echo "$c1"
a-b-c
$ echo "$c2"
a b c
The quotes aren't particularly important here, as expansions on the right-hand side of an assignment aren't subject to word-splitting or pathname expansion; content1 and content2 should always be identical, as should be content3 and content4.
Which is better depends on what you want.
There is a quite nasty expression that want to echo using bash.
The expression is:
'one two --
Note: There is white space after --.
So I have:
IFS=
echo 'one$IFStwoIFS--$IFS
But the result is:
one$IFStwo$IFS--$IFS
You have few issues with your approach:
Within single quote variables are not expanded in shell
In the string one$IFStwo$IFS--$IFS first instance of $IFS will not be expanded since you have string two next to $IFS so it attempts to expand non-existent variable $IFStwo.
Default value of $IFS is $' \t\n'
You can use:
echo "one${IFS}two$IFS--$IFS"
which will expand to (cat -A output):
one ^I$
two ^I$
-- ^I$
This is driving me absolutely crazy:
$ a="/"
echo $a # note empty output line below
$ var="/home/vivek/foo/bar"
$ echo $var
home vivek foo bar
What's going on in my bash shell on OS X?
I've tried this on my other Mac.. and it works perfectly!
tl;dr:
Reset the special $IFS variable to its default - IFS=$' \t\n' - or, preferably, double-quote your variable reference (echo "$var") to print the value as-is.
You're referencing $var unquoted, which means that its value is subject to word splitting (one of the many expansions that Bash applies to unquoted tokens).
Word splitting happens by any of the characters defined in the built-in $IFS variable (the Internal Field Separator), which defaults to $' \t\n' (space, tab, newline).
In your case, $IFS contains / (possibly among other chars.), which means that /home/vivek/foo/bar is split into separate arguments home, vivek, foo, bar, which are then passed to echo.
echo, when given multiple arguments, prints them separated with a space, which is what you're seeing.
(Similarly, / as the value of $var is interpreted as just a separator, with no fields, which means that no arguments are passed to echo, which just prints a newline).
There are 2 lessons here:
Only temporarily change $IFS; restore the previous value once you're done with the custom value.
Generally, double-quote all variable references to ensure that their values are preserved as-is; only use unquoted variable references if you explicitly want shell expansions applied to their values.
I'm almost certain the code I have here worked before. Here's a simplified version and what it produces:
a="atext"
b="btext"
var=$'${a}\n${b}\n'
printf "var=$var"
Which produces output:
var=${a}
${b}
The real code outputs var to file, but the variable expansions aren't happening for some reason.
If this can't work, can you suggest a nice alternative way, and why one uses $' '? Thanks.
GNU bash, version 4.3.42
$'' is a quoting type used to allow backslash escape sequences to describe literal strings with nonprintable characters and other such oddities. Thus, $'\n' evaluates to a single character -- a newline -- whereas '\n' and "\n" both evaluate to two characters, the first being a backslash and the second being an n.
If you want to have the exact behavior of your original code -- putting a literal newline between the results of two different expansions -- you can switch quote types partway through a string:
a="atext"
b="btext"
var="$a"$'\n'"$b"
printf '%s' "var=$var"
That is, right next to each other, with no spaces between:
"$a"
$'\n'
"$b"
This gives you $a and $b expanded, with a literal newline between them.
Why does this matter? Try the following:
$ a=atext
$ b=btext
$ var1="$a\n$b" # Assign with literal "\" and "n" characters
$ printf "$var1" # Here, printf changes the "\n" into the newline
atext
btext
$ printf '%s' "$var1" # ...but this form shows that the "\n" are really there
atext\nbtext
$ var2="$a"$'\n'"$b" # now, we put a single newline in the string
$ printf '%s' "$var2" # and now even accurate use of printf shows that newline
atext
btext
Just replace the single quotes with double quotes.
$ cat test
a="atext"
b="btext"
var=$"${a}\n${b}\n"
printf "var=$var"
$ sh test
var=atext
btext
For variable expansion you either need to use double quotes or no quotes. Single quotes negate expansion.