In bash, you can use printf "%q" to escape the special characters in a string. I've added a line break in the following examples for clarity's sake:
$ printf "%q\n" "foo"
foo
$ printf "%q\n" 'foo$bar'
foo\$bar
$ printf "%q\n" "foo bar" # Tab entered with Ctrl+V Tab
$'foo\tbar'
You can supply the -v option to printf to stick the output into a variable, rather than echoing to stdout.
Now what if I want to echo the original, unescaped string back to stdout? If I just do a simple echo, it includes all the meta/control characters; echo -e gets me slightly further, but not to a fully unescaped state.
Use printf and eval to "unescape" the string (it is perfectly safe, since it is quoted by bash already). echo is, in general, dangerous with arbitrary user input.
eval printf '%s\\n' "$var"
The argument to printf is %s\\n with double backslashes even within the single quotes, since eval will take it as one argument and then strip it (evaluate it) as is, so it effectively becomes something like: printf %s\\n $'12\n34 56' which is a perfectly safe and valid command (the variable even includes a newline and space, as you can see).
Related
cat file.json gives me what I want to capture inside of $JSON:
{
key: "value\nwith\nnewline\nchars"
}
I can't do JSON=$(cat file.json) though because then the newline characters are translated and I get after echo $JSON or echo -e $JSON.
{
key: "value
with
newline
chars"
}.
How can I preserve the newline characters inside of $JSON?
Capture using command substitution doesn't perform the translation you're worried about here, but using echo (or misusing printf by substituting into the format string rather than a separate parameter) will.
To emit a variable with backslash sequences intact, use:
printf '%s\n' "$JSON"
This avoids behavior that echo can have (either explicitly with bash's noncompliant extension for echo -e, or implicitly when the xpg_echo flag is enabled in bash, or as default out-of-the-box behavior with other, POSIX+XSI-compatible /bin/sh implementations) wherein escape sequences are replaced by echo, even if the variable passed as an argument had a multi-character backslash sequence.
I am using following shebang in my script:
#!/bin/sh
To check whether I haven't used any bash syntax, I checked my script with checkbashisms script and it says
possible bashism in my_script.sh line 3 (echo -e):
echo -e "hello world"
And I am running this script on embedded board with busybox shell.
How to resolve this?
POSIX allows echo -e as the default behavior. You may do
var="string with escape\nsequence"
printf "%s\n" "$var" # not interpreting backslashes
printf "%b\n" "$var" # interpreting backslashes
This should pass the checkbashisms test
help echo
-e enable interpretation of the following backslash escapes
...
If your code doesn't need to interpret backslash escapes then you can safely remove the -e flag.
Sometimes, people tend to use -e just for interpreting \n as newline. If that's the case, know that you can have multiline strings just by quoting them and writing as it is. An example:
var="Hi,
I am
a multiline
string"
But if you do need to interpret backslash escapes, then using printf will be your best bet.
Though I would recommend using printf whether you need backslash interpretation or not.
With printf:
printf "%s\n" "$var" # no backslash interpretation
printf "$var" # backslash interpretation occurs
I'm almost certain the code I have here worked before. Here's a simplified version and what it produces:
a="atext"
b="btext"
var=$'${a}\n${b}\n'
printf "var=$var"
Which produces output:
var=${a}
${b}
The real code outputs var to file, but the variable expansions aren't happening for some reason.
If this can't work, can you suggest a nice alternative way, and why one uses $' '? Thanks.
GNU bash, version 4.3.42
$'' is a quoting type used to allow backslash escape sequences to describe literal strings with nonprintable characters and other such oddities. Thus, $'\n' evaluates to a single character -- a newline -- whereas '\n' and "\n" both evaluate to two characters, the first being a backslash and the second being an n.
If you want to have the exact behavior of your original code -- putting a literal newline between the results of two different expansions -- you can switch quote types partway through a string:
a="atext"
b="btext"
var="$a"$'\n'"$b"
printf '%s' "var=$var"
That is, right next to each other, with no spaces between:
"$a"
$'\n'
"$b"
This gives you $a and $b expanded, with a literal newline between them.
Why does this matter? Try the following:
$ a=atext
$ b=btext
$ var1="$a\n$b" # Assign with literal "\" and "n" characters
$ printf "$var1" # Here, printf changes the "\n" into the newline
atext
btext
$ printf '%s' "$var1" # ...but this form shows that the "\n" are really there
atext\nbtext
$ var2="$a"$'\n'"$b" # now, we put a single newline in the string
$ printf '%s' "$var2" # and now even accurate use of printf shows that newline
atext
btext
Just replace the single quotes with double quotes.
$ cat test
a="atext"
b="btext"
var=$"${a}\n${b}\n"
printf "var=$var"
$ sh test
var=atext
btext
For variable expansion you either need to use double quotes or no quotes. Single quotes negate expansion.
When trying to use ANSI color escapes from a shell script, I was completely stuck, as the escape sequences (\e) were printed verbatim to the output.
#!/bin/sh
GREEN="\e[32m"
RED="\e[31m"
CLEAR="\e[0m"
printf "${GREEN}test passed${CLEAR}\n"
printf "${RED}test failed${CLEAR}\n"
Produces
\e[32mtest passed\e[0m
\e[31mtest failed\e[0m
\e is not recognized by POSIX sh (as mentioned by honzasp), but \033 is.
GREEN='\033[32m'
CLEAR='\033[0m'
printf "${GREEN}testpassed${CLEAR}\n"
Generally, it's safer to not expand parameters inside the first argument to printf (consider, for example FOO="hello %s"; printf "$FOO bar \n" baz;). However, this requires you to embed an actual escape character in your parameters, rather than a string that printf interprets as an escape character.
GREEN=$(printf '\033[32m')
CLEAR=$(printf '\033[0m')
printf '%stest passed%s' "$GREEN" "$CLEAR"
The solution is to use #!/bin/bash instead of #!/bin/sh in the first line, because raw sh's printf doesn't understand the escapes.
I had a previous question with bash eval, the responder confirmed to me I have to use eval in that case.
The manual on eval says - that's all it says I am afraid -
eval [arguments]
The arguments are concatenated together into a single command, which is then read and executed, and its exit status returned as the exit status of eval. If there are no arguments or only empty arguments, the return status is zero.
Now I have this problem:
bash> printf '\n'
bash> eval printf '\n'
nbash>
Why is that? Why does eval (which I have to use, not in this case of SSCCE, but in the real case) damage the effect of printf ?
I wish the manual said more about how bash and eval behave. Where did the previous responder find all that information, seemingly more than what the manual says (as I shown above).
OK, the responder suggested the quote. Great, but now I really need this
bash> printf "foo bar"'\n'
to work with eval. If I do
bash eval "printf foo bar'\n'"
does not work as before
eval printf '\n'
The shell reads this command as three words: eval, printf and \n. The single quotes don’t exist anymore after parsing these words.
The eval command now gets this to evaluate: printf \n, and, since the backslash is not enclosed in quotes, it is just discarded. Therefore this is equivalent to:
eval printf n
This is the actual command you get to evaluate:
# echo printf '\n'
printf \n
And this is the result of your evaluated command:
# printf \n
n
echo is your friend!!
Replace the eval by echo to "see" what is being executed:
$ echo printf '\n'
printf '\n'
What will happen with that command?, try it:
$ printf \n
n
As the value is not quoted it lost its special meaning to printf.
So, in the end, the problem is one of quoting (as always is with eval).
$ eval printf \''done %s\n'\' \'test\'
done test