VAR="-e xyz"
echo $VAR
This prints xyz, for some reason. I don't seem to be able to find a way to get a string to start with -e.
What is going on here?
The answers that say to put $VAR in quotes are only correct by side effect. That is, when put in quotes, echo(1) receives a single argument of -e xyz, and since that is not a valid option string, echo just prints it out. It is a side effect as echo could just as easily print an error regarding malformed options. Most programs will do this, but it seems GNU echo (from coreutils) and the version built into bash simply echo strings that start with a hyphen but are not valid argument strings. This behaviour is not documented so it should not be relied upon.
Further, if $VAR contains a valid echo option argument, then quoting $VAR will not help:
$ VAR="-e"
$ echo "$VAR"
$
Most GNU programs take -- as an argument to mean no more option processing — all the arguments after -- are to be processed as non-option arguments. bash echo does not support this so you cannot use it. Even if it did, it would not be portable. echo has other portability issues (-n vs \c, no -e).
The correct and portable solution is to use printf(1).
printf "%s\n" "$VAR"
The variable VAR contains -e xyz, if you access the variable via $ the -e is interpreted as a command-line option for echo. Note that the content of $VAR is not wrapped into "" automatically.
Use echo "$VAR" to fix your problem.
In zsh, you can use a single dash (-) before your arguments. This ensures that no following arguments are interpreted as options.
% VAR="-e xyz"
% echo - $VAR
-e xyz
From the zsh docs:
echo [ -neE ] [ arg ... ]
...
Note that for standards compliance a double dash does not
terminate option processing; instead, it is printed directly.
However, a single dash does terminate option processing, so the
first dash, possibly following options, is not printed, but
everything following it is printed as an argument.
The single dash behaviour is different from other shells.
Keep in mind this behavior is specific to zsh.
Try:
echo "$VAR"
instead.
(-e is a valid option for echo - this is what causes this phenomenon).
Related
I'm writing a shell script that should be somewhat secure, i.e., does not pass secure data through parameters of commands and preferably does not use temporary files. How can I pass a variable to the standard input of a command?
Or, if it's not possible, how can I correctly use temporary files for such a task?
Passing a value to standard input in Bash is as simple as:
your-command <<< "$your_variable"
Always make sure you put quotes around variable expressions!
Be cautious, that this will probably work only in bash and will not work in sh.
Simple, but error-prone: using echo
Something as simple as this will do the trick:
echo "$blah" | my_cmd
Do note that this may not work correctly if $blah contains -n, -e, -E etc; or if it contains backslashes (bash's copy of echo preserves literal backslashes in absence of -e by default, but will treat them as escape sequences and replace them with corresponding characters even without -e if optional XSI extensions are enabled).
More sophisticated approach: using printf
printf '%s\n' "$blah" | my_cmd
This does not have the disadvantages listed above: all possible C strings (strings not containing NULs) are printed unchanged.
(cat <<END
$passwd
END
) | command
The cat is not really needed, but it helps to structure the code better and allows you to use more commands in parentheses as input to your command.
Note that the 'echo "$var" | command operations mean that standard input is limited to the line(s) echoed. If you also want the terminal to be connected, then you'll need to be fancier:
{ echo "$var"; cat - ; } | command
( echo "$var"; cat - ) | command
This means that the first line(s) will be the contents of $var but the rest will come from cat reading its standard input. If the command does not do anything too fancy (try to turn on command line editing, or run like vim does) then it will be fine. Otherwise, you need to get really fancy - I think expect or one of its derivatives is likely to be appropriate.
The command line notations are practically identical - but the second semi-colon is necessary with the braces whereas it is not with parentheses.
This robust and portable way has already appeared in comments. It should be a standalone answer.
printf '%s' "$var" | my_cmd
or
printf '%s\n' "$var" | my_cmd
Notes:
It's better than echo, reasons are here: Why is printf better than echo?
printf "$var" is wrong. The first argument is format where various sequences like %s or \n are interpreted. To pass the variable right, it must not be interpreted as format.
Usually variables don't contain trailing newlines. The former command (with %s) passes the variable as it is. However tools that work with text may ignore or complain about an incomplete line (see Why should text files end with a newline?). So you may want the latter command (with %s\n) which appends a newline character to the content of the variable. Non-obvious facts:
Here string in Bash (<<<"$var" my_cmd) does append a newline.
Any method that appends a newline results in non-empty stdin of my_cmd, even if the variable is empty or undefined.
I liked Martin's answer, but it has some problems depending on what is in the variable. This
your-command <<< """$your_variable"""
is better if you variable contains " or !.
As per Martin's answer, there is a Bash feature called Here Strings (which itself is a variant of the more widely supported Here Documents feature):
3.6.7 Here Strings
A variant of here documents, the format is:
<<< word
The word is expanded and supplied to the command on its standard
input.
Note that Here Strings would appear to be Bash-only, so, for improved portability, you'd probably be better off with the original Here Documents feature, as per PoltoS's answer:
( cat <<EOF
$variable
EOF
) | cmd
Or, a simpler variant of the above:
(cmd <<EOF
$variable
EOF
)
You can omit ( and ), unless you want to have this redirected further into other commands.
Try this:
echo "$variable" | command
If you came here from a duplicate, you are probably a beginner who tried to do something like
"$variable" >file
or
"$variable" | wc -l
where you obviously meant something like
echo "$variable" >file
echo "$variable" | wc -l
(Real beginners also forget the quotes; usually use quotes unless you have a specific reason to omit them, at least until you understand quoting.)
I don't seem to be able to assign a string that begins with "-e" or "-E" to a bash shell variable:
$ options="-e stuff"
$ echo $options
stuff
Other letters work fine:
$ options="-g stuff"
$ echo $options
-g stuff
What is the reason for this?
You should quote your variable:
echo "${options}"
otherwise it's being expanded to
echo -g stuff
which is being interpreter by echo as its -e option, which actually exists (see man echo), and that's why -e "did not work" while other letters you tried "did".
First: To reliably determine the value of a variable in bash, use declare -p, not echo. Thus:
declare -p options
will emit something like:
declare -- options="-e stuff"
This tells you much more than echo does:
Because it's declare -- rather than declare -x, you know that the variable is not exported.
Because it's not declare -a, you know it's not giving you an array (echo "$array" will print only the first element of a shell array and ignore the rest).
Because it's not declare -i, you know the value wasn't declared to be an integer... etc.
If you're only worried about the string case, but want to ensure that you get a printable value no matter which version of bash is in use (as some historical releases will not always guarantee printable escaping for values printed with declare -p), consider instead:
printf '%q=%q\n' options "$options"
...which will emit unambiguous output even if there are cursor control characters, newlines, or other non-textual contents in your string.
Follow the advice of the POSIX specification for echo, and use printf instead. To quote the APPLICATION USAGE section in full, emphasis added, noting that in bash, -e enables XSI-style interpretation of escape sequences:
It is not possible to use echo portably across all POSIX systems
unless both -n (as the first argument) and escape sequences are
omitted.
The printf utility can be used portably to emulate any of the
traditional behaviors of the echo utility as follows (assuming that
IFS has its standard value or is unset):
The historic System V echo and the requirements on XSI implementations
in this volume of POSIX.1-2008 are equivalent to:
printf "%b\n" "$*"
The BSD echo is equivalent to:
if [ "X$1" = "X-n" ]
then
shift
printf "%s" "$*"
else
printf "%s\n" "$*"
fi
New applications are encouraged to use printf instead of echo.
So, how does this apply to you? Since you want -e to be treated as data, not part of echo's setup, the BSD, non--n branch of that applies:
options="-e stuff"
printf '%s\n' "$options"
I would like to use echo in bash to print out a string of characters followed by only a carriage return. I've looked through the man page and have found that echo -e will make echo interpret backslash escape characters. Using that I can say echo -e 'hello\r' and it will print like this
$>echo -e 'hello\r'
hello
$>
So it looks like it handled the carriage return properly. I also found echo -n in the man page will stop echo from inserting a newline character and it looks like it works when I do this
$>echo -n 'hello\r'
hello\r$>
The problem I'm having is in combining both -e and -n. I've tried each of echo -e -n 'hello\r', echo -n -e 'hello\r', echo -en 'hello\r', and echo -ne 'hello\r' and nothing gets printed like so:
$>echo -ne 'hello\r'
$>
Is there something I'm missing here or can the -e and -n options not be used together?
I think it's working, you're just not seeing it. This is your command:
$> echo -ne 'hello\r'
Because of the carriage return (\r), that will leave the cursor at the start of the same line on the terminal where it wrote the hello - which means that's where the next thing output to the terminal will be written. So if your actual prompt is longer than the $> you show here, it will overwrite the hello completely.
This sequence will let you see what's actually happening:
echo -ne 'hello\r'; sleep 5; echo 'good-bye'
But for better portability to other shells, I would avoid using options on echo like that. Those are purely bashisms, not supported by the POSIX standard. The printf builtin, however, is specified by POSIX. So if you want to display strings with no newline and parsing of backslash sequences, you can just use it:
printf '%s\r' 'hello'
There are numerous different implementations of the echo command. There's one built into most shells (with different behavior for each), and the behavior of /bin/echo varies considerably from one system to another.
Rather than echo, use printf. It's built into bash as well as being available as an external command, and its behavior is much more consistent across implementations. (The major variation is that the GNU coreutils printf recognizes --help and --version options.)
Just use:
printf 'hello\r'
I'd like you to introduce you to printf.
OP, meet printf. printf. This is the OP...
Whenever you are trying to do anything unusual with output in BASH, you should switch to printf. In fact, I use printf all the time, so my scripts will run under both BASH and Kornshell.
Although BASH and Kornshell are 99% the same, the echo command is different. In Kornshell, it's deprecated, and you're supposed to use the builtin print. However, there's no print in BASH. Using printf solves the problem because it works (sort of) the same in both shells.
Since printf doesn't automatically end each line with a \n, you don't have to worry about how to prevent the \n from being appended. Heck, if you want a \n, you have to put it yourself. In your case:
printf `hello\r`
However, for safety reasons, you should really do this:
printf '%s\r' 'hello'
This is a habit I've gotten into when I'm using printf. Imagine if I want to print out $FOO, and I do this:
printf "$FOO\n"
That will normally work, but what if $FOO has a dash at the beginning, or has a % sign in it. The above won't do what I want to do. However, if I do this:
printf '%s\n' "$FOO"
This will print out $FOO no matter what $FOO has in it.
POSIX 7 says it is not possible:
If the first operand is -n, or if any of the operands contain a backslash character, the results are implementation-defined.
On Ubuntu 12.04 for example:
Bash's built-in echo echo -e '\n' interprets the newline, echo '\n' does not.
/bin/sh's built-in echo does not have the -e option, and interprets the newline even without it. This can byte when writing Makefiles, which use /bin/sh by default.
POSIX 7 solution: printf as the others mentioned.
Quoting. It's the wave of the future. Always quote.
echo -ne 'hello\r'
Use double quotes if you put a variable inside that you want expanded.
I've read the man pages on echo, and it tells me that the -e parameter will allow an escaped character, such as an escaped n for newline, to have its special meaning. When I type the command
$ echo -e 'foo\nbar'
into an interactive bash shell, I get the expected output:
foo
bar
But when I use this same command (i've tried this command character for character as a test case) I get the following output:
-e foo
bar
It's as if echo is interpretting the -e as a parameter (because the newline still shows up) yet also it interprets the -e as a string to echo. What's going on here? How can I prevent the -e showing up?
You need to use #!/bin/bash as the first line in your script. If you don't, or if you use #!/bin/sh, the script will be run by the Bourne shell and its echo doesn't recognize the -e option. In general, it is recommended that all new scripts use printf instead of echo if portability is important.
In Ubuntu, sh is provided by a symlink to /bin/dash.
Different implementations of echo behave in annoyingly different ways. Some don't take options (i.e. will simply echo -e as you describe) and automatically interpret escape sequences in their parameters. Some take flags, and don't interpret escapes unless given the -e flag. Some take flags, and interpret different escape sequences depending on whether the -e flag was passed. Some will cause you to tear your hair out if you try to get them to behave in a predictable manner... oh, wait, that's all of them.
What you're probably seeing here is a difference between the version of echo built into bash vs /bin/echo or maybe vs. some other shell's builtin. This bit me when Mac OS X v10.5 shipped with a bash builtin echo that echoed flags, unlike what all my scripts expected...
In any case, there's a solution: use printf instead. It always interprets escape sequences in its first argument (the format string). The problems are that it doesn't automatically add a newline (so you have to remember do that explicitly), and it also interprets % sequences in its first argument (it is, after all, a format string). Generally, you want to put all the formatting stuff in the format string, then put variable strings in the rest of the arguments so you can control how they're interpreted by which % format you use to interpolate them into the output. Some examples:
printf "foo\nbar\n" # this does what you're trying to do in the example
printf "%s\n" "$var" # behaves like 'echo "$var"', except escapes will never be interpreted
printf "%b\n" "$var" # behaves like 'echo "$var"', except escapes will always be interpreted
printf "%b\n" "foo\nbar" # also does your example
Use
alias echo /usr/bin/echo
to force 'echo' invoking coreutils' echo which interpret '-e' parameter.
Try this:
import subprocess
def bash_command(cmd):
subprocess.Popen(['/bin/bash', '-c', cmd])
code="abcde"
// you can use echo options such as -e
bash_command('echo -e "'+code+'"')
Source: http://www.saltycrane.com/blog/2011/04/how-use-bash-shell-python-subprocess-instead-binsh/
How do I echo one or more tab characters using a bash script?
When I run this code
res=' 'x # res = "\t\tx"
echo '['$res']' # expect [\t\tx]
I get this
res=[ x] # that is [<space>x]
echo -e ' \t '
will echo 'space tab space newline' (-e means 'enable interpretation of backslash escapes'):
$ echo -e ' \t ' | hexdump -C
00000000 20 09 20 0a | . .|
Use printf, not echo.
There are multiple different versions of the echo command. There's /bin/echo (which may or may not be the GNU Coreutils version, depending on the system), and the echo command is built into most shells. Different versions have different ways (or no way) to specify or disable escapes for control characters.
printf, on the other hand, has much less variation. It can exist as a command, typically /bin/printf, and it's built into some shells (bash and zsh have it, tcsh and ksh don't), but the various versions are much more similar to each other than the different versions of echo are. And you don't have to remember command-line options (with a few exceptions; GNU Coreutils printf accepts --version and --help, and the built-in bash printf accepts -v var to store the output in a variable).
For your example:
res=' 'x # res = "\t\tx"
printf '%s\n' "[$res]"
And now it's time for me to admit that echo will work just as well for the example you're asking about; you just need to put double quotes around the argument:
echo "[$res]"
as kmkaplan wrote (two and a half years ago, I just noticed!). The problem with your original commands:
res=' 'x # res = "\t\tx"
echo '['$res']' # expect [\t\tx]
isn't with echo; it's that the shell replaced the tab with a space before echo ever saw it.
echo is fine for simple output, like echo hello world, but you should use printf whenever you want to do something more complex. You can get echo to work, but the resulting code is likely to fail when you run it with a different echo implementation or a different shell.
You can also try:
echo Hello$'\t'world.
Put your string between double quotes:
echo "[$res]"
you need to use -e flag for echo then you can
echo -e "\t\t x"
From the bash man page:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
So you can do this:
echo $'hello\tworld'
Use the verbatim keystroke, ^V (CTRL+V, C-v, whatever).
When you type ^V into the terminal (or in most Unix editors), the following character is taken verbatim. You can use this to type a literal tab character inside a string you are echoing.
Something like the following works:
echo "^V<tab>" # CTRL+V, TAB
Bash docs (q.v., "quoted-insert")
quoted-insert (C-q, C-v)
Add the next character that you type to the line verbatim. This is how to insert key sequences like C-q, for example.
side note: according to this, ALT+TAB should do the same thing, but we've all bound that sequence to window switching so we can't use it
tab-insert (M-TAB)
Insert a tab character.
--
Note: you can use this strategy with all sorts of unusual characters. Like a carriage return:
echo "^V^M" # CTRL+V, CTRL+M
This is because carriage return is ASCII 13, and M is the 13th letter of the alphabet, so when you type ^M, you get the 13th ASCII character. You can see it in action using ls^M, at an empty prompt, which will insert a carriage return, causing the prompt to act just like you hit return. When these characters are normally interpreted, verbatim gets you get the literal character.
Using echo to print values of variables is a common Bash pitfall.
Reference link:
http://mywiki.wooledge.org/BashPitfalls#echo_.24foo
If you want to use echo "a\tb" in a script, you run the script as:
# sh -e myscript.sh
Alternatively, you can give to myscript.sh the execution permission, and then run the script.
# chmod +x myscript.sh
# ./myscript.sh
res="\t\tx"
echo -e "[${res}]"