Original string getting mutated while printing with echo statement.
#!/bin/bash
response='{\\\"test\\\":\\\"data\\\"}'
echo $response;
Actual Output - {\\"test\\":\\"data\\"}
Expected output - {\\\"test\\\":\\\"data\\\"}
quote your variables (see https://mywiki.wooledge.org/Quotes)
use printf, not echo (see https://unix.stackexchange.com/q/65803/133219)
e.g.:
$ response='{\\\"test\\\":\\\"data\\\"}'
$ printf '%s\n' "$response"
{\\\"test\\\":\\\"data\\\"}
This works as expected in bash, but you are instead running it with sh. See: Why does my bash code fail when I run it with sh?
However, when you want to print a string exactly as is, use printf:
response='{\\\"test\\\":\\\"data\\\"}'
printf '%s\n' "$response"
This works correctly for all values in all shells, including response='*' reponse='-n' and response='foo bar'
Related
I am trying to Concatenate variables with Strings in my bash script, the variables are being read independently but whenever I try to concatenate them, it doesn't recognize the variable values.
ex-
echo $CONFIG_PROTOCOL (Prints the variable value, HTTP)
echo $CONFIG_PROTOCOL'://'$CONFIG_SERVER_SOURCE:$CONFIG_PORT'/api/creation/objects/export?collection.ids='$sf
The above echo with the URL prints out /api/creations/objects/export?collection.ids=value_1, while it should print out http://localhost:8080/api/creation/objects/export?collection.ids=value_1
Any inputs will be appreciated.
This happens because $CONFIG_PORT has a trailing carriage return. Here's a MCVE:
CONFIG_PROTOCOL="http"
CONFIG_SERVER_SOURCE="example.com"
CONFIG_PORT="8080"
sf="42"
# Introduce bug:
CONFIG_PORT+=$'\r'
echo $CONFIG_PROTOCOL'://'$CONFIG_SERVER_SOURCE:$CONFIG_PORT'/api/creation/objects/export?collection.ids='$sf
When executed, this prints:
/api/creation/objects/export?collection.ids=42
When you comment out the buggy line, you get:
http://example.com:8080/api/creation/objects/export?collection.ids=42
In both cases, echo will appear to show the correct values because echo is a tool for showing text to humans and not useful for showing the underlying data. printf '%q\n' "$CONFIG_PORT" will instead show it in an unambiguous format:
$ echo $CONFIG_PORT
8080
$ printf '%q\n' "$CONFIG_PORT"
$'8080\r'
The best way to fix this is to ensure that whatever supplies the value does so correctly. But the easiest way is to just strip them:
echo $CONFIG_PROTOCOL'://'$CONFIG_SERVER_SOURCE:$CONFIG_PORT'/api/creation/objects/export?collection.ids='$sf | tr -d '\r'
echo "$CONFIG_PROTOCOL://$CONFIG_SERVER_SOURCE:$CONFIG_PORT/api/creation/objects/export?collection.ids=$sf"
Try above echo statement
I'm trying to write a "phone home" script, which will log the exact command line (including any single or double quotes used) into a MySQL database. As a backend, I have a cgi script which wraps the database. The scripts themselves call curl on the cgi script and include as parameters various arguments, including the verbatim command line.
Obviously I have quite a variety of quote escaping to do here and I'm already stuck at the bash stage. At the moment, I can't even get bash to print verbatim the arguments provided:
Desired output:
$ ./caller.sh -f -hello -q "blah"
-f hello -q "blah"
Using echo:
caller.sh:
echo "$#"
gives:
$ ./caller.sh -f -hello -q "blah"
-f hello -q blah
(I also tried echo $# and echo $*)
Using printf %q:
caller.sh:
printf %q $#
printf "\n"
gives:
$ ./caller.sh -f hello -q "blah"
-fhello-qblah
(I also tried print %q "$#")
I would welcome not only help to fix my bash problem, but any more general advice on implementing this "phone home" in a tidier way!
There is no possible way you can write caller.sh to distinguish between these two commands invoked on the shell:
./caller.sh -f -hello -q "blah"
./caller.sh -f -hello -q blah
There are exactly equivalent.
If you want to make sure the command receives special characters, surround the argument with single quotes:
./caller.sh -f -hello -q '"blah"'
Or if you want to pass just one argument to caller.sh:
./caller.sh '-f -hello -q "blah"'
You can get this info from the shell history:
function myhack {
line=$(history 1)
line=${line#* }
echo "You wrote: $line"
}
alias myhack='myhack #'
Which works as you describe:
$ myhack --args="stuff" * {1..10} $PATH
You wrote: myhack --args="stuff" * {1..10} $PATH
However, quoting is just the user's way of telling the shell how to construct the program's argument array. Asking to log how the user quotes their arguments is like asking to log how hard the user punched the keys and what they were wearing at the time.
To log a shell command line which unambiguously captures all of the arguments provided, you don't need any interactive shell hacks:
#!/bin/bash
line=$(printf "%q " "$#")
echo "What you wrote would have been indistinguishable from: $line"
I understand you want to capture the arguments given by the caller.
Firstly, quotes used by the caller are used to protect during the interpretation of the call. But they do not exist as argument.
An example: If someone call your script with one argument "Hello World!" with two spaces between Hello and World. Then you have to protect ALWAYS $1 in your script to not loose this information.
If you want to log all arguments correctly escaped (in the case where they contains, for example, consecutive spaces...) you HAVE to use "$#" with double quotes. "$#" is equivalent to "$1" "$2" "$3" "$4" etc.
So, to log arguments, I suggest the following at the start of the caller:
i=0
for arg in "$#"; do
echo "arg$i=$arg"
let ++i
done
## Example of calls to the previous script
#caller.sh '1' "2" 3 "4 4" "5 5"
#arg1=1
#arg2=2
#arg3=3
#arg4=4 4
#arg5=5 5
#Flimm is correct, there is no way to distinguish between arguments "foo" and foo, simply because the quotes are removed by the shell before the program receives them. What you need is "$#" (with the quotes).
#!/usr/local/bin/bash
out=`grep apache README`
echo $out;
Usually grep shows each match on a separate line when run on the command line. However, in the above scripts, the newline separating each match disappears. Does anyone know how the newline can be preserved?
You're not losing it in the assignment but in the echo. You can see this clearly if you:
echo "${out}"
You'll see a similar effect with the following script:
x="Hello,
I
am
a
string
with
newlines"
echo "====="
echo ${x}
echo "====="
echo "${x}"
echo "====="
which outputs:
=====
Hello, I am a string with newlines
=====
Hello,
I
am
a
string
with
newlines
=====
And, irrelevant to your question but I'd like to mention it anyway, I prefer to use the $() construct rather than backticks, just for the added benefit of being able to nest commands. So your script line becomes:
out=$(grep apache README)
Now that may not look any different (and it isn't) but it makes possible more complex commands like:
lines_with_nine=$(grep $(expr 7 + 2) inputfile)
Put $out in quotes:
#!/usr/local/bin/bash
out=`grep apache README`
echo "$out";
Quoting variables in bash preserves the whitespace.
For instance:
#!/bin/bash
var1="A B C D"
echo $var1 # A B C D
echo "$var1" # A B C D
since newlines are whitespace they get "removed"
Combining other answers into a one liner:
echo "($(grep apache README))"
When we use shell expansion, it gives all the expanded word in one line. For example:
#!/bin/bash
data="Hello\ {World,Rafi}"
eval echo $data
This produces the following output:
Hello World Hello Rafi
Is it possible to output each line on a separate line like this?
Hello World
Hello Rafi
If I understand you right, you want to generate multiple words using brace expansion ({...}), then print each word on a separate line.
If you don't absolutely have to store "Hello\ {World,Rafi}" in a variable, you can do this with printf shell-builtin
printf "%s\n" "Hello "{Rafi,World}
Some explanation:
The format string (here: %s\n) is reused until all the arguments to printf is used up (Reference).
%s\n consumes 1 argument
"Hello "{Rafi,World} returns 2 words/arguments i.e. "Hello Rafi" and "Hello World"
So, this printf command is equivalent to
printf "%s\n%s\n" "Hello Rafi" "Hello World"
except you don't have to type all that up.
#!/bin/bash
data="Hello\ {World'\n',Rafi'\n',Kamal'\n'}"
eval echo -e "$data"
echo -e will evaluate newline characters.
Same as Antarus'a answer, except that echo has "-n". From http://unixhelp.ed.ac.uk/CGI/man-cgi?echo
-n do not output the trailing newline
#!/bin/bash
data="Hello\ {World,Rafi}'\n'"
eval echo -n -e "$data"
Actually, your problem is not the expansion but the echo command. Depending on your system, you might get what you want by
#!/bin/bash
data="Hello\ {World\\\\n,Rafi}"
eval echo -e "$data"
It is different solution but a very clean one.
#!/bin/bash
names="World Rafi"
for name in $names
do
echo Hello $name
done
Instead of using eval (which is dangerous and really not a good practice — see my comment in your post), another strategy would be to use an array. The following will do exactly what you want, in a clean and safe way:
data=( "Hello "{World,Rafi} )
printf "%s\n" "${data[#]}"
How do I print a newline? This merely prints \n:
$ echo -e "Hello,\nWorld!"
Hello,\nWorld!
Use printf instead:
printf "hello\nworld\n"
printf behaves more consistently across different environments than echo.
Make sure you are in Bash.
$ echo $0
bash
All these four ways work for me:
echo -e "Hello\nworld"
echo -e 'Hello\nworld'
echo Hello$'\n'world
echo Hello ; echo world
echo $'hello\nworld'
prints
hello
world
$'' strings use ANSI C Quoting:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
You could always do echo "".
For example,
echo "Hello,"
echo ""
echo "World!"
On the off chance that someone finds themselves beating their head against the wall trying to figure out why a coworker's script won't print newlines, look out for this:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
echo $(GET_RECORDS);
As in the above, the actual running of the method may itself be wrapped in an echo which supersedes any echos that may be in the method itself. Obviously, I watered this down for brevity. It was not so easy to spot!
You can then inform your comrades that a better way to execute functions would be like so:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
GET_RECORDS;
Simply type
echo
to get a new line
POSIX 7 on echo
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html
-e is not defined and backslashes are implementation defined:
If the first operand is -n, or if any of the operands contain a <backslash> character, the results are implementation-defined.
unless you have an optional XSI extension.
So I recommend that you should use printf instead, which is well specified:
format operand shall be used as the format string described in XBD File Format Notation [...]
the File Format Notation:
\n <newline> Move the printing position to the start of the next line.
Also keep in mind that Ubuntu 15.10 and most distros implement echo both as:
a Bash built-in: help echo
a standalone executable: which echo
which can lead to some confusion.
str='hello\nworld'
$ echo | sed "i$str"
hello
world
You can also do:
echo "hello
world"
This works both inside a script and from the command line.
On the command line, press Shift+Enter to do the line break inside the string.
This works for me on my macOS and my Ubuntu 18.04 (Bionic Beaver) system.
For only the question asked (not special characters etc) changing only double quotes to single quotes.
echo -e 'Hello,\nWorld!'
Results in:
Hello,
World!
There is a new parameter expansion added in Bash 4.4 that interprets escape sequences:
${parameter#operator} - E operator
The expansion is a string that is the value of parameter with
backslash escape sequences expanded as with the $'…' quoting
mechanism.
$ foo='hello\nworld'
$ echo "${foo#E}"
hello
world
I just use echo without any arguments:
echo "Hello"
echo
echo "World"
To print a new line with echo, use:
echo
or
echo -e '\n'
This could better be done as
x="\n"
echo -ne $x
-e option will interpret backslahes for the escape sequence
-n option will remove the trailing newline in the output
PS: the command echo has an effect of always including a trailing newline in the output so -n is required to turn that thing off (and make it less confusing)
My script:
echo "WARNINGS: $warningsFound WARNINGS FOUND:\n$warningStrings
Output:
WARNING : 2 WARNINGS FOUND:\nWarning, found the following local orphaned signature file:
On my Bash script I was getting mad as you until I've just tried:
echo "WARNING : $warningsFound WARNINGS FOUND:
$warningStrings"
Just hit Enter where you want to insert that jump. The output now is:
WARNING : 2 WARNINGS FOUND:
Warning, found the following local orphaned signature file:
If you're writing scripts and will be echoing newlines as part of other messages several times, a nice cross-platform solution is to put a literal newline in a variable like so:
newline='
'
echo "first line${newline}second line"
echo "Error: example error message n${newline}${usage}" >&2 #requires usage to be defined
If the previous answers don't work, and there is a need to get a return value from their function:
function foo()
{
local v="Dimi";
local s="";
.....
s+="Some message here $v $1\n"
.....
echo $s
}
r=$(foo "my message");
echo -e $r;
Only this trick worked on a Linux system I was working on with this Bash version:
GNU bash, version 2.2.25(1)-release (x86_64-redhat-linux-gnu)
You could also use echo with braces,
$ (echo hello; echo world)
hello
world
This got me there....
outstuff=RESOURCE_GROUP=[$RESOURCE_GROUP]\\nAKS_CLUSTER_NAME=[$AKS_CLUSTER_NAME]\\nREGION_NAME=[$REGION_NAME]\\nVERSION=[$VERSION]\\nSUBNET-ID=[$SUBNET_ID]
printf $outstuff
Yields:
RESOURCE_GROUP=[akswork-rg]
AKS_CLUSTER_NAME=[aksworkshop-804]
REGION_NAME=[eastus]
VERSION=[1.16.7]
SUBNET-ID=[/subscriptions/{subidhere}/resourceGroups/makeakswork-rg/providers/Microsoft.Network/virtualNetworks/aks-vnet/subnets/aks-subnet]
Sometimes you can pass multiple strings separated by a space and it will be interpreted as \n.
For example when using a shell script for multi-line notifcations:
#!/bin/bash
notify-send 'notification success' 'another line' 'time now '`date +"%s"`
With jq:
$ jq -nr '"Hello,\nWorld"'
Hello,
World
Additional solution:
In cases, you have to echo a multiline of the long contents (such as code/ configurations)
For example:
A Bash script to generate codes/ configurations
echo -e,
printf might have some limitation
You can use some special char as a placeholder as a line break (such as ~) and replace it after the file was created using tr:
echo ${content} | tr '~' '\n' > $targetFile
It needs to invoke another program (tr) which should be fine, IMO.