Concatenate variables to a String bash script - bash

I am trying to Concatenate variables with Strings in my bash script, the variables are being read independently but whenever I try to concatenate them, it doesn't recognize the variable values.
ex-
echo $CONFIG_PROTOCOL (Prints the variable value, HTTP)
echo $CONFIG_PROTOCOL'://'$CONFIG_SERVER_SOURCE:$CONFIG_PORT'/api/creation/objects/export?collection.ids='$sf
The above echo with the URL prints out /api/creations/objects/export?collection.ids=value_1, while it should print out http://localhost:8080/api/creation/objects/export?collection.ids=value_1
Any inputs will be appreciated.

This happens because $CONFIG_PORT has a trailing carriage return. Here's a MCVE:
CONFIG_PROTOCOL="http"
CONFIG_SERVER_SOURCE="example.com"
CONFIG_PORT="8080"
sf="42"
# Introduce bug:
CONFIG_PORT+=$'\r'
echo $CONFIG_PROTOCOL'://'$CONFIG_SERVER_SOURCE:$CONFIG_PORT'/api/creation/objects/export?collection.ids='$sf
When executed, this prints:
/api/creation/objects/export?collection.ids=42
When you comment out the buggy line, you get:
http://example.com:8080/api/creation/objects/export?collection.ids=42
In both cases, echo will appear to show the correct values because echo is a tool for showing text to humans and not useful for showing the underlying data. printf '%q\n' "$CONFIG_PORT" will instead show it in an unambiguous format:
$ echo $CONFIG_PORT
8080
$ printf '%q\n' "$CONFIG_PORT"
$'8080\r'
The best way to fix this is to ensure that whatever supplies the value does so correctly. But the easiest way is to just strip them:
echo $CONFIG_PROTOCOL'://'$CONFIG_SERVER_SOURCE:$CONFIG_PORT'/api/creation/objects/export?collection.ids='$sf | tr -d '\r'

echo "$CONFIG_PROTOCOL://$CONFIG_SERVER_SOURCE:$CONFIG_PORT/api/creation/objects/export?collection.ids=$sf"
Try above echo statement

Related

Escape charactors getting removed in echo statement

Original string getting mutated while printing with echo statement.
#!/bin/bash
response='{\\\"test\\\":\\\"data\\\"}'
echo $response;
Actual Output - {\\"test\\":\\"data\\"}
Expected output - {\\\"test\\\":\\\"data\\\"}
quote your variables (see https://mywiki.wooledge.org/Quotes)
use printf, not echo (see https://unix.stackexchange.com/q/65803/133219)
e.g.:
$ response='{\\\"test\\\":\\\"data\\\"}'
$ printf '%s\n' "$response"
{\\\"test\\\":\\\"data\\\"}
This works as expected in bash, but you are instead running it with sh. See: Why does my bash code fail when I run it with sh?
However, when you want to print a string exactly as is, use printf:
response='{\\\"test\\\":\\\"data\\\"}'
printf '%s\n' "$response"
This works correctly for all values in all shells, including response='*' reponse='-n' and response='foo bar'

How to write the content of a unknown shell variable to stdout in a safe way

I only know what I have read so far, and I am confused about how to actually echo a variable as is.
echo "$var" might fail if var='-n'
printf '%s\n' "$var" might fail because of shell not implementig printf
echo -- "$var" might fail because it is a gnu extension
So if i would have to guess:
echo x"$var"|sed 's#^x##1' would be the only way, but I have never encountered that pattern. Why?
As a concrete question:
for source; do
target="$(echo "$source"|sed 's#[^a-z0-9]\+#.#')"
# do stuff with $source and $target
done
Does this work, or could someone "hack" / "break" my script by putting a file named '-n' somewhere, assuming my script is executed by some my_script * cron?
How do I write echo "$var" so it does not break?
Does this work, or could someone "hack" / "break" my script by putting
a file named '-n' somewhere?
There is nothing wrong with:
target="$(echo "$source"|sed 's#[^a-z0-9]\+#.#')"
What is happening:
"$(...)" is a command substitution which will substitute the results of the command within as the value -- in which case the result is assigned to target.
echo "$source"|sed 's#[^a-z0-9]\+#.#' simply pipes the output of echo (e.g. what is in source) to sed for the simple substitution of every character not lowercase or a digit followed by + with a period 1. Note: the quotes ".." around $source ARE proper within the command substitution.
There is no inherent reason assigning -n to a variable will cause any mischief. What you do with the variable is another question, but suffice it to say it is hard to see any problem.
"POSIX-shell's out there not implementing printf" -- Huh? Any shell not implementing printf would be more an exception rather than the rule. See printf - The Open Group Library that is POSIX.
If you are attempting to printf output that begins with '-' simply precede the output with "--" to indicate End-of-Options before the string your want to print and things will go fine. With your example of "-n", printf is about the only way you will output a variable beginning with the single '-', for example:
$ t="-n"
$ printf -- "%s\n" "$t"
-n
(note: you don't have to include "--" in printf "%s\n" "$var", the only time you must include it is with printf -- "-foo\n" or you will receive an "invalid option error".
For echo you can enable interpretation of backslash escapes with -e and include a backspace, e.g.
$ echo -e " \b$t"
-n
I think that has covered all issues. If not, let me know. Also, if you have any additional questions, drop a comment below or edit and add to your question.
footnotes:
note: + isn't part of basic regular expressions and it need not be escaped, but if there is any question, it is safer to include in a character class of its own, e.g. [^a-z0-9][+].

bash printf: ignore parameters or access by index?

I have a bash function that fetches values (using curl and cut) and creates a file name from them. Now I want to support a second naming scheme that needs a different set of parameters.
Example:
#!/bin/bash
TEMPLATE="%02i. %s.txt"
foo() {
a="Imagine these to"
b="be set dynamically"
c="42"
filename="$(printf "$TEMPLATE" "$c" "$a")"
# second: filename="$a - $b.txt"
# or: filename="$(printf "%s - %s.txt" "$a" "$b")"
echo "$filename"
# generate file
}
# actual script loops over:
foo
One of the values is a number that should be padded with leading zeros if required, thus printf in the current implementation.
Is there a way to implement this with just setting a different template globally? This would require that the template can access parameters by index or at least skip some of them.
If not, what are my alternatives? The template is to be chosen by command line parameter and does not change after initialization.
What does not work:
bash man page suggests that zero length output is not possible (to skip values)
C's printf man page mentions a "%m$" construct, which apparently is not supported in bash
the function itself generates the values, so it cannot receive the full filename as parameter
If you need to skip an argument, you can use %.s. Examples:
$ printf "%.s%s\n" "Bonjour" "Hello"
Hello
$ printf "%s%.s\n" "Bonjour" "Hello"
Bonjour
AFAIK, you can't access arguments by index.
If you need to store the formated string in a variable, please don't use a subshell as in:
$ variable=$(printf "%.s%s" "Bonjour" "Hello")
$ echo "$variable"
Hello
Instead, use the -v option to printf (type help printf to have the details) as in:
$ printf -v variable "%.s%s" "Bonjour" "Hello"
$ echo "$variable"
Hello
Also, if your template can come from user input, I would advise you to add -- just before it (this is to end the options of the command, just in case a user wants to start a template with a dash). Hence I would replace you line
filename="$(printf "$TEMPLATE" "$c" "$a")"
with
printf -v filename -- "$TEMPLATE" "$c" "$a"
Finally, having upper case variable names is considered bad bash practice.

Echo newline in Bash prints literal \n

How do I print a newline? This merely prints \n:
$ echo -e "Hello,\nWorld!"
Hello,\nWorld!
Use printf instead:
printf "hello\nworld\n"
printf behaves more consistently across different environments than echo.
Make sure you are in Bash.
$ echo $0
bash
All these four ways work for me:
echo -e "Hello\nworld"
echo -e 'Hello\nworld'
echo Hello$'\n'world
echo Hello ; echo world
echo $'hello\nworld'
prints
hello
world
$'' strings use ANSI C Quoting:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
You could always do echo "".
For example,
echo "Hello,"
echo ""
echo "World!"
On the off chance that someone finds themselves beating their head against the wall trying to figure out why a coworker's script won't print newlines, look out for this:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
echo $(GET_RECORDS);
As in the above, the actual running of the method may itself be wrapped in an echo which supersedes any echos that may be in the method itself. Obviously, I watered this down for brevity. It was not so easy to spot!
You can then inform your comrades that a better way to execute functions would be like so:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
GET_RECORDS;
Simply type
echo
to get a new line
POSIX 7 on echo
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html
-e is not defined and backslashes are implementation defined:
If the first operand is -n, or if any of the operands contain a <backslash> character, the results are implementation-defined.
unless you have an optional XSI extension.
So I recommend that you should use printf instead, which is well specified:
format operand shall be used as the format string described in XBD File Format Notation [...]
the File Format Notation:
\n <newline> Move the printing position to the start of the next line.
Also keep in mind that Ubuntu 15.10 and most distros implement echo both as:
a Bash built-in: help echo
a standalone executable: which echo
which can lead to some confusion.
str='hello\nworld'
$ echo | sed "i$str"
hello
world
You can also do:
echo "hello
world"
This works both inside a script and from the command line.
On the command line, press Shift+Enter to do the line break inside the string.
This works for me on my macOS and my Ubuntu 18.04 (Bionic Beaver) system.
For only the question asked (not special characters etc) changing only double quotes to single quotes.
echo -e 'Hello,\nWorld!'
Results in:
Hello,
World!
There is a new parameter expansion added in Bash 4.4 that interprets escape sequences:
${parameter#operator} - E operator
The expansion is a string that is the value of parameter with
backslash escape sequences expanded as with the $'…' quoting
mechanism.
$ foo='hello\nworld'
$ echo "${foo#E}"
hello
world
I just use echo without any arguments:
echo "Hello"
echo
echo "World"
To print a new line with echo, use:
echo
or
echo -e '\n'
This could better be done as
x="\n"
echo -ne $x
-e option will interpret backslahes for the escape sequence
-n option will remove the trailing newline in the output
PS: the command echo has an effect of always including a trailing newline in the output so -n is required to turn that thing off (and make it less confusing)
My script:
echo "WARNINGS: $warningsFound WARNINGS FOUND:\n$warningStrings
Output:
WARNING : 2 WARNINGS FOUND:\nWarning, found the following local orphaned signature file:
On my Bash script I was getting mad as you until I've just tried:
echo "WARNING : $warningsFound WARNINGS FOUND:
$warningStrings"
Just hit Enter where you want to insert that jump. The output now is:
WARNING : 2 WARNINGS FOUND:
Warning, found the following local orphaned signature file:
If you're writing scripts and will be echoing newlines as part of other messages several times, a nice cross-platform solution is to put a literal newline in a variable like so:
newline='
'
echo "first line${newline}second line"
echo "Error: example error message n${newline}${usage}" >&2 #requires usage to be defined
If the previous answers don't work, and there is a need to get a return value from their function:
function foo()
{
local v="Dimi";
local s="";
.....
s+="Some message here $v $1\n"
.....
echo $s
}
r=$(foo "my message");
echo -e $r;
Only this trick worked on a Linux system I was working on with this Bash version:
GNU bash, version 2.2.25(1)-release (x86_64-redhat-linux-gnu)
You could also use echo with braces,
$ (echo hello; echo world)
hello
world
This got me there....
outstuff=RESOURCE_GROUP=[$RESOURCE_GROUP]\\nAKS_CLUSTER_NAME=[$AKS_CLUSTER_NAME]\\nREGION_NAME=[$REGION_NAME]\\nVERSION=[$VERSION]\\nSUBNET-ID=[$SUBNET_ID]
printf $outstuff
Yields:
RESOURCE_GROUP=[akswork-rg]
AKS_CLUSTER_NAME=[aksworkshop-804]
REGION_NAME=[eastus]
VERSION=[1.16.7]
SUBNET-ID=[/subscriptions/{subidhere}/resourceGroups/makeakswork-rg/providers/Microsoft.Network/virtualNetworks/aks-vnet/subnets/aks-subnet]
Sometimes you can pass multiple strings separated by a space and it will be interpreted as \n.
For example when using a shell script for multi-line notifcations:
#!/bin/bash
notify-send 'notification success' 'another line' 'time now '`date +"%s"`
With jq:
$ jq -nr '"Hello,\nWorld"'
Hello,
World
Additional solution:
In cases, you have to echo a multiline of the long contents (such as code/ configurations)
For example:
A Bash script to generate codes/ configurations
echo -e,
printf might have some limitation
You can use some special char as a placeholder as a line break (such as ~) and replace it after the file was created using tr:
echo ${content} | tr '~' '\n' > $targetFile
It needs to invoke another program (tr) which should be fine, IMO.

Shell scripting: cat vs echo for output

I've written a small script in bash that parses either the provided files or stdin if no file is given to produce some output. What is the best way to redirect the parsed output to stdout (at the end of the script the result is stored in a variable). Should I use cat or echo, or is there another preferred method?
Use the printf command:
printf '%s\n' "$var"
echo is ok for simple cases, but it can behave oddly for certain arguments. For example, echo has a -n option that tells it not to print a newline. If $var happens to be -n, then
echo "$var"
won't print anything. And there are a number of different versions of echo (either built into various shells or as /bin/echo) with subtly different behaviors.
echo. You have your parsed data in a variable, so just echo "$var" should be fine. cat is used to print the contents of files, which isn't what you want here.
echo is a fine way to do it. You will have to jump through a few hoops if you want cat to work.

Resources