can you please help me to understand what does this piece of code exactly do?
I guess it takes two arguments from shell script that this is part of, but I cannot quite grasp meaning of lines with //, %% and ##. Thanks in advance, for your help.
getEnvVal()
{
tNm=`grep $1 $2`
tNm=${tNm//$1}
tNm=${tNm%% }
tNm=${tNm## }
echo $tNm
}
tNm=`grep $1 $2` saves the output of grep to a variable
tNm=${tNm//$1} removes any occurrences of the value of $1 from the variable
tNm=${tNm%% } removes a single space from the end of the variable
tNm=${tNm## } removes a single space from the beginning of the variable
echo $tNm prints the variable value
I would not recommend you use this script, because there are many problems with it:
Uses obsolete `foo` construct. You should use $(foo).
Strips newlines from the end of the grep output. To get the literal output from grep you'll have to use something like this:
tNm=$(grep "$1" "$2"; printf x)
tNm=${tNm%x}
Variables are not quoted properly.
If $1 contains any characters which have special meaning in Bash, ${tNm//$1} will likely behave unexpectedly (thanks #chepner).
The author probably meant to remove multiple spaces from the start and end of the lines:
tNm=${tNm%% *}
tNm=${tNm## *}
echo would behave unexpectedly if the variable starts with a dash. Use printf instead.
The variable name is not meaningful.
Related
I'm writing a shell script that should be somewhat secure, i.e., does not pass secure data through parameters of commands and preferably does not use temporary files. How can I pass a variable to the standard input of a command?
Or, if it's not possible, how can I correctly use temporary files for such a task?
Passing a value to standard input in Bash is as simple as:
your-command <<< "$your_variable"
Always make sure you put quotes around variable expressions!
Be cautious, that this will probably work only in bash and will not work in sh.
Simple, but error-prone: using echo
Something as simple as this will do the trick:
echo "$blah" | my_cmd
Do note that this may not work correctly if $blah contains -n, -e, -E etc; or if it contains backslashes (bash's copy of echo preserves literal backslashes in absence of -e by default, but will treat them as escape sequences and replace them with corresponding characters even without -e if optional XSI extensions are enabled).
More sophisticated approach: using printf
printf '%s\n' "$blah" | my_cmd
This does not have the disadvantages listed above: all possible C strings (strings not containing NULs) are printed unchanged.
(cat <<END
$passwd
END
) | command
The cat is not really needed, but it helps to structure the code better and allows you to use more commands in parentheses as input to your command.
Note that the 'echo "$var" | command operations mean that standard input is limited to the line(s) echoed. If you also want the terminal to be connected, then you'll need to be fancier:
{ echo "$var"; cat - ; } | command
( echo "$var"; cat - ) | command
This means that the first line(s) will be the contents of $var but the rest will come from cat reading its standard input. If the command does not do anything too fancy (try to turn on command line editing, or run like vim does) then it will be fine. Otherwise, you need to get really fancy - I think expect or one of its derivatives is likely to be appropriate.
The command line notations are practically identical - but the second semi-colon is necessary with the braces whereas it is not with parentheses.
This robust and portable way has already appeared in comments. It should be a standalone answer.
printf '%s' "$var" | my_cmd
or
printf '%s\n' "$var" | my_cmd
Notes:
It's better than echo, reasons are here: Why is printf better than echo?
printf "$var" is wrong. The first argument is format where various sequences like %s or \n are interpreted. To pass the variable right, it must not be interpreted as format.
Usually variables don't contain trailing newlines. The former command (with %s) passes the variable as it is. However tools that work with text may ignore or complain about an incomplete line (see Why should text files end with a newline?). So you may want the latter command (with %s\n) which appends a newline character to the content of the variable. Non-obvious facts:
Here string in Bash (<<<"$var" my_cmd) does append a newline.
Any method that appends a newline results in non-empty stdin of my_cmd, even if the variable is empty or undefined.
I liked Martin's answer, but it has some problems depending on what is in the variable. This
your-command <<< """$your_variable"""
is better if you variable contains " or !.
As per Martin's answer, there is a Bash feature called Here Strings (which itself is a variant of the more widely supported Here Documents feature):
3.6.7 Here Strings
A variant of here documents, the format is:
<<< word
The word is expanded and supplied to the command on its standard
input.
Note that Here Strings would appear to be Bash-only, so, for improved portability, you'd probably be better off with the original Here Documents feature, as per PoltoS's answer:
( cat <<EOF
$variable
EOF
) | cmd
Or, a simpler variant of the above:
(cmd <<EOF
$variable
EOF
)
You can omit ( and ), unless you want to have this redirected further into other commands.
Try this:
echo "$variable" | command
If you came here from a duplicate, you are probably a beginner who tried to do something like
"$variable" >file
or
"$variable" | wc -l
where you obviously meant something like
echo "$variable" >file
echo "$variable" | wc -l
(Real beginners also forget the quotes; usually use quotes unless you have a specific reason to omit them, at least until you understand quoting.)
I've written a general purpose logger function in bash called trace() whose 3rd argument and beyond supposed to club and print as text, preserving any embedded newlines. But it doesn't. I'm looking for an effect similar to echo command that does it as intended (tried below).
Looks like the club-together thing is to blame (${#:3})?
timeStamp() { echo `date "+%Y-%m-%d %H:%M:%S:%3N %Z"` ;}
trace() {
lineNum=$1
traceType=$2
traceText=${#:3} #Input Parameters 3rd and beyond
#echo -en "[${lineNum}][$(timeStamp)][${traceType}]: ${traceText}"
#printf "[%s][%s][%s]: %s\n" ${lineNum} "$(timeStamp)" ${traceType} "${traceText}"
printf "[${lineNum}][$(timeStamp)][${traceType}]: ${traceText}"
}
Trials/Output:
$ trace $LINENO ERR "This is
a multiline
text
that's supposed to go
upto 5th line"
[342][2017-08-04 00:42:02:062 EDT][ERR]: This is a multiline text that's supposed to go upto 5th line
$ echo "This is a
> multiline text
> that's supposed
> to go
> upto 5th line"
This is a
multiline text
that's supposed
to go
upto 5th line
The problem is indeed in traceText=${#:3}. Normally, $# is equivalent to $1 $2 $3... (and ${#:3} to $3 $4 $5...), and since it isn't in double-quotes, each of those goes through word splitting and wildcard expansion. But in the case of var=$# it can't assign multiple values to var, so the word splitting and wildcard expansion gets suppressed... but apparently not completely. It's apparently doing enough word splitting to convert newlines to spaces.
I'm not clear if this is a bug or not; IMO it's a situation that doesn't really make sense because of the conflict between $# (treat each parameter as a separate item) and = (which can only assign one value). IMO what you should be using is traceText="${*:3}", which is unambiguous -- the double-quotes explicitly suppress word splitting and wildcard expansion, and the $* means take all the arguments stuck together with spaces (or whatever the first character of IFS is).
In my testing (with bash v3.2.57), all of these work as expected:
traceText=${*:3}
traceText="${*:3}"
traceText="${#:3}"
The only one that gives weird results is with # and without double-quotes.
In addition to the comment, you can also rewrite trace() altogether to include a command substitution for date and simply use the positional parameters in the printf format string. For example:
trace() {
printf "[$1]$(date "+%Y-%m-%d %H:%M:%S:%3N %Z")][$2]: ${#:3}"
}
Example Use/Output
$ trace 123 "some error" "this
is my
multi-line
text
"
[123]2017-08-04 01:01:29:765 CDT][some error]: this
is my
multi-line
text
note: there is a limit on the number of positional parameters that can be passed, so if your multi-line text exceeds the limit, trace will not work. See: Bash command line and input limit
I was looking to try and figure out how trim a string in Bash, from the trailing end, once I hit a certain character.
Example: if my string is this (or any link): https://www.cnpp.usda.gov/Innovations/DataSource/MyFoodapediaData.zip
(I'll set that as my variable).
(I.e. if I echo $var it will return that link:)
I'm looking to use Bash, I'm guessing I will need to utilize sed or awk, but I want to trim, starting from the end until I see the first / (since the will be the file name) and strip that out.
So using that link, I'm trying to just get after the / so jus "MyFoodapediaData.zip" and set that to a different variable.
So in the end, if I echo $var2 (if I call it that) it will just return: MyFoodapediaData.zip"
I tried working with sed 's.*/" and that will start from the beginning until it finds the first slash. I was looking for the reverse order if possible.
You can use bash builtin parameter substitution for this:
$ var='https://www.cnpp.usda.gov/Innovations/DataSource/MyFoodapediaData.zip'
$ echo "$var"
https://www.cnpp.usda.gov/Innovations/DataSource/MyFoodapediaData.zip
$ var2=${var##*/}
$ echo "$var2"
MyFoodapediaData.zip
${var##*/} means "from the beginning of the value of the var variable, remove everything up to the last slash."
See parameter substitution in the manual
I'm writing a script in bash and I want it to execute a command and to handle each line separately. for example:
LINES=$(df)
echo $LINES
it will return all the output converting new lines with spaces.
example:
if the output was supposed to be:
1
2
3
then I would get
1 2 3
how can I place the output of a command into a variable allowing new lines to still be new lines so when I print the variable i will get proper output?
Generally in bash $v is asking for trouble in most cases. Almost always what you really mean is "$v" in double quotes:
LINES="$(df)"
echo "$LINES"
No, it will not. The $(something) only strips trailing newlines.
The expansion in argument to echo splits on whitespace and than echo concatenates separate arguments with space. To preserve the whitespace, you need to quote again:
echo "$LINES"
Note, that the assignment does not need to be quoted; result of expansion is not word-split in assignment to variable and in argument to case. But it can be quoted and it's easier to just learn to just always put the quotes in.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Split string based on delimiter in Bash?
In a bash script how do I split string with a separator like ; and loop through the resulting array?
You can probably skip the step of explicitly creating an array...
One trick that I like to use is to set the inter-field separator (IFS) to the delimiter character. This is especially handy for iterating through the space or return delimited results from the stdout of any of a number of unix commands.
Below is an example using semicolons (as you had mentioned in your question):
export IFS=";"
sentence="one;two;three"
for word in $sentence; do
echo "$word"
done
Note: in regular Bourne-shell scripting setting and exporting the IFS would occur on two separate lines (IFS='x'; export IFS;).
If you don't wish to mess with IFS (perhaps for the code within the loop) this might help.
If know that your string will not have whitespace, you can substitute the ';' with a space and use the for/in construct:
#local str
for str in ${STR//;/ } ; do
echo "+ \"$str\""
done
But if you might have whitespace, then for this approach you will need to use a temp variable to hold the "rest" like this:
#local str rest
rest=$STR
while [ -n "$rest" ] ; do
str=${rest%%;*} # Everything up to the first ';'
# Trim up to the first ';' -- and handle final case, too.
[ "$rest" = "${rest/;/}" ] && rest= || rest=${rest#*;}
echo "+ \"$str\""
done
Here's a variation on ashirazi's answer which doesn't rely on $IFS. It does have its own issues which I ouline below.
sentence="one;two;three"
sentence=${sentence//;/$'\n'} # change the semicolons to white space
for word in $sentence
do
echo "$word"
done
Here I've used a newline, but you could use a tab "\t" or a space. However, if any of those characters are in the text it will be split there, too. That's the advantage of $IFS - it can not only enable a separator, but disable the default ones. Just make sure you save its value before you change it - as others have suggested.
Here is an example code that you may use:
$ STR="String;1;2;3"
$ for EACH in `echo "$STR" | grep -o -e "[^;]*"`; do
echo "Found: \"$EACH\"";
done
grep -o -e "[^;]*" will select anything that is not ';', therefore spliting the string by ';'.
Hope that help.
sentence="one;two;three"
a="${sentence};"
while [ -n "${a}" ]
do
echo ${a%%;*}
a=${a#*;}
done