Delayed expansion of composite variable in Bash - bash

I'm defining a variable as a composition of other variables and some text, and I'm trying to get this variable to not expand its containing variables on the assigning. But I want it to expand when called later. That way I could reuse the same template to print different results as the inner variables keep changing. I'm truing to avoid eval as much as possible as I will be receiving some of the inner variables from third parties, and I do not know what to expect.
My use case, as below, is to have some "calling stack" so I can log all messages with the same format and keep a record of the script, function, and line of the logged message in some format like this: script.sh:this_function:42.
My attempted solution
called.sh:
#!/bin/bash
SCRIPT_NAME="`basename "${BASH_SOURCE[0]}"`"
CURR_STACK="${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}"
echo "${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}"
echo "${CURR_STACK}"
echo
function _func_1 {
echo "${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}"
echo "${CURR_STACK}"
}
_func_1
So, I intend to get the same results while printing the "${CURR_STACK}" as when printing the previous line.
If there is some built-in or other clever way to log this 'call stack', by all means, let me know! I'll gladly wave my code good-bye, but I'd still like to know how to prevent the variables from expanding right away on the assigning of CURR_STACK, but still keep them able to expand further ahead.
Am I missing some shopt?
What I've tried:
Case 1 (expanding on line 4):
CURR_STACK="${SCRIPT_NAME}:${FUNNAME[0]}:${LINENO[0]}"
CURR_STACK="`echo "${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}"`"
CURR_STACK="`echo "\${SCRIPT_NAME}:\${FUNCNAME[0]}:\${LINENO[0]}"`"
called.sh::7 <------------------| These are control lines
called.sh::4 <---------------. .------------| With the results I expect to get.
X
called.sh:_func_1:12 <---´ `-------| Both indicate that the values expanded
called.sh::4 <-------------------------| on line 4 - when CURR_STACK was set.
Case 2 (not expanding at all):
CURR_STACK="\${SCRIPT_NAME}:\${FUNNAME[0]}:\${LINENO[0]}"
CURR_STACK=\${SCRIPT_NAME}:\${FUNCNAME[0]}:\${LINENO[0]}
CURR_STACK="`echo '${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}'`"
called.sh::7
${SCRIPT_NAME}:${FUNNAME[0]}:${LINENO[0]} <-------.----| No expansion at all!...
/
called.sh::12 /
${SCRIPT_NAME}:${FUNNAME[0]}:${LINENO[0]} <----´

Shell variables are store plain inert text(*), not executable code; there isn't really any concept of delayed evaluation here. To make something that does something when used, create a function instead of a variable:
print_curr_stack() {
echo "$(basename "${BASH_SOURCE[1]}"):${FUNCNAME[1]}:${BASH_LINENO[0]}"
}
# ...
echo "We are now at $(print_curr_stack)"
# Or just run it directly:
print_curr_stack
Note: using BASH_SOURCE[1] and FUNCNAME[1] gets info about context the function was run from, rather than where it is in the function itself. But for some reason I'm not clear on, BASH_LINENO[1] gets the wrong info, and BASH_LINENO[0] is what you want.
You could also write it to allow the caller to specify additional text to print:
print_curr_stack() {
echo "$#" "$(basename "${BASH_SOURCE[1]}"):${FUNCNAME[1]}:${BASH_LINENO[0]}"
}
# ...
print_curr_stack "We are now at"
(* There's an exception to what I said about variables just contain inert text: some variables -- like $LINENO, $RANDOM, etc -- are handled specially by the shell itself. But you can't create new ones like this except by modifying the shell itself.)

Are you familiar with eval?
$ a=this; b=is; c=a; d=test;
$ e='echo "$a $b $c $d"';
$ eval $e;
this is a test
$ b='is NOT'; # modify one of the variables
$ eval $e;
this is NOT a test
$ f=$(eval $e); # capture the value of the "eval" statement
$ echo $f;
this is NOT a test

Related

Loading a while true loop into a variable

I'm having a bit of trouble getting this to work/ knowing if its possible. I'm creating a game using little other than bash, this requires a lot of repeated case statements. I am trying to load all the repeated case statements into a variable, then repeat them when necessary to limit the amount of work it will take to update the shared case statements between different scripts.
Here is what I have:
#!/bin/bash
moo="[m][o][o]) echo 'thank you for following instructions' ;;"
test=$(echo "while true ; do
read -p 'type moo: ' case
case $case in
$moo
*) echo 'type moo please'
esac
done")
"$test"
The problem I run into is:
./case.sh: line 13: $'while true ; do\nread -p \'type moo: \' case\ncase in\n[m][o][o]) echo \'thank you for following instructions\' ;;\n*) echo \'type moo please\' ;;\nesac\ndone': command not found
The information in the moo variable will eventually be in a separate script and will be set by invoking it as a function within that script when I finally get a working model.
It looks like this is a workable idea, I've just reached a loss on how to invoke the variable without it acting up. If anyone has any ideas, I would greatly appreciate it.
Thank you in advance!
It doesn't work because the quotes make the variable expansion be treated as a single word.
But it wouldn't work without quotes, either, because the shell doesn't parse the output of variables for syntax like semicolon and newline. Variable expansion is done after that stage of command parsing. The only processing that's done on expanded variables is word-splitting and wildcard matching.
You need to use eval to perform all command parsing:
eval "$test"
Another problem is that the variable $case is being expanded when you assign the variable test, it's not getting the value being read by read. Since the variable doesn't have a value yet, it's being executed as:
case in ...
and this is invalid syntax. You need to escape the $ so it will be passed through literally.
There's also no need for echo, you can simply assign the string directly.
test="while true ; do
read -p 'type moo: ' case
case \$case in
$moo
*) echo 'type moo please'
esac
done"

Deferred evaluation of bash variables

I need to define a string (options) which contains a variable (group) that is going to be available later in the script.
This is what I came up with, using a literal string that gets evaluated later.
#!/bin/bash
options='--group="$group"' #$group is not available at this point
#
# Some code...
#
group='trekkie'
eval echo "$options" # the result is used elsewhere
It works, however it makes use of eval which I would like to avoid if not absolutely necessary (I don't want to risk potential problems because of unpredictable data).
I've asked for help in multiple places and I've got a couple of answers that were directing me to use indirect variables.
The problem is I simply fail to see how indirect variables might help me with my problem. As far as I understand they only offer a way of indirectly referencing other variables like this:
options="--group="$group""
a=options
group='trekkies'
echo "${!a}" # spits out --group=
I would also like to avoid using functions if possible because I don't want to make things more complicated than they need to be.
More Idiomatic: Using Parameter Expansion
Don't attempt to define the --group="$group" argument up-front when you don't yet know the group name; instead, set a flag that indicates whether the argument is needed, and honor that flag when forming your final argument list.
By going the below approach, you avoid any need for "deferred evaluation":
#!/bin/bash
# initialize your flag as unset
unset needs_group
# depending on your application logic, optionally set that flag
if [[ $application_logic_here ]]; then
needs_group=1
fi
# ...so, the actual group can be defined later, when it's known...
group=trekkies
# and then check the flag to determine whether to pass the argument:
yourcommand ${needs_group+--group="$group"}
If you don't need the flag to be separate from the group variable, this is even easier:
# pass --group="$group" only if "$group" is a defined shell variable
yourcommand ${group+--group="$group"}
The relevant syntax is a parameter expansion: ${var+value} expands to value only if var is defined; and unlike most parameter expansions, its value can parse to multiple words with quoting applied.
Alternately: One-Liner Function Shims
Here, you really are defining --group="$group" before the group is known:
#!/bin/bash
if [[ $application_logic_here ]]; then
with_optional_group() { "$#" --group="$group"; }
else
with_optional_group() { "$#"; }
fi
group=trekkies
with_optional_group yourcommand

return a value from shell script, into another shell script

I know how to return an exit code, but I would like to return the result of an operation done in a shell script function, so I can eventually use it in another script or function.
Something like
var1=$(myfunction)
function2 var1
Where myfunction could be something like A+B=C
I looked into "return", but it will return a code, not a value.
I am looking into various sites that show how to write functions, but I don't see how you actually return values.
In C++ you would use return "variable name", but shell script won't allow this. It says that the variable do not exist (which is logical, it is a variable created in a function, so when the function is released, that memory space assigned to it is gone). Can't use global variables since the function may be in one script and the calling function that needs the return value, may be in a different one.
myfunction could be something like A+B=C
Just echo the result:
$ myfunction() { echo $(($1+$2)); }
The above myfunction adds two numbers and echoes the result.
The return value can then be captured just as you had it:
$ var=$(myfunction 12 5)
$ echo $var
17
The construct var=$(myfunction) captures the standard out from myfunction and saves it in var. Thus, when you want to return something from myfunction, just send it to standard, like we did with echo in the example above.
In cases where you want the return value to be carefully formatted, you should consider using printf in place of echo.
More: How to return multiple values
Let's define a function that produces two outputs:
$ f() { echo "output1" ; echo "output2" ; }
$ f
output1
output2
If you want to get those values back separately, the most reliable method is to use bash's arrays:
$ a=($(f))
The above executes f, via $(f) and saves the results in an array called a. We can see what is in a by using declare -p:
$ declare -p a
declare -a a='([0]="output1" [1]="output2")'
I use the same sorta thing for returning values from other scripts to my main script like the title suggests.
At the end of the 2nd script, I echo the variable I want to return to the main script:
#!/bin/bash
# This is the Second Script.
# Store the variables passed from the main script:
VAR1_FROM_MAIN_SCRIPT=$1
VAR2_FROM_MAIN_SCRIPT=$2
# Add the 2 variables and store as another variable to return:
RETURN_THIS=$(($VAR1_FROM_MAIN_SCRIPT + VAR2_FROM_MAIN_SCRIPT))
# This is the variable you are sending back to the main script:
echo "$RETURN_THIS" #<---- This won't print to screen!!!
Then in the main script I pass in a couple variables to, and execute, the 2nd script like this:
#!/bin/bash
# This is the Main Script.
PASS_VAR1_TO_SCRIPT=1
PASS_VAR2_TO_SCRIPT=2
# Call the second script and store it's results in this variable:
RETURN_VARIABLE=$(./secondScriptName "$PASS_VAR1_TO_SCRIPT" "$PASS_VAR2_TO_SCRIPT")
# Display the returned variable from the second script:
echo $RETURN_VARIABLE #<---- Will display 3
The reason the echo in the second script won't print to screen, is because it's running that second script in a subshell from the RETURN_VARIABLE... I know my explanation of the subshell sucks, but that's besides the point...
Also, I know you can source the other script, but this might help others.
In shell scripting you don't return a value but just echo (print) it and caller would capture the output of your script/function to grab the returned value.
Example:
dateval=$(date)
echo $dateval
Wed Apr 23 18:35:45 EDT 2014
Instead of date you can place your function or your shell script.

Shell scripting return values not correct, why?

In a shell script I wrote to test how functions are returning values I came across an odd unexpected behavior. The code below assumes that when entering the function fnttmpfile the first echo statement would print to the console and then the second echo statement would actually return the string to the calling main. Well that's what I assumed, but I was wrong!
#!/bin/sh
fntmpfile() {
TMPFILE=/tmp/$1.$$
echo "This is my temp file dude!"
echo "$TMPFILE"
}
mainname=main
retval=$(fntmpfile "$mainname")
echo "main retval=$retval"
What actually happens is the reverse. The first echo goes to the calling function and the second echo goes to STDOUT. why is this and is there a better way....
main retval=This is my temp file dude!
/tmp/main.19121
The whole reason for this test is because I am writing a shell script to do some database backups and decided to use small functions to do specific things, ya know make it clean instead of spaghetti code. One of the functions I was using was this:
log_to_console() {
# arg1 = calling function name
# arg2 = message to log
printf "$1 - $2\n"
}
The whole problem with this is that the function that was returning a string value is getting the log_to_console output instead depending on the order of things. I guess this is one of those gotcha things about shell scripting that I wasn't aware of.
No, what's happening is that you are running your function, and it outputs two lines to stdout:
This is my temp file dude!
/tmp/main.4059
When you run it $(), bash will intercept the output and store it in the value. The string that is stored in the variable contains the first linebreak (the last one is removed). So what is really in your "retval" variable is the following C-style string:
"This is my temp file dude!\n/tmp/main.4059"
This is not really returning a string (can't do that in a shell script), it's just capturing whatever output your function returns. Which is why it doesn't work. Call your function normally if you want to log to console.

Bash parameter quotes and eval

I've written a bash logging library to be implemented with some complex scripts that my company is currently using. I've been deadset on providing the script filename (${BASH_SOURCE}) and the line number (${LINENO}) of the calling script when making the log calls. However, I didn't want to have to rely on the user or implementing script to pass in these two variables as parameters. If this were C/C++, I would just create a macro that prepends "__FILE__" and "__LINE__" to the parameter list.
I was finally able to get this part working. Here are some much-simplified abstracts as a proof of concept:
Here's the logging library:
# log.sh
LOG="eval _log \${BASH_SOURCE} \${LINENO}"
_log () {
_BASH_SOURCE=`basename "${1}"` && shift
_LINENO=${1} && shift
echo "(${_BASH_SOURCE}:${_LINENO}) $#"
}
And an implementing test script:
# MyTest.sh
. ./log.sh
${LOG} "This is a log message"
# (test.sh:5) This is a log message
This works pretty well (and, I was thrilled to get it working at first). However, this has one glaring problem: the interaction between quotes and eval. If I make the call:
${LOG} "I'm thrilled that I got this working"
# ./test.sh: eval: line 5: unexected EOF while looking for matching `''
# ./test.sh: eval: line 6: syntax error: unexpected end of file
Now, I believe that I understand why this is happening. The quoted parameters are kept intact as they're passed to eval, but at that point, the content is placed as-is into the resulting command string. I know that I can fix this by doing some escaping; however, I REALLY do not want to force the implementing scripts to have to do this. Before I implemented this "eval macro" capability, I had users make calls directly to "_log" and allowed them to optionally pass in "${LINENO}." With this implementation, The failing call above (with only a quoted sentence) worked just fine.
At the most basic level, all I really want is for a script to be able to call [log function/macro] "String to log with special characers" and have the resulting log message contain the filename and line number of the calling script, followed by the log message. If it's possible, I would assume that I'm very close, but if there's something I'm overlooking that would require a different approach, I'm open to that as well. I can't force the users to escape all of their messages, as that will likely cause them to not use this library. This is such a problem that if I can't find a solution to this, I'll likely revert to the old capability (which required ${LINENO} to be passed as a function parameter - this was much less intrusive).
TLDR: Is there any way to get eval to respect special characters within a quoted parameter, without having to escape them?
I recommend avoiding eval if possible. For your logging use case, you could take a look at the shell builtin caller. If you need more information, you can use the variables BASH_SOURCE, BASH_LINENO and FUNCNAME. Note that all of these variables are arrays and contain the full call stack. See the following example:
#! /bin/bash
function log() {
echo "[$( caller )] $*" >&2
echo "BASH_SOURCE: ${BASH_SOURCE[*]}"
echo "BASH_LINENO: ${BASH_LINENO[*]}"
echo "FUNCNAME: ${FUNCNAME[*]}"
}
function foobar() {
log "failed:" "$#"
}
foobar "$#"
(Note: this solves the immediate problem of quoting, but #nosid's answer about
accessing the call stack is much better)
Change your definition of _log slightly, to read from standard input instead of
taking the log message from positional parameters:
_log () {
# Set up _BASH_SOURCE and _LINENO the same way
cat <(echo -n "$(_BASH_SOURCE:$_LINENO) ") -
}
Then pass your log message via standard input using a here doc or a here string:
${LOG} <<<"This is a log message"
${LOG} <<<"I'm thrilled this works, too!"
${LOG} <<HERE
Even this long
message works as
intended!
HERE
For your specific case of logging, you may want to look at this function which prints the caller's context's file function and line number.
Reusable quoting function
This function will do correct quoting for you:
function token_quote {
local quoted=()
for token; do
quoted+=( "$(printf '%q' "$token")" )
done
printf '%s\n' "${quoted[*]}"
}
Example usage:
Example usage:
$ token_quote token 'single token' token
token single\ token token
Above, note the single token's space is quoted as \.
$ set $(token_quote token 'single token' token)
$ eval printf '%s\\n' "$#"
token
single token
token
$
This shows that the spaces in tokens are preserved.

Resources