I have two shell scripts that I'd like to invoke from a C program. I would like shell variables set in the first script to be visible in the second. Here's what it would look like:
a.sh:
var=blah
<save vars>
b.sh:
<restore vars>
echo $var
The best I've come up with so far is a variant on "set > /tmp/vars" to save the variables and "eval $(cat /tmp/vars)" to restore them. The "eval" chokes when it tries to restore a read-only variable, so I need to grep those out. A list of these variables is available via "declare -r". But there are some vars which don't show up in this list, yet still can't be set in eval, e.g. BASH_ARGC. So I need to grep those out, too.
At this point, my solution feels very brittle and error-prone, and I'm not sure how portable it is. Is there a better way to do this?
One way to avoid setting problematic variables is by storing only those which have changed during the execution of each script. For example,
a.sh:
set > /tmp/pre
foo=bar
set > /tmp/post
grep -v -F -f/tmp/pre /tmp/post > /tmp/vars
b.sh:
eval $(cat /tmp/vars)
echo $foo
/tmp/vars contains this:
PIPESTATUS=([0]="0")
_=
foo=bar
Evidently evaling the first two lines has no adverse effect.
If you can use a common prefix on your variable names, here is one way to do it:
# save the variables
yourprefix_width=1200
yourprefix_height=2150
yourprefix_length=1975
yourprefix_material=gravel
yourprefix_customer_array=("Acme Plumbing" "123 Main" "Anytown")
declare -p $(echo ${!yourprefix#}) > varfile
# load the variables
while read -r line
do
if [[ $line == declare\ * ]]
then
eval "$line"
fi
done < varfile
Of course, your prefix will be shorter. You could do further validation upon loading the variables to make sure that the variable names conform to your naming scheme.
The advantage of using declare is that it is more secure than just using eval by itself.
If you need to, you can filter out variables that are marked as readonly or select variables that are marked for export.
Other commands of interest (some may vary by Bash version):
export - without arguments, lists all exported variables using a declare format
declare -px - same as the previous command
declare -pr - lists readonly variables
If it's possible for a.sh to call b.sh, it will carry over if they're exported. Or having a parent set all the values necessary and then call both. That's the most secure and sure method I can think of.
Not sure if it's accepted dogma, but:
bash -c 'export foo=bar; env > xxxx'
env `cat xxxx` otherscript.sh
The otherscript will have the env printed to xxxx ...
Update:
Also note:
man execle
On how to set environment variables for another system call from within C, if you need to do that. And:
man getenv
and http://www.crasseux.com/books/ctutorial/Environment-variables.html
An alternative to saving and restoring shell state would be to make the C program and the shell program work in parallel: the C program starts the shell program, which runs a.sh, then notifies the C program (perhaps passing some information it's learned from executing a.sh), and when the C program is ready for more it tells the shell program to run b.sh. The shell program would look like this:
. a.sh
echo "information gleaned from a"
arguments_for_b=$(read -r)
. b.sh
And the general structure of the C program would be:
set up two pairs of pipes, one for C->shell and one for shell->C
fork, exec the shell wrapper
read information gleaned from a on the shell->C pipe
more processing
write arguments for b on the C->shell pipe
wait for child process to end
I went looking for something similar and couldn't find it either, so I made the two scripts below. To start, just say shellstate, then probably at least set -i and set -o emacs which this reset_shellstate doesn't do for you. I don't know a way to ask bash which variables it thinks are special.
~/bin/reset_shellstate:
#!/bin/bash
__="$PWD/shellstate_${1#_}"
trap '
declare -p >"'"$__"'"
trap >>"'"$__"'"
echo cd \""$PWD"\" >>"'"$__"'" # setting PWD did this already, but...
echo set +abefhikmnptuvxBCEHPT >>"'"$__"'"
echo set -$- >>"'"$__"'" # must be last before sed, see $s/s//2 below
sed -ri '\''
$s/s//2
s,^trap --,trap,
/^declare -[^ ]*r/d
/^declare -[^ ]* [A-Za-z0-9_]*[^A-Za-z0-9_=]/d
/^declare -[^ ]* [^= ]*_SESSION_/d
/^declare -[^ ]* BASH[=_]/d
/^declare -[^ ]* (DISPLAY|GROUPS|SHLVL|XAUTHORITY)=/d
/^declare -[^ ]* WINDOW(ID|PATH)=/d
'\'' "'"$__"'"
shopt -op >>"'"$__"'"
shopt -p >>"'"$__"'"
declare -f >>"'"$__"'"
echo "Shell state saved in '"$__"'"
' 0
unset __
~/bin/shellstate:
#!/bin/bash
shellstate=shellstate_${1#_}
test -s $shellstate || reset_shellstate $1
shift
bash --noprofile --init-file shellstate_${1#_} -is "$#"
exit $?
Related
This is almost the exact same question as in this post, except that I do not want to use eval.
Quick question short, I want to execute the command echo aaa | grep a by first storing it in a string variable Command='echo aaa | grep a', and then running it without using eval.
In the post above, the selected answer used eval. That works for me too. What concerns me a lot is that there are plenty of warnings about eval below, followed by some attempts to circumvent it. However, none of them are able to solve my problem (essentially the OP's). I have commented below their attempts, but since it has been there for a long time, I suppose it is better to post the question again with the restriction of not using eval.
Concrete Example
What I want is a shell script that runs my command when I am happy:
#!/bin/bash
# This script run-this-if.sh runs the commands when I am happy
# Warning: the following script does not work (on nose)
if [ "$1" == "I-am-happy" ]; then
"$2"
fi
$ run-if.sh I-am-happy [insert-any-command]
Your sample usage can't ever work with an assignment, because assignments are scoped to the current process and its children. Because there's no reason to try to support assignments, things get suddenly far easier:
#!/bin/sh
if [ "$1" = "I-am-happy" ]; then
shift; "$#"
fi
This then can later use all the usual techniques to run shell pipelines, such as:
run-if-happy "$happiness" \
sh -c 'echo "$1" | grep "$2"' _ "$untrustedStringOne" "$untrustedStringTwo"
Note that we're passing the execve() syscall an argv with six elements:
sh (the shell to run; change to bash etc if preferred)
-c (telling the shell that the following argument is the code for it to run)
echo "$1" | grep "$2" (the code for sh to parse)
_ (a constant which becomes $0)
...whatever the shell variable untrustedStringOne contains... (which becomes $1)
...whatever the shell variable untrustedStringTwo contains... (which becomes $2)
Note here that echo "$1" | grep "$2" is a constant string -- in single-quotes, with no parameter expansions or command substitutions -- and that untrusted values are passed into the slots that fill in $1 and $2, out-of-band from the code being evaluated; this is essential to have any kind of increase in security over what eval would give you.
I executed a background process that was obtained as a parameter and didn't success to get the process's name after the execution.
I do the following:
#! /bin/bash
filePath=$1
$filePath > log.txt &
echo `jobs -l`
Actual result:
[1]+ 2381 Running $filePath > log.txt &
Expected result:
[1]+ 2381 Running /home/user/Desktop/script.sh > log.txt &
The best answer is don't; job control is a feature designed for interactive use, and is not guaranteed to be available at all in noninteractive shells, much less guaranteed to behave in any useful or meaningful way. However, if you insist, you can use printf %q to generate an eval-safe string with the post-expansion form of your variables, and then use eval to run it as code:
#!/bin/bash
printf -v logfile_q '%q' "${log:-log.txt}" # use "$logfile", or default to log.txt
printf -v cmd_q '%q ' "$#" # quote our arguments into one eval-safe string
eval "$cmd_str >$logfile_q &" # Parts that aren't hardcoded must be printf-q'd for safety.
jobs -l
Note that I added some extra configurability for the sake of demonstration -- it's okay to have >log.txt inside your eval'd code, but it's not safe to have >$logfile, because if logfile=$'foo$(rm -rf ~)\'$(rm -rf ~)\'' (a perfectly legal filename!) then you're going to lose your home directory. Thus, any variables needing to be used inside an argument to eval need to be escaped with printf %q beforehand.
To get started, here's the script I'm running to get the offending string:
# sed finds all sourced file paths from inputted file.
#
# while reads each match output from sed to $SOURCEFILE variable.
# Each should be a file path, or a variable that represents a file path.
# Any variables found should be expanded to the full path.
#
# echo and calls are used for demonstractive purposes only
# I intend to do something else with the path once it's expanded.
PATH_SOME_SCRIPT="/path/to/bash/script"
while read -r SOURCEFILE; do
echo "$SOURCEFILE"
"$SOURCEFILE"
$SOURCEFILE
done < <(cat $PATH_SOME_SCRIPT | sed -n -e "s/^\(source\|\.\|\$include\) //p")
You may also wish to use the following to test this out as mock data:
[ /path/to/bash/script ]
#!/bin/bash
source "$HOME/bash_file"
source "$GLOBAL_VAR_SCRIPT_PATH"
echo "No cow powers here"
For the tl;dr crew, basically the while loop spits out the following on the mock data:
"$HOME/bash_file"
bash: "$HOME/bash_file": no such file or directory
bash: "$HOME/bash_file": no such file or directory
"$GLOBAL_VAR_SCRIPT_PATH"
"$GLOBAL_VAR_SCRIPT_PATH": command not found
"$GLOBAL_VAR_SCRIPT_PATH": command not found
My question is, can you get the variable to expand correctly, e.g., print "/home//bash_file" and "/expanded/variable/path"? I should also state that although eval works I do not intend to use it because of its potential insecurities.
Protip that any variable value used in cat | sed would be available globally, including to the calling script, so it's not because the script cannot call the variable value.
FIRST SOLUTION ATTEMPT
Using anubhava's envsubst solution:
SOMEVARIABLE="/home/nick/.some_path"
while read -r SOURCEFILE; do
echo "$SOURCEFILE"
envsubst <<< "$SOURCEFILE";
done < <(echo -e "\"\$SOMEVARIABLE\"\n\"$HOME/.another_file\"")
This outputs the following:
"$SOMEVARIABLE"
""
"/home/nick/.another_file"
"/home/nick/.another_file"
Unfortunately, it does not expand the variable! Oh dear :(
SECOND SOLUTION ATTEMPT
Based upon the first attempt:
export SOMEVARIABLE="/home/nick/.some_path"
while read -r SOURCEFILE; do
echo "$SOURCEFILE"
envsubst <<< "$SOURCEFILE";
done < <(echo -e "\"\$SOMEVARIABLE\"\n\"$HOME/.another_file\"")
unset SOMEVARIABLE
which produces the results we wanted without eval and without messing with global variables (for too long anyway), hoorah!
Good runner-ups were further suggested using eval (although potentially unsafe) which can be found in this answer and here (link courtesy of anubhava's extended comments).
My question is, can you get the variable to expand correctly, e.g., print "/home//bash_file" and "/expanded/variable/path"?
Yes you can use envsubst program, that substitutes the values of environment variables:
while read -r sourceFile; do
envsubst <<< "$sourceFile"
done < <(sed -n "s/^\(source\|\.\|\$include\) //p" "$PATH_SOME_SCRIPT")
I think you are asking how to recursively expand variables in bash. Try
expanded=$(eval echo $SOURCEFILE)
inside your loop. eval runs the expanded command you give it. Since $SOURCEFILE isn't in quotes, it will be expanded to, e.g., $HOME/whatever. Then the eval will expand the $HOME before passing it to echo. echo will print the result, and expanded=$(...) will put the printed result in $expanded.
I have a bash script that I wish to read from a file to get it's arguments set. Basically my script reads arguments positionally ($1, $2, $3, etc.)
while test $# -gt 0; do
case $1 in
-h | --help)
echo "Help cruft"
exit 0
;;
esac
shift
done
One of the options I was hoping could be a config file that reads in arguments (for simple and easy config) so I was hoping the set -- command would work (-- to over ride the arguments). However, since they are defined in a file I have to read it in and use xargs to pass them:
-c | --config)
cat $2 | xargs set --
continue
;;
The trouble is that xargs buggers up the -- so I don't know how to accomplish this.
Note: I realize I could use source config_file and have it set variable; might be the final option. I wanted to know if I could do it like above and simplify the documentation.
A simplified example script:
# foo.sh
echo "x y z" | xargs set --
echo $*
# Command line
$ bash foo.sh a b c
xargs: set: No such file or directory
a b c
xargs can't execute set because:
set is a shell built-in, not an external command. xargs only knows how to execute commands. (Some shell built-ins shadow commands with the same name, such as printf, true, and [. So xargs can execute those commands, but the semantics might not be identical to the built-in.)
Even if xargs could execute set, it would have no effect because xargs does not run inside of the shell's environment; every command executed by xargs is a separate process. So you will get no error if you do this:
echo a b c | xargs bash -c 'set -- "${#}"' _
But it also won't do anything useful. (Substitute set with echo and you'll see that it does actually invoke the command.)
How to read arguments from a file.
First, you need to answer the question: what does it mean to have arguments in a file? Are they individual whitespace-separated words with no mechanism to include whitespace in any argument? (That would also be required for xargs to work in its default mode, so it is not a totally unreasonable assumption, although it is almost certainly going to get you into trouble at some point.)
In that case you don't need xargs at all; you can just use command substitution:
set -- $(<file)
While that will work fine, this won't:
echo a b c | set -- $(</dev/stdin)
because the pipeline (created by the | operator) causes the processes on either side to be run in subshells, and consequently the set doesn't modify the current shell's environment variables.
A more robust solution
Suppose that each argument is in a single line in the file, which makes it possible to include whitespace in an argument, but not a newline. Then we could use the useful mapfile built-in to read the arguments into an array, and set the positional arguments from the array. (Or just use the array directly, but that would be a different question.)
mapfile -t args < file
set -- "${args[#]}"
Again, watch out for piping into mapfile; it won't work, for the same reason that it didn't work with set.
I can redirect the output and then cat the file and grep/awk the variable, but I would like to use this file for multiple variables.
So If it was one variable say STATUS then i could do some thing like
echo "STATUS $STATUS" >> variable.file
#later perhaps in a remote shell where varible.file was copied
NEW_VAR=`cat variable.file | awk print '{$2}'`
I guess some inline editing with sed would help. The smaller the code the better.
One common way of storing variables in a file is to just store NAME=value lines in the file, and then just source that in to the shell you want to pick up the variables.
echo 'STATUS="'"$STATUS"'"' >> variable.file
# later
. variable.file
In Bash, you can also use source instead of ., though this may not be portable to other shells. Note carefully the exact sequence of quotes necessary to get the correct double quotes printed out in the file.
If you want to put multiple variables at once into the file, you could do the following. Apologies for the quoting contortions that this takes to do properly and portably; if you restrict yourself to Bash, you can use $"" to make the quoting a little simpler:
for var in STATUS FOO BAR
do
echo "$var="'"'"$(eval echo '$'"$var")"'"'
done >> variable.file
The declare builtin is useful here
for var in STATUS FOO BAR; do
declare -p $var | cut -d ' ' -f 3- >> filename
done
As Brian says, later you can source filename
declare is great because it handles quoting for you:
$ FOO='"I'"'"'m here," she said.'
$ declare -p FOO
declare -- FOO="\"I'm here,\" she said."
$ declare -p FOO | cut -d " " -f 3-
FOO="\"I'm here,\" she said."