Problem description
In a shell script, I want to iterate over all command line arguments ("$#") from inside a function. However, inside a function, $# refers to the function arguments, not the command line arguments. I tried passing the arguments to the function using a variable, but that doesn't help, since it breaks arguments with whitespaces.
How can I pass $# to a function in such a way that it does not break whitespace? I am sorry if this has been asked before, I tried searching for this question and there are a lot similar ones, but I didn't find an answer nevertheless.
Illustration
I made a shell script to illustrate the problem.
print_args.sh source listing
#!/bin/sh
echo 'Main scope'
for arg in "$#"
do
echo " $arg"
done
function print_args1() {
echo 'print_args1()'
for arg in "$#"
do
echo " $arg"
done
}
function print_args2() {
echo 'print_args2()'
for arg in $ARGS
do
echo " $arg"
done
}
function print_args3() {
echo 'print_args3()'
for arg in "$ARGS"
do
echo " $arg"
done
}
ARGS="$#"
print_args1
print_args2
print_args3
print_args.sh execution
$ ./print_args.sh foo bar 'foo bar'
Main scope
foo
bar
foo bar
print_args1()
print_args2()
foo
bar
foo
bar
print_args3()
foo bar foo bar
As you can see, I can't get the last foo bar to appear as a single argument. I want a function that gives the same output as the main scope.
You can use this BASH function:
#!/bin/bash
echo 'Main scope'
for arg in "$#"
do
echo " $arg"
done
function print_args1() {
echo 'print_args1()'
for arg in "$#"; do
echo " $arg"
done
}
function print_args3() {
echo 'print_args3()'
for arg in "${ARGS[#]}"; do
echo " $arg"
done
}
ARGS=( "$#" )
print_args1 "$#"
print_args3
You can see use of bash shebang at top:
#!/bin/bash
required be able to use BASH arrays.
Output:
bash ./print_args.sh foo bar 'foo bar'
Main scope
foo
bar
foo bar
print_args1()
foo
bar
foo bar
print_args3()
foo
bar
foo bar
Related
I want a shell script that can take advantage of a few optional flags in this style:
script.sh --foo "foo message" --bar "bar message"
script.sh --foo "foo message"
script.sh --bar "bar message"
script.sh --bar "bar message" --foo "foo message"
What would be the simplest methods of accessing these values (ex: "foo message" and "bar message" by their flag names (ex: foo and bar) ?
With bash, you can do something like:
#!/bin/bash
args=()
while test $# -gt 0; do
case $1 in
--foo) foo_msg=$2; shift;;
--bar) bar_msg=$2; shift;;
*) args+=($1);;
esac
shift
done
set -- "${args[#]}"
echo "remainging args: $#"
echo "foo_msg=$foo_msg"
echo "bar_msg=$bar_msg"
If you're using a shell without arrays, it's more difficult to retain the other arguments, but you can usually just consume them in a similar loop.
I can't understand why after attempt to call read after subshell my script stops, but works ok without susbshell call before read
command_not_found_handle()
{
(true) # subshell call
read line
echo "$line"
}
bash-4.4$ foo
[1]+ Stopped foo
And without subshell call it works fine:
command_not_found_handle()
{
read line
echo "$line"
}
bash-4.4$ foo
smth
smth
bash-4.4$
Upd. Fixed in bash 5.0
As #chepner said this appears to be a bug. They pointed out in a since-deleted answer that running fg allows the command to continue, which suggests Bash thinks command_not_found_handle is not connected to stdin.
Adding some debugging statements demonstrates that we get past the subshell but some state about the subshell leaks into the command_not_found_handle environment, causing Bash to think stdin isn't available to read when it clearly is.
subshell() {
echo "Start subshell"; (echo "In sub-shell"); echo "End subshell"
}
try_read() {
printf "Start try_read\nInput: "; read line; echo "End try_read $line"
}
command_not_found_handle() {
echo "Start CNFH $*"; subshell; try_read; echo "End CNFH"
}
This causes us to hang at read, after subshell has finished, until we fg the stopped job:
$ foo
Start CNFH foo
Start subshell
In sub-shell
End subshell
Start try_read
Input:
[1]+ Stopped foo
$ fg
foo
bar
End try_read bar
End CNFH
Removing the subshell call allows read to make progress as expected:
$ command_not_found_handle() { echo "Start CNFH $*"; try_read; echo "End CNFH"; }
$ foo
Start CNFH foo
Start try_read
Input: bar
End try_read bar
End CNFH
Why does the code
date
bash -c "date"
declare -x date='() { echo today; }' #aka export date='() { echo today; }'
date
bash -c "date"
print
Wed Sep 24 22:01:50 CEST 2014
Wed Sep 24 22:01:50 CEST 2014
Wed Sep 24 22:01:50 CEST 2014
today
?
Where (and why) does the evaluation
date$date
happen and getting
date() {echo today; }
Ad: #Etan Reisner
I exporting a variable - not a function. Bash makes a function from it. The
export date='someting'
is still a variable regardless of its content. So, why is
export date='() { echo something; }' #Note, it is a variable, not function.
converted to an function?
The mentioned security advisory talks about the execution of the command following the variable, for example,
x='() { echo I do nothing; }; echo vulnerable' bash -c ':'
^^^^^^^^^^^^^^^
This is executed - this vunerability is CLOSED in version 4.3.25(1).
The command after the env-definition isn't executed in the latest Bash.
But the question remains - Why does Bash convert the exported variable to a function?
It is a bug ;) Full demo, based on #chepner's answer:
#Define three variables
foo='() { echo variable foo; }' # ()crafted
qux='() { echo variable qux; }' # ()crafted
bar='variable bar' # Normal
export foo qux bar # Export
#Define the same name functions (but not qux!)
foo() { echo "function foo"; }
bar() { echo "function bar"; }
declare -fx foo bar #Export
#printouts
echo "current shell foo variable:=$foo="
echo "current shell foo function:=$(foo)="
echo "current shell bar variable:=$bar="
echo "current shell bar function:=$(bar)="
echo "current shell qux variable:=$qux="
echo "current shell qux function:=$(qux)="
#subshell
bash -c 'echo subshell foo variable:=$foo='
bash -c 'echo subshell foo command :=$(foo)='
bash -c 'echo subshell bar variable:=$bar='
bash -c 'echo subshell bar command :=$(bar)='
bash -c 'echo subshell qux variable:=$qux='
bash -c 'echo subshell qux command :=$(qux)='
prints
current shell foo variable:=() { echo variable foo; }=
current shell foo function:=function foo=
current shell bar variable:=variable bar=
current shell bar function:=function bar=
current shell qux variable:=() { echo variable qux; }=
tt: line 20: qux: command not found
current shell qux function:==
subshell foo variable:== #<-- LOST the exported foo variable
subshell foo command :=function foo=
subshell bar variable:=variable bar=
subshell bar command :=function bar=
subshell qux variable:== #<-- And the variable qux got converted to
subshell qux command :=variable qux= #<-- function qux in the subshell (!!!).
Avoiding the long comments, here is code from the Bash sources:
if (privmode == 0 && read_but_dont_execute == 0 && STREQN ("() {", string, 4))
^^^^^^^^ THE PROBLEM
{
string_length = strlen (string);
temp_string = (char *)xmalloc (3 + string_length + char_index);
strcpy (temp_string, name);
temp_string[char_index] = ' ';
strcpy (temp_string + char_index + 1, string);
if (posixly_correct == 0 || legal_identifier (name))
parse_and_execute (temp_string, name, SEVAL_NONINT|SEVAL_NOHIST);
/* Ancient backwards compatibility. Old versions of bash exported
functions like name()=() {...} */
The "ancient" (seems) was better... :)
if (name[char_index - 1] == ')' && name[char_index - 2] == '(')
name[char_index - 2] = '\0';
The key point to remember is that
foo='() { echo 5; }'
only defines a string parameter with a string that looks a lot like a function. It's still a regular string:
$ echo $foo
() { echo 5; }
And not a function:
$ foo
bash: foo: command not found
Once foo is marked for export,
$ export foo
any child Bash will see the following string in its environment:
foo=() { echo 5; }
Normally, such strings become shell variables, using the part preceding the = as the name and the part following the value. However, Bash treats such strings specially by defining a function instead:
$ echo $foo
$ foo
5
You can see that the environment itself is not changed by examining it with something other than Bash:
$ perl -e 'print $ENV{foo}\n"'
() { echo 5
}
(The parent Bash replaces the semicolon with a newline when creating the child's environment, apparently). It's only the child Bash that creates a function instead of a shell variable from such a string.
The fact that foo could be both a parameter and a function within the same shell;
$ foo=5
$ foo () { echo 9; }
$ echo $foo
5
$ foo
9
explains why -f is needed with export. export foo would cause the string foo=5 to be added to the environment of a child; export -f foo is used to add the string foo=() { echo 9; }.
You are essentially manually exporting a function with the name date. (Since that is the format that bash uses internally to export functions. Which is suggested by Barmar in his answer. This mechanism is mentioned here at the very least.)
Then when you run bash it sees that exported function and uses it when you tell it to run date.
Is the question then where is that mechanism specified? My guess is it isn't since it is an internal detail.
This should show the merging of the behaviours if that helps anything.
$ bar() { echo automatic; }; export -f bar
$ declare -x foo='() { echo manual; }'
$ declare -p foo bar
declare -x foo="() { echo manual; }"
-bash: declare: bar: not found
$ type foo bar
-bash: type: foo: not found
bar is a function
bar ()
{
echo automatic
}
$ bash -c 'type foo bar'
foo is a function
foo ()
{
echo manual
}
bar is a function
bar ()
{
echo automatic
}
The answer to your question comes directly from man bash:
The export and declare -x commands allow parameters and functions
to be added to and deleted from the environment. If the value of a
parameter in the environment is modified, the new value becomes part
of the environment, replacing the old.
Thus
declare -x date='() { echo today; }'
replaces date in the environment. The next immediate call to date gives date as it exists in the script (which is unchanged). The call to bash -c "date" creates a new shell and executes date as defined by declare -x.
I have one shell script that sources a second one within the context of a function taking parameters:
#!/bin/bash
# bar.sh
function f()
{
source foo.sh
echo "Do something else with $1, after foo.sh is sourced."
}
f bar
And:
#!/bin/bash
# foo.sh
x=${1:-"default"}
echo $x
Execution output is the following:
$ ./bar.sh
bar
Do something else with bar, after foo.sh is sourced.
I was expecting to get default as first line output instead of bar. So it turns out that even though I'm not passing any arguments to foo.sh, it is taking $1 from the context of function f. I can understand this behavior from reading bash documentation, but what would be the best way to override it?
EDIT: Based on your comment and edited question:
#!/bin/bash
# bar.sh
function f()
{
# save $1
arg1="$1"
# unset $1
shift
# source your script; prints default
source ./foo.sh
# restore $1
set -- $arg1
# should print bar
echo $1
echo "Do something else with $1, after foo.sh is sourced."
}
f bar
Can some one give an example where declare -x would be useful ?
declare -x FOO is the same as export FOO. It "exports" the FOO variable as an environment variable, so that programs you run from that shell session would see it.
Declare -x can be used instead of eval to allow variables to be set as arguments to the shell. For example, you can replace the extremely insecure:
# THIS IS NOT SAFE
while test $# -gt 0; do
eval export $1
shift
done
with the safer:
while test $# -gt 0; do
declare -x $1
shift
done
As an aside, this construct allows the user to invoke the script as:
$ ./test-script foo=bar
rather than the more idiomatic (but confusing to some):
$ foo=bar ./test-script
Use declare -x when you want to pass a variable to a different program, but don't want the variable to be used in global scope of the parent shell (i.e. when declaring inside a function).
From the bash help:
When used in a function, declare makes NAMEs local, as with the local
command. The -g option suppresses this behavior.
-x to make NAMEs export
Using + instead of - turns off the given attribute.
So declare -gx NAME=X will effectively behave the same as export NAME=X, but declare -x does not when the declare statements are inside functions.
The accepted solution is not accurate, as commented by #sampablokuper.
However both export variable a to subshells (below within ()).
test.sh:
#/bin/bash
foo() {
declare -x a="OK"
([ "$a" = "OK" ]) && echo "foo exported a locally"
}
bar() {
export a="OK"
([ "$a" = "OK" ]) && echo "bar exported a locally"
}
foo
echo "Global value of a by foo: ${a}"
([ "$a" = "OK" ]) && echo "foo exported a globally" \
|| echo 'foo did not export a globally'
bar
echo "Global value of a by bar: ${a}"
([ "$a" = "OK" ]) && echo "bar exported a globally" \
|| echo 'bar did not export a globally'
Runs as follows:
$ ./test.sh
foo exported a locally
Global value of a by foo:
foo did not export a globally
bar exported a locally
Global value of a by bar: OK
bar exported a globally