I need to extract some variables and functions from a zsh script into a bash script. Is there any way to do this? What I've tried (some are embarrassingly wrong, but covering everything):
. /script/path.zsh (zsh-isms exist, so it fails)
exec zsh
. /script/path.zsh
exec bash
zsh << 'EOF'
. /script/path.zsh
EOF
chsh -s zsh
. /script/path.zsh
chsh -s bash
This thread is the closest I've found. Unfortunately, I have too many items to import for that to be feasible, and neither script is anywhere near a polyglot. However, the functions and variables that I need to import are polyglots.
You can "scrape" the zsh source file for what you need, then execute the code in bash using eval. Here's an example for doing this for a few functions:
File script.zsh:
test1() {
echo "Hello from test1"
}
test2() {
echo $((1 + $1))
}
File script.sh (bash):
# Specify source script and functions
source_filename="script.zsh"
source_functions=" \
test1 \
test2 \
"
# Perform "sourcing"
function_definitions="$(python -B -c "
import re
with open('${source_filename}', mode='r') as file:
content = file.read()
for func in '${source_functions}'.split():
print(re.search(func + r'\(\).*?\n}', content, flags=re.DOTALL).group())
" )"
eval "${function_definitions}"
# Try out test functions
test1 # Hello from test1
n=5
echo "$n + 1 is $(test2 $n)" # 5 + 1 is 6
Run the bash script and it will make use of the functions test1 and test2 defined in the zsh script:
bash script.sh
The above makes use of Python, specifically its re module. It simply looks for character sequences of the form funcname(), and assumes that the function ends at the first }. So it's not very general, but works if you write your functions in this manner.
Related
This is one sample program in the GNU parallel documentation for executing via the shell script shebang.
#!/usr/bin/parallel --shebang-wrap --colsep " " /bin/bash
echo Arguments: $#
The output for
./bash_echo.parl gracias 'buenos dias'
is
gracias
buenos
dias
The above script does not handle command line arguments that are quoted and contain spaces. The arguments are expanded instead and treated as individual inputs.
How do I obtain the correct output as for the bash script below?
#!/usr/bin/env bash
for i in "$#"; do
echo "$i"
done
This, obviously, handles quoted command line args.
Output:
gracias
buenos dias
I've tried using the option 'colseps' setting the separator to ' ' but that isn't the solution.
You have found a bug: --shebang-wrap was never tested with spaces.
Possible fix:
diff --git a/src/parallel b/src/parallel
index 69adfdac..e7c0d930 100755
--- a/src/parallel
+++ b/src/parallel
## -3302,9 +3302,10 ## sub read_options() {
#options = shift #ARGV;
}
my $script = Q(shift #ARGV);
+ my #args = map{ Q($_) } #ARGV;
# exec myself to split $ARGV[0] into separate fields
- exec "$0 --_pipe-means-argfiles #options #parser $script ".
- "::: #ARGV";
+ exec "$0 --_pipe-means-argfiles #options #parser $script ".
+ "::: #args";
}
}
if($ARGV[0] =~ / --shebang(-?wrap)? /) {
It seems to fix your issue, but it may introduce others.
For updates: Follow https://savannah.gnu.org/bugs/index.php?63703
I have a custom 'runner'-script that I need to use to run all of my terminal commands. Below you can see the general idea in the script.
#!/usr/bin/env bash
echo "Running '$#'"
# do stuff before running cmd
$#
echo "Done"
# do stuff after running cmd
I can use the script in bash as follows:
$ ./run.sh echo test
Running 'echo test'
test
Done
$
I would like to use it like this:
$ echo test
Running 'echo test'
test
Done
$
Bash has the trap ... DEBUG and PROMPT_COMMAND, which lets me execute something before and after a command, but is there something that would allow me to execute instead of the command?
There is also the command_not_found_handle which would work if I had an empty PATH env variable, but that seems too dirty.
After some digging, I ended up looking at the source code and found that bash does not support custom executors. Below is a patch to add a new handle that works similarly as the command_not_found_handler.
diff --git a/eval.c b/eval.c
index f02d6e40..8d32fafa 100644
--- a/eval.c
+++ b/eval.c
## -52,6 +52,10 ##
extern sigset_t top_level_mask;
#endif
+#ifndef EXEC_HOOK
+# define EXEC_HOOK "command_exec_handle"
+#endif
+
static void send_pwd_to_eterm __P((void));
static sighandler alrm_catcher __P((int));
## -172,7 +176,15 ## reader_loop ()
executing = 1;
stdin_redir = 0;
- execute_command (current_command);
+ SHELL_VAR *hookf = find_function (EXEC_HOOK);
+ if (hookf == 0) {
+ execute_command (current_command);
+ } else {
+ char *command_to_print = make_command_string (current_command);
+ WORD_LIST *og = make_word_list(make_word(command_to_print), (WORD_LIST *)NULL);
+ WORD_LIST *wl = make_word_list(make_word(EXEC_HOOK), og);
+ execute_shell_function (hookf, wl);
+ }
exec_done:
QUIT;
One can then define function command_exec_handle() { eval $1; } which will be executed instead of the original command given in the prompt. The original command is fully in the first parameter. The command_exec_handle can be given in .bashrc and it works as expected.
Notice: this is very dangerous! If you mess up and put a bad command_exec_handler in your .bashrc, you might end up with a shell that does not execute commands. It will be quite hard to fix without booting from a live cd.
It seems you have the same problem listed here. If you want to run some commands if your original command was not found, the Bash 4's command_not_found_handler will certainly fit your needs.
Try to be more specific, maybe with some code snippets that do or do not work, in order to help us to help you...
A recent vulnerability, CVE-2014-6271, in how Bash interprets environment variables was disclosed. The exploit relies on Bash parsing some environment variable declarations as function definitions, but then continuing to execute code following the definition:
$ x='() { echo i do nothing; }; echo vulnerable' bash -c ':'
vulnerable
But I don't get it. There's nothing I've been able to find in the Bash manual about interpreting environment variables as functions at all (except for inheriting functions, which is different). Indeed, a proper named function definition is just treated as a value:
$ x='y() { :; }' bash -c 'echo $x'
y() { :; }
But a corrupt one prints nothing:
$ x='() { :; }' bash -c 'echo $x'
$ # Nothing but newline
The corrupt function is unnamed, and so I can't just call it. Is this vulnerability a pure implementation bug, or is there an intended feature here, that I just can't see?
Update
Per Barmar's comment, I hypothesized the name of the function was the parameter name:
$ n='() { echo wat; }' bash -c 'n'
wat
Which I could swear I tried before, but I guess I didn't try hard enough. It's repeatable now. Here's a little more testing:
$ env n='() { echo wat; }; echo vuln' bash -c 'n'
vuln
wat
$ env n='() { echo wat; }; echo $1' bash -c 'n 2' 3 -- 4
wat
…so apparently the args are not set at the time the exploit executes.
Anyway, the basic answer to my question is, yes, this is how Bash implements inherited functions.
This seems like an implementation bug.
Apparently, the way exported functions work in bash is that they use specially-formatted environment variables. If you export a function:
f() { ... }
it defines an environment variable like:
f='() { ... }'
What's probably happening is that when the new shell sees an environment variable whose value begins with (), it prepends the variable name and executes the resulting string. The bug is that this includes executing anything after the function definition as well.
The fix described is apparently to parse the result to see if it's a valid function definition. If not, it prints the warning about the invalid function definition attempt.
This article confirms my explanation of the cause of the bug. It also goes into a little more detail about how the fix resolves it: not only do they parse the values more carefully, but variables that are used to pass exported functions follow a special naming convention. This naming convention is different from that used for the environment variables created for CGI scripts, so an HTTP client should never be able to get its foot into this door.
The following:
x='() { echo I do nothing; }; echo vulnerable' bash -c 'typeset -f'
prints
vulnerable
x ()
{
echo I do nothing
}
declare -fx x
seems, than Bash, after having parsed the x=..., discovered it as a function, exported it, saw the declare -fx x and allowed the execution of the command after the declaration.
echo vulnerable
x='() { x; }; echo vulnerable' bash -c 'typeset -f'
prints:
vulnerable
x ()
{
echo I do nothing
}
and running the x
x='() { x; }; echo Vulnerable' bash -c 'x'
prints
Vulnerable
Segmentation fault: 11
segfaults - infinite recursive calls
It doesn't overrides already defined function
$ x() { echo Something; }
$ declare -fx x
$ x='() { x; }; echo Vulnerable' bash -c 'typeset -f'
prints:
x ()
{
echo Something
}
declare -fx x
e.g. the x remains the previously (correctly) defined function.
For the Bash 4.3.25(1)-release the vulnerability is closed, so
x='() { echo I do nothing; }; echo Vulnerable' bash -c ':'
prints
bash: warning: x: ignoring function definition attempt
bash: error importing function definition for `x'
but - what is strange (at least for me)
x='() { x; };' bash -c 'typeset -f'
STILL PRINTS
x ()
{
x
}
declare -fx x
and the
x='() { x; };' bash -c 'x'
segmentation faults too, so it STILL accept the strange function definition...
I think it's worth looking at the Bash code itself. The patch gives a bit of insight as to the problem. In particular,
*** ../bash-4.3-patched/variables.c 2014-05-15 08:26:50.000000000 -0400
--- variables.c 2014-09-14 14:23:35.000000000 -0400
***************
*** 359,369 ****
strcpy (temp_string + char_index + 1, string);
! if (posixly_correct == 0 || legal_identifier (name))
! parse_and_execute (temp_string, name, SEVAL_NONINT|SEVAL_NOHIST);
!
! /* Ancient backwards compatibility. Old versions of bash exported
! functions like name()=() {...} */
! if (name[char_index - 1] == ')' && name[char_index - 2] == '(')
! name[char_index - 2] = '\0';
if (temp_var = find_function (name))
--- 364,372 ----
strcpy (temp_string + char_index + 1, string);
! /* Don't import function names that are invalid identifiers from the
! environment, though we still allow them to be defined as shell
! variables. */
! if (legal_identifier (name))
! parse_and_execute (temp_string, name, SEVAL_NONINT|SEVAL_NOHIST|SEVAL_FUNCDEF|SEVAL_ONECMD);
if (temp_var = find_function (name))
When Bash exports a function, it shows up as an environment variable, for example:
$ foo() { echo 'hello world'; }
$ export -f foo
$ cat /proc/self/environ | tr '\0' '\n' | grep -A1 foo
foo=() { echo 'hello world'
}
When a new Bash process finds a function defined this way in its environment, it evalutes the code in the variable using parse_and_execute(). For normal, non-malicious code, executing it simply defines the function in Bash and moves on. However, because it's passed to a generic execution function, Bash will correctly parse and execute additional code defined in that variable after the function definition.
You can see that in the new code, a flag called SEVAL_ONECMD has been added that tells Bash to only evaluate the first command (that is, the function definition) and SEVAL_FUNCDEF to only allow functio0n definitions.
In regard to your question about documentation, notice here in the commandline documentation for the env command, that a study of the syntax shows that env is working as documented.
There are, optionally, 4 possible options
An optional hyphen as a synonym for -i (for backward compatibility I assume)
Zero or more NAME=VALUE pairs. These are the variable assignment(s) which could include function definitions.
Note that no semicolon (;) is required between or following the assignments.
The last argument(s) can be a single command followed by its argument(s). It will run with whatever permissions have been granted to the login being used. Security is controlled by restricting permissions on the login user and setting permissions on user-accessible executables such that users other than the executable's owner can only read and execute the program, not alter it.
[ spot#LX03:~ ] env --help
Usage: env [OPTION]... [-] [NAME=VALUE]... [COMMAND [ARG]...]
Set each NAME to VALUE in the environment and run COMMAND.
-i, --ignore-environment start with an empty environment
-u, --unset=NAME remove variable from the environment
--help display this help and exit
--version output version information and exit
A mere - implies -i. If no COMMAND, print the resulting environment.
Report env bugs to bug-coreutils#gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
Report env translation bugs to <http://translationproject.org/team/>
In the bash manual section 3.7.4, it states
The environment for any simple command or function may be augmented
temporarily by prefixing it with parameter assignments, as described
in Shell Parameters [section 3.4].
And a trivial example of this is
MYVAR=MYVALUE mycommand
I have read section 3.4 and I still can't figure out how to specify multiple parameter assignments. Note that the statement in 3.7.4 is definitely plural, implying that it is possible.
The following does not seem to work:
MYVAR1=abc MYVAR2=xyz mycommand
I am using bash version 4.1, 23 December 2009.
It should work. That is acceptable syntax. Here's an example:
$ cat a
#!/bin/sh
echo $MYVAR1
echo $MYVAR2
$ ./a
$ MYVAR1=abc MYVAR2=xyz ./a
abc
xyz
$
UPDATE: Your updated example given in your answer will work if you precede the simple command with the variables as required:
mycommand () { echo MYVAR1=[$MYVAR1]; echo MYVAR2=[$MYVAR2]; }
for f in ~/*.txt ; do MYVAR1=abc MYVAR2=xyz mycommand; done
Oops, my example was over-simplified. This works fine:
mycommand () { echo MYVAR1=[$MYVAR1]; echo MYVAR2=[$MYVAR2]; }
MYVAR1=abc MYVAR2=xyz mycommand
but this does not:
mycommand () { echo MYVAR1=[$MYVAR1]; echo MYVAR2=[$MYVAR2]; }
MYVAR1=abc MYVAR2=xyz for f in ~/*.txt; do mycommand; done
and the key difference is this phrase:
for any simple command or function
A for command is a complex command, not "a simple command or function", according to section 3.2.
I've never seen that method used.
You could always just pass in regular parameters to the script.
Updating for new example:
This works in both situations:
> mycommand () { echo MYVAR1=[$1]; echo MYVAR2=[$2]; }
> mycommand a b
MYVAR1=[a]
MYVAR2=[b]
> for f in ~/file*.txt; do mycommand a b; done
MYVAR1=[a]
MYVAR2=[b]
MYVAR1=[a]
MYVAR2=[b]
Here's a shell script:
globvar=0
function myfunc {
let globvar=globvar+1
echo "myfunc: $globvar"
}
myfunc
echo "something" | myfunc
echo "Global: $globvar"
When called, it prints out the following:
$ sh zzz.sh
myfunc: 1
myfunc: 2
Global: 1
$ bash zzz.sh
myfunc: 1
myfunc: 2
Global: 1
$ zsh zzz.sh
myfunc: 1
myfunc: 2
Global: 2
The question is: why this happens and what behavior is correct?
P.S. I have a strange feeling that function behind the pipe is called in a forked shell... So, can there be a simple workaround?
P.P.S. This function is a simple test wrapper. It runs test application and analyzes its output. Then it increments $PASSED or $FAILED variables. Finally, you get a number of passed/failed tests in global variables. The usage is like:
test-util << EOF | myfunc
input for test #1
EOF
test-util << EOF | myfunc
input for test #2
EOF
echo "Passed: $PASSED, failed: $FAILED"
Korn shell gives the same results as zsh, by the way.
Please see BashFAQ/024. Pipes create subshells in Bash and variables are lost when subshells exit.
Based on your example, I would restructure it something like this:
globvar=0
function myfunc {
echo $(($1 + 1))
}
myfunc "$globvar"
globalvar=$(echo "something" | myfunc "$globalvar")
Piping something into myfunc in sh or bash causes a new shell to spawn. You can confirm this by adding a long sleep in myfunc. While it's sleeping call ps and you'll see a subprocess. When the function returns, that sub shell exits without changing the value in the parent process.
If you really need that value to be changed, you'll need to return a value from the function and check $PIPESTATUS after, I guess, like this:
globvar=0
function myfunc {
let globvar=globvar+1
echo "myfunc: $globvar"
return $globvar
}
myfunc
echo "something" | myfunc
globvar=${PIPESTATUS[1]}
echo "Global: $globvar"
The problem is 'which end of a pipeline using built-ins is executed by the original process?'
In zsh, it looks like the last command in the pipeline is executed by the main shell script when the command is a function or built-in.
In Bash (and sh is likely to be a link to Bash if you're on Linux), then either both commands are run in a sub-shell or the first command is run by the main process and the others are run by sub-shells.
Clearly, when the function is run in a sub-shell, it does not affect the variable in the parent shell (only the global in the sub-shell).
Consider adding an extra test:
echo Something | { myfunc; echo $globvar; }
echo $globvar