In a shell script (in .zshrc) I am trying to execute a command that is stored as a string in another variable. Various sources on the web say this is possible, but I'm not getting the behavior i expect. Maybe it's the ~ at the beginning of the command, or maybe it's the use of sudo, I'm not sure. Any ideas? Thanks
function update_install()
{
# builds up a command as a string...
local install_cmd="$(make_install_command $#)"
# At this point the command as a string looks like: "sudo ~some_server/bin/do_install arg1 arg2"
print "----------------------------------------------------------------------------"
print "Will update install"
print "With command: ${install_cmd}"
print "----------------------------------------------------------------------------"
echo "trying backticks"
`${install_cmd}`
echo "Trying \$()"
$(${install_cmd})
echo "Trying \$="
$=install_cmd
}
Output:
Will update install
With command: sudo ~some_server/bin/do_install arg1 arg2
trying backticks
update_install:9: no such file or directory: sudo ~some_server/bin/do_install arg1 arg2
Trying $()
update_install:11: no such file or directory: sudo ~some_server/bin/do_install arg1 arg2
Trying $=
sudo ~some_server/bin/do_install arg1 arg2: command not found
Use eval:
eval ${install_cmd}
As explained in §3.1 "Why does $var where var="foo bar" not do what I expect?" of the Z-Shell FAQ, you can use the shwordsplit shell option to tell zsh that you want it to split variables up by spaces and treat them as multiple words. That same page also discusses alternatives that you might want to consider.
I believe you have two problems here - the first is that your install_cmd is being interpreted as a single string, instead of a command (sudo) with 3 arguments.
Your final attempt $=install_cmd actually does solve that problem correctly (though I'd write it as ${=install_cmd} instead), but then you hit your second problem: ~some_server/bin/do_install is not a known command. This is because sudo doesn't interpret the ~ like you intend, for safety reasons; it would need to evaluate its arguments using the shell (or do some special-casing for ~, which is really none of sudo's business), which opens up a whole can of worms that, understandably, sudo does its best to avoid.
That's also why it worked to do eval ${install_cmd} - because that's literally treating the whole string as a thing to be evaluated, possibly containing multiple commands (e.g. if install_cmd contained echo foo; sudo rm -rf / it would be happy to wipe your system).
You have to be the one to decide whether you want install_cmd to allow full shell semantics, including variable interpolation, path expansion, multiple commands, etc. or whether it should just expand the words out and run them as a single command.
Previous answers are correct, what worked for me was the below command in zsh
totalin=$(eval echo testing |wc -l)
echo "Total: $totalin"
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
Is there a way to debug a Bash script?
E.g., something that prints a sort of an execution log, like "calling line 1", "calling line 2", etc.
sh -x script [arg1 ...]
bash -x script [arg1 ...]
These give you a trace of what is being executed. (See also 'Clarification' near the bottom of the answer.)
Sometimes, you need to control the debugging within the script. In that case, as Cheeto reminded me, you can use:
set -x
This turns debugging on. You can then turn it off again with:
set +x
(You can find out the current tracing state by analyzing $-, the current flags, for x.)
Also, shells generally provide options '-n' for 'no execution' and '-v' for 'verbose' mode; you can use these in combination to see whether the shell thinks it could execute your script — occasionally useful if you have an unbalanced quote somewhere.
There is contention that the '-x' option in Bash is different from other shells (see the comments). The Bash Manual says:
-x
Print a trace of simple commands, for commands, case commands, select commands, and arithmetic for commands and their arguments or associated word lists after they are expanded and before they are executed. The value of the PS4 variable is expanded and the resultant value is printed before the command and its expanded arguments.
That much does not seem to indicate different behaviour at all. I don't see any other relevant references to '-x' in the manual. It does not describe differences in the startup sequence.
Clarification: On systems such as a typical Linux box, where '/bin/sh' is a symlink to '/bin/bash' (or wherever the Bash executable is found), the two command lines achieve the equivalent effect of running the script with execution trace on. On other systems (for example, Solaris, and some more modern variants of Linux), /bin/sh is not Bash, and the two command lines would give (slightly) different results. Most notably, '/bin/sh' would be confused by constructs in Bash that it does not recognize at all. (On Solaris, /bin/sh is a Bourne shell; on modern Linux, it is sometimes Dash — a smaller, more strictly POSIX-only shell.) When invoked by name like this, the 'shebang' line ('#!/bin/bash' vs '#!/bin/sh') at the start of the file has no effect on how the contents are interpreted.
The Bash manual has a section on Bash POSIX mode which, contrary to a long-standing but erroneous version of this answer (see also the comments below), does describe in extensive detail the difference between 'Bash invoked as sh' and 'Bash invoked as bash'.
When debugging a (Bash) shell script, it will be sensible and sane — necessary even — to use the shell named in the shebang line with the -x option. Otherwise, you may (will?) get different behaviour when debugging from when running the script.
I've used the following methods to debug my script.
set -e makes the script stop immediately if any external program returns a non-zero exit status. This is useful if your script attempts to handle all error cases and where a failure to do so should be trapped.
set -x was mentioned above and is certainly the most useful of all the debugging methods.
set -n might also be useful if you want to check your script for syntax errors.
strace is also useful to see what's going on. Especially useful if you haven't written the script yourself.
Jonathan Leffler's answer is valid and useful.
But, I find that the "standard" script debugging methods are inefficient, unintuitive, and hard to use. For those used to sophisticated GUI debuggers that put everything at your fingertips and make the job a breeze for easy problems (and possible for hard problems), these solutions aren't very satisfactory.
I do use a combination of DDD and bashdb. The former executes the latter, and the latter executes your script. This provides a multi-window UI with the ability to step through code in context and view variables, stack, etc., without the constant mental effort to maintain context in your head or keep re-listing the source.
There is guidance on setting that up in DDD and BASHDB.
I found the shellcheck utility and maybe some folks find it interesting.
A little example:
$ cat test.sh
ARRAY=("hello there" world)
for x in $ARRAY; do
echo $x
done
$ shellcheck test.sh
In test.sh line 3:
for x in $ARRAY; do
^-- SC2128: Expanding an array without an index only gives the first element.
Fix the bug. First try...
$ cat test.sh
ARRAY=("hello there" world)
for x in ${ARRAY[#]}; do
echo $x
done
$ shellcheck test.sh
In test.sh line 3:
for x in ${ARRAY[#]}; do
^-- SC2068: Double quote array expansions, otherwise they're like $* and break on spaces.
Let's try again...
$ cat test.sh
ARRAY=("hello there" world)
for x in "${ARRAY[#]}"; do
echo $x
done
$ shellcheck test.sh
Found now!
It's just a small example.
You can also write "set -x" within the script.
Install Visual Studio Code, and then add the Bash debug extension and you are ready to debug in visual mode. See it here in action.
Use Eclipse with the plugins Shelled and BashEclipse.
DVKit - Eclipse-based IDE for design verification tasks
BashEclipse
For Shelled: Download the ZIP file and import it into Eclipse via menu Help → Install new software: local archive. For BashEclipse: Copy the JAR files into the dropins directory of Eclipse
Follow the steps provided in BashEclipse files
I wrote a tutorial with many screenshots at Bash: enabling Eclipse for Bash Programming | Plugin Shelled (shell editor)
I built a Bash debugger: Bash Debuging Bash. Just give it a try.
set +x = #ECHO OFF, set -x = #ECHO ON.
You can add the -xv option to the standard shebang as follows:
#!/bin/bash -xv
-x : Display commands and their arguments as they are executed.
-v : Display shell input lines as they are read.
ltrace is another Linux utility similar to strace. However, ltrace lists all the library calls being called in an executable or a running process. Its name itself comes from library-call tracing.
For example:
ltrace ./executable <parameters>
ltrace -p <PID>
Source
I think you can try this Bash debugger: http://bashdb.sourceforge.net/.
Some trick to debug Bash scripts:
Using set -[nvx]
In addition to
set -x
and
set +x
for stopping dump.
I would like to speak about set -v which dump as smaller as less developed output.
bash <<<$'set -x\nfor i in {0..9};do\n\techo $i\n\tdone\nset +x' 2>&1 >/dev/null|wc -l
21
for arg in x v n nx nv nvx;do echo "- opts: $arg"
bash 2> >(wc -l|sed s/^/stderr:/) > >(wc -l|sed s/^/stdout:/) <<eof
set -$arg
for i in {0..9};do
echo $i
done
set +$arg
echo Done.
eof
sleep .02
done
- opts: x
stdout:11
stderr:21
- opts: v
stdout:11
stderr:4
- opts: n
stdout:0
stderr:0
- opts: nx
stdout:0
stderr:0
- opts: nv
stdout:0
stderr:5
- opts: nvx
stdout:0
stderr:5
Dump variables or tracing on the fly
For testing some variables, I use sometime this:
bash <(sed '18ideclare >&2 -p var1 var2' myscript.sh) args
for adding:
declare >&2 -p var1 var2
at line 18 and running the resulting script (with args), without having to edit them.
Of course, this could be used for adding set [+-][nvx]:
bash <(sed '18s/$/\ndeclare -p v1 v2 >\&2/;22s/^/set -x\n/;26s/^/set +x\n/' myscript) args
It will add declare -p v1 v2 >&2 after line 18, set -x before line 22 and set +x before line 26.
A little sample:
bash <(sed '2,3s/$/\ndeclare -p LINENO i v2 >\&2/;5s/^/set -x\n/;7s/^/set +x\n/' <(
seq -f 'echo $#, $((i=%g))' 1 8)) arg1 arg2
arg1 arg2, 1
arg1 arg2, 2
declare -i LINENO="3"
declare -- i="2"
/dev/fd/63: line 3: declare: v2: not found
arg1 arg2, 3
declare -i LINENO="5"
declare -- i="3"
/dev/fd/63: line 5: declare: v2: not found
arg1 arg2, 4
+ echo arg1 arg2, 5
arg1 arg2, 5
+ echo arg1 arg2, 6
arg1 arg2, 6
+ set +x
arg1 arg2, 7
arg1 arg2, 8
Note: Care about $LINENO. It will be affected by on-the-fly modifications!
(To see resulting script without executing, simply drop bash <( and ) arg1 arg2)
Step by step, execution time
Have a look at my answer about how to profile Bash scripts.
There's a good amount of detail on logging for shell scripts via the global variables of the shell. We can emulate a similar kind of logging in a shell script: Log tracing mechanism for shell scripts
The post has details on introducing log levels, like INFO, DEBUG, and ERROR. Tracing details like script entry, script exit, function entry, function exit.
Sample log:
In BASH, I use "pushd . " command to save the current directory on the stack.
After issuing this command in couple of different directories, I have multiple directories saved on the stack which I am able to see by issuing command "dirs".
For example, the output of "dirs" command in my current bash session is given below -
0 ~/eclipse/src
1 ~/eclipse
2 ~/parboil/src
Now, to switch to 0th directory, I issue a command "cd ~0".
I want to create a bash alias command or a function for this command.
Something like "xya 0", which will switch to 0th directory on stack.
I wrote following function to achieve this -
xya(){
cd ~$1
}
Where "$1" in above function, is the first argument passed to the function "xya".
But, I am getting the following error -
-bash: cd: ~1: No such file or directory
Can you please tell what is going wrong here ?
Generally, bash parsing happens in the following order:
brace expansion
tilde expansion
parameter, variable, arithmetic expansion; command substitution (same phase, left-to-right)
word splitting
pathname expansion
Thus, by the time your parameter is expanded, tilde expansion is already finished and will not take place again, without doing something explicit like use of eval.
If you know the risks and are willing to accept them, use eval to force parsing to restart at the beginning after the expansion of $1 is complete. The below tries to mitigate the damage should something that isn't eval-safe is passed as an argument:
xya() {
local cmd
printf -v cmd 'cd ~%q' "$1"
eval "$cmd"
}
...or, less cautiously (which is to say that the below trusts your arguments to be eval-safe):
xya() {
eval "cd ~$1"
}
You can let dirs print the absolute path for you:
xya(){
cd "$(dirs -${1-0} -l)"
}
UPDATE:
I made progress, but I'm still unsure what I'm not understanding. I've reduced my script down to a single line, and I still run into the same issue.
Any insight on the following?
NOTE: if I enter "/Users/Da.../Docker" or as it is below using a backspace for the whitespace, the result is the same.
for myvm in $(find /Users/David/Documents/Virtual\ Machines/Docker -name *.vmx); do echo -e "VM name: $myvm"; done
VM name: /Users/David/Documents/Virtual
VM name: Machines/Docker/RHEL7-DockerHost-Node04.vmwarevm/RHEL7-DockerHost-Node04.vmx
...
VM name: /Users/David/Documents/Virtual
VM name: Machines/Docker/RHEL7-DockerHost.vmwarevm/RHEL7-DockerHost.vmx
VM name: /Users/David/Documents/Virtual
VM name: Machines/Docker/RHEL7-DockerHost-Node01.vmwarevm/RHEL7-DockerHost-Node01.vmx
What am I missing?
--------------------------
I've found similar questions, but the answers aren't working for me.
I'm writing a script that will allow me to start and stop multiple VMs within VMware Fusion without having to click each one using the vmrun command, found at "/Applications/VMware Fusion.app/Contents/Library/vmrun".
Problem
When I try to alias vmrun:
alias myvmcli='/Applications/VMware\ Fusion.app/Contents/Library/vmrun'
...I get the following error from bash:
line 3: /Applications/VMware\: No such file or directory
Discussion
It obviously is missing the whitespace, but I thought using the backspace character would indicate that whitespace should be used.
I've tried single quotes and double quotes in the alias line to no avail.
If I run alias from the command line, it works fine.
Solution/Thoughts?
I'm sure that creating a link can solve my problem, but I really want to know why this isn't working.
You can avoid both functions and aliases by adding the directory to your path:
PATH="/Applications/VMware Fusion.app/Contents/Library:$PATH"
Then vmrun by itself will work.
You need to quote twice:
alias myvmcli='"/Applications/VMware Fusion.app/Contents/Library/vmrun"'
or escape:
alias myvmcli=\''/Applications/VMware Fusion.app/Contents/Library/vmrun'\'
or
alias myvmcli="\"/Applications/VMware Fusion.app/Contents/Library/vmrun\""
Be careful of the way the 2nd and 3rd are escaped. They differ.
You need the double quotation because you need to quote once because the alias command requires it (alias asd="echo asd"), and then once again because the command inside the alias requires it.
EDIT:
Even though I answered this, the alias you posted, works with me just fine. Could be due to differences in bash version though:
[ 0][s: ~ ]$ cd /tmp
[ 0][s: tmp ]$ alias asd='asd\ zxc/test'
[ 0][s: tmp ]$ asd
woooo!
A backslash \ is not required inside quotes, it is retained. You would only need the backslash if you didn't use quotes, but generally quotes are good.
$ echo 'Hello\ world'
Hello\ world
$ echo "Hello\ world"
Hello\ world
$ echo Hello\ world
Hello world
$ echo "Hello world"
Hello world
Using an alias inside a script is not good, aliases are really designed as productivity aids on the command-line.
In this case just assign it to a variable, you don't need an alias or a function.
cmd='/Applications/VMware Fusion.app/Contents/Library/vmrun'
Then, when you want to run it:
"$cmd"
Note the double quotes, required because of the embedded space.
Tested on OS X with vmware.
As others have said, something more complex than this would require a function.
I tried following code.
command1="echo"
"${command1}" 'case1'
command2="echo -e"
"${command2}" 'case2'
echo -e 'case3'
The outputs are following,
case1
echo -e: command not found
case3
The case2 results in an error but similar cases, case1 and case3 runs well. It seems command with option cannot be recognized as valid command.
I would like to know why it does not work. Please teach me. Thank you very much.
Case 1 (Unmodified)
command1="echo"
"${command1}" 'case1'
This is bad practice as an idiom, but there's nothing actively incorrect about it.
Case 2 (Unmodified)
command2="echo -e"
"${command2}" 'case2'
This is looking for a program named something like /usr/bin/echo -e, with the space as part of its name.
Case 2 (Reduced Quotes)
# works in this very specific case, but bad practice
command2="echo -e"
$command2 'case2' # WITHOUT THE QUOTES
...this one works, but only because your command isn't interesting enough (doesn't have quotes, doesn't have backslashes, doesn't have other shell syntax). See BashFAQ #50 for a description of why it isn't an acceptable practice in general.
Case X (eval -- Bad Practice, Oft Advised)
You'll often see this:
eval "$command1 'case1'"
...in this very specific case, where command1 and all arguments are hardcoded, this isn't exceptionally harmful. However, it's extremely harmful with only a small change:
# SECURITY BUGS HERE
eval "$command1 ${thing_to_echo}"
...if thing_to_echo='$(rm -rf $HOME)', you'll have a very bad day.
Best Practices
In general, commands shouldn't be stored in strings. Use a function:
e() { echo -e "$#"; }
e "this works"
...or, if you need to build up your argument list incrementally, an array:
e=( echo -e )
"${e[#]}" "this works"
Aside: On echo -e
Any implementation of echo where -e does anything other than emit the characters -e on output is failing to comply with the relevant POSIX standard, which recommends using printf instead (see the APPLICATION USAGE section).
Consider instead:
# a POSIX-compliant alternative to bash's default echo -e
e() { printf '%b\n' "$*"; }
...this not only gives you compatibility with non-bash shells, but also fixes support for bash in POSIX mode if compiled with --enable-xpg-echo-default or --enable-usg-echo-default, or if shopt -s xpg_echo was set, or if BASHOPTS=xpg_echo was present in the shell's environment at startup time.
If the variable command contains the value echo -e.
And the command line given to the shell is:
"$command" 'case2'
The shell will search for a command called echo -e with spaces included.
That command doesn't exist and the shell reports the error.
The reason of why this happen is depicted in the image linked below, from O'Reilly's Learning the Bash Shell, 3rd Edition:
Learning the bash Shell, 3rd Edition
By Cameron Newham
...............................................
Publisher: O'Reilly
Pub Date: March 2005
ISBN: 0-596-00965-8
Pages: 352
If the variable is quoted (follow the right arrows) it goes almost (passing steps 6,7, and 8) directly to execution in step 12.
Therefore, the command searched has not been split on spaces.
Original image (removed because of #CharlesDuffy complaint, I don't agree, but ok, let's move to the impossible to be in fault side) is here:
Link to original image in the web site where I found it.
If the command line given to the shell is un-quoted:
$command 'case2'
The string command gets expanded in step 6 (Parameter expansion) and then the value of the variable $command: echo -e gets divided in step 9: "Word splitting".
Then the shell search for command echo with argument -e.
The command echo "see" an argument of -e and echo process it as an option.
Trying to store commands inside an string is a very bad idea.
Try this, think very carefully of what you would expect the out put to be, and then be surprised on execution:
$ command='echo -e case2; echo "next line"'; $command
To take a look at what happens, execute the command as this:
$ set -vx; $command; set +vx
It works on my machine if I give the command this way:
cmd2="echo -e"
if you are still facing a problem I would suggest storing the options in another variable so that if you are doing shell scripting then multiple commands that use similar option values you can leverage the variable so also try something like this.
cmd1="echo"
opt1="-e"
$cmd1 $opt1 Hello
I often forget to run commands with sudo. I'm looking for a way to make a bash function (or alias) for repeating the last command with sudo. Something like:
S() {
sudo $(history 1)
}
Any ideas?
You can write:
sudo !!
(See §9.3 "History Expansion" in the Bash Reference Manual.)
Not enough?
sudo !!
if you want the S simply put:
alias S=sudo
and use it
S !!
the !! mean the last command
Use alias redo='sudo $(history -p !!)'. This is the only thing that I
have found that works with aliases. The other answers do not work in aliases for some reason, having tried them myself, even though they do work when directly running them.
!! can be used to reference the last command. So:
sudo !!
Adding and expanding upon aaron franke's response (which is correct but lacking a why) such that simply typing redo works after configuring the alias alias redo='sudo $(history -p !!);
This is the only thing I found that works with aliases
There are a few things going on that warrant explanation.
alias redo='sudo !!'
doesn't work as it doesn't execute the sudo portion, though it does resolve the !!.
To get the sudo !! to resolve correctly, a shell resolution directive has to execute and concatenate along with sudo. So we do;
alias redo='sudo $(history -p !!)'
By specifying the right side of the alias to $(history -p !!), what this does is say to the shell;
redo is an alias, assess the right side of the =
sudo remains as is and it concatenated to...
$() is a shell directive, to execute the contents within the current execution process (as oppose to a sub-shell ${}, which spawns a different process to execute within)
Resolve history -p !!
The !! gets expanded to history -p <lastCommandHere> as a result of step 4.
The -p part says to history to only print the results, not execute
The sudo, executes the remaining (now printed out on the same command line), with elevated privileges, assuming password was entered correctly
This ultimately means that the command history -p !! effectively writes out the resolution rather than executing after the sudo.
NOTE: The ' is significant. If you use ", the !! gets interpolated twice and to achieve our purpose we need to very carefully control the resolution steps; single/double quotes in bash
PS
If you're using zsh + oh-my-zsh, there is a sudo alias setup for _ such that you can do;
_ !!
Which will rerun the last command as sudo. However, if you're configuring the alias in ~/.zshrc, the alias redo='sudo $(history -p !!) will not work as is from the zsh config as the resolution steps are different.
That said, you can put the alias into ~/.bashrc instead of ~/.zshrc and have the same result when executing the alias redo from zsh (assuming you're sub-shelling zsh from bash, though this is admittedly a bit of a kludge - though a functional one).