Is there a way to suspend the execution of the shell script to inspect the state of the environment or execute random commands?
alias combined with eval gives you basic functionality of breakpoints in calling context:
#!/bin/bash
shopt -s expand_aliases
alias breakpoint='
while read -p"Debugging(Ctrl-d to exit)> " debugging_line
do
eval "$debugging_line"
done'
f(){
local var=1
breakpoint
echo $'\n'"After breakpoint, var=$var"
}
f
At the breakpoint, you can input
echo $var
followed by
var=2
then Ctrl-d to exit from breakpoint.
Due to eval in the while loop, use with caution.
Bash or shell scripts do not have such debugging capabilities as other programming languages like Java, Python, etc.
We can put the echo "VAR_NAME=$VAR_NAME" command in the code where we want to log the variable value.
Also, a little bit more flexible solution is to put this code somewhere at the beginning in the shell script we want to debug:
function BREAKPOINT() {
BREAKPOINT_NAME=$1
echo "Enter breakpoint $BREAKPOINT_NAME"
set +e
/bin/bash
BREAKPOINT_EXIT_CODE=$?
set -e
if [[ $BREAKPOINT_EXIT_CODE -eq 0 ]]; then
echo "Continue after breakpoint $BREAKPOINT_NAME"
else
echo "Terminate after breakpoint $BREAKPOINT_NAME"
exit $BREAKPOINT_EXIT_CODE
fi
}
export -f BREAKPOINT
and then later, at the line of code where we need to break we invoke this function like this:
# some shell script here
BREAKPOINT MyBreakPoint
# and some other shell script here
So then the BREAKPOINT function will log some output then launch /bin/bash where we can run any echo or some other shell command we want. When we want to continue running the rest of the shell script (release breakpoint) we just need to execute exit command. If we need to terminate script execution we would run exit 1 command.
There exist solutions like bash-debug.
A poor-man's solution which works for me is the interactive shell.
By adding three lines of code, you can introspect and alter variables as follows:
Let's assume, that you have the script test.bash
A=FOO
export B=BAR
echo $A
echo $B
$ test.bash
FOO
BAR
If you add an interactive shell at line 3, you can look around and inspect variables which have been exported before:
A=FOO
export B=BAR
bash -c "$SHELL"
echo $A
echo $B
$ test.bash
$ echo $A
$ echo $B
BAR
$ exit
FOO
BAR
If you want to see all variables in your interactive shell, you have to add set -a to the preamble of your script, such that all variables and functions are exported:
set -a
A=FOO
export B=BAR
bash -c "$SHELL"
echo $A
echo $B
$ test.bash
$ echo $A
FOO
$ echo $B
BAR
$ exit
FOO
BAR
Note, that you cannot change the variables in your interactive shell. The only solution for me is to source an additional script of variables, which will be sourced rightafter the interactive shell
set -a
A=FOO
export B=BAR
bash -c "$SHELL"
source /tmp/var
echo $A
echo $B
$ test.bash
$ echo "export A=alice" > /tmp/var
$ echo "export B=bob" >> /tmp/var
$ exit
alice
bob
In a shell script, how do I echo all shell commands called and expand any variable names?
For example, given the following line:
ls $DIRNAME
I would like the script to run the command and display the following
ls /full/path/to/some/dir
The purpose is to save a log of all shell commands called and their arguments. Is there perhaps a better way of generating such a log?
set -x or set -o xtrace expands variables and prints a little + sign before the line.
set -v or set -o verbose does not expand the variables before printing.
Use set +x and set +v to turn off the above settings.
On the first line of the script, one can put #!/bin/sh -x (or -v) to have the same effect as set -x (or -v) later in the script.
The above also works with /bin/sh.
See the bash-hackers' wiki on set attributes, and on debugging.
$ cat shl
#!/bin/bash
DIR=/tmp/so
ls $DIR
$ bash -x shl
+ DIR=/tmp/so
+ ls /tmp/so
$
set -x will give you what you want.
Here is an example shell script to demonstrate:
#!/bin/bash
set -x #echo on
ls $PWD
This expands all variables and prints the full commands before output of the command.
Output:
+ ls /home/user/
file1.txt file2.txt
I use a function to echo and run the command:
#!/bin/bash
# Function to display commands
exe() { echo "\$ $#" ; "$#" ; }
exe echo hello world
Which outputs
$ echo hello world
hello world
For more complicated commands pipes, etc., you can use eval:
#!/bin/bash
# Function to display commands
exe() { echo "\$ ${#/eval/}" ; "$#" ; }
exe eval "echo 'Hello, World!' | cut -d ' ' -f1"
Which outputs
$ echo 'Hello, World!' | cut -d ' ' -f1
Hello
You can also toggle this for select lines in your script by wrapping them in set -x and set +x, for example,
#!/bin/bash
...
if [[ ! -e $OUT_FILE ]];
then
echo "grabbing $URL"
set -x
curl --fail --noproxy $SERV -s -S $URL -o $OUT_FILE
set +x
fi
shuckc's answer for echoing select lines has a few downsides: you end up with the following set +x command being echoed as well, and you lose the ability to test the exit code with $? since it gets overwritten by the set +x.
Another option is to run the command in a subshell:
echo "getting URL..."
( set -x ; curl -s --fail $URL -o $OUTFILE )
if [ $? -eq 0 ] ; then
echo "curl failed"
exit 1
fi
which will give you output like:
getting URL...
+ curl -s --fail http://example.com/missing -o /tmp/example
curl failed
This does incur the overhead of creating a new subshell for the command, though.
According to TLDP's Bash Guide for Beginners: Chapter 2. Writing and debugging scripts:
2.3.1. Debugging on the entire script
$ bash -x script1.sh
...
There is now a full-fledged debugger for Bash, available at SourceForge. These debugging features are available in most modern versions of Bash, starting from 3.x.
2.3.2. Debugging on part(s) of the script
set -x # Activate debugging from here
w
set +x # Stop debugging from here
...
Table 2-1. Overview of set debugging options
Short | Long notation | Result
-------+---------------+--------------------------------------------------------------
set -f | set -o noglob | Disable file name generation using metacharacters (globbing).
set -v | set -o verbose| Prints shell input lines as they are read.
set -x | set -o xtrace | Print command traces before executing command.
...
Alternatively, these modes can be specified in the script itself, by
adding the desired options to the first line shell declaration.
Options can be combined, as is usually the case with UNIX commands:
#!/bin/bash -xv
Another option is to put "-x" at the top of your script instead of on the command line:
$ cat ./server
#!/bin/bash -x
ssh user#server
$ ./server
+ ssh user#server
user#server's password: ^C
$
You can execute a Bash script in debug mode with the -x option.
This will echo all the commands.
bash -x example_script.sh
# Console output
+ cd /home/user
+ mv text.txt mytext.txt
You can also save the -x option in the script. Just specify the -x option in the shebang.
######## example_script.sh ###################
#!/bin/bash -x
cd /home/user
mv text.txt mytext.txt
##############################################
./example_script.sh
# Console output
+ cd /home/user
+ mv text.txt mytext.txt
Type "bash -x" on the command line before the name of the Bash script. For instance, to execute foo.sh, type:
bash -x foo.sh
Combining all the answers I found this to be the best, simplest
#!/bin/bash
# https://stackoverflow.com/a/64644990/8608146
exe(){
set -x
"$#"
{ set +x; } 2>/dev/null
}
# example
exe go generate ./...
{ set +x; } 2>/dev/null from https://stackoverflow.com/a/19226038/8608146
If the exit status of the command is needed, as mentioned here
Use
{ STATUS=$?; set +x; } 2>/dev/null
And use the $STATUS later like exit $STATUS at the end
A slightly more useful one
#!/bin/bash
# https://stackoverflow.com/a/64644990/8608146
_exe(){
[ $1 == on ] && { set -x; return; } 2>/dev/null
[ $1 == off ] && { set +x; return; } 2>/dev/null
echo + "$#"
"$#"
}
exe(){
{ _exe "$#"; } 2>/dev/null
}
# examples
exe on # turn on same as set -x
echo This command prints with +
echo This too prints with +
exe off # same as set +x
echo This does not
# can also be used for individual commands
exe echo what up!
For zsh, echo
setopt VERBOSE
And for debugging,
setopt XTRACE
To allow for compound commands to be echoed, I use eval plus Soth's exe function to echo and run the command. This is useful for piped commands that would otherwise only show none or just the initial part of the piped command.
Without eval:
exe() { echo "\$ $#" ; "$#" ; }
exe ls -F | grep *.txt
Outputs:
$
file.txt
With eval:
exe() { echo "\$ $#" ; "$#" ; }
exe eval 'ls -F | grep *.txt'
Which outputs
$ exe eval 'ls -F | grep *.txt'
file.txt
For csh and tcsh, you can set verbose or set echo (or you can even set both, but it may result in some duplication most of the time).
The verbose option prints pretty much the exact shell expression that you type.
The echo option is more indicative of what will be executed through spawning.
http://www.tcsh.org/tcsh.html/Special_shell_variables.html#verbose
http://www.tcsh.org/tcsh.html/Special_shell_variables.html#echo
Special shell variables
verbose
If set, causes the words of each command to be printed, after history substitution (if any). Set by the -v command line option.
echo
If set, each command with its arguments is echoed just before it is executed. For non-builtin commands all expansions occur before echoing. Builtin commands are echoed before command and filename substitution, because these substitutions are then done selectively. Set by the -x command line option.
$ cat exampleScript.sh
#!/bin/bash
name="karthik";
echo $name;
bash -x exampleScript.sh
Output is as follows:
I have this code snippet running on several Linux boxes, and a Solaris 10 box with bash 3.6 (iirc). However, on a Solaris 11 box, with GNU bash, version 4.4.11(1)-release (sparc-sun-solaris2.11) it gives the following error.
#!/bin/env bash
CLEAN_COUNT() {
local L_STRING=$(sed '/[^[:graph:][:space:]]/{
s/[^[:graph:][:space:]]//g; s/\[[0-9]*m//g; s/(B//g
}' <<<$*) || return 1
echo ${#L_STRING}
}
f() {
ARGS=($#)
echo $((${#ARGS[1]:-0} - $(CLEAN_COUNT ${ARGS[1]:-0}) ))
}
f one two three four
Error received: ./gather_data.bash: line 15: ${#ARGS[1]:-0} - $(CLEAN_COUNT ${ARGS[1]:-0}) : bad substitution
I've isolated the above code in it's own script, I've compared the shopt and set -o settings on that box with another one. I'm perplexed. If I can get the code to work without the substitution, even if ARGS has no element 1 and I'm running set -o nounset, then I will use another piece of code.
Changes affecting this happened in Bash 4.3 and Bash 4.4. Observe:
No error in Bash 4.2:
$ docker run --rm -it bash:4.2 bash -u
bash-4.2$ bash --version | head -n 1
GNU bash, version 4.2.53(2)-release (x86_64-pc-linux-musl)
bash-4.2$ declare -a var && echo "${#var[1]:-1}"
0
but this doesn't actually print my default value: var[1] is the empty string, hence 0. -u seems to ignore that var has no elements. There is no difference in behaviour between echo "${#var[1]:-1}", echo "${#var[1]}" and echo "${#var[1]}", they all print 0.
Bash 4.3 complains about unbound variable:
$ docker run --rm -it bash:4.3 bash -u
bash-4.3$ bash --version | head -n 1
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-musl)
bash-4.3$ declare -a var && echo "${#var[1]:-1}"
bash: var: unbound variable
Bash 4.4 complains about substitution:
$ docker run --rm -it bash:4.4 bash -u
bash-4.4$ bash --version | head -n 1
GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-musl)
bash-4.4$ declare -a var && echo "${#var[1]:-1}"
bash: ${#var[1]:-1}: bad substitution
even without set -u:
bash-4.4# set +o nounset
bash-4.4# declare -a var && echo "${#var[1]:-1}"
bash: ${#var[1]:-1}: bad substitution
Also, ${#var:-1} is considered "bad substitution" in all versions, even without set -u:
$ for v in 3.2 4.{0..4}; do docker run --rm -it bash:$v; done
bash-3.2# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-3.2# exit
exit
bash-4.0# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.0# exit
exit
bash-4.1# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.1# exit
exit
bash-4.2# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.2# exit
exit
bash-4.3# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.3# exit
exit
bash-4.4# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.4# exit
exit
I can't see any mention of changes to this behaviour in NEWS, but it seems to make sense, as ${#var[0]:-1} doesn't default to 1 anyway, so now the behaviour is consistent across scalars and arrays.
This being said, I'd rewrite your function as follows:
f () {
local args=("$#")
if [[ -z ${args[1]:-} ]]; then
echo 0
else
echo $(( ${#args[1]} - $(clean_count "${args[1]}") ))
fi
}
Rename uppercase variable names to lowercase to avoid clash with shell and environment variables
Make args local to function
Quote "$#" in args to avoid splitting before assigning to array elements
Check if args[1] is the empty string, make sure no unset complaint is triggered with ${args[1]:-}
Treat cases for empty and non-empty string separately
Alternatively, if f () is not a simplification and you never access elements other than what you show, you could further simplify to
f () {
if [[ -z ${2:-} ]]; then
echo 0
else
echo $(( ${#2} - $(clean_count "$2") ))
fi
}
When cap3 try to execute comands on my FreeBSD server - I have an errors and my cap3 tasks doesn't work
DEBUG [0bb99d53] Command: if test ! -d /home/web_server/data/www/capistrano/site/shared/dumps; then echo "Directory does not exist '/home/web_server/data/www/capistrano/site/shared/dumps'" 1>&2; false; fi
DEBUG [0bb99d53] if: Expression Syntax.
DEBUG [0bb99d53] fi: Command not found.
And I know why - because my server use csh shell by default
% echo $0
-csh
Following cap3 variable doesn't work for me
set :shell, '/usr/local/bin/bash'
set :default_shell, '/usr/local/bin/bash'
How can I set shell for cap3 tasks?
You could set the default shell to bash for the deploy user on your server.
chsh -s /usr/local/bin/bash [your_deploy_user]
This worked for me.
My opinion is that you can attempt to use:
/bin/bash -c 'if test ! -d /home/web_server/data/www/capistrano/site/shared/dumps; then echo "Directory does not exist \'/home/web_server/data/www/capistrano/site/shared/dumps\'" 1>&2; false; fi'
From the Bash man page:
If the -c option is present, then commands are read from string. If
there are arguments after the string, they are assigned to the
positional parameters, starting with $0.
I did a test and it worked for me:
1#one:~>%bash -c "/bin/echo Hello; echo Goodbye"
Hello
Goodbye
Is there any way to run bash script X so that if X call executable bash script Y then Y starts by 'sh -eux'?
X.sh:
./Y.sh
Y.sh:
#!/bin/sh
echo OK
It is possible to make a subshell run using the same shell options set in the parent by exporting the SHELLOPTS environment variable.
In your case where X.sh and Y.sh cannot be edited, I'd create a wrapper script that simply exports SHELLOPTS before calling X.sh.
Example:
#!/bin/sh
# example X.sh which calls Y.sh
./Y.sh
.
#!/bin/sh
# example Y.sh which needs to be called using sh -eux
echo $SHELLOPTS
.
#!/bin/sh -eux
# wrapper.sh which sets the options for all sub shells
export SHELLOPTS
./X.sh
Calling X.sh directly shows that -eux options are not set in Y.sh
[lsc#aphek]$ ./X.sh
braceexpand:hashall:interactive-comments:posix
Calling it via wrapper.sh shows that the options have propagated to the subshells.
[lsc#aphek]$ ./wrapper.sh
+ export SHELLOPTS
+ ./x.sh
+ ./y.sh
+ echo braceexpand:errexit:hashall:interactive-comments:nounset:posix:xtrace
braceexpand:errexit:hashall:interactive-comments:nounset:posix:xtrace
Tested on GNU bash, version 3.00.15(1)-release. YMMV.
So i've got tool to debug shell scripts:
#!/bin/sh
# this tool allows to get full xtrace of any shell script execution
# only argument is script name
out=out.$1
err=err.$1
tmp=tmp.$1
echo "export SHELLOPTS; sh $# 1>> $out 2>> $err" > $tmp
sh -eux $tmp &> /dev/null
rm $tmp