In a shell script, how do I echo all shell commands called and expand any variable names?
For example, given the following line:
ls $DIRNAME
I would like the script to run the command and display the following
ls /full/path/to/some/dir
The purpose is to save a log of all shell commands called and their arguments. Is there perhaps a better way of generating such a log?
set -x or set -o xtrace expands variables and prints a little + sign before the line.
set -v or set -o verbose does not expand the variables before printing.
Use set +x and set +v to turn off the above settings.
On the first line of the script, one can put #!/bin/sh -x (or -v) to have the same effect as set -x (or -v) later in the script.
The above also works with /bin/sh.
See the bash-hackers' wiki on set attributes, and on debugging.
$ cat shl
#!/bin/bash
DIR=/tmp/so
ls $DIR
$ bash -x shl
+ DIR=/tmp/so
+ ls /tmp/so
$
set -x will give you what you want.
Here is an example shell script to demonstrate:
#!/bin/bash
set -x #echo on
ls $PWD
This expands all variables and prints the full commands before output of the command.
Output:
+ ls /home/user/
file1.txt file2.txt
I use a function to echo and run the command:
#!/bin/bash
# Function to display commands
exe() { echo "\$ $#" ; "$#" ; }
exe echo hello world
Which outputs
$ echo hello world
hello world
For more complicated commands pipes, etc., you can use eval:
#!/bin/bash
# Function to display commands
exe() { echo "\$ ${#/eval/}" ; "$#" ; }
exe eval "echo 'Hello, World!' | cut -d ' ' -f1"
Which outputs
$ echo 'Hello, World!' | cut -d ' ' -f1
Hello
You can also toggle this for select lines in your script by wrapping them in set -x and set +x, for example,
#!/bin/bash
...
if [[ ! -e $OUT_FILE ]];
then
echo "grabbing $URL"
set -x
curl --fail --noproxy $SERV -s -S $URL -o $OUT_FILE
set +x
fi
shuckc's answer for echoing select lines has a few downsides: you end up with the following set +x command being echoed as well, and you lose the ability to test the exit code with $? since it gets overwritten by the set +x.
Another option is to run the command in a subshell:
echo "getting URL..."
( set -x ; curl -s --fail $URL -o $OUTFILE )
if [ $? -eq 0 ] ; then
echo "curl failed"
exit 1
fi
which will give you output like:
getting URL...
+ curl -s --fail http://example.com/missing -o /tmp/example
curl failed
This does incur the overhead of creating a new subshell for the command, though.
According to TLDP's Bash Guide for Beginners: Chapter 2. Writing and debugging scripts:
2.3.1. Debugging on the entire script
$ bash -x script1.sh
...
There is now a full-fledged debugger for Bash, available at SourceForge. These debugging features are available in most modern versions of Bash, starting from 3.x.
2.3.2. Debugging on part(s) of the script
set -x # Activate debugging from here
w
set +x # Stop debugging from here
...
Table 2-1. Overview of set debugging options
Short | Long notation | Result
-------+---------------+--------------------------------------------------------------
set -f | set -o noglob | Disable file name generation using metacharacters (globbing).
set -v | set -o verbose| Prints shell input lines as they are read.
set -x | set -o xtrace | Print command traces before executing command.
...
Alternatively, these modes can be specified in the script itself, by
adding the desired options to the first line shell declaration.
Options can be combined, as is usually the case with UNIX commands:
#!/bin/bash -xv
Another option is to put "-x" at the top of your script instead of on the command line:
$ cat ./server
#!/bin/bash -x
ssh user#server
$ ./server
+ ssh user#server
user#server's password: ^C
$
You can execute a Bash script in debug mode with the -x option.
This will echo all the commands.
bash -x example_script.sh
# Console output
+ cd /home/user
+ mv text.txt mytext.txt
You can also save the -x option in the script. Just specify the -x option in the shebang.
######## example_script.sh ###################
#!/bin/bash -x
cd /home/user
mv text.txt mytext.txt
##############################################
./example_script.sh
# Console output
+ cd /home/user
+ mv text.txt mytext.txt
Type "bash -x" on the command line before the name of the Bash script. For instance, to execute foo.sh, type:
bash -x foo.sh
Combining all the answers I found this to be the best, simplest
#!/bin/bash
# https://stackoverflow.com/a/64644990/8608146
exe(){
set -x
"$#"
{ set +x; } 2>/dev/null
}
# example
exe go generate ./...
{ set +x; } 2>/dev/null from https://stackoverflow.com/a/19226038/8608146
If the exit status of the command is needed, as mentioned here
Use
{ STATUS=$?; set +x; } 2>/dev/null
And use the $STATUS later like exit $STATUS at the end
A slightly more useful one
#!/bin/bash
# https://stackoverflow.com/a/64644990/8608146
_exe(){
[ $1 == on ] && { set -x; return; } 2>/dev/null
[ $1 == off ] && { set +x; return; } 2>/dev/null
echo + "$#"
"$#"
}
exe(){
{ _exe "$#"; } 2>/dev/null
}
# examples
exe on # turn on same as set -x
echo This command prints with +
echo This too prints with +
exe off # same as set +x
echo This does not
# can also be used for individual commands
exe echo what up!
For zsh, echo
setopt VERBOSE
And for debugging,
setopt XTRACE
To allow for compound commands to be echoed, I use eval plus Soth's exe function to echo and run the command. This is useful for piped commands that would otherwise only show none or just the initial part of the piped command.
Without eval:
exe() { echo "\$ $#" ; "$#" ; }
exe ls -F | grep *.txt
Outputs:
$
file.txt
With eval:
exe() { echo "\$ $#" ; "$#" ; }
exe eval 'ls -F | grep *.txt'
Which outputs
$ exe eval 'ls -F | grep *.txt'
file.txt
For csh and tcsh, you can set verbose or set echo (or you can even set both, but it may result in some duplication most of the time).
The verbose option prints pretty much the exact shell expression that you type.
The echo option is more indicative of what will be executed through spawning.
http://www.tcsh.org/tcsh.html/Special_shell_variables.html#verbose
http://www.tcsh.org/tcsh.html/Special_shell_variables.html#echo
Special shell variables
verbose
If set, causes the words of each command to be printed, after history substitution (if any). Set by the -v command line option.
echo
If set, each command with its arguments is echoed just before it is executed. For non-builtin commands all expansions occur before echoing. Builtin commands are echoed before command and filename substitution, because these substitutions are then done selectively. Set by the -x command line option.
$ cat exampleScript.sh
#!/bin/bash
name="karthik";
echo $name;
bash -x exampleScript.sh
Output is as follows:
i have a bash script show below in a file called test.sh
#!/usr/bin/env bash
echo $1
echo "execution done"
when i execute this script using
Case-1
./test.sh "started"
started
execution done
showing properly
Case-2
If i execute with
bash test.sh "started"
i'm getting the out put as
started
execution done
But i would like to execute this using a cat or wget command with arguments
For example like.
Q1
cat test.sh |bash
Or using a command
Q2
wget -qO - "url contain bash" |bash
So in Q1 and Q2 how do i pass argument
Something simlar to this shown in this github
https://github.com/creationix/nvm
Please refer installation script
$ bash <(curl -Ls url_contains_bash_script) arg1 arg2
Explanation:
$ echo -e 'echo "$1"\necho "done"' >test.sh
$ cat test.sh
echo "$1"
echo "done"
$ bash <(cat test.sh) "hello"
hello
done
$ bash <(echo -e 'echo "$1"\necho "done"') "hello"
hello
done
You don't need to pipe to bash; bash runs as standard in your terminal.
If I have a script and I have to use cat, this is what I'll do:
cat script.sh > file.sh; chmod 755 file.sh; ./file.sh arg1 arg2 arg3
script.sh is the source script. You can replace that call with anything you want.
This has security implications though; just running an arbitrary code in your shell - especially with wget where the code comes from a remote location.
I'm getting started with bash scripting and made this little script following along a short guide but for some reason when I run the script with sh myscript I get
myscript: 5: myscript: 0: not found running on ubuntu 12.04
here is my script below I should at least see the echo message if no args are set:
#!/bin/bash
#will do something
name=$1
username=$2
if (( $# == 0 ))
then
echo "##############################"
echo "myscript [arg1] [arg2]"
echo "arg1 is your name"
echo "and arg2 is your username"
fi
var1="Your name is ${name} and your username is ${username}"
`echo ${var1} > yourname.txt`
`echo ${var1} > yourname.txt`
Get rid of the backticks.
echo ${var1} > yourname.txt
...for some reason when I run the script with sh myscript...
Don't run it that way. Make the script executable and run it directly
chmod +x myscript
./script
(or run with bash myscript explicitly).
It looks like that expression will work in bash but not in sh. As others pointed out change it to executable, make sure your shebang line is using bash and run it like this:
./myscript
If you want to run it with sh then it is complaining about line 5. Change it to this and it will work in /bin/sh.
if [ $# -ne 0 ]
Check out the man page for test.
Also you don't need the backticks on this line:
echo ${var1} > yourname.txt
What is a procedure to decorate an arbitrary bash command to execute it in a subshell? I cannot change the command, I have to decorate it on the outside.
the best I can think of is
>bash -c '<command>'
works on these:
>bash -c 'echo'
>bash -c 'echo foobar'
>bash -c 'echo \"'
but what about the commands such as
echo \'
and especially
echo \'\"
The decoration has to be always the same for all commands. It has to always work.
You say "subshell" - you can get one of those by just putting parentheses around the command:
x=outer
(x=inner; echo "x=$x"; exit)
echo "x=$x"
produces this:
x=inner
x=outer
You could (ab)use heredocs:
bash -c "$(cat <<-EOF
echo \'\"
EOF
)"
This is one way without using -c option:
bash <<EOF
echo \'\"
EOF
What you want to do is exactly the same as escapeshellcmd() in PHP (http://php.net/manual/fr/function.escapeshellcmd.php)
You just need to escape #&;`|*?~<>^()[]{}$\, \x0A and \xFF. ' and " are escaped only if they are not paired.
But beware of security issues...
Let bash take care of it this way:
1) prepare the command as an array:
astrCmd=(echo \'\");
2) export the array as a simple string:
export EXPORTEDastrCmd="`declare -p astrCmd| sed -r "s,[^=]*='(.*)',\1,"`";
3) restore the array and run it as a full command:
bash -c "declare -a astrCmd='$EXPORTEDastrCmd';\${astrCmd[#]}"
Create a function to make these steps more easy like:
FUNCbash(){
astrCmd=("$#");
export EXPORTEDastrCmd="`declare -p astrCmd| sed -r "s,[^=]*='(.*)',\1,"`";
bash -c "declare -a astrCmd='$EXPORTEDastrCmd';\${astrCmd[#]}";
}
FUNCbash echo \'\"
In answering another question I created the following script bash script:
#!/bin/bash
files1=( file1.txt file2.txt file3.txt )
files2=( file1_.txt file2_.txt file3_.txt )
cmd="vim -c 'set diffopt=filler,vertical' -c 'edit ${files1[0]}' -c 'diffsplit ${files2[0]}' "
echo $cmd
for i in {1..2}; do
cmd="${cmd} -c 'tabe ${files1[i]}' -c 'diffsplit ${files2[i]}' "
done
#$cmd
echo $cmd
the problem is that if I try to run
$cmd
in the end of the script I get errors, but if I just use echo $cmd and then copy and paste in the command line it works just fine.
Any ideas what I am doing wrong?
Thanks.
Use:
eval $cmd
So that the variables within the expression are expanded before execution.
BASH FAQ entry #50: "I'm trying to put a command in a variable, but the complex cases always fail!"