I have a bash script which sequentially executes many tasks.
However, because I do not want to see simple status messages (such as the long output of yum -y update), I ignored all those messages using:
#!/bin/bash
(
yum -y update
cd /foo/bar
cp ~/bar /usr/bin/foo
...
...
) > /dev/null
This does the job just fine, but what if something went wrong, like cp failed to copy some file? If this happens, I would like to catch the error and exit immediately, before the process continues.
How can I exit the process and show the related error? Normally,
an if/else clause would have to be used, like this:
#!/bin/bash
yum -y update
if [ $? -ne 0 ]; then
echo "error "
exit 1
fi
But the problem with this approach is that it would show the process; therefore, I would have to use > /dev/null on each line; more importantly, if I had more than 100 things to do, then the I would have to use many if/else statements.
Is there a convenient solution for this?
Rather than running your commands in (...) use set -e OR bash -ec to execute them:
bash -ec 'set -e
yum -y update
cd /foo/bar
...
...
cp ~/bar /usr/bin/foo' > /dev/null 2> errlog
OR using set -e:
(
set -e
yum -y update
cd /foo/bar
...
...
cp ~/bar /usr/bin/foo
) > /dev/null 2> errlog
-e option will make sure to exit the sub shell as soon as an error occurs.
Related
I came across an example for the using tee utility in the tee info page:
wget -O - http://example.com/dvd.iso | tee >(sha1sum > dvd.sha1) > dvd.iso
I looked up the >(...) syntax and found something called "process substitution". From what I understand, it makes a process look like a file that another process could write/append its output to. (Please correct me if I'm wrong on that point.)
How is this different from a pipe? (|) I see a pipe is being used in the above example—is it just a precedence issue? or is there some other difference?
There's no benefit here, as the line could equally well have been written like this:
wget -O - http://example.com/dvd.iso | tee dvd.iso | sha1sum > dvd.sha1
The differences start to appear when you need to pipe to/from multiple programs, because these can't be expressed purely with |. Feel free to try:
# Calculate 2+ checksums while also writing the file
wget -O - http://example.com/dvd.iso | tee >(sha1sum > dvd.sha1) >(md5sum > dvd.md5) > dvd.iso
# Accept input from two 'sort' processes at the same time
comm -12 <(sort file1) <(sort file2)
They're also useful in certain cases where you for any reason can't or don't want to use pipelines:
# Start logging all error messages to file as well as disk
# Pipes don't work because bash doesn't support it in this context
exec 2> >(tee log.txt)
ls doesntexist
# Sum a column of numbers
# Pipes don't work because they create a subshell
sum=0
while IFS= read -r num; do (( sum+=num )); done < <(curl http://example.com/list.txt)
echo "$sum"
# apt-get something with a generated config file
# Pipes don't work because we want stdin available for user input
apt-get install -c <(sed -e "s/%USER%/$USER/g" template.conf) mysql-server
Another major difference is the propagation of return values / exit codes (I'll use simpler commands to illustrate):
Pipe:
$ ls -l /notthere | tee listing.txt
ls: cannot access '/notthere': No such file or directory
$ echo $?
0
-> exit code of tee is propagated
Process substitution:
$ ls -l /notthere > >(tee listing.txt)
ls: cannot access '/notthere': No such file or directory
$ echo $?
2
-> exit code of ls is propagated
There are of course several methods to work around this (e.g. set -o pipefail, variable PIPESTATUS), but I think it's worth mentioning since this is the default behavior.
Yet another rather subtle, yet potentially annoying difference lies in subprocess termination (best illustrated using commands that produce lots of output):
Pipe:
#!/usr/bin/env bash
tar --create --file /tmp/etc-backup.tar --verbose --directory /etc . 2>&1 | tee /tmp/etc-backup.log
retval=${PIPESTATUS[0]}
(( ${retval} == 0 )) && echo -e "\n*** SUCCESS ***\n" || echo -e "\n*** FAILURE (EXIT CODE: ${retval}) ***\n"
-> after the line containing the pipe construct, all commands of the pipe have already terminated (otherwise PIPESTATUS could not contain their respective exit codes)
Process substitution:
#!/usr/bin/env bash
tar --create --file /tmp/etc-backup.tar --verbose --directory /etc . &> >(tee /tmp/etc-backup.log)
retval=$?
(( ${retval} == 0 )) && echo -e "\n*** SUCCESS ***\n" || echo -e "\n*** FAILURE (EXIT CODE: ${retval}) ***\n"
-> after the line containing the process substitution, the command within >(...), i.e. tee in this example, may still be running, potentially causing desynchronized console output (SUCCESS / FAILURE message gets mixed in with still flowing tar output) [*]
[*] Can be reproduced on the framebuffer console, but does not seem to affect GUI terminals like KDE's Konsole (likely due to different buffering strategies).
I'm trying to run a script that is required to have an exit code of 0. Unfortunalty I cannot use an init.d or other startup script to control this this, so I must make this work.
Basically if I understand AWS's docs correctly (elastic beanstalk), I need be able to run the following two commands and exit with a 0 and provide no other output to stdout.
As the root user I need to cd to a particular dir and run these two commands:
pkill -f que
bundle exec que
In my actually script I have:
#!/usr/bin/env bash
su -s /bin/bash -c "cd /some/dir && nohup pkill -f que &>/dev/null &"
sleep 10
su -s /bin/bash -c "cd /some/dir && nohup bundle exec que &"
Which still causes this error to be raised:
returned non-zero exit status 1 (Executor::NonZeroExitStatus)
Any tips for how to silently run those commands correctly?
I'm also looking at these for ideas:
https://blog.eq8.eu/article/aws-elasticbeanstalk-hooks.html
http://www.dannemanne.com/posts/post-deployment_script_on_elastic_beanstalk_restart_delayed_job
But its still not clear to me how this is supposed to exit successfully
Perhaps I'm missing something, but wouldn't this be easily solved by using two shell scripts? One with cd, pkill, and bundle. Call this script (foo.sh) something like:
#!/usr/bin/env bash
su -c ./foo.sh > /dev/null 2>&1 < /dev/null
exit 0
Is it possible to execute 2 or more commands in one line but return the status of the 1st command in bash?
I have the following step in Docker build:
RUN bin/myserver && cat tmp/log && rm -rf tmp
It is essential to run that process inside my docker build, and it makes sense to make a cleanup afterwards to remove unneeded things to be stored as docker layer.
The myserver logs to log file, not console output, so I need to cat the log to know what was going on, especially in case of failure.
How can I put those things together?
How about:
RUN sh -c 'bin/myserver; status=$?; cat tmp/log; rm -rf tmp; exit $status'
You can redirect the stderr of a command using 2>
To ignore the errors, you can redirect it to /dev/null: 2> /dev/null
So your command line becomes:
RUN bin/myserver && cat tmp/log 2> /dev/null && rm -rf tmp 2> /dev/null
I have a shell script when need to run as a particular user. So I call that script as below,
su - testuser -c "/root/check_package.sh | tee -a /var/log/check_package.log"
So after this when I check the last execution exitcode it returns always 0 only even if that script fails.
I tried something below also which didn't help,
su - testuser -c "/root/check_package.sh | tee -a /var/log/check_package.log && echo $? || echo $?"
Is there way to get the exitcode of command whatever running through su.
The problem here is not su, but tee: By default, the shell exits with the exit status of the last pipeline component; in your code, that component is not check_package.sh, but instead is tee.
If your /bin/sh is provided by bash (as opposed to ash, dash, or another POSIX-baseline shell), use set -o pipefail to cause the entirely pipeline to fail if any component of it does:
su - testuser -c "set -o pipefail; /root/check_package.sh | tee -a /var/log/check_package.log"
Alternately, you can do the tee out-of-band with redirection to a process substitution (though this requires your current user to have permission to write to check_package.log):
su - testuser -c "/root/check_package.sh" > >(tee -a /var/log/check_package.log
Both su and sudo exit with the exit status of the command they execute (if authentication succeeded):
$ sudo false; echo $?
1
$ su -c false; echo $?
1
Your problem is that the command pipeline that su runs is a pipeline. The exit status of your pipeline is that of the tee command (which succeeds), but what you really want is that of the first command in the pipeline.
If your shell is bash, you have a couple of options:
set -o pipefail before your pipeline, which will make it return the rightmost failure value of all the commands if any of them fail
Examine the specific member of the PIPESTATUS array variable - this can give you the exit status of the first command whether or not tee succeeds.
Examples:
$ sudo bash -c "false | tee -a /dev/null"; echo $?
0
$ sudo bash -c "set -o pipefail; false | tee -a /dev/null"; echo $?
1
$ sudo bash -c 'false | tee -a /dev/null; exit ${PIPESTATUS[0]}'; echo $?
1
You will get similar results using su -c, if your system shell (in /bin/sh) is Bash. If not, then you'd need to explicitly invoke bash, at which point sudo is clearly simpler.
I was facing a similar issue today, in case the topic is still open here my solution, otherwise just ignore it...
I wrote a bash script (let's say my_script.sh) which looks more or less like this:
### FUNCTIONS ###
<all functions listed in the main script which do what I want...>
### MAIN SCRIPT ### calls the functions defined in the section above
main_script() {
log_message "START" 0
check_env
check_data
create_package
tar_package
zip_package
log_message "END" 0
}
main_script |tee -a ${var_log} # executes script and writes info into log file
var_sta=${PIPESTATUS[0]} # captures status of pipeline
exit ${var_sta} # exits with value of status
It works when you call the script directly or in sudo mode
Is there an alternative for the timeout command on Mac OSx. The basic requirement is I am able to run a command for a specified amount of time.
e.g:
timeout 10 ping google.com
This program runs ping for 10s on Linux.
You can use
brew install coreutils
And then whenever you need timeout, use
gtimeout
..instead. To explain why here's a snippet from the Homebrew Caveats section:
Caveats
All commands have been installed with the prefix 'g'.
If you really need to use these commands with their normal names, you
can add a "gnubin" directory to your PATH from your bashrc like:
PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"
Additionally, you can access their man pages with normal names if you add
the "gnuman" directory to your MANPATH from your bashrc as well:
MANPATH="/usr/local/opt/coreutils/libexec/gnuman:$MANPATH"
Another simple approach that works pretty much cross platform (because it uses perl which is nearly everywhere) is this:
function timeout() { perl -e 'alarm shift; exec #ARGV' "$#"; }
Snagged from here:
https://gist.github.com/jaytaylor/6527607
Instead of putting it in a function, you can just put the following line in a script, and it'll work too:
timeout.sh
perl -e 'alarm shift; exec #ARGV' "$#";
or a version that has built in help/examples:
timeout.sh
#!/usr/bin/env bash
function show_help()
{
IT=$(cat <<EOF
Runs a command, and times out if it doesnt complete in time
Example usage:
# Will fail after 1 second, and shows non zero exit code result
$ timeout 1 "sleep 2" 2> /dev/null ; echo \$?
142
# Will succeed, and return exit code of 0.
$ timeout 1 sleep 0.5; echo \$?
0
$ timeout 1 bash -c 'echo "hi" && sleep 2 && echo "bye"' 2> /dev/null; echo \$?
hi
142
$ timeout 3 bash -c 'echo "hi" && sleep 2 && echo "bye"' 2> /dev/null; echo \$?
hi
bye
0
EOF
)
echo "$IT"
exit
}
if [ "$1" == "help" ]
then
show_help
fi
if [ -z "$1" ]
then
show_help
fi
#
# Mac OS-X does not come with the delightfully useful `timeout` program. Thankfully a rough BASH equivalent can be achieved with only 2 perl statements.
#
# Originally found on SO: http://stackoverflow.com/questions/601543/command-line-command-to-auto-kill-a-command-after-a-certain-amount-of-time
#
perl -e 'alarm shift; exec #ARGV' "$#";
As kvz stated simply use homebrew:
brew install coreutils
Now the timeout command is already ready to use - no aliases are required (and no gtimeout required, although also available).
You can limit execution time of any program using this command:
ping -t 10 google.com & sleep 5; kill $!
The Timeout Package from Ubuntu / Debian can be made to compile on Mac and it works.
The package is available at http://packages.ubuntu.com/lucid/timeout
You can do ping -t 10 google.com >nul
the >nul gets rid of the output. So instead of showing 64 BYTES FROM 123.45.67.8 BLAH BLAH BLAH it'll just show a blank newline until it times out. -t flag can be changed to any number.