We have a shell script that is called by cron and runs as root.
This script outputs logging and debug info, and has been failing at one certain point. This point varies based on how much output the script creates (it fails sooner if we enable more debugging output, for example).
However, if the script is called directly, as a user, then it works without a problem.
We have since created a simplified test case which demonstrates the problem.
The script is:
#!/bin/bash
function log_so () {
local msg="$1"
if [ -z "${LOG_FILE}" ] ; then warn_so "It's pointless use log_so() if LOG_FILE variable is undefined!" ; return 1 ; fi
echo -e "${msg}"
echo -e "${msg}" >> ${LOG_FILE}
(
/bin/true
)
}
LOG_FILE="/usr/local/bin/log_bla"
linenum=1
while [[ $linenum -lt 2000 ]] ; do
log_so "short text: $linenum"
let linenum++
done
The highest this has reached is 244 before dying (when called via cron).
Some other searches recommended using a no-op subshell from the function and also calling /bin/true but not only did this not work, the subshell option is not feasible in the main script.
We have also tried changing the file descriptor limit for root, but that did not help, and have tried using both #!/bin/sh and #!/bin/bash for the script.
We are using bash 4.1.5(1)-release on Ubuntu 10.04 LTS.
Any ideas or recommendations for a workaround would be appreciated.
What about opening a fd by hand and cleaning it up afterwards? I don't have a bash 4.1 to test with, but it might help.
LOG_FILE="/usr/local/bin/log_bla"
exec 9<> "$LOG_FILE"
function log_so () {
local msg="$1"
if [ -z "${LOG_FILE}" ] ; then warn_so "It's pointless use log_so() if LOG_FILE variable is undefined!" ; return 1 ; fi
echo -e "${msg}"
echo -e "${msg}" >&9
return 0
}
linenum=1
while [[ $linenum -lt 2000 ]] ; do
log_so "short text: $linenum"
let linenum++
done
exec 9>&-
Related
I have a function that runs a set of scripts that set variables, functions, and aliases in the current shell.
reloadVariablesFromScript() {
for script in "${scripts[#]}"; do
. "$script"
done
}
If one of the scripts has an error, I want to exit the script and then exit the function, but not to kill the shell.
reloadVariablesFromScript() {
for script in "${scripts[#]}"; do
{(
set -e
. "$script"
)}
if [[ $? -ne 0 ]]; then
>&2 echo $script failed. Skipping remaining scripts.
return 1
fi
done
}
This would do what I want except it doesn't set the variables in the script whether the script succeeds or fails.
Without the subshell, set -e causes the whole shell to exit, which is undesirable.
Is there a way I can either prevent the called script from continuing on an error without killing the shell or else set/export variables, aliases, and functions from within a subshell?
The following script simulates my problem:
test() {
{(
set -e
export foo=bar
false
echo Should not have gotten here!
export bar=baz
)}
local errorCode=$?
echo foo="'$foo'". It should equal 'bar'.
echo bar="'$bar'". It should not be set.
if [[ $errorCode -ne 0 ]]; then
echo Script failed correctly. Exiting function.
return 1
fi
echo Should not have gotten here!
}
test
If worst comes to worse, since these scripts don't actually edit the filesystem, I can run each script in a subshell, check the exit code, and if it succeeds, run it outside of a subshell.
Note that set -e has a number of surprising behaviors -- relying on it is not universally considered a good idea. That caveat being give, though: We can shuffle environment variables, aliases, and shell functions out as text:
envTest() {
local errorCode newVars
newVars=$(
set -e
{
export foo=bar
false
echo Should not have gotten here!
export bar=baz
} >&2
# print generate code which, when eval'd, recreates our functions and variables
declare -p | egrep -v '^declare -[^[:space:]]*r'
declare -f
alias -p
); errorCode=$?
if (( errorCode == 0 )); then
eval "$newVars"
fi
printf 'foo=%q. It should equal %q\n' "$foo" "bar"
printf 'bar=%q. It should not be set.\n' "$bar"
if [[ $errorCode -ne 0 ]]; then
echo 'Script failed correctly. Exiting function.'
return 1
fi
echo 'Should not have gotten here!'
}
envTest
Note that this code only evaluates either export should the entire script segment succeed; the question text and comments appear to indicate that this is acceptable if not desired.
I have a bash script which is only meant to used be when sourced.
I want to return from it automatically on any error, similar to what set -e does.
However setting set -e doesn't work for me because it will also exit the users shell.
Right now I'm handling returning manually like this command || return 1, for each command.
You can also use command || true or command || return.
If your requirement is something different, please update more precisely.
You can use trap. E.g.:
// foo.sh
function func() {
trap 'if [ $? -ne 0 ]; then echo "Trapped!"; return ; fi' DEBUG
echo 'foo'
find -name "foo" . 2> /dev/null
echo 'bar'
}
func
Two notes. First, the trap needs to be inside the function as shown. It won't work if it's just inside the script.
Two, there is a significant limitation. Even if you set the return to the trap (e.g., return 1), while func exists after the bad find command, $? is still zero, no matter what. I'm not sure if there's a way around that, so if it's important to preserve the exit value of the failed command, this may not work.
E.g., if you had:
func
func_return=$?
echo "return value is: $func_return"
func_return will always be zero. I've played around with trying to get the exit value of the failed command to pass out of the function trap and into the function exit value, but have not found a way to do it.
If you need to preserve the return value, you could update a global variable inside the debug trap.
If I understand well, you can set -e locally in each function.
cat sourced
f1 () {
local -
set -e
[ "$1" -eq "$1" ] 2> /dev/null && echo "$1"
}
cat script.sh
. sourced
param='bad'
ret=$(f1 "$param")
[ $? -eq 0 ] && echo "result = $ret" || \
echo "error in sourced file with param $param"
param=3
ret=$(f1 "$param")
[ $? -eq 0 ] && echo "result = $ret" || \
echo "error in sourced file with param $param"
Im modifying an existing bash script and having some trouble getting the while loop behaving correctly. This is the original code
while ! /usr/bin/executable1
do
# executable1 returned an error. So sleep for some time try again
sleep 2
done
I would like to change this to the following
while ! /usr/bin/executable1 && ! $(myfunc)
do
# executable1 and myfunc both were unsuccessful. So sleep for some time
sleep 2
done
executable1 returns 0 on success and 1 on failure. I understand that "true" in bash evaluates to 0 so thats why the original script would keep looping till executable returned success
Accordingly myfunc is coded like this
myfunc ()
{
# Check if file exists. If exits, return 0, If not, return 1
if [ -e someFile ]; then
return 0
fi
return 1
}
I notice that the my new while loop does not seem to call executable1. It always calls myfunc() and then exits out of the loop immediately. What am I doing wrong?
I tried various ways of coding the while loop (with (( )), [ ], [[ ]] etc), but nothing seems to fix it
You don't need $(...) to call a function, just to capture its standard output. You simply want
while ! /usr/bin/executable1 && ! myfunc
do
sleep 2
done
Note that myfunc can also be more simply written
myfunc () {
[ -e someFile ]
}
or even (in bash)
myfunc () [[ -e someFile ]]
Either way, it's almost not worth defining myfunc separately; just use
while ! /usr/bin/executable1 && ! [[ -e someFile ]]
do
sleep 2
done
It might also be simpler to use an until loop:
until /usr/bin/executable1 || [[ -e someFile ]]; do
sleep 2
done
I'm looking for the best way to duplicate the Linux 'watch' command on Mac OS X. I'd like to run a command every few seconds to pattern match on the contents of an output file using 'tail' and 'sed'.
What's my best option on a Mac, and can it be done without downloading software?
With Homebrew installed:
brew install watch
You can emulate the basic functionality with the shell loop:
while :; do clear; your_command; sleep 2; done
That will loop forever, clear the screen, run your command, and wait two seconds - the basic watch your_command implementation.
You can take this a step further and create a watch.sh script that can accept your_command and sleep_duration as parameters:
#!/bin/bash
# usage: watch.sh <your_command> <sleep_duration>
while :;
do
clear
date
$1
sleep $2
done
Use MacPorts:
$ sudo port install watch
The shells above will do the trick, and you could even convert them to an alias (you may need to wrap in a function to handle parameters):
alias myWatch='_() { while :; do clear; $2; sleep $1; done }; _'
Examples:
myWatch 1 ls ## Self-explanatory
myWatch 5 "ls -lF $HOME" ## Every 5 seconds, list out home directory; double-quotes around command to keep its arguments together
Alternately, Homebrew can install the watch from http://procps.sourceforge.net/:
brew install watch
It may be that "watch" is not what you want. You probably want to ask for help in solving your problem, not in implementing your solution! :)
If your real goal is to trigger actions based on what's seen from the tail command, then you can do that as part of the tail itself. Instead of running "periodically", which is what watch does, you can run your code on demand.
#!/bin/sh
tail -F /var/log/somelogfile | while read line; do
if echo "$line" | grep -q '[Ss]ome.regex'; then
# do your stuff
fi
done
Note that tail -F will continue to follow a log file even if it gets rotated by newsyslog or logrotate. You want to use this instead of the lower-case tail -f. Check man tail for details.
That said, if you really do want to run a command periodically, the other answers provided can be turned into a short shell script:
#!/bin/sh
if [ -z "$2" ]; then
echo "Usage: $0 SECONDS COMMAND" >&2
exit 1
fi
SECONDS=$1
shift 1
while sleep $SECONDS; do
clear
$*
done
I am going with the answer from here:
bash -c 'while [ 0 ]; do <your command>; sleep 5; done'
But you're really better off installing watch as this isn't very clean...
If watch doesn't want to install via
brew install watch
There is another similar/copy version that installed and worked perfectly for me
brew install visionmedia-watch
https://github.com/tj/watch
Or, in your ~/.bashrc file:
function watch {
while :; do clear; date; echo; $#; sleep 2; done
}
To prevent flickering when your main command takes perceivable time to complete, you can capture the output and only clear screen when it's done.
function watch {while :; do a=$($#); clear; echo "$(date)\n\n$a"; sleep 1; done}
Then use it by:
watch istats
Try this:
#!/bin/bash
# usage: watch [-n integer] COMMAND
case $# in
0)
echo "Usage $0 [-n int] COMMAND"
;;
*)
sleep=2;
;;
esac
if [ "$1" == "-n" ]; then
sleep=$2
shift; shift
fi
while :;
do
clear;
echo "$(date) every ${sleep}s $#"; echo
$#;
sleep $sleep;
done
Here's a slightly changed version of this answer that:
checks for valid args
shows a date and duration title at the top
moves the "duration" argument to be the 1st argument, so complex commands can be easily passed as the remaining arguments.
To use it:
Save this to ~/bin/watch
execute chmod 700 ~/bin/watch in a terminal to make it executable.
try it by running watch 1 echo "hi there"
~/bin/watch
#!/bin/bash
function show_help()
{
echo ""
echo "usage: watch [sleep duration in seconds] [command]"
echo ""
echo "e.g. To cat a file every second, run the following"
echo ""
echo " watch 1 cat /tmp/it.txt"
exit;
}
function show_help_if_required()
{
if [ "$1" == "help" ]
then
show_help
fi
if [ -z "$1" ]
then
show_help
fi
}
function require_numeric_value()
{
REG_EX='^[0-9]+$'
if ! [[ $1 =~ $REG_EX ]] ; then
show_help
fi
}
show_help_if_required $1
require_numeric_value $1
DURATION=$1
shift
while :; do
clear
echo "Updating every $DURATION seconds. Last updated $(date)"
bash -c "$*"
sleep $DURATION
done
Use the Nix package manager!
Install Nix, and then do nix-env -iA nixpkgs.watch and it should be available for use after the completing the install instructions (including sourcing . "$HOME/.nix-profile/etc/profile.d/nix.sh" in your shell).
The watch command that's available on Linux does not exist on macOS. If you don't want to use brew you can add this bash function to your shell profile.
# execute commands at a specified interval of seconds
function watch.command {
# USAGE: watch.commands [seconds] [commands...]
# EXAMPLE: watch.command 5 date
# EXAMPLE: watch.command 5 date echo 'ls -l' echo 'ps | grep "kubectl\\\|node\\\|npm\\\|puma"'
# EXAMPLE: watch.command 5 'date; echo; ls -l; echo; ps | grep "kubectl\\\|node\\\|npm\\\|puma"' echo date 'echo; ls -1'
local cmds=()
for arg in "${#:2}"; do
echo $arg | sed 's/; /;/g' | tr \; \\n | while read cmd; do
cmds+=($cmd)
done
done
while true; do
clear
for cmd in $cmds; do
eval $cmd
done
sleep $1
done
}
https://gist.github.com/Gerst20051/99c1cf570a2d0d59f09339a806732fd3
I want to write code like this:
command="some command"
safeRunCommand $command
safeRunCommand() {
cmnd=$1
$($cmnd)
if [ $? != 0 ]; then
printf "Error when executing command: '$command'"
exit $ERROR_CODE
fi
}
But this code does not work the way I want. Where did I make the mistake?
Below is the fixed code:
#!/bin/ksh
safeRunCommand() {
typeset cmnd="$*"
typeset ret_code
echo cmnd=$cmnd
eval $cmnd
ret_code=$?
if [ $ret_code != 0 ]; then
printf "Error: [%d] when executing command: '$cmnd'" $ret_code
exit $ret_code
fi
}
command="ls -l | grep p"
safeRunCommand "$command"
Now if you look into this code, the few things that I changed are:
use of typeset is not necessary, but it is a good practice. It makes cmnd and ret_code local to safeRunCommand
use of ret_code is not necessary, but it is a good practice to store the return code in some variable (and store it ASAP), so that you can use it later like I did in printf "Error: [%d] when executing command: '$command'" $ret_code
pass the command with quotes surrounding the command like safeRunCommand "$command". If you don’t then cmnd will get only the value ls and not ls -l. And it is even more important if your command contains pipes.
you can use typeset cmnd="$*" instead of typeset cmnd="$1" if you want to keep the spaces. You can try with both depending upon how complex is your command argument.
'eval' is used to evaluate so that a command containing pipes can work fine
Note: Do remember some commands give 1 as the return code even though there isn't any error like grep. If grep found something it will return 0, else 1.
I had tested with KornShell and Bash. And it worked fine. Let me know if you face issues running this.
Try
safeRunCommand() {
"$#"
if [ $? != 0 ]; then
printf "Error when executing command: '$1'"
exit $ERROR_CODE
fi
}
It should be $cmd instead of $($cmd). It works fine with that on my box.
Your script works only for one-word commands, like ls. It will not work for "ls cpp". For this to work, replace cmd="$1"; $cmd with "$#". And, do not run your script as command="some cmd"; safeRun command. Run it as safeRun some cmd.
Also, when you have to debug your Bash scripts, execute with '-x' flag. [bash -x s.sh].
There are several things wrong with your script.
Functions (subroutines) should be declared before attempting to call them. You probably want to return() but not exit() from your subroutine to allow the calling block to test the success or failure of a particular command. That aside, you don't capture 'ERROR_CODE' so that is always zero (undefined).
It's good practice to surround your variable references with curly braces, too. Your code might look like:
#!/bin/sh
command="/bin/date -u" #...Example Only
safeRunCommand() {
cmnd="$#" #...insure whitespace passed and preserved
$cmnd
ERROR_CODE=$? #...so we have it for the command we want
if [ ${ERROR_CODE} != 0 ]; then
printf "Error when executing command: '${command}'\n"
exit ${ERROR_CODE} #...consider 'return()' here
fi
}
safeRunCommand $command
command="cp"
safeRunCommand $command
The normal idea would be to run the command and then use $? to get the exit code. However, sometimes you have multiple cases in which you need to get the exit code. For example, you might need to hide its output, but still return the exit code, or print both the exit code and the output.
ec() { [[ "$1" == "-h" ]] && { shift && eval $* > /dev/null 2>&1; ec=$?; echo $ec; } || eval $*; ec=$?; }
This will give you the option to suppress the output of the command you want the exit code for. When the output is suppressed for the command, the exit code will directly be returned by the function.
I personally like to put this function in my .bashrc file.
Below I demonstrate a few ways in which you can use this:
# In this example, the output for the command will be
# normally displayed, and the exit code will be stored
# in the variable $ec.
$ ec echo test
test
$ echo $ec
0
# In this example, the exit code is output
# and the output of the command passed
# to the `ec` function is suppressed.
$ echo "Exit Code: $(ec -h echo test)"
Exit Code: 0
# In this example, the output of the command
# passed to the `ec` function is suppressed
# and the exit code is stored in `$ec`
$ ec -h echo test
$ echo $ec
0
Solution to your code using this function
#!/bin/bash
if [[ "$(ec -h 'ls -l | grep p')" != "0" ]]; then
echo "Error when executing command: 'grep p' [$ec]"
exit $ec;
fi
You should also note that the exit code you will be seeing will be for the grep command that's being run, as it is the last command being executed. Not the ls.