I am having difficulty running a 1300 line bash script in sudo crontab. Also note this is on Ubuntu 10.04.
I suspect there is a "hidden" timeout somewhere because the script seems to stop at exactly 3 minutes very consistently but "timeout" isn't listed in the crontab file:
MAILTO="my#email.com"
00 12 * * * /path/to/script.sh
The script runs fine if I run it in a terminal window OR if I add >> anOutputFile.txt to the end of the crontab line. I could just use that as my solution but I want to understand what is causing the script to stop.
It almost seems like there's a timeout on the run time of the crontab script because it doesn't always stop at the exact same spot, but it does end within the same few lines consistently. I've tried adding a sleep 300 at the beginning of the script to see if it's really a timeout issue but that does not seem to be the case because it will still stop within the same few lines.
If it isn't a timeout issue, I am thinking there may be an issue with the for i in loop that the script fails around. If I comment out a the last two echo lines in the for i in {1..5} loop, the script finishes fine but if only one is commented out, it seems to be right at that unknown timeout limit and will have a 50-50 chance of finishing.
if ( [ ${#modemusbpath0} != 0 ] ); then
if test -e ModemPath; then
. /path/to/ModemPath
echo LEGACY MODEM USB PATH: $modemusbpath
echo LEGACY MODEM USB PATH: $modemusbpath >> /home/path/to/file/genrpt$DATESTR.log
BUS=`echo $modemusbpath | awk -F / '{print "/dev/bus/usb/"$5"/"}'`
echo BUS=$BUS
DEVICE=`echo $modemusbpath | awk -F / '{print $6}' | sed 's/^0*//' `
for i in {1..5} ; do
DEVICES=$(printf %03d $(($DEVICE + $i )) )
echo DEVICES=$DEVICES
NEXTMODEMPATH=$BUS$DEVICES
echo NEXTMODEMPATH=$NEXTMODEMPATH
echo "NEXTMODEMPATH=$NEXTMODEMPATH" >> /home/path/to/file/genrpt$DATESTR.log
#
modemusbpath="$modemusbpath $NEXTMODEMPATH"
done
echo modemusbpath=$modemusbpath
/home/path/to/file/TelemetryFailureReport.sh "TELEMETRY: NO CURRENT MODEM PATH; USING PATHS: $modemusbpath"
else
echo LEGACY MODEM PATH DOES NOT EXIST!
echo "LEGACY MODEM PATH DOES NOT EXIST!" >> /home/path/to/file/genrpt$DATESTR.log
fi
fi
Should I be specifying an output file? I think that solves my issue but I'd like to understand the cause.
A few more notes:
- I haven't been able to make MAILTO="my#email.com" work by setting it in the crontab in attempt to see the output, and even if I did, this script is used when there is no internet connection so an email shouldn't work.
- Also, I am using "timeout" as a variable at other points in the script, but there is no /bin/timeout or /sbin/timeout or /usr/bin/timeout or /usr/sbin/timeout. Should I stop using this as a variable name?
- PATH is the same between the terminal and a cron script.
Thank you.
The usual if format is a bit different. Can you try instead of this:
if ( [ ${#modemusbpath0} != 0 ] );
to be this:
if [ "${#modemusbpath0}" -ne 0 ];
Related
I am trying to create an init script for a program in bash. (rhel6)
It checks for the processes first. If processes are found it will echo that program is already online and if not it'll move on to to start the program as a certain user by using launch script. After doing that it should tail the log file of the program and check for a string of words together. If the words are found it should kill tail and echo that program is online.
Here's the start segment.
prog=someProg
user=someUser
threadCount=$(ps -ef | grep $prog |grep -v 'grep' |awk '{ print $2 }'| wc -l)
startb() {
if [ "$threadCount" -eq 2 ]; then
echo "$prog already online."
else
echo "Bringing $prog online."
su $user -c "/path/to/start/script.sh"
tail -f /path/to/$prog/log/file |
while IFS=$'\n' read line
do
if [[ $line == *started\ up\ and\ registered\ in* ]]; then
pkill tail
echo "$prog now online."
fi
done
fi
}
My problems:
The variable $prog doesn't get picked in $threadcount no
matter how I try. (with single and double quotes)
The logic about tailing the log file works randomly. Some times it
just works perfect. It tails and waits till the string is found
before echoing program is online and at times it just starts script
and then echoes that program is online without the tail or wait.
It's unpredictable. I implemented the same logic in stop segment too to monitor log and then echo but even that works the same way as start. Just random.
I am sure that this might look dumb and broken. This is made by picking pieces here and there with my beginner bash skills.
Thanks in advance for suggestions and help.
I can't reproduce the error you are experiencing with the "grep $prog"...sorry.
But for the other part.
I will assume that the script starting your program, the line with su, is starting something in background and that the script end by itself. If not, your example will wait indefinitely.
Could be a personal preference, but when I'm using something like tail to verify lines, I use a named pipe (mkfifo).
That would give something like :
# Getting the tail in background
tail -f /path/to/$prog/log/file > some_fifo &
# Getting the tail PID
tailPID=$!
while read line; do #You don't need to modify/use IFS here.
if [[ $line == *started\ up\ and\ registered\ in* ]]; then
kill -15 $tailPID #since you know the PID you won't kill another tail
echo "$prog now online."
break # don't like the possibility, even remote, of an infinite loop :)
fi
done < some_fifo #reading from the named pipe
Hope it can help you
I'm trying to match the output of an SSH command but for some reason it's not working. I'm using authorized_keys to restrict the login to one command - which is a wrapper script that takes parameters.
Here's my code:
for i in `seq 2 254`; do
myoutput=$( ssh system#server01 ip 192.168.0.$i )
echo "$myoutput"
echo "$myoutput" | grep -q "not found"
if [ $? -eq 0 ]; then
echo 192.168.0.$i
exit
fi
done
Here's the output of the script:
192.168.0.2 not found in DB. Try searching live (--forcelive)
192.168.0.3 not found in DB. Try searching live (--forcelive)
192.168.0.4 not found in DB. Try searching live (--forcelive)
This should stop at the first instance and echo the IP but it's continuing along because ALL of the greps are returning 1 rather than 0. The SSH command itself (without the grep) returns 0 every time.
Why is grep miscoding the response? Is there a better way to accomplish what I'm wanting to do?
The content you're trying to search for is almost certainly on stderr, so your substitution isn't capturing it at all; thus, it's being emitted to the console by the same command that invokes ssh rather than placed in the myoutput variable.
Consider using the redirection 2>&1 to redirect stderr to stdout, as so:
myoutput=$( ssh system#server01 "ip 192.168.0.$i" 2>&1 )
By the way, if you're okay with the overhead of calling ssh over two hundred times, I would seriously consider rewriting this like so:
for ((i=2; i<254; i++)); do
if ssh system#server01 "ip 192.168.0.$i" 2>&1 | grep -q 'not found'; then
echo "192.168.0.$i"
exit
fi
done
Why?
seq is neither built into bash, nor POSIX-specified. Thus, it's not guaranteed even to exist, much less to behave in any particular way.
The command line passed by ssh to a remote system is built as a single string generated by concatenating arguments. Passing only a single string with the exact code you wish to execute is thus a more honest representation of what goes on under the hood, and avoids bugs that can be created by forgetting what ssh really does with its command line.
Using $? is bad form when you can simply test for success of a command directly.
If you weren't restricted by authorized_keys, by the way, you'd be much better off moving the script evaluation to the remote system, something like:
ip_found=$(ssh system#server01 bash -s <<'EOF'
for ((i=2; i<254; i++)); do
content=$(ip "192.168.0.$i" 2>&1)
if [[ $content = *"not found"* ]]; then
echo "192.168.0.$i"
exit 0
fi
done
exit 1
EOF
)
echo "Remote system found $ip_found"
...since you are so restricted, you might -- for performance reasons -- consider using SSH multiplexing to reuse a single authenticated connection over your many queries.
I need to get the full output of a command and also get its pid. The problem is that getting the pid works like this:
(nmcli d wifi connect "$1" password "$2") & PID=$!
and grabbing the output works like this:
output="$(nmcli d wifi connect $1 password $2)"
How do I combine these two?
I think your best bet is to use a temporary file:
f="${TMPDIR:-/tmp}/tmp.$$"
trap "rm $f" EXIT
nmcli d wifi connect "$1" password "$2" >"$f" &
nm_pid=$!
# ... do stuff
wait $nm_pid
# ... use contents of "$f"
(I've added quotes around your $1 and $2 - it's rare to encounter too much quoting in scripts).
This cannot be done. You want to run program in background and capture it's output simultaneously into a variable. If that would be possible then the value of the variable would be changing during the execution of background program without your explicit commands. For example:
output="$(nmcli d wifi connect $1 password $2)" &
echo $output #maybe empty
sleep 10
echo $output #what now?
In the above script we did not change the value of "output" variable explicitly (by assignment). But if the feature you are asking about would be possible then the value of the variable would change implicitly (which is error-prone).
You can achieve same functionality by redirecting output of a background command to a file.
Do you mean something like this?
#/bin/env bash
nmcli d wifi connect "$1" password "$2" > /tmp/$$.tmp &
PID=$!
wait
output=$(</tmp/$$.tmp)
...
I have a shell script in which in the first line I ask the user to input how many minutes they want the script to run for:
#!/usr/bin/ksh
echo "How long do you want the script to run for in minutes?:\c"
read scriptduration
loopcnt=0
interval=1
date2=$(date +%H:%M%S)
(( intervalsec = $interval * 1 ))
totalmin=${1:-$scriptduration}
(( loopmax = ${totalmin} * 60 ))
ofile=/home2/s499929/test.log
echo "$date2 total runtime is $totalmin minutes at 2 sec intervals"
while(( $loopmax > $loopcnt ))
do
date1=$(date +%H:%M:%S)
pid=`/usr/local/bin/lsof | grep 16752 | grep LISTEN |awk '{print $2}'` > /dev/null 2>&1
count=$(netstat -an|grep 16752|grep ESTABLISHED|wc -l| sed "s/ //g")
process=$(ps -ef | grep $pid | wc -l | sed "s/ //g")
port=$(netstat -an | grep 16752 | grep LISTEN | wc -l| sed "s/ //g")
echo "$date1 activeTCPcount:$count activePID:$pid activePIDcount=$process listen=$port" >> ${ofile}
sleep $intervalsec
(( loopcnt = loopcnt + 1 ))
done
It works great if I kick it off an input the values manually. But if I want to run this for 3 hours I need to kick off the script to run in the background.
I have tried just running ./scriptname & and I get this:
$ How long do you want the test to run for in minutes:360
ksh: 360: not found.
[2] + Stopped (SIGTTIN) ./test.sh &
And the script dies. Is this possible, any suggestions on how I can accept this one input and then run in the background?? Thanks!!!
You could do something like this:
test.sh arg1 arg2 &
Just refer to arg1 and arg2 as $1 and $2, respectively, in the bash script. ($0 is the name of the script)
So,
test.sh 360 &
will pass 360 as the first argument to the bash or ksh script which can be referred to as $1 in the script.
So the first few lines of your script would now be:
#!/usr/bin/ksh
scriptduration=$1
loopcnt=0
...
...
With bash you can start the script in the foreground and after you finished with the user input, interrupt it by hitting Ctrl-Z.
Then type
$ bg %
and the script will continue to run in the background.
Why You're Getting What You're Getting
When you run the script in the background, it can't take any user input. In fact, the program will freeze if it expects user input until its put back in the foreground. However, output has to go somewhere. Thus, the output goes to the screen (even though the program is running in the background. Thus, you see the prompt.
The prompt you see your program displaying is meaningless because you can't input at the prompt. Instead, you type in 360 and your shell is interpreting it as a command you want because you're not putting it in the program, you're putting it in the command prompt.
You want your program to be in the foreground for the input, but run in the background. You can't do both at once.
Solutions To Your Dilemma
You can have two programs. The first takes the input, and the second runs the actual program in the background.
Something like this:
#! /bin/ksh
read time?"How long in seconds do you want to run the job? "
my_actual_job.ksh $time &
In fact, you could even have a mechanism to run the job in the background if the time is over a certain limit, but otherwise run the job in the foreground.
#! /bin/ksh
readonly MAX_FOREGROUND_TIME=30
read time?"How long in seconds do you want to run the job? "
if [ $time -gt $MAX_FOREGROUND_TIME ]
then
my_actual_job.ksh $time &
else
my_actual_job.ksh $time
fi
Also remember if your job is in the background, it cannot print to the screen. You can redirect the output elsewhere, but if you don't, it'll print to the screen at inopportune times. For example, you could be in VI editing a file, and suddenly have the output appear smack in the middle of your VI session.
I believe there's an easy way to tell if your job is in the background, but I can't remember it offhand. You could find your current process ID by looking at $$, then looking at the output of jobs -p and see if that process ID is in the list. However, I'm sure someone will come up with an easy way to tell.
It is also possible that a program could throw itself into the background via the bg $$ command.
Some Hints
If you're running Kornshell, you might consider taking advantage of many of Kornshell's special features:
print: The print command is more flexible and robust than echo. Take a look at the manpage for Kornshell and see all of its features.
read: You notice that you can use the read var?"prompt" form of the read command.
readonly: Use readonly to declare constants. That way, you don't accidentally change the value of that variable later. Besides, it's good programming technique.
typeset: Take a look at typeset in the ksh manpage. The typeset command can help you declare particular variables as floating point vs. real, and can automatically do things like zero fill, right or left justify, etc.
Some things not specific to Kornshell:
The awk and sed commands can also do what grep does, so there's no reason to filter something through grep and then through awk or sed.
You can combine greps by using the -e parameter. grep foo | grep bar is the same as grep -e foo -e bar.
Hope this helps.
I've tested this with ksh and it worked. The trick is to let the script call itself with the time to wait as parameter:
if [ -z "$1" ]; then
echo "How long do you want the test to run for in minutes:\c"
read scriptduration
echo "running task in background"
$0 $scriptduration &
exit 0
else
scriptduration=$1
fi
loopcnt=0
interval=1
# ... and so on
So are you using bash or ksh? In bash, you can do this:
{ echo 360 | ./test.sh ; } &
It could work for ksh also.
I have a bash script that sources contents from another file. The contents of the other file are commands I would like to execute and compare the return value. Some of the commands are have multiple commands separated by either a semicolon (;) or by ampersands (&&) and I can't seem to make this work. To work on this, I created some test scripts as shown:
test.conf is the file being sourced by test
Example-1 (this works), My output is 2 seconds in difference
test.conf
CMD[1]="date"
test.sh
. test.conf
i=2
echo "$(${CMD[$i]})"
sleep 2
echo "$(${CMD[$i]})"
Example-2 (this does not work)
test.conf (same script as above)
CMD[1]="date;date"
Example-3 (tried this, it does not work either)
test.conf (same script as above)
CMD[1]="date && date"
I don't want my variable, CMD, to be inside tick marks because then, the commands would be executed at time of invocation of the source and I see no way of re-evaluating the variable.
This script essentially calls CMD on pass-1 to check something, if on pass-1 I get a false reading, I do some work in the script to correct the false reading and re-execute & re-evaluate the output of CMD; pass-2.
Here is an example. Here I'm checking to see if SSHD is running. If it's not running when I evaluate CMD[1] on pass-1, I will start it and re-evaluate CMD[1] again.
test.conf
CMD[1]=`pgrep -u root -d , sshd 1>/dev/null; echo $?`
So if I modify this for my test script, then test.conf becomes:
NOTE: Tick marks are not showing up but it's the key below the ~ mark on my keyboard.
CMD[1]=`date;date` or `date && date`
My script looks like this (to handle the tick marks)
. test.conf
i=2
echo "${CMD[$i]}"
sleep 2
echo "${CMD[$i]}"
I get the same date/time printed twice despite the 2 second delay. As such, CMD is not getting re-evaluate.
First of all, you should never use backticks unless you need to be compatible with an old shell that doesn't support $() - and only then.
Secondly, I don't understand why you're setting CMD[1] but then calling CMD[$i] with i set to 2.
Anyway, this is one way (and it's similar to part of Barry's answer):
CMD[1]='$(date;date)' # no backticks (remember - they carry Lime disease)
eval echo "${CMD[1]}" # or $i instead of 1
From the couple of lines of your question, I would have expected some approach like this:
#!/bin/bash
while read -r line; do
# munge $line
if eval "$line"; then
# success
else
# fail
fi
done
Where you have backticks in the source, you'll have to escape them to avoid evaluating them too early. Also, backticks aren't the only way to evaluate code - there is eval, as shown above. Maybe it's eval that you were looking for?
For example, this line:
CMD[1]=`pgrep -u root -d , sshd 1>/dev/null; echo $?`
Ought probably look more like this:
CMD[1]='`pgrep -u root -d , sshd 1>/dev/null; echo $?`'