am writing a crontab script, which will run on each Saturday for every 15minutes. The idea is to validate an external api status =SUCCESS or not. If its success, then the cronjob for the day should not trigger any more.
Right now am trying with recursion, but I dont think so that is a best solution.
Is there any other solution to achieve this? am using Shell script to invoke api.
Here is the existing snippet:
Cronjob:
*/15 * * * 6 validate.sh
script:
status='curl -X GET "api"'
if [[ $status == "SUCCEEDED" ]];then
trigger email
else sleep 180
./validate.sh
fi
Add another cron job so it removes the flag file on Friday evening, before the other job starts running:
59 23 * * 5 rm .succeeded.txt
Then change your script so it aborts if this file exists, and creates it when it succeeds.
#!/bin/bash
test -e .succeeded.txt && exit
if [[ $(curl -X GET "api") == "SUCCEEDED" ]];then
trigger email
touch .succeeded.txt
fi
I tried to fix other errors in your script, too, but I had to guess many things. This assumes "SUCCEEDED" is the sole output from curl when the GET works.
Putting the command in a variable is a useless complication which makes your script longer and (very slightly) slower, but in addition, it creates problems of its own when the command contains embedded quotes; see e.g. http://mywiki.wooledge.org/BashFAQ/050
... But of course, presumably you wanted to actually run the command. Your attempt would merely check whether the string in the variable was equal to "SUCEEDED" which of course it would never be.
Another problem was that you were spawning multiple validate.sh jobs, each of which would recurse and retry. You want one or the other, not both. I went with keeping your schedule and just trying once in each job.
Related
I have a Bash script that gets executed inside Codebuild--it currently reads the last line in a file and, depending on what's in it, kicks off some AWS cli commands to deploy a new Lambda. I'm trying to update the script to read multiple lines and be able to deploy multiple Lambdas, but my current attempt at just putting the original code in a loop for each line in the file failed with the error An error occurred (ResourceConflictException) when calling the UpdateFunctionCode operation: The operation cannot be performed at this time. An update is in progress for resource blahblahblah.
I think putting the CLI command inside a loop and sleeping until the command executes successfully should work. However, I'm pretty inexperienced with Bash and not quite sure of the syntax to do it, and it's not the easiest thing to test locally. Something like:
mins=0
while [aws lambda update-function-code blahblahblah -ne 0] && [ $mins -lt 5 ]
do
echo "Resource conflict...sleeping for 1 min"
sleep 1m
mins=$(( $mins + 1 ))
done
There are actual args instead of blahblahblah, and I'm not sure how long I actually want to wait for the resource conflict to resolve. Also, earlier in the script, set -eou pipefail has been called, so I don't know if I'd need to unset it or do something special to capture the output of the AWS command.
I'd most like to know how to actually write the code using this logic, but if there are any problems with doing it this way that I should account for, or entirely different and better ways of doing it, that would be helpful, too.
How do I avoid cronjob from executing multiple times on the same command? I had tried to look around and try to check and kill in processes but it doesn't work with the below code. With the below code it keeps entering into else condition where it suppose to be "running". Any idea which part I did it wrongly?
#!/bin/sh
devPath=`ps aux | grep "[i]mport_shell_script"` | xargs
if [ ! -z "$devPath" -a "$devPath" != " " ]; then
echo "running"
exit
else
while true
do
sudo /usr/bin/php /var/www/html/xxx/import_from_datafile.php /dev/null 2>&1
sleep 5
done
fi
exit
cronjob:
*/2 * * * * root /bin/sh /var/www/html/xxx/import_shell_script.sh /dev/null 2>&1
I don't see the point to add a cron job which then starts a loop that runs a job. Either use cron to run the job every minute or use a daemon script to make sure your service is started and is kept running.
To check whether your script is already running, you can use a lock directory (unless your daemon framework already does that for you):
LOCK=/tmp/script.lock # You may want a better name here
mkdir $LOCK || exit 1 # Exit with error if script is already running
trap "rmdir $LOCK" EXIT # Remove the lock when the script terminates
...normal code...
If your OS supports it, then /var/lock/script might be a better path.
Your next question is probably how to write a daemon. To answer that, I need to know what kind of Linux you're using and whether you have things like systemd, daemonize, etc.
check the presence of a file at the beginning of your script ( for example /tmp/runonce-import_shell_script ). If it exists, that means the same script is already running (or the previous one halted with an error).
You can also add a timestamp in that file so you can check since when the script was running (and maybe decide to run it again after 24h even if the file is present)
I have a simple bash script which calls a php script every 10 minutes thats performs some maintenance. Every once in a while this php script terminates while it's running and when this happens the bash script exits.
I'd like to make it so the bash script keeps on looping even if the php script falters. Can anyone point me in the right direction? I've been searching for a while but I can't seem to find the answer, maybe I'm not using the right search terms.
#!/bin/sh
set -e
while :
do
/usr/bin/php /path/to/maintenance/script.php
sleep 600
done
Rjz's comment is correct, you should use cron. To do that, run crontab -e and add this line:
*/10 * * * * /usr/bin/php /path/to/maintenance/script.php
If it's set up properly, cron will email you any output (including error messages).
The:
set -e
line sets the shell's "exit on error" flag, which tells it that if a program it runs exits with a non-zero status, the shell should also exit:
set -e
false
echo if this prints, your shell is not honoring "set -e"
There are exceptions for programs whose status is being tested, of course, so that:
set -e
if prog; then
echo program succeeded
else
echo program failed
fi
echo this will still print
will work correctly (one or or the other echo will occur, and then the last one will as well).
Back in the Dim Time, when /bin/sh was non-POSIX and was written in Bournegol, there was a bug in some versions of sh that broke || expressions:
set -e
false || true
echo if this prints, your shell is OK
(The logic bug applied to && expressions internally as well, but was harmless there, since false && anything is itself false which means the whole expression fails anyway!) Ever since then, I've been wary of "-e".
Not having much luck Googling this question and I thought about posting it on SF, but it actually seems like a development question. If not, please feel free to migrate.
So, I have a script that runs via cron every morning at about 3 am. I also run the same scripts manually sometimes. The problem is that every time I run my script manually and it fails, it sends me an e-mail; even though I can look at the output and view the error in the console.
Is there a way for the bash script to tell that it's being run through cron (perhaps by using whoami) and only send the e-mail if so? I'd love to stop receiving emails when I'm doing my testing...
you can try "tty" to see if it's run by a terminal or not. that won't tell you that it's specifically run by cron, but you can tell if its "not a user as a prompt".
you can also get your parent-pid and follow it up the tree to look for cron, though that's a little heavy-handed.
I had a similar issue. I solved it with checking if stdout was a TTY. This is a check to see if you script runs in interactive mode:
if [ -t 1 ] ; then
echo "interacive mode";
else
#send mail
fi
I got this from: How to detect if my shell script is running through a pipe?
The -t test return true if file descriptor is open and refers to a terminal. '1' is stdout.
Here's two different options for you:
Take the emailing out of your script/program and let cron handle it. If you set the MAILTO variable in your crontab, cron will send anything printed out to that email address. eg:
MAILTO=youremail#example.com
# run five minutes after midnight, every day
5 0 * * * $HOME/bin/daily.job
Set an environment variable in your crontab that is used to determine if running under cron. eg:
THIS_IS_CRON=1
# run five minutes after midnight, every day
5 0 * * * $HOME/bin/daily.job
and in your script something like
if [ -n "$THIS_IS_CRON" ]; then echo "I'm running in cron"; else echo "I'm not running in cron"; fi
Why not have a command line argument that is -t for testing or -c for cron.
Or better yet:
-e=email#address.com
If it's not specified, don't send an email.
I know the question is old, but I just came across the same problem. This was my solution:
CRON=$(pstree -s $$ | grep -q cron && echo true || echo false)
then test with
if $CRON
then
echo "Being run by cron"
else
echo "Not being run by cron"
fi
same idea as the one that #eruciform mentioned - follows your PID up the process tree checking for cron.
Note: This solution only works specifically for cron, unlike some of the other solutions, which work anytime the script is being run non-interactively.
What works for me is to check $TERM. Under cron it's "dumb" but under a shell it's something else. Use the set command in your terminal, then in a cron-script and check it out
if [ "dumb" == "$TERM" ]
then
echo "cron"
else
echo "term"
fi
I'd like to suggest a new answer to this highly-voted question. This works only on systemd systems with loginctl (e.g. Ubuntu 14.10+, RHEL/CentOS 7+) but is able to give a much more authoritative answer than previously presented solutions.
service=$(loginctl --property=Service show-session $(</proc/self/sessionid))
if [[ ${service#*=} == 'crond' ]]; then
echo "running in cron"
fi
To summarize: when used with systemd, crond (like sshd and others) creates a new session when it starts a job for a user. This session has an ID that is unique for the entire uptime of the machine. Each session has some properties, one of which is the name of the service that started it. loginctl can tell us the value of this property, which will be "crond" if and only if the session was actually started by crond.
Advantages over using environment variables:
No need to modify cron entries to add special invocations or environment variables
No possibility of an intermediate process modifying environment variables to create a false positive or false negative
Advantages over testing for tty:
No false positives in pipelines, startup scripts, etc
Advantages over checking the process tree:
No false positives from processes that also have crond in their name
No false negatives if the script is disowned
Many of the commands used in prior posts are not available on every system (pstree, loginctl, tty). This was the only thing that worked for me on a ten years old BusyBox/OpenWrt router that I'm currently using as a blacklist DNS server. It runs a script with an auto-update feature. Running from crontab, it sends an email out.
[ -z "$TERM" ] || [ "$TERM" = "dumb" ] && echo 'Crontab' || echo 'Interactive'
In an interactive shell the $TERM-variable returns the value vt102 for me. I included the check for "dumb" since #edoceo mentioned it worked for him. I didn't use '==' since it's not completely portable.
I also liked the idea from Tal, but also see the risk of having undefined returns. I ended up with a slightly modified version, which seems to work very smooth in my opinion:
CRON="$( pstree -s $$ | grep -c cron )"
So you can check for $CRON being 1 or 0 at any time.
I have a cron job that runs every hour. It accesses an xml feed. If the xml feed is unvailable (which seems to happen once a day or so) it creates a "failure" file. This "failure" file has some metadata in it and is erased at the next hour when the script runs again and the XML feed works again.
What I want is to make a 2nd cron job that runs a few minutes after the first one, looks into the directory for a "failure" file and, if it's there, retries the 1st cron job.
I know how to set up cron jobs, I just don't know how to make scripting conditional like that. Am I going about this in entirely the wrong way?
Possibly. Maybe what you'd be better off doing is having the original script sleep and retry a (limited) number of times.
Sleep is a shell command and shells support looping so it could look something like:
for ((retry=0;retry<12;retry++)); do
try the thing
if [[ -e my_xlm_file ]]; then break; fi
sleep 300
# five minutes later...
done
As the command to run, try:
/bin/bash -c 'test -e failurefile && retrycommand -someflag -etc'
It runs retrycommand if failurefile exists
Why not have your set your script touch a status file when it has successfully completed. Have it run every 5 minutes, and have the first check of the script be to see if the status file is less then 60 minutes old, and if it is young, then quit, if it is old, then fetch.
I agree with MarkusQ that you should retry in the original job instead of creating another job to watch the first job.
Take a look at this tool to make retrying easier: https://github.com/kadwanev/retry
You can just wrap the original cron in a retry very easily and the final existence of the failure file would indicate if it failed even after the retries.
If somebody will need a bash script to ping an endpoint (for example, run scheduled API tasks via cron), retry it, if the response status was bad, then:
#!/bin/bash
echo "Start pinch.sh script.";
# run 5 times
for ((i=1;i<=5;i++))
do
# run curl to do a POST request to https://www.google.com
# silently flush all its output
# get the response status code as a bash variable
http_response=$(curl -o /dev/null -s -w "%{response_code}" https://www.google.com);
# check for the expected code
if [ $http_response != "200" ]
then
# process fail
echo "The pinch is Failed. Sleeping for 5 minutes."
# wait for 300 seconds, then start another iteration
sleep 300
else
# exit from the cycle
echo "The pinch is OK. Finishing."
break;
fi
done
exit 0