It is common to use crontab to schedule a task. However, I found that crontab can only schedule tasks on a minutely, hourly or daily basis. If I want to schedule task on every 4 hours, I can still make it by setting up 6 daily task. However, what if I want to have a period that doesn't make up one day, say, a task every 5 hours / 7 hours? Is there a way to set up the task by hour period in crontab or using other tools? (would be even better if that tool can be run on PaaS like Heroku / Openshift)
Here is an example of using the minutely crontab to setup a task that happens every 5 minutes (https://github.com/openshift-quickstart/openshift-cacti-quickstart/blob/master/.openshift/cron/minutely/cactipoll)
if [ ! -f $OPENSHIFT_DATA_DIR/last_run ]; then
touch $OPENSHIFT_DATA_DIR/last_run
fi
if [[ $(find $OPENSHIFT_DATA_DIR/last_run -mmin +4) ]]; then #run every 5 mins
rm -f $OPENSHIFT_DATA_DIR/last_run
touch $OPENSHIFT_DATA_DIR/last_run
php $OPENSHIFT_REPO_DIR/php/poller.php > $OPENSHIFT_DATA_DIR/cacti.log 2>&1
fi
Basically the task runs every minute, but only gets executed after 5 minutes have passed. You can easily adapt this to use hours instead of minutes.
Related
The problem I want to tackle is as follows. I have a long(1 to 2 hours) running task that has to be run everyday. So the goto option was cron. But the catch is that I have to give a 24 hour gap between successive runs. So using cron now would involve rewriting the cron job file after every run. This might be clear after this example.
The long running job 'LR' starts at 6PM on Monday and finishes at 7:30PM sameday.
On Tuesday it's supposed to start at 7:30 PM and not 6PM (like it did on monday). This is because there has to be a 24hr gap between successive runs.
The obvious option here was to have a process running an infinite loop. start the LR job. Then sleep for 24hr and continue with the loop. This works perfectly too. In my setup there is a bash script which is running this loop.
while [ 1 == 1 ]; do
/bin/jobs/long_run.py
/bin/jobs/cleanup.sh
sleep 86400
done
So my question is what is the total amount of CPU resource spent and what is the RAM usage.
Not sure if this affects the answer in anyway; I'm running this on termux on an android phone.
Also please recommend other light weight options.
There is nothing to worry about resources, while a script executes sleep, it really sleeps. You should worry for if anything happens between two executions, like restart, downtime etc. This structure:
while true; do
sh script.sh
sleep 86400
done
does not resume and you don't save the time for the next execution anywhere. Similar to this structure is to have a wrapper, suppose f() is your jobs
f() {
echo working
}
wrapper() {
f
echo sleeping
sleep 86400
wrapper
}
wrapper
so now you call the wrapper, which works, sleeps and calls itself. You could use just this, if you are ok with what could go wrong, at least print the datetime somewhere.
You can replace the internal sleep and wrapper call with job scheduling with cron or at. Probably at is not a standard packet for all distributions (at least not for mine) while cron is. You could install it. For at the wrapper would be like this:
wrapper() {
f
at now +1 day wrapper
}
With cron, you could edit the crontab, like this but better use a crontab file like this, what you have to do is to parse date command, create the date prefix, update crontab.
Note: There may be other cron jobs for user, existing or added after that, this is considered in the last link.
I have a script that runs through a list of servers to connect to and grabs by SCP over files to store
Occasionally due to various reasons one of the servers crashes and my script gets stuck for around 4 hours before moving on through the list.
I would like to be able to detect a connection issue or a period of time elapsed after script has started and kill that command and move on to next.
I suspect that this would involve a wait or sleep and continue but I am new to loops and bash
#!/bin/bash
#
# Generate a list of backups to grab
df|grep backups|awk -F/ '{ print $NF }'>/tmp/backuplistsmb
# Get each backup in turn
for BACKUP in `cat /tmp/backuplistsmb`
do
cd /srv/backups/$BACKUP
scp -o StrictHostKeyChecking=no $BACKUP:* .
sleep 3h
done
The above script works fine but does get stuck for 4 hours should there be a connection issue. It is worth noting that some of the transfers take 10 mins and some 2.5 hours
Any ideas or help would very appreciated
Try to use the timeout program for that:
Usage:
timeout [OPTION] DURATION COMMAND [ARG]...
E.g. time (timeout 3 sleep 5) will run for 3 secs.
So in your code you can use:
timeout 300 scp -o StrictHostKeyChecking=no $BACKUP:* .
This limits the copy to 5 minutes.
I need to load a file every seconds, which means, there is not a determinate time of the day.
I am using cPanel to run the CronJob and the task (the wget) is already working fine and I am just having trouble with the time schedules.
I have tried:
Minute Hour Day Month Weekday
*/1 * * * *
running every minute:
Minute Hour Day Month Weekday
* * * * *
it seems a bit odd why you run it every second. Cron job cannot be used to schedule a job in seconds interval. i.e You cannot schedule a cron job to run every 1 seconds. The alternative is to write a shell script that uses ‘sleep 1′ command in it.
$ cat every-1-second.sh
#!/bin/bash
while true
do
/home/ramesh/backup.sh
sleep 1
done
This question already has answers here:
Cron jobs and random times, within given hours
(13 answers)
Closed 9 years ago.
Need run a shell script once a day at random time. (so once every day between 00:00-23:59).
I know the sleep command, and the cron too, but
the cron has not random times
and the sleep solution - not very nice - my idea is launch the script every midnight and sleep random time at the start of the script.
Is here something more elegant?
If you have the at command, you can combinte the cron and the at.
Run from a cron every midnight the next script:
#!/bin/bash
script="/tmp/script.sh" #insert the path to your script here
min=$(( 24 * 60 ))
rmin=$(( $RANDOM % $min ))
at -f "$script" now+${rmin}min
The above will run the at command every midnight and will execute your script at random time . You should check your crontab how often is the atrun command started. (The atrun runs the commands stored with the at)
The main benefit in comparison with the sleep method: this "survives" the system reboot.
I would simply launch you script at midnight, and sleep for a random time between 0 and 86400 seconds. Since my bash's $RANDOM returns a number between 0 and 32767:
sleep $(( ($RANDOM % 1440)*60 + ($RANDOM % 60) ))
The best alternative to cron is probably at
See at man page
Usually, at reads commands from standard input, but you can give a file of jobs with -f.
Time wise, you can specify many formats. Maybe in your case the most convenient would be
at -f jobs now + xxx minutes
where your scripts gives xxx as a random value from 1 to 1440 (1440 minutes in a day), and jobs contains the commands you want to be executed.
Nothing prevents you from running sed to patch your crontab as the last thing your program does and just changing the next start time. I wouldn't sleep well though.
You can use cron to launch bash script, which generates pseudorandom timestamp and gives it to unix program at
I see you are familiar with bash and cron enough, so at will be a piece of cake for you. Documentation as always "man at" or you can try wiki
http://en.wikipedia.org/wiki/At_(Unix)
How to start a shell script in one minute later?
Suppose there are two bash files a.sh and b.sh
I want to execute b.sh one minute(or several seconds) after a.sh executed.
what should I code in a.sh ?
Simple. you want to use 'at' to schedule your job. and 'date' to calculate your moment in the future.
Example:
echo b.sh | at now + 1 minute
or:
echo b.sh | at -t `date -v+60S "+%Y%m%d%H%M%S"`
-v+60S adds 60 seconds to current time. You can control exactly how many seconds you want to add.
But usually, when people wants one program to launch a minute after the other, they are not 100% sure it will not take more or less than a minute. that's it. b.sh could be launched before a.sh is finished. or a.sh could have finished 30 seconds earlier than "planned" and b.sh could have started faster.
I would recommend a different model. Where b.sh is launched first.
a.sh creates a temp file when it starts. execute is tasks and delete its temp file at the end.
b.sh watch for the temp file to be created, then deleted. and start its tasks.
Make the final line of a.sh:
sleep 60 && b.sh
(If b.sh is not in a directory in PATH, make that a full path to b.sh.)
You can just sleep:
a.sh
sleep 60
b.sh
Or for more complicated cases you can use the at command:
echo b.sh | at now + 1 minute
See the at man page for more information.
Use the at command.
See man at for how to use it.
You could use the command to sleep your script for 1 minute.
sleep 1m
Then when you wish to call the 2nd script
bash a.sh
If you want to execute the second script some number of seconds after the start of the first script, you can do this in the first:
b.sh &
and this in the second:
sleep 10
# more commands
You could pass the number of seconds as an argument from the first to the second.
Unfortunately, at doesn't do time increments finer than one minute.
Schedule both the scripts to run at the same time in cron and put the required delay in b.sh.