I am facing a problem with the scripts which i am using in my code, my cronjob runs every 5 mins but the scripts which it is running some time takes more time and i want my cronjob to wait for those files to finish its processing and then execute in the earliest interval is it possible?
Please see below example. Kindly propose me a solution. TIA.
I am running a cronjob for e.g.:
*/5 * * * * /home/Sti/New_Int/fetch_My_Data.sh
This job is invoking below scripts and few details about what each script is doing
fetch_Some_Data.sh --> This script is just moving few files from one location to another so that the only required files can be processed.
tran.sh --> This script opens a for loop and for each file it will open a DB connection by invoking PostP.sh script and for processing it has a sleep time of 60 seconds.
PostP.sh --> This is a script which creates a DB connection and terminates it for each file which is being processed in point 2.
So can you provide me a solution so that if the files are not processed in point 2 the cronjob won't run till then
I usually use a temporary file in such cases to indicate a running instance, and till that file exists all other instances simply exit or Error.
Add this logic to your shell script before doing anything else:
if exists file
then
exit
end if
else
touch empty file
end else
Related
I already built a script trigger_jenkins_job.sh which works perfectly fine for now. It’s composed mainly of 3 functions:
input_checkpoint
run_remotejob #: Running Jenkins job remotely using Json api.
sleep 10 #: 10 sec estimated time until pending duration is over
#and Jenkins job start running, i.e. a given slave was
#assigned to run the job.
get_buildID #: Retrieving build state, last build ID and last stable
#build ID using
The problem is I want to get rid of that sleep 10 seconds. And in the same time, I want to be sure before executing the function get_buildID that the remotely- triggered job is actually running on a node.
That way I will be retrieving the triggered job’s id, and not the last one in the queue before triggering that job.
Regarding the Jenkins file of the job, I specified:
agent {
label 'linux-node'
}
So, I guess the question is, I need some how from by bash script, to test if linux-node is running the remotely-triggered job, and if yes I execute the function get_buildID.
Get rid of the sleep command and use the wait command.
If you are triggering Job with tokens,it command itself should return you buildNumber.
Another way could be REST API. Please see "nextBuildNumber" field there (if build is still pending) else "number"
I have the following cron tasks:
/etc/cron.d/mongo
/etc/cron.d/elastic
These cron jobs only executes scripts located here:
/etc/script/mongo
/etc/script/elastic
These tasks execute every 30 minuts, that's the following cron format:
0,30 * * * *
I don't know why but these tasks aren't executing all times. In 2 hours, for example, they only execute 2-3 times and not 4. These tasks are performing backups, I need to be sure that they execute every 30 minuts.
Why this is happening?
PD: If i execute crontab -e the file is empty, this may cause any problem?
I have written a shell script which includes some commands to collect various Test logs.But I want to put waiting time before a particular command so that it will run after 30 mins after starting the shell script.Anyone having the knowledge regarding it please help me out.
Thanks in advance
You can use the at command to schedule a job to run at any time:
echo "/my_path/my_script" | at now + 30 minutes
If you want a delay inside your script you can sleep to a specific time using this solution.
I have a simple bash script that runs some tasks which can take varying amounts of time to complete (from 15 mins to 5 hours). The script loops using a for loop, so that I can run it an arbitrary number of times, normally back-to-back.
However, I have been requested to have each iteration of the script start at the top of the hour. Normally, I would use cron and kick it off that way, every hour, but since the runtime of the script is highly variable, that becomes trickier.
It is not allowable for multiple instances of the script to be running at once.
So, I'd like to include the logic to wait for 'top of the hour' within the script, but I'm not sure of the best way to do that, or if there's some way to (ab)use 'at' or something more elegant like that. Any ideas?
You can still use cron. Just make your script use a lock file. With the flock utility you can do:
#!/bin/bash
exec 42> /tmp/myscriptname.lock
flock -n 42 || { echo "Previous instance still running"; exit 1; }
rest of your script here
Now, simply schedule your job every hour in cron, and the new instance will simply exit if the old one's still running. There is no need to clean up any lock files.
I have a question.
I want to run more instance of same job in parallel from within a script: I have a loop in which I invoke jobs with dsjob and without option "-wait" and "-jobstatus".
I want that jobs completed before script termination, but I don't know how to verify if job instance terminated.
I though to use wait command but it is not appropriate.
Thanks in advance
First,you should assure job compile option "Allow Multiple Instance" choose.
Second:
#!/bin/bash
. /home/dsadm/.bash_profile
INVOCATION=(1 2 3 4 5)
cd $DSHOME/bin
for id in ${INVOCATION[#]}
do
./dsjob -run -mode NORMAL -wait test demo.$id
done
project -- test
job -- demo
$id -- invocation id
the two line in shell scipt:guarantee the environment path can work.
Run the jobs like you say without the -wait, and then loop around running dsjob -jobinfo and parse the output for a job status of 1 or 2. When all jobs return this status, they are all finished.
You might find, though, that you check the status of the job before it actually starts running and you might pick up an old status. You might be able to fix this by first resetting the job instance and waiting for a status of "Not running", prior to running the job.
Invoke the jobs in loop without wait or job-status option
after your loop , check the jobs status by dsjob command
Example - dsjob -jobinfo projectname jobname.invocationid
you can code one more loop for this also and use sleep command under that
write yours further logic as per status of the jobs
but its good to create Job Sequence to invoke this multi-instance job simultaneously with the help of different invoaction-ids
create a sequence job if these are in same process
create different sequences or directly create different scripts to trigger these jobs simultaneously with invocation- ids and schedule in same time.
Best option create a standard generalized script where each thing will be getting created or getting value as per input command line parameters
Example - log files on the basis of jobname + invocation-id
then schedule the same script for different parameters or invocations .