I'm working on a bash script that will run for appoximately 30 minutes at a time. I've got it running stable as far as that part goes. I've been looking for a way to make it fire certain commands at inervals of every 3 minutes while running. I've not had any luck, so I turn to those of you that may know more about bash than I.
Any suggestions?
Here is what I have in mind of doing.
START=$(date +%s);
while read LINE <&3; do
END=$(date +%s);
if [[ $(($END-$START)) > 180 || $(($END-$START)) == 180 ]]
then
$START=$(date +%s);
run command
fi
done
Add a cron job to make it run every 3 minutes.
*/3 * * * * /path/to/script
What about the watch command?? (https://unix.stackexchange.com/questions/10646/repeat-a-unix-command-every-x-seconds-forever)
(Second answer on here: Run command every second)
You can run a loop in the background:
{ while /bin/true; do some_command; sleep 180; done; } &
loop_pid=$!
Then before the main script exits, kill the background loop:
kill $loop_pid
You can also call the same script from same script.
$ cat script.sh
#!/bin/bash
# commands
# commands
sleep 1800
sh $0
Related
right now I'm using this script for a program:
export FREESURFER_HOME=$HOME/freesurfer
source $FREESURFER_HOME/SetUpFreeSurfer.sh
cd /home/ubuntu/fastsurfer
datadir=/home/ubuntu/moya/data
fastsurferdir=/home/ubuntu/moya/output
mkdir -p $fastsurferdir/logs # create log dir for storing nohup output log (optional)
while read p ; do
echo $p
nohup ./run_fastsurfer.sh --t1 $datadir/$p/orig.nii \
--parallel --threads 16 --sid $p --sd $fastsurferdir > $fastsurferdir/logs/out-${p}.log &
sleep 3600s
done < /home/ubuntu/moya/data/subjects-list.txt
Instead of using sleep 3600s, as the program needs around an hour, I'd like to use wait until all processes (several PIDS) are finished.
If this is the right way, can you tell me how to do that?
BR Alex
wait will wait for all background processes to finish (see help wait). So all you need is to run wait after creating all of the background processes.
This may be more than what you are asking for but I figured I would provide some methods for controlling the number of threads you want to have running at once. I find that I always want to limit the number for various reasons.
Explaination
The following will limit concurrent threads to max_threads running at one time. I am also using the main design pattern so we have a main that runs the script with a function run_jobs that handles the calling and waiting. I read all of $p into an array, then traverse that array as we launch threads. It will either launch a thread up to 4 or wait 5 seconds, once there are at least one less than four it will start another thread. When finished it waits for any remaining to be done. If you want something more simplistic I can do that as well.
#!/usr/bin/env bash
export FREESURFER_HOME=$HOME/freesurfer
source $FREESURFER_HOME/SetUpFreeSurfer.sh
typeset max_threads=4
typeset subjects_list="/home/ubuntu/moya/data/subjects-list.txt"
typeset subjectsArray
run_jobs() {
local child="$$"
local num_children=0
local i=0
while [[ 1 ]] ; do
num_children=$(ps --no-headers -o pid --ppid=$child | wc -w) ; ((num_children-=1))
echo "Children: $num_children"
if [[ ${num_children} -lt ${max_threads} ]] ;then
if [ $i -lt ${#subjectsArray[#]} ] ;then
((i+=1))
# RUN COMMAND HERE &
./run_fastsurfer.sh --t1 $datadir/${subjectsArray[$i]}/orig.nii \
--parallel --threads 16 --sid ${subjectsArray[$i]} --sd $fastsurferdir
fi
fi
sleep 10
done
wait
}
main() {
cd /home/ubuntu/fastsurfer
datadir=/home/ubuntu/moya/data
fastsurferdir=/home/ubuntu/moya/output
mkdir -p $fastsurferdir/logs # create log dir for storing nohup output log (optional)
mapfile -t subjectsArray < ${subjects_list}
run_jobs
}
main
Note: I did not run this code since you have not provided enough information to actually do so.
I have an Shell script that runs a PHP script every 8 seconds as seen below:
enter code here
#!/bin/bash
while true; do
php /var/www/html/folder1/scripts/processtask.php /var/www/html/folder1/app
sleep 8;
done
enter code here
I need it to continue this way, however I need it to stop running at 2am until 3am then resume after that on a daily basis
You can use cronetab for the same. Check the below link for configuration:
https://www.baeldung.com/linux/schedule-script-execution
If you only want to start the script if not between 2am and 3am (but an allready started script does not need to be stopped) this could be easily checked with an additional if-statement:
#!/bin/bash
while true; do
if [[ 2 -le $(date '+%H') && $(date '+%H') -le 3 ]]; then
sleep 60
else
php /var/www/html/folder1/scripts/processtask.php /var/www/html/folder1/app
sleep 8
fi
done
I have written a shell script that runs some commands. I have added a logic to run this script once every 24 hours. But it runs once and then doesn't run.
The script is as below:
#!/bin/bash
while true; do
cd /home/ubuntu/;
DATE=`date '+%Y-%m-%d'`;
aws s3 cp --recursive "/home/ubuntu/" s3://bucket_name/$DATE/;
rm -r -f ./*;
# sleep 24 hours
sleep $((24 * 60 * 60))
done
Why does it not run once every 24 hours ? I do not get any errors when the script runs. The copy takes about 10 mins.
The good practice is to protect your script againt multirunning.
In this case, you can be sure that only 1 instance is running.
#!/bin/bash
LOCKFILE=/tmp/block_file
if ( set -o noclobber; echo "$$" > "$LOCKFILE") 2> /dev/null;
then
trap 'rm -f "$LOCKFILE"; exit $?' INT TERM EXIT
while true; do
cd /home/ubuntu/;
DATE=`date '+%Y-%m-%d'`;
aws s3 cp --recursive "/home/ubuntu/" s3://bucket_name/$DATE/;
rm -r -f ./*;
# sleep 24 hours
sleep $((24 * 60 * 60))
done
rm -f "$LOCKFILE"
trap - INT TERM EXIT
else
echo "Warning. Script is already running!"
echo "Block by PID $(cat $LOCKFILE) ."
exit
fi
You can run a script immune to hangups.
nohup is a UNIX utility that runs the specified command ignoring communication loss signals (SIGHUP). Thus, the script will continue to work in the background even after the user logs out.
nohup ./yourscript.sh
The created file /tmp/block_file will safe runned script against multirunning. To complete it press ctrl+c or run kill -11 pidofyourscript in terminal, in this way /tmp/block_file will be deleted.
The output of script puts on file nohup.out.
To run in background (preferred way):
nohup ./yourscript.sh &
Your script is probably killed due to inactivity, or when you exit the shell. The proper way to do this is use cron, as #Christian.K mentioned. See https://help.ubuntu.com/community/CronHowto
I have a code which crontab is not able to run
i am using ./filename.sh to run it. when i do this manually this runs fine
but via cron when i try
*/5 * * * * . /home/ubuntu/filename.sh >> /filename.sh
This doesn't work
for ((i=0; i<retries; i++)); do
curl -1 --cipher ALL --connect-timeout 90 -T $zip_name ftps://ftp.box.com/Backup/$zip_name --user admin:pas [[ $? -eq 0 ]] && break
echo "something went wrong, let's wait 6 seconds and retry"
sleep 6
done
[[ $retries -eq i ]] && { echo "This email is being sent as a notifer of Failure, Support" | mail -s "Dump Failed" "asdfas4#gmail.com" ; exit 1; }
Also when i run this using sh
it says Syntax error: Bad for loop variable
Presumably, the shell of your cron is dash (not bash). In Ubuntu (and derivatives) sh is just a symlink for dash.
You can:
Add a shebang at the top of of your script -- #!/bin/bash (or #!/usr/bin/env bash), recommended approach
Run the script as an argument to bash: /bin/bash /path/to/script.sh, moderately recommended
Set SHELL=/bin/bash in your crontab (not recommended), or even you can set the SHELL as bash for the run of any single command but again use the shebang approach
Also, always try to use absolute path to any file in crontab as cron runs with a modified PATH.
Now, even in bash, your for construct's syntax is incorrect, you need the arithmetic operator (()), not subshell (()):
for ((i=0; i<retries; i++)); do ...; done
If you source the script by . operator, then shebang #!/bin/bash in the script is ignored. The script is executed in the shell of your cron, which is not bash and it doesn't support for loop you are using.
Add shebang in your script and remove . from cron file:
*/5 * * * * /home/ubuntu/filename.sh >> /filename.sh
or just edit cron file in the following way:
*/5 * * * * /bin/bash /home/ubuntu/filename.sh >> /filename.sh
Don't use . when you are giving full path to the script, try running below command:
*/5 * * * * /home/ubuntu/filename.sh
It should work, Let me know in case of more details.
I have a bash script in Ubuntu, I want it to run every 10 minutes for example after it's done. How can I do this? Thanks!
You can check watch.
From the man pages of watch the description says watch - execute a program periodically, showing output fullscreen, you can try watch -n 600 my_script.sh which will execute myscript.sh every 600 seconds i.e. 10 minutes. watch shows the output to full screen, you can redirect it to say /dev/null in case you are not interested in the output to the screen.
Hope this helps!
Cronjobs is what you need.
My blog post:- http://linux-junky.blogspot.com/2010/10/guide-to-add-cronjob-simplified.html
Or you can also use sleep 600 in your script.
You can use at to reschedule the script from within the script. At the end of the script put:
at now + 10 minutes << END
"$0" "$#"
END
Or with crontab -e, or another option, to check the date. for example, if you want to do something every 10 minutes, you can write:
if [ $((`date +%M`%10)) -eq 0 ] && [ `date +%S` -lt 10 ]; then
#your code
fi