i'm having a difficult time writing a bash script, hoping someone could help. basically i'm trying to run a number of processes at the same time and then kill them all after an interval.
so for example, if i want to run my_long_running_task 50 times and kill after 10 minutes this is what i came up with:
#!/bin/bash
PIDS=()
(while :
do
my_long_running_task;
sleep 1
done ) &
PIDS+=($!)
...{repeat while loop 50 times or stick it in a for loop)...
sleep 600; # 10 minutes * 60 seconds
for p in "${PIDS[#]}"
do
kill $p
done
i'm not a bash expert but that seems like it should work - fork all the processes adding their pids to an array. then at the end just sleep for a certain amount of time before iterating over the array and killing all the pids. and indeed this worked for my very simple poc:
#!/bin/bash
PIDS=()
(while :
do
echo '1'
sleep 1;
done) &
PIDS+=($!)
(while :
do
echo '2'
sleep 1;
done) &
PIDS+=($!)
(sleep 10; \
for p in "${PIDS[#]}"
do
kill $p
done)
but when i do something more interesting than echo - like, in my case, running phantomjs, the processes don't get killed after the interval.
any thoughts? what am i missing?
Your wish is my command (at least, when your wish aligns sufficiently with my desires):
When you run phantomjs, do you run it with exec or just as a normal process?
Does it make any difference if you do use exec?
The thought behind the questions is that you kill the shell that runs the other process (which, in the case of echo, is the shell), but that doesn't necessarily kill the children of the process. Maybe you need to use something like:
kill -TERM -- -$p
kill -- -$p
to send a signal to the process group, rather than just the process.
Also, consider whether a 'time out' command would make your life easier (timeout on Linux).
Related
I have written a bash script to carry out some tests on my system. The tests run in the background and in parallel. The tests can take a long time and sometimes I may wish to abort the tests part way through.
If I Control+C then it aborts the parent script, but leaves the various children running. I wish to make it so that I can hit Control+C or otherwise to quit and then kill all child processes running in the background. I have a bit of code that does the job if I'm running running the background jobs directly from the terminal, but it doesn't work in my script.
I have a minimal working example.
I have tried using trap in combination with pgrep -P $$.
#!/bin/bash
trap 'kill -n 2 $(pgrep -P $$)' 2
sleep 10 &
wait
I was hoping that on hitting control+c (SIGINT) would kill everything that the script started but it actually says:
./breakTest.sh: line 1: kill: (3220) - No such process
This number changes, but doesn't seem to apply to any running processes, so I don't know where it is coming from.
I guess if the contents of the trap command get evaluated where the trap command occurs then it might explain the outcome. The 3220 pid might be for pgrep itself.
I'd appreciate some insight here
Thanks
I have found a solution using pkill. This example also deals with many child processes.
#!/bin/bash
trap 'pkill -P $$' SIGINT SIGTERM
for i in {1..10}; do
sleep 10 &
done
wait
This appears to kill all the child processes elegantly. Though I don't properly understand what the issue was with my original code, apart from sending the correct signal.
in bash whenever you you use & after a command it places that command as a background job ( this background jobs are called job_spec ) which is incremented by one until you exit that terminal session. You can use the jobs command to get the list of the background jobs running. To work with this jobs you have to use the % with the job id. The jobs command also accept other options such as jobs -p to see the proces sids of all jobs , jobs -p %JOB_SPEC to see the process of id of that particular job.
#!/usr/bin/env bash
trap 'kill -9 %1' 2
sleep 10 &
wait
or
#!/usr/bin/env bash
trap 'kill -9 $(jobs -p %1)' 2
sleep 10 &
wait
I implemented something like this few years back, you can take a look at it async bash
You can try something like the following:
pkill -TERM -P <your_parent_id_here>
How do I repeatedly start and kill a bash script that takes a long time. I have a analyze_realtime.sh that runs indefinitely, but I only want to run it for X second bursts (just say 15s for now).
while true; do analyze_realtime.sh; sleep 15; done
The problem with this is that analyze_realtime.sh never finishes, so this logic doesn't work. Is there a way to kill the process after 15 seconds, then start it again?
I was thinking something with analyze_realtime.sh&, ps, and kill may work. Is there anything simpler?
while true;
do
analyze_realtime.sh &
jobpid=$! # This gets the pid of the bg job
sleep 15
kill $jobpid
if ps -p $jobpid &>/dev/null; then
echo "$jobpid didn't get killed. Moving on..."
fi
done
You can do more under the if-statement, sending other SIGNALs if SIGHUP didn't work.
Try this out
while true; do
analyze_realtime.sh & # put script execution in background
sleep 15
kill %1
done
Explanation
%1 refer to the latest process ran in background
You can use timeout utility from coreutils:
while true; do
timeout 15 analyze_realtime.sh
done
(Inspired by this answer)
Is there any built in feature in bash to wait for 1 out of many processes to finish? And then kill remaining processes?
pids=""
# Run five concurrent processes
for i in {1..5}; do
( longprocess ) &
# store PID of process
pids+=" $!"
done
if [ "one of them finished" ]; then
kill_rest_of_them;
fi
I'm looking for "one of them finished" command. Is there any?
bash 4.3 added a -n flag to the built-in wait command, which causes the script to wait for the next child to complete. The -p option to jobs also means you don't need to store the list of pids, as long as there aren't any background jobs that you don't want to wait on.
# Run five concurrent processes
for i in {1..5}; do
( longprocess ) &
done
wait -n
kill $(jobs -p)
Note that if there is another background job other than the 5 long processes that completes first, wait -n will exit when it completes. That would also mean you would still want to save the list of process ids to kill, rather than killing whatever jobs -p returns.
It's actually fairly easy:
#!/bin/bash
set -o monitor
killAll()
{
# code to kill all child processes
}
# call function to kill all children on SIGCHLD from the first one
trap killAll SIGCHLD
# start your child processes here
# now wait for them to finish
wait
You just have to be really careful in your script to use only bash built-in commands. You can't start any utilities that run as a separate process after you issue the trap command - any child process exiting will send SIGCHLD - and you can't tell where it came from.
I have a simple shell script whose also is below:
#!/usr/bin/sh
echo "starting the process which is a c++ process which does some database action for around 30 minutes"
#this below process should be run in the background
<binary name> <arg1> <arg2>
exit
Now what I want is to monitor and display the status information of the process.
I don't want to go deep into its functionality. Since I know that the process will complete in 30 minutes, I want to show to the user that 3.3% is completed for every 1 min and also check whether the process is running in the background and finally if the process is completed I want to display that it is completed.
could anybody please help me?
The best thing you could do is to put some kind of instrumentation in your application,
and let it report the actual progress in terms of work items processed / total amount of work.
Failing that, you can indeed refer to the time that the thing has been running.
Here's a sample of what I've used in the past. Works in ksh93 and bash.
#! /bin/ksh
set -u
prog_under_test="sleep"
args_for_prog=30
max=30 interval=1 n=0
main() {
($prog_under_test $args_for_prog) & pid=$! t0=$SECONDS
while is_running $pid; do
sleep $interval
(( delta_t = SECONDS-t0 ))
(( percent=100*delta_t/max ))
report_progress $percent
done
echo
}
is_running() { (kill -0 ${1:?is_running: missing process ID}) 2>& -; }
function report_progress { typeset percent=$1
printf "\r%5.1f %% complete (est.) " $(( percent ))
}
main
If your process involves a pipe than http://www.ivarch.com/programs/quickref/pv.shtml would be an excellent solution or an alternative is http://clpbar.sourceforge.net/ . But these are essentially like "cat" with a progress bar and need something to pipe through them. There is a small program that you could compile and then execute as a background process then kill when things finish up, http://www.dreamincode.net/code/snippet3062.htm that would probablly work if you just want to dispaly something for 30 minutes and then print out almost done in the console if your process runs long and it exits, but you would have to modify it. Might be better just to create another shell script that displays a character every few seconds in a loop and checks if the pid of the previous process is still running, I believe you can get the parent pid by looking at the $$ variable then check if it is still running in /proc/pid .
You really should let the command output statistics, but for simplicity's sake you can do something like this to simply increment a counter while your process runs:
#!/bin/sh
cmd & # execute a command
pid=$! # Record the pid of the command
i=0
while sleep 60; do
: $(( i += 1 ))
e=$( echo $i 3.3 \* p | dc ) # compute percent completed
printf "$e percent complete\r" # report completion
done & # reporter is running in the background
pid2=$! # record reporter's pid
# Wait for the original command to finish
if wait $pid; then
echo cmd completed successfully
else
echo cmd failed
fi
kill $pid2 # Kill the status reporter
Let's say I have a bash script that executes three scripts in parallel
./script1 &
./script2 &
./script3 &
Now, let us say that ./script4 depends on script1, script2 and script3. How can I force it to wait for those, while still executing the three scripts in parallel?
You can use wait a built-in command available in Bash and in some other shells.
(see equivalent command WAITFOR on Windows)
wait documentation
Wait for each specified process to complete and return its termination
status.
Syntax
wait [n ...]
Key
n A process ID or a job specification
Each n can be a process ID or a job specification; if a job
specification is given, all processes in that job's pipeline are
waited for.
If n is not given, all currently active child processes are waited
for, and the return status is zero.
If n specifies a non-existent process or job, the return status is
127. Otherwise, the return status is the exit status of the last process or job waited for.
Simple solution
Below wait waits indefinitely for all currently active child processes to be all ended (i.e. in this case the three scripts).
./script1 &
./script2 &
./script3 &
wait # waits for all child processes
./script4
Store the PIDs in shell local variables
./script1 & pid1=$!
./script2 & pid2=$!
./script3 & pid3=$!
wait $pid1 $pid2 $pid3 # waits for 3 PIDs
./script4
Store the PIDs in temporary files
./script1 & echo $! >1.pid
./script2 & echo $! >2.pid
./script3 & echo $! >3.pid
wait $(<1.pid) $(<2.pid) $(<3.pid)
rm 1.pid 2.pid 3.pid # clean up
./script4
This last solution pollutes the current directory with three files (1.pid, 2.pid and 3.pid). One of these file may be corrupted before wait call. Moreover these files could be left in the file-system in case of crash.
From the bash man page:
wait [n ...]
Wait for each specified process and return its termination status.
Each `n` may be a process ID or a job specification.... If `n` is not
given, all currently active child processes are waited for, and the return
status is zero.
The easiest implementation might be for your last script to start the others. That way it's easy for it to store their PIDs and pass them to wait.
I whipped up something quickly years ago, but now I wanted nested parallelism. This is what I came up with:
# Run each supplied argument as a bash command, inheriting calling environment.
# bash_parallel's can be nested, though escaping quotes can be tricky -- define helper function for such cases.
# Example: bash_parallel "sleep 10" "ls -altrc"
function bash_parallel
{
(
i=0
unset BASH_PARALLEL_PIDS # Do not inherit BASH_PARALLEL_PIDS from parent bash_parallel (if any)
for cmd in "$#"
do
($cmd) & # In subshell, so sibling bash_parallel's wont interfere
BASH_PARALLEL_PIDS[$i]=$!
echo "bash_parallel started PID ${BASH_PARALLEL_PIDS[$i]}: $cmd"
i=$(($i + 1))
done
echo "bash_parallel waiting for PIDs: ${BASH_PARALLEL_PIDS[#]}"
wait ${BASH_PARALLEL_PIDS[#]}
) # In subshell, so ctrl-c will kill still-running children.
}
Use:
eisbaw#leno:~$ time (bash_parallel "sleep 10" "sleep 5")
bash_parallel started PID 30183: sleep 10
bash_parallel started PID 30184: sleep 5
bash_parallel waiting for PIDs: 30183 30184
real 0m10.007s
user 0m0.000s
sys 0m0.004s