I have a shell script that executes several sub scripts in the background using & and some scripts by just calling them with ./
This all happens in a loop in the parent script.
At at the end of the loop in the parent script i want to wait until all child processes are finished and get their exit status. If one of the child scripts has failed I want to exit the loop and the parent script.
What i currently do is after each subscript is executed i collect its pid using $!. Then i loop through the pids and call wait $pid || exit 1.
But for some reason i get the error: pid 12345 is not a child of this shell. What could be the reason for this error?
Is there another way to do this?
EDIT:
pids=""
extract () {
./p1.sh &
pids="$pids $!"
./p2.sh &
pids="$pids $!"
}
load () {
./p3.sh
pids="$pids $!"
}
extract
load
echo ${pids[#]}
for p in $pids; do
wait $p
done
The output is:
pid: 13725 end process 1
pid: 13726 end process 2
pid: 13727 end p3
13725 13726 13726
As you can see the pid 13726 is somehow added twice. The error message i get is: wait: pid 13726 is not a child of this shell
Related
Using ksh93 i'm attempting to wait for a background process ,run_cataloguer(), to finish, from within a separate background process ,send_mail(), using the script below:
#!/usr/bin/env ksh
function run_cataloguer
{
echo "In run_cataloguer()"
sleep 2
echo "leaving run_cataloguer()"
}
function send_mail
{
echo "In send_mail()"
#jobs
wait_for_cataloguer
sleep 1
echo "Leaving send_mail() "
}
function wait_for_cataloguer
{
echo "In wait_for_cataloguer() PID_CAT = $PID_CAT"
wait $PID_CAT
waitRet=$?
echo "waitRet = $waitRet"
}
run_cataloguer &
PID_CAT=$!
echo "PID_CAT = $PID_CAT"
send_mail &
wait # Wait for all
echo "Finished main"
The following output is seen:
PID_CAT = 1265
In run_cataloguer()
In send_mail()
In wait_for_cataloguer() PID_CAT = 1265
waitRet = 127 # THIS SHOULD be 0
Leaving send_mail()
leaving run_cataloguer()
Finished main
The problem is
waitRet = 127
which means the wait command can't see $PID_CAT, so it doesn't wait for run_cataloguer() to finish and
"leaving send_mail()"
is printed before
"leaving run_cataloguer()"
If I run send_mail in the foreground then waitRet = 0, which is correct.
So, it appears that you cannot wait for a background process from within a separate background process.
Also, if I uncomment the jobs command, nothing is returned , which appears to confirm the previous statement.
If anyone has a solution ,apart form using flag files, :), it would be much appreciated.
It looks like this cannot be done. The solution I used was from Parvinder here:
wait child process but get error: 'pid is not a child of this shell'
I am running a for loop in which a command is run in background using &. In the end i want all commands to return value..
Here is the code i tried
for((i=0 ;i<3;i++)) {
// curl command which returns a value &
}
wait
// next piece of code
I want to get all three returned value and then proceed.. But the wait command does not wait for background processes to complete and runs the next part of code. I need the returned values to proceed..
Shell builtins have documentation accessible with help BUILTIN_NAME.
help wait yields:
wait: wait [-n] [id ...]
Wait for job completion and return exit status.
Waits for each process identified by an ID, which may be a process ID or a
job specification, and reports its termination status. If ID is not
given, waits for all currently active child processes, and the return
status is zero. If ID is a a job specification, waits for all processes
in that job's pipeline.
If the -n option is supplied, waits for the next job to terminate and
returns its exit status.
Exit Status:
Returns the status of the last ID; fails if ID is invalid or an invalid
option is given.
which implies that to get the return statuses, you need to save the pid and then wait on each pid, using wait $THE_PID.
Example:
sl() { sleep $1; echo $1; return $(($1+42)); }
pids=(); for((i=0;i<3;i++)); do sl $i & pids+=($!); done;
for pid in ${pids[#]}; do wait $pid; echo ret=$?; done
Example output:
0
ret=42
1
ret=43
2
ret=44
Edit:
With curl, don't forget to pass -f (--fail) to make sure the process will fail if the HTTP request did:
CURL Example:
#!/bin/bash
URIs=(
https://pastebin.com/raw/w36QWU3D
https://pastebin.com/raw/NONEXISTENT
https://pastebin.com/raw/M9znaBB2
)
pids=(); for((i=0;i<3;i++)); do
curl -fL "${URIs[$i]}" &>/dev/null &
pids+=($!)
done
for pid in "${pids[#]}"; do
wait $pid
echo ret=$?
done
CURL Example output:
ret=0
ret=22
ret=0
GNU Parallel is a great way to do high-latency things like curl in parallel.
parallel curl --head {} ::: www.google.com www.hp.com www.ibm.com
Or, filtering results:
parallel curl --head -s {} ::: www.google.com www.hp.com www.ibm.com | grep '^HTTP'
HTTP/1.1 302 Found
HTTP/1.1 301 Moved Permanently
HTTP/1.1 301 Moved Permanently
Here is another example:
parallel -k 'echo -n Starting {} ...; sleep 5; echo done.' ::: 1 2 3 4
Starting 1 ...done.
Starting 2 ...done.
Starting 3 ...done.
Starting 4 ...done.
It seems that bash's wait doesn't honor set -e as I would expect. Or it somehow loses track of the child process exiting with an error. Consider the example.
set -e # exit immediately on error
function child()
{
if [ $1 -eq 3 ]; then
echo "child $1 performing error"
# exit 1 ## I also tried this
false
else
echo "child $1 performing successful"
true
fi
echo "child $1 exiting normally"
}
# parent
child 1 & # succeeds
child 2 & # fails
child 3 & # succeeds
wait # why doesn't wait indicate an error?
echo "Launch nukes!" # don't want this to execute if a child failed
I want the set -e semantics, but wait doesn't seem to honor them.
A parent launches three children. One of them chokes and exits with an error (honoring the set -e). The problem is that the parent process plunders on as if nothing bad happened silently. I.e. I want to propagate the error.
Is there a way to enable this behavior? I.e. have wait return non-zero if any child exits non-zero.
If you are using bash 4.3, you can use wait -n in a loop to wait for each child in turn. You won't know which child failed, but whenever one does fail, the exit status of wait will be non-zero.
child 1 & # succeeds
child 2 & # fails
child 3 & # succeeds
for i in 1 2 3; do
wait -n
done
According to the Man page, that's what should happen (emphasis mine):
If n is not given, all currently active child processes are waited
for, and the return status is zero.
You want the second form where you specify wait {pid1} {pid2} {...} and get back the right error code.
I am experimenting with multiple processes. I am trapping SIGCLD to execute something when the child is done. It is working on IRB but not when I execute as a ruby script.
pid = fork {sleep 2; puts 'hello'}
trap('CLD') { puts "pid: #{pid} exited with code"}
When I run the above from IRB, I both lines are printed but when I run it as a ruby script, the line within the trap procedure does not show up.
IRB gives you an outer loop, which means that the ruby process doesn't exit until you decide to kill it. The problem with your ruby script is that the main process is finishing and killing your child (yikes) before it has the chance to trap the signal.
My guess is that this is a test script, and the chances are that your desired program won't have the case where the parent finishes before the child. To see your trap working in a plain ruby script, add a sleep at the end:
pid = fork {sleep 2; puts 'hello'}
trap('CLD') { puts "pid: #{pid} exited with code"}
sleep 3
To populate the $? global variable, you should explicitly wait for the child process to exit:
pid = fork {sleep 2; puts 'hello'}
trap('CLD') { puts "pid: #{pid} exited with code #{$? >> 8}" }
Process.wait
If you do want the child to run after the parent process has died, you want a daemon (double fork).
When you run your code in IRB, the main thread belongs to IRB so that all the stuff you’ve called is living within virtually infinite time loop.
In a case of script execution, the main thread is your own and it dies before trapping. Try this:
pid = fork {sleep 2; puts 'hello'}
trap('CLD') { puts "pid: #{pid} exited with code"}
sleep 5 # this is needed to prevent main thread to die ASAP
Hope it helps.
In shell script I am trying to wait for non-child process. I got reference on how to do it from:
WAIT for "any process" to finish
My shell script structure is:
Main.sh
func1(){
return 1
}
func2(){
# Wait for func1 to finish
while kill -0 "$pid_func1"; do
sleep 0.5
done
}
# Call function 1 in background
func1 &
pid_func1=$!
func2 &
In this case how do I receive the return value of func1 inside function func2?
You generally cannot capture the exit status of non-child processes. You may be able to work something involving logging the exit codes to status files and then reading the values, but otherwise you're not going to be able to capture the values
I used anothe shell variable to store the return status in this case and checked value of this shell variable whereever required. Find a sample shell script below to simulate the scenario.
#!/bin/bash
func1(){
retvalue=23 # return value which needs to be returned
status_func1=$retvalue # store this value in shell variable
echo "func1 executing"
return $retvalue
}
func2(){
# Not possible to use wait command for pid of func1 as it is not a child of func2
#wait $pid_func1
#ret_func1=$?
while kill -0 "$pid_func1"; do
echo "func1 is still executing"
sleep 0.5
done
echo "func2 executing"
#echo "func1 ret: $ret_func1"
echo "func1 ret: $status_func1"
}
# Main shell script starts here
func1 &
pid_func1=$!
func2 &
Hope its useful for others who are facing the same issue.