I have a Bash script that runs a long running process in the foreground. When it receives a SIGQUIT signal, it should perform various cleanup operations such as killing itself and all of its child processes (via kill of process group etc.). A minimal script, that should catch the signal, is shown below (called test_trap.sh):
#!/bin/bash
trap 'echo "TRAP CAUGHT"; exit 1' QUIT # other required signals are omitted for brevity
echo starting sleep
sleep 11666
echo ending sleep
echo done
I would like to send the SIGHUP signal to the process of the test_trap.sh script. However, sending a SIGHUP to the test_trap.sh does not trigger the trap expression, but only when I send the signal to the child sleep 11666 process does the trap fire. Below is a bash session demonstrating this:
bash-4.1$ test_trap.sh &
[1] 19633
bash-4.1$ starting sleep
bash-4.1$ kill -s SIGQUIT 19633
bash-4.1$ jobs
[1]+ Running test_trap.sh &
bash-4.1$ ps -ef --forest --cols=10000 | grep '11666\|test_trap.sh' | grep -v grep
theuser 19633 12227 0 07:40 pts/4 00:00:00 \_ /bin/bash ./test_trap.sh
theuser 19634 19633 0 07:40 pts/4 00:00:00 | \_ sleep 11666
bash-4.1$ kill -s SIGQUIT 19634
bash-4.1$ Quit (core dumped)
TRAP CAUGHT
[1]+ Exit 1 test_trap.sh
bash-4.1$ ps -ef --forest --cols=10000 | grep '11666\|test_trap.sh' | grep -v grep
bash-4.1$
Note that the "sleep 11666" is just a representative process. That process can actually be an interactive subshell (e.g., bash -i).
Why doesn't the parent test_trap.sh process catch the SIGHUP signal? Why would the trap fire only when the process for sleep 11666 was signaled?
I do not want to use uncatchable SIGKILL as I do need to do an assortment of cleanup operations in the trap expression.
This script is intended run on any fairly recent version of any Linux distribution containing Bash (e.g., not Cygwin).
References:
killing Parent process along with child process using SIGKILL
Kill bash and child process
bash must wait for sleep to complete before it can execute the handler. A good workaround is to run sleep in the background, then immediately wait for it. While sleep is uninterruptible, wait is not.
trap 'kill $sleep_pid; echo "TRAP CAUGHT"; exit 1' QUIT
echo starting sleep
sleep 11666 &
sleep_pid=$!
wait
echo ending sleep
echo done
The recording of sleep_pid and using it to kill sleep from the handler are optional.
Actually, bash is receiving the signal, but it is in an uninterruptible state waiting for the sleep command to end. When it ends, bash will react to the signal and execute the trap.
You can replace the long sleep command with a loop of short sleep commands:
while true
do
sleep 1
done
With that, if you send the signal to the bash process, it will react as soon as the currently executing sleep command ends, that is, at most 1 second after it was sent.
Try with the signal SIGINT (the same which is sent by pressing Ctrl+C) instead of SIGKILL. Other signals only work when the bash can process I/O or some other condition.
Related
I have a bash script that among other things, launches a background process. I use a function that setups some configuration for the process, launches it, checks it started correctly, and returns its PID, which is used later to kill the subprocess. The sample code below has the same structure but simplified logic:
function launcher(){
sleep 30 &
echo $!
PID=$(launcher)
echo $PID
kill $PID
The issue I'm facing is that the subshell that executes the launcher function does not return until the sleep command ends. Therefore the echo $PID statement is not executed until the subshell ends.
what surprises me is that if I check the sleep command, it does not have the script as parent id:
UID PID PPID C STIME TTY STAT TIME CMD
user 20135 1 0 18:39 pts/8 S+ 0:00 sleep 30
How can I start the sleep & in the background to allow the subshell to end before it ends?
Note: Please notice in my case, the background process will never end until I kill it, so I need the subshell to end get the PID. Also notice in my real code, the logic of the launcher function is quite complex and I'm running it as a subshell to isolate the main process from it.
Thanks in advance
It happens that the problem was about stdin because the main shell was reading from the subshell's stdout, which is inherited by the background process. Just redirecting the stdout when invoking the background process makes it work as expected.
sleep 100 > /dev/null &
I'm not sure if this gets it done but
function launcher(){
echo "start launching"
sleep 100 &
echo "end launching"
}
launcher
PID=$!
# Here $PID is the process id of `sleep`
echo $PID
kill $PID
Without the kill, this runs the sleep command forked and the shell script ends leaving the sleep command running with the pid set in PID which means you could kill it later or not.
Is this what you need? If not, can you clarify what you're expecting?
I also noticed that if the parent script stays alive, the PPID of the sleep process is correct and stays in tact.
# sleeper_test.sh
#!/bin/bash
function launcher(){
echo "start launching"
sleep 100 &
echo "end launching"
}
launcher
PID=$!
# Here $PID is the process id of `sleep`
echo $PID
sleep 10
#kill $PID
$ ps -ef | grep sleep
501 13748 5471 0 1:54PM ttys000 0:00.00 /bin/bash ./sleeper_test.sh
501 13749 13748 0 1:54PM ttys000 0:00.00 sleep 100 <- child correctly tied to the parent sh script
501 13750 13748 0 1:54PM ttys000 0:00.00 sleep 10
$ ps -ef | grep sleep
501 13749 1 0 1:54PM ttys000 0:00.00 sleep 100 <- since the parent ended - it's parent becomes the root process parent
``
What is the shortest way to sleep a bash script at a certain location until another script wakes it up to continue it's job?
Mayby using flock -u .. or blocking read on a pipe ?
Say scriptA sleeps and waits for being waken up by scriptB.
One way is, in A, before you sleep, write the pid to some file say scriptA.pid then falling in sleep.
When B is running, at the right moment, you can read the scriptA.pid file, to get the pid of A, then do pkill -P pidofA sleep thus, the sleep sub-process will be killed, and A will continue its execution.
I'm a fan of named pipes (fifo). scriptA.sh:
pipe='/tmp/mypipe'
mkfifo "$pipe"
echo "$0 going to sleep..."
# Should block
read < "$pipe"
echo "$0 continuing"
scriptB.sh
pipe='/tmp/mypipe'
mkfifo "$pipe"
echo "$0 waking other process"
# might block
echo > "$pipe"
echo "$0 exiting"
You will get a mkfifo: /tmp/mypipe: File exists from the second mkfifo, if that bothers you then test for existence first (-e "$pipe"). This does not tidy-up (rm) the fifo, not sure where that should go because timing of the application is critical to where you put that.
You could use the inter process signals: the kill command should be used to send a signal to a process using its pid.
The SIGSTOP signal stops the execution of the process.
The SIGCONT signal resumes the process execution.
The example script below:
stores the pid of the process in a file.
the script sends to its own process the SIGSTOP signal ($$ is the pid of the current bash process).
Hopefully, another process will resume the execution.
Give a try to this:
#!/bin/bash --
printf "%s" $$ > /tmp/aScript.pid
kill -STOP $$ # STOP the execution here
# execution continues here when the SIGCONT signal is received
printf "script %s: received the SIGCONT signal\n" $$
Test in a terminal:
$ ./aScript.sh &
[1] 26444
$ kill -CONT $(cat /tmp/aScript.pid)
script 26444: received the SIGCONT signal
1st method
The running script can stop itself -
$: cat flagfile
#!/usr/bin/bash
echo $$ > /tmp/flagfile.pid
kill -STOP $$
date
$: ./flagfile &
[1] 24679
$: ps -fu $LOGNAME | grep 'flagfile$'
P2759474 24679 24521 0 13:29 pts/0 00:00:00 /usr/bin/bash ./flagfile
[1]+ Stopped ./flagfile
Then any other script can restart it.
$: kill -CONT $(</tmp/flagfile.pid)
$: Wed Dec 12 13:36:01 CST 2018
That last line gave me back a prompt before the background process managed to output the date. :)
2nd method
If a delay is ok, you can have a trap break it out.
This isn't totally stopping the script, but you can set the delay and make it as freindly as you have leeway to wait for it to wake up.
$: cat flagfile
#!/usr/bin/bash
trap 'loop=0' USR1
loop=1
delay=2
echo $$ > /tmp/flagfile.pid
while (( loop )); do sleep $delay; done
date
$: ./flagfile &
[1] 25018
$: ps -fu $LOGNAME | grep 'flagfile$'
P2759474 25018 24521 0 13:42 pts/0 00:00:00 /usr/bin/bash ./flagfile
Wait as long as you like....
$: kill -USR1 $(</tmp/flagfile.pid)
$: Wed Dec 12 13:42:43 CST 2018
[1]+ Done ./flagfile
I've got 2 shell scripts:
# subshell.sh
trap "echo Caught SIGTERM" 15
echo $$
sleep 100000
# parent.sh
setsid sh subshell.sh &
pid=$!
echo "sid=$pid"
sleep 2
# This won't work!
kill -15 -$pid
The main purpose is to send SIGTERM to subshell and all its children. After googling for a while (there is a tricky problem of how bash handles signal), I choose setsid to create a new session and sending the signal used -pid. However, the message won't be printed although pid is correct. If I manually execuate kill -15 -$pid, this can work. So how can I send a signal to the subshell?
Well finally I managed to make this work by creating another subshell..., and then call kill -15 -$pid inside that subshell. Still don't know why parent shell can't do this
There's a nodejs script called mimosa (https://github.com/dbashford/mimosa)
Nodejs uses USR1 to switch the running process to debug mode
Here's how I do it manually
$ cd myproj
$ mimosa watch -s # this runs node /path/to/mimosa watch -s
22:16:03 - Watching /Users/admin/Work/test-mimosa/assets
... # some more output
# check the pid from a different terminal
$ ps aux | grep mimosa
admin 79284 0.7 0.8 3153812 129272 s006 S+ 10:16PM 0:03.57 node /opt/local/bin/mimosa watch -s
# send debug signal from the 2nd terminal
kill -s USR1 79284
# nodejs output in the 1st terminal
Hit SIGUSR1 - starting debugger agent.
debugger listening on port 5858
The same works if I run mimosa as a background process (mimosa watch -s &)
Now I need to automate the process: run mimosa, get its pid, send USR1, wait for user's SIGTERM, kill mimosa:
mimosa watch -s &
pid=$!
echo "mimosa pid: $pid"
trap "echo '\nSTOP'; kill $pid; exit" SIGHUP SIGINT SIGTERM
echo 'send debug'
kill -s USR1 $pid
wait $pid
This script exits immediately, so does the mimosa process (I check it with grep again).
The output in the console
$ ./debug.sh
mimosa pid: 79516
send debug
./debug.sh: line 11: 79516 User defined signal 1: 30 mimosa watch -s
What's wrong, how to fix?
Could mimosa be sending a signal to its own process group when you send the debug signal? That would explain it.
In interactive shells, doing ./program starts program with its own process group. If program does something like kill -s USR1 0, it'll never exit that group.
In non-interactive shells / scripts, doing ./program will start it as a child but in the same process group. If the child does kill -s USR1 0, it'll kill the calling script.
You could do trap 'echo ignoring' USR1 USR2 in your debug.sh in case those are the signals being sent by mimosa.
Alternatively, try turning on job control with set -m before starting mimosa.
See also I have "trap 'echo ignore' USR1" in my called script, why does the calling script get killed?
Say I have a script:
#!/bin/bash
# test_trap.sh
trap "echo SIGINT captured!" SIGINT
echo $$
sleep 1000
I know trap COMMAND will only be executed after sleep 1000 finishes when it receives SIGINT signal. But the command of trap will be executed when I pressed keyboard Ctrl-C:
> sh test_sh.sh
50138
^CSIGINT captured!
And using kill -s SIGINT will not.
What am I missing here?
The bash version is GNU bash, 4.2.46(2)-release
With kill -s SIGINT 50138, you are only sending the signal to the shell's process, and that has to wait for sleep 1000 to finish, because sleep doesn't receive the signal.
Control-C, though, causes the terminal to send SIGINT to every process in the current process group, so both your shell script and sleep receive it. Your script still doesn't process the trap command until sleep completes, but sleep exits immediately in response to the SIGINT it just received from the terminal.
If your kill supports it, you can also use kill -s SIGINT -50138 (note the negative process id) to send SIGINT to the entire process group.