Sending kill -s USR1 from bash script stops the process while doing the same from terminal does not - bash

There's a nodejs script called mimosa (https://github.com/dbashford/mimosa)
Nodejs uses USR1 to switch the running process to debug mode
Here's how I do it manually
$ cd myproj
$ mimosa watch -s # this runs node /path/to/mimosa watch -s
22:16:03 - Watching /Users/admin/Work/test-mimosa/assets
... # some more output
# check the pid from a different terminal
$ ps aux | grep mimosa
admin 79284 0.7 0.8 3153812 129272 s006 S+ 10:16PM 0:03.57 node /opt/local/bin/mimosa watch -s
# send debug signal from the 2nd terminal
kill -s USR1 79284
# nodejs output in the 1st terminal
Hit SIGUSR1 - starting debugger agent.
debugger listening on port 5858
The same works if I run mimosa as a background process (mimosa watch -s &)
Now I need to automate the process: run mimosa, get its pid, send USR1, wait for user's SIGTERM, kill mimosa:
mimosa watch -s &
pid=$!
echo "mimosa pid: $pid"
trap "echo '\nSTOP'; kill $pid; exit" SIGHUP SIGINT SIGTERM
echo 'send debug'
kill -s USR1 $pid
wait $pid
This script exits immediately, so does the mimosa process (I check it with grep again).
The output in the console
$ ./debug.sh
mimosa pid: 79516
send debug
./debug.sh: line 11: 79516 User defined signal 1: 30 mimosa watch -s
What's wrong, how to fix?

Could mimosa be sending a signal to its own process group when you send the debug signal? That would explain it.
In interactive shells, doing ./program starts program with its own process group. If program does something like kill -s USR1 0, it'll never exit that group.
In non-interactive shells / scripts, doing ./program will start it as a child but in the same process group. If the child does kill -s USR1 0, it'll kill the calling script.
You could do trap 'echo ignoring' USR1 USR2 in your debug.sh in case those are the signals being sent by mimosa.
Alternatively, try turning on job control with set -m before starting mimosa.
See also I have "trap 'echo ignore' USR1" in my called script, why does the calling script get killed?

Related

cannot trap SIGINT signal for bash shell

I have a script that calls
#!/usr/bin/env bash
my_trap(){
echo "signal was trapped";
}
trap my_trap INT
trap my_trap SIGINT
echo "here is the pid: $$"
echo "here is the parent id just in case: $(ps -o ppid= -p $$)"
tsc --watch # open a server / watch process
I call that script in terminal window 1. In another terminal window (terminal 2) I call
kill -INT <pid>
I call that for both the pid and parent pid echoed in the other terminal
nothing gets trapped - nothing gets logged ("signal was trapped" does not get logged)..
anyone know why SIGINT cannot be caught? Note that if I use ctrl-c in terminal 1, it does get caught, so something about sending SIGINT from another terminal window is not working.

How to use trap reliably using Bash running foreground child processes

I have a Bash script that runs a long running process in the foreground. When it receives a SIGQUIT signal, it should perform various cleanup operations such as killing itself and all of its child processes (via kill of process group etc.). A minimal script, that should catch the signal, is shown below (called test_trap.sh):
#!/bin/bash
trap 'echo "TRAP CAUGHT"; exit 1' QUIT # other required signals are omitted for brevity
echo starting sleep
sleep 11666
echo ending sleep
echo done
I would like to send the SIGHUP signal to the process of the test_trap.sh script. However, sending a SIGHUP to the test_trap.sh does not trigger the trap expression, but only when I send the signal to the child sleep 11666 process does the trap fire. Below is a bash session demonstrating this:
bash-4.1$ test_trap.sh &
[1] 19633
bash-4.1$ starting sleep
bash-4.1$ kill -s SIGQUIT 19633
bash-4.1$ jobs
[1]+ Running test_trap.sh &
bash-4.1$ ps -ef --forest --cols=10000 | grep '11666\|test_trap.sh' | grep -v grep
theuser 19633 12227 0 07:40 pts/4 00:00:00 \_ /bin/bash ./test_trap.sh
theuser 19634 19633 0 07:40 pts/4 00:00:00 | \_ sleep 11666
bash-4.1$ kill -s SIGQUIT 19634
bash-4.1$ Quit (core dumped)
TRAP CAUGHT
[1]+ Exit 1 test_trap.sh
bash-4.1$ ps -ef --forest --cols=10000 | grep '11666\|test_trap.sh' | grep -v grep
bash-4.1$
Note that the "sleep 11666" is just a representative process. That process can actually be an interactive subshell (e.g., bash -i).
Why doesn't the parent test_trap.sh process catch the SIGHUP signal? Why would the trap fire only when the process for sleep 11666 was signaled?
I do not want to use uncatchable SIGKILL as I do need to do an assortment of cleanup operations in the trap expression.
This script is intended run on any fairly recent version of any Linux distribution containing Bash (e.g., not Cygwin).
References:
killing Parent process along with child process using SIGKILL
Kill bash and child process
bash must wait for sleep to complete before it can execute the handler. A good workaround is to run sleep in the background, then immediately wait for it. While sleep is uninterruptible, wait is not.
trap 'kill $sleep_pid; echo "TRAP CAUGHT"; exit 1' QUIT
echo starting sleep
sleep 11666 &
sleep_pid=$!
wait
echo ending sleep
echo done
The recording of sleep_pid and using it to kill sleep from the handler are optional.
Actually, bash is receiving the signal, but it is in an uninterruptible state waiting for the sleep command to end. When it ends, bash will react to the signal and execute the trap.
You can replace the long sleep command with a loop of short sleep commands:
while true
do
sleep 1
done
With that, if you send the signal to the bash process, it will react as soon as the currently executing sleep command ends, that is, at most 1 second after it was sent.
Try with the signal SIGINT (the same which is sent by pressing Ctrl+C) instead of SIGKILL. Other signals only work when the bash can process I/O or some other condition.

Why does my bash script take so long to respond to kill when it runs in the background?

(Question revised, now that I understand more about what's actually happening):
I have a script that runs in the background, periodically doing some work and then sleeping for 30 seconds:
echo "background script PID: $$"
trap 'echo "Exiting..."' INT EXIT
while true; do
# check for stuff to do, do it
sleep 30
done &
If I try to kill this script via kill or kill INT, it takes 30 seconds to respond to the signal.
I will answer this question below, since I found a good explanation online.
(My original, embarrassingly un-researched question)
This question is for a bash script that includes the following trap:
trap 'echo "Exiting...">&2; kill $childPID 2>/dev/null; exit 0' \
SIGALRM SIGHUP SIGINT SIGKILL SIGPIPE SIGPROF SIGTERM \
SIGUSR1 SIGUSR2 SIGVTALRM SIGSTKFLT
If I run the script in the foreground, and hit
CTRL-C, it gets the signal immediately and exits
(under one sec).
If I run the same script in the background (&), and kill it via
kill or kill -INT, it takes 30 seconds before getting the signal.
Why is that, and how can I fix it?
As explained in http://mywiki.wooledge.org/SignalTrap --
"When bash is executing an external command in the foreground, it does not handle any signals received until the foreground process terminates" - and since sleep is an external command, bash does not even see the signal until sleep finishes.
That page has a very good overview of signal processing in bash, and work-arounds to this issue. Briefly, one correct way of handling the situation is to send the signal to the process group instead of just the parent process:
kill -INT -123 # will kill the process group with the ID 123
Head over to the referenced page for a full explanation (no sense in my reproducing any more of it here).
Possible reason: signals issued while a process is sleeping are not delivered until wake-up of the process. When started via the command line, the process doesn't sleep, so the signal gets delivered immediately.
#RashaMatt, I was unable to get the read command to work as advertised on Greg's wiki. Sending a signal to the script simply did not interrupt the read. I needed to do this:
#!/bin/bash
bail() {
echo "exiting"
kill $readpid
rm -rf $TMPDIR
exit 0
}
sig2() {
echo "doing stuff"
}
echo Shell $$ started.
trap sig2 SIGUSR2
trap bail SIGUSR1 SIGHUP SIGINT SIGQUIT SIGTERM
trap -p
TMPDIR=$(mktemp -p /tmp -d .daemonXXXXXXX)
chmod 700 $TMPDIR
mkfifo $TMPDIR/fifo
chmod 400 $TMPDIR/fifo
while : ; do
read < $TMPDIR/fifo & readpid=$!
wait $readpid
done
...send the desired signal to the shell's pid displayed from the Shell $$ started line, and watch the excitement.
waiting on a sleep is simpler, true, but some os' don't have sleep infinity, and I wanted to see how Greg's read example would work (which it didn't).

Set trap in bash for different process with PID known

I need to set a trap for a bash process I'm starting in the background. The background process may run very long and has its PID saved in a specific file.
Now I need to set a trap for that process, so if it terminates, the PID file will be deleted.
Is there a way I can do that?
EDIT #1
It looks like I was not precise enough with my description of the problem. I have full control over all the code, but the long running background process I have is this:
cat /dev/random >> myfile&
When I now add the trap at the beginning of the script this statement is in, $$ will be the PID of that bigger script not of this small background process I am starting here.
So how can I set traps for that background process specifically?
(./jobsworthy& echo $! > $pidfile; wait; rm -f $pidfile)&
disown
Add this to the beginning of your Bash script.
#!/bin/bash
trap 'rm "$pidfile"; exit' EXIT SIGQUIT SIGINT SIGSTOP SIGTERM ERR
pidfile=$(tempfile -p foo -s $$)
echo $$ > "$pidfile"
# from here, do your long running process
You can run your long running background process in an explicit subshell, as already shown by Petesh's answer, and set a trap inside this specific subshell to handle the exiting of your long running background process. The parent shell remains unaffected by this subshell trap.
(
trap '
trap - EXIT ERR
kill -0 ${!} 1>/dev/null 2>&1 && kill ${!}
rm -f pidfile.pid
exit
' EXIT QUIT INT STOP TERM ERR
# simulate background process
sleep 15 &
echo ${!} > pidfile.pid
wait
) &
disown
# remove background process by hand
# kill -TERM ${!}
You do not need trap to just run some command after a background process terminates, you can instead run through a shell command line and add the command following after the background process, separated with semicolon (and let this shell run in the background instead of the background process).
If you still would like to have some notification in your shell script send and trap SIGUSR2 for instance:
#!/bin/sh
BACKGROUND_PROCESS=xterm # for my testing, replace with what you have
sh -c "$BACKGROUND_PROCESS; rm -f the_pid_file; kill -USR2 $$" &
trap "echo $BACKGROUND_PROCESS ended" USR2
while sleep 1
do
echo -n .
done

bash trap will echo from keyboard Ctrl-C while not kill 2

Say I have a script:
#!/bin/bash
# test_trap.sh
trap "echo SIGINT captured!" SIGINT
echo $$
sleep 1000
I know trap COMMAND will only be executed after sleep 1000 finishes when it receives SIGINT signal. But the command of trap will be executed when I pressed keyboard Ctrl-C:
> sh test_sh.sh
50138
^CSIGINT captured!
And using kill -s SIGINT will not.
What am I missing here?
The bash version is GNU bash, 4.2.46(2)-release
With kill -s SIGINT 50138, you are only sending the signal to the shell's process, and that has to wait for sleep 1000 to finish, because sleep doesn't receive the signal.
Control-C, though, causes the terminal to send SIGINT to every process in the current process group, so both your shell script and sleep receive it. Your script still doesn't process the trap command until sleep completes, but sleep exits immediately in response to the SIGINT it just received from the terminal.
If your kill supports it, you can also use kill -s SIGINT -50138 (note the negative process id) to send SIGINT to the entire process group.

Resources