Command below prints pid of subshell and subshell of subshell:
$ ( ( echo $BASHPID )& echo $BASHPID )& sleep 1
[1] 9885
9885
9887
[1]+ Done ( ( echo $BASHPID ) & echo $BASHPID )
Now command below is more complicated, but it indicates that second subshell is in 'process group' of first subshell:
$ ( ( echo $$ $BASH_SUBSHELL $BASHPID ; export BBB=$BASHPID; ps -e -o pid,pgid,ppid,comm | grep -E "$$|$BBB|PGID" | grep -E "bash|PGID" )& echo $$ $BASH_SUBSHELL $BASHPID; sleep 1 )& sleep 1
[3] 9973
2787 1 9973
2787 2 9975
PID PGID PPID COMMAND
2787 2787 2769 bash
9973 9973 2787 bash
9975 9973 9973 bash
Is there simple way to create similar command which will show unique number for last row in second column?
There's no way to do that since you are detaching the subshells from the current process (PGID) with the & operator therefore will run on a separate process (new PGID).
Check this ones also: How to set process group of a shell script
Related
Can somebody explain the following behaviour with a here-string, please?
$ echo "$SHLVL $BASH_SUBSHELL $BASHPID $$"
1 0 18689 18689
$ cat <<< "$SHLVL $BASH_SUBSHELL $BASHPID $$"
1 0 19078 18689
$ cat <<< "$SHLVL $BASH_SUBSHELL $BASHPID $$"
1 0 19079 18689
The BASHPID is different from the shell PID in the here-string, and change each time, but I don't understand why.
The same happens in here-documents:
$ cat << EOT
> $SHLVL $BASH_SUBSHELL $BASHPID $$
> EOT
1 0 19096 18689
Surprisingly, BASHPID doesn't change in a command group:
$ { cat ;} <<< "$SHLVL $BASH_SUBSHELL $BASHPID $$"
1 0 18689 18689
On the other hand, it changes in a subshell
$ (echo $BASHPID ; cat) <<< "$SHLVL $BASH_SUBSHELL $BASHPID $$"
20465
1 1 20465 18689
while the here-string is supposed to expand in the current shell.
Note: my bash version is 4.3+
(Just guessing ...)
The behavior is similar to this:
# echo $$
35130
# echo $( echo $$ $BASHPID )
35130 88025
# echo $( echo $$ $BASHPID )
35130 88026
#
# # or
#
# echo $$ $BASHPID | cat
35130 88028
# echo $$ $BASHPID | cat
35130 88030
Apparently $BASHPID is not expanded at the same time as $$. According to man bash:
BASHPID
Expands to the process ID of the current bash process. This differs from $$ under certain circumstances, such as subshells that do not require bash to be re-initialized.
This implies $BASHPID is not expanded the time Bash parses the command line otherwise it'll be the same as $$. And in Bash source code there's a func initialize_dynamic_variables() (in file variables.c):
1905 static void
1906 initialize_dynamic_variables ()
1907 {
1908 SHELL_VAR *v;
1909
1910 v = init_seconds_var ();
1911
1912 INIT_DYNAMIC_VAR ("BASH_ARGV0", (char *)NULL, get_bash_argv0, assign_bash_argv0);
....
....
1924 INIT_DYNAMIC_VAR ("BASHPID", (char *)NULL, get_bashpid, null_assign);
1925 VSETATTR (v, att_integer);
....
As it shows, vars like BASHPID are called dynamic variables. I guess these vars are handled specially and are expanded in the last minute when it knows that no more sub-shell will be forked (the fork may be followed by exec(), e.g., to run external commands).
#!/usr/bin/env bash
sleep 3 & # Spawn a child
trap '
pgrep -P $$ # Outputs one PID as expected
PIDS=( $( pgrep -P $$ ) ) # Saves an extra nonexistant PID
echo "PIDS: ${PIDS[#]}" # You can see it is the last one
ps -o pid= "${PIDS[#]:(-1)}" ||
echo "Dafuq is ${PIDS[#]:(-1)}?" # Yep, it does not exist!
' 0 1 2 3 15
It outputs
11800
PIDS: 11800 11802
Dafuq is 11802?
It only happens with traps.
Why is a nonexistent PID appended to the array? And how to avoid this odd behaviour?
By using $(...), you've created a subprocess which will execute that code.
Naturally, the parent of that process will be the current shell, so it's going to be listed.
As for the workaround, you could remove that PID from the list. First you have to know how to access the subshell PID: $$ in a script vs $$ in a subshell . Now you can filter it out (nope, it doesn't work):
PIDS=( $( pgrep -P $$ | grep -v ^$BASHPID$ ) )
Here is minimal code for issue demonstration:
http://pastebin.com/5TXDpSh5
#!/bin/bash
set -e
set -o pipefail
function echoTraps() {
echo "= on start:"
trap -p
trap -- 'echo func-EXIT' EXIT
echo "= after set new:"
trap -p
# we can ensure after script done - file '/tmp/tmp.txt' was not created
trap -- 'echo SIG 1>/tmp/tmp.txt' SIGPIPE SIGHUP SIGINT SIGQUIT SIGTERM
}
trap -- 'echo main-EXIT1' EXIT
echo "===== subshell trap"
( echoTraps; )
echo "===== pipe trap"
echoTraps | cat
echo "===== done everything"
output
===== subshell trap
= on start:
= after set new:
trap -- 'echo func-EXIT' EXIT
func-EXIT
===== pipe trap
= on start:
= after set new:
trap -- 'echo func-EXIT' EXIT
===== done everything
main-EXIT1
expected output
===== subshell trap
= on start:
= after set new:
trap -- 'echo func-EXIT' EXIT
func-EXIT
===== pipe trap
= on start:
= after set new:
trap -- 'echo func-EXIT' EXIT
func-EXIT <---- here is the expected difference
===== done everything
main-EXIT1
NB: i tested for OSX 10.9.2 bash (3.2.51) - other versions of bash has same difference between actual an expected output, and described bellow
The only way to find out if this behavior is expected or not is to ask Chet Ramey (GNU bash maintainer). Please send an email with your report to bug-bash#gnu.org
You can see that the current behavior seems to be correct, given that it handles the subshell case explicitly in:http://git.savannah.gnu.org/cgit/bash.git/tree/execute_cmd.c#n621
/* We want to run the exit trap for forced {} subshells, and we
want to note this before execute_in_subshell modifies the
COMMAND struct. Need to keep in mind that execute_in_subshell
runs the exit trap for () subshells itself. */
/* This handles { command; } & */
s = user_subshell == 0 && command->type == cm_group && pipe_in == NO_PIPE && pipe_out == NO_PIPE && asynchronous;
/* run exit trap for : | { ...; } and { ...; } | : */
/* run exit trap for : | ( ...; ) and ( ...; ) | : */
s += user_subshell == 0 && command->type == cm_group && (pipe_in != NO_PIPE || pipe_out != NO_PIPE) && asynchronous == 0;
last_command_exit_value = execute_in_subshell (command, asynchronous, pipe_in, pipe_out, fds_to_close);
if (s)
subshell_exit (last_command_exit_value);
else
sh_exit (last_command_exit_value);
As you can see, the explicit subshell case is handled as a special case (and so is the case with the command grouping). This behavior has evolved historically, as Adrian found out, due to multiple bug reports.
This is the list of changes for this particular feature (triggering EXIT trap on subshells):
Commit: http://git.savannah.gnu.org/cgit/bash.git/commit/?id=a37d979e7b706ce9babf1306c6b370c327038eb9
+execute_cmd.c
+ - execute_command_internal: make sure to run the EXIT trap for group
+ commands anywhere in pipelines, not just at the end. From a point
+ raised by Andreas Schwab <schwab#linux-m68k.org>
Report: https://lists.gnu.org/archive/html/bug-bash/2013-04/msg00126.html (Re: trap EXIT in piped subshell not triggered during wait)
Commit: http://git.savannah.gnu.org/cgit/bash.git/commit/?id=1a81420a36fafc5217e770e042fd39a1353a41f9
+execute_cmd.c
+ - execute_command_internal: make sure any subshell forked to run a
+ group command or user subshell at the end of a pipeline runs any
+ EXIT trap it sets. Fixes debian bash bug 698411
+ http://bugs.debian.org/cgi-big/bugreport.cgi?bug=698411
Report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=698411 (EXIT trap and pipeline and subshell)
Commit: http://git.savannah.gnu.org/cgit/bash.git/commit/?id=fd58d46e0d058aa983eea532bfd7d4c597adef54
+execute_cmd.c
+ - execute_command_internal: make sure to call subshell_exit for
+ {} group commands executed asynchronously (&). Part of fix for
+ EXIT trap bug reported by Maarten Billemont <lhunath#lyndir.com>
Report: http://lists.gnu.org/archive/html/bug-bash/2012-07/msg00084.html (EXIT traps in interactive shells)
There is also a recent bug report in relation to the EXIT trap not executing in some expected contexts: http://lists.gnu.org/archive/html/bug-bash/2016-11/msg00054.html
Here are some more test cases for your amusement:
$ cat traps.sh
#!/bin/bash
echoTraps() {
echo "entering echoTraps()"
printf " traps: %s\n" "$(trap -p)"
echo " setting trap"
trap -- 'echo "func-exit()"' EXIT
printf " traps: %s\n" "$(trap -p)"
echo "exiting echoTraps()"
}
trap -- 'echo "main-exit()"' EXIT
echo "===== calling '( echoTraps; )'"
( echoTraps; )
echo
echo "===== calling 'echoTraps | cat'"
echoTraps | cat
echo
echo "===== calling '( echoTraps; ) | cat'"
( echoTraps; ) | cat
echo
echo "===== calling '{ echoTraps; } | cat'"
{ echoTraps; } | cat
echo
bash-4.2.25(1)
$ ./traps.sh
===== calling '( echoTraps; )'
entering echoTraps()
traps:
setting trap
traps: trap -- 'echo "func-exit()"' EXIT
exiting echoTraps()
func-exit()
===== calling 'echoTraps | cat'
entering echoTraps()
traps: trap -- 'echo "main-exit()"' EXIT
setting trap
traps: trap -- 'echo "func-exit()"' EXIT
exiting echoTraps()
===== calling '( echoTraps; ) | cat'
entering echoTraps()
traps:
setting trap
traps: trap -- 'echo "func-exit()"' EXIT
exiting echoTraps()
func-exit()
===== calling '{ echoTraps; } | cat'
entering echoTraps()
traps: trap -- 'echo "main-exit()"' EXIT
setting trap
traps: trap -- 'echo "func-exit()"' EXIT
exiting echoTraps()
main-exit()
bash-4.3.0(1)
$ bash-static-4.3.2/bin/bash-static traps.sh
===== calling '( echoTraps; )'
entering echoTraps()
traps:
setting trap
traps: trap -- 'echo "func-exit()"' EXIT
exiting echoTraps()
func-exit()
===== calling 'echoTraps | cat'
entering echoTraps()
traps: trap -- 'echo "main-exit()"' EXIT
setting trap
traps: trap -- 'echo "func-exit()"' EXIT
exiting echoTraps()
===== calling '( echoTraps; ) | cat'
entering echoTraps()
traps:
setting trap
traps: trap -- 'echo "func-exit()"' EXIT
exiting echoTraps()
func-exit()
===== calling '{ echoTraps; } | cat'
entering echoTraps()
traps: trap -- 'echo "main-exit()"' EXIT
setting trap
traps: trap -- 'echo "func-exit()"' EXIT
exiting echoTraps()
func-exit()
main-exit()
Bottom line: Don't rely on edge-cases like this. I remember investigating other inconsistencies (not about traps) with regards to subshells and pipes and tried to wrap my head around the bash source code and you don't want to go down that route and try to understand why it behaves like it does in certain situations (the code is really horrible, btw). As you can see, some things seem to have been "fixed" and/but both my examples already behave differently than yours.
I have a bash that runs endless commands as background processes:
#!/bin/bash
function xyz() {
# some awk command
}
endlesscommand "param 1" | xyz & # async
pids=$!
endlesscommand "param 2" | xyz & # async
pids="$pids "$!
endlesscommand "param 3" | xyz # sync so the script doesn't leave
The only way to stop this script is (must be) Ctrl-C or kill and when that happens, I need to kill all the background processes listed in the $pids variable.
How do I do that?
If it was possible to catch the kill signal on the main process and execute a function when that happens (shutdown hook), I would do something like:
for $pid in $pids; do kill $pid; done;
But I can't find how to do this...
Here's a trap that doesn't need you to track pids:
trap 'jobs -p | xargs kill' EXIT
EDIT: #Barmar asked if this works within non-sourced scripts, where job control isn't usually available. It does. Consider this script:
$ cat no-job-control
#! /bin/bash
set -e -o pipefail
# Prove job control is off
if suspend
then
echo suspended
else
echo suspension failed, job control must be off
fi
echo
# Set up the trap
trap 'jobs -p | xargs kill' EXIT
# Make some work
(echo '=> Starting 0'; sleep 5; echo '=> Finishing 0') &
(echo '=> Starting 1'; sleep 5; echo '=> Finishing 1') &
(echo '=> Starting 2'; sleep 5; echo '=> Finishing 2') &
echo "What's in jobs -p?"
echo
jobs -p
echo
echo "Ok, exiting now"
echo
When run we see the pids of the three group leaders, and then see them killed:
$ ./no-job-control
./no-job-control: line 6: suspend: cannot suspend: no job control
suspension failed, job control must be off
=> Starting 0
What's in jobs -p?
=> Starting 1
54098
54099
54100
Ok, exiting now
=> Starting 2
./no-job-control: line 31: 54098 Terminated: 15 ( echo '=> Starting 0'; sleep 5; echo '=> Finishing 0' )
./no-job-control: line 31: 54099 Terminated: 15 ( echo '=> Starting 1'; sleep 5; echo '=> Finishing 1' )
./no-job-control: line 31: 54100 Terminated: 15 ( echo '=> Starting 2'; sleep 5; echo '=> Finishing 2' )
If we instead comment out the trap line and re-run, the three jobs do not die and in fact print out their final messages a few seconds later. Notice the returned prompt interleaved with the final outputs.
$ ./no-job-control
./no-job-control: line 6: suspend: cannot suspend: no job control
suspension failed, job control must be off
=> Starting 0
What's in jobs -p?
54110
54111
54112
=> Starting 1
Ok, exiting now
=> Starting 2
$ => Finishing 0
=> Finishing 2
=> Finishing 1
You can make use of pgrep and a function to kill all processes created under the main process like this. This would not only kill the direct child processes but also those created under it.
#!/bin/bash
function killchildren {
local LIST=() IFS=$'\n' A
read -a LIST -d '' < <(exec pgrep -P "$1")
local A SIGNAL="${2:-SIGTERM}"
for A in "${LIST[#]}"; do
killchildren_ "$A" "$SIGNAL"
done
}
function killchildren_ {
local LIST=()
read -a LIST -d '' < <(exec pgrep -P "$1")
kill -s "$2" "$1"
if [[ ${#LIST[#]} -gt 0 ]]; then
local A
for A in "${LIST[#]}"; do
killchildren_ "$A" "$2"
done
fi
}
trap 'killchildren "$BASHPID"' EXIT
endlesscommand "param 1" &
endlesscommand "param 2" &
endlesscommand "param 3" &
while pgrep -P "$BASHPID" >/dev/null; do
wait
done
As for your original code, it would be better to just use arrays, and you also don't need to use a for loop:
#!/bin/bash
trap 'kill "${pids[#]}"' EXIT
pids=()
endlesscommand "param 1" & # async
pids+=("$!")
endlesscommand "param 2" & # async
pids+=("$!")
endlesscommand "param 3" & # syncing this is not a good idea since if the main process would end along with it if it ends earlier.
pids+=("$!")
while pgrep -P "$BASHPID" >/dev/null; do
wait
done
Original function reference: http://www.linuxquestions.org/questions/blog/konsolebox-210384/bash-functions-to-list-and-kill-or-send-signals-to-process-trees-34624/
kill `ps axl | grep "endlesscommand" | awk '{printf $4" "}'`
This will look for the parent processes that is affecting "endlesscommand"
Is it possible to get $BASHPID value in bash3.x?
Actually, I cannot find any description about it in bash3.x man, but it's available in bash4.x.
You can try: bash -c 'echo $PPID'. This works for me on bash 4.1:
$ echo $$
8792
$ ( echo $BASHPID; echo $(bash -c 'echo $PPID') )
12987
12987