Can somebody explain the following behaviour with a here-string, please?
$ echo "$SHLVL $BASH_SUBSHELL $BASHPID $$"
1 0 18689 18689
$ cat <<< "$SHLVL $BASH_SUBSHELL $BASHPID $$"
1 0 19078 18689
$ cat <<< "$SHLVL $BASH_SUBSHELL $BASHPID $$"
1 0 19079 18689
The BASHPID is different from the shell PID in the here-string, and change each time, but I don't understand why.
The same happens in here-documents:
$ cat << EOT
> $SHLVL $BASH_SUBSHELL $BASHPID $$
> EOT
1 0 19096 18689
Surprisingly, BASHPID doesn't change in a command group:
$ { cat ;} <<< "$SHLVL $BASH_SUBSHELL $BASHPID $$"
1 0 18689 18689
On the other hand, it changes in a subshell
$ (echo $BASHPID ; cat) <<< "$SHLVL $BASH_SUBSHELL $BASHPID $$"
20465
1 1 20465 18689
while the here-string is supposed to expand in the current shell.
Note: my bash version is 4.3+
(Just guessing ...)
The behavior is similar to this:
# echo $$
35130
# echo $( echo $$ $BASHPID )
35130 88025
# echo $( echo $$ $BASHPID )
35130 88026
#
# # or
#
# echo $$ $BASHPID | cat
35130 88028
# echo $$ $BASHPID | cat
35130 88030
Apparently $BASHPID is not expanded at the same time as $$. According to man bash:
BASHPID
Expands to the process ID of the current bash process. This differs from $$ under certain circumstances, such as subshells that do not require bash to be re-initialized.
This implies $BASHPID is not expanded the time Bash parses the command line otherwise it'll be the same as $$. And in Bash source code there's a func initialize_dynamic_variables() (in file variables.c):
1905 static void
1906 initialize_dynamic_variables ()
1907 {
1908 SHELL_VAR *v;
1909
1910 v = init_seconds_var ();
1911
1912 INIT_DYNAMIC_VAR ("BASH_ARGV0", (char *)NULL, get_bash_argv0, assign_bash_argv0);
....
....
1924 INIT_DYNAMIC_VAR ("BASHPID", (char *)NULL, get_bashpid, null_assign);
1925 VSETATTR (v, att_integer);
....
As it shows, vars like BASHPID are called dynamic variables. I guess these vars are handled specially and are expanded in the last minute when it knows that no more sub-shell will be forked (the fork may be followed by exec(), e.g., to run external commands).
Related
I am not able to understand why zsh is generating the same random value when calling a subshell.
Consider the following code:
$ cat script.sh
#!/bin/zsh
checkFieldConvergence () {
echo $RANDOM
echo $RANDOM
}
echo $(checkFieldConvergence)
echo $(checkFieldConvergence)
echo $(checkFieldConvergence)
checkFieldConvergence
checkFieldConvergence
checkFieldConvergence
$ ./script.sh
4049 24768
4049 24768
4049 24768
4049
24768
20764
3330
17114
1195
whereas the same with bash gives
$ cat script.sh
#!/bin/bash
checkFieldConvergence () {
echo $RANDOM
echo $RANDOM
}
echo $(checkFieldConvergence)
echo $(checkFieldConvergence)
echo $(checkFieldConvergence)
checkFieldConvergence
checkFieldConvergence
checkFieldConvergence
$ ./script.sh
12274 28155
27609 10269
14100 14662
6945
17897
20354
29817
14495
27552
why zsh is generating the same random value when calling a subshell
Bash detects if it's run in the subshell and if it is, bash re-seeds the random generator using the current value of gettimeofday(), while Zsh does not re-seed the generator and just calls rand().
References: https://github.com/bminor/bash/blob/f3a35a2d601a55f337f8ca02a541f8c033682247/variables.c#L1371 , https://github.com/bminor/bash/blob/f3a35a2d601a55f337f8ca02a541f8c033682247/lib/sh/random.c#L87 vs https://github.com/zsh-users/zsh/blob/00d20ed15e18f5af682f0daec140d6b8383c479a/Src/params.c#L4294 .
For a truly random number different between sub-shells, use SRANDOM.
Command below prints pid of subshell and subshell of subshell:
$ ( ( echo $BASHPID )& echo $BASHPID )& sleep 1
[1] 9885
9885
9887
[1]+ Done ( ( echo $BASHPID ) & echo $BASHPID )
Now command below is more complicated, but it indicates that second subshell is in 'process group' of first subshell:
$ ( ( echo $$ $BASH_SUBSHELL $BASHPID ; export BBB=$BASHPID; ps -e -o pid,pgid,ppid,comm | grep -E "$$|$BBB|PGID" | grep -E "bash|PGID" )& echo $$ $BASH_SUBSHELL $BASHPID; sleep 1 )& sleep 1
[3] 9973
2787 1 9973
2787 2 9975
PID PGID PPID COMMAND
2787 2787 2769 bash
9973 9973 2787 bash
9975 9973 9973 bash
Is there simple way to create similar command which will show unique number for last row in second column?
There's no way to do that since you are detaching the subshells from the current process (PGID) with the & operator therefore will run on a separate process (new PGID).
Check this ones also: How to set process group of a shell script
I'm trying to understand why whenever I'm using function 2>&1 | tee -a $LOG tee creates a subshell in function that can't be exited by simple exit 1 (and if I'm not using tee it works fine). Below the example:
#!/bin/bash
LOG=/root/log.log
function first()
{
echo "Function 1 - I WANT to see this."
exit 1
}
function second()
{
echo "Function 2 - I DON'T WANT to see this."
exit 1
}
first 2>&1 | tee -a $LOG
second 2>&1 | tee -a $LOG
Output:
[root#linuxbox ~]# ./1.sh
Function 1 - I WANT to see this.
Function 2 - I DON'T WANT to see this.
So. if I remove | tee -a $LOG part, it's gonna work as expected (script will be exited in the first function).
Can you, please, explain how to overcome this and exit properly in the function while being able to tee output?
If you create a pipeline, the function is run in a subshell, and if you exit from a subshell, only the subshell will be affected, not the parent shell.
printPid(){ echo $BASHPID; }
printPid #some value
printPid #same value
printPid | tee #an implicit subshell -- different value
( printPid ) #an explicit subshell -- also a different value
If, instead of aFunction | tee you do:
aFunction > >(tee)
it'll be essential the same, except aFunction won't run in a subshell, and thus will be able to affect the current environment (set variables, call exit, etc.).
Use PIPESTATUS to retrieve the exit status of the first command in the pipeline.
first 2>&1 | tee -a $LOG; test ${PIPESTATUS[0]} -eq 0 || exit ${PIPESTATUS[0]}
second 2>&1 | tee -a $LOG; test ${PIPESTATUS[0]} -eq 0 || exit ${PIPESTATUS[0]}
You can tell bash to fail if anything in the pipeline fails with set -e -o pipefail:
$ cat test.sh
#!/bin/bash
LOG=~/log.log
set -e -o pipefail
function first()
{
echo "Function 1 - I WANT to see this."
exit 1
}
function second()
{
echo "Function 2 - I DON'T WANT to see this."
exit 1
}
first 2>&1 | tee -a $LOG
second 2>&1 | tee -a $LOG
$ ./test.sh
Function 1 - I WANT to see this.
#!/usr/bin/env bash
sleep 3 & # Spawn a child
trap '
pgrep -P $$ # Outputs one PID as expected
PIDS=( $( pgrep -P $$ ) ) # Saves an extra nonexistant PID
echo "PIDS: ${PIDS[#]}" # You can see it is the last one
ps -o pid= "${PIDS[#]:(-1)}" ||
echo "Dafuq is ${PIDS[#]:(-1)}?" # Yep, it does not exist!
' 0 1 2 3 15
It outputs
11800
PIDS: 11800 11802
Dafuq is 11802?
It only happens with traps.
Why is a nonexistent PID appended to the array? And how to avoid this odd behaviour?
By using $(...), you've created a subprocess which will execute that code.
Naturally, the parent of that process will be the current shell, so it's going to be listed.
As for the workaround, you could remove that PID from the list. First you have to know how to access the subshell PID: $$ in a script vs $$ in a subshell . Now you can filter it out (nope, it doesn't work):
PIDS=( $( pgrep -P $$ | grep -v ^$BASHPID$ ) )
The program:
function import {
set -x
# read NAME < <(/usr/bin/pwd)
NAME=$(/usr/bin/pwd)
echo 123 >&2
set +x
}
echo aaaaaaaaaaa
import
echo bbbbbbbbbbb
OUT=$( import 2>&1 )
echo "$OUT"
echo ccccccccccc
I hoped to have the output between 'aaa' and 'bbb' to be the same as in between 'bbb' and 'ccc'. But it is not the case with ksh:
aaaaaaaaaaa
+ /usr/bin/pwd
+ NAME=/home/neuron
+ echo 123
+ 1>& 2
123
bbbbbbbbbbb
+ /usr/bin/pwd
ccccccccccc
If I change $( ... ) into < <(...), stderr works as usual and I have the same output. I tried that on solaris and linux and it behaves the same, so I guess it's not ksh bug. Please note that it's not just 'set -x' being disabled, also the 'echo 123 1>&2' output disapears. In bash the code works as I would suppose.
My questions are 'why' and 'how to capture the function's stdout and stderr?'
Thank you
Vlad
It's confirmed to be a bug. And we can reproduce it very simply:
Something appears broken with nested command substitution:
for command in true /bin/true; do
a=$( ( b=$( $command ); echo 123 >& 3; ) 3>& 1 ) &&
echo a=$a command=$command
done
a=123 command=true
a= command=/bin/true
We run the same assignment twice, once with a builtin, and once with
an external command. We would expect the same results, but this fails
when we include the external command.
Glenn Fowler:
I believe this was fixed between 2012-08-23 and 2012-10-12
Tested with the latest beta:
$ ${ksh} test.sh
Version AIJM 93v- 2014-01-14
a=123 command=true
a=123 command=/bin/true
$
$ ksh test.sh
Version JM 93u 2011-02-08
a=123 command=true
a= command=/bin/true
$
This looks like a bug in ksh (My version is u), and it is not specific to pwd or stderr.
Below I can reproduce the effect with true and fd 3. (This allows us to keep use the shell trace)
The effect appears to be triggered by assigning output from an external process
into a variable inside a function. If the assignment is then followed by output
to some other file descriptor, that output gets lost.
The idea here is that the $( ... ) construct somehow conflicts with the subsequent redirect,
but only when the code in ... is run externally. The true and pwd builtins don't trigger a subshell in ksh93.
I will ask David Korn for confirmation.
function f1 {
var=$( /bin/true )
echo 123 >& 3
}
function f2 {
var=$( true )
echo 123 >& 3
}
function f3 {
typeset var
var=$( /bin/true )
echo 123 >& 3
}
functions="f1 f2 f3"
typeset -tf ${functions}
exec 3>& 1
echo ${.sh.version}
for f in ${functions}; do
echo TEST $f
functions $f
echo "123 expected: "
$f
OUT=$( $f 3>& 1 )
echo "OUT='123' expected"
echo "OUT='$OUT'" Captured output
echo
done
output:
Version JM 93u 2011-02-08
TEST f1
function f1 {
var=$( /bin/true )
echo 123 >& 3
}
123 expected:
123
OUT='123' expected
OUT='' Captured output
TEST f2
function f2 {
var=$( true )
echo 123 >& 3
}
123 expected:
123
OUT='123' expected
OUT='123' Captured output
TEST f3
function f3 {
typeset var
var=$( /bin/true )
echo 123 >& 3
}
123 expected:
123
OUT='123' expected
OUT='' Captured output