Using bash, I would like to understand the different outputs between :
ps |wc -l
4
and
n=$(ps|wc -l)
echo $n
5
I guess the $(ps|wc -l) instruction is creating an additional subprocess, but I don't really understand why it is addded to the ps count
You said it: $( ) creates a subprocess. Because the command ps without options precisely lists all subprocesses of the current shell, you get one more line. I checked this by replacing wc with tee:
$ ps | tee four
PID TTY TIME CMD
XXXXXXX pts/5 00:00:00 bash
YYYYYYY pts/5 00:00:00 ps
ZZZZZZZ pts/5 00:00:00 tee
$ : $(ps -H | tee five)
$ cat five
PID TTY TIME CMD
XXXXXXX pts/5 00:00:00 bash
YYYYYYY pts/5 00:00:00 bash
ZZZZZZZ pts/5 00:00:00 ps
ΩΩΩΩΩΩΩ pts/5 00:00:00 tee
I also passed -H to ps so that it evidences the process tree.
man bash seems to suggest that if I want to execute a command in a separate bash shell all I have to do is bash -c command:
-c string If the -c option is present, then commands are read from string.
I want to do that because I need to run a few things in different environments:
bash --rcfile ~/.bashrc.a -c mytest.a
bash --rcfile ~/.bashrc.b -c mytest.b
However, that didn't work as expected; one can see that by the number of bash shells running, for example:
$ bash
$ ps
PID TTY TIME CMD
7554 pts/0 00:00:00 bash
7573 pts/0 00:00:00 ps
28616 pts/0 00:00:00 bash
$ exit
exit
$ ps
PID TTY TIME CMD
7582 pts/0 00:00:00 ps
28616 pts/0 00:00:00 bash
$ bash -c ps
PID TTY TIME CMD
7583 pts/0 00:00:00 ps
28616 pts/0 00:00:00 bash
How should the invocation of bash should be modified so that it would start a new shell with the specified rc, execute the given command in that shell (with the env modified according to the rc), and exit back?
It's already working exactly the way you want it to. The lack of an extra process is simply due to bash's tail-call optimization.
Bash recognizes that there's no point in having a shell instance whose only job is to wait for a process and exit. It will instead skip the fork and exec the process directly. This is a huge win for e.g. var=$(ps), where it cuts the number of expensive forks from 2 to 1.
If you give it additional commands to run afterwards, this optimization is no longer valid, and then you'll see the additional process:
$ bash -c 'ps'
PID TTY TIME CMD
4531 pts/10 00:00:00 bash
4540 pts/10 00:00:00 ps
$ bash -c 'ps; exit $?'
PID TTY TIME CMD
4531 pts/10 00:00:00 bash
4549 pts/10 00:00:00 bash
4550 pts/10 00:00:00 ps
bash --rcfile ~/.bashrc.a mytest.a will already run mytest.a in a separate process. -c is for specifying a shell command directly, rather than running a script.
# NO!
bash for x in 1 2 3; do echo "$x"; done
# Yes.
bash -c 'for x in 1 2 3; do echo "$x"; done'
In bash, when I run the following command:
sh -c "command"
is there created a subshell and then the command is executed?
My guess is that the command will run in the current shell, but I'm not sure. This guess come from the fact that I already tested using the following commands:
echo $BASHPID, $BASH_SUBSHELL
and
sh -c "echo $BASHPID, $BASH_SUBSHELL"
and the results are the same. But this could be a little misleading as someone told me because the variables may be substituted before the command is executed. Is this the truth?
I think that with examples it's possible to understand the situation
(in my case sh is a symbolic link to /bin/dash).
I did this tests for sh:
echo $$ ; sh -c 'echo $$ ; sh -c '"'"'echo $$'"'"' '
16102
7569
7570
Three different PID, three different shell. (If there are different shells, there is not a subshell spawn).
In a similar way for BASH
echo $$ $BASHPID, $BASH_SUBSHELL ; bash -c 'echo $$ $BASHPID $BASH_SUBSHELL ; bash -c '"'"'echo $$ $BASHPID $BASH_SUBSHELL '"'"' '
16102 16102, 0
10337 10337 0
10338 10338 0
Three different $BASHPID no different $BASH_SUBSHELL (see note below for differences between $$ and $BASHPID).
If we were in a subshell that do not require to be reinitialized, then $$ and $BASHPID should be different.
In the same way $BASH_SUBSHELL is not incremented, it is always 0.
So 2 clues to say again that no new subshell are spawned, we have only new shells.
From man bash (4.2.45(1)-release) I report some pertinent parts about when a subshell is spawned:
Each command in a pipeline is executed as a separate process
(i.e., in a subshell).
If a command is terminated by the control operator &, the shell executes the
command in the background in a subshell. The shell does not wait for the
command to finish, and the return status is 0.
Commands separated by a ; are executed sequentially; the shell waits for each
command to terminate in turn. The return status is the exit status of the last
command executed.
...
( list ) list is executed in a subshell environment
{ list; } list is simply executed in the current shell environment.
A coprocess is a shell command preceded by the coproc reserved word. A coprocess is executed asynchronously in a subshell...
$ Expands to the process ID of the shell. In a () subshell, it expands to the
process ID of the current shell, not the subshell.
Notes:
BASHPID Expands to the process ID of the current bash process. This differs from $$
under certain circumstances, such as subshells that do not require bash to be
re-initialized.
BASH_SUBSHELL Incremented by one each time a subshell or subshell environment is spawned. The initial value is 0.
For the differences between the use of single quote '' an double quote "" you can see this question. Let we remember only that if you write the commands within double quote"" the variables will be evaluated via parameter expansion from the original shell, if extquote is enabled as it is by default from shopt.(cfr. 4.3.2 The shopt builtin in the Bash Reference Manual)
*extquote*
If set, $'string' and $"string" quoting is performed within ${parameter} expansions enclosed in double quotes. This option is enabled by default.
For further references you may find useful e.g.
man bash.
The section Shell Expansions of the bash manual.
The double quote section or
the Parameter Expansion of Shell Command Language as defined in The Open Group Base Specifications Issue 6 IEEE Std 1003.1, 2004 Edition.
I would say not.
A subshell is typically reserved for any instance where the shell process needs to perform an operation in an isolated environment, but with access to (a copy of) the current state of the shell, including all global and local variables.
Examples:
Pipelines: echo x | read y
Command substitution: line=$( read < file && echo "$REPLY" )
In most cases, this would result in the shell process forking itself.
In recent updates of ksh93, some subshells may not actually fork a new process.
The crux of the matter is that a subshell is always created implicitly.
Running sh -c ... will create a new shell process, which will discard most of the state and start from scratch. That means that normal (local, global) shell variables are gone. Naturally, the new shell will have a copy of all exported variables.
A different interpretation of the question could be Does the -c option fork a new process for running ...? Answer: No. It doesn't. However, certain commands passed into the -c argument may require the shell to spawn a subshell, just as when they'd be part of a script, or typed interactively.
I made a check of it as well and no, I don't think it (-c) summons a subshell. If we refer to sh itself, the yes sh is a shell that gets summoned, but not if it's about a subshell within sh itself. We can verify this by running a command like:
# bash -c 'pstree -p; echo Value of PPID in Callee: $PPID; echo Callee PID: $BASHPID / $$'; echo "Caller PID: $$"
bash(28860)───bash(17091)───pstree(17092)
Value of PPID in Callee: 28860
Callee PID: 17091 / 17091
Caller PID: 28860
As you can see the called shell (17091) and the caller shell (28860) are both connected directly as child-parent. There's nothing in between them. $BASHPID and $$ are even the same, in which case should be different if you're on a subshell. This just tells that there's no subshell summoned when calling commands with -c.
There's only one special behavior to this, and that is when summoning a single external binary e.g.:
# bash -c 'pstree -p'; echo "Caller PID: $$"
bash(28860)───pstree(17124)
Caller PID: 28860
There bash saves itself from forking and decided to just directly exec() the only command. You might think that perhaps bash always does that to the last command if the command refers to an external executable binary, but no it doesn't:
# bash -c 'echo Value of PPID in Callee: $PPID; echo Callee PID: $BASHPID / $$; pstree -p'; echo "Caller PID: $$"
Value of PPID in Callee: 28860
Callee PID: 17128 / 17128
bash(28860)───bash(17128)───pstree(17129)
Caller PID: 28860
Now about
echo $BASHPID, $BASH_SUBSHELL
and
sh -c "echo $BASHPID, $BASH_SUBSHELL"
and the results are the same.
It should be the same if echo $BASHPID, $BASH_SUBSHELL is executed in the same shell since "echo $BASHPID, $BASH_SUBSHELL" is first expanded by the shell that interprets the command before it's passed as an argument to sh. If BASHPID is let's say 28860 and BASH_SUBSHELL 0, then the expanded value of "echo $BASHPID, $BASH_SUBSHELL" is 'echo 28860, 0' in which case the command would actually be sh -c 'echo 28860, 0'. The proper way to this actually is to use a single quote to allow interpretation only within the context of the new called shell: sh -c 'echo $BASHPID, $BASH_SUBSHELL', although I'm not really sure if the command itself would be helpful for testing.
So basically what I'm saying is that the test echo $BASHPID, $BASH_SUBSHELL + sh -c "echo $BASHPID, $BASH_SUBSHELL" doesn't prove anything if -c summons a subshell or not and that the guy who told you that it could mislead since the variables may be substituted is correct.
Nevertheless, my own test showed that Bash really doesn't summon a subshell with it (-c).
Check this:
$ name=foo
$ echo $name
foo
$ sh -c "echo $name"
foo
$ sh -c 'echo $name'
$
You need ' in stead of " in your command. Else $name will get evaluated to foo before execution
And to answer your question, yes, it does create/spawn a new shell.
sh will spawn a subshell. But the subshell id stays the same
21934 pts/0 Ss 0:00 -bash
21963 pts/0 S 0:00 \_ /usr/bin/sudo -u root -H /bin/bash -c export AUDITUSER=ch002854; cd $HOME && exec -a '-bash' /bin/bash
22031 pts/0 S 0:00 \_ -bash
2969 pts/0 S 0:00 \_ sleep 1000
2993 pts/0 S 0:00 \_ sleep 1000
3726 pts/0 R+ 0:00 \_ ps af
With sh -c it will just run the command. This will reinitialize your environment variables and therefore it resets $BASH_SUBSHELL to its default 0.
# sh -c 'echo $BASHPID, $BASH_SUBSHELL'
12671, 0
While with `` or () you can create a subshell
# (echo $BASHPID, $BASH_SUBSHELL)
13214, 1
Regardless bash
Running this simple line will show you a lot:
$ echo $$;sh -c 'echo $$;ps axfw | grep -B2 $$'
14152
12352
14147 ? S 0:00 xterm
14152 pts/4 Ss 0:00 \_ bash
12352 pts/4 S+ 0:00 \_ sh -c echo $$;ps axfw | grep -B2 $$
12353 pts/4 R+ 0:00 \_ ps axfw
12354 pts/4 S+ 0:00 \_ grep -B2 12352
This is clearly a child but what's a subshell?
Well, my current running shell pid is 14152
$ echo $$ $BASHPID $BASH_SUBSHELL
14152 14152 0
$ (echo $$ $BASHPID $BASH_SUBSHELL)
14152 12356 1
$ ( (echo $$ $BASHPID $BASH_SUBSHELL) )
14152 12357 2
Well nice: BASHPID is a dynamic variable which take alway the value of executing pid:
$ echo $BASHPID | sed \$a$BASHPID | sed \$a$BASHPID
12371
12372
12373
Hmmm where is my current working pid?
$ sed "\$a$BASHPID $$" < <(echo "$BASHPID $$"| sed "\$a$BASHPID $$")
12386 14152
12387 14152
14152 14152
Well I've find them!
$ echo $BASHPID $$;(echo $BASHPID $$;ps axfw | grep -B3 $BASHPID)
14152 14152
12469 14152
14152 pts/4 Ss 0:00 \_ bash
12469 pts/4 S+ 0:00 \_ bash
12470 pts/4 R+ 0:00 \_ ps axfw
12471 pts/4 S+ 0:00 \_ grep -B3 12471
Conclusion
When you
$ echo $BASHPID $$ $BASH_SUBSHELL;(echo $BASHPID $$ $BASH_SUBSHELL)
14152 14152 0
12509 14152 1
use parenthesis or pipe (|), you will create subshell which is a forked child of a running shell.
But when you
$ echo $BASHPID $$ $BASH_SUBSHELL;bash -c 'echo $BASHPID $$ $BASH_SUBSHELL'
14152 14152 0
12513 12513 0
You will burn a child, running a shell interpreter, so it's a subshell, but not linked to his parent.
Nota: If the child is not linked to his parent, they use same I/O files descriptors. So if you close your window (pts/4 in my run), you will send a SIGHUP to all process using them.
In this answer I'm thinking of subshell the same way as I think of a subprocess.
First off, don't be confused by variable expansion. If you write a variable in double quotes, it will be expanded by the calling shell, not the sh or its -c command. So
sh -c "echo $$"
gives you the PID of the invoking shell, because it expands $$ before it invokes sh, whereas
sh -c 'echo $$'
gives you the PID of the sh command that has been invoked, because the single quotes tell the invoking shell to pass the string $$ to sh unchanged.
That said, the general answer to your question is:
The sh command is definitely a subshell, invoked by the parent shell when it sees 'sh'
When you note that the argument to -c is treated by it as a shell script, then you see that you'll get a grand-subshell of the invoked sh, whenever a regular shell script would get one.
More on item 2: Some shell builtins create subshells; others do not. The sh man page talks about this; check out the "if" builtin, or constucts that use backquotes ``. Usage of pipes also causes subshells. You can also deliberately force a subshell by enclosing commands in parentheses. However, enclosing commands in braces {} does NOT of itself cause a subshell.
sh is designed to run commands, so in most cases you will have subshells. Notable exceptions are the 'case' builtin, variable assignment, option setting, and the . command.
Please explain the Ruby code below, I can't able to understand the code.
In command_substitution.rb
a = %x!ps -def |grep bash!
puts a
Output
1000 3806 3799 0 10:54 pts/0 00:00:00 -bash
1000 4981 4979 0 12:50 pts/0 00:00:00 sh -c ps -def |grep bash
1000 4984 4981 0 12:50 pts/0 00:00:00 grep bash
I searched to find out what is command substitution using Google. But, I didn't get the exact explanation about it.
Please explain .
The %x notation is a literal for executing a shell script. Here, the character ! is used to indicate the start and the end of the literal. It runs the command ps -def |grep bash in the shell, assigns the result to variable a, and prints it.
Within the shell script, ps gets the running processes, | passes that to the next command, which is grep, which searches for the string bash within the processes shown.
I was trying to write a set of functions that could check to see if a process name was running when I encountered some unexpected output. I've condensed the issue in the following script names isRunning.sh which depends on a system ps command that can take the '-fC' arguments...
#!/bin/bash
progname=isRunning.sh
ps -fC isRunning.sh
pRet=`ps -fC ${progname} | wc -l`
echo pRet $pRet
psOut=`ps -fC ${progname}`
wcOut=`echo "${psOut}" | wc -l`
echo
echo ps output
echo "${psOut}"
echo
echo wcOut $wcOut
The first attempt at piping the ps output to wc gets a return of 3. The second attempt gets the expected return value of 2. Can anyone explain this behavior? I figure it's got to be something stupid I am overlooking.
Thanks,
bbb
edit: my output
UID PID PPID C STIME TTY TIME CMD
root 6717 5940 0 13:10 pts/0 00:00:00 /bin/bash ./isRunning.sh
pRet 3
ps output
UID PID PPID C STIME TTY TIME CMD
root 6717 5940 0 13:10 pts/0 00:00:00 /bin/bash ./isRunning.sh
wcOut 2
I get 2 both attempts. Your ps might be outputting an extra blank line, or somesuch, and then your shell's backtick expansion stripping it. Or maybe you actually had two processes matching the first time you ran it.
If you just want to see if its running, check the exit code from your ps:
if ps -C "${progname}" > /dev/null; then
echo its running
else
echo not running
fi
Even better, you should take a look at pidof and pgrep if you can rely on them being present on whichever systems you're targeting. Or use the LSB functions, if you're on Linux.
edit: Actually, since you're looking for copies of yourself running, you might be picking up the shell doing a fork to implement the pipe.