When using Upstart, controlling subprocesses (child process) is quite important. But what confused me is as following, which has gone beyond upstart itself:
scenario 1:
root#ubuntu-jstorm:~/Desktop# su cr -c 'sleep 20 > /tmp/a.out'
I got 3 processes by: cr#ubuntu-jstorm:~$ ps -ef | grep -v grep | grep sleep
root 8026 6544 0 11:11 pts/2 00:00:00 su cr -c sleep 20 > /tmp/a.out
cr 8027 8026 0 11:11 ? 00:00:00 bash -c sleep 20 > /tmp/a.out
cr 8028 8027 0 11:11 ? 00:00:00 sleep 20
scenario 2:
root#ubuntu-jstorm:~/Desktop# su cr -c 'sleep 20'
I got 2 processes by: cr#ubuntu-jstorm:~$ ps -ef | grep -v grep | grep sleep
root 7975 6544 0 10:03 pts/2 00:00:00 su cr -c sleep 20
cr 7976 7975 0 10:03 ? 00:00:00 sleep 20
The process of sleep 20 is the one I care, especially in Upstart, the process managed by Upstart should be this while not bash -c sleep 20 > /tmp/a.out is managed by Upstart, while not the sleep 20.
In scenario 1, upstart will not work correctly, above is the reason.
Therefore, why scenario 1 got 3 process, this doesn't make sense for me. Even though I know I can use command 'exec' to fix it, I just want to get the procedure what happened when the two command committed.
su -c starts the shell and passes it the command via its -c option. The shell may spawn as many processes as it likes (it depends on the given command).
It appears the shell executes the command directly without forking in some cases e.g., if you run su -c '/bin/sleep $$' then the apparent behaviour as if:
su starts a shell process (e.g., /bin/sh)
the shell gets its own process id (PID) and substitute $$ with it
the shell exec() /bin/sleep.
You should see in ps output that sleep's argument is equal to its pid in this case.
If you run su -c '/bin/sleep $$ >/tmp/sleep' then /bin/sleep argument is different from its PID (it is equal to the ancestor's PID) i.e.:
su starts a shell process (e.g., /bin/sh)
the shell gets its own process id (PID) and substitute $$ with it
the shell double forks and exec() /bin/sleep.
The double fork indicates that the actual sequence of events might be different e.g., su could orchestrate the forking or not forking, not the shell (I don't know). It seems the double fork is there to make sure that the command won't get a controlling terminal.
command > file
This is not atomic action, and actually done in 2 process.
One is execute the command;
the other do the output redirection.
Above two action can not done in one process.
Am I right?
Related
Creating a bash script with this command:
cat <<"END"> z
#! /bin/bash
sleep 20 && exit 1 &
ret=$!
ps $ret | grep $ret
END
and then running it gives:
7230 pts/39 S+ 0:00 /bin/bash ./z
I was expecting to see sleep 20 ... which is the child process. If I remove the && exit 1 it does return the child process.
Whats the reason? How can I get the child process id in above statement?
You already get the right information about the child process. Only in your case, ps doesn't know or want to show a proper COMMAND name for your chained sub-process you start in the background - what probably confused you.
Looks like this is the case with the chained commands (.. && ..., thus it has nothing to do with exit 1 could be also echo 5 etc.) where the process group leader name is showed as cmd name instead.
From the (ps man page)
`cmd | COMMAND`: simple name of executable
# Process state codes
`S`: interruptible sleep (waiting for an event to complete)
`+`: is in the foreground process group
See the S+ in your ps | grep output.
So, you can adapt your script a bit to confirm that you actually capture(d) the right information about the child process, like so:
cat <<"END"> z
#! /bin/bash
sleep 20 && exit 1 &
ret=$!
echo $ret
jobs -l
# display parent and child process info
# -j Jobs format
ps -j $$ $ret
END
Output of echo $ret:
30274
Output of jobs -l:
[1]+ 30274 Running sleep 20 && exit 1 &
Output of ps -j $$ $ret:
PID PGID SID TTY STAT TIME COMMAND
30273 30273 21804 pts/0 S+ 0:00 /bin/bash ./z
30274 30273 21804 pts/0 S+ 0:00 /bin/bash ./z
Note that both the parent and child have the same PGID, whereas the pid 30274 of the child process displayed by jobs -l and ps ... matches.
Further, if you change sleep 20 && exit 1 & as bash -c 'sleep 20 && exit 1' & you would get a proper command name for the child this time, as follows (cf. output order above):
30384
[1]+ 30384 Running bash -c 'sleep 20 && exit 1' &
PID PGID SID TTY STAT TIME COMMAND
30383 30383 21804 pts/0 S+ 0:00 /bin/bash ./z
30384 30383 21804 pts/0 S+ 0:00 bash -c sleep 20 && exit 1
Last but not least, in your original version instead of ps $ret | grep $ret you could also try
pstree -s $ret
From pstree man page
-s: Show parent processes of the specified process.
Which will provide you with an output similar to that one below, which would also confirm that you get the right process info for sleep 20 && exit 1 &:
systemd───systemd───gnome-terminal-───bash───bash───sleep
What you see is not parent pid, but sub-shell pid
When you run :
sleep 20 && exit 1 &
The processes tree is like :
current-shell ---> sub-shell ---> 'sleep 20 && exit 1'
When you run :
sleep 20 &
The processes tree is like :
current-shell ---> 'sleep 20'
Reason why you see pid for 'sleep 20'
Whats the reason?
The reason is that some entity has to do &&. It can't be sleep, because sleep only sleeps, and after sleep terminates (so there is no longer sleep to make any decision), some "entity" needs to compare the exit status of sleep and decide and then execute exit 1. That "entity" is the shell, that has to be "above" sleep to do the action. So the "real" background process is the shell, and sleep is it's child process.
In case of only sleep 20 & there is an optimization in bash that the parent shell in case bash sees there is only a single command to do. So bash scans the whole command command bla bla & and sees there is only one command to do. Because of that bash does only call to exec instead of the standard fork+exec and becomes sleep itself instead of running a child process. Because of the exec the subshell becomes sleep, so you see it in process name. It's a resource optimization done bash.
man bash seems to suggest that if I want to execute a command in a separate bash shell all I have to do is bash -c command:
-c string If the -c option is present, then commands are read from string.
I want to do that because I need to run a few things in different environments:
bash --rcfile ~/.bashrc.a -c mytest.a
bash --rcfile ~/.bashrc.b -c mytest.b
However, that didn't work as expected; one can see that by the number of bash shells running, for example:
$ bash
$ ps
PID TTY TIME CMD
7554 pts/0 00:00:00 bash
7573 pts/0 00:00:00 ps
28616 pts/0 00:00:00 bash
$ exit
exit
$ ps
PID TTY TIME CMD
7582 pts/0 00:00:00 ps
28616 pts/0 00:00:00 bash
$ bash -c ps
PID TTY TIME CMD
7583 pts/0 00:00:00 ps
28616 pts/0 00:00:00 bash
How should the invocation of bash should be modified so that it would start a new shell with the specified rc, execute the given command in that shell (with the env modified according to the rc), and exit back?
It's already working exactly the way you want it to. The lack of an extra process is simply due to bash's tail-call optimization.
Bash recognizes that there's no point in having a shell instance whose only job is to wait for a process and exit. It will instead skip the fork and exec the process directly. This is a huge win for e.g. var=$(ps), where it cuts the number of expensive forks from 2 to 1.
If you give it additional commands to run afterwards, this optimization is no longer valid, and then you'll see the additional process:
$ bash -c 'ps'
PID TTY TIME CMD
4531 pts/10 00:00:00 bash
4540 pts/10 00:00:00 ps
$ bash -c 'ps; exit $?'
PID TTY TIME CMD
4531 pts/10 00:00:00 bash
4549 pts/10 00:00:00 bash
4550 pts/10 00:00:00 ps
bash --rcfile ~/.bashrc.a mytest.a will already run mytest.a in a separate process. -c is for specifying a shell command directly, rather than running a script.
# NO!
bash for x in 1 2 3; do echo "$x"; done
# Yes.
bash -c 'for x in 1 2 3; do echo "$x"; done'
What happens if you have multiple exec commands in a shell script, for example:
#!/bin/sh
exec yes > /dev/null &
exec yes alex > /dev/null
I assume that a fork is still needed in order to execute the first command since the shell needs to continue executing?
Or does the & specify to create a sub process in which the exec is actually then run?
The use of & implie a sub-process.
So exec have no effect.
Demo:
export LANG=C
echo $$
17259
exec sh -c 'echo $$;read foo' &
[1] 17538
17538
[1]+ Stopped exec sh -c 'echo $$;read foo'
fg
exec sh -c 'echo $$;read foo'
17259
I run the script: echo $$;read foo in order to prevent exit before having quietly read previous output.
In this sample, the current process ID is 17259.
When run with ampersand (&), the output is another pid (bigger). when run without ampersand, the new shell replace the command and is not forked.
Replacing the command by:
sh -c 'echo $$;set >/tmp/fork_test-$$.env;read'
re-running the whole test will generate two files in /tmp.
On my desk, I could read:
19772
19994
19772
So I found two files in /tmp:
-rw-r--r-- 1 user0 user0 2677 jan 22 00:26 /tmp/fork_test-19772.env
-rw-r--r-- 1 user0 user0 2689 jan 22 00:27 /tmp/fork_test-19994.env
If I run: diff /tmp/fork_test-19*env, I read:
29c29
< SHLVL='0'
---
> SHLVL='1'
46a47
> _='/bin/sh'
So the first run, with ampersand is in a sublevel.
Nota: This was tested under many different shell.
The shell forks to run the background process, but that means the new shell still needs to fork to run yes. Using exec eliminates the fork in the subshell.
I'm running two levels of xterms. In the first level I run "xterm -e bsub -Ip master.tcl". The master.tcl script invokes yet another xterm with "xterm -e bsub -Ip slave.tcl".
From some reason, when slave.tcl finishes executing, the second xterm is not closing. However, the second xterm does display the following message once the slave script finishes:
<< JobExitInfo: Job <128309> is done successfully. >>
Also, when looking at the LSF system, the job does not appear, as if it really finished. But the xterm window stays open, instead of closing.
Any idea why?
Thanks.
It is unlikely that xterm would stay open unless there was something still running there.
I'd check (using ps -ef for instance) to see what processes are still running in the remaining xterm. xterm would only be still open if there were something running, e.g., waiting for input.
Using ps -ef (assuming this is not a BSD system), you would see a listing with a heading like this:
UID PID PPID C STIME TTY TIME CMD
and later in the listing, the relevant information, e.g.,
tom 3647 20185 0 06:17 pts/2 00:00:00 sh -c xterm -e vile
tom 3648 3647 0 06:17 pts/2 00:00:00 xterm -e vile
tom 3649 3648 0 06:17 pts/3 00:00:00 vile
tom 3650 3649 0 06:17 pts/3 00:00:00 sh -c ps -ef
tom 3651 3650 0 06:17 pts/3 00:00:00 ps -ef
xterm's process-id (PID) is the place to start. It would be found in the PPID (parent's process-ID) column at least one other place. In turn, that process's PID may be used in further child-processes.
BSD systems use a different set of options (see FreeBSD for example), but in general you can obtain the necessary information from ps.
If I run:
$> bash script.sh
a fork-and-exec happens to run the bash binary. Does that process execute script.sh or does it create a sub-shell in turn in the same way that
$> ./script.sh
first creates a sub-shell to execute the script?
The bash process that runs bash script.sh executes the script directly, not as a second layer of fork and exec. Obviously, individual commands within the script are forked and executed separately, but not the script itself.
You could use ps to show that. For example, script.sh might contain:
tty
echo $$
sleep 20
You could run that and in another terminal window run ps -ft tty0 (if the tty command indicated tty0), and you'd see the shell in which you ran the bash script.sh command, the shell which is running script.sh and the sleep command.
Example
In ttys000:
$ bash script.sh
/dev/ttys000
65090
$
In ttys001:
$ ps -ft ttys000
UID PID PPID C STIME TTY TIME CMD
0 2422 2407 0 9Jul14 ttys000 0:00.13 login -pfl jleffler /bin/bash -c exec -la bash /bin/bash
199484 2428 2422 0 9Jul14 ttys000 0:00.56 -bash
199484 65090 2428 0 3:58PM ttys000 0:00.01 bash script.sh
199484 65092 65090 0 3:58PM ttys000 0:00.00 sleep 20
$
You can use pstree or ps -fax to look at the process tree. In your case when specifying bash as a (forked) command with a script parameter it will not (need) to fork a subshell as running with "command file" is one mode of operation (if not used -c).
BTW: you can also use exec sh script.sh to replace your current shell process with the new sub shell.
When you call a shell script without the source (or .) command, it will run in a subshell. This is the case for your second line. If you want to run the script in the current one, you would need to use . ./script.sh.