number of forks a shell script performs during execution - shell

Question

Is there a way to compute the number of forks a shell script performs while it is executing? I've been looking at maybe writing a C wrapper using getrusage(2) and analyzing the various fields of
struct rusage {
struct timeval ru_utime; /* user time used */
struct timeval ru_stime; /* system time used */
long ru_maxrss; /* max resident set size */
long ru_ixrss; /* integral shared text memory size */
long ru_idrss; /* integral unshared data size */
long ru_isrss; /* integral unshared stack size */
long ru_minflt; /* page reclaims */
long ru_majflt; /* page faults */
long ru_nswap; /* swaps */
long ru_inblock; /* block input operations */
long ru_oublock; /* block output operations */
long ru_msgsnd; /* messages sent */
long ru_msgrcv; /* messages received */
long ru_nsignals; /* signals received */
long ru_nvcsw; /* voluntary context switches */
long ru_nivcsw; /* involuntary context switches */
};
but the number of forks isn't available here. Next idea is to strace shell and children and look for the forks. Is there a simpler way with less overhead? Is there some shell with a nonstandard option/variable/mechanism to show the number of forks?

Answer 1

There are a few options:
the best multi-platform approach is likely strace or its equivalent (truss, ktrace), or dtrace. See below. This also lets you attach to a running process.
a workable, if slightly tricky multi-platform approach is to create a dynamic library with your own versions of fork/execve etc, which log the calls, and then call the real C library functions. Search SO for LD_PRELOAD to get some ideas. This won't work on statically linked binaries though.
on Linux and Solaris you can set the environment variable LD_DEBUG=files, and the dynamic linker will issue various diagnostics as both executables and libraries are loaded, it should be enlightening for such a simple step. On Linux each new process should output some or most of "initialize", "init" and "fini" entries, along with PIDs. This won't work on statically linked binaries though.
if you are on Linux or *BSD or Solaris, and have root access, and process accounting is available you can run your command(s), and then inspect the output of lastcomm or dump-acct. This might require accounting to be started (if it is not already running). This might not provide the details you need on some platforms. It can be done easily on RH/CentOS 6, and provides all the details needed. Other systems also have process accounting.
if you are on Linux and have auditd support you can use autrace myscript.sh to log system calls. (auditd should be running for this so the kernel data is logged to the audit file)
for completeness: you could use a debugger, but that's about the most tedious approach I can think of ;-)
On Linux you can trace execution (moderate performance penalty) with:
strace -f -o /tmp/myscript.trace -e trace=process ./myscript.sh
Then inspect the .trace file. The parameter -e trace=process filters to show only process related syscalls.
On Solaris you can trace with:
truss -f -o /tmp/myscript.trace \
-u libc:fork,execl,execv,execle,execve,execlp,execvp ./myscript.sh
Solaris truss lets you trace both userland libraries and kernel syscalls. You could also use dtrace, see here for some ideas: http://www.brendangregg.com/DTrace/lostcpu.html
Other platforms have variables similar to LD_DEBUG or LD_VERBOSE, see the linker documentation (e.g. man ld.so).
In the above cases you should understand that what programs (usually) call are C library functions, e.g. fork(), what is requested of the kernel depends on the OS at least, and it may result in a syscall of vfork, execve or clone.