pipe of "ps -e", "tee" and "head" behaves strange - bash

I meant to use the following command to grep the "zsh" process with header. But this command only showed the header and then exited. The grep didn't run. I can do this task by other ways but I want to understand what happened here. The interesting thing is that the pipe works as expected if other command like ls replaces ps -e. In fact, the pipe works even with ps (without -e). The pipe also works if head is replaced by another grep. Seemingly, there are some interesting interactions between ps -e, head and tee in this pipe. Any explanation? I ran the commands in MacOS-iterm2 and zsh. Thanks.
ps -e|tee >(head -n 1) >(grep "zsh") >/dev/null
the command above produces the following incorrect output: only header is shown, grep didn't run.
PID TTY TIME CMD
The correct output can be generated by the following command:
ps -e|awk 'FNR==1{print};/zsh/{print}'
PID TTY TIME CMD
1658 ttys000 0:00.30 -zsh
2817 ttys001 0:00.49 /bin/zsh -i
12890 ttys002 0:00.26 -zsh
13332 ttys003 0:00.23 -zsh
13469 ttys004 0:00.19 /bin/zsh -i
Update: The problem is indeed in the tee shipped with MacOS. As #OndrejK. pointed out in the comments below, tee from GNU works as expected.

I think Ondrej K. nailed it in a comment: head is reading just the first line it receives from tee, then exiting, so when tee tries to write the second line to that pipe, it gets an error (probably actually a SIGPIPE signal) and exits without writing the rest of ps's output.
I was able to reproduce this problem under both macOS and Raspbian, using both zsh and bash as my shell. I was able to get the full output by replacing head with something that reads (but doesn't output) the rest of the input from tee. Here's an example:
% ps -e|tee >(head -n 1) >(grep "zsh") >/dev/null
PID TTY TIME CMD
% ps -e|tee >(head -n 1; cat>/dev/null) >(grep "zsh") >/dev/null
PID TTY TIME CMD
13127 pts/0 00:00:00 zsh
13353 pts/0 00:00:00 zsh
% ps -e|tee >(sed -n '1p') >(grep "zsh") >/dev/null
PID TTY TIME CMD
13127 pts/0 00:00:00 zsh
In the second version above, adding cat>/dev/null after head lets it consume (and discard) the rest of the input after head exits. In the third, sed -n '1p' reads its entire input, but only prints the first line.
Note that the result here will depend on things like buffer sizes and the timing of various events, so it may be inconsistent between different environments or even vary randomly under apparently consistent conditions. I suspect the reason it works with ps (without the -e option) is that it doesn't produce enough output to trigger the problem: tee may be able to forward the entire output from ps before head gets around to exiting.
EDIT: At least in my environments, adding a delay after head exits prevents the problem:
% ps -e|tee >(head -n 1; sleep 1) >(grep "zsh") >/dev/null
PID TTY TIME CMD
8002 pts/0 00:00:00 zsh
8164 pts/0 00:00:00 zsh
This means it's timing-limited, rather than buffer-limited. But if the output from ps were big enough, it'd run into buffer limits at some point, and the output would be truncated no matter what delays were inserted.

Related

how to extract the PID of a process by command line

I want to get the PID of a process namely "cron" by command line.
I tried the following script.
ps ax|grep 'cron'
but I am getting a part of a table,
1427 ? Ss 0:00 /usr/sbin/cron -f
24160 pts/5 S+ 0:00 grep --color=auto cron
How I extract the pid from this ?
The pgrep utility will return the process IDs for the currently running processes matching its argument:
$ pgrep cron
228
It may also be used to "grep for" things on the command line:
$ pgrep -f uerfale
69749
69752
$ pgrep -l -f uerfale
69749 slogin uerfale
69752 slogin: /home/kk/.ssh/sockets/uerfale-9022-kk.sock [mux] m
To kill a process by name, use pkill. It works in the same way as pgrep but will send a signal to the matched processes instead of outputting a process ID.
Just use pidof, rather to use other commands and apply post-processing actions on them.
$ pidof cron
22434
To make the command return only one PID pertaining to to the process, use the -s flag
-s
Single shot - this instructs the program to only return one pid.
Like this, for example:
ps -ef|grep 'cron'|grep -v grep|awk '{print $2}'
You can try this;
ps -o pid,sess,cmd afx | egrep "( |/)cron( -f)?$"
or
pstree -pas <cronPID>

ksh on BSD style UNIX. How to redirect stderr to tee and a file

A similar question as been asked here
But it wont work for me
"shell.ksh" > >(tee here.log ) 2> >(tee err.log)
ksh: 0403-057 Syntax error: `>' is not expected.
this command will not redirect stderr just stdop
"shell.ksh" | tee file.log 2>&1
I am on AIX running some UNIX similar to BSD that does not have GNU extensions
and this is ksh
ps $$
PID TTY STAT TIME COMMAND
40632390 pts/6 A 0:00 -ksh
Any short and sweet solns folks ?
Here is a wrapper script that should do what you want. I can't confirm that it works on AIX ksh, but it works using sh (AT&T Research) 93u+ 2012-08-01 (Debian's ksh package calls it the "Real, AT&T version of the Korn shell." This is intentionally neither pdksh nor its mksh fork).
This is portable and should work in all POSIX shells.
pipe1+2.ksh
#!/bin/ksh
touch err.log here.log # create the logs now so tail -f doesn't complain
tail -f here.log & # follow the output we'll pipe to later (run in bg)
HERE=$! # save the process ID of this backgrounded process
tail -f err.log >&2 & # follow output (as above), pipe output to stderr
ERR=$! # save the process ID
"$#" > here.log 2> err.log # run given command and pipe stdout and stderr
RET=$? # save return value
sleep 1 # tail polls at 1/s, so wait one second
kill $HERE $ERR # stop following those logs
exit $RET # restore the exit code from given command
So you invoke it as pipe1+2.ksh shell.ksh (assuming both commands are in your path).
Here's a test version of shell.ksh:
#!/bin/ksh
echo this output is stdout 1
echo this output is stderr >&2
echo this output is stdout 2
and a trial run:
# ksh pipe1+2.ksh ksh shell.ksh
this output is stdout 1
this output is stdout 2
this output is stderr
# cat here.log
this output is stdout 1
this output is stdout 2
# cat err.log
this output is stderr
A note: because tail -f isn't instantaneous, stdout vs stderr will be slightly out of order. It won't even be consistent between runs, at least with code that runs as fast as echo.
I doubt your version of tail supports it, but GNU tail has a -s SECONDS flag that lets you specify a different polling interval as a decimal value, so you can try e.g. tail -f -s 0.01 … to improve the ordering of what is displayed. If your version of sleep also supports decimal values, change it to match that number.
(I had originally crafted a complicated file descriptor piping response based on one of the "more elaborate combinations" in the amazing Csh Programming Considered Harmful, but that didn't seem to work. See the comments and/or the initial version of this answer.)

xterm -e not terminating when script finishes execution

I'm running two levels of xterms. In the first level I run "xterm -e bsub -Ip master.tcl". The master.tcl script invokes yet another xterm with "xterm -e bsub -Ip slave.tcl".
From some reason, when slave.tcl finishes executing, the second xterm is not closing. However, the second xterm does display the following message once the slave script finishes:
<< JobExitInfo: Job <128309> is done successfully. >>
Also, when looking at the LSF system, the job does not appear, as if it really finished. But the xterm window stays open, instead of closing.
Any idea why?
Thanks.
It is unlikely that xterm would stay open unless there was something still running there.
I'd check (using ps -ef for instance) to see what processes are still running in the remaining xterm. xterm would only be still open if there were something running, e.g., waiting for input.
Using ps -ef (assuming this is not a BSD system), you would see a listing with a heading like this:
UID PID PPID C STIME TTY TIME CMD
and later in the listing, the relevant information, e.g.,
tom 3647 20185 0 06:17 pts/2 00:00:00 sh -c xterm -e vile
tom 3648 3647 0 06:17 pts/2 00:00:00 xterm -e vile
tom 3649 3648 0 06:17 pts/3 00:00:00 vile
tom 3650 3649 0 06:17 pts/3 00:00:00 sh -c ps -ef
tom 3651 3650 0 06:17 pts/3 00:00:00 ps -ef
xterm's process-id (PID) is the place to start. It would be found in the PPID (parent's process-ID) column at least one other place. In turn, that process's PID may be used in further child-processes.
BSD systems use a different set of options (see FreeBSD for example), but in general you can obtain the necessary information from ps.

Bash hide "killed"

I cannot see to hide the "killed" output from my script. Please help.
killall -9 $SCRIPT
I have tried:
killall -9 $SCRIPT >/dev/null 2>&1
and everyone redirect combination it seems. Thanks for the help.
* UPDATE *
The main script cannot run in the background. It outputs a bunch on information to the user while running. Thanks for the input though. Any other ideas?
Here is the desired output
HEV Terminated
administrator#HEV-DEV:~/hev-1.2.7$
Here is the current output:
HEV Terminated
Killed
administrator#HEV-DEV:~/hev-1.2.7$
It's not the script printing it, it's the shell. You can suppress it by putting it in the background and waiting for it to finish, redirecting the error from there:
./script &
wait 2>/dev/null
It looks like this can be avoided if you execute the script (including the &) in a subshell:
$ ( ./script.sh & )
$ ps
PID TTY TIME CMD
14208 pts/0 00:00:00 bash
16134 pts/0 00:00:00 script.sh
16135 pts/0 00:00:00 sleep
16136 pts/0 00:00:00 ps
$ killall script.sh
$
If I execute it (without the subshell) like ./script.sh then I get the output
[1]+ Terminated ./script.sh &>/dev/null
Try ...
(killall -9 $SCRIPT) 2>/dev/null
This executes the "killall" in a subshell and all output from that subshell is redirected, including the the "KILLED" output.
Try killall -q,--quiet don't print complaints
killall -q -9 $SCRIPT

Can't understand ps | wc output differences

I was trying to write a set of functions that could check to see if a process name was running when I encountered some unexpected output. I've condensed the issue in the following script names isRunning.sh which depends on a system ps command that can take the '-fC' arguments...
#!/bin/bash
progname=isRunning.sh
ps -fC isRunning.sh
pRet=`ps -fC ${progname} | wc -l`
echo pRet $pRet
psOut=`ps -fC ${progname}`
wcOut=`echo "${psOut}" | wc -l`
echo
echo ps output
echo "${psOut}"
echo
echo wcOut $wcOut
The first attempt at piping the ps output to wc gets a return of 3. The second attempt gets the expected return value of 2. Can anyone explain this behavior? I figure it's got to be something stupid I am overlooking.
Thanks,
bbb
edit: my output
UID PID PPID C STIME TTY TIME CMD
root 6717 5940 0 13:10 pts/0 00:00:00 /bin/bash ./isRunning.sh
pRet 3
ps output
UID PID PPID C STIME TTY TIME CMD
root 6717 5940 0 13:10 pts/0 00:00:00 /bin/bash ./isRunning.sh
wcOut 2
I get 2 both attempts. Your ps might be outputting an extra blank line, or somesuch, and then your shell's backtick expansion stripping it. Or maybe you actually had two processes matching the first time you ran it.
If you just want to see if its running, check the exit code from your ps:
if ps -C "${progname}" > /dev/null; then
echo its running
else
echo not running
fi
Even better, you should take a look at pidof and pgrep if you can rely on them being present on whichever systems you're targeting. Or use the LSB functions, if you're on Linux.
edit: Actually, since you're looking for copies of yourself running, you might be picking up the shell doing a fork to implement the pipe.

Resources