I have a couple of scripts to control some applications (start/stop/list/etc). Currently my "stop" script just sends an interrupt signal to an application, but I'd like to have more feedback about what application does when it is shutting down. Ideally, I'd like to start tailing its log, then send an interrupt signal and then keep tailing that log until the application stops.
How to do this with a shell script?
For just tailing a log file until a certain process stops (using tail from GNU coreutils):
do_something > logfile &
tail --pid $! -f logfile
UPDATE The above contains a race condition: In case do_something spews many lines into logfile, the tail will skip all of them but the last few. To avoid that and always have tail print the complete logfile, add a -n +1 parameter to the tail call (that is even POSIX tail(1)):
do_something > logfile &
tail --pid $! -n +1 -f logfile
Here's a Bash script that works without --pid. Change $log_file and $p_name to suit your need:
#!/bin/bash
log_file="/var/log/messages"
p_name="firefox"
tail -n10 $log_file
curr_line="$(tail -n1 $log_file)"
last_line="$(tail -n1 $log_file)"
while [ $(ps aux | grep $p_name | grep -v grep | wc -l) -gt 0 ]
do
curr_line="$(tail -n1 $log_file)"
if [ "$curr_line" != "$last_line" ]
then
echo $curr_line
last_line=$curr_line
fi
done
echo "[*] $p_name exited !!"
If you need to tail log until process exited, but watch stdout / sdterr at the same time, try this:
# Run some process in bg (background):
some_process &
# Get process id:
pid=$!
# Tail the log once it is created, but watch process stdout/stderr at the same time:
tail --pid=$pid -f --retry log_file_path &
# Since tail is running in bg also - wait until the process has completed:
tail --pid=$pid -f /dev/null
Related
i have some Watcher.sh script running on a directory /foo/bar. When something happens in /foo/bar it starts another script (Updater.sh) which first sleeps for 30 seconds and then do stuff. The Watcher.sh script is managed by supervisor, which is configured to restart the script if it ends. So when there was a change in /foo/bar and the Updater.sh is started (in a subprocess), the Watcher script should end and get started again by supervisor. If there is a second change in /foo/bar the new Updater.sh script should end the first one and start again its 30 second sleep before doing stuff.
Watcher.sh script:
#!/usr/bin/env bash
echo "Started inotifywait with $BASHPID ID"
inotifywait -q -e modify,move,create,delete /foo/bar
/updater.sh &
Updater.sh script:
#!/usr/bin/env bash
echo "Start Update process $BASHPID"
FILE=/var/run/updater.pid
if test -f "$FILE" ; then
kill -9 `cat /var/run/updater.pid`
rm /var/run/updater.pid
fi
echo $BASHPID > /var/run/updater.pid
echo "Change in /foo/bar, reload soon"
sleep 30
supervisorctl reload
My problem is that it just don't happen. The Watcher.sh script is restarted, but the Updater.sh script is not killed at all. I checked the /var/run/updater.pid which is definitely set after the first change, but it just don't enter the if case and even when it does, it does not continue after the kill.
Would be great if somebody knows where i do something wrong, it should be simple right?..
EDIT 1:
Use the following code in the updater.sh to not listen on a pid file
if ps -ef | grep -vE "grep|$BASHPID" | grep updater | awk '{print $2}' ; then
echo "Kill it"
kill -9 `ps -ef | grep -vE "grep|$BASHPID" | grep updater | awk '{print $2}'`
fi
I'm writing a barrier to stall the execution of a script until a certain keyword is logged. The script is pretty simple:
tail -F -n0 logfile.log | while read LINE; do
[[ "$LINE" == *'STOP'* ]] && echo ${LINE} && break;
done
or
tail -F -n0 logfile.log | grep -m1 STOP
The thing is it doesn't quit as soon as the keyword is detected, but only after the next line is written. I.e:
printf "foo\n" >> logfile.log # keeps reading
printf "foo\n" >> logfile.log # keeps reading
printf "STOP\n" >> logfile.log # STOP printed
printf "foo\n" >> logfile.log # code exits at last
Unfortunately I can't rely on the fact that another line will be logged after the "STOP" (not within an interval useful for my purposes at least).
The workaround found so far is to tail also another file I know for sure gets updated quite frequently, but what is the "clean" solution so that the code will exit right after it logs STOP?
In bash, when executing a command of the form
command1 | command2
and command2 dies or terminates, the pipe which receives /dev/stdout from command1 becomes broken. This, however, does not terminate command1 instantly.
So to achieve what you want is to use process substitution and not a pipe
awk '/STOP/{exit}1' < <(tail -f logfile)
When you use awk, you can see the behaviour in a bit more detail:
$ touch logfile
$ tail -f logfile | awk '/STOP/{exit}1;END{print "end"}'
This awk program will check if "STOP" is seen, and if not print the line again. If "STOP" is seen it will print "end"
When you do in another terminal
$ echo "a" >> logfile
$ echo "STOP" >> logfile
$ echo "b" >> logfile
You see that awk prints the following output:
a # result of print
end # awk test STOP, exits and executes END statement
Furthermore, if you look more closely, you see that awk is at this point already terminated.
ps before sending "STOP":
13625 pts/39 SN 0:00 | \_ bash
32151 pts/39 SN+ 0:00 | \_ tail -f foo
32152 pts/39 SN+ 0:00 | \_ awk 1;/STOP/{exit}1;END{print "end"}
ps after sending "STOP":
13625 pts/39 SN 0:00 | \_ bash
32151 pts/39 SN+ 0:00 | \_ tail -f foo
So the awk program terminated, but tail did not crash because it is not yet aware the pipe is broken as it did not attempt to write to it.
When you do the following in the terminal with the pipeline, you see the exit status of tail:
$ echo "${PIPESTATUS[0]} ${PIPESTATUS[1]}"
$ 141 0
Which states that awk terminated nicely, but tail terminated with exit code 141 which means SIGPIPE.
I want to stall the execution of my script until a process is closed (I have the PID stored in a variable).
#!/bin/bash
outputl=$( ps -ef | grep $var4 | awk '{print $2}' ) >> $logfile
while [ "ps -p $outputl" ] > /dev/null;
do
sleep 1;
done
echo "Stopped $instance" >> $logfile
//command...
It stays in the "while" and not continue whit script.
This line:
while [ "ps -p $output1" ]
does not execute the ps command. It simply tests whether the string "ps -p $output1" is not empty, and it obviously isn't. To test the output of a command, use $():
while [ "$(ps -p "$output1")" ]
But since ps produces a header, this will always be true. The best way to test if a PID exists is to use the kill command with signal 0; this doesn't actually send a signal, it just tests whether it's possible to send a signal. I'm assuming this code is being run either by root or the userid running the application being checked. So you can write:
while kill -0 "$output1" 2>/dev/null
Also, your code for getting the PID into $output1 is wrong. ps -ef will also include the grep command, which matches the name you're looking for, so you need to filter that out. Use:
output1=$(ps -ef | grep "$var4" | awk '!/grep/ { print $2 }')
Redirecting the output to $logfile is not necessary, since variable assignments don't print anything.
Many systems have a pgrep command, which can be used by itself to test if a process with a given name exists; if you have this, you can use it instead of reinventing the wheel (and if not, you should be able to install it).
If you have the PID then just wait for it to complete. Try:
outputl=$( ps -ef | awk -v v="$var4" '$0~v{print $2}' )
wait "$outputl"
echo "Stopped $instance" >> $logfile
then look for a better way to find the pid in the first line.
I want to process each stdout-line for a shell, the moment it is created. I want to grab the output of test.sh (a long process). My current approach is this:
./test.sh >tmp.txt &
PID=$!
tail -f tmp.txt | while read line; do
echo $line
ps ${PID} > /dev/null
if [ $? -ne 0 ]; then
echo "exiting.."
fi
done;
But unfortunately, this will print "exiting" and then wait, as the tail -f is still running. I tried both break and exit
I run this on FreeBSD, so I cannot use the --pid= option of some linux tails.
I can use ps and grep to get the pid of the tail and kill it, but thats seems very ugly to me.
Any hints?
why do you need the tail process?
Could you instead do something along the lines of
./test.sh | while read line; do
# process $line
done
or, if you want to keep the output in tmp.txt :
./test.sh | tee tmp.txt | while read line; do
# process $line
done
If you still want to use an intermediate tail -f process, maybe you could use a named pipe (fifo) instead of a regular pipe, to allow detaching the tail process and getting its pid:
./test.sh >tmp.txt &
PID=$!
mkfifo tmp.fifo
tail -f tmp.txt >tmp.fifo &
PID_OF_TAIL=$!
while read line; do
# process $line
kill -0 ${PID} >/dev/null || kill ${PID_OF_TAIL}
done <tmp.fifo
rm tmp.fifo
I should however mention that such a solution presents several heavy problems of race conditions :
the PID of test.sh could be reused by another process;
if the test.sh process is still alive when you read the last line, you won't have any other occasion to detect its death afterwards and your loop will hang.
I want to do a tail -F on a file until matching a pattern. I found a way using awk, but IMHO my command is not really clean. The problem is that I need to do it in only one line, because of some limitations.
tail -n +0 -F /tmp/foo | \
awk -W interactive '{if ($1 == "EOF") exit; print} END {system("echo EOF >> /tmp/foo")}'
The tail will block until EOF appears in the file. It works pretty well. The END block is mandatory because awk's exit does not exit right away. It makes awk to eval the END block before quitting. The END block hangs on a read call (because of tail), so the last thing I need to do, is to write another line in the file to force tail to exit.
Does someone know a better way to do that?
Use tail's --pid option and tail will stop when the shell dies. No need to add extra to the tailed file.
sh -c 'tail -n +0 --pid=$$ -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'
Try this:
sh -c 'tail -n +0 -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'
The whole command-line will exit as soon as the "EOF" string is seen in /tmp/foo.
There is one side-effect: the tail process will be left running (in the background) until anything is written to /tmp/foo.
I've not results with the solution:
sh -c 'tail -n +0 -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'
There is some issue related with the buffer because if there aren't more lines appended to the file, then sed will not read the input. So, with a little more research i came up with this:
sed '/EOF/q' <(tail -n 0 -f /tmp/foo)
The script is in https://gist.github.com/2377029
This is something Tcl is quite good at. If the following is "tail_until.tcl",
#!/usr/bin/env tclsh
proc main {filename pattern} {
set pipe [open "| tail -n +0 -F $filename"]
set pid [pid $pipe]
fileevent $pipe readable [list handler $pipe $pattern]
vwait ::until_found
catch {exec kill $pid}
}
proc handler {pipe pattern} {
if {[gets $pipe line] == -1} {
if {[eof $pipe]} {
set ::until_found 1
}
} else {
puts $line
if {[string first $pattern $line] != -1} {
set ::until_found 1
}
}
}
main {*}$argv
Then you'd do tail_until.tcl /tmp/foo EOF
Does this work for you?
tail -n +0 -F /tmp/foo | sed '/EOF/q'
I'm assuming that 'EOF' is the pattern you're looking for. The sed command quits when it finds it, which means that the tail should quit the next time it writes.
I suppose that there is an outside chance that tail would hang around if the pattern is found at about the end of the file, waiting for more output to appear in the file which will never appear. If that's really a concern, you could probably arrange to kill it - the pipeline as a whole will terminate when sed terminates (unless you're using a funny shell that decides that isn't the correct behaviour).
Grump about Bash
As feared, bash (on MacOS X, at least, but probably everywhere) is a shell that thinks it needs to hang around waiting for tail to finish even though sed quit. Sometimes - more often than I like - I prefer the behaviour of good old Bourne shell which wasn't so clever and therefore guessed wrong less often than Bash does. dribbler is a program which dribbles out messages one per second ('1: Hello' etc in the example), with the output going to standard output. In Bash, this command sequence hangs until I did 'echo pqr >>/tmp/foo' in a separate window.
date
{ timeout -t 2m dribbler -t -m Hello; echo EOF; } >/tmp/foo &
echo Hi
sleep 1 # Ensure /tmp/foo is created
tail -n +0 -F /tmp/foo | sed '/EOF/q'
date
Sadly, I don't immediately see an option to control this behaviour. I did find shopt lithist, but that's unrelated to this problem.
Hooray for Korn Shell
I note that when I run that script using Korn shell, it works as I'd expect - leaving a tail lurking around to be killed somehow. What works there is 'echo pqr >> /tmp/foo' after the second date command completes.
Here's an extended version of Jon's solution which uses sed instead of grep so that the output of tail goes to stdout:
sed -r '/EOF/q' <( exec tail -n +0 -f /tmp/foo ); kill $! 2> /dev/null
This works because sed gets created before tail so $! holds the PID of tail
The main advantage of this over the sh -c solutions is that killing a sh seems to print something to the output such as 'Terminated' which is unwelcome
sh -c 'tail -n +0 --pid=$$ -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'
Here the main problem is with $$.
If you run command as is, $$ is set not to sh but to the PID of the current shell where command is run.
To make kill work you need to change kill $$ to kill \$$
After that you can safely get rid of --pid=$$ passed to tail command.
Summarising, following will work just fine:
/bin/sh -c 'tail -n 0 -f /tmp/foo | { sed "/EOF/ q" && kill \$$ ;}
Optionally you can pass -n to sed to keep it quiet :)
To kill the dangling tail process as well you may execute the tail command in a (Bash) process substituion context which can later be killed as if it had been a backgrounded process. (Code taken from How to read one line from 'tail -f' through a pipeline, and then terminate?).
: > /tmp/foo
grep -m 1 EOF <( exec tail -f /tmp/foo ); kill $! 2> /dev/null
echo EOF > /tmp/foo # terminal window 2
As an alternative you could use a named pipe.
(
: > /tmp/foo
rm -f pidfifo
mkfifo pidfifo
sh -c '(tail -n +0 -f /tmp/foo & echo $! > pidfifo) |
{ sed "/EOF/ q" && kill $(cat pidfifo) && kill $$ ;}'
)
echo EOF > /tmp/foo # terminal window 2
ready to use for tomcat =
sh -c 'tail -f --pid=$$ catalina.out | { grep -i -m 1 "Server startup in" && kill $$ ;}'
for above scenario :
sh -c 'tail -f --pid=$$ /tmp/foo | { grep -i -m 1 EOF && kill $$ ;}'
tail -f <filename> | grep -q "<pattern>"