I have named pipe that gets data very slowly but endlessly, I want to copy the contents of the named pipe to date formatted files while they arrive.
I have something like this
do
cat /tmp/big_file > `printf '%(%Y/%m/%d)T' -1`.output &
sleep 3590
kill $!
sleep 10
done
Is it safe to just kill cat? Or could I lose some data in its buffer? How do I tell cat it's time to stop?
If you do not want to use while read you can send signals also with kill like in https://unix.stackexchange.com/questions/2107/how-to-suspend-and-resume-processes By default kill sends SIGTERM. You can also send signals using C or shell code like in https://www.thegeekstuff.com/2011/02/send-signal-to-process/ With SIGTSTP you suspend a process, SIGCONT continues it, a list of all signals is in http://man7.org/linux/man-pages/man7/signal.7.html
https://unix.stackexchange.com/questions/2107/how-to-suspend-and-resume-processes
There are some pitfalls described in https://unix.stackexchange.com/questions/149741/why-is-sigint-not-propagated-to-child-process-when-sent-to-its-parent-process
Related
I'm trying to write a bash script that starts a program, waits for x seconds and than sends the ctrl+c signal to the program to stop it.
The program is "trace-cmd" (which is a frontend for ftrace) that records/traces data until ctrl+c is pressed.
I already found a solution to get the PID of trace-cmd, and to send the SIGINT signal using kill to it. Yet, somehow it does not work.
This is my command that is executed (all in one line, just formatted for readability):
sudo trace-cmd record -p function -P $PID & echo $! > ./pid_trace.txt &
echo "[+] Stored PID:" $(cat ./pid_trace.txt) &
sleep $seconds; printf "Killing trace-cmd\n"; sudo kill -INT $(cat ./pid_trace.txt)
The 'echo' is just for testing, I used the txt file as I could not assign the $! value to a variable. As far as I understood: the "&" is used so that these commands are executed concurrently and the ";" so that they are executed after each other. So: i should start trace-cmd, store the PID than start the time, and only after the timer is done execute the kill.
When the ctrl+c signal is pressed while executing trace-cmd, a specific output can be seen (basically that the trace is getting stored). Yet, with my bash program I cannot see it. I assume that the kill signal is either not "delivered" or that the SIGINT is not the signal that trace-cmd expects (can a program intercept these signals? or the key-strokes rather)
Any help would be appreciated!
The 'sudo' process does not pass INT signal to it's children. The CTRL/C processing passed the INT signal to all processes running in the foreground for the connected terminal.
Try one of the options:
Consider using regular kill on sudo (NO -INT). This will use TERM signal, which will result in sudo terminating sub processes.
Send the INT signal directly to the frace (pkill -INT trace_cmd)
Please note that this questions was edited after a couple of comments I received. Initially I wanted to split my goal into smaller pieces to make it simpler (and perhaps expand my knowledge on various fronts), but it seems I went too far with the simplicity :). So, here I am asking the big question.
Using bash, is there a way one can actually create an anonymous pipe between two child processes and know their pids?
The reason I'm asking is when you use the classic pipeline, e.g.
cmd1 | cmd2 &
you lose the ability to send signals to cmd1. In my case the actual commands I am running are these
./my_web_server | ./my_log_parser &
my_web_server is a basic web server that dump a lot of logging information to it's stdout
my_log_parser is a log parser that I wrote that reads through all the logging information it receives from my_web_server and it basically selects only certain values from the log (in reality it actually stores the whole log as it received it, but additionally it creates an extra csv file with the values it finds).
The issue I am having is that my_web_server actually never stops by itself (it is a web server, you don't want that from a web server :)). So after I am done, I need to stop it myself. I would like for the bash script to do this when I stop it (the bash script), either via SIGINT or SIGTERM.
For something like this, traps are the way to go. In essence I would create a trap for INT and TERM and the function it would call would kill my_web_server, but... I don't have the pid and even though I know I could look for it via ps, I am looking for a pretty solution :).
Some of you might say: "Well, why don't you just kill my_log_parser and let my_web_server die on its own with SIGPIPE?". The reason why I don't want to kill it is when you kill a process that's at the end of the pipeline, the output buffer of the process before it, is not flushed. Ergo, you lose stuff.
I've seen several solutions here and in other places that suggested to store the pid of my_web_server in a file. This is a solution that works. It is possible to write the pipeline by fiddling with the filedescriptors a bit. I, however don't like this solution, because I have to generate files. I don't like the idea of creating arbitrary files just to store a 5-character PID :).
What I ended up doing for now is this:
#!/bin/bash
trap " " HUP
fifo="$( mktemp -u "$( basename "${0}" ).XXXXXX" )"
mkfifo "${fifo}"
<"${fifo}" ./my_log_parser &
parser_pid="$!"
>"${fifo}" ./my_web_server &
server_pid="$!"
rm "${fifo}"
trap '2>/dev/null kill -TERM '"${server_pid}"'' INT TERM
while true; do
wait "${parser_pid}" && break
done
This solves the issue with me not being able to terminate my_web_server when the script receives SIGINT or SIGTERM. It seems more readable than any hackery fiddling with file descriptors in order to eventually use a file to store my_web_server's pid, which I think is good, because it improves the readability.
But it still uses a file (named pipe). Even though I know it uses the file (named pipe) for my_web_server and my_log_parser to talk (which is a pretty good reason) and the file gets wiped from the disk very shortly after it's created, it's still a file :).
Would any of you guys know of a way to do this task without using any files (named pipes)?
From the Bash man pages:
! Expands to the process ID of the most recently executed back-
ground (asynchronous) command.
You are not running a background command, you are running process substitution to read to file descriptor 3.
The following works, but I'm not sure if it is what you are trying to achieve:
sleep 120 &
child_pid="$!"
wait "${child_pid}"
sleep 120
Edit:
Comment was: I know I can pretty much do this the silly 'while read i; do blah blah; done < <( ./my_proxy_server )'-way, but I don't particularly like the fact that when a script using this approach receives INT or TERM, it simply dies without telling ./my_proxy_server to bugger off too :)
So, it seems like your problem stems from the fact that it is not so easy to get the PID of the proxy server. So, how about using your own named pipe, with the trap command:
pipe='/tmp/mypipe'
mkfifo "$pipe"
./my_proxy_server > "$pipe" &
child_pid="$!"
echo "child pid is $child_pid"
# Tell the proxy server to bugger-off
trap 'kill $child_pid' INT TERM
while read
do
echo $REPLY
# blah blah blah
done < "$pipe"
rm "$pipe"
You could probably also use kill %1 instead of using $child_pid.
YAE (Yet Another Edit):
You ask how to get the PIDS from:
./my_web_server | ./my_log_parser &
Simples, sort of. To test I used sleep, just like your original.
sleep 400 | sleep 500 &
jobs -l
Gives:
[1]+ 8419 Running sleep 400
8420 Running | sleep 500 &
So its just a question of extracting those PIDS:
pid1=$(jobs -l|awk 'NR==1{print $2}')
pid2=$(jobs -l|awk 'NR==2{print $1}')
I hate calling awk twice here, but anything else is just jumping through hoops.
I have a couple of bahs scripts running at the same time, and they communicate with each other by saving trigger variables in a folder. So one script will do something, and when its done it will echo "done" > variablefolder. The second script has a loop, checking every now and then if there is a "done" in the variable folder. If it is, the script executes something.
Does Bash support any better way of doing this? I know about export name=value, but that in practice does pretty much the same as what I'm doing now. I'm thinking, is there any way of pushing information to a Bash script that reacts on it? So when something is pushed to it, the Bash script will run a function, or something?
One way to handle inter-process communications is to use signals...
To send a signal to another process you can use the kill command.
The kill command uses the process id to identify the process.
You can save the process id to a file after the script starts using the $$ variable
Here is an example of a script that will catch a signal:
#!/bin/bash
echo $$ > /tmp/pid # Save the pid
function do_stuff {
echo "I am doing stuff"
exit
}
trap do_stuff SIGINT
while `true`
do
echo "Waiting for a signal"
sleep 1
done
So to send it a signal you can do this:
#!/bin/bash
pid=`cat /tmp/pid` # Read the pid
kill -s INT $pid
I am working with programs that use CTRL-C to stop a task, and what I want to do is to run that task for certain number of minutes and then have it stop like CTRL-C was pressed. The reason why I want it to stop like ctrl+c was pressed is because it auto saves when you stop the program instead of killing it and possibly losing the saved data.
edit; I don't want to use cron unless if it stops my script it will have the program save the data, I am hoping to accomplish this inside the shell script.
The trap statement catches these sequences and can be programmed to execute a list of commands upon catching those signals.
-#!/bin/bash
trap "echo Saving Data" SIGINT
while :
do
sleep 60
done
For Information on Traps : http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_12_02.html
Using timeout command to send SIGINT after 60 seconds:
timeout --signal=INT 60 /path/to/script.sh params
If you need to intercept ctrl+c, you should use the trap builtin, like this :
cleanup(){ # do something...; }
trap 'cleanup' 2
# rest of the code
2 on trap line is the SIGINT signal sended by ctrl+c, see man 7 signals
Try the following:
#!/bin/bash
set -m
/path/to/script.sh params &
set +m
bg_pid=$!
sleep 60
kill -2 $bg_pid
This should allow you to send SIGINT to a backgrounded process using Job Control and the set builtin
Is it possible to set a process to die in its start parameters?
What I'm trying to do is set a process to die either before it's started or when it's starting.
When I try to get a pid of an opened named pipe with cat > $pipe by adding an ampersand and $! after it, it spams the terminal with ">" symbols, so I was wondering if it were possible to start cat > $pipe with a parameter to die on a PID's SIGHUP.
How would I go about doing this? If I put anything after the cat > $pipe, it will not work correctly for me.
"get a pid of an opened named pipe"
A named pipe does not have a pid, only processes have pids (the clue is in the 'p')
However, that does not appear to be anything to do with the title of the question. By default, a process will die on a SIGHUP. However, a child process inherits the parent's signal mask, so if the parent ignored SIGHUP then that will be the case in the child (not true for handlers). So you can force a die with (for example):
trap 'exit 128' SIGHUP
But how does that part of the question relate to named pipes? Are you trying to find which processes have the pipe open? You can iterate through /proc for that.
EDIT after comments from the OP:
If you run cat > mypipe & then the cat will hang trying to access the keyboard - cat by default reads STDIN.
[1]+ Stopped cat > mypipe
So then you have to bring it into forground (fg) to enter data, normally terminated by <CTRL>+D. I am at a loss as to why you want to use cat in this way.
Anyway, if you run in background it is very easy to get a background job's pid:
assuming it is job number 1
set $(jobs -l %1)
pid=$2
Maybe you could further investigate why you can't run the job in background, and show an example (use the script command to get a copy of your session in a file called typescript)
Have you tried putting it in parentheses? (cat > $pipe) &$