So heres the situation (in windows):
Theres a child process started by the parent, which only has one pipe open, stdout.
In order for the parent to end the process, it calls pclose
We can't call read on the pipe to detect if its broken to end the process (because it's a write only pipe, read will always return immediately with an error)
Is there a way to get an event from the pipe when the read end (on the parent) closes? If not, we have to continuously write garbage to the pipe in order to detect when it closes, which is a sub-optimal and wasteful solution.
Related
I faced a concurrency problem when writing to the same named pipe created with mkfifo by multiple processes at the same time, where some writes got lost. Since the number of writing processes are limited I want to switch from "writing to 1 pipe from n processes and reading from 1 separate" to "writing to n pipes by n processes and reading from 1 separate process".
Currently I'm reading via read line <"$pipe" in a loop until a condition is met. read blocks here until a line was read.
How can I read from multiples pipes ($pipe1, $pipe2 … $pipeN) via one loop until a condition is met, while honouring newly written lines on all pipes the same?
One way to deal with the initially described problem of multiple children writing to a single FIFO is to have a process open the FIFO for reading but never actually read it. This will allow writers to write unless the FIFO is full. I don't think there's a standard program that simply goes to sleep forever until signalled. I use a home-brew program pause, which is a pretty minimal C program:
#include <unistd.h>
int main(void)
{
pause();
}
It never exits until signalled. In your shell script, first launch the children, telling them to write to $FIFO, then run:
pause <$FIFO &
pid=$!
Note that pause-like command will not be launched into the background until the redirection completes, and the open of the FIFO won't complete until there is a process to write to the FIFO — so at least one child needs to be launched in background before the pause-like process is executed. Or write a variant of pause (I call mine sleepon) which opens the files named in its argument list — then the command line is similar to sleepon $FIFO & and the backgrounding operation completes and the pause-like program blocks until it is able to open the FIFO (which will be when one of the children opens the FIFO for writing), and then goes to sleep indefinitely. But the code for sleepon is a lot more complex than the code for pause.
Once the children and the pause-like process are launched, the parent can continue with the main processing loop.
while read line
do
…
done < $FIFO
The main thing to be aware of is that the parent loop will exit whenever the FIFO is emptied. You need to know when it should terminate, if ever. At the point where it does terminate, it should kill the pause process: kill $pid. You may need to wrap a while true; do … done loop around the line-reading loop — but you may need something cleverer than that. It depends, in part, on what your "until a condition is met" requirement is.
Your requirement to 'read from multiple FIFOs, all of which may intermittently have data on them' is not easy to do. It's not particularly trivial in C; I don't think there's a standard (POSIX) shell command to assist with that. In C, you'd end up using POSIX select() or
poll() or one of their many variants — some of which are platform-specific. There might be a platform-specific command that will help; I have my doubts, though.
In Linux/Unix when a process in the background mode tries to read from stdin it gets terminated. What is causing this termination ? There is no core file. So it doesn't look like termination is by a signal that generates core.
One reason for termination could be a signal.
When process are not connected directly to a tty device, stdin/stdout are typically handled with pipes.
The pipe(7) man page says:
If all file descriptors referring to the read end of a pipe have been closed, then a write(2) will cause a SIGPIPE signal to be generated for the calling process.
However, this applies only to writing.
For reading,
if all file descriptors referring to the write end of a pipe have been closed, then an attempt to read(2) from the pipe will see end-of-file (read(2) will return 0).
It is quite possible that the program, when it cannot read anything, decides to terminate. (What else could it do?)
I'm currently using this example as a guide to redirect standard error of a child process launched by CreateProcess.
However unlike the example currently I'm waiting until the process finishes (checking GetExitCodeProcess), closing the pipe and then reading the error if a non-zero return code comes back.
However I've since read if the pipe fills up the client process will block until the pipe is cleared. The reason I'm not currently reading from the pipe during execution is that the ReadFile call blocks during execution (standard error is only output at the end) so I can't pump the message queue to avoid the GUI from "ghosting" and being marked not responding.
I can't find any reference to how big the pipe is by default (although I can set a size myself), is this something I need to worry about given I'm buffering the output into a string variable for later use anyway? (ie. it would need to fit into the available memory for the process so it has a hard limit there, it's not going to a file like most of the examples have)
I'm writing a program that communicate with its child process using anonymous pipe. And I want to know if there are contents in the pipe for me to read. Because calling ReadFile for an empty pipe would result in halting.
The child process is another software so I can't change it. I can only redirect its i/o.
You can use the PeekNamedPipe function to examine the state of the pipe without blocking.
I have some basic questions about pipes I am unsure about.
a) What is the standard behavior if a process writing to a pipe gets killed (ie. SIGKILL SIGINT) Does it close the pipe? Does it flush the pipe? Or is the behavior undefined?
b) What is the standard behavior if a process returns normally? Is it guaranteed to flush the pipe and close the pipe? (without explicitly doing so of course).
I would like these answers to be as general as possible, but in reality if it depends entirely on the OS specs I can accept that! However, if there is a Posix standard or a current defined Windows behavior I would be very grateful to know.
Thanks.
a. What is the standard behavior if a process writing to a pipe gets killed (ie. SIGKILL SIGINT) Does it close the pipe? Does it flush the pipe? Or is the behavior undefined?
SIGKILL never allows any cleanup - the process dies, dead. With SIGINT, it depends on whether the process handles the signal. If so, it is likely to exit via exit(2), which flushes standard I/O file handles. The question is - was the pipe connected to standard output or via popen()? If so, outstanding buffered data may be flushed; if not, there is no buffered data so flushing is immaterial.
If there is unread data in the pipe, that data remains in the pipe, ready for the reader to collect - assuming there is a reader.
b. What is the standard behavior if a process returns normally? Is it guaranteed to flush the pipe and close the pipe? (without explicitly doing so of course).
It depends on whether the pipe was connected via standard I/O or not. If not, there is nothing pending. If so, then yes, any material in the buffers will be flushed as the standard I/O stream is closed.
c. Thanks for the info on signals and the unread data, but I'm a little confused about the standard I/O pipe connection. After you mentioned popen() I looked it up and the man page says its return value identical to an I/O stream and the streams are fully buffered by default. I'm just not clear on the difference between the two nor do I understand where the difference comes from.
The basic system call for creating pipes is pipe(2). It creates two file descriptors, one for the read end of the pipe, one for the write end. If you do nothing else with them, then they remain as file descriptors, with unbuffered output (via write(2) and related system calls). If the process terminates, there is no buffering in the application; the pipe is closed.
If you use popen(3), then it does a whole lot more work for you. It still invokes pipe(2) to create the pipes, but it then does a fork(2). The child arranges the correct configuration of the pipes and launches the child process. The parent also closes the unused end of the pipe, and uses fdopen(3) to create a standard I/O file stream for the calling process to use.
With the file stream, if there is data in the I/O buffer, then a close or equivalent will ensure that the outstanding data is flushed and the file descriptor is closed.
The normal behaviour is that all file descriptors are closed when a process terminates. This means that a pipe, like any other open file descriptor, is closed normally.
One interesting thing about pipes, though: in POSIX, if a process writes to a pipe that has been closed, the writer will get a signal, SIGPIPE.
Edit:
A caveat: The difference between s SIGx termination and a normal termination is that, like any other file write, you may loose data that has been buffered (via a FILE write) and not yet written to the file descriptor.