How to check if stdout was closed - without writing data to it? - go

Question

I wrote a program that reads data, filters and processes it and writes it to stdout. If stdout is piped to another process, and the piped process terminates, I get SIGPIPEd, which is great, because the program terminates, and the pipeline comes to a timely end.
Depending on the filter parameters however, there may not be a single write for tens of seconds, and during that time there won't be a SIGPIPE, although the downstream process has long finished. How can I detect this, without actually writing something to stdout? Currently, the pipeline is just hanging, until my program terminates of natural causes.
I tried writing a zero-length slice
if _, err := os.Stdout.Write([]byte{}); err != nil
but unfortunately that does not result in an error.
N.B. Ideally, this should work regardless of the platform, but if it works on Linux only, that's already an improvement.

Answer 1

This doesn't answer it in Go, but you can likely find a way to use this.
If you can apply Poll(2) to the write end of your pipe, you will get an notification when it becomes un-writable. How to integrate that into your Go code depends upon your program; hopefully it could be useful:
#include <errno.h>
#include <poll.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
void sp(int sno) {
write(2, "sigpipe!\n", 9);
_exit(1);
}
int waitfd(int fd) {
int n;
struct pollfd p;
p.fd = fd;
p.events = POLLOUT | POLLRDBAND;
/* RDBAND is for what looks like a bug in illumos fifovnops.c */
p.revents = 0;
if ((n=poll(&p, 1, -1)) == 1) {
if (p.revents & POLLOUT) {
return fd;
}
if (p.revents & (POLLERR|POLLHUP)) {
return -1;
}
}
fprintf(stderr, "poll=%d (%d:%s), r=%#x\n",
n, errno, strerror(errno), p.revents);
return -1;
}
int main() {
int count = 0;
char c;
signal(SIGPIPE, sp);
while (read(0, &c, 1) > 0) {
int w;
while ((w=waitfd(1)) != -1 &&
write(1, &c, 1) != 1) {
}
if (w == -1) {
break;
}
count++;
}
fprintf(stderr, "wrote %d\n", count);
return 0;
}
In linux, you can run this program as: ./a.out < /dev/zero | sleep 1 and it will print something like: wrote 61441. You can change it to sleep for 3s, and it will print the same thing. That is pretty good evidence that it is has filled the pipe, and is waiting for space.
Sleep will never read from the pipe, so when its time is up, it closes the read side, which wakes up poll(2) with a POLLERR event.
If you change the poll event to not include POLLOUT, you get the simpler program:
#include <errno.h>
#include <fcntl.h>
#include <poll.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
int waitfd(int fd) {
int n;
struct pollfd p;
p.fd = fd;
p.events = POLLRDBAND;
p.revents = 0;
if ((n=poll(&p, 1, -1)) == 1) {
if (p.revents & (POLLERR|POLLHUP)) {
return -1;
}
}
fprintf(stderr, "poll=%d (%d:%s), r=%#x\n",
n, errno, strerror(errno), p.revents);
return -1;
}
int main() {
if (waitfd(1) == -1) {
fprintf(stderr, "Got an error!\n");
}
return 0;
}
where "Got an error!" indicates the pipe was closed. I don't know how portable this is, as poll(2) documentation is kinda sketchy.
Without the POLLRDBAND (so events is 0), this works on Linux, but wouldn't on UNIX (at least Solaris and macos). Again, docs were useless, but having the kernel source answers many questions :)
This example, using threads, can be directly mapped to go:
#include <pthread.h>
#include <errno.h>
#include <poll.h>
#include <signal.h>
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
int Events = POLLRDBAND;
void sp(int sno) {
char buf[64];
write(2, buf, snprintf(buf, sizeof buf, "%d: sig%s(%d)\n", getpid(), sys_siglist[sno], sno));
_exit(1);
}
int waitfd(int fd) {
int n;
struct pollfd p;
p.fd = fd;
p.events = Events;
/* RDBAND is for what looks like a bug in illumos fifovnops.c */
p.revents = 0;
if ((n=poll(&p, 1, -1)) == 1) {
if (p.revents & (POLLERR|POLLHUP)) {
return -1;
}
return fd;
}
return -1;
}
void *waitpipe(void *t) {
int x = (int)(intptr_t)t; /*gcc braindead*/
waitfd(x);
kill(getpid(), SIGUSR1);
return NULL;
}
int main(int ac) {
pthread_t killer;
int count = 0;
char c;
Events |= (ac > 1) ? POLLOUT : 0;
signal(SIGPIPE, sp);
signal(SIGUSR1, sp);
pthread_create(&killer, 0, waitpipe, (int *)1);
while (read(0, &c, 1) > 0) {
write(1, &c, 1);
count++;
}
fprintf(stderr, "wrote %d\n", count);
return 0;
}
Note that it parks a thread on poll, and it generates a SIGUSR1. Here is running it:
mcloud:pipe $ ./spthr < /dev/zero | hexdump -n80
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0000050
185965: sigUser defined signal 1(10)
mcloud:pipe $ ./spthr < /dev/zero | sleep 1
185969: sigUser defined signal 1(10)
mcloud:pipe $ ./spthr | sleep 1
185972: sigUser defined signal 1(10)
mcloud:pipe $ ./spthr < /dev/zero | hexdump -n800000
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
00c3500
185976: sigBroken pipe(13)
In the first command, hexdump quits after 80 bytes, the poll is fundamentally racing with the read+write loop, so it could have generated either a sigpipe or sigusr1.
The second two demonstrate that sleep will cause a sigusr1 (poll returned an exception event) whether or not the write side of the pipe is full when the pipe reader exits.
The fourth, uses hexdump to read a lot of data, way more than pipe capacity, which more deterministically causes a sigpipe.
You can generate test programs which model it more exactly, but the point is that the program gets notification as soon as the pipe is closed; not having to wait until its next write.

Answer 2

Not a real solution to the problem - namely, detecting if the process down the pipe has terminated without writing to it - but here is a workaround, suggested in a comment by Daniel Farrell: (Define and) use a heartbeat signal that will get ignored downstream.
As this workaround is not transparent, it may not be possible if you don't control all processes involved.
Here's an example that uses the NUL byte as heartbeat signal for text based data:
my-cmd | head -1 | tr -d '\000' > file
my-cmd would send NUL bytes in times of inactivity to get a timely EPIPE / SIGPIPE.
Note the use of tr to strip off the heartbeats again once it has served its purpose - otherwise they would end up in file.