Why does io.Pipe() continue to block even when EOF is reached? - go

Question

While playing with subprocesses and reading stdout through pipes I noticed interesting behaviour.
If I use an io.Pipe() to read the stdout of a subprocess created through os/exec, reading from that pipe hangs forever even when EOF is reached (the process is finished):
cmd := exec.Command("/bin/echo", "Hello, world!")
r, w := io.Pipe()
cmd.Stdout = w
cmd.Start()
io.Copy(os.Stdout, r) // Prints "Hello, World!" but never returns
However, if I use the built-in method StdoutPipe() it works:
cmd := exec.Command("/bin/echo", "Hello, world!")
p := cmd.StdoutPipe()
cmd.Start()
io.Copy(os.Stdout, p) // Prints "Hello, World!" and returns
Digging into the source code of /usr/lib/go/src/os/exec/exec.go, I can see that the StdoutPipe() method actually uses os.Pipe(), not io.Pipe():
pr, pw, err := os.Pipe()
cmd.Stdout = pw
cmd.closeAfterStart = append(c.closeAfterStart, pw)
cmd.closeAfterWait = append(c.closeAfterWait, pr)
return pr, nil
This gives me two clues:
File descriptors are being closed at certain points. Critically, the "write" end of the pipe is being closed after process start.
Instead of io.Pipe() as I used above, os.Pipe() (a lower level call that roughly maps to pipe(2) in POSIX) is used.
However I am still unable to understand why my original example behaves the way it does after taking into account this newfound knowledge.
If I try to close the write end of an io.Pipe() (instead of an os.Pipe()) then it appears to break it completely and nothing gets read (as if I'm reading from a closed pipe even though I thought I passed it to the subprocess):
cmd := exec.Command("/bin/echo", "Hello, world!")
r, w := io.Pipe()
cmd.Stdout = w
cmd.Start()
w.Close()
io.Copy(os.Stdout, r) // Prints nothing, no read buffer available
Okay, so I guess an io.Pipe() is quite different than an os.Pipe(), and probably doesn't behave like Unix pipes where one close() doesn't close it for everybody.
Just so you don't think I'm asking for a quick fix, I already know I can achieve my expected behaviour by using this code:
cmd := exec.Command("/bin/echo", "Hello, world!")
r, w, _ := os.Pipe() // using os.Pipe() instead of io.Pipe()
cmd.Stdout = w
cmd.Start()
w.Close()
io.Copy(os.Stdout, r) // Prints "Hello, World!" and returns on EOF. Works. :-)
What I'm asking for is why does io.Pipe() seem to ignore an EOF from the writer, leaving the reader blocking forever? A valid answer could be that io.Pipe() is the wrong tool for the job because $REASONS but I can't figure out what those $REASONS are because according to the documentation what I'm trying to do seems perfectly reasonable.
Here is a complete example to illustrate what I'm talking about:
package main
import (
"fmt"
"os"
"os/exec"
"io"
)
func main() {
cmd := exec.Command("/bin/echo", "Hello, world!")
r, w := io.Pipe()
cmd.Stdout = w
cmd.Start()
io.Copy(os.Stdout, r) // Blocks here even though EOF is reached
fmt.Println("Finished io.Copy()")
cmd.Wait()
}

Answer 1

"why does io.Pipe() seem to ignore an EOF from the writer, leaving the reader blocking forever?" Because there is no such thing as "EOF from the writer". All an EOF is (in unix) is an indication to the reader that no processes hold the write side of the pipe open. When a process attempts to read from a pipe which has no writers, the read system call returns a value that is conveniently named EOF. Since your parent still has one copy of the write side of the pipe open, read blocks. Stop thinking of EOF as a thing. It is merely an abstraction, and the writer never "sends" it.

Answer 2

You can use a goroutine:
package main
import (
"os"
"os/exec"
"io"
)
func main() {
r, w := io.Pipe()
c := exec.Command("go", "version")
c.Stdout = w
c.Start()
go func() {
io.Copy(os.Stdout, r)
r.Close()
}()
c.Wait()
}