exec.Wait() with a modified Stdin waits indefinitely - go

Question

I encounter a weird behavior with exec.Wait() with a modified Stdin.
I'm just modifying Stdin in order to be able to duplicate its content, count the amount of data… but that's not the problem here.
I've made this stripped down program just to demonstrate the strange behavior :
with a modified Stdin, cmd.Wait() waits indefinitely… until I press "enter" or "^C"
with an unmodified Stdin (uncomment the line cmd.Stdin = os.Stdin), the program processes to the end flawlessly.
when I launch this program (with a modified Stdin) with Delve (dlv debug), the program processes to the end flawlessly !
I also added a time.Sleep of 30 seconds between cmd.Start() and cmd.Wait(), and then attached the program to Delve (dlv attach PID). When I enter continue, cmd.Wait() waits indefinitely… until I press "enter" or "^C"
I tested these behaviours with go1.11 and go1.12
package main
import (
"fmt"
"os"
"os/exec"
)
type Splitter struct {
f *os.File
fd int
}
func NewSplitter(f *os.File) *Splitter {
return &Splitter{f, int(f.Fd())}
}
func (s *Splitter) Close() error {
return s.f.Close()
}
func (s *Splitter) Read(p []byte) (int, error) {
return s.f.Read(p)
}
func (s *Splitter) Write(p []byte) (int, error) {
return s.f.Write(p)
}
func main() {
var cmd *exec.Cmd
cmd = exec.Command("cat", "foobarfile")
cmd.Stdin = NewSplitter(os.Stdin)
//cmd.Stdin = os.Stdin
cmd.Stdout = NewSplitter(os.Stdout)
cmd.Stderr = NewSplitter(os.Stderr)
cmd.Start()
cmd.Wait()
fmt.Println("done")
}
Is there something I'm doing wrong ?
Thanks for your help.

Answer 1

This program duplicates the content as you asked. You can although try the commented part as well. And the comments are self - explanatory, I hope it explains your query.
package main
import (
"io"
"log"
"os"
"os/exec"
)
func main() {
// Execute cat command w/ arguments
// cmd := exec.Command("cat", "hello.txt")
// Execute cat command w/o arguments
cmd := exec.Command("cat")
// Attach STDOUT stream
stdout, err := cmd.StdoutPipe()
if err != nil {
log.Println(err)
}
// Attach STDIN stream
stdin, err := cmd.StdinPipe()
if err != nil {
log.Println(err)
}
// Attach STDERR stream
stderr, err := cmd.StderrPipe()
if err != nil {
log.Println(err)
}
// Spawn go-routine to copy os's stdin to command's stdin
go io.Copy(stdin, os.Stdin)
// Spawn go-routine to copy command's stdout to os's stdout
go io.Copy(os.Stdout, stdout)
// Spawn go-routine to copy command's stderr to os's stderr
go io.Copy(os.Stderr, stderr)
// Run() under the hood calls Start() and Wait()
cmd.Run()
// Note: The PIPES above will be closed automatically after Wait sees the command exit.
// A caller need only call Close to force the pipe to close sooner.
log.Println("Command complete")
}

Answer 2

You are replacing the process file descriptors, which are normally *os.File, with other Go types. In order for stdin to act like a stream, the os/exec package needs to launch a goroutine to copy the data between the io.Reader and the process. This is documented in the os/exec package:
// Otherwise, during the execution of the command a separate
// goroutine reads from Stdin and delivers that data to the command
// over a pipe. In this case, Wait does not complete until the goroutine
// stops copying, either because it has reached the end of Stdin
// (EOF or a read error) or because writing to the pipe returned an error.
If you look at the stack trace from your program, you'll see that it is waiting for the io goroutines to complete in Wait():
goroutine 1 [chan receive]:
os/exec.(*Cmd).Wait(0xc000076000, 0x0, 0x0)
/usr/local/go/src/os/exec/exec.go:510 +0x125
main.main()
Because you are now in control of the data stream, it is up to you to close it as necessary. If Stdin is not necessary here, then don't assign it at all. If it is going to be used, then you must Close() it to have Wait() return.
Another option is to ensure that you are using an *os.File, which the easiest method is to use the StdinPipe, StdoutPipe and StderrPipe methods, which in turn use os.Pipe(). This way ensures that the process is dealing only with *os.File, and not with other Go types.