given the following example:
// test.go
package main
import (
"fmt"
"os/exec"
)
func main() {
cmd := exec.Command("login")
in, _ := cmd.StdinPipe()
in.Write([]byte("user"))
out, err := cmd.CombinedOutput()
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%s", out)
}
How can I detect that the process is not going to finish, because it is waiting for user input?
I'm trying to be able to run any script, but abort it if for some reason it tries to read from stdin.
Thanks!
Detecting that the process is not going to finish is a difficult problem. In fact, it is one of the classic "unsolvable" problems in Computer Science: the Halting Problem.
In general, when you are calling exec.Command and will not be passing it any input, it will cause the program to read from your OS's null device (see documentation in the exec.Cmd fields). In your code (and mine below), you explicitly create a pipe (though you should check the error return of StdinPipe in case it is not created correctly), so you should subsequently call in.Close(). In either case, the subprocess will get an EOF and should clean up after itself and exit.
To help with processes that don't handle input correctly or otherwise get themselves stuck, the general solution is to use a timeout. In Go, you can use goroutines for this:
// Set your timeout
const CommandTimeout = 5 * time.Second
func main() {
cmd := exec.Command("login")
// Set up the input
in, err := cmd.StdinPipe()
if err != nil {
log.Fatalf("failed to create pipe for STDIN: %s", err)
}
// Write the input and close
go func() {
defer in.Close()
fmt.Fprintln(in, "user")
}()
// Capture the output
var b bytes.Buffer
cmd.Stdout, cmd.Stderr = &b, &b
// Start the process
if err := cmd.Start(); err != nil {
log.Fatalf("failed to start command: %s", err)
}
// Kill the process if it doesn't exit in time
defer time.AfterFunc(CommandTimeout, func() {
log.Printf("command timed out")
cmd.Process.Kill()
}).Stop()
// Wait for the process to finish
if err := cmd.Wait(); err != nil {
log.Fatalf("command failed: %s", err)
}
// Print out the output
fmt.Printf("Output:\n%s", b.String())
}
In the code above, there are actually three main goroutines of interest: the main goroutine spawns the subprocess and waits for it to exit; a timer goroutine is sent off in the background to kill the process if it's not Stopped in time; and a goroutine that writes the output to the program when it's ready to read it.
Although this would not allow you to "detect" the program trying to read from stdin, I would just close stdin. This way, the child process will just receive an EOF when it tried to read. Most programs know how to handle a closed stdin.
// All error handling excluded
cmd := exec.Command("login")
in, _ := cmd.StdinPipe()
cmd.Start()
in.Close()
cmd.Wait()
Unfortunately, this means you can't use combined output, the following code should allow you to do the same thing. It requires you to import the bytes package.
var buf = new(bytes.Buffer)
cmd.Stdout = buf
cmd.Stderr = buf
After cmd.Wait(), you can then do:
out := buf.Bytes()
I think the solution is to run the child process with closed stdin - by adjusting the Cmd.Stdin appropriately and then Runinng it afterwards instead of using CombinedOutput().
Finally, I'm going to implement a combination of Kyle Lemons answer and forcing the new process have it's own session without a terminal attached to it, so that the executed comand will be aware that there is no terminal to read from.
// test.go
package main
import (
"log"
"os/exec"
"syscall"
)
func main() {
cmd := exec.Command("./test.sh")
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
out, err := cmd.CombinedOutput()
if err != nil {
log.Fatal("error:", err)
}
log.Printf("%s", out)
}
Related
I would like to manage a process in Go with the package os/exec. I would like to start it and be able to read the output and write several times to the input.
The process I launch in the code below, menu.py, is just a python script that does an echo of what it has in input.
func ReadOutput(rc io.ReadCloser) (string, error) {
x, err := ioutil.ReadAll(rc)
s := string(x)
return s, err
}
func main() {
cmd := exec.Command("python", "menu.py")
stdout, err := cmd.StdoutPipe()
Check(err)
stdin, err := cmd.StdinPipe()
Check(err)
err = cmd.Start()
Check(err)
go func() {
defer stdin.Close() // If I don't close the stdin pipe, the python code will never take what I write in it
io.WriteString(stdin, "blub")
}()
s, err := ReadOutput(stdout)
if err != nil {
Log("Process is finished ..")
}
Log(s)
// STDIN IS CLOSED, I CAN'T RETRY !
}
And the simple code of menu.py :
while 1 == 1:
name = raw_input("")
print "Hello, %s. \n" % name
The Go code works, but if I don't close the stdin pipe after I write in it, the python code never take what is in it. It is okay if I want to send only one thing in the input on time, but what is I want to send something again few seconds later? Pipe is closed! How should I do? The question could be "How do I flush a pipe from WriteCloser interface?" I suppose
I think the primary problem here is that the python process doesn't work the way you might expect. Here's a bash script echo.sh that does the same thing:
#!/bin/bash
while read INPUT
do echo "Hello, $INPUT."
done
Calling this script from a modified version of your code doesn't have the same issue with needing to close stdin:
func ReadOutput(output chan string, rc io.ReadCloser) {
r := bufio.NewReader(rc)
for {
x, _ := r.ReadString('\n')
output <- string(x)
}
}
func main() {
cmd := exec.Command("bash", "echo.sh")
stdout, err := cmd.StdoutPipe()
Check(err)
stdin, err := cmd.StdinPipe()
Check(err)
err = cmd.Start()
Check(err)
go func() {
io.WriteString(stdin, "blab\n")
io.WriteString(stdin, "blob\n")
io.WriteString(stdin, "booo\n")
}()
output := make(chan string)
defer close(output)
go ReadOutput(output, stdout)
for o := range output {
Log(o)
}
}
The Go code needed a few minor changes - ReadOutput method needed to be modified in order to not block - ioutil.ReadAll would have waited for an EOF before returning.
Digging a little deeper, it looks like the real problem is the behaviour of raw_input - it doesn't flush stdout as expected. You can pass the -u flag to python to get the desired behaviour:
cmd := exec.Command("python", "-u", "menu.py")
or update your python code to use sys.stdin.readline() instead of raw_input() (see this related bug report: https://bugs.python.org/issue526382).
Even though there is some problem with your python script. The main problem is the golang pipe. A trick to solve this problem is use two pipes as following:
// parentprocess.go
package main
import (
"bufio"
"log"
"io"
"os/exec"
)
func request(r *bufio.Reader, w io.Writer, str string) string {
w.Write([]byte(str))
w.Write([]byte("\n"))
str, err := r.ReadString('\n')
if err != nil {
panic(err)
}
return str[:len(str)-1]
}
func main() {
cmd := exec.Command("bash", "menu.sh")
inr, inw := io.Pipe()
outr, outw := io.Pipe()
cmd.Stdin = inr
cmd.Stdout = outw
if err := cmd.Start(); err != nil {
panic(err)
}
go cmd.Wait()
reader := bufio.NewReader(outr)
log.Printf(request(reader, inw, "Tom"))
log.Printf(request(reader, inw, "Rose"))
}
The subprocess code is the same logic as your python code as following:
#!/usr/bin/env bash
# menu.sh
while true; do
read -r name
echo "Hello, $name."
done
If you want to use your python code you should do some changes:
while 1 == 1:
try:
name = raw_input("")
print "Hello, %s. \n" % name
sys.stdout.flush() # there's a stdout buffer
except:
pass # make sure this process won't die when come across 'EOF'
// StdinPipe returns a pipe that will be connected to the command's
// standard input when the command starts.
// The pipe will be closed automatically after Wait sees the command exit.
// A caller need only call Close to force the pipe to close sooner.
// For example, if the command being run will not exit until standard input`enter code here`
// is closed, the caller must close the pipe.
func (c *Cmd) StdinPipe() (io.WriteCloser, error) {}
I have to process a long output of a script and find some data. This data most likely will be located at the almost very beginning of the output. After data found I do not need to process output anymore and can quit.
The issue that I cannot stop processing output because exec.Cmd does not have any function to close opened command.
Here are some simplified code (error handling was ommited):
func processOutput(r *bufio.Reader)(){
for {
line, _, err := r.ReadLine()
if some_condition_meet {
break
}
......
}
return
}
func execExternalCommand(){
cmdToExecute := "......"
cmd := exec.Command("bash", "-c", cmdToExecute)
output, _ := cmd.StdoutPipe()
cmd.Start()
r := bufio.NewReader(output)
go processOutput(r)
cmd.Wait()
return
}
What should I do at the end at processOutput function to stop cmd? Maybe there is another way how to solve it.
Thanks
As it stands, you can't do this from processOutput because all it receives is a bufio.Reader. You would need to pass the exec.Cmd to it for it to do anything with the forked process.
To kill the forked process, you can send it a signal, e.g.: cmd.Process.Kill() or cmd.Process.Signal(os.SIGINT). See documentation on exec.Cmd and os.Process.
You could probably use "context" for example:
package main
import (
"bufio"
"context"
"os/exec"
)
func processOutput(r *bufio.Reader, cancel context.WithCancel) {
for {
line, _, err := r.ReadLine()
if some_condition_meet {
break
}
}
cancel()
return
}
func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
cmdToExecute := "......"
cmd := exec.CommandContext(ctx, "bash", "-c", cmdToExecute)
output, _ := cmd.StdoutPipe()
cmd.Start()
r := bufio.NewReader(output)
go processOutput(r, cancel)
cmd.Wait()
return
}
In case need to end with a timeout this could work (example is taken from here: https://golang.org/src/os/exec/example_test.go)
func ExampleCommandContext() {
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
if err := exec.CommandContext(ctx, "sleep", "5").Run(); err != nil {
// This will fail after 100 milliseconds. The 5 second sleep
// will be interrupted.
}
}
A basic example just using sleep but closing it after 1 second: https://play.golang.org/p/gIXKuf5Oga
I'm trying to use Go's os/exec Command() to simulate a keypress, and sometimes I'll want to use this keypress multiple times in quick succession. I'm using exec.Command to call "xte", "key XF86AudioPlay", which pauses music on a Linux OS. While the Command can Start() or Run() no problem, if I try to execute again, I get an error:
exec: already started
I've tried using Process.Kill() immediately after the execution, in order to free it up, but this will make the execution not work in the first place. I got this idea from here: Terminating a Process Started with os/exec in Golang
My code uses a switch and calls this pause function accordingly, but I'll simply share the basis of the code I wrote, with the case as an example function:
cmd := exec.Command("xte", "key XF86AudioPlay")
//...
func Pause() {
err := cmd.Start() // or cmd.Run()
if err != nil {
fmt.Println(err)
}
err = cmd.Process.Kill()
if err != nil {
fmt.Printf("Failed to kill: %s", err)
}
}
So, to recap, I'm successful at calling it one time, but upon success calls to Pause(), I get the error from cmd.Start()/Run() which read: exec: already started.
I also tried going lower, that is, using syscall, but I ran into some trouble. I tried:
args[0] = "xte"
args[1] = "key"
args[2] = "XF86AudioPlay"
//... Pause():
err := syscall.Exec("/bin", args, os.Environ())
if err != nil {
fmt.Println(err)
}
And here I got a permission denied error, even running as super user (sudo).
How should I proceed call this Command() and then free it up for immediate recall? Or am I on the right track with syscall?
Edit
So the solution as both Amd and Son Bui state, was to create the Command everytime I intend to call it, basically putting the assignment cmd := exec.Command()inside my Pause() method.
As the type Cmd struct { Docs said:
A Cmd cannot be reused after calling its Run, Output or CombinedOutput
methods.
1- Use two separate exec.Command like this,
Also you may need some delay between runs (simulating two separate keystrokes):
package main
import (
"fmt"
"os"
"os/exec"
"time"
)
func main() {
Pause()
fmt.Println("Once")
time.Sleep(100 * time.Millisecond)
Pause()
fmt.Println("Twice")
}
func Pause() {
cmd := exec.Command("xte", "key XF86AudioPlay")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
err := cmd.Run()
if err != nil {
fmt.Println(err)
}
}
2- You may run it once:
You may use something like this (not tested):
xte 'key XF86AudioPlay' 'key XF86AudioPlay'
and consider adding a short delay to the xte command (simulating two separate keystrokes):
xte 'key XF86AudioPlay' 'usleep 100000' 'key XF86AudioPlay'
Like this:
package main
import (
"fmt"
"os"
"os/exec"
)
func main() {
Pause()
fmt.Println("Once")
}
func Pause() {
cmd := exec.Command("xte", `key XF86AudioPlay`, `usleep 100000`, `key XF86AudioPlay`)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
err := cmd.Run()
if err != nil {
fmt.Println(err)
}
}
See:
https://askubuntu.com/questions/499926/why-do-these-xte-commands-work-in-terminal-but-not-when-bound-with-xbindkeys
http://manpages.ubuntu.com/manpages/wily/man1/xte.1.html
http://wiki.robotz.com/index.php/Linux_Tools_to_Remap_Keys_and_Mouse_Buttons
I hope this helps.
See from source code:
cmd struct: (https://golang.org/src/os/exec/exec.go line 99)
// Process is the underlying process, once started.
Process *os.Process
And in Start function (https://golang.org/src/os/exec/exec.go line 327)
:
if c.Process != nil {
return errors.New("exec: already started")
}
So you can only use cmd.Start once time. If you want to use multiple time, you can create new Cmd OR run multiple command in once, ex:
cmd := exec.Command("/bin/sh", "-c", "command1; command2; command3; ...")
I have this code
subProcess := exec.Cmd{
Path: execAble,
Args: []string{
fmt.Sprintf("-config=%s", *configPath),
fmt.Sprintf("-serverType=%s", *serverType),
fmt.Sprintf("-reload=%t", *reload),
fmt.Sprintf("-listenFD=%d", fd),
},
Dir: here,
}
subProcess.Stdout = os.Stdout
subProcess.Stderr = os.Stderr
logger.Info("starting subProcess:%s ", subProcess.Args)
if err := subProcess.Run(); err != nil {
logger.Fatal(err)
}
and then I do os.Exit(1) to stop the main process
I can get output from the subprocess
but I also want to put stdin to
I try
subProcess.Stdin = os.Stdin
but it does not work
I made a simple program (for testing). It reads a number and writes the given number out.
package main
import (
"fmt"
)
func main() {
fmt.Println("Hello, What's your favorite number?")
var i int
fmt.Scanf("%d\n", &i)
fmt.Println("Ah I like ", i, " too.")
}
And here is the modified code
package main
import (
"fmt"
"io"
"os"
"os/exec"
)
func main() {
subProcess := exec.Command("go", "run", "./helper/main.go") //Just for testing, replace with your subProcess
stdin, err := subProcess.StdinPipe()
if err != nil {
fmt.Println(err) //replace with logger, or anything you want
}
defer stdin.Close() // the doc says subProcess.Wait will close it, but I'm not sure, so I kept this line
subProcess.Stdout = os.Stdout
subProcess.Stderr = os.Stderr
fmt.Println("START") //for debug
if err = subProcess.Start(); err != nil { //Use start, not run
fmt.Println("An error occured: ", err) //replace with logger, or anything you want
}
io.WriteString(stdin, "4\n")
subProcess.Wait()
fmt.Println("END") //for debug
}
You interested about these lines
stdin, err := subProcess.StdinPipe()
if err != nil {
fmt.Println(err)
}
defer stdin.Close()
//...
io.WriteString(stdin, "4\n")
//...
subProcess.Wait()
Explanation of the above lines
We gain the subprocess' stdin, now we can write to it
We use our power and we write a number
We wait for our subprocess to complete
Output
START
Hello, What's your favorite number?
Ah I like 4 too.
END
For better understanding
There's now an updated example available in the Go docs: https://golang.org/pkg/os/exec/#Cmd.StdinPipe
If the subprocess doesn't continue before the stdin is closed, the io.WriteString() call needs to be wrapped inside an anonymous function:
func main() {
cmd := exec.Command("cat")
stdin, err := cmd.StdinPipe()
if err != nil {
log.Fatal(err)
}
go func() {
defer stdin.Close()
io.WriteString(stdin, "values written to stdin are passed to cmd's standard input")
}()
out, err := cmd.CombinedOutput()
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s\n", out)
}
Though this question is a little old, but here is my answer:
This question is of course very platform specific as how standard IO is handled depends on the OS implementation and not on Go language. However, as general rule of thumb (due to some OSes being prevailing), "what you ask is not possible".
On most of modern operating systems you can pipe standard streams (as in #mraron's answer), you can detach them (this is how daemons work), but you cannot reassign or delegate them to another process.
I think this limitation is more because of security concern. There are still from time to time bugs being discovered that allow remote code execution, imagine if OS was allowing to reassign/delegate STDIN/OUT, then with such vulnerabilities the consequences would be disastrous.
While you cannot directly do this as #AlexKey wrote earlier still you can make some workarounds. If os prevents you to pipe your own standard streams who cares all you need 2 channels and 2 goroutines
var stdinChan chan []byte
var stdoutChan chan []byte
//when something happens in stdout of your code just pipe it to stdout chan
stdoutChan<-somehowGotDataFromStdOut
then you need two funcs as i mentioned before
func Pipein(){
for {
stdinFromProg.Write(<-stdInChan)
}
}
The same idea for the stdout
I'm writing a program in Go that executes a server like program (also Go). Now I want to have the stdout of the child program in my terminal window where I started the parent program. One way to do this is with the cmd.Output() function, but this prints the stdout only after the process has exited. (That's a problem because this server-like program runs for a long time and I want to read the log output)
The variable out is of type io.ReadCloser and I don't know what I should do with it to achieve my task, and I can't find anything helpful on the web on this topic.
func main() {
cmd := exec.Command("/path/to/my/child/program")
out, err := cmd.StdoutPipe()
if err != nil {
fmt.Println(err)
}
err = cmd.Start()
if err != nil {
fmt.Println(err)
}
//fmt.Println(out)
cmd.Wait()
}
Explanation to the code: uncomment the Println function to get the code to compile, I know that Println(out io.ReadCloser) is not a meaningful function.
(it produces the output &{3 |0 <nil> 0} ) These two lines are just required to get the code to compile.
Now I want to have the stdout of the child program in my terminal
window where I started the parent program.
No need to mess with pipes or goroutines, this one is easy.
func main() {
// Replace `ls` (and its arguments) with something more interesting
cmd := exec.Command("ls", "-l")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Run()
}
I believe that if you import io and os and replace this:
//fmt.Println(out)
with this:
go io.Copy(os.Stdout, out)
(see documentation for io.Copy and for os.Stdout), it will do what you want. (Disclaimer: not tested.)
By the way, you'll probably want to capture standard-error as well, by using the same approach as for standard-output, but with cmd.StderrPipe and os.Stderr.
For those who don't need this in a loop, but would like the command output to echo into the terminal without having cmd.Wait() blocking other statements:
package main
import (
"fmt"
"io"
"log"
"os"
"os/exec"
)
func checkError(err error) {
if err != nil {
log.Fatalf("Error: %s", err)
}
}
func main() {
// Replace `ls` (and its arguments) with something more interesting
cmd := exec.Command("ls", "-l")
// Create stdout, stderr streams of type io.Reader
stdout, err := cmd.StdoutPipe()
checkError(err)
stderr, err := cmd.StderrPipe()
checkError(err)
// Start command
err = cmd.Start()
checkError(err)
// Don't let main() exit before our command has finished running
defer cmd.Wait() // Doesn't block
// Non-blockingly echo command output to terminal
go io.Copy(os.Stdout, stdout)
go io.Copy(os.Stderr, stderr)
// I love Go's trivial concurrency :-D
fmt.Printf("Do other stuff here! No need to wait.\n\n")
}