Hi guys for some reason Wait() when I execute mysql command hangs for ever, does anyone know why?
Here is my code.
// Import imports data into Local database
func (x MySQL) Import(data string, opt LocalDb) {
var stderr bytes.Buffer
cmd := exec.Command("mysql", x.importOptions(opt)...)
// Set < pipe variable
stdin, err := cmd.StdinPipe()
errChk(err)
cmd.Stderr = &stderr
cmd.Start()
// Write data to pipe
io.WriteString(stdin, data)
fmt.Println("Importing " + x.DB + " to localhost...")
// Log mysql error
if err := cmd.Wait(); err != nil {
log.Fatal(stderr.String())
} else {
fmt.Println("Importing complete")
}
}
This function accomplishes everything and mysql imports the data into database but it never return from Wait() just freezes there even though is completed.
The problem is that you haven't closed the stdin pipe. MySQL will remain active until it is.
The fix is thus simple:
// Write data to pipe
io.WriteString(stdin, data)
stdin.Close()
fmt.Println("Importing " + x.DB + " to localhost...")
The fact that StdinPipe() acts in this way is documented as such:
StdinPipe returns a pipe that will be connected to the command's standard input when the command starts. The pipe will be closed automatically after Wait sees the command exit. A caller need only call Close to force the pipe to close sooner. For example, if the command being run will not exit until standard input is closed, the caller must close the pipe.
Related
I have a program which makes an ssh connection to a new (every time the program is executed) gcp instance to retrieve information. The problem is that sometimes I got this error and I don't know why:
2019/08/22 12:30:37 ssh: Stdout already set
My code(avoiding error handle):
results := /home/example.txt
client, err := ssh.Dial("tcp", addrIP+":22", clientConfig)
session, err := client.NewSession()
defer session.Close()
data, err := session.Output(" cat " + results)
if err != nil {
log.Print("Fails when new output")
log.Fatal(err)
}
During the output is when the error occurs.
Calling session.Output would set the Stdout of the session to a buffer, then run the command provided, and return the contents in the buffer.
If the Stdout of this session is already set (for example, if you call the session.Output multiple times), an error of "Stdout already set" will be returned.
If you need to run multiple commands in one session, just manually set the Stdout to some buffer maintained by yourself, and use the session.Run() method instead of session.Output.
This question already has an answer here:
How can I redirect the stdout and stderr of a command to both the console and a log file while outputting in real time?
(1 answer)
Closed 5 years ago.
I have a small Go tool which basically allows the user to define an command that than will be run using os/exec.
My problem is that I want to show the user the output (stdout/stderr) of the command.
An example could look like this:
The user defines a command that in the end is sh test.sh.
Content of test.sh:
echo "Start"
sleep 7s
echo "Done"
With my current implementation the user can only see the output once the complete command finished. In the example above the user wouldn't see the output Start until the sleep command and the second echo finish.
I currently retrieve the output of the command like this:
cmd := exec.Command(command, args...)
cmd.Dir = dir
// Attach to the standard out to read what the command might print
stdout, err := cmd.StdoutPipe()
if err != nil {
log.Panic(err)
}
// Execute the command
if err := cmd.Start(); err != nil {
log.Panic(err)
}
buf := new(bytes.Buffer)
buf.ReadFrom(stdout)
log.Print(buf.String())
Is it somehow possible to read the stdout/stderr in real-time. Meaning that as soon as the user defined command creates and output it is printed?
Thank you mh-cbon. That pushed me in the right direction.
The code now looks like this and does exactly what I want it to do. I also found that when I use Run() instead of Start() the execution of the program only continues after the command has finished.
cmd := exec.Command(command, args...)
cmd.Dir = dir
var stdBuffer bytes.Buffer
mw := io.MultiWriter(os.Stdout, &stdBuffer)
cmd.Stdout = mw
cmd.Stderr = mw
// Execute the command
if err := cmd.Run(); err != nil {
log.Panic(err)
}
log.Println(stdBuffer.String())
I'm writing a program which opens a named pipe for reading, and then processes any lines written to this pipe:
err = syscall.Mkfifo("/tmp/myfifo", 0666)
if err != nil {
panic(err)
}
pipe, err := os.OpenFile("/tmp/myfifo", os.O_RDONLY, os.ModeNamedPipe)
if err != nil {
panic(err)
}
reader := bufio.NewReader(pipe)
scanner := bufio.NewScanner(reader)
for scanner.Scan() {
line := scanner.Text()
process(line)
}
This works fine as long as the writing process does not restart or for other reasons send an EOF. When this happens, the loop terminates (as expected from the specifications of Scanner).
However, I want to keep the pipe open to accept further writes. I could just reinitialize the scanner of course, but I believe this would create a race condition where the scanner might not be ready while a new process has begun writing to the pipe.
Are there any other options? Do I need to work directly with the File type instead?
From the bufio GoDoc:
Scan ... returns false when the scan stops, either by reaching the end of the input or an error.
So you could possibly leave the file open and read until EOF, then trigger scanner.Scan() again when the file has changed or at a regular interval (i.e. make a goroutine), and make sure the pipe variable doesn't go out of scope so you can reference it again.
If I understand your concern about a race condition correctly, this wouldn't be an issue (unless write and read operations must be synchronized) but when the scanner is re-initialized it will end up back at the beginning of the file.
I execute process with Go and write output to file (log file)
cmd := exec.Command(path)
cmd.Dir = dir
t := time.Now()
t1 := t.Format("20060102-150405")
fs, err := os.Create(dir + "/var/log/" + t1 + ".std")
if err == nil {
cmd.Stdout = fs
}
I wish to rotate logs and change log file daily
http://golang.org/pkg/os/exec/
// Stdout and Stderr specify the process's standard output and error.
//
// If either is nil, Run connects the corresponding file descriptor
// to the null device (os.DevNull).
//
// If Stdout and Stderr are the same writer, at most one
// goroutine at a time will call Write.
Stdout io.Writer
Stderr io.Writer
Is it safe to change cmd.Stdout variable daily from arbitary goroutine or I have to implement goroutine that will copy from Stdout to another file and switch files?
It is safe to change those variables directly. However, if you change them once the command has actually been run then they will have no effect on the actual running child process. To rotate the output of the running process "live" you will have to implement that in the process itself, or pipe everything through the parent and use a goroutine as you suggest.
given the following example:
// test.go
package main
import (
"fmt"
"os/exec"
)
func main() {
cmd := exec.Command("login")
in, _ := cmd.StdinPipe()
in.Write([]byte("user"))
out, err := cmd.CombinedOutput()
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%s", out)
}
How can I detect that the process is not going to finish, because it is waiting for user input?
I'm trying to be able to run any script, but abort it if for some reason it tries to read from stdin.
Thanks!
Detecting that the process is not going to finish is a difficult problem. In fact, it is one of the classic "unsolvable" problems in Computer Science: the Halting Problem.
In general, when you are calling exec.Command and will not be passing it any input, it will cause the program to read from your OS's null device (see documentation in the exec.Cmd fields). In your code (and mine below), you explicitly create a pipe (though you should check the error return of StdinPipe in case it is not created correctly), so you should subsequently call in.Close(). In either case, the subprocess will get an EOF and should clean up after itself and exit.
To help with processes that don't handle input correctly or otherwise get themselves stuck, the general solution is to use a timeout. In Go, you can use goroutines for this:
// Set your timeout
const CommandTimeout = 5 * time.Second
func main() {
cmd := exec.Command("login")
// Set up the input
in, err := cmd.StdinPipe()
if err != nil {
log.Fatalf("failed to create pipe for STDIN: %s", err)
}
// Write the input and close
go func() {
defer in.Close()
fmt.Fprintln(in, "user")
}()
// Capture the output
var b bytes.Buffer
cmd.Stdout, cmd.Stderr = &b, &b
// Start the process
if err := cmd.Start(); err != nil {
log.Fatalf("failed to start command: %s", err)
}
// Kill the process if it doesn't exit in time
defer time.AfterFunc(CommandTimeout, func() {
log.Printf("command timed out")
cmd.Process.Kill()
}).Stop()
// Wait for the process to finish
if err := cmd.Wait(); err != nil {
log.Fatalf("command failed: %s", err)
}
// Print out the output
fmt.Printf("Output:\n%s", b.String())
}
In the code above, there are actually three main goroutines of interest: the main goroutine spawns the subprocess and waits for it to exit; a timer goroutine is sent off in the background to kill the process if it's not Stopped in time; and a goroutine that writes the output to the program when it's ready to read it.
Although this would not allow you to "detect" the program trying to read from stdin, I would just close stdin. This way, the child process will just receive an EOF when it tried to read. Most programs know how to handle a closed stdin.
// All error handling excluded
cmd := exec.Command("login")
in, _ := cmd.StdinPipe()
cmd.Start()
in.Close()
cmd.Wait()
Unfortunately, this means you can't use combined output, the following code should allow you to do the same thing. It requires you to import the bytes package.
var buf = new(bytes.Buffer)
cmd.Stdout = buf
cmd.Stderr = buf
After cmd.Wait(), you can then do:
out := buf.Bytes()
I think the solution is to run the child process with closed stdin - by adjusting the Cmd.Stdin appropriately and then Runinng it afterwards instead of using CombinedOutput().
Finally, I'm going to implement a combination of Kyle Lemons answer and forcing the new process have it's own session without a terminal attached to it, so that the executed comand will be aware that there is no terminal to read from.
// test.go
package main
import (
"log"
"os/exec"
"syscall"
)
func main() {
cmd := exec.Command("./test.sh")
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
out, err := cmd.CombinedOutput()
if err != nil {
log.Fatal("error:", err)
}
log.Printf("%s", out)
}