I'm using Go's syscall package Ptrace interface to trace a process. The problem is, if the tracee is long-running, the tracing seems to hang. I tried replicating the issue with C implementation, but there everything seems to work fine.
Here's a Go code to reproduce the issue:
import (
"fmt"
"os"
"os/exec"
"syscall"
)
func main() {
len := "9999999"
cmd := exec.Command("openssl", "rand", "-hex", len)
cmd.SysProcAttr = &syscall.SysProcAttr{Ptrace: true}
cmd.Stdout = os.Stdout
cmd.Stdin = os.Stdin
cmd.Start()
pid, _ := syscall.Wait4(-1, nil, syscall.WALL, nil)
for {
syscall.PtraceSyscall(pid, 0)
_, err := syscall.Wait4(-1, nil, syscall.WALL, nil)
if err != nil {
fmt.Println(err)
break
}
}
}
When running the above code, the process never completes and it has to be interrupted. If the len variable is changed to something smaller, for example 9, the process will complete without issues and output will be something following:
$ go run main.go
d2ff963e65e8e1926b
no child processes
Found it. The program hangs when Go runtime changes the thread in which the goroutine is running. Can be verified in the example code by printing fmt.Println(syscall.Gettid()) inside the loop:
package main
import (
"fmt"
"os/exec"
"syscall"
)
func main() {
len := "9999999"
cmd := exec.Command("openssl", "rand", "-hex", len)
cmd.SysProcAttr = &syscall.SysProcAttr{Ptrace: true}
cmd.Start()
pid, _ := syscall.Wait4(-1, nil, syscall.WALL, nil)
for {
fmt.Println(syscall.Gettid())
syscall.PtraceSyscall(pid, 0)
_, err := syscall.Wait4(-1, nil, syscall.WALL, nil)
if err != nil {
fmt.Println(err)
break
}
}
}
Solution: lock the execution of the goroutine to its current thread by using runtime.LockOSThread():
....
func main() {
runtime.LockOSThread()
len := "9999999"
cmd := exec.Command("openssl", "rand", "-hex", len)
....
Related
I am trying to run a simple program that spawns a vim process.
The user should be able (when the exec.Command starts) to switch to vim window and the process execution should halt there.
When user closes vim (wq!) the program execution should resume from that point.
The following simple attempt fails but I cannot figure out why
package main
import (
"log"
"os/exec"
)
func main() {
cmd := exec.Command("vim", "lala")
err := cmd.Run()
if err != nil {
log.Fatal(err)
}
}
▶ go run main.go
2022/11/25 09:16:44 exit status 1
exit status 1
Why the exit status 1?
You missed these two lines:
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
Thanks to these two lines the user is able to edit with vim the file in the terminal. The control is returned to the program when the user quit from the terminal (e.g., with the command :wq). Below, you can find the whole code:
package main
import (
"log"
"os"
"os/exec"
)
func main() {
cmd := exec.Command("vim", "lala")
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
err := cmd.Run()
if err != nil {
log.Fatal(err)
}
}
Hope this helps!
Because you should set Stdin and Stdoutfor cmd:
package main
import (
"log"
"os"
"os/exec"
)
func main() {
cmd := exec.Command("vim", "lala")
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
err := cmd.Run()
if err != nil {
log.Fatal(err)
}
}
I am building a terminal emulator in golang and I was trying to run detached processes from which I can copy output and display it to the user but the io.Copy function blocks and hence I cannot continue to the output part
I looked in the source code and it blocks in the internal ReadFrom method, I cannot understand why this is happening
package main
import (
"bytes"
"fmt"
"io"
"os"
)
func main() {
inputReader, inputWriter, _ := os.Pipe()
outputReader, outputWriter, _ := os.Pipe()
io.Copy(inputWriter, bytes.NewReader([]byte("\n")))
stdin := inputReader
stdout := outputWriter
stderr := outputWriter
var attr = os.ProcAttr{
Dir: "/tmp",
Env: nil,
Files: []*os.File{
stdin,
stdout,
stderr,
},
Sys: nil,
}
process, startProcessErr := os.StartProcess("/usr/bin/ls", []string{"ls"}, &attr)
if startProcessErr != nil {
panic(startProcessErr)
}
if releaseProcessErr := process.Release(); releaseProcessErr != nil {
panic(releaseProcessErr)
}
var output bytes.Buffer
io.Copy(&output, outputReader)
fmt.Println(output)
}
Maybe it is because I release the process but I dont think it should happen
The call io.Copy(&output, outputReader) blocks until read on outputReader returns EOF or some other error. Read on outputReader does not return EOF because the write side of the pipe is still open in the parent process. Fix by closing the writer in the parent process.
...
if releaseProcessErr := process.Release(); releaseProcessErr != nil {
panic(releaseProcessErr)
}
outputWriter.Close() // <-- add this line
var output bytes.Buffer
io.Copy(&output, outputReader)
fmt.Println(output)
...
Use the os/exec package to simplify the code:
cmd := exec.Command("/usr/bin/ls")
cmd.Dir = "/tmp"
output, err := cmd.CombinedOutput()
if err != nil {
log.Fatal(err)
}
fmt.Println(string(output))
I am executing bash commands from a golang application. Now the stdout and stderr go directly to console:
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
But I would like stdout and stderr to be returned as string variables from the runBashCommandAndKillIfTooSlow function without printing to the console immediately.
How to implement this?
The code:
package main
import (
"fmt"
"log"
"os"
"os/exec"
"time"
)
func main() {
ok, outString, errString := runBashCommandAndKillIfTooSlow("ls -la", 2000)
fmt.Println("ok")
fmt.Println(ok)
fmt.Println("outString")
fmt.Println(outString)
fmt.Println("errString")
fmt.Println(errString)
}
/*
run bash command and kill it if it works longer than "killInMilliSeconds" milliseconds
*/
func runBashCommandAndKillIfTooSlow(command string, killInMilliSeconds time.Duration) (okResult bool, stdout, stderr string) {
fmt.Println("running bash command...")
fmt.Println(command)
cmd := exec.Command("sh", "-c", command)
cmd.Stdout = os.Stdout // cmd.Stdout -> stdout
cmd.Stderr = os.Stderr // cmd.Stderr -> stderr
okResult = true
err := cmd.Start()
log.Printf("Waiting for command to finish...")
done := make(chan error, 1)
go func() {
done <- cmd.Wait()
}()
select {
case <-time.After(killInMilliSeconds * time.Millisecond):
if err := cmd.Process.Kill(); err != nil {
log.Fatal("failed to kill: ", err)
okResult = false
}
<-done // allow goroutine to exit
// log.Println("process killed")
case err := <-done:
if err != nil {
log.Printf("process done with error = %v", err)
okResult = false
}
}
if err != nil {
log.Fatal(err)
okResult = false
}
return
}
By the way, the program should keep its ability to kill the bash command if it was too slow (killInMilliSeconds parameter).
Set the output to a strings.Builder (in Go versions 1.10 or later) or a bytes.Buffer
var outbuf, errbuf strings.Builder // or bytes.Buffer
cmd.Stdout = &outbuf
cmd.Stderr = &errbuf
After running the command, you can get the stdout and stderr as a string by calling the Builder.String() method:
stdout := outbuf.String()
stderr := errbuf.String()
You can simplify this quite a bit by using cmd.Run() instead of cmd.Start() to have it automatically wait for it to be finish, and use exec.CommandContext() to have it timeout. This will also output in the correct order, whereas the original program is out of order due to go routines.
Here's the exact same program simplified and using #Mello Marmot's answer:
package main
import (
"bytes"
"fmt"
"log"
"os"
"os/exec"
"time"
"golang.org/x/net/context"
)
func main() {
ctx := context.Background()
ok, outString, errString := runBashCommandAndKillIfTooSlow(ctx, "ls -la", 2000*time.Millisecond)
fmt.Println("ok")
fmt.Println(ok)
fmt.Println("outString")
fmt.Println(outString)
fmt.Println("errString")
fmt.Println(errString)
}
/*
run bash command and kill it if it works longer than "killIn"
*/
func runBashCommandAndKillIfTooSlow(ctx context.Context, command string, killIn time.Duration) (okResult bool, stdout, stderr string) {
fmt.Println("running bash command...")
fmt.Println(command)
ctx, _ = context.WithTimeout(ctx, killIn)
cmd := exec.CommandContext(ctx, "sh", "-c", command)
// Set output to Byte Buffers
var outb, errb bytes.Buffer
cmd.Stdout = &outb
cmd.Stderr = &errb
okResult = true
err := cmd.Run()
stdout = outb.String()
stderr = errb.String()
if err != nil {
log.Fatal(err)
okResult = false
}
return
}
Another option is strings.Builder:
package main
import (
"os/exec"
"strings"
)
func main() {
b := new(strings.Builder)
c := exec.Command("go", "version")
c.Stdout = b
c.Run()
println(b.String() == "go version go1.16.3 windows/amd64\n")
}
https://golang.org/pkg/strings#Builder
I read the explanation from golang.org, it says like below.
// ExtraFiles specifies additional open files to be inherited by the
// new process. It does not include standard input, standard output, or
// standard error. If non-nil, entry i becomes file descriptor 3+i.
//
// BUG: on OS X 10.6, child processes may sometimes inherit unwanted fds.
// http://golang.org/issue/2603
ExtraFiles []*os.File
I'm not very understand about it ? For example I have such code below.
cmd := &exec.Cmd{
Path: init,
Args: initArgs,
}
cmd.Stdin = Stdin
cmd.Stdout = Stdout
cmd.Stderr = Stderr
cmd.Dir = Rootfs
cmd.ExtraFiles = []*os.File{childPipe}
Is that mean, since I have written a childpipe in cmd.ExtraFiles = []*os.File{childPipe}, I can use it by writing fd 3 directly.
pipe = os.NewFile(uintptr(3), "pipe")
json.NewEncoder(pipe).Encode(newThing)
Thanks if anyone can give some help!
Correct; you can read from the pipe by creating a new *File whose file descriptor is that of the child pipe. Below is a example of piping data from the child process to the parent:
Parent:
package main
import (
"fmt"
"os/exec"
"os"
"encoding/json"
)
func main() {
init := "child"
initArgs := []string{"hello world"}
r, w, err := os.Pipe()
if err != nil {
panic(err)
}
cmd := exec.Command(init, initArgs...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.ExtraFiles = []*os.File{w}
if err := cmd.Start(); err != nil {
panic(err)
}
var data interface{}
decoder := json.NewDecoder(r)
if err := decoder.Decode(&data); err != nil {
panic(err)
}
fmt.Printf("Data received from child pipe: %v\n", data)
}
Child:
package main
import (
"os"
"encoding/json"
"strings"
"fmt"
)
func main() {
if len(os.Args) < 2 {
os.Exit(1)
}
arg := strings.ToUpper(os.Args[1])
pipe := os.NewFile(uintptr(3), "pipe")
err := json.NewEncoder(pipe).Encode(arg)
if err != nil {
panic(err)
}
fmt.Println("This message printed to standard output, not to the pipe")
}
Isn't this Golang program supposed to output a directory listing to stdout?
It compiles ok, but does nothing.
package main
import "exec"
func main() {
argv := []string{"-la"}
envv := []string{}
exec.Run("ls", argv, envv, "", exec.DevNull, exec.PassThrough, exec.MergeWithStdout)
}
this works:
package main
import "exec"
func main() {
cmd, err := exec.Run("/bin/ls", []string{"/bin/ls", "-la"}, []string{}, "", exec.DevNull, exec.PassThrough, exec.PassThrough)
if (err != nil) {
return
}
cmd.Close()
}
You could also do it in native go using: ioutil.ReadDir(dir), like so:
//listdir.go
package main
import (
"os"
"io/ioutil"
"fmt"
)
func ListDir(dir string) ([]os.FileInfo, error) {
return ioutil.ReadDir(dir)
}
func main() {
dir := "./"
if len(os.Args) > 1 {
dir = os.Args[1]
}
fi, err := ListDir(dir)
if err != nil {
fmt.Println("Error", err)
}
for _, f := range fi {
d := "-"
if f.IsDir() { d = "d" }
fmt.Printf("%s %o %d %s %s\n", d, f.Mode() & 0777, f.Size(), f.ModTime().Format("Jan 2 15:04"), f.Name())
}
}
Checkout the documentation available for ioutil and os packages.
By default exec.Command will leave standard input, output and error connected to /dev/null. So, your 'ls' command is running fine but the output is just being thrown away. If you add:
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
before the exec.Run call then your output will go where you probably expect it.
exec.Run replaces your program with the one it executes -- it never returns to your app. This means that when 'cd' completes, it will exit as normal, and the only effect should be of changing the directory; 'ls' will never run.