Last days i am kinda struggling with forking a process and kill the parent from the forked one(child)
I don't know why but it seems not to kill the parent at all. first i tough it where the open connections that keeps the process running because of the graceful shutdown process, But its not that issue.
if i send SIGTERM to the parent from the terminal it worked perfect, but when the child send SIGTERM it does not stop, force quitting is no option cause of graceful shutdown.
edit * the processes are still in the process list. Maybe that is for parent to track its children?
Some code that does the fork, maybe i do something wrong here
func (s *Server) Upgrade() error {
tl := s.listener.(*listener)
addr := s.listener.Addr()
name := fmt.Sprintf("%s:%s->", addr.Network(), addr.String())
os.Setenv("PROX_NAME", name)
fl, err := tl.File()
if err != nil {
return fmt.Errorf("Failed to extract file desciptor, %v", err)
}
fd := fl.Fd()
argv0, err := exec.LookPath(os.Args[0])
if err != nil {
return fmt.Errorf("Failed to execute lookpath, %v", err)
}
noCloseOnExec(fd)
files := make([]*os.File, fd+1)
files[syscall.Stdin] = os.Stdin
files[syscall.Stdout] = os.Stdout
files[syscall.Stderr] = os.Stderr
files[fd] = os.NewFile(fd, name)
wd, err := os.Getwd()
if err != nil {
return err
}
os.Setenv("GPROXY_FD", fmt.Sprintf("%d", fd))
os.Setenv("GPROXY_PID", fmt.Sprintf("%d", syscall.Getpid()))
args := []string{"gproxy", "-debug", "start"}
_, err = os.StartProcess(argv0, args, &os.ProcAttr{
Dir: wd,
Env: os.Environ(),
Files: files,
})
return err
}
the termination of the parent
func termParentProcess() error {
pid := syscall.Getppid()
return syscall.Kill(pid, syscall.SIGTERM)
}
Long story short, you can't fork, there's a very old and ongoing issue for it here.
You could use something like https://github.com/icattlecoder/godaemon but the parent will have to terminate itself not the child.
var isChild = flag.Bool("child", false, "parent pid")
func main() {
flag.Parse()
var s string
if !*isChild { //parent
fmt.Println("forking")
files := make([]*os.File, 3)
files[syscall.Stdin] = os.Stdin
files[syscall.Stdout] = os.Stdout
files[syscall.Stderr] = os.Stderr
fmt.Println(os.StartProcess(os.Args[0], append(os.Args, "-child"), &os.ProcAttr{
Dir: "/tmp",
Env: os.Environ(),
Files: files,
}))
fmt.Scan(&s) // block
} else {
ppid := os.Getppid()
fmt.Println("ppid", ppid, "kill:", syscall.Kill(ppid, syscall.SIGTERM))
time.Sleep(5 * time.Second)
fmt.Println("child dying")
}
}
The problem my parent process wont terminate on a TERM signal is because of a inner for loop that wont break. I fixed the for loop and let the main function return. If in Go the main returns the program exits. sorry for bothering you guys with a big mistake of myself
Related
Is it possible to, in a goroutine, stream a file as it is being written to by a subprocess command? The goal here is to capture the output as both a file and stream it live. I have:
cmd := exec.CommandContext(ctx, c.Bin, args...)
// CANT USE NON FILE!!
// https://github.com/golang/go/issues/23019
tempout, err := ioutil.TempFile("", "workerout")
if err != nil {
return "", err
}
tempoutName := tempout.Name()
defer os.Remove(tempoutName) // clean up
temperr, err := ioutil.TempFile("", "workererr")
if err != nil {
return "", err
}
temperrName := temperr.Name()
defer os.Remove(temperrName) // clean up
cmd.Stdout = tempout
cmd.Stderr = temperr
err = cmd.Start()
// Stream the logs
// Does not work. Flushing issue???
/*
ro := bufio.NewReader(tempout)
go func() {
line, _, _ := ro.ReadLine()
logger.Debug(line)
}()
re := bufio.NewReader(temperr)
go func() {
line, _, _ := re.ReadLine()
logger.Error(line)
}()
*/
cmd.Wait()
return tempout.Read(... // read the file into a string and return it
The commented out section of the code seems to show the logs only once the command exits (either by ctx being cancelled, or it finishes), in that it does not log in real time. Is there a way to make this log in real time?
I've been trying to write a program that record what is passed to a subprocess and the console returns in live (in the future, to record SSH sessions, for now on Python shell for testing)
I can record without issue stdout and stderr (it shows and record it correctly) but I can't find a way to do the same on stdin ?
Basically that my stdin will both map to the subprocess stdin and write to the log file.
There is my current code :
func SSH(cmd *cobra.Command, args []string) {
logFile := fmt.Sprintf("%v#%s.log", args[0], time.Now().Format(SSHLogDateFormat))
usr, _ := user.Current()
home := usr.HomeDir
logDir := fmt.Sprintf("%s/%s/logs", home, config.ConfigDir)
if _, err := os.Stat(logDir); os.IsNotExist(err) {
err = os.Mkdir(logDir, os.FileMode(int(0700)))
if err != nil {
log.Fatalf("Failed to create %s: %s", logDir, err)
}
}
fullLogFile := fmt.Sprintf("%s/%s", logDir, logFile)
log.Infof("Started recording to %s", fullLogFile)
bash, err := exec.LookPath("bash")
if err != nil {
log.Errorf("Could not locate bash: %v", err)
}
f, err := os.Create(fullLogFile)
if err != nil {
log.Fatalf("Failed to open device logs: %s", err)
}
command := exec.Command(bash, "-c", "python")
out := io.MultiWriter(os.Stdout, f)
command.Stderr = out
command.Stdout = out
if err := command.Start(); nil != err {
log.Fatalf("Error starting program: %s, %s", command.Path, err.Error())
}
err = command.Wait()
if err != nil {
log.Fatalf("Error waiting program: %s, %s", command.Path, err.Error())
}
f.Close()
log.Infof("Finished recording to %s", fullLogFile)
}
Tried this too without success :
out := io.MultiWriter(os.Stdout, f)
in := io.TeeReader(os.Stdin, out)
command.Stderr = out
command.Stdout = out
command.Stdin = in
You need to write to the process's stdin. Get a write pipe to that:
procIn, err := command.StdinPipe()
if nil!=err {
log.Fatal(err)
}
Then create a multiWriter to write to both log and process:
inWriter := io.MultiWriter(procIn, f)
Finally, copy Stdin into the MultiWriter:
go func() {
io.Copy(inWriter, os.Stdin)
procIn.Close()
}()
We do the copy in a goroutine, so as not to hang everything up: we haven't started the command yet, so there's nothing receiving the written bytes. It needs to occur in parallel to the command running.
Here's a very simple example:
package main
import (
`os`
`os/exec`
`io`
)
// pipeto copies stdin to logOut and to the command,
// and copies the commands stdout and stderr to logOut and
// to our stderr.
func pipeto(logOut os.Writer, cmd string, args ...string) error {
cmd := exec.Command(cmd, args...)
out := io.MultiWriter(os.Stdout, logOut)
cmd.Stderr, cmd.Stdout = out, out
procIn, err := cmd.StdinPipe()
if nil!=err {
return err
}
go func() {
io.Copy( io.MultiWriter(procIn, logOut) , os.Stdin )
procIn.Close()
}()
return cmd.Run()
}
func main() {
logOut, err := os.Create(`logout.log`)
if nil!=err {
panic(err)
}
defer logOut.Close()
if err := pipeto(logOut, `sed`, `s/tea/coffee/g`); nil!=err {
panic(err)
}
}
You can test it, where I've named my go file pipetest.go:
echo this is a test of tea | go run pipetest.go
The you will see both the input and the output reflected in logout.log:
this is a test of tea
this is a test of coffee
At the end I found the solution by using the PTY library (That would have been needed anyway to handle special signals and tabs on subprocesses): https://github.com/creack/pty
I took the Shell example and just replaced the io.Copy with my MultiWriter
In Go, I'm trying to:
start a subprocess
read from stdout and stderr separately
implement an overall timeout
After much googling, we've come up with some code that seems to do the job, most of the time. But there seems to be a race condition whereby some output is not read.
The problem seems to only occur on Linux, not Windows.
Following the simplest possible solution found with google, we tried creating a context with a timeout:
context.WithTimeout(context.Background(), 10*time.Second)
While this worked most of the time, we were able to find cases where it would just hang forever. There was some aspect of the child process that caused this to deadlock. (Something to do with grandchildren that were not sufficiently dissasociated from the child process, and thus caused the child to never completely exit.)
Also, it seemed that in some cases the error that is returned when the timeout occurrs would indicate a timeout, but would only be delivered after the process had actually exited (thus making the whole concept of the timeout useless).
func GetOutputsWithTimeout(command string, args []string, timeout int) (io.ReadCloser, io.ReadCloser, int, error) {
start := time.Now()
procLogger.Tracef("Initializing %s %+v", command, args)
cmd := exec.Command(command, args...)
// get pipes to standard output/error
stdout, err := cmd.StdoutPipe()
if err != nil {
return emptyReader(), emptyReader(), -1, fmt.Errorf("cmd.StdoutPipe() error: %+v", err.Error())
}
stderr, err := cmd.StderrPipe()
if err != nil {
return emptyReader(), emptyReader(), -1, fmt.Errorf("cmd.StderrPipe() error: %+v", err.Error())
}
// setup buffers to capture standard output and standard error
var buf bytes.Buffer
var ebuf bytes.Buffer
// create a channel to capture any errors from wait
done := make(chan error)
// create a semaphore to indicate when both pipes are closed
var wg sync.WaitGroup
wg.Add(2)
go func() {
if _, err := buf.ReadFrom(stdout); err != nil {
procLogger.Debugf("%s: Error Slurping stdout: %+v", command, err)
}
wg.Done()
}()
go func() {
if _, err := ebuf.ReadFrom(stderr); err != nil {
procLogger.Debugf("%s: Error Slurping stderr: %+v", command, err)
}
wg.Done()
}()
// start process
procLogger.Debugf("Starting %s", command)
if err := cmd.Start(); err != nil {
procLogger.Errorf("%s: failed to start: %+v", command, err)
return emptyReader(), emptyReader(), -1, fmt.Errorf("cmd.Start() error: %+v", err.Error())
}
go func() {
procLogger.Debugf("Waiting for %s (%d) to finish", command, cmd.Process.Pid)
err := cmd.Wait() // this can be 'forced' by the killing of the process
procLogger.Tracef("%s finished: errStatus=%+v", command, err) // err could be nil here
//notify select of completion, and the status
done <- err
}()
// Wait for timeout or completion.
select {
// Timed out
case <-time.After(time.Duration(timeout) * time.Second):
elapsed := time.Since(start)
procLogger.Errorf("%s: timeout after %.1f\n", command, elapsed.Seconds())
if err := TerminateTree(cmd); err != nil {
return ioutil.NopCloser(&buf), ioutil.NopCloser(&ebuf), -1,
fmt.Errorf("failed to kill %s, pid=%d: %+v",
command, cmd.Process.Pid, err)
}
wg.Wait() // this *should* take care of waiting for stdout and stderr to be collected after we killed the process
return ioutil.NopCloser(&buf), ioutil.NopCloser(&ebuf), -1,
fmt.Errorf("%s: timeout %d s reached, pid=%d process killed",
command, timeout, cmd.Process.Pid)
//Exited normally or with a non-zero exit code
case err := <-done:
wg.Wait() // this *should* take care of waiting for stdout and stderr to be collected after the process terminated naturally.
elapsed := time.Since(start)
procLogger.Tracef("%s: Done after %.1f\n", command, elapsed.Seconds())
rc := -1
// Note that we have to use go1.10 compatible mechanism.
if err != nil {
procLogger.Tracef("%s exited with error: %+v", command, err)
exitErr, ok := err.(*exec.ExitError)
if ok {
ws := exitErr.Sys().(syscall.WaitStatus)
rc = ws.ExitStatus()
}
procLogger.Debugf("%s exited with status %d", command, rc)
return ioutil.NopCloser(&buf), ioutil.NopCloser(&ebuf), rc,
fmt.Errorf("%s: process done with error: %+v",
command, err)
} else {
ws := cmd.ProcessState.Sys().(syscall.WaitStatus)
rc = ws.ExitStatus()
}
procLogger.Debugf("%s exited with status %d", command, rc)
return ioutil.NopCloser(&buf), ioutil.NopCloser(&ebuf), rc, nil
}
//NOTREACHED: should not reach this line!
}
Calling GetOutputsWithTimeout("uname",[]string{"-mpi"},10) will return the expected single line of output most of the time. But sometimes it will return no output, as if the goroutine that reads stdout didn't start soon enough to "catch" all the output (or exited early?) The "most of the time" strongly suggests a race condition.
We will also sometimes see errors from the goroutines about "file already closed" (this seems to happen with the timeout condition, but will happen at other "normal" times as well).
I would have thought that starting the goroutines before the cmd.Start() would have ensured that no output would be missed, and that using the WaitGroup would guarantee they would both complete before reading the buffers.
So how are we missing output? Is there still a race condition between the two "reader" goroutines and the cmd.Start()? Should we ensure those two are running using yet another WaitGroup?
Or is there a problem with the implementation of ReadFrom()?
Note that we are currently using go1.10 due to backward-compatibility problems with older OSs but the same effect occurs with go1.12.4.
Or are we overthinking this, and a simple implementation with context.WithTimeout() would do the job?
But sometimes it will return no output, as if the goroutine that reads stdout didn't start soon enough to "catch" all the output
This is impossible, because a pipe can't "lose" data. If the process is writing to stdout and the Go program isn't reading yet, the process will block.
The simplest way to approach the problem is:
Launch goroutines to collect stdout, stderr
Launch a timer that kills the process
Start the process
Wait for it to finish (or be killed by the timer) with .Wait()
If timer is fired, return timeout error
Handle wait error
func GetOutputsWithTimeout(command string, args []string, timeout int) ([]byte, []byte, int, error) {
cmd := exec.Command(command, args...)
// get pipes to standard output/error
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, nil, -1, fmt.Errorf("cmd.StdoutPipe() error: %+v", err.Error())
}
stderr, err := cmd.StderrPipe()
if err != nil {
return nil, nil, -1, fmt.Errorf("cmd.StderrPipe() error: %+v", err.Error())
}
// setup buffers to capture standard output and standard error
var stdoutBuf, stderrBuf []byte
// create 3 goroutines: stdout, stderr, timer.
// Use a waitgroup to wait.
var wg sync.WaitGroup
wg.Add(2)
go func() {
var err error
if stdoutBuf, err = ioutil.ReadAll(stdout); err != nil {
log.Printf("%s: Error Slurping stdout: %+v", command, err)
}
wg.Done()
}()
go func() {
var err error
if stderrBuf, err = ioutil.ReadAll(stderr); err != nil {
log.Printf("%s: Error Slurping stderr: %+v", command, err)
}
wg.Done()
}()
t := time.AfterFunc(time.Duration(timeout)*time.Second, func() {
cmd.Process.Kill()
})
// start process
if err := cmd.Start(); err != nil {
t.Stop()
return nil, nil, -1, fmt.Errorf("cmd.Start() error: %+v", err.Error())
}
err = cmd.Wait()
timedOut := !t.Stop()
wg.Wait()
// check if the timer timed out.
if timedOut {
return stdoutBuf, stderrBuf, -1,
fmt.Errorf("%s: timeout %d s reached, pid=%d process killed",
command, timeout, cmd.Process.Pid)
}
if err != nil {
rc := -1
if exitErr, ok := err.(*exec.ExitError); ok {
rc = exitErr.Sys().(syscall.WaitStatus).ExitStatus()
}
return stdoutBuf, stderrBuf, rc,
fmt.Errorf("%s: process done with error: %+v",
command, err)
}
// cmd.Wait docs say that if err == nil, exit code is 0
return stdoutBuf, stderrBuf, 0, nil
}
I'm having trouble sending a signal from a parent process and receiving it in the child process.
This is the code for the child process. It exits when it receives SIGINT.
// child.go
func main() {
stop := make(chan os.Signal, 1)
signal.Notify(stop, os.Interrupt)
fmt.Println("started")
<-stop
fmt.Println("stopped")
}
This is the parent process. It starts child.go, sends SIGINT, then waits for it to exit.
// main.go
func main() {
// Start child process
cmd := exec.Command("go", "run", "child.go")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
err := cmd.Start()
if err != nil {
_, _ = fmt.Fprintf(os.Stderr, "Start: " + err.Error())
return
}
// Wait, then send signal
time.Sleep(time.Millisecond * 500)
err = cmd.Process.Signal(os.Interrupt)
if err != nil {
_, _ = fmt.Fprintf(os.Stderr, "Signal: " + err.Error())
return
}
// Wait for child process to finish
err = cmd.Wait()
if err != nil {
_, _ = fmt.Fprintf(os.Stderr, "Wait: " + err.Error())
}
return
}
This code should print started\nstopped to show that it worked as expected, but it only prints started and hangs at cmd.Wait(), meaning the child process did not receive the signal.
When I run go run child.go it works fine, so I don't think the problem is with that file. I understand that func (*Process) Signal doesn't work on Windows; I am using Linux.
How can I fix the code so that the child process gets the signal sent by the parent process?
As mentioned by #JimB in the comments section, the go run is your problem.
go run child.go will compile child and execute it as it's own process. If you run a ps after go run child.go, you will see two processes running.
The process you are watching and signalling is the go executable, not the child.
Replace the exec.Command("go", "run", "child.go") with the compiled binary exec.Command("child")and it should work.
What would be an effective way to kill a process with Go code if you only know the process name? I see some functions provided by the os package like:
func FindProcess(pid int) (*Process, error)
func (p *Process) Kill() error
func (p *Process) Signal(sig Signal) error
Is there a good/common practice to get the pid without having to execute commands and then parse the output?
I have found a way to get back the pid using a command like the following:
echo $(ps cax | grep myapp | grep -o '^[ ]*[0-9]*')
and I have used it with exec.Command() but I would like to avoid it if there is a better approach.
Running external commands is probably the best way to do this. However, the following code runs on Ubuntu at least as long as you are the owner of the process to kill.
// killprocess project main.go
package main
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"log"
"os"
"path/filepath"
"strconv"
"strings"
)
// args holds the commandline args
var args []string
// findAndKillProcess walks iterative through the /process directory tree
// looking up the process name found in each /proc/<pid>/status file. If
// the name matches the name in the argument the process with the corresponding
// <pid> will be killed.
func findAndKillProcess(path string, info os.FileInfo, err error) error {
// We just return in case of errors, as they are likely due to insufficient
// privileges. We shouldn't get any errors for accessing the information we
// are interested in. Run as root (sudo) and log the error, in case you want
// this information.
if err != nil {
// log.Println(err)
return nil
}
// We are only interested in files with a path looking like /proc/<pid>/status.
if strings.Count(path, "/") == 3 {
if strings.Contains(path, "/status") {
// Let's extract the middle part of the path with the <pid> and
// convert the <pid> into an integer. Log an error if it fails.
pid, err := strconv.Atoi(path[6:strings.LastIndex(path, "/")])
if err != nil {
log.Println(err)
return nil
}
// The status file contains the name of the process in its first line.
// The line looks like "Name: theProcess".
// Log an error in case we cant read the file.
f, err := ioutil.ReadFile(path)
if err != nil {
log.Println(err)
return nil
}
// Extract the process name from within the first line in the buffer
name := string(f[6:bytes.IndexByte(f, '\n')])
if name == args[1] {
fmt.Printf("PID: %d, Name: %s will be killed.\n", pid, name)
proc, err := os.FindProcess(pid)
if err != nil {
log.Println(err)
}
// Kill the process
proc.Kill()
// Let's return a fake error to abort the walk through the
// rest of the /proc directory tree
return io.EOF
}
}
}
return nil
}
// main is the entry point of any go application
func main() {
args = os.Args
if len(args) != 2 {
log.Fatalln("Usage: killprocess <processname>")
}
fmt.Printf("trying to kill process \"%s\"\n", args[1])
err := filepath.Walk("/proc", findAndKillProcess)
if err != nil {
if err == io.EOF {
// Not an error, just a signal when we are done
err = nil
} else {
log.Fatal(err)
}
}
}
It's just an example that certainly can be improved. I wrote this for Linux and tested the code on Ubuntu 15.10. It will not run on Windows.
I finally used something like the following:
// `echo "sudo_password" | sudo -S [command]`
// is used in order to run the command with `sudo`
_, err := exec.Command("sh", "-c", "echo '"+ sudopassword +"' | sudo -S pkill -SIGINT my_app_name").Output()
if err != nil {
// ...
} else {
// ...
}
I used the SIGINT signal to gracefully stop the app.
From wikipedia:
SIGINT
The SIGINT signal is sent to a process by its controlling terminal when a user wishes to interrupt the process. This is typically initiated by pressing Ctrl+C, but on some systems, the "delete" character or "break" key can be used.
SIGKILL
The SIGKILL signal is sent to a process to cause it to terminate immediately (kill). In contrast to SIGTERM and SIGINT, this signal cannot be caught or ignored, and the receiving process cannot perform any clean-up upon receiving this signal. The following exceptions apply:
Cross-Platform (3rd party) Solution
I've implemented various solutions to do this for months now, and for some reason it took me that long to find gopsutil. It is a 3rd party library and that may or may not be a deal breaker for you, but it has worked flawlessly for our cross-platform projects. The following example will kill the first process with the matching name, but it can easily be adapted to kill all processes with the name.
import "github.com/shirou/gopsutil/v3/process"
func KillProcess(name string) error {
processes, err := process.Processes()
if err != nil {
return err
}
for _, p := range processes {
n, err := p.Name()
if err != nil {
return err
}
if n == name {
return p.Kill()
}
}
return fmt.Errorf("process not found")
}
With Context Support
As an added bonus, the library also supports context cancellation on all process related operations including process queries, and killing the process.
func KillAllProcessesCtx(ctx context.Context, name string) error {
processes, err := process.ProcessesWithContext(ctx)
if err != nil {
return err
}
for _, p := range processes {
n, err := p.NameWithContext(ctx)
if err != nil {
return err
}
if n == name {
err = p.KillWithContext(ctx)
if err != nil {
return err
}
}
}
return nil
}
Graceful Termination
The library also supports graceful termination by sending your own signal to the process.
// Do this
err = p.SendSignal(syscall.SIGINT)
// Instead of this
err = p.Kill()
You can can already kill a process by process ID with Go, so the real question
here is getting the process ID from the process name. Here is example for
Windows:
package main
import (
"fmt"
"golang.org/x/sys/windows"
)
// unsafe.Sizeof(windows.ProcessEntry32{})
const processEntrySize = 568
func processID(name string) (uint32, error) {
h, e := windows.CreateToolhelp32Snapshot(windows.TH32CS_SNAPPROCESS, 0)
if e != nil { return 0, e }
p := windows.ProcessEntry32{Size: processEntrySize}
for {
e := windows.Process32Next(h, &p)
if e != nil { return 0, e }
if windows.UTF16ToString(p.ExeFile[:]) == name {
return p.ProcessID, nil
}
}
return 0, fmt.Errorf("%q not found", name)
}
func main() {
n, e := processID("WindowsTerminal.exe")
if e != nil {
panic(e)
}
println(n)
}
https://pkg.go.dev/golang.org/x/sys/windows#CreateToolhelp32Snapshot
For Windows:
You can use below method. Pass process name which you want to terminate.
func killProcessByName(procname string) int {
kill := exec.Command("taskkill", "/im", procname, "/T", "/F")
err := kill.Run()
if err != nil {
return -1
}
return 0
}
Ref: https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/taskkill