how to handle process output in go using named pipe - go

I am trying to set up a pipe from a running process in tmux,
in order to handle its output line by line.
I have had a look at this guide to pipe the output of a tmux session to stdout
and this article about (named) pipes in go.
I have been trying around with that for quite a while now,
but still didn't get any noteworthy results.
I would really appreciate any thoughts on how to set that pipe up,
Ideally in a way so I can range over it linewise.
Thanks a lot

Here is the solution which I found here (thank you Malcolm)
func Readln(r *bufio.Reader) (string, error) {
var (
isPrefix = true
err error
line, ln []byte
)
for isPrefix && err == nil {
line, isPrefix, err = r.ReadLine()
ln = append(ln, line...)
}
return string(ln), err
}
func handle(s string) {
//Do something with your string
}
func main() {
c := exec.Command("sh", "./tmuxpipe.sh")
err := c.Run()
if err != nil {
log.Fatal(err)
}
f, err := os.Open("/tmp/tmuxpipe")
if err != nil {
fmt.Printf("error opening file: %v\n", err)
os.Exit(1)
}
r := bufio.NewReader(f)
s, e := Readln(r)
for e == nil {
handle(s)
log.Println(s)
s, e = Readln(r)
}
}
here is the tmuxpipe.sh:
mkfifo /tmp/tmuxpipe
tmux pipe-pane -o -t tmuxSession 'cat >> /tmp/tmuxpipe'
The reason I did not just use exec.Command() there, is because for some reason beyond my comprehension this:
c := exec.Command("tmux", "pipe-pane", "-o", "-t", "tmuxSession", 'cat >> /tmp/tmuxpipe'")
err := c.Run()
handleError(err)
did not work (for me).
There was no error occuring, but the output of the tmux session wasn't displayd either.
I hope this helps anybody

Related

finding absolute path name of a process on macOS

I am new in GO and mac,
and I'm trying to use sysctl for finding the full path name of running processes.
kprocs, err := unix.SysctlKinfoProcSlice("kern.proc.all")
if err != nil {
fmt.Println("error: ", err)
}
for _, proc := range kprocs {
name := string(proc.Proc.P_comm[:])
pid := proc.Proc.P_pid
extName, err := unix.SysctlKinfoProc("kern.proc.pathname", int(pid))
if err != nil {
fmt.Println("error: ", err)
}
and getting error: no such file or directory
am I using this function correcly?
EDIT
If I run the process like this: ./processName ,then I am not getting it's full path, for example /Users/username/go/src/processName - which is what I need.
All the solutions with ps will give the relative path, and I need someting that gives the absolute path of the process.
Here you go:
func printPath(pid int32) {
cmd := exec.Command("ps", append([]string{"xuwww", "-p", strconv.Itoa(int(pid))})...)
output, e := cmd.CombinedOutput()
if e != nil {
fmt.Println(e)
}
lines := strings.Split(string(output), "\n")
comps := strings.Fields(lines[1])
fmt.Println(comps[len(comps)-1])
}
func main() {
kprocs, err := unix.SysctlKinfoProcSlice("kern.proc.all")
if err != nil {
fmt.Println("error: ", err)
}
for _, proc := range kprocs {
pid := proc.Proc.P_pid
printPath(pid)
}
}
I didn't use unix.SysctlKinfoProc(), instead ran a shell command ps xuwww -p PID using exec.command(). It returns output as string. Parse the output string to get absolute path.
Output:
649 /System/Library/PrivateFrameworks/IMDPersistence.framework/XPCServices/IMDPersistenceAgent.xpc/Contents/MacOS/IMDPersistenceAgent 646 /System/Library/PrivateFrameworks/AuthKit.framework/Versions/A/Support/akd 643 /System/Library/CoreServices/pbs
Following my comment on your question you can get the information you are looking for by using lsof.
Similar to the other code example you can iterate over all known processes but for some processes it takes some time for the command to return.
package main
import (
"fmt"
"os/exec"
"golang.org/x/sys/unix"
)
func printPath(pid int32) {
cmd := exec.Command("bash", "-c", fmt.Sprintf("lsof -p %d -Fn | awk 'NR==5{print}' | sed \"s/n\\//\\//\"", pid))
output, e := cmd.Output()
if e != nil {
fmt.Println(e)
return
}
fmt.Printf("%s", output)
}
func main() {
kprocs, err := unix.SysctlKinfoProcSlice("kern.proc.all")
if err != nil {
fmt.Println("error: ", err)
}
for _, proc := range kprocs {
pid := proc.Proc.P_pid
fmt.Printf("getting for pid %d: ", pid)
printPath(pid)
}
}
The code output can be improved. Processes that have exited already now make the output kind of a mess :-/

How to open a process and record stdin and stdout properly in Go?

I've been trying to write a program that record what is passed to a subprocess and the console returns in live (in the future, to record SSH sessions, for now on Python shell for testing)
I can record without issue stdout and stderr (it shows and record it correctly) but I can't find a way to do the same on stdin ?
Basically that my stdin will both map to the subprocess stdin and write to the log file.
There is my current code :
func SSH(cmd *cobra.Command, args []string) {
logFile := fmt.Sprintf("%v#%s.log", args[0], time.Now().Format(SSHLogDateFormat))
usr, _ := user.Current()
home := usr.HomeDir
logDir := fmt.Sprintf("%s/%s/logs", home, config.ConfigDir)
if _, err := os.Stat(logDir); os.IsNotExist(err) {
err = os.Mkdir(logDir, os.FileMode(int(0700)))
if err != nil {
log.Fatalf("Failed to create %s: %s", logDir, err)
}
}
fullLogFile := fmt.Sprintf("%s/%s", logDir, logFile)
log.Infof("Started recording to %s", fullLogFile)
bash, err := exec.LookPath("bash")
if err != nil {
log.Errorf("Could not locate bash: %v", err)
}
f, err := os.Create(fullLogFile)
if err != nil {
log.Fatalf("Failed to open device logs: %s", err)
}
command := exec.Command(bash, "-c", "python")
out := io.MultiWriter(os.Stdout, f)
command.Stderr = out
command.Stdout = out
if err := command.Start(); nil != err {
log.Fatalf("Error starting program: %s, %s", command.Path, err.Error())
}
err = command.Wait()
if err != nil {
log.Fatalf("Error waiting program: %s, %s", command.Path, err.Error())
}
f.Close()
log.Infof("Finished recording to %s", fullLogFile)
}
Tried this too without success :
out := io.MultiWriter(os.Stdout, f)
in := io.TeeReader(os.Stdin, out)
command.Stderr = out
command.Stdout = out
command.Stdin = in
You need to write to the process's stdin. Get a write pipe to that:
procIn, err := command.StdinPipe()
if nil!=err {
log.Fatal(err)
}
Then create a multiWriter to write to both log and process:
inWriter := io.MultiWriter(procIn, f)
Finally, copy Stdin into the MultiWriter:
go func() {
io.Copy(inWriter, os.Stdin)
procIn.Close()
}()
We do the copy in a goroutine, so as not to hang everything up: we haven't started the command yet, so there's nothing receiving the written bytes. It needs to occur in parallel to the command running.
Here's a very simple example:
package main
import (
`os`
`os/exec`
`io`
)
// pipeto copies stdin to logOut and to the command,
// and copies the commands stdout and stderr to logOut and
// to our stderr.
func pipeto(logOut os.Writer, cmd string, args ...string) error {
cmd := exec.Command(cmd, args...)
out := io.MultiWriter(os.Stdout, logOut)
cmd.Stderr, cmd.Stdout = out, out
procIn, err := cmd.StdinPipe()
if nil!=err {
return err
}
go func() {
io.Copy( io.MultiWriter(procIn, logOut) , os.Stdin )
procIn.Close()
}()
return cmd.Run()
}
func main() {
logOut, err := os.Create(`logout.log`)
if nil!=err {
panic(err)
}
defer logOut.Close()
if err := pipeto(logOut, `sed`, `s/tea/coffee/g`); nil!=err {
panic(err)
}
}
You can test it, where I've named my go file pipetest.go:
echo this is a test of tea | go run pipetest.go
The you will see both the input and the output reflected in logout.log:
this is a test of tea
this is a test of coffee
At the end I found the solution by using the PTY library (That would have been needed anyway to handle special signals and tabs on subprocesses): https://github.com/creack/pty
I took the Shell example and just replaced the io.Copy with my MultiWriter

exec.Command with input redirection

I'm trying to run a fairly simple bash command from my Go code. My program writes out an IPTables config file and I need to issue a command to make IPTables refresh from this config. This is very straightforward at the commandline:
/sbin/iptables-restore < /etc/iptables.conf
However, I can't for the life of me figure out how to issue this command with exec.Command(). I tried a few things to accomplish this:
cmd := exec.Command("/sbin/iptables-restore", "<", "/etc/iptables.conf")
// And also
cmd := exec.Command("/sbin/iptables-restore", "< /etc/iptables.conf")
No surprise, neither of those worked. I also tried to feed the filename into the command by piping in the file name to stdin:
cmd := exec.Command("/sbin/iptables-restore")
stdin, err := cmd.StdinPipe()
if err != nil {
log.Fatal(err)
}
err = cmd.Start()
if err != nil {
log.Fatal(err)
}
io.WriteString(stdin, "/etc/iptables.conf")
That doesn't work either, no surprise. I can use stdin to pipe in the contents of the file, but this seems silly when I can just tell iptables-restore what data to go read. So how might I get Go to run the command /sbin/iptables-restore < /etc/iptables.conf?
first read this /etc/iptables.conf file content then write it to cmd.StdinPipe() like this:
package main
import (
"io"
"io/ioutil"
"log"
"os/exec"
)
func main() {
bytes, err := ioutil.ReadFile("/etc/iptables.conf")
if err != nil {
log.Fatal(err)
}
cmd := exec.Command("/sbin/iptables-restore")
stdin, err := cmd.StdinPipe()
if err != nil {
log.Fatal(err)
}
err = cmd.Start()
if err != nil {
log.Fatal(err)
}
_, err = io.WriteString(stdin, string(bytes))
if err != nil {
log.Fatal(err)
}
}
cmd := exec.Command("/usr/sbin/iptables-restore", "--binary", iptablesFilePath)
_, err := cmd.CombinedOutput()
if err != nil {
return err
}
return nil
this work fine on my Raspberry Pi3
The os/exec package does not invoke the system shell, nor does it implement the < redirection syntax typically handled by a shell.
Open the input file and use that file as stdin:
stdin, err := os.Open("/etc/iptables.conf")
if err != nil {
log.Fatal(err)
}
defer stdin.Close()
cmd := exec.Command("/sbin/iptables-restore")
cmd.Stdin = stdin // <-- use open file as stdin
result, err := cmd.CombinedOutput()
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s\n", result)

ReadLine from io.ReadCloser

I need to find a way to read a line from a io.ReadCloser object OR find a way to split a byte array on a "end line" symbol. However I don't know the end line symbol and I can't find it.
My application execs a php script and needs to get the live output from the script and do "something" with it when it gets it.
Here's a small piece of my code:
cmd := exec.Command(prog, args)
/* cmd := exec.Command("ls")*/
out, err := cmd.StdoutPipe()
if err != nil {
fmt.Println(err)
}
err = cmd.Start()
if err != nil {
fmt.Println(err)
}
after this I monitor the out buffer in a go routine. I've tried 2 ways.
1) nr, er := out.Read(buf) where buf is a byte array. the problem here is that I need to brake the array for each new line
2) my second option is to create a new bufio.reader
r := bufio.NewReader(out)
line,_,e := r.ReadLine()
it runs fine if I exec a command like ls, I get the output line by line, but if I exec a php script it immediately get an End Of File error and exits(I'm guessing that's because of the delayed output from php)
EDIT: My problem was I was creating the bufio.Reader inside the go routine whereas if I do it right after the StdoutPipe() like minikomi suggested, it works fine
You can create a reader using bufio, and then read until the next line break character (Note, single quotes to denote character!):
stdout, err := cmd.StdoutPipe()
rd := bufio.NewReader(stdout)
if err := cmd.Start(); err != nil {
log.Fatal("Buffer Error:", err)
}
for {
str, err := rd.ReadString('\n')
if err != nil {
log.Fatal("Read Error:", err)
return
}
fmt.Println(str)
}
If you're trying to read from the reader in a goroutine with nothing to stop the script, it will exit.
Another option is bufio.NewScanner:
package main
import (
"bufio"
"os/exec"
)
func main() {
cmd := exec.Command("go", "env")
out, err := cmd.StdoutPipe()
if err != nil {
panic(err)
}
buf := bufio.NewScanner(out)
cmd.Start()
defer cmd.Wait()
for buf.Scan() {
println(buf.Text())
}
}
https://golang.org/pkg/bufio#NewScanner

How do you get the output of a system command in Go?

Let's say I want to run 'ls' in a go program, and store the results in a string. There seems to be a few commands to fork processes in the exec and os packages, but they require file arguments for stdout, etc. Is there a way to get the output as a string?
There is an easier way now:
package main
import (
"fmt"
"log"
"os/exec"
)
func main() {
out, err := exec.Command("date").Output()
if err != nil {
log.Fatal(err)
}
fmt.Printf("The date is %s\n", out)
}
Where out is the standard output. It's in the format []byte, but you can change it to string easily with:
string(out)
You can also use CombinedOutput() instead of Output() which returns standard output and standard error.
exec.Command
To get both stdout and stderr into separate strings, you can use byte buffers like so:
cmd := exec.Command("date")
var outb, errb bytes.Buffer
cmd.Stdout = &outb
cmd.Stderr = &errb
err := cmd.Run()
if err != nil {
log.Fatal(err)
}
fmt.Println("out:", outb.String(), "err:", errb.String())
cmd := exec.Command("ls", "-al")
output, _ := cmd.CombinedOutput()
fmt.Println(string(output))
or
cmd := exec.Command(name, arg...)
stdout, err := cmd.StdoutPipe()
cmd.Stderr = cmd.Stdout
if err != nil {
return err
}
if err = cmd.Start(); err != nil {
return err
}
for {
tmp := make([]byte, 1024)
_, err := stdout.Read(tmp)
fmt.Print(string(tmp))
if err != nil {
break
}
}
I used this with a recent version of GO (~1.11)
// CmdExec Execute a command
func CmdExec(args ...string) (string, error) {
baseCmd := args[0]
cmdArgs := args[1:]
log.Debugf("Exec: %v", args)
cmd := exec.Command(baseCmd, cmdArgs...)
out, err := cmd.Output()
if err != nil {
return "", err
}
return string(out), nil
}
// Usage:
// out, err := CmdExec("ls", "/home")
Two options, depending on the paradigm you prefer:
os.ForkExec()
exec.Run()
Use exec.Run, passing Pipe for stdout. Read from the pipe that it returns.
If you are wanting string output, strings.Builder is more efficient [1] than
bytes.Buffer:
package main
import (
"os/exec"
"strings"
)
func main() {
c, b := exec.Command("go", "version"), new(strings.Builder)
c.Stdout = b
c.Run()
print(b.String())
}
https://golang.org/pkg/bytes#Buffer.String
Edit: This answer is obsolete. Please see Fatih Arslan's answer below.
Use exec.Run by specifying Pipe as the stdout (and stderr if you want). It will return cmd, which contains an os.File in the Stdout (and Stderr) fields. Then you can read it using for example ioutil.ReadAll.
Example:
package main
import (
"exec";
"io/ioutil";
)
func main() {
if cmd, e := exec.Run("/bin/ls", nil, nil, exec.DevNull, exec.Pipe, exec.MergeWithStdout); e == nil {
b, _ := ioutil.ReadAll(cmd.Stdout)
println("output: " + string(b))
}
}

Resources