I'm trying to have a pipe go to the cmd.ExtraFiles
I currently have the error saying
cannot use cmdstdout (type io.ReadCloser) as type []byte in argument to pipeR.Read
cannot use cmdstdout (type io.ReadCloser) as type []byte in argument to fd3.Write
This is the gocode I have thus far
cmd2 = exec.Command("-i", "pipe:0", "-i", "pipe:1")
cmd1 := exec.Command("command", "-o", "-")
pipeR, pipeW, _ := os.Pipe()
cmd2.ExtraFiles = []*os.File{
pipeW,
}
cmd1.Start()
cmd1stdout, err := cmd1.StdoutPipe()
if err != nil {
log.Printf("pipeThruError: %v\n", err)
return err
}
fd3 := os.NewFile(3, "/proc/self/fd/3")
fd3.Write(cmd1stdout)
pipeR.Read(cmd1stdout)
pipeR.Close()
pipeW.Close()
fd3.Close()
cmd3 = exec.Command("command", "-o", "-")
stdin, stdinErr := cmd3.StdoutPipe()
if stdinErr != nil {
log.Printf("pipeThruFStdinErr: %v\n", stdinErr)
return stdinErr
}
cmd3.Start()
cmd2.Stdin = stdin
EDIT: Added full scope
The goal is to have cmd2 accept input via cmd3 by Stdin, and have cmd1 output piped via ExtraFiles
The types don't quite line up here. Specifically,
cmd.StdoutPipe
returns an io.ReadCloser
whereas
pipeR.Read
is expecting an []byte as input.
I believe you are ultimately looking to utilize the Read and Write functions of the os package to accomplish your task as shown below:
package main
import (
"log"
"os"
"os/exec"
)
func main() {
cmd := exec.Command("command", "-o", "-")
pipeR, pipeW, _ := os.Pipe()
cmd.ExtraFiles = []*os.File{
pipeW,
}
cmd.Start()
cmdstdout, err := cmd.StdoutPipe()
if err != nil {
log.Printf("pipeThruError: %v\n", err)
os.Exit(1)
}
buf := make([]byte, 100)
cmdstdout.Read(buf)
pipeR.Close()
pipeW.Close()
fd3 := os.NewFile(3, "/proc/self/fd/3")
fd3.Write(buf)
fd3.Close()
}
Related
So I have a Go program that reads from STDIN as such below. I want the username and password to be entered from keyboard or device but the string slice can be passed using pipe. If I run the command as below:
echo "Hello World" | go run main.go
os.Stdin will be set to read from pipes and never the keyboard. Is there a way that I can change os.Stdin FileMode as such that it will be reading from device, i.e. keyboard for username and password?
I tried using os.Stdin.Chmod(FileMode) but received this error:
chmod /dev/stdin: invalid argument
func main() {
var n = []string{}
scanner := bufio.NewScanner(os.Stdin)
fmt.Println("Please type anything with Newline Separated, empty line signals termination")
for scanner.Scan() {
h := scanner.Text()
if h == "" {
break
}
n = append(n, h)
}
if err := scanner.Err(); err != nil {
fmt.Printf("Error in reading from STDIN: %v\n", err)
}
reader := bufio.NewReader(os.Stdin)
os.Stdout.WriteString("Username: ")
username, err := reader.ReadString('\n')
if err != nil {
fmt.Printf("Unable to read username: %v\n", err)
}
username = strings.TrimSpace(username)
os.Stdout.WriteString("Password: ")
bytePassword, _ := terminal.ReadPassword(int(os.Stdin.Fd()))
password := string(bytePassword)
os.Stdout.WriteString("\n")
}
Probably scanf could help, check this example:
https://play.golang.org/p/tteQNl0trJp
package main
import (
"fmt"
)
func main() {
fmt.Println("Enter your name")
var name string
fmt.Scanf("%s", &name)
fmt.Printf("name = %s\n", name)
}
Something a little more elaborated to check if there is something to read from stdin and if not prompt the user:
https://play.golang.org/p/7qeAQ5UNhdQ
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
// check if there is somethinig to read on STDIN
stat, _ := os.Stdin.Stat()
if (stat.Mode() & os.ModeCharDevice) == 0 {
var stdin []byte
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
stdin = append(stdin, scanner.Bytes()...)
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
fmt.Printf("stdin = %s\n", stdin)
} else {
fmt.Println("Enter your name")
var name string
fmt.Scanf("%s", &name)
fmt.Printf("name = %s\n", name)
}
}
You can instead read from /dev/tty as this is always the terminal (if the program runs on a terminal). This is portable only to Unix-like systems (Linux, BSD, macOS, etc) and won't work on Windows.
// +build !windows
tty, err := os.Open("/dev/tty")
if err != nil {
log.Fatalf("can't open /dev/tty: %s", err)
}
scanner := bufio.NewScanner(tty)
// as you were ...
I read the explanation from golang.org, it says like below.
// ExtraFiles specifies additional open files to be inherited by the
// new process. It does not include standard input, standard output, or
// standard error. If non-nil, entry i becomes file descriptor 3+i.
//
// BUG: on OS X 10.6, child processes may sometimes inherit unwanted fds.
// http://golang.org/issue/2603
ExtraFiles []*os.File
I'm not very understand about it ? For example I have such code below.
cmd := &exec.Cmd{
Path: init,
Args: initArgs,
}
cmd.Stdin = Stdin
cmd.Stdout = Stdout
cmd.Stderr = Stderr
cmd.Dir = Rootfs
cmd.ExtraFiles = []*os.File{childPipe}
Is that mean, since I have written a childpipe in cmd.ExtraFiles = []*os.File{childPipe}, I can use it by writing fd 3 directly.
pipe = os.NewFile(uintptr(3), "pipe")
json.NewEncoder(pipe).Encode(newThing)
Thanks if anyone can give some help!
Correct; you can read from the pipe by creating a new *File whose file descriptor is that of the child pipe. Below is a example of piping data from the child process to the parent:
Parent:
package main
import (
"fmt"
"os/exec"
"os"
"encoding/json"
)
func main() {
init := "child"
initArgs := []string{"hello world"}
r, w, err := os.Pipe()
if err != nil {
panic(err)
}
cmd := exec.Command(init, initArgs...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.ExtraFiles = []*os.File{w}
if err := cmd.Start(); err != nil {
panic(err)
}
var data interface{}
decoder := json.NewDecoder(r)
if err := decoder.Decode(&data); err != nil {
panic(err)
}
fmt.Printf("Data received from child pipe: %v\n", data)
}
Child:
package main
import (
"os"
"encoding/json"
"strings"
"fmt"
)
func main() {
if len(os.Args) < 2 {
os.Exit(1)
}
arg := strings.ToUpper(os.Args[1])
pipe := os.NewFile(uintptr(3), "pipe")
err := json.NewEncoder(pipe).Encode(arg)
if err != nil {
panic(err)
}
fmt.Println("This message printed to standard output, not to the pipe")
}
I have an app called "myapp". That app simply writes to stderr.
The important bit is, I want to capture what is written in stderr and process it in real-time. How would I go about doing that?
I tried the code below. :
cmd := exec.Command("myapp") // this app prints lines to stderr
stderr, err := cmd.StderrPipe()
if err != nil {
log.Fatal(err)
}
if err := cmd.Start(); err != nil {
log.Fatal(err)
}
if b, err := ioutil.ReadAll(stderr); err == nil {
log.Println(string(b))
}
if err := cmd.Wait(); err != nil {
log.Fatal(err)
}
The code doesn't print out anyting. I suspect it's because ioutil.ReadAll() is not the proper func to call since it waits for EOF. How else would I read from the stderr pipe?
You can replace the command executed with anything that outputs to stdout or stderr like tail -f mylogfile. The point is, I want to process the lines as they are written to stdout.
StderrPipe returns a ReadCloser. You can use that to create a bufio.Scanner and then read lines one by one:
sc := bufio.NewScanner(stderr)
for sc.Scan() {
fmt.Printf("Line: %s\n", sc.Text());
}
Create a type that implements io.Writer and set that as the command's stderr writer.
type Processor struct{}
func (Processor) Write(b []byte) (int, error) {
// intercept data here
return os.Stdout.Write(b)
}
func main() {
cmd := exec.Command("mycommand")
cmd.Stderr = Processor{}
_ = cmd.Run()
}
I'm writing a small program with an interpreter, I would like to pipe any command that is not recognized by my shell to bash, and print the output as if written in a normal terminal.
func RunExtern(c *shell.Cmd) (string, os.Error) {
cmd := exec.Command(c.Cmd(), c.Args()...)
out, err := cmd.Output()
return string(out), err
}
this is what I've written so far, but it only executes a program with its args, I would like to send the whole line to bash and get the output, any idea how to do so ?
For example, to list directory entries in columns,
package main
import (
"exec"
"fmt"
"os"
)
func BashExec(argv []string) (string, os.Error) {
cmdarg := ""
for _, arg := range argv {
cmdarg += `"` + arg + `" `
}
cmd := exec.Command("bash", "-c", cmdarg)
out, err := cmd.Output()
return string(out), err
}
func main() {
out, err := BashExec([]string{`ls`, `-C`})
if err != nil {
fmt.Println(err)
}
fmt.Println(out)
}
Let's say I want to run 'ls' in a go program, and store the results in a string. There seems to be a few commands to fork processes in the exec and os packages, but they require file arguments for stdout, etc. Is there a way to get the output as a string?
There is an easier way now:
package main
import (
"fmt"
"log"
"os/exec"
)
func main() {
out, err := exec.Command("date").Output()
if err != nil {
log.Fatal(err)
}
fmt.Printf("The date is %s\n", out)
}
Where out is the standard output. It's in the format []byte, but you can change it to string easily with:
string(out)
You can also use CombinedOutput() instead of Output() which returns standard output and standard error.
exec.Command
To get both stdout and stderr into separate strings, you can use byte buffers like so:
cmd := exec.Command("date")
var outb, errb bytes.Buffer
cmd.Stdout = &outb
cmd.Stderr = &errb
err := cmd.Run()
if err != nil {
log.Fatal(err)
}
fmt.Println("out:", outb.String(), "err:", errb.String())
cmd := exec.Command("ls", "-al")
output, _ := cmd.CombinedOutput()
fmt.Println(string(output))
or
cmd := exec.Command(name, arg...)
stdout, err := cmd.StdoutPipe()
cmd.Stderr = cmd.Stdout
if err != nil {
return err
}
if err = cmd.Start(); err != nil {
return err
}
for {
tmp := make([]byte, 1024)
_, err := stdout.Read(tmp)
fmt.Print(string(tmp))
if err != nil {
break
}
}
I used this with a recent version of GO (~1.11)
// CmdExec Execute a command
func CmdExec(args ...string) (string, error) {
baseCmd := args[0]
cmdArgs := args[1:]
log.Debugf("Exec: %v", args)
cmd := exec.Command(baseCmd, cmdArgs...)
out, err := cmd.Output()
if err != nil {
return "", err
}
return string(out), nil
}
// Usage:
// out, err := CmdExec("ls", "/home")
Two options, depending on the paradigm you prefer:
os.ForkExec()
exec.Run()
Use exec.Run, passing Pipe for stdout. Read from the pipe that it returns.
If you are wanting string output, strings.Builder is more efficient [1] than
bytes.Buffer:
package main
import (
"os/exec"
"strings"
)
func main() {
c, b := exec.Command("go", "version"), new(strings.Builder)
c.Stdout = b
c.Run()
print(b.String())
}
https://golang.org/pkg/bytes#Buffer.String
Edit: This answer is obsolete. Please see Fatih Arslan's answer below.
Use exec.Run by specifying Pipe as the stdout (and stderr if you want). It will return cmd, which contains an os.File in the Stdout (and Stderr) fields. Then you can read it using for example ioutil.ReadAll.
Example:
package main
import (
"exec";
"io/ioutil";
)
func main() {
if cmd, e := exec.Run("/bin/ls", nil, nil, exec.DevNull, exec.Pipe, exec.MergeWithStdout); e == nil {
b, _ := ioutil.ReadAll(cmd.Stdout)
println("output: " + string(b))
}
}