I have the following bash script bash_loop.shthat prints 1 to 10 and sleep 3 s in between.
#!/bin/bash
# Basic while loop
counter=1
while [ $counter -le 10 ]
do
echo $counter
((counter++))
sleep 3
done
echo All done
Right now, I have my go code as follow:
burstingScript := "bash_loop.sh"
cmd := exec.Command("/bin/sh", burstingScript)
var out bytes.Buffer
cmd.Stdout = &out
if err := cmd.Run(); err != nil {
fmt.Println("An error has occurred..")
log.Fatal(err)
}
fmt.Println(out.String())
However, this only print out everything after the cmd finish running after 30s, instead of print things as they become available.
So my question is that if I can print each number while the bash script is still running instead of printing everything all together after the bash script finishes execution.
P.S.1: In the real use case, I have to process the output of bash script in realtime, instead of simply printing things out to os.Stdout, so I am wondering if there's any command poll() interface or equivalence in go.
P.S.2：In the real use case, I want to detach from the child process as soon as I find interesting message. For example, after I read 3, I want my function return 3 immediately and not wait for the rest of output anymore, though I still want the child process (the bash script) itself to be up and running.
P.S.3: In python, I would do something like this
cmd = "./bash_loop.sh"
p = subprocess.Popen(cmd, shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
result = {}
while True:
out = p.stdout.readline()
if out == '' and p.poll() != None:
break
if out != '':
#process(out)
sys.stdout.write(out)
sys.stdout.flush()
P.S.4: Now I have evolved my go snippet to follow. Will the child become a zombie if I return before the command finishes running?
burstingScript := path.Join(rootDir, "bash_loop.sh")
cmd := exec.Command("/bin/sh", burstingScript)
stdout, _ := cmd.StdoutPipe()
cmd.Start()
scanner := bufio.NewScanner(stdout)
for scanner.Scan() {
m := scanner.Text()
fmt.Println(m)
//if m=="3" {
// return process(m)
//}
}
cmd.Wait()
Set the command output to stdout:
cmd.Stdout = os.Stdout
Related
I am refer to the post How to execute system command with unknown arguments? to run a jq command on my ubuntu shell.
Below is the code I tried
import (
"fmt"
"os/exec"
"sync"
"strings"
)
func exeCmd(cmd string, wg *sync.WaitGroup) {
fmt.Println("command is ",cmd)
// splitting head => g++ parts => rest of the command
parts := strings.Fields(cmd)
head := parts[0]
parts = parts[1:len(parts)]
out, err := exec.Command(head,parts...).Output()
if err != nil {
fmt.Printf("%s", err)
}
fmt.Printf("%s", out)
wg.Done() // Need to signal to waitgroup that this goroutine is done
}
func main() {
wg := new(sync.WaitGroup)
wg.Add(1)
x := []string{"jq '(.data.legacyCollection.collectionsPage.stream.edges|map({node:(.node|{url,firstPublished,headline:{default:.headline.default},summary})})) as $edges|{data:{legacyCollection:{collectionsPage:{stream:{$edges}}}}}' long-response.json > short-response.json"}
exeCmd(x[0], wg)
wg.Wait()
}
The output is as below, seems like the commans is correctly detected but shell returns exit status 3 which is "no such process" ?
command is jq '(.data.legacyCollection.collectionsPage.stream.edges|map({node:(.node|{url,firstPublished,headline:{default:.headline.default},summary})})) as $edges|{data:{legacyCollection:{collectionsPage:{stream:{$edges}}}}}' long-response.json > short-repsonse.json exit status 3
Anyone can help on this ?
What I want is a go function that can wrapper and run bash command line the same way as I do on Linux shell
PS: the jq command I tried above works pretty well when I paste it on my Linux shell
Tried somethine else: deleted the single quote in my jq command and my command got executed with output I expect - a parsed json file
but still, I got a exist status 2 , anyone can explain
why the single quote in my command line affects how G parse the command ?
why I still go exist 2 after my shell commands complete?
The program executes the jq command, not a shell. The jq command does not understand the shell syntax passed to the command (the quotes and I/O redirection).
Use the following code to run the command with stdout redirected to short-response.json.
cmd := exec.Command("jq",
"(.data.legacyCollection.collectionsPage.stream.edges|map({node:(.node|{url,firstPublished,headline:{default:.headline.default},summary})})) as $edges|{data:{legacyCollection:{collectionsPage:{stream:{$edges}}}}}",
"long-response.json")
f, err := os.Create("short-response.json")
if err != nil {
log.Fatal(err)
}
defer f.Close()
cmd.Stdout = f // set stdout to short-response.json
err = cmd.Run()
if err != nil {
log.Fatal(err)
}
I am trying to redirect the output of a process to one place, and then to open a pipe to the process , read its data and send the data to a predefined web socket connection --> but i get EOF.
For example :
myCmd := exec.Command("bash", "-c", "some command")
myCmd.Stdout = os.Stdout
myCmd.Stderr = os.Stderr
appOut, _ := lsCmd.StdoutPipe()
And then i am calling this go routine:
data := make([]byte, 1000)
for {
for {
n, err := appOut.Read(data)
if err == nil && n > 0 {
myConnection.WriteMessage(websocket.TextMessage, data[0:n])
}
if err != nil {
log.Printf("App exited. err: %s, Error ", err)
}
}
}
Can someone tell me what i have done wrong ?
I must mention that even if I change the order of the line (openning the pipe before the redirection it still happens)
This question already has an answer here:
How can I redirect the stdout and stderr of a command to both the console and a log file while outputting in real time?
(1 answer)
Closed 5 years ago.
I have a small Go tool which basically allows the user to define an command that than will be run using os/exec.
My problem is that I want to show the user the output (stdout/stderr) of the command.
An example could look like this:
The user defines a command that in the end is sh test.sh.
Content of test.sh:
echo "Start"
sleep 7s
echo "Done"
With my current implementation the user can only see the output once the complete command finished. In the example above the user wouldn't see the output Start until the sleep command and the second echo finish.
I currently retrieve the output of the command like this:
cmd := exec.Command(command, args...)
cmd.Dir = dir
// Attach to the standard out to read what the command might print
stdout, err := cmd.StdoutPipe()
if err != nil {
log.Panic(err)
}
// Execute the command
if err := cmd.Start(); err != nil {
log.Panic(err)
}
buf := new(bytes.Buffer)
buf.ReadFrom(stdout)
log.Print(buf.String())
Is it somehow possible to read the stdout/stderr in real-time. Meaning that as soon as the user defined command creates and output it is printed?
Thank you mh-cbon. That pushed me in the right direction.
The code now looks like this and does exactly what I want it to do. I also found that when I use Run() instead of Start() the execution of the program only continues after the command has finished.
cmd := exec.Command(command, args...)
cmd.Dir = dir
var stdBuffer bytes.Buffer
mw := io.MultiWriter(os.Stdout, &stdBuffer)
cmd.Stdout = mw
cmd.Stderr = mw
// Execute the command
if err := cmd.Run(); err != nil {
log.Panic(err)
}
log.Println(stdBuffer.String())
I have a method that can spawn an interactive process, now how do I log everything (including stdin and stdout) after spawning ?
e.g.,
func execute(cmd1 string, slice []string) {
cmd := exec.Command(cmd1, slice...)
// redirect the output to terminal
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Stdin = os.Stdin
cmd.Run()
}
..
The interactive program could be :
execute(ftp)
I think I have to dup stdin, stdout and read write in separate thread.
Rather than redirecting it's output to the terminal read it and then you can log/print do whatever you want with it.
stdout, err := cmd.StdoutPipe()
b, _ := ioutil.ReadAll(stdout)
fmt.Println(string(b))
Something like the code above would work though there are many options. I think you'll want to remove all that code you have to redirect to the terminal.
you could store the output in a temporary buffer and write it to several places
outBuf := bytes.Buffer{}
cmd := exec.Command(cmd1, slice...)
cmd.Stdout = &outBuf
cmd.Run()
if outBuf.Len() > 0 {
log.Printf("%s", outBuf.String())
fmt.Fprintf(os.Stdout, "%s", outBuf.String())
}
Im trying the following, to use go to bundle a folder of html files using the CMD web2exe.
cmd := exec.Command("web2exe-win.exe", "html-folder --main index.html --export- to windows-x32 --output-dir")
var out bytes.Buffer
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
fmt.Println(err)
}
fmt.Println(out)
When a program exits non-zero it means that it could not run successfully and typically it has written an error message to STDERR (or STDOUT). You should somehow capture or print the output streams so you can inspect them for error messages. For example:
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
Note also that your command line arguments should be separate array elements (instead of space separated elements in a single string as they are now):
cmd := exec.Command("web2exe-win.exe", "html-folder", "--main", "index.html", "--export-to", "windows-x32", "--output-dir")