I'm facing with a weird golang issue. The following code will clarify:
package main
import (
"os/exec"
"io"
"fmt"
"os"
)
var (
pw io.WriteCloser
pr io.ReadCloser
)
func main() {
term := exec.Command("/bin/sh")
// Get stdin writer pipe
pw, _ = term.StdinPipe()
pr, _ = term.StdoutPipe()
term.Start()
run("cd ~")
pwd := run("pwd");
// Do something with pwd output
...
term.Wait()
}
func run(c string) string {
io.WriteString(pw, fmt.Sprintln(c))
buf := make([]byte, 32 * 1024)
pr.Read(buf)
return string(buf)
}
I'd like to run some commands in a shell env and read their output. There's no problem on write/run command but it seems that there're some limitations while reading:
you can't know if a command doesn't output anything or not;
there's no way to check if stdout is ready to be read or not.
The pr.Read(dest) method will block the code flow until something is read from stdout. As said, the goal is to read sequentially (without using a go routine and/or an infinite loop). This means that if we send a cd command the func end is never reached.
Setting the non-block flag through unix.SetNonblock on stdout file descriptor seems to solve the above issue but you can't know prior if it's ready or not and an error saying "resource temporary not available" is returned from .Read call.
As Cerise Limón mentioned go functions whould be the way to go here, since these sorts of interactive scripting exercises are traditionally done with expect.
You can wrap the the parrellel execution into a library to it might still look like sequencial code, so this might be helpful: https://github.com/ThomasRooney/gexpect
From the readme:
child, err := gexpect.Spawn("python")
if err != nil { panic(err) }
child.Expect(">>>")
child.SendLine("print 'Hello World'")
child.Interact()
child.Close()
Related
package main
import (
"bufio"
"os"
)
func main() {
bw := bufio.NewWriter(os.Stdin)
bw2 := bufio.NewWriter(os.Stdout)
bw.WriteString("Hello, world 1\n")
bw2.WriteString("Hello, world 2\n")
bw.Flush()
bw2.Flush()
}
This code show both string in a local environment.
But why does it work differently in different environments?
My local environment
OS : macOS 12.6
go : go1.19.2 darwin/amd64
ide : vscode
on my local machine :
$ go run myworkspace/main/main.go
Hello, world 1
Hello, world 2
on the playground :
# in the 'Output' section
---
Hello, world 2
Program exited.
The Go Playground
https://go.dev/play/p/PtoDwCZGggd
Scroll all the way down to "About the Playground"
About the Playground
The Go Playground runs the program inside a sandbox.
There are limitations to the programs that can be run in the playground:
The only communication a playground program has to the outside world is by writing to standard output and standard error.
The Go Playground discards writes to standard input.
Locally, os.Stdin is a file.
var (
Stdin = NewFile(uintptr(syscall.Stdin), "/dev/stdin")
Stdout = NewFile(uintptr(syscall.Stdout), "/dev/stdout")
Stderr = NewFile(uintptr(syscall.Stderr), "/dev/stderr")
)
Stdin, Stdout, and Stderr are open Files pointing to the standard input, standard output, and standard error file descriptors.
The bottom line is : your program does not control what thingy will be used as Stdin and Stdout when it is started.
On your local machine, try running : echo "" | go run main.go or go run main.go < anyfile.
(also : you should check for errors in your go code)
It is a standard setup, when running a command from a terminal, that the shell uses the same file descriptor, which points to a read/write terminal, for stdin and stdout/stderr. With that setup, writing to stdin "works".
But you can't rely on it (you shouldn't, actually), and it turns out that the playground doesn't start its processes that way :
package main
import (
"fmt"
"os"
)
func main() {
_, err := os.Stdin.Write([]byte("Hello\n"))
if err != nil {
fmt.Fprintln(os.Stderr, "*** error:", err)
}
}
// Output:
// *** error: write /dev/stdin: bad file descriptor
//
// Program exited.
https://go.dev/play/p/NmxgOsK2ovp
see also :
Why is it possible to write() to STDIN?
Writing to stdin and reading from stdout (UNIX/LINUX/C Programming)
(found by googling "writing to stdin")
I am trying to learn TUI programming in Go using the TCell API.
It is a simple app that print word "hello". However, when I run the program below, nothing happens. Please tell me what I am doing wrong.
package main
import (
"fmt"
"github.com/gdamore/tcell"
"os"
)
func main() {
scn, err := tcell.NewScreen()
if err != nil {
fmt.Fprintf(os.Stderr, "%v\n", err)
os.Exit(1)
}
hhh := []rune("hello")
scn.SetContent(10, 10, rune(' '), hhh, tcell.StyleDefault)
scn.Show()
}
The creator of this api (https://github.com/gdamore/tcell.git) provided the solution. Here is his respond:
There are three potential issues.
First, you need to initialize the screen. Call scn.Init() after creating the screen.
The second is that your call to SetContent is misguided. The string you are passing is to accommodate combining characters. Instead you need to call SetContent 5 times (one for each letter of "hello") with a different offset, and the appropriate letter of "hello". You probably want to just pass "" for the 4th argument (the string), since none of this is combining characters..
The third problem is that your program just exits. On most terminals this will cause the reset of the terminal to occur, losing your output. (On xterm, for example, tcell uses the alternate screen buffer by default, which leads to exit causing the contents of that screen to be lost, when it switches back to the primary screen buffer at program termination.) The simplest way to prove this is to add a time.Sleep(time.Second * 10) or similar as the last line of your program.
Here is the modified code:
import (
"fmt"
"github.com/gdamore/tcell"
"github.com/gdamore/tcell/encoding"
"os"
"time"
)
func main() {
encoding.Register()
scn, err := tcell.NewScreen()
scn.Init()
if err != nil {
fmt.Fprintf(os.Stderr, "%v\n", err)
os.Exit(1)
}
scn.Clear()
scn.SetContent(10, 10, rune('h'), []rune(""), tcell.StyleDefault)
scn.Show()
time.Sleep(time.Second * 2)
}
Hope this help.
I have been trying to run a command and parse the output in golang. Here is a sample of what I am trying to do:
package main
import (
"fmt"
"os/exec"
)
func main() {
out,err := exec.Command("ls -ltr").Output()
if err != nil {
fmt.Println("Error: %s", err)
}
fmt.Printf("%s",out)
}
Now, when I am trying to run "ls -ltr", I get this error:
Error: %s exec: "ls -ltr": executable file not found in $PATH
So, basically go is looking for whole "ls -ltr" in PATH. And it's not there obviously. Is there any way I can pass a flag to any argument?TIA.
You pass arguments to the program by passing more arguments to the function - it's variadic:
out,err := exec.Command("ls","-ltr").Output()
https://golang.org/pkg/os/exec/#Command
This is a pretty common convention with exec-style functions which you will see in most languages. The other common pattern is builders.
Sometimes the layout of arguments you need to pass won't be known at compile-time (though it's not a good idea to send arbitrary commands to the system - stay safe!). If you want to pass an unknown number of arguments, you can use an array with some special syntax:
// Populate myArguments however you like
myArguments := []string{"bar","baz"}
// Pass myArguments with "..." to use variadic behaviour
out,err := exec.Command("foo", myArguments...).Output()
I'm trying to cross compile a Go program that will execute a bash script. Is it possible to embed the bash script in the binary?
I've referenced:
Golang serve static files from memory
Not sure if this applies to executing bash scripts though. Am I missing something here? Some clarification or pointers will be very helpful, thanks!
Since bash can execute scripts from stdin, you just send your script to the bash command via command.Stdin.
An example without go embed:
package main
import (
"bytes"
"fmt"
"os/exec"
"strings"
)
var script = `
echo $PWD
pwd
echo "-----------------------------------"
# print process
ps aux | grep code
`
func main() {
c := exec.Command("bash")
c.Stdin = strings.NewReader(script)
b, e := c.Output()
if e != nil {
fmt.Println(e)
}
fmt.Println(string(b))
}
With go 1.16 embed (https://golang.org/pkg/embed/):
package main
import (
"bytes"
_ "embed"
"fmt"
"os/exec"
"strings"
)
//go:embed script.sh
var script string
func main() {
c := exec.Command("bash")
c.Stdin = strings.NewReader(script)
b, e := c.Output()
if e != nil {
fmt.Println(e)
}
fmt.Println(string(b))
}
Bonus
Passing parameters to your script with -s -.
The following example will pass -la /etc to the script.
package main
import (
"fmt"
"os/exec"
"strings"
)
func main() {
// pass parameters to your script as a safe way.
c := exec.Command("sh", "-s", "-", "-la", "/etc")
// use $1, $2, ... $# as usual
c.Stdin = strings.NewReader(`
echo $#
ls $1 "$2"
`)
b, e := c.Output()
if e != nil {
fmt.Println(e)
}
fmt.Println(string(b))
}
Playground: https://go.dev/play/p/T1lMSrXcOIL
You can actually directly interface with the system shell in Go. Depending on what's in your bash script you can probably convert everything completely to go. For example things like handling files, extracting archives, outputting text, asking for user input, downloading files, and so much more can be done natively in Go. For anything you absolutely need the shell for you can always use golang.org/pkg/os/exec.
I wrote a snippet that demonstrates a really simple Go based command shell. Basically it pipes input, output, and error between the user and the shell. It can be used interactively or to directly run most shell commands. I'm mentioning it here mostly to demonstrate Go's OS capabilities. Check it out: github.com/lee8oi/goshell.go
Did you try writing the stream-data (according to the reference go-bindata provides a function that returns []byte) into a temporary file?
see: http://golang.org/pkg/io/ioutil/#TempFile
you can then execute it with a syscall
http://golang.org/pkg/syscall/#Exec
where the first argument needs to be a shell.
I have an application that usually runs silent in the background, so I compile it with
go build -ldflags -H=windowsgui <gofile>
To check the version at the command line, I wanted to pass a -V flag to the command line to get the string holding the version to be printed to the command prompt then have the application exit. I added the flag package and code. When I test it with
go run <gofile> -V
...it prints the version fine. When I compile the exe, it just exits, printing nothing. I suspect it's the compilation flag causing it to not access the console and sending my text into the bit bucket.
I've tried variations to print to stderr and stdout, using println and fprintf and os.stderr.write, but nothing appears from the compiled application. How should I try printing a string to the command prompt when compiled with those flags?
The problem is that when a process is created using an executable which has the "subsystem" variable in its PE header set to "Windows", the process has its three standard handles closed and it is not associated with any console—no matter if you run it from the console or not. (In fact, if you run an executable which has its subsystem set to "console" not from a console, a console is forcibly created for that process and the process is attached to it—you usually see it as a console window popping up all of a sudden.)
Hence, to print anything to the console from a GUI process on Windows you have to explicitly connect that process to the console which is attached to its parent process (if it has one), like explained here for instance. To do this, you call the AttachConsole API function. With Go, this can be done using the syscall package:
package main
import (
"fmt"
"syscall"
)
const (
ATTACH_PARENT_PROCESS = ^uint32(0) // (DWORD)-1
)
var (
modkernel32 = syscall.NewLazyDLL("kernel32.dll")
procAttachConsole = modkernel32.NewProc("AttachConsole")
)
func AttachConsole(dwParentProcess uint32) (ok bool) {
r0, _, _ := syscall.Syscall(procAttachConsole.Addr(), 1, uintptr(dwParentProcess), 0, 0)
ok = bool(r0 != 0)
return
}
func main() {
ok := AttachConsole(ATTACH_PARENT_PROCESS)
if ok {
fmt.Println("Okay, attached")
}
}
To be truly complete, when AttachConsole() fails, this code should probably take one of these two routes:
Call AllocConsole() to get its own console window created for it.
It'd say this is pretty much useless for displaying version information as the process usually quits after printing it, and the resulting user experience will be a console window popping up and immediately disappearing; power users will get a hint that they should re-run the application from the console but mere mortals won't probably cope.
Post a GUI dialog displaying the same information.
I think this is just what's needed: note that displaying help/usage messages in response to the user specifying some command-line argument is quite often mentally associated with the console, but this is not a dogma to follow: for instance, try running msiexec.exe /? at the console and see what happens.
One problem with the solutions already posted here is that they redirect all output to the console, so if I run ./myprogram >file, the redirection to file gets lost. I've written a new module, github.com/apenwarr/fixconsole, that avoids this problem. You can use it like this:
import (
"fmt"
"github.com/apenwarr/fixconsole"
"os"
)
func main() {
err := fixconsole.FixConsoleIfNeeded()
if err != nil {
fmt.Fatalf("FixConsoleOutput: %v\n", err)
}
os.Stdout.WriteString(fmt.Sprintf("Hello stdout\n"))
os.Stderr.WriteString(fmt.Sprintf("Hello stderr\n"))
}
Answer above was helpful but alas it did not work for me out of the box. After some additional research I came to this code:
// go build -ldflags -H=windowsgui
package main
import "fmt"
import "os"
import "syscall"
func main() {
modkernel32 := syscall.NewLazyDLL("kernel32.dll")
procAllocConsole := modkernel32.NewProc("AllocConsole")
r0, r1, err0 := syscall.Syscall(procAllocConsole.Addr(), 0, 0, 0, 0)
if r0 == 0 { // Allocation failed, probably process already has a console
fmt.Printf("Could not allocate console: %s. Check build flags..", err0)
os.Exit(1)
}
hout, err1 := syscall.GetStdHandle(syscall.STD_OUTPUT_HANDLE)
hin, err2 := syscall.GetStdHandle(syscall.STD_INPUT_HANDLE)
if err1 != nil || err2 != nil { // nowhere to print the error
os.Exit(2)
}
os.Stdout = os.NewFile(uintptr(hout), "/dev/stdout")
os.Stdin = os.NewFile(uintptr(hin), "/dev/stdin")
fmt.Printf("Hello!\nResult of console allocation: ")
fmt.Printf("r0=%d,r1=%d,err=%s\nFor Goodbye press Enter..", r0, r1, err0)
var s string
fmt.Scanln(&s)
os.Exit(0)
}
The key point: after allocating/attaching the console, there is need to get stdout handle, open file using this handle and assign it to os.Stdout variable. If you need stdin you have to repeat the same for stdin.
You can get the desired behavior without using -H=windowsgui; you'd basically create a standard app (with its own console window), and hide it until the program exits.
func Console(show bool) {
var getWin = syscall.NewLazyDLL("kernel32.dll").NewProc("GetConsoleWindow")
var showWin = syscall.NewLazyDLL("user32.dll").NewProc("ShowWindow")
hwnd, _, _ := getWin.Call()
if hwnd == 0 {
return
}
if show {
var SW_RESTORE uintptr = 9
showWin.Call(hwnd, SW_RESTORE)
} else {
var SW_HIDE uintptr = 0
showWin.Call(hwnd, SW_HIDE)
}
}
And then use it like this:
func main() {
Console(false)
defer Console(true)
...
fmt.Println("Hello World")
...
}
If you build a windowless app you can get output with PowerShell command Out-String
.\\main.exe | out-string
your build command may look like:
cls; go build -i -ldflags -H=windowsgui main.go; .\\main.exe | out-string;
or
cls; go run -ldflags -H=windowsgui main.go | out-string
No tricky syscalls nor kernel DLLs needed!