Why operate normally when bufio.NewWriter get os.Stdin in local - go

package main
import (
"bufio"
"os"
)
func main() {
bw := bufio.NewWriter(os.Stdin)
bw2 := bufio.NewWriter(os.Stdout)
bw.WriteString("Hello, world 1\n")
bw2.WriteString("Hello, world 2\n")
bw.Flush()
bw2.Flush()
}
This code show both string in a local environment.
But why does it work differently in different environments?
My local environment
OS : macOS 12.6
go : go1.19.2 darwin/amd64
ide : vscode
on my local machine :
$ go run myworkspace/main/main.go
Hello, world 1
Hello, world 2
on the playground :
# in the 'Output' section
---
Hello, world 2
Program exited.

The Go Playground
https://go.dev/play/p/PtoDwCZGggd
Scroll all the way down to "About the Playground"
About the Playground
The Go Playground runs the program inside a sandbox.
There are limitations to the programs that can be run in the playground:
The only communication a playground program has to the outside world is by writing to standard output and standard error.
The Go Playground discards writes to standard input.
Locally, os.Stdin is a file.
var (
Stdin = NewFile(uintptr(syscall.Stdin), "/dev/stdin")
Stdout = NewFile(uintptr(syscall.Stdout), "/dev/stdout")
Stderr = NewFile(uintptr(syscall.Stderr), "/dev/stderr")
)
Stdin, Stdout, and Stderr are open Files pointing to the standard input, standard output, and standard error file descriptors.

The bottom line is : your program does not control what thingy will be used as Stdin and Stdout when it is started.
On your local machine, try running : echo "" | go run main.go or go run main.go < anyfile.
(also : you should check for errors in your go code)
It is a standard setup, when running a command from a terminal, that the shell uses the same file descriptor, which points to a read/write terminal, for stdin and stdout/stderr. With that setup, writing to stdin "works".
But you can't rely on it (you shouldn't, actually), and it turns out that the playground doesn't start its processes that way :
package main
import (
"fmt"
"os"
)
func main() {
_, err := os.Stdin.Write([]byte("Hello\n"))
if err != nil {
fmt.Fprintln(os.Stderr, "*** error:", err)
}
}
// Output:
// *** error: write /dev/stdin: bad file descriptor
//
// Program exited.
https://go.dev/play/p/NmxgOsK2ovp
see also :
Why is it possible to write() to STDIN?
Writing to stdin and reading from stdout (UNIX/LINUX/C Programming)
(found by googling "writing to stdin")

Related

Writing to CON (Windows console) doesn't print to STDOUT (console)

I understood that the CON name is equivalent to /dev/tty in Linux.
But when I used a program that write to this file, it doesn't print anything, it works only on Linux.
I think I am not using it correctly.
IMPORTANT: I am not looking for workarounds like fmt.Println, I need to be able to print to this file, like I did in Linux (/dev/tty).
This is the program:
package main
import (
"fmt"
"os"
"runtime"
)
func main() {
var ttyName string
if runtime.GOOS == "windows" {
fmt.Println("*** Using `con`")
ttyName = "con"
} else {
fmt.Println("*** Using `/dev/tty`")
ttyName = "/dev/tty"
}
f, _ := os.OpenFile(ttyName, os.O_WRONLY, 0644)
fmt.Fprintln(f, "*** Stdout redirected")
}
The program prints "*** Stdout redirected" in Linux but not in Windows.
The problem is related to the ttyName.
I used con as the name but it seems that it doesn't work.
I also tried: "con:", "\\\\.\\con", "\\\\.\\con:", "\\\\?\\con", and conout.
How can I use it to print to the console?
I took some of the ideas from this website:
https://www.mkssoftware.com/docs/man5/dev_console.5.asp
Make sure you test this with Windows Command Prompt as CON might not work as expected with 3rd party terminal emulators (like embedded ones in IDE, Hyper.js, etc.).
Options you listed should work, CON (legacy DOS name) or \\.\CON (UNC name) in uppercase are safe bet:
f, _ := os.OpenFile("CON", os.O_WRONLY, 0644)
f, _ := os.OpenFile("\\\\.\\CON", os.O_WRONLY, 0644)

Reading from stdout pipe once ready in golang

I'm facing with a weird golang issue. The following code will clarify:
package main
import (
"os/exec"
"io"
"fmt"
"os"
)
var (
pw io.WriteCloser
pr io.ReadCloser
)
func main() {
term := exec.Command("/bin/sh")
// Get stdin writer pipe
pw, _ = term.StdinPipe()
pr, _ = term.StdoutPipe()
term.Start()
run("cd ~")
pwd := run("pwd");
// Do something with pwd output
...
term.Wait()
}
func run(c string) string {
io.WriteString(pw, fmt.Sprintln(c))
buf := make([]byte, 32 * 1024)
pr.Read(buf)
return string(buf)
}
I'd like to run some commands in a shell env and read their output. There's no problem on write/run command but it seems that there're some limitations while reading:
you can't know if a command doesn't output anything or not;
there's no way to check if stdout is ready to be read or not.
The pr.Read(dest) method will block the code flow until something is read from stdout. As said, the goal is to read sequentially (without using a go routine and/or an infinite loop). This means that if we send a cd command the func end is never reached.
Setting the non-block flag through unix.SetNonblock on stdout file descriptor seems to solve the above issue but you can't know prior if it's ready or not and an error saying "resource temporary not available" is returned from .Read call.
As Cerise Limón mentioned go functions whould be the way to go here, since these sorts of interactive scripting exercises are traditionally done with expect.
You can wrap the the parrellel execution into a library to it might still look like sequencial code, so this might be helpful: https://github.com/ThomasRooney/gexpect
From the readme:
child, err := gexpect.Spawn("python")
if err != nil { panic(err) }
child.Expect(">>>")
child.SendLine("print 'Hello World'")
child.Interact()
child.Close()

How can stdout be captured or suppressed for Go(lang) testing?

How can stdout be captured or suppressed for Go testing?
I am trying to teach myself go(lang) testing. In the code below, myshow.LoadPath prints lots of information to stdout (which is a normal side effect). It does however make for very noisy output when I run "go test" Is there a way to suppress or capture stdout?
For comparison, I'm thinking about something like this from the python world. http://pytest.org/latest/capture.html#captures
package slideshow_test
import (
"os"
"testing"
"github.com/golliher/go-hpf/slideshow"
)
func setupTest() {
myshow := slideshow.Slideshow{Name: "This is my show"}
myshow.LoadPath("..")
}
func TestStub(t *testing.T) {
if true == false {
t.Fail()
}
}
func TestMain(m *testing.M) {
setupTest()
os.Exit(m.Run())
}
os.Stdout which is used by the fmt.Printf and others is just a variable. So you can overwrite it at any time and restore it back when necessary. https://golang.org/pkg/os/#pkg-variables
To suppress the output during the test I use the following code. I fixes output as well as logging. After test is done it resets the output streams.
func TestStartStowWrongCommand(t *testing.T) {
defer quiet()()
...
}
func quiet() func() {
null, _ := os.Open(os.DevNull)
sout := os.Stdout
serr := os.Stderr
os.Stdout = null
os.Stderr = null
log.SetOutput(null)
return func() {
defer null.Close()
os.Stdout = sout
os.Stderr = serr
log.SetOutput(os.Stderr)
}
}
The output can be suppressed
by running the tests with go test .:
$ go help test
Go test runs in two different modes: local
directory mode when invoked with no package arguments (for example,
'go test'), and package list mode when invoked with package arguments
(for example 'go test math', 'go test ./...', and even 'go test .').
In local directory mode, go test compiles and tests the package
sources found in the current directory and then runs the resulting
test binary. In this mode, caching (discussed below) is disabled.
After the package test finishes, go test prints a summary line showing
the test status ('ok' or 'FAIL'), package name, and elapsed time.
In package list mode, go test compiles and tests each of the packages
listed on the command line. If a package test passes, go test prints
only the final 'ok' summary line. If a package test fails, go test
prints the full test output. If invoked with the -bench or -v flag, go
test prints the full output even for passing package tests, in order
to display the requested benchmark results or verbose logging.
Not exactly what you are asking for but still might be helpful.
You can use t.Log (http://golang.org/pkg/testing/#T.Log) and t.Logf (http://golang.org/pkg/testing/#T.Logf) methods in the test method. The output will be printed only if the test fails or the -test.v flag is set.
I also would suggest to use log package to print to the console in myshow.LoadPath. Then you can disable (or capture) the output in the test by setting custom writer using log.SetOutput

Go: embed bash script in binary

I'm trying to cross compile a Go program that will execute a bash script. Is it possible to embed the bash script in the binary?
I've referenced:
Golang serve static files from memory
Not sure if this applies to executing bash scripts though. Am I missing something here? Some clarification or pointers will be very helpful, thanks!
Since bash can execute scripts from stdin, you just send your script to the bash command via command.Stdin.
An example without go embed:
package main
import (
"bytes"
"fmt"
"os/exec"
"strings"
)
var script = `
echo $PWD
pwd
echo "-----------------------------------"
# print process
ps aux | grep code
`
func main() {
c := exec.Command("bash")
c.Stdin = strings.NewReader(script)
b, e := c.Output()
if e != nil {
fmt.Println(e)
}
fmt.Println(string(b))
}
With go 1.16 embed (https://golang.org/pkg/embed/):
package main
import (
"bytes"
_ "embed"
"fmt"
"os/exec"
"strings"
)
//go:embed script.sh
var script string
func main() {
c := exec.Command("bash")
c.Stdin = strings.NewReader(script)
b, e := c.Output()
if e != nil {
fmt.Println(e)
}
fmt.Println(string(b))
}
Bonus
Passing parameters to your script with -s -.
The following example will pass -la /etc to the script.
package main
import (
"fmt"
"os/exec"
"strings"
)
func main() {
// pass parameters to your script as a safe way.
c := exec.Command("sh", "-s", "-", "-la", "/etc")
// use $1, $2, ... $# as usual
c.Stdin = strings.NewReader(`
echo $#
ls $1 "$2"
`)
b, e := c.Output()
if e != nil {
fmt.Println(e)
}
fmt.Println(string(b))
}
Playground: https://go.dev/play/p/T1lMSrXcOIL
You can actually directly interface with the system shell in Go. Depending on what's in your bash script you can probably convert everything completely to go. For example things like handling files, extracting archives, outputting text, asking for user input, downloading files, and so much more can be done natively in Go. For anything you absolutely need the shell for you can always use golang.org/pkg/os/exec.
I wrote a snippet that demonstrates a really simple Go based command shell. Basically it pipes input, output, and error between the user and the shell. It can be used interactively or to directly run most shell commands. I'm mentioning it here mostly to demonstrate Go's OS capabilities. Check it out: github.com/lee8oi/goshell.go
Did you try writing the stream-data (according to the reference go-bindata provides a function that returns []byte) into a temporary file?
see: http://golang.org/pkg/io/ioutil/#TempFile
you can then execute it with a syscall
http://golang.org/pkg/syscall/#Exec
where the first argument needs to be a shell.

Printing output to a command window when golang application is compiled with -ldflags -H=windowsgui

I have an application that usually runs silent in the background, so I compile it with
go build -ldflags -H=windowsgui <gofile>
To check the version at the command line, I wanted to pass a -V flag to the command line to get the string holding the version to be printed to the command prompt then have the application exit. I added the flag package and code. When I test it with
go run <gofile> -V
...it prints the version fine. When I compile the exe, it just exits, printing nothing. I suspect it's the compilation flag causing it to not access the console and sending my text into the bit bucket.
I've tried variations to print to stderr and stdout, using println and fprintf and os.stderr.write, but nothing appears from the compiled application. How should I try printing a string to the command prompt when compiled with those flags?
The problem is that when a process is created using an executable which has the "subsystem" variable in its PE header set to "Windows", the process has its three standard handles closed and it is not associated with any console—no matter if you run it from the console or not. (In fact, if you run an executable which has its subsystem set to "console" not from a console, a console is forcibly created for that process and the process is attached to it—you usually see it as a console window popping up all of a sudden.)
Hence, to print anything to the console from a GUI process on Windows you have to explicitly connect that process to the console which is attached to its parent process (if it has one), like explained here for instance. To do this, you call the AttachConsole API function. With Go, this can be done using the syscall package:
package main
import (
"fmt"
"syscall"
)
const (
ATTACH_PARENT_PROCESS = ^uint32(0) // (DWORD)-1
)
var (
modkernel32 = syscall.NewLazyDLL("kernel32.dll")
procAttachConsole = modkernel32.NewProc("AttachConsole")
)
func AttachConsole(dwParentProcess uint32) (ok bool) {
r0, _, _ := syscall.Syscall(procAttachConsole.Addr(), 1, uintptr(dwParentProcess), 0, 0)
ok = bool(r0 != 0)
return
}
func main() {
ok := AttachConsole(ATTACH_PARENT_PROCESS)
if ok {
fmt.Println("Okay, attached")
}
}
To be truly complete, when AttachConsole() fails, this code should probably take one of these two routes:
Call AllocConsole() to get its own console window created for it.
It'd say this is pretty much useless for displaying version information as the process usually quits after printing it, and the resulting user experience will be a console window popping up and immediately disappearing; power users will get a hint that they should re-run the application from the console but mere mortals won't probably cope.
Post a GUI dialog displaying the same information.
I think this is just what's needed: note that displaying help/usage messages in response to the user specifying some command-line argument is quite often mentally associated with the console, but this is not a dogma to follow: for instance, try running msiexec.exe /? at the console and see what happens.
One problem with the solutions already posted here is that they redirect all output to the console, so if I run ./myprogram >file, the redirection to file gets lost. I've written a new module, github.com/apenwarr/fixconsole, that avoids this problem. You can use it like this:
import (
"fmt"
"github.com/apenwarr/fixconsole"
"os"
)
func main() {
err := fixconsole.FixConsoleIfNeeded()
if err != nil {
fmt.Fatalf("FixConsoleOutput: %v\n", err)
}
os.Stdout.WriteString(fmt.Sprintf("Hello stdout\n"))
os.Stderr.WriteString(fmt.Sprintf("Hello stderr\n"))
}
Answer above was helpful but alas it did not work for me out of the box. After some additional research I came to this code:
// go build -ldflags -H=windowsgui
package main
import "fmt"
import "os"
import "syscall"
func main() {
modkernel32 := syscall.NewLazyDLL("kernel32.dll")
procAllocConsole := modkernel32.NewProc("AllocConsole")
r0, r1, err0 := syscall.Syscall(procAllocConsole.Addr(), 0, 0, 0, 0)
if r0 == 0 { // Allocation failed, probably process already has a console
fmt.Printf("Could not allocate console: %s. Check build flags..", err0)
os.Exit(1)
}
hout, err1 := syscall.GetStdHandle(syscall.STD_OUTPUT_HANDLE)
hin, err2 := syscall.GetStdHandle(syscall.STD_INPUT_HANDLE)
if err1 != nil || err2 != nil { // nowhere to print the error
os.Exit(2)
}
os.Stdout = os.NewFile(uintptr(hout), "/dev/stdout")
os.Stdin = os.NewFile(uintptr(hin), "/dev/stdin")
fmt.Printf("Hello!\nResult of console allocation: ")
fmt.Printf("r0=%d,r1=%d,err=%s\nFor Goodbye press Enter..", r0, r1, err0)
var s string
fmt.Scanln(&s)
os.Exit(0)
}
The key point: after allocating/attaching the console, there is need to get stdout handle, open file using this handle and assign it to os.Stdout variable. If you need stdin you have to repeat the same for stdin.
You can get the desired behavior without using -H=windowsgui; you'd basically create a standard app (with its own console window), and hide it until the program exits.
func Console(show bool) {
var getWin = syscall.NewLazyDLL("kernel32.dll").NewProc("GetConsoleWindow")
var showWin = syscall.NewLazyDLL("user32.dll").NewProc("ShowWindow")
hwnd, _, _ := getWin.Call()
if hwnd == 0 {
return
}
if show {
var SW_RESTORE uintptr = 9
showWin.Call(hwnd, SW_RESTORE)
} else {
var SW_HIDE uintptr = 0
showWin.Call(hwnd, SW_HIDE)
}
}
And then use it like this:
func main() {
Console(false)
defer Console(true)
...
fmt.Println("Hello World")
...
}
If you build a windowless app you can get output with PowerShell command Out-String
.\\main.exe | out-string
your build command may look like:
cls; go build -i -ldflags -H=windowsgui main.go; .\\main.exe | out-string;
or
cls; go run -ldflags -H=windowsgui main.go | out-string
No tricky syscalls nor kernel DLLs needed!

Resources