I am attempting to write/print text to the screen from a Go program launched from another console/terminal application--a "door" program that launches from an old-school Bulletin Board System (BBS).
The BBS itself runs over a telnet connection, localhost:2323. And when launching my program, the BBS automatically adds the correct socket handle as an argument, which I can then read using Flag (it's an integer, like 236).
Obviously, in Linux, I'd just use fmt.Println("Hello World!") using os.Stdout... But on Windows, I need to somehow pipe/redirect the Go program's output to the provided socket.
Here's the function I started with:
func writeOut(fd int, buf []byte) bool {
for len(buf) > 0 {
n, err := syscall.Write(syscall.Handle(fd), buf)
if err != nil {
fmt.Println(err)
return false
}
buf = buf[n:]
}
return true
}
called from:
writeOut(socketInt, []byte("Writing to Windows socket..."))
The error returned is: The parameter is incorrect
What am I doing wrong, and how would this be accomplished in Go?
You can't arbitrarily pass file or socket handles to another process that isn't inheriting them from your process in the first place. Each process has its own unique set of handles. In POSIX inheriting socket handles is possible (albeit not recommended) but in Windows they simply cannot be inherited (see Are TCP SOCKET handles inheritable?).
You can redirect stdout to a TCP socket when calling CreateProcess though, so that when invoked, your program can indeed fmt.Println to stdout and the output would go straight to the socket:
func serveDoor(conn *net.TCPConn, name string, args ...string) {
defer conn.Close()
cmd := exec.Command(name, args...)
cmd.Stdin = conn
cmd.Stdout = conn
cmd.Stderr = conn
err := cmd.Run()
fmt.Println("door finished:", err)
}
(full gist)
Another solution is to use a pipe and pump it to the socket.
Related
How to read/manipulate the input from a connection that is passed to a command stdin?
For example, given the following code.
c, _ := net.Dial("tcp", somehost)
cmd := exec.Command("/bin/sh")
cmd.Stdin, cmd.Stdout, cmd.Stderr = c, c, c
cmd.Run()
How would it be possible to reverse the string from the connection before it is passed to the cmd.Stdin or how could I parse the string and not pass it on to cmd.Stdin?
Ive considered reading from the connection with bufio and then passing it to Command second argument, the params, but I was hoping for a better solution that does not require me to handle all the different cases for args input in a command, but instead just passing it on to Stdin after analysing the input
Ok since you mentioned in comments that "my real issue is how to intercept the input from the connection, parse it and parse it to the Stdin of the command. Seems when I do cmd.Run() I block and hence cant really continously parse"
Here is how I will do it:
import (
"io"
"os"
"os/exec"
)
func main() {
//All errors are not checked
cmd := exec.Command("/bin/sh")
cmdStdin, _ := cmd.StdinPipe()
go func() {
defer cmdStdin.Close()
//here you will need to loop on reading the connection,
//for simplicity lets assume you do that & receive data
//let says you got ls from connection
cmdStdin.Write([]byte("ls\n"))
}()
cmdStdout, _ := cmd.StdoutPipe()
go io.Copy(os.Stdout, cmdStdout)
cmd.Run()
}
I encounter a weird behavior with exec.Wait() with a modified Stdin.
I'm just modifying Stdin in order to be able to duplicate its content, count the amount of data… but that's not the problem here.
I've made this stripped down program just to demonstrate the strange behavior :
with a modified Stdin, cmd.Wait() waits indefinitely… until I press "enter" or "^C"
with an unmodified Stdin (uncomment the line cmd.Stdin = os.Stdin), the program processes to the end flawlessly.
when I launch this program (with a modified Stdin) with Delve (dlv debug), the program processes to the end flawlessly !
I also added a time.Sleep of 30 seconds between cmd.Start() and cmd.Wait(), and then attached the program to Delve (dlv attach PID). When I enter continue, cmd.Wait() waits indefinitely… until I press "enter" or "^C"
I tested these behaviours with go1.11 and go1.12
package main
import (
"fmt"
"os"
"os/exec"
)
type Splitter struct {
f *os.File
fd int
}
func NewSplitter(f *os.File) *Splitter {
return &Splitter{f, int(f.Fd())}
}
func (s *Splitter) Close() error {
return s.f.Close()
}
func (s *Splitter) Read(p []byte) (int, error) {
return s.f.Read(p)
}
func (s *Splitter) Write(p []byte) (int, error) {
return s.f.Write(p)
}
func main() {
var cmd *exec.Cmd
cmd = exec.Command("cat", "foobarfile")
cmd.Stdin = NewSplitter(os.Stdin)
//cmd.Stdin = os.Stdin
cmd.Stdout = NewSplitter(os.Stdout)
cmd.Stderr = NewSplitter(os.Stderr)
cmd.Start()
cmd.Wait()
fmt.Println("done")
}
Is there something I'm doing wrong ?
Thanks for your help.
This program duplicates the content as you asked. You can although try the commented part as well. And the comments are self - explanatory, I hope it explains your query.
package main
import (
"io"
"log"
"os"
"os/exec"
)
func main() {
// Execute cat command w/ arguments
// cmd := exec.Command("cat", "hello.txt")
// Execute cat command w/o arguments
cmd := exec.Command("cat")
// Attach STDOUT stream
stdout, err := cmd.StdoutPipe()
if err != nil {
log.Println(err)
}
// Attach STDIN stream
stdin, err := cmd.StdinPipe()
if err != nil {
log.Println(err)
}
// Attach STDERR stream
stderr, err := cmd.StderrPipe()
if err != nil {
log.Println(err)
}
// Spawn go-routine to copy os's stdin to command's stdin
go io.Copy(stdin, os.Stdin)
// Spawn go-routine to copy command's stdout to os's stdout
go io.Copy(os.Stdout, stdout)
// Spawn go-routine to copy command's stderr to os's stderr
go io.Copy(os.Stderr, stderr)
// Run() under the hood calls Start() and Wait()
cmd.Run()
// Note: The PIPES above will be closed automatically after Wait sees the command exit.
// A caller need only call Close to force the pipe to close sooner.
log.Println("Command complete")
}
You are replacing the process file descriptors, which are normally *os.File, with other Go types. In order for stdin to act like a stream, the os/exec package needs to launch a goroutine to copy the data between the io.Reader and the process. This is documented in the os/exec package:
// Otherwise, during the execution of the command a separate
// goroutine reads from Stdin and delivers that data to the command
// over a pipe. In this case, Wait does not complete until the goroutine
// stops copying, either because it has reached the end of Stdin
// (EOF or a read error) or because writing to the pipe returned an error.
If you look at the stack trace from your program, you'll see that it is waiting for the io goroutines to complete in Wait():
goroutine 1 [chan receive]:
os/exec.(*Cmd).Wait(0xc000076000, 0x0, 0x0)
/usr/local/go/src/os/exec/exec.go:510 +0x125
main.main()
Because you are now in control of the data stream, it is up to you to close it as necessary. If Stdin is not necessary here, then don't assign it at all. If it is going to be used, then you must Close() it to have Wait() return.
Another option is to ensure that you are using an *os.File, which the easiest method is to use the StdinPipe, StdoutPipe and StderrPipe methods, which in turn use os.Pipe(). This way ensures that the process is dealing only with *os.File, and not with other Go types.
There is a client and a server that communicates through stdio. I think I am basically confused about the stdin and stdout. I have some questions about the stdio.
Does the server read the request from stdin or stdout where the client writes to?
Does the server write response to stdin or stdout where the client can read?
Below is the code snippet of connection part on server side.
case "stdio":
log.Println("server: reading on stdin, writing on stdout")
<-jsonrpc2.NewConn(context.Background(), jsonrpc2.NewBufferedStream(stdrwc{}, jsonrpc2.VSCodeObjectCodec{}), handler, connOpt...).DisconnectNotify()
log.Println("connection closed")
return nil
type stdrwc struct{}
func (stdrwc) Read(p []byte) (int, error) {
return os.Stdin.Read(p)
}
func (stdrwc) Write(p []byte) (int, error) {
return os.Stdout.Write(p)
}
func (stdrwc) Close() error {
if err := os.Stdin.Close(); err != nil {
return err
}
return os.Stdout.Close()
}
It's hard to say what this program is doing (as there's just a portion of it). Looks like you have an implementation of ReadWriteCloser that reads from stdin and writes to stdout (and a portion of a switch statement).
Generally any program can read from stdin and write to stdout (and stderr). You can link stdout of one program to stdin of other program with a pipe (e.g. client | server), but that's unidirectional. In your case, it sounds like you want client's stdin to go to server's stdout and vice versa. In local development, Unix sockets are usually used for that, but you might be able to create a named pipe (with mkfifo) like shown here.
Also, it might be easier to start with a super simple toy program, that doesn't include jsonrpc2 and any other packages.
I hope that helps!
I have a Go program hosting a simple HTTP service on localhost:8080 so I can connect my public nginx host to it via the proxy_pass directive, as a reverse proxy to serve part of my site's requests. This is all working great, no problems there.
I want to convert the Go program to host the HTTP service on a Unix domain socket instead of a local TCP socket for improved security and to reduce the unnecessary protocol overhead of TCP.
PROBLEM:
The problem is that Unix domain sockets cannot be reused once they are bind() to, even after program termination. The second time (and every time after) I run the Go program it exits with a fatal error "address already in use".
Common practice is to unlink() Unix domain sockets (i.e. remove the file) when the server shuts down. However, this is tricky in Go. My first attempt was to use the defer statement in my main func (see below), but it is not getting run if I interrupt the process with a signal like CTRL-C. I suppose this is to be expected. Disappointing, but not unexpected.
QUESTION: Is there a best practice on how to unlink() the socket when the server process shuts down (either gracefully or ungracefully)?
Here's part of my func main() that starts the server listening for reference:
// Create the HTTP server listening on the requested socket:
l, err := net.Listen("unix", "/tmp/mysocket")
if err != nil {
log.Fatal(err)
} else {
// Unix sockets must be unlink()ed before being reused again.
// Unfortunately, this defer is not run when a signal is received, e.g. CTRL-C.
defer func() {
os.Remove("/tmp/mysocket")
}()
log.Fatal(http.Serve(l, http.HandlerFunc(indexHtml)))
}
Here is the complete solution I used. The code I posted in my question was a simplified version for clear demonstration purposes.
// Create the socket to listen on:
l, err := net.Listen(socketType, socketAddr)
if err != nil {
log.Fatal(err)
return
}
// Unix sockets must be unlink()ed before being reused again.
// Handle common process-killing signals so we can gracefully shut down:
sigc := make(chan os.Signal, 1)
signal.Notify(sigc, os.Interrupt, os.Kill, syscall.SIGTERM)
go func(c chan os.Signal) {
// Wait for a SIGINT or SIGKILL:
sig := <-c
log.Printf("Caught signal %s: shutting down.", sig)
// Stop listening (and unlink the socket if unix type):
l.Close()
// And we're done:
os.Exit(0)
}(sigc)
// Start the HTTP server:
log.Fatal(http.Serve(l, http.HandlerFunc(indexHtml)))
I sure hope this is good and effective Go code that would make the Go authors proud. It certainly looks so to me. If it is not, that would be embarrassing on my part. :)
For anyone curious, this is part of https://github.com/JamesDunne/go-index-html which is a simple HTTP directory listing generator with some extra features that web servers don't give you out of the box.
You can end your main func with the signal handler and spawn separate go routines for your other tasks instead. That way, you can leverage the defer mechanism and handle all (signal-based or not) shut downs cleanly:
func main() {
// Create the HTTP server listening on the requested socket:
l, err := net.Listen("unix", "/tmp/mysocket")
if err != nil {
log.Fatal(err)
return
}
// Just work with defer here; this works as long as the signal handling
// happens in the main Go routine.
defer l.Close()
// Make sure the server does not block the main
go func() {
log.Fatal(http.Serve(l, http.HandlerFunc(indexHtml)))
}()
// Use a buffered channel so we don't miss any signals
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, os.Kill, syscall.SIGTERM)
// Block until a signal is received.
s := <-c
fmt.Println("Got signal:", s)
// ...and exit, running all the defer statements
}
In modern Go, you may use the syscall.Unlink() - docs here:
import (
"net"
"syscall"
...
)
...
socketpath := "/tmp/somesocket"
// carry on with your socket creation:
addr, err := net.ResolveUnixAddr("unixgram", socketpath)
if err != nil {
return err;
}
// always remove the named socket from the fs if its there
err = syscall.Unlink(socketpath)
if err != nil {
// not really important if it fails
log.Error("Unlink()",err)
}
// carry on with socket bind()
conn, err := net.ListenUnixgram("unixgram", addr);
if err != nil {
return err;
}
given the following example:
// test.go
package main
import (
"fmt"
"os/exec"
)
func main() {
cmd := exec.Command("login")
in, _ := cmd.StdinPipe()
in.Write([]byte("user"))
out, err := cmd.CombinedOutput()
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%s", out)
}
How can I detect that the process is not going to finish, because it is waiting for user input?
I'm trying to be able to run any script, but abort it if for some reason it tries to read from stdin.
Thanks!
Detecting that the process is not going to finish is a difficult problem. In fact, it is one of the classic "unsolvable" problems in Computer Science: the Halting Problem.
In general, when you are calling exec.Command and will not be passing it any input, it will cause the program to read from your OS's null device (see documentation in the exec.Cmd fields). In your code (and mine below), you explicitly create a pipe (though you should check the error return of StdinPipe in case it is not created correctly), so you should subsequently call in.Close(). In either case, the subprocess will get an EOF and should clean up after itself and exit.
To help with processes that don't handle input correctly or otherwise get themselves stuck, the general solution is to use a timeout. In Go, you can use goroutines for this:
// Set your timeout
const CommandTimeout = 5 * time.Second
func main() {
cmd := exec.Command("login")
// Set up the input
in, err := cmd.StdinPipe()
if err != nil {
log.Fatalf("failed to create pipe for STDIN: %s", err)
}
// Write the input and close
go func() {
defer in.Close()
fmt.Fprintln(in, "user")
}()
// Capture the output
var b bytes.Buffer
cmd.Stdout, cmd.Stderr = &b, &b
// Start the process
if err := cmd.Start(); err != nil {
log.Fatalf("failed to start command: %s", err)
}
// Kill the process if it doesn't exit in time
defer time.AfterFunc(CommandTimeout, func() {
log.Printf("command timed out")
cmd.Process.Kill()
}).Stop()
// Wait for the process to finish
if err := cmd.Wait(); err != nil {
log.Fatalf("command failed: %s", err)
}
// Print out the output
fmt.Printf("Output:\n%s", b.String())
}
In the code above, there are actually three main goroutines of interest: the main goroutine spawns the subprocess and waits for it to exit; a timer goroutine is sent off in the background to kill the process if it's not Stopped in time; and a goroutine that writes the output to the program when it's ready to read it.
Although this would not allow you to "detect" the program trying to read from stdin, I would just close stdin. This way, the child process will just receive an EOF when it tried to read. Most programs know how to handle a closed stdin.
// All error handling excluded
cmd := exec.Command("login")
in, _ := cmd.StdinPipe()
cmd.Start()
in.Close()
cmd.Wait()
Unfortunately, this means you can't use combined output, the following code should allow you to do the same thing. It requires you to import the bytes package.
var buf = new(bytes.Buffer)
cmd.Stdout = buf
cmd.Stderr = buf
After cmd.Wait(), you can then do:
out := buf.Bytes()
I think the solution is to run the child process with closed stdin - by adjusting the Cmd.Stdin appropriately and then Runinng it afterwards instead of using CombinedOutput().
Finally, I'm going to implement a combination of Kyle Lemons answer and forcing the new process have it's own session without a terminal attached to it, so that the executed comand will be aware that there is no terminal to read from.
// test.go
package main
import (
"log"
"os/exec"
"syscall"
)
func main() {
cmd := exec.Command("./test.sh")
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
out, err := cmd.CombinedOutput()
if err != nil {
log.Fatal("error:", err)
}
log.Printf("%s", out)
}