I am using the built-in testing module to run some functional tests I have in my GO project. In my project I have external dependencies, which I connect to in my TestMain method. I save these connections to variables which then I use in the tests themselves, and the connections can take a long time to establish properly (Kafka anyone?). I would like to run the tests on-demand, but after these variables have been setup.
So what I want is to listen to stdin in my TestMain function and either run or quit the tests. But I want it to be controlled by the user so I can have my test environment setup, and the tests will run on my command.
But sadly, it seems that when running go test ... that stdin is mapped directly to /dev/null. So when I try to read os.Stdin I get an EOF error. The minimum code for this is:
package tests
import (
"bufio"
"fmt"
"os"
"testing"
)
func TestMain(m *testing.M) {
reader := bufio.NewReader(os.Stdin)
if input, err := reader.ReadString('\n'); err != nil {
fmt.Println(err)
fmt.Println("-----------")
fmt.Println(input)
os.Exit(1)
}
fmt.Println("ESCAPED!")
os.Exit(m.Run())
}
I have read in circles how to mock this for unit tests and the sort, but my case is more of a functional test runner. Is there a way, or even some trickery, that will allow me to open or change the test processes stdin?
You can redirect os.Stdin, it depends on OS though:
package tests
import (
"fmt"
"os"
"testing"
"bufio"
"runtime"
)
func TestMain(m *testing.M) {
var ttyName string
if runtime.GOOS == "windows" {
ttyName = "con"
} else {
ttyName = "/dev/tty"
}
f, err := os.Open(ttyName)
if err != nil {
panic(err)
}
defer f.Close()
oldStdin := os.Stdin
defer func() { os.Stdin = oldStdin }()
os.Stdin = f
reader := bufio.NewReader(os.Stdin)
if input, err := reader.ReadString('\n'); err != nil {
fmt.Println("Error:", err)
fmt.Println("-----------")
fmt.Println(input)
os.Exit(1)
}
fmt.Println("ESCAPED!")
os.Exit(m.Run())
}
func TestHello(t *testing.T){
fmt.Println("Hello")
}
Related
Using the os/exec package, I want to run an external command on a *nix OS with another user instead of root. (The main process runs under root user).
The external command runs by go app. But my app can not read /proc/pid/smaps file, following error:
panic: open /proc/2962/smaps: permission denied
goroutine 6 [running]:
main.memwatch(0xc000094000, 0xc00007a0c0)
/src/main.go:41 +0x298
created by main.main
/src/main.go:25 +0x18f
exit status 2
Here is my code:
// main.go
package main
import (
"fmt"
"os"
"os/exec"
"syscall"
"time"
)
func main() {
cmd := exec.Command("sleep", "3")
cmd.SysProcAttr = &syscall.SysProcAttr{
Setpgid: true,
Credential: &syscall.Credential{Uid: 65534, Gid: 65534}, // External command expect run with `nobody` instead of `root` for security reason
}
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
err := cmd.Start()
done := make(chan struct{})
go memwatch(cmd, done)
if err != nil {
panic(err)
}
cmd.Wait()
close(done)
}
func memwatch(cmd *exec.Cmd, done <-chan struct{}) {
// Reuse the reader so we don't have to close and reopen it all the time
smaps, err := os.Open(fmt.Sprintf("/proc/%d/smaps", cmd.Process.Pid))
if err != nil {
panic(err)
}
defer smaps.Close()
for {
select {
case <-done:
return
default:
fmt.Println("running")
time.Sleep(10 * time.Millisecond)
}
}
}
I'm tired. Anyone here for help, please.
I test my code inside a docker container that doesn't have SYS_PTRACE capability. That's why the error shows. The error was gone when I added the SYS_PTRACE capability for that container.
I try to setup a small Golang Microservice for users with Gin and Mongodb.
package main
import (
"context"
"fmt"
"github.com/wzslr321/artiver/entity"
"github.com/wzslr321/artiver/settings"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"log"
"os"
"os/signal"
"syscall"
"time"
)
type application struct {
users *entity.UserCollection
}
var app *application
func init() {
initMongo()
}
func initMongo() {
oc := options.Client().ApplyURI(settings.MongodbSettings.Uri)
client, err := mongo.NewClient(oc)
if err != nil {
log.Fatalf("Error occured while initializing a new mongo client: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
err = client.Connect(ctx)
if err != nil {
log.Fatalf("Errorr occurred while connecting to a client: %v", err)
}
defer func() {
if err = client.Disconnect(ctx); err != nil {
panic(err)
}
}()
log.Println("Successfully connected to the database!")
app = &application{
users: &entity.UserCollection{
C: client.Database("artiver").Collection("users"),
},
}
}
func main() {
router := app.InitRouter()
It doesn't show any errors in my IDE ( GoLand ), but when I try to build it I get an error:
# command-line-arguments
users/cmd/app/main.go:67:15: app.InitRouter undefined (type *application has no field or method InitRouter)
It it easily visible on the image above, that I do have access to such a method. It is defined in the same package.
package main
import (
"github.com/gin-gonic/gin"
cors "github.com/rs/cors/wrapper/gin"
"net/http"
)
func (app *application) InitRouter() *gin.Engine {
r := gin.New()
r.Use(gin.Recovery())
r.Use(cors.Default())
r.GET("/", func(ctx *gin.Context) {
ctx.String(http.StatusOK, "Hello World")
})
user := r.Group("/api/user")
{
user.POST("/add", app.CreateUser)
}
return r
}
I have no idea how am I supposed to fix it and what is done wrong. I'd appreciate any hint about what isn't done correctly.
Answer based on #mkopriva help in comments.
The issue was related to not running all needed .go files.
In my case, the solution was to build it this way in my Makefile:
go build -o $(path)users cmd/app/*
In similar cases, go run . most likely will do the job.
I am trying to execute tskarh from golang script using the example from
https://tutorialedge.net/golang/executing-system-commands-with-golang/
The script works fine, but i don't receive any kind of output
What i want to get is the following:
Continuously run the script,
capture some packets,
extract some fields values,
and assign to variables
Any help please ?
https://pastebin.com/PeAz7vh9
package main
import (
"fmt"
"os/exec"
"runtime"
)
func execute() {
// here we perform the pwd command.
// we can store the output of this in our out variable
// and catch any errors in err
out, err := exec.Command("tshark", "-i", "em1").CombinedOutput()
// if there is an error with our execution
// handle it here
if err != nil {
fmt.Printf("%s", err)
}
fmt.Println("Command Successfully Executed")
// as the out variable defined above is of type []byte we need to convert
// this to a string or else we will see garbage printed out in our console
// this is how we convert it to a string
output := string(out[:])
// once we have converted it to a string we can then output it.
fmt.Println(output)
}
func main() {
fmt.Println("Simple Shell")
fmt.Println("---------------------")
if runtime.GOOS == "windows" {
fmt.Println("Can't Execute this on a windows machine")
} else {
execute()
}
}
I have no idea of tshark, but here is a code that will work continously, you need os.Interrupt, and select.
package main
import (
"os"
"os/exec"
"os/signal"
)
func main() {
out := exec.Command("ping", "8.8.8.8")
f1, _ := os.OpenFile("./outfile.txt", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0755)
f2, _ := os.OpenFile("./errfile.txt", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0755)
out.Stdout = f1
out.Stderr = f2
defer func() {
f1.Close()
f2.Close()
}()
err := out.Run()
if err != nil {
panic(err)
}
var ctrlcInt chan os.Signal
ctrlcInt = make(chan os.Signal, 1)
signal.Notify(ctrlcInt, os.Interrupt)
for {
select {
case <-ctrlcInt:
break
default:
continue
}
}
return
}
this code pings 8.8.8.8 and writes out put to outfile.txt, it will exit when you press ctrl+c. If there is error it will write to errfile.txt. You can tail the files and see the output. Hope this helps.
I'm having some trouble linking things up here.
What am I doing incorrectly?
package main
import (
"encoding/xml"
"fmt"
"log"
"os/exec"
)
func checkError(err error) {
if err != nil {
log.Fatalf("Error: %s", err)
}
}
func metrics() {
cmd := exec.Command(
"nvidia-smi",
"--query",
"--xml-format")
out, err := cmd.StdoutPipe()
checkError(err)
cmd.Start()
defer cmd.Wait()
go func() {
var data interface{}
dec := xml.NewDecoder(out)
dec.Decode(&data)
fmt.Printf("Data: %+v\n", data)
}()
//go io.Copy(os.Stdout, out)
}
func main() {
metrics()
}
Result after running program is:
Data:
Things seem to be "linked" correctly.
Problem is likely to be here:
var data interface{}
You then do:
dec.Decode(&data)
But that won't work.
You need to pass in a struct that can actually be used to decode the fields in the XML that the nvidia-smi command returns.
Find below a modified example (replacing your nvidia-smi for an echo command to make it return a sample XML).
You should adjust the struct to be able to map to the actual XML you'll receive.
By the way:
You should check the error returned by decode just in case
I don't understand why you are decoding in a separate goroutine. I left it like that in the modified example, but it would work if you do it right in the same goroutine as well.
Example:
package main
import (
"log"
"os/exec"
"fmt"
"encoding/xml"
)
func checkError(err error) {
if err != nil {
log.Fatalf("Error: %s", err)
}
}
type Result struct {
Value int `xml:"value"`
}
func metrics() {
cmd := exec.Command(
"echo", "-n",
`<result><value>1</value></result>`)
out, err := cmd.StdoutPipe()
checkError(err)
cmd.Start()
defer cmd.Wait()
go func() {
var data Result
dec := xml.NewDecoder(out)
err = dec.Decode(&data)
checkError(err)
fmt.Printf("Data: %+v\n", data)
}()
//go io.Copy(os.Stdout, out)
}
func main() {
metrics()
}
I got the go routine below to work but the problem is that it prints to the console instead of to the screen. My idea is to have a running log of what commands or output is happening in a script show on a webpage where it can be watched in real time. Using fmt.Fprint doesn't do the trick. All that happens is that my webpage will never fully load. Help please?
Running external python in Golang, Catching continuous exec.Command Stdout
go code
package main
import (
"log"
"net/http"
"time"
"os/exec"
"io"
"bufio"
"fmt"
"github.com/gorilla/mux"
)
func main() {
r := mux.NewRouter()
s := r.PathPrefix("/api/").Subrouter()
s.HandleFunc("/export", export).Methods("GET")
http.Handle("/", r)
log.Panic(http.ListenAndServe(":80", nil))
}
func export(w http.ResponseWriter, r *http.Request) {
cmd := exec.Command("python", "game.py")
stdout, err := cmd.StdoutPipe()
if err != nil {
panic(err)
}
stderr, err := cmd.StderrPipe()
if err != nil {
panic(err)
}
err = cmd.Start()
if err != nil {
panic(err)
}
go copyOutput(stdout)
go copyOutput(stderr)
cmd.Wait()
}
func copyOutput(r io.Reader, w http.ResponseWriter) {
scanner := bufio.NewScanner(r)
for scanner.Scan() {
fmt.Fprint(w, scanner.Text()) //line I expect to print to the screen, but doesn't
}
}
python script
import time
import sys
while True:
print "Hello"
sys.stdout.flush()
time.sleep(1)
There's a lot more to the site so I know the route is configured correctly because printing to the screen works when I'm not using the go routine'
UPDATE:
Here is my new update function which prints to the screen, but only after the entire script has ran, not as it goes
func export(w http.ResponseWriter, r *http.Request) {
cmd := exec.Command("python", "game.py")
cmd.Stdout = w
cmd.Start()
cmd.Wait()
}
I believe I may still need a go routine in order to get it to print as I go, but putting cmd.Start and/or cmd.Wait in one doesn't work
UPDATE:
So even with everything given, I have not been able to get having the outputs show on a browser as they are ran working. It simply locks up the browser, even with the headers and flush. I will hopefully have time to give a complete, working answer to this but for now, the code above prints the code to the browser correctly after it has ran. I found a repo that I think may be what I'm looking for and maybe it will help others who come across this question as well.
https://github.com/yudai/gotty
This is a very basic (naive) example but how can give you an idea of how to stream data continuously:
https://play.golang.org/p/vtXPEHSv-Sg
The code for game.py is:
import time
import sys
while True:
print("Hello")
sys.stdout.flush()
time.sleep(1)
The web app code:
package main
import (
"bufio"
"fmt"
"io"
"log"
"net/http"
"os"
"os/exec"
"github.com/nbari/violetear"
)
func stream(w http.ResponseWriter, r *http.Request) {
cmd := exec.Command("python", "game.py")
rPipe, wPipe, err := os.Pipe()
if err != nil {
log.Fatal(err)
}
cmd.Stdout = wPipe
cmd.Stderr = wPipe
if err := cmd.Start(); err != nil {
log.Fatal(err)
}
go writeOutput(w, rPipe)
cmd.Wait()
wPipe.Close()
}
func writeOutput(w http.ResponseWriter, input io.ReadCloser) {
flusher, ok := w.(http.Flusher)
if !ok {
http.Error(w, "Streaming not supported", http.StatusInternalServerError)
return
}
// Important to make it work in browsers
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
in := bufio.NewScanner(input)
for in.Scan() {
fmt.Fprintf(w, "data: %s\n", in.Text())
flusher.Flush()
}
input.Close()
}
func main() {
router := violetear.New()
router.HandleFunc("/", stream, "GET")
log.Fatal(http.ListenAndServe(":8080", router))
}
The key part here is the use of http.Flusher and some headers to make it work within a browser:
w.Header().Set("Content-Type", "text/event-stream")
Note the problem with this code is that once a request arrives it will exec the command that loops forever, so the wPipe.Close() will never be called
cmd.Wait()
wPipe.Close()
To be more verbose you could print the output the terminal beside the browser:
for in.Scan() {
data := in.Text()
log.Printf("data: %s\n", data)
fmt.Fprintf(w, "data: %s\n", data)
flusher.Flush()
}
If you have more than one request you will notice it will write faster in the terminal, not bad but you will also notice that if the client closed the connection/browser you will still see data going out.
A better way could execute the command within a context, from the example: https://golang.org/pkg/os/exec/#CommandContext
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
if err := exec.CommandContext(ctx, "sleep", "5").Run(); err != nil {
// This will fail after 100 milliseconds. The 5 second sleep
// will be interrupted.
}
Also take a look at the context (https://stackoverflow.com/a/44146619/1135424) not replaces http.CloseNotifier so could be usefull for terminate the process once the client close browser, disconetcs.
At the end depends on your needs but hope can give you an idea about how to stream data in an easy way by using the http.Flusher interface.
Just for fun here is an example using the context:
https://play.golang.org/p/V69BuDUceBA
Still very basic, but in this case if client closes the browser the program also terminates, as an exercice could be nice to improve it an share back ;-), notice the use of CommandContext and ctx.Done()
package main
import (
"bufio"
"fmt"
"io"
"log"
"net/http"
"os"
"os/exec"
"github.com/nbari/violetear"
)
func stream(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
ch := make(chan struct{})
cmd := exec.CommandContext(ctx, "python", "game.py")
rPipe, wPipe, err := os.Pipe()
if err != nil {
log.Fatal(err)
}
cmd.Stdout = wPipe
cmd.Stderr = wPipe
if err := cmd.Start(); err != nil {
log.Fatal(err)
}
go writeOutput(w, rPipe)
go func(ch chan struct{}) {
cmd.Wait()
wPipe.Close()
ch <- struct{}{}
}(ch)
select {
case <-ch:
case <-ctx.Done():
err := ctx.Err()
log.Printf("Client disconnected: %s\n", err)
}
}
func writeOutput(w http.ResponseWriter, input io.ReadCloser) {
flusher, ok := w.(http.Flusher)
if !ok {
http.Error(w, "Streaming not supported", http.StatusInternalServerError)
return
}
// Important to make it work in browsers
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
in := bufio.NewScanner(input)
for in.Scan() {
data := in.Text()
log.Printf("data: %s\n", data)
fmt.Fprintf(w, "data: %s\n", data)
flusher.Flush()
}
input.Close()
}
func main() {
router := violetear.New()
router.HandleFunc("/", stream, "GET")
log.Fatal(http.ListenAndServe(":8080", router))
}