golang exec incorrect behavior - go

I'm using following code segment to get the XML definition of a virtual machine running on XEN Hypervisor. The code is trying to execute the command virsh dumpxml Ubutnu14 which will give the XML of the VM named Ubuntu14
virshCmd := exec.Command("virsh", "dumpxml", "Ubuntu14")
var virshCmdOutput bytes.Buffer
var stderr bytes.Buffer
virshCmd.Stdout = &virshCmdOutput
virshCmd.Stderr = &stderr
err := virshCmd.Run()
if err != nil {
fmt.Println(err)
fmt.Println(stderr.String())
}
fmt.Println(virshCmdOutput.String())
This code always goes into the error condition for the given domain name and I get the following output.
exit status 1
error: failed to get domain 'Ubuntu14'
error: Domain not found: no domain with matching name 'Ubuntu14'
But if I run the standalone command virsh dumpxml Ubuntu14, I get the correct XML definition.
I would appreciate if someone could give me some hints on what I'm doing wrong. My host machine is Ubuntu-16.04 and golang version is go1.6.2 linux/amd64

I expect you are running virsh as a different user in these two scenarios, and since you don't provide any URI, it is connecting to a different libvirtd instance. If you run virsh as non-root, then it'll usually connect to qemu:///session, but if you run virsh as root, then it'll usually connect to qemu:///system. VMs registered against one URI, will not be visible when connecting to the other URI.
BTW, if you're using go, you'd be much better off using the native Go library bindings for libvirt instead of exec'ing virsh. Your "virsh dumpxml" invokation is pretty much equivalent to this:
import (
"github.com/libvirt/libvirt-go"
)
conn, err := libvirt.NewConnect("qemu:///system")
dom, err := conn.LookupDomainByName("Ubuntu14")
xml, err := dom.GetXMLDesc(0)
(obviously do error handling too)

Related

Cloud Run Golang container issue/missunderstanding

I'm trying to do a report of all the objects in all the projects we have in Cloud Storage of our Org. I'm using this repo from the Google Professionnal Services as it's doing exactly what we want: https://github.com/GoogleCloudPlatform/professional-services/tree/main/tools/gcs2bq
We want to use containers instead of just the go code on a Cloud Function for portability mainly.
Locally everything is good and the program behave as expected but when I try in Cloud Run things get tricky. From what I understand, the go part needs to listen to a port, which I added at the beginning of the main so the container can be deployed, which it is:
// Determine port for HTTP service
port := os.Getenv("PORT")
if port == "" {
port = "8080"
log.Printf("defaulting to port %s", port)
}
Start HTTP server.
log.Printf("listening on port %s", port)
if err := http.ListenAndServe(":"+port, nil); err != nil {
log.Fatal(err)
}
But as you can see in the repo, the first file called is the run.sh one. Which set environment variables and then call the main.go. It sucessfully complete it's task, which is get all the size of the different files. But after that the run.sh doesnt "resume" and go to the part where it uploads the data in a BigQuery table, which locally work.
Here is the part in the run.sh file where I have a problem. Note : I don't have errors from executing the ./gcs2bq Note 2 : Every environment variable has a correct value
./gcs2bq $GCS2BQ_FLAGS || error "Export failed!" 2 <- doesnt get past this line
gsutil mb -p "${GCS2BQ_PROJECT}" -c standard -l "${GCS2BQ_LOCATION}" -b on "gs://${GCS2BQ_BUCKET}" || echo "Info: Storage bucket already exists: ${GCS2BQ_BUCKET}"
gsutil cp "${GCS2BQ_FILE}" "gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME}" || error "Failed copying ${GCS2BQ_FILE} to gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME}!" 3
bq mk --project_id="${GCS2BQ_PROJECT}" --location="${GCS2BQ_LOCATION}" "${GCS2BQ_DATASET}" || echo "Info: BigQuery dataset already exists: ${GCS2BQ_DATASET}"
bq load --project_id="${GCS2BQ_PROJECT}" --location="${GCS2BQ_LOCATION}" --schema bigquery.schema --source_format=AVRO --use_avro_logical_types --replace=true "${GCS2BQ_DATASET}.${GCS2BQ_TABLE}" "gs://${GCS2BQ_BUCKET}/${GCS2BQ_FIL$
error "Failed to load gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME} to BigQuery table ${GCS2BQ_DATASET}.${GCS2BQ_TABLE}!" 4
gsutil rm "gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME}" || error "Failed deleting gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME}!" 5
rm -f "${GCS2BQ_FILE}"
I'm kinda new to containers and Cloud Run and even after reading projects and documentation, I'm not sure what I'm doing wrong, Is it normal that the .sh is "stuck" when calling the main.go? I can provide more details/explaination if needed.
Okay so for anyone who encounter similar situation this is how I made it work for me.
The container isn't supposed to stop so no exit, it will just go back to the main function.
That means that when I called executable it just looped and never exited and completed the task. So the solution here is to "recode" everything past the call in golang directly into the main.go
Here the run.sh is then useless so I used another .go file that listen for http request and then call the code that gather data and send it to Bigquery.

How can I get HTTP RPC Server and client to work?

I am trying to send messages from client to server and back using the exact HTTP RPC server/client code given here.
However, when I run the server, my command line becomes blank because the server starts listening using:
err := http.ListenAndServe(":1234", nil)
if err != nil {
fmt.Println(err.Error())
}
In the client code, I need to get an argument from the command line for client to run:
serverAddress := os.Args[1]
However, this argument is not available because the server code makes my command line blank.
How can I get the server and client to work on the same command line window?

Why Logrus writes logs to Linux logs?

I use Logrus library to write my golang logs to file. The problem is that logrus also writes the logs to Linux system logs. Here is my log output in log configs.
log.SetOutput(os.Stderr)
I do not want Logrus to write application logs to Linux system logs. Any idea about solving this problem?
To write output to file, you can set output to a file handler.
file, err := os.OpenFile(filename, os.O_WRONLY | os.O_CREATE, 0755)
if err != nil {
// handle error
}
logrus.SetOutput(f)
In case you want to introduce more advanced logic to handle the output, you can use this library Lumberjack as log output

go-ping library for unprivileged ICMP ping in golang

I have been using go-ping library for the unprivileged ping and calculate various statistics of network in golang.
code snippet is as->
func (p *Ping) doPing() (latency, jitter, packetLoss float64, err error) {
timeout := time.Second*1000
interval := time.Second
count := 5
host := p.ipAddr
pinger, cmdErr := ping.NewPinger(host)
if cmdErr != nil {
glog.Error("Failed to ping " + p.ipAddr)
err = cmdErr
return
}
pinger.Count = count
pinger.Interval = interval
pinger.Timeout = timeout
pinger.SetPrivileged(false)
pinger.Run()
stats := pinger.Statistics()
latency = float64(stats.AvgRtt)
jitter = float64(stats.StdDevRtt)
packetLoss = stats.PacketLoss
return
}
It was working fine but now it has started throwing :-
"Error listening for ICMP packets: socket: permission denied" error.
Anyone knows the reason behind this? Go version I am using is go1.7.4.
This is in the README.md of the library you're using :
This library attempts to send an "unprivileged" ping via UDP. On linux, this must be enabled by setting
sudo sysctl -w net.ipv4.ping_group_range="0 2147483647"
If you do not wish to do this, you can set pinger.SetPrivileged(true) and use setcap to allow your binary using go-ping to bind to raw sockets (or just run as super-user):
setcap cap_net_raw=+ep /bin/goping-binary
See this blog and the Go icmp library for more details.
Hope it helps !
Make sure your setting haven't changed in any way. Using ping from the package still works for me on a 32-bit Ubuntu 16.04 with Go 1.7.4 (linux/386) if I previousely set the net.ipv4.ping_group_range according to the instructions on Github.
Note on Linux Support:
This library attempts to send an "unprivileged" ping via UDP. On linux, this must be enabled by setting
sudo sysctl -w net.ipv4.ping_group_range="0 2147483647"
If you do not wish to do this, you can set pinger.SetPrivileged(true) and use setcap to allow your binary
using go-ping to bind to raw sockets (or just run as super-user):
setcap cap_net_raw=+ep /bin/goping-binary
See this blog
and the Go icmp library for
more details.

exec.Command does not register error from Go's own pprof tool

Here is my code:
cmd := exec.Command("go", "tool", "pprof", "-dot", "-lines", "http://google.com")
out, err := cmd.Output()
if err != nil {
panic(err)
}
println(string(out))
When I run the exact same command in my console, I see:
$ go tool pprof -dot -lines http://google.com
Fetching profile from http://google.com/profilez
Please wait... (30s)
server response: 404 Not Found
However, my go program does not register that this is an error. Oddly, the variable out prints as an empty string and err is nil. What is going on?
To clarify, I am profiling http://google.com to purposefully create an error. I would normally profile a real Go application.
The text
Fetching profile from http://google.com/profilez
Please wait... (30s)
server response: 404 Not Found
is written to stderr. Your program captures stdout, which is empty. Consider calling:
out, err := cmd.CombinedOutput()
to grab both stdout and stderr.
cmd.Output() and cmd.CombinedOutput() return err == nil because the command exits with status zero. Perhaps an issue should be filed requesting that the command exit with non-zero status.

Resources