I have some trouble with execute shell commands from a Go program:
var command = pwd + "/build " + file_name
dateCmd := exec.Command(string(command))
dateOut, err := dateCmd.Output()
check(err)
If command variable equals a single word like /home/slavik/project/build (build is shell script) it works, but if I try to pass some arg i.e. /home/slavik/project/build xxx or /home/slavik/project/build -v=1 the Go program raises an exception like file /home/slavik/project/build not found
What's wrong with my code?
You have to pass the program and the arguments separately. See the signature of exec.Command:
func Command(name string, arg ...string) *Cmd
So if you want to pass e.g. -v=1, your call probably should look something like:
dateCmd := exec.Command(pwd + "/build", "-v=1")
Use
exec.Command(pwd + "/build", fileName)
Related
I'd like to write a function that puts two slices of strings as arguments in a shell command. What I'm trying to do is to unpack the 2 slices, and pass them to exec.Command.
image_tag := "mydocker/mytag"
func buildDockerContainer(dockerArgs []string, otherArgs []string) {
cmd := exec.Command("docker", "run", "-d", dockerArgs..., image_tag, otherArgs...)
}
However, when writing this, Goland keeps giving me syntax error:
Invalid use of '...', the corresponding parameter is non-variadic
I know I can do the following:
cmdToRun := []string{"run", "-d"}
cmdToRun = append(cmdToRun, append(append(dockerArgs, image_tag), otherArgs...)...)
cmd := exec.Command("docker", cmdToRun...)
But is there a more elegant way that I can do all these inline?
Use append:
args:= append(append(append([]string{"run","-d"},dockerArgs...),image_tag),otherArgs...)
cmd := exec.Command("docker", args...)
exec.Command(name string, arg ...string) behave same as append function in this case. the problem is that if the first arg is string, later it expect only string, but if you pass instead only slice that is fine.
https://go.dev/play/p/TBi3nsKue5n
and don't forget the ... at the end of a slice, when the function expect a string and you pass []string
go version go1.15.6 windows/amd64
dev os Windows [Version 10.0.19041.630]
I have a Go app in which I am running the AWS CLI using exec.Cmd.Run(). I build out the Cmd class and populate the arguments.
Before I run the Cmd, I use the .String() method to view the command to be ran. If I take this value, copy it to a shell, the command executes with no modifications to the output given to me with no issues reported.
However, when I run the command, it fails returning an error. When I debug the script, it is failing because it says the AWS CLI is saying a parameter is incorrect.
Questions:
Is it possible to see the 100% raw representation of what is being ran? It does not match the return value of .String()
Is there a better way to call an os level command that I am missing?
Real Example:
cmd := &exec.Cmd{
Path: awsPath,
Args: args,
Stdout: &stdout,
Stderr: &stderr,
}
fmt.Printf("Command: %s\n", cmd.String())
// c:\PROGRA~1\Amazon\AWSCLIV2\aws.exe --profile testprofile --region us-east-1 --output json ec2 describe-network-interfaces --filters Name=group-id,Values=sg-abc123
// Running above works 100% of the time if ran from a shell window
err := cmd.Run()
// always errors out saying the format is incorrect
GoPlayground Replication of Issue
https://play.golang.org/p/mvV9VG8F0oz
From cmd.String source:
// String returns a human-readable description of c.
// It is intended only for debugging.
// In particular, it is not suitable for use as input to a shell.
// The output of String may vary across Go releases.
You are seeing the reverse, but the problem is the same: eye-balling a printed command string does not show the exact executable path (is there a rogue space or unprintable character?), same with the arguments (rogue characters?).
Use fmt.Printf("cmd : %q\n", cmd.Path) to show any hidden unicode characters etc. And use the same technique with each of the arguments.
EDIT:
I have found the root cause of your problem you met: os/exec
// Path is the path of the command to run.
//
// This is the only field that must be set to a non-zero
// value. If Path is relative, it is evaluated relative
// to Dir.
Path string
// Args holds command line arguments, including the command as **Args[0]**.
// If the Args field is empty or nil, Run uses {Path}.
//
// In typical use, both Path and Args are set by calling Command.
Args []string
So if you have declare the Cmd.Path := "/usr/local/bin/aws", you have to declare Cmd. Args like this: Args: []string{"", "s3", "help"}, because the Args including the command as Args[0] in above document link.
Final, I think you can exec command like this for simple and effectively:
package main
import (
"bytes"
"fmt"
"os/exec"
)
func main() {
stdout := &bytes.Buffer{}
stderr := &bytes.Buffer{}
name := "/usr/local/bin/aws"
arg := []string{"s3", "help"}
cmd := exec.Command(name, arg...)
cmd.Stderr = stderr
cmd.Stdout = stdout
fmt.Printf("Command: %q\n", cmd.String())
err := cmd.Run()
if err != nil {
fmt.Println("Error: ", stderr.String())
}
fmt.Println("Output: ", stdout.String())
}
=========
$ go run main.go
Command: "/usr/local/bin/aws s3 help"
Done.
I would like to run following code in Go:
out, err := exec.Command("echo", "$PATH").Output()
The result was:
$PATH
Instead of the expected value of "PATH=/bin...".
Why? And how can I get the expected value?
Your command is not being interpreted by a shell, which is why the expected variable substitution is not occurring.
From the exec package documentation:
...the os/exec package intentionally does not invoke the system shell
and does not expand any glob patterns or handle other expansions,
pipelines, or redirections typically done by shells.
...
To expand environment variables, use package os's ExpandEnv.
So to achieve the desired result in your example:
out, err := exec.Command("echo", os.ExpandEnv("$PATH")).Output()
It's worth reviewing the set of functions for getting environment variables and using what works best for your particular use case:
func ExpandEnv(s string) string - ExpandEnv replaces ${var} or $var in the string according to the values of the current environment variables. References to undefined variables are replaced by the empty string.
func Getenv(key string) string - Getenv retrieves the value of the environment variable named by the key. It returns the value, which will be empty if the variable is not present. To distinguish between an empty value and an unset value, use LookupEnv.
func LookupEnv(key string) (string, bool) - LookupEnv retrieves the value of the environment variable named by the key. If the variable is present in the environment the value (which may be empty) is returned and the boolean is true. Otherwise the returned value will be empty and the boolean will be false.
From Go by Example: Environment Variables, you can try (instead of a system command echo) to use os.Getenv:
fmt.Println("PATH:", os.Getenv("PATH"))
Since you are using a shell command, you need to provide the shell to run the command.
cmd=exec.Command("/bin/sh","-c", "echo $PATH")
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
err := cmd.Run()
if err != nil {
panic(err)
}
I have been trying to run a command and parse the output in golang. Here is a sample of what I am trying to do:
package main
import (
"fmt"
"os/exec"
)
func main() {
out,err := exec.Command("ls -ltr").Output()
if err != nil {
fmt.Println("Error: %s", err)
}
fmt.Printf("%s",out)
}
Now, when I am trying to run "ls -ltr", I get this error:
Error: %s exec: "ls -ltr": executable file not found in $PATH
So, basically go is looking for whole "ls -ltr" in PATH. And it's not there obviously. Is there any way I can pass a flag to any argument?TIA.
You pass arguments to the program by passing more arguments to the function - it's variadic:
out,err := exec.Command("ls","-ltr").Output()
https://golang.org/pkg/os/exec/#Command
This is a pretty common convention with exec-style functions which you will see in most languages. The other common pattern is builders.
Sometimes the layout of arguments you need to pass won't be known at compile-time (though it's not a good idea to send arbitrary commands to the system - stay safe!). If you want to pass an unknown number of arguments, you can use an array with some special syntax:
// Populate myArguments however you like
myArguments := []string{"bar","baz"}
// Pass myArguments with "..." to use variadic behaviour
out,err := exec.Command("foo", myArguments...).Output()
I would like to run my program like this:
go run launch.go http://example.com --m=2 --strat=par
"http://example.com" gets interpreted as the first command line argument, which is ok, but the flags are not parsed after that and stay at the default value. If I put it like this:
go run launch.go --m=2 --strat=par http://example.com
then "--m=2" is interpreted as the first argument (which should be the URL).
I could also just remove the os.Args completely, but then I would have only optional flags and I want one (the URL) to be mandatory.
Here's my code:
package main
import (
"fmt"
"webcrawler/crawler"
"webcrawler/model"
"webcrawler/urlutils"
"os"
"flag"
)
func main() {
if len(os.Args) < 2 {
log.Fatal("Url must be provided as first argument")
}
strategy := flag.String("strat", "par", "par for parallel OR seq for sequential crawling strategy")
routineMultiplier := flag.Int("m", 1, "Goroutine multiplier. Default 1x logical CPUs. Only works in parallel strategy")
page := model.NewBasePage(os.Args[1])
urlutils.BASE_URL = os.Args[1]
flag.Parse()
pages := crawler.Crawl(&page, *strategy, *routineMultiplier)
fmt.Printf("Crawled: %d\n", len(pages))
}
I am pretty sure that this should be possible, but I can't figure out how.
EDIT:
Thanks justinas for the hint with the flag.Args(). I now adapted it like this and it works:
...
flag.Parse()
args := flag.Args()
if len(args) != 1 {
log.Fatal("Only one argument (URL) allowed.")
}
page := model.NewBasePage(args[0])
...
os.Args doesn't really know anything about the flag package and contains all command-line arguments. Try flag.Args() (after calling flag.Parse(), of course).
As a followup, to parse flags that follow a command like
runme init -m thisis
You can create your own flagset to skip the first value like
var myValue string
mySet := flag.NewFlagSet("",flag.ExitOnError)
mySet.StringVar(&myValue,"m","mmmmm","something")
mySet.Parse(os.Args[2:])
This tripped me up too, and since I call flag.String/flag.Int64/etc in a couple of places in my app, I didn't want to have to pass around a new flag.FlagSet all over the place.
// If a commandline app works like this: ./app subcommand -flag -flag2
// `flag.Parse` won't parse anything after `subcommand`.
// To still be able to use `flag.String/flag.Int64` etc without creating
// a new `flag.FlagSet`, we need this hack to find the first arg that has a dash
// so we know when to start parsing
firstArgWithDash := 1
for i := 1; i < len(os.Args); i++ {
firstArgWithDash = i
if len(os.Args[i]) > 0 && os.Args[i][0] == '-' {
break
}
}
flag.CommandLine.Parse(os.Args[firstArgWithDash:])
The reason I went with this is because flag.Parse just calls flag.CommandLine.Parse(os.Args[1:]) under the hood anyway.
You can check if the Arg starts with "--" or "-" and avoid using that Arg in a loop.
For example:
for _, file := range os.Args[1:] {
if strings.HasPrefix(file, "--") {
continue
}
//do stuff
}