I would like to run following code in Go:
out, err := exec.Command("echo", "$PATH").Output()
The result was:
$PATH
Instead of the expected value of "PATH=/bin...".
Why? And how can I get the expected value?
Your command is not being interpreted by a shell, which is why the expected variable substitution is not occurring.
From the exec package documentation:
...the os/exec package intentionally does not invoke the system shell
and does not expand any glob patterns or handle other expansions,
pipelines, or redirections typically done by shells.
...
To expand environment variables, use package os's ExpandEnv.
So to achieve the desired result in your example:
out, err := exec.Command("echo", os.ExpandEnv("$PATH")).Output()
It's worth reviewing the set of functions for getting environment variables and using what works best for your particular use case:
func ExpandEnv(s string) string - ExpandEnv replaces ${var} or $var in the string according to the values of the current environment variables. References to undefined variables are replaced by the empty string.
func Getenv(key string) string - Getenv retrieves the value of the environment variable named by the key. It returns the value, which will be empty if the variable is not present. To distinguish between an empty value and an unset value, use LookupEnv.
func LookupEnv(key string) (string, bool) - LookupEnv retrieves the value of the environment variable named by the key. If the variable is present in the environment the value (which may be empty) is returned and the boolean is true. Otherwise the returned value will be empty and the boolean will be false.
From Go by Example: Environment Variables, you can try (instead of a system command echo) to use os.Getenv:
fmt.Println("PATH:", os.Getenv("PATH"))
Since you are using a shell command, you need to provide the shell to run the command.
cmd=exec.Command("/bin/sh","-c", "echo $PATH")
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
err := cmd.Run()
if err != nil {
panic(err)
}
Related
go version go1.15.6 windows/amd64
dev os Windows [Version 10.0.19041.630]
I have a Go app in which I am running the AWS CLI using exec.Cmd.Run(). I build out the Cmd class and populate the arguments.
Before I run the Cmd, I use the .String() method to view the command to be ran. If I take this value, copy it to a shell, the command executes with no modifications to the output given to me with no issues reported.
However, when I run the command, it fails returning an error. When I debug the script, it is failing because it says the AWS CLI is saying a parameter is incorrect.
Questions:
Is it possible to see the 100% raw representation of what is being ran? It does not match the return value of .String()
Is there a better way to call an os level command that I am missing?
Real Example:
cmd := &exec.Cmd{
Path: awsPath,
Args: args,
Stdout: &stdout,
Stderr: &stderr,
}
fmt.Printf("Command: %s\n", cmd.String())
// c:\PROGRA~1\Amazon\AWSCLIV2\aws.exe --profile testprofile --region us-east-1 --output json ec2 describe-network-interfaces --filters Name=group-id,Values=sg-abc123
// Running above works 100% of the time if ran from a shell window
err := cmd.Run()
// always errors out saying the format is incorrect
GoPlayground Replication of Issue
https://play.golang.org/p/mvV9VG8F0oz
From cmd.String source:
// String returns a human-readable description of c.
// It is intended only for debugging.
// In particular, it is not suitable for use as input to a shell.
// The output of String may vary across Go releases.
You are seeing the reverse, but the problem is the same: eye-balling a printed command string does not show the exact executable path (is there a rogue space or unprintable character?), same with the arguments (rogue characters?).
Use fmt.Printf("cmd : %q\n", cmd.Path) to show any hidden unicode characters etc. And use the same technique with each of the arguments.
EDIT:
I have found the root cause of your problem you met: os/exec
// Path is the path of the command to run.
//
// This is the only field that must be set to a non-zero
// value. If Path is relative, it is evaluated relative
// to Dir.
Path string
// Args holds command line arguments, including the command as **Args[0]**.
// If the Args field is empty or nil, Run uses {Path}.
//
// In typical use, both Path and Args are set by calling Command.
Args []string
So if you have declare the Cmd.Path := "/usr/local/bin/aws", you have to declare Cmd. Args like this: Args: []string{"", "s3", "help"}, because the Args including the command as Args[0] in above document link.
Final, I think you can exec command like this for simple and effectively:
package main
import (
"bytes"
"fmt"
"os/exec"
)
func main() {
stdout := &bytes.Buffer{}
stderr := &bytes.Buffer{}
name := "/usr/local/bin/aws"
arg := []string{"s3", "help"}
cmd := exec.Command(name, arg...)
cmd.Stderr = stderr
cmd.Stdout = stdout
fmt.Printf("Command: %q\n", cmd.String())
err := cmd.Run()
if err != nil {
fmt.Println("Error: ", stderr.String())
}
fmt.Println("Output: ", stdout.String())
}
=========
$ go run main.go
Command: "/usr/local/bin/aws s3 help"
Done.
I have a function that returns to values user and err . When I call it in scope I already have variable user, but don’t have variable err , so compiler/linter tells me to use := operator syntax (I know I can declare err somewhere before this call with var declaration), making it look like this:
user := User{"Name"}
...
user, err := functionThatReturnsTwoValues()
if err != nil {
...
Question: In this specific case, in the line user, err := functionThatReturnsTwoValues , will user variable be redeclared?
P.S. I also understand that from actual results, it doesn't matter for me, as in the end, I will have variable with correct data after function call in any case. Also the fact, that variable will be defined in the stack in our case, means there will be no garbage collection involved to clean it up, even if there were 2 User structs initialized.
I think user variable is not re-declared, but of course its value is overridden. I've tested via checking the variable's pointer addresses as below. As you may see the pointers which are used for capturing variable addresses stays same.
https://play.golang.org/p/bj3QwSgCCiG
Snipped:
func main() {
user := User{"Name"}
up1 := &user
user, err := functionThatReturnsTwoValues()
up2 := &user
if err == nil {
fmt.Printf("User: %v \n", user)
fmt.Printf("Pointer check : up1 ?= up2 --> %t [up1=%p, up2=%p]\n", up1 == up2, up1, up2)
fmt.Printf("Value check : *up1 ?= *up2 --> %t \n", *up1 == *up2)
}
}
The output is:
User: {Name2}
Pointer check : up1 ?= up2 --> true [up1=0x40c138, up2=0x40c138]
Value check : *up1 ?= *up2 --> true
The user variable will be used to store the result of the function, see below an example
https://play.golang.org/p/eHHycX4p16j
Yes, the variable is redeclared and its value is overridden. But it must still have the same type declaration. The redeclaration essentially drops down to an assignment and the same stack memory is used.
This is explicitly allowed by the spec to make cases such as defining err neater:
Unlike regular variable declarations, a short variable declaration may redeclare variables provided they were originally declared earlier in the same block (or the parameter lists if the block is the function body) with the same type, and at least one of the non-blank variables is new. As a consequence, redeclaration can only appear in a multi-variable short declaration. Redeclaration does not introduce a new variable; it just assigns a new value to the original.
- https://golang.org/ref/spec#Short_variable_declarations
For example:
var user User
user, err := fetchUser() // OK: user redeclared but new err variable declared
user, err := fetchUser() // Bad: user and err variable redeclared, but no new variables declared
user, err = fetchUser() // OK: No declarations, only assignments
In short, yes, the variable user will be redeclared. From the Go Language Specification:
Unlike regular variable declarations, a short variable declaration may redeclare variables provided they were originally declared earlier in the same block. As a consequence, redeclaration can only appear in a multi-variable short declaration. Redeclaration does not introduce a new variable; it just assigns a new value to the original.
See the specification of short variable declaration for more details.
Yes, shorthand declaration will redefine it.
package main
import "fmt"
func main() {
i := 1
i, x := giveMeTwos()
fmt.Println(i, x)
}
func giveMeTwos() (int, int) {
return 2, 2
}
I have a simple piece of code where I want to convert elements of a slice into json and then print them out to my http.responseWriter.
for _, element := range customers {
result, _ := json.Marshal(element)
fmt.Fprintf(w, string(result))
}
However when I compile this I get the error "result declared and not used". If I add a simple line:
_ = result
Then everything compiles and works fine. Why does the compiler complain about this usage, and what is the correct way to do this in go?
Any insight is appreciated, my searches so far seem to indicate the call to Fprintf should count as a usage.
The code in question does not result in the error posted, for proof, check it on the Go Playground.
This error usually is (and the op confirmed it is too in this case) caused by having a local variable with same name outside of the block, and when using the short variable declaration, that shadows that variable.
This error can be reproduced with the following code:
var result []byte
customers := []int{}
w := os.Stdout
for _, element := range customers {
result, _ := json.Marshal(element)
fmt.Fprintf(w, string(result))
}
Attempting to compile and run it, we get the error (try it on the Go Playground):
prog.go:10:6: result declared and not used
Solution is to use a simple assignment instead of the short variable declaration if intention is to use the existing variable (in which case no new variable will be created), or use a different name for the variable if intention is not to use the outer, existing variable (but then the outer variable is to be removed or be used of course).
I have been trying to run a command and parse the output in golang. Here is a sample of what I am trying to do:
package main
import (
"fmt"
"os/exec"
)
func main() {
out,err := exec.Command("ls -ltr").Output()
if err != nil {
fmt.Println("Error: %s", err)
}
fmt.Printf("%s",out)
}
Now, when I am trying to run "ls -ltr", I get this error:
Error: %s exec: "ls -ltr": executable file not found in $PATH
So, basically go is looking for whole "ls -ltr" in PATH. And it's not there obviously. Is there any way I can pass a flag to any argument?TIA.
You pass arguments to the program by passing more arguments to the function - it's variadic:
out,err := exec.Command("ls","-ltr").Output()
https://golang.org/pkg/os/exec/#Command
This is a pretty common convention with exec-style functions which you will see in most languages. The other common pattern is builders.
Sometimes the layout of arguments you need to pass won't be known at compile-time (though it's not a good idea to send arbitrary commands to the system - stay safe!). If you want to pass an unknown number of arguments, you can use an array with some special syntax:
// Populate myArguments however you like
myArguments := []string{"bar","baz"}
// Pass myArguments with "..." to use variadic behaviour
out,err := exec.Command("foo", myArguments...).Output()
I have some trouble with execute shell commands from a Go program:
var command = pwd + "/build " + file_name
dateCmd := exec.Command(string(command))
dateOut, err := dateCmd.Output()
check(err)
If command variable equals a single word like /home/slavik/project/build (build is shell script) it works, but if I try to pass some arg i.e. /home/slavik/project/build xxx or /home/slavik/project/build -v=1 the Go program raises an exception like file /home/slavik/project/build not found
What's wrong with my code?
You have to pass the program and the arguments separately. See the signature of exec.Command:
func Command(name string, arg ...string) *Cmd
So if you want to pass e.g. -v=1, your call probably should look something like:
dateCmd := exec.Command(pwd + "/build", "-v=1")
Use
exec.Command(pwd + "/build", fileName)