I want to test different (correct/incorrect) command line arguments passed to my CLI program, but I am not sure how to achieve this with go/testing package because I am getting flag redefined error. Looks like it happens because flag.Parse() can be called only once. What is the proper approach to test different command line arguments passed into the go program? Is there is any way to define something like setup()/teardown() or run every case in isolation (but in the same file)?
Here is my code:
Function to test:
func (p *Params) Parse() (*Params, error) {
param1Ptr := flag.String("param1", "default", "param1 desc")
param2Ptr := flag.String("param2", "default", "param1 desc")
...
...
flag.Parse()
...
}
Test file:
package main
import (
"os"
"testing"
)
func TestParam1(t *testing.T) {
os.Args = []string{"cmd", "-param1", "incorrect", "-param2", "correct"}
params := Params{}
_, err := params.Parse()
...
...
}
func TestParam2(t *testing.T) {
os.Args = []string{"cmd", "-param1", "correct", "-param2", "incorrect"}
params := Params{}
_, err := params.Parse()
...
...
}
Don't use the global FlagSet object in the flags package. Create your own FlagSet as a field of Params: https://golang.org/pkg/flag/#FlagSet
All that flag.String et al do is pass through the function call to a global FlagSet object in the flag package (specifically flag.CommandLine is the variable). This is easy to use but not a generally good practice. Using your own flagset would avoid the issues you described as well as other potential side effects from using global variables.
Clear the global FlagSet before each test using:
flag.CommandLine = flag.NewFlagSet(os.Args[0], flag.ExitOnError)
See How to unset flags Visited on command line in GoLang for Tests
Related
I have go test code similar to this:
func TestRollback(t *testing.T) {
store := NewStore()
// do some stuff
err := store.Rollback()
// checks
}
The problem is store.Rollback() has a prompt read from the stdin for y or n
How do I send "y" to the test process when running go test -v --run TestRollback
The difficulty in testing your Rollback method stems from hardcoding its dependency on singleton os.Stdin.
Tinkerer's answer is viable but, because it mutates that package-level variable, it doesn't lend itself to running tests in parallel.
A preferable alternative (IMO) consists in using an interface. Testing often rhymes with interface, in Go. Here, because os.Stdin satisfies the io.Reader interface, you could parameterise your Store type with an io.Reader passed to your factory function:
type Store struct {
// other fields, omitted here
in io.Reader
}
func NewStore(in io.Reader) *Store {
store := Store {
// other fields, omitted here
in: in,
}
return &store
}
Then, in your test functions, you could use a concrete type that satisfies io.Reader and is easily configurable, such as a *strings.Reader:
func TestRollback(t *testing.T) {
// arrange
in := strings.Reader("-- put contents of stdin here --")
store := NewStore(in)
// act
err := store.Rollback()
// assert
// ...
}
The following can redirect stdin temporarily.
rd,wr,err := os.Pipe()
saved := os.Stdin
os.Stdin = rd
... Test code feeds wr ...
os.Stdin = saved
I've just recently started working with Go, and I've run into some
behavior working with Cobra and Viper that I'm not sure I understand.
This is a slightly modified version of the sample code you get by
running cobra init. In main.go I have:
package main
import (
"github.com/larsks/example/cmd"
"github.com/spf13/cobra"
)
func main() {
rootCmd := cmd.NewCmdRoot()
cobra.CheckErr(rootCmd.Execute())
}
In cmd/root.go I have:
package cmd
import (
"fmt"
"os"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var cfgFile string
func NewCmdRoot() *cobra.Command {
config := viper.New()
var cmd = &cobra.Command{
Use: "example",
Short: "A brief description of your application",
PersistentPreRun: func(cmd *cobra.Command, args []string) {
initConfig(cmd, config)
},
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("This is a test\n")
},
}
cmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $HOME/.example.yaml)")
cmd.PersistentFlags().String("name", "", "a name")
// *** If I move this to the top of initConfig
// *** the code runs correctly.
config.BindPFlag("name", cmd.Flags().Lookup("name"))
return cmd
}
func initConfig(cmd *cobra.Command, config *viper.Viper) {
if cfgFile != "" {
// Use config file from the flag.
config.SetConfigFile(cfgFile)
} else {
config.AddConfigPath(".")
config.SetConfigName(".example")
}
config.AutomaticEnv() // read in environment variables that match
// If a config file is found, read it in.
if err := config.ReadInConfig(); err == nil {
fmt.Fprintln(os.Stderr, "Using config file:", config.ConfigFileUsed())
}
// *** This line triggers a nil pointer reference.
fmt.Printf("name is %s\n", config.GetString("name"))
}
This code will panic with a nil pointer reference at the final call to
fmt.Printf:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x6a90e5]
If I move the call to config.BindPFlag from the NewCmdRoot
function to the top of the initConfig command, everything runs
without a problem.
What's going on here? According to the Viper docs regarding the use of
BindPFlags:
Like BindEnv, the value is not set when the binding method is
called, but when it is accessed. This means you can bind as early as
you want, even in an init() function.
That's almost exactly what I'm doing here. At the time I call
config.BindPflag, config is non-nil, cmd is non-nil, and the
name argument has been registered.
I assume there's something going on with my use of config in a
closure in PersistentPreRun, but I don't know exactly why that is
causing this failure.
I thought this was interesting so I did some digging and found your exact problem documented in an issue. The problematic line is this:
config.BindPFlag("name", cmd.Flags().Lookup("name"))
// ^^^^^^^
You created a persistent flag, but bound the flag to the Flags property. If you change your code to bind to PersistentFlags, everything will work as intended even with this line in NewCmdRoot:
config.BindPFlag("name", cmd.PersistentFlags().Lookup("name"))
I don't have any issue if I use cmd.PersistentFlags().Lookup("name").
// *** If I move this to the top of initConfig
// *** the code runs correctly.
pflag := cmd.PersistentFlags().Lookup("name")
config.BindPFlag("name", pflag)
Considering you just registered persistent flags (flag will be available to the command it's assigned to as well as every command under that command), it is safer to call cmd.PersistentFlags().Lookup("name"), rather than cmd.Flags().Lookup("name").
The latter returns nil, since the PersistentPreRun is only called when rootCmd.Execute() is called, which is after cmd.NewCmdRoot().
At cmd.NewCmdRoot() levels, flags have not yet been initialized, even after some were declared "persistent".
This ends up being a little more complex than it might appear at first glance, so while the other answers here helped me resolve the problem I'd like to add a little detail.
There are some nuances in the documentation that aren't particularly clear if you're just starting to work with Cobra. Let's start with the documentation for the PersistentFlags method:
PersistentFlags returns the persistent FlagSet specifically set in the current command.
The key is in ...in the current command. In my NewCmdRoot root method, we can use cmd.PersistentFlags() because the root command is the current command. We can even use cmd.PersistentFlags() in the PersistentPreRun method, as long as we're not processing a subcommand.
If we were to re-write cmd/root.go from the example so that it includes a subcommand, like this...
package cmd
import (
"fmt"
"os"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var cfgFile string
func NewCmdSubcommand() *cobra.Command {
var cmd = &cobra.Command{
Use: "subcommand",
Short: "An example subcommand",
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("This is an example subcommand\n")
},
}
return cmd
}
func NewCmdRoot() *cobra.Command {
config := viper.New()
var cmd = &cobra.Command{
Use: "example",
Short: "A brief description of your application",
PersistentPreRun: func(cmd *cobra.Command, args []string) {
initConfig(cmd, config)
},
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("Hello, world\n")
},
}
cmd.PersistentFlags().StringVar(
&cfgFile, "config", "", "config file (default is $HOME/.example.yaml)")
cmd.PersistentFlags().String("name", "", "a name")
cmd.AddCommand(NewCmdSubcommand())
err := config.BindPFlag("name", cmd.PersistentFlags().Lookup("name"))
if err != nil {
panic(err)
}
return cmd
}
func initConfig(cmd *cobra.Command, config *viper.Viper) {
name, err := cmd.PersistentFlags().GetString("name")
if err != nil {
panic(err)
}
fmt.Printf("name = %s\n", name)
if cfgFile != "" {
// Use config file from the flag.
config.SetConfigFile(cfgFile)
} else {
config.AddConfigPath(".")
config.SetConfigName(".example")
}
config.AutomaticEnv() // read in environment variables that match
// If a config file is found, read it in.
if err := config.ReadInConfig(); err == nil {
fmt.Fprintln(os.Stderr, "Using config file:", config.ConfigFileUsed())
}
// *** This line triggers a nil pointer reference.
fmt.Printf("name is %s\n", config.GetString("name"))
}
...we would find that it works when executing the root command:
$ ./example
name =
name is
Hello, world
But it fails when we run the subcommand:
[lars#madhatter go]$ ./example subcommand
panic: flag accessed but not defined: name
goroutine 1 [running]:
example/cmd.initConfig(0xc000172000, 0xc0001227e0)
/home/lars/tmp/go/cmd/root.go:55 +0x368
example/cmd.NewCmdRoot.func1(0xc000172000, 0x96eca0, 0x0, 0x0)
/home/lars/tmp/go/cmd/root.go:32 +0x34
github.com/spf13/cobra.(*Command).execute(0xc000172000, 0x96eca0, 0x0, 0x0, 0xc000172000, 0x96eca0)
/home/lars/go/pkg/mod/github.com/spf13/cobra#v1.1.3/command.go:836 +0x231
github.com/spf13/cobra.(*Command).ExecuteC(0xc00011db80, 0x0, 0xffffffff, 0xc0000240b8)
/home/lars/go/pkg/mod/github.com/spf13/cobra#v1.1.3/command.go:960 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
/home/lars/go/pkg/mod/github.com/spf13/cobra#v1.1.3/command.go:897
main.main()
/home/lars/tmp/go/main.go:11 +0x2a
This is because the subcommand inherits the PersistentPreRun command from the root (this is what the Persistent part means), but when this method runs, the cmd argument passwd to PersistentPreRun is no longer the root command; it's the subcommand command. When we try to call cmd.PersistentFlags(), it fails because the current command doesn't have any persistent flags associated with it.
In this case, we need to instead use the Flags method:
Flags returns the complete FlagSet that applies to this command (local and persistent declared here and by all parents).
This gives us access to persistent flags declared by parents.
An additional issue, that doesn't appear to be called out explicitly in the documentation, is that Flags() is only available after command processing has been run (that is, after you call cmd.Execute() on the command or a parent). That means we can use it in PersistentPreRun, but we can't use it in NewCmdRoot (because that method finishes before we process the command line).
TL;DR
We have to use cmd.PersistentFlags() in NewCmdRoot because we're looking for persistent flags applied to the current command, and the value from Flags() won't be available yet.
We need to use cmd.Flags() in PersistentPreRun (and other persistent commands methods) because when processing a subcommand, PersistentFlags will only look for persistent flags on the current command, but won't traverse parents. We need to use cmd.Flags() instead, which will roll up persistent flags declared by parents.
Does testing package support snapshot testing?
Here is my case:
package main
import (
"bytes"
"fmt"
"html/template"
)
func main() {
query := `
INSERT INTO "ADGROUP_PERFORMANCE_REPORT" (
{{.columnPrefix}}_adgroup_id,
{{.columnPrefix}}_adgroup_nme,
{{.columnPrefix}}_adgroup_status,
{{.columnPrefix}}_campaign_id,
{{.columnPrefix}}_campaign_nme,
{{.columnPrefix}}_campaign_status,
{{.columnPrefix}}_clicks,
{{.columnPrefix}}_impressions,
{{.columnPrefix}}_ctr,
{{.columnPrefix}}_average_cpc,
{{.columnPrefix}}_cost,
{{.columnPrefix}}_conversions,
{{.columnPrefix}}_average_position,
{{.columnPrefix}}_device,
google_adwords_client_customer_id
) VALUES
`
vars := make(map[string]interface{})
vars["columnPrefix"] = "adgroup_performance_report"
result := processString(query, vars)
fmt.Printf("result=%s\n", result)
}
func process(t *template.Template, vars interface{}) string {
var tmplBytes bytes.Buffer
err := t.Execute(&tmplBytes, vars)
if err != nil {
panic(err)
}
return tmplBytes.String()
}
func processString(str string, vars interface{}) string {
tmpl, err := template.New("tmpl").Parse(str)
if err != nil {
panic(err)
}
return process(tmpl, vars)
}
Now I am going to write unit test for it, I would like use snapshot testing to test the structure of the SQL query string processed by html/template pkg.
Here is the output in the stdout:
result=
INSERT INTO "ADGROUP_PERFORMANCE_REPORT" (
adgroup_performance_report_adgroup_id,
adgroup_performance_report_adgroup_nme,
adgroup_performance_report_adgroup_status,
adgroup_performance_report_campaign_id,
adgroup_performance_report_campaign_nme,
adgroup_performance_report_campaign_status,
adgroup_performance_report_clicks,
adgroup_performance_report_impressions,
adgroup_performance_report_ctr,
adgroup_performance_report_average_cpc,
adgroup_performance_report_cost,
adgroup_performance_report_conversions,
adgroup_performance_report_average_position,
adgroup_performance_report_device,
google_adwords_client_customer_id
) VALUES
I don't want to write this expected value duplicately in unit test file and assert it. I prefer using snapshot testing and it will generate a snapshot file. Something like jestjs snapshot-testing
As far as I know, the testing Package does not support something like this out of the box. There is a pattern for Go that you can utilise called "Golden file testing". The convention is to store testdata in a testdata folder alongside your test. In this case you would store the rendered template in a so called "golden file". The test itself provides an update flag to write out the latest version (so that you don't have to manually maintain the output):
var update = flag.Bool("update", false, "update .golden files")
func TestProcessString(t *testing.T) {
vars := make(map[string]interface{})
vars["columnPrefix"] = "adgroup_performance_report"
actual := processString(query, vars)
golden := filepath.Join(“testdata”, ”performance_report.golden”)
if *update {
ioutil.WriteFile(golden, actual, 0644)
}
expected, _ := ioutil.ReadFile(golden)
if !bytes.Equal(actual, expected) {
t.Fatalf("Output did not match, expected %v, recieved %v, expected, actual)
}
}
A nice example of this pattern can be found in the gofmt source code: https://golang.org/src/cmd/gofmt/gofmt_test.go
This is probably too late as an answer, but I have been working on a snapshot testing library for Golang that is "like" jest toMatchSnapshot.
I believe it matches exactly your use case, how to test long strings that you don't want to keep that string on or unit tests.
It would be as simple as and also provides a nice diff view
func TestExample(t *testing.T) {
snaps.MatchSnapshot(t ,processString(query, vars))
}
You can have a look go-snaps.
I have been trying to run a command and parse the output in golang. Here is a sample of what I am trying to do:
package main
import (
"fmt"
"os/exec"
)
func main() {
out,err := exec.Command("ls -ltr").Output()
if err != nil {
fmt.Println("Error: %s", err)
}
fmt.Printf("%s",out)
}
Now, when I am trying to run "ls -ltr", I get this error:
Error: %s exec: "ls -ltr": executable file not found in $PATH
So, basically go is looking for whole "ls -ltr" in PATH. And it's not there obviously. Is there any way I can pass a flag to any argument?TIA.
You pass arguments to the program by passing more arguments to the function - it's variadic:
out,err := exec.Command("ls","-ltr").Output()
https://golang.org/pkg/os/exec/#Command
This is a pretty common convention with exec-style functions which you will see in most languages. The other common pattern is builders.
Sometimes the layout of arguments you need to pass won't be known at compile-time (though it's not a good idea to send arbitrary commands to the system - stay safe!). If you want to pass an unknown number of arguments, you can use an array with some special syntax:
// Populate myArguments however you like
myArguments := []string{"bar","baz"}
// Pass myArguments with "..." to use variadic behaviour
out,err := exec.Command("foo", myArguments...).Output()
I would like to run my program like this:
go run launch.go http://example.com --m=2 --strat=par
"http://example.com" gets interpreted as the first command line argument, which is ok, but the flags are not parsed after that and stay at the default value. If I put it like this:
go run launch.go --m=2 --strat=par http://example.com
then "--m=2" is interpreted as the first argument (which should be the URL).
I could also just remove the os.Args completely, but then I would have only optional flags and I want one (the URL) to be mandatory.
Here's my code:
package main
import (
"fmt"
"webcrawler/crawler"
"webcrawler/model"
"webcrawler/urlutils"
"os"
"flag"
)
func main() {
if len(os.Args) < 2 {
log.Fatal("Url must be provided as first argument")
}
strategy := flag.String("strat", "par", "par for parallel OR seq for sequential crawling strategy")
routineMultiplier := flag.Int("m", 1, "Goroutine multiplier. Default 1x logical CPUs. Only works in parallel strategy")
page := model.NewBasePage(os.Args[1])
urlutils.BASE_URL = os.Args[1]
flag.Parse()
pages := crawler.Crawl(&page, *strategy, *routineMultiplier)
fmt.Printf("Crawled: %d\n", len(pages))
}
I am pretty sure that this should be possible, but I can't figure out how.
EDIT:
Thanks justinas for the hint with the flag.Args(). I now adapted it like this and it works:
...
flag.Parse()
args := flag.Args()
if len(args) != 1 {
log.Fatal("Only one argument (URL) allowed.")
}
page := model.NewBasePage(args[0])
...
os.Args doesn't really know anything about the flag package and contains all command-line arguments. Try flag.Args() (after calling flag.Parse(), of course).
As a followup, to parse flags that follow a command like
runme init -m thisis
You can create your own flagset to skip the first value like
var myValue string
mySet := flag.NewFlagSet("",flag.ExitOnError)
mySet.StringVar(&myValue,"m","mmmmm","something")
mySet.Parse(os.Args[2:])
This tripped me up too, and since I call flag.String/flag.Int64/etc in a couple of places in my app, I didn't want to have to pass around a new flag.FlagSet all over the place.
// If a commandline app works like this: ./app subcommand -flag -flag2
// `flag.Parse` won't parse anything after `subcommand`.
// To still be able to use `flag.String/flag.Int64` etc without creating
// a new `flag.FlagSet`, we need this hack to find the first arg that has a dash
// so we know when to start parsing
firstArgWithDash := 1
for i := 1; i < len(os.Args); i++ {
firstArgWithDash = i
if len(os.Args[i]) > 0 && os.Args[i][0] == '-' {
break
}
}
flag.CommandLine.Parse(os.Args[firstArgWithDash:])
The reason I went with this is because flag.Parse just calls flag.CommandLine.Parse(os.Args[1:]) under the hood anyway.
You can check if the Arg starts with "--" or "-" and avoid using that Arg in a loop.
For example:
for _, file := range os.Args[1:] {
if strings.HasPrefix(file, "--") {
continue
}
//do stuff
}