Global flags and subcommands - go

I'm implementing a little CLI with multiple subcommands. I'd like to support global flags, that is flags that apply to all subcommands to avoid repeating them.
For example, in the example below I'm trying to have -required flag that is required for all subcommands.
package main
import (
"flag"
"fmt"
"log"
"os"
)
var (
required = flag.String(
"required",
"",
"required for all commands",
)
fooCmd = flag.NewFlagSet("foo", flag.ExitOnError)
barCmd = flag.NewFlagSet("bar", flag.ExitOnError)
)
func main() {
flag.Parse()
if *required == "" {
fmt.Println("-required is required for all commands")
}
switch os.Args[1] {
case "foo":
fooCmd.Parse(os.Args[2:])
fmt.Println("foo")
case "bar":
barCmd.Parse(os.Args[2:])
fmt.Println("bar")
default:
log.Fatalf("[ERROR] unknown subcommand '%s', see help for more details.", os.Args[1])
}
}
I would expect usage to be like:
$ go run main.go foo -required helloworld
but if I ran that with the above code I get:
$ go run main.go foo -required hello
-required is required for all commands
flag provided but not defined: -required
Usage of foo:
exit status 2
It looks like flag.Parse() is not capturing -required from the CLI, and then the fooCmd is complaining that I've given it a flag it doesn't recognize.
What's the easiest way to have subcommands with global flags in Golang?

If you intend to implement subcommands, you shouldn't call flag.Parse().
Instead decide which subcommand to use (as you did with os.Args[1]), and call only its FlagSet.Parse() method.
Yes, for this to work, all flag sets should contain the common flags. But it's easy to register them once (in one place). Create a package level variable:
var (
required string
fooCmd = flag.NewFlagSet("foo", flag.ExitOnError)
barCmd = flag.NewFlagSet("bar", flag.ExitOnError)
)
And use a loop to iterate over all flagsets, and register the common flags, pointing to your variable using FlagSet.StringVar():
func setupCommonFlags() {
for _, fs := range []*flag.FlagSet{fooCmd, barCmd} {
fs.StringVar(
&required,
"required",
"",
"required for all commands",
)
}
}
And in main() call Parse() of the appropriate flag set, and test required afterwards:
func main() {
setupCommonFlags()
switch os.Args[1] {
case "foo":
fooCmd.Parse(os.Args[2:])
fmt.Println("foo")
case "bar":
barCmd.Parse(os.Args[2:])
fmt.Println("bar")
default:
log.Fatalf("[ERROR] unknown subcommand '%s', see help for more details.", os.Args[1])
}
if required == "" {
fmt.Println("-required is required for all commands")
}
}
You can improve the above solution by creating a map of flag sets, so you can use that map to register common flags, and also to do the parsing.
Full app:
var (
required string
fooCmd = flag.NewFlagSet("foo", flag.ExitOnError)
barCmd = flag.NewFlagSet("bar", flag.ExitOnError)
)
var subcommands = map[string]*flag.FlagSet{
fooCmd.Name(): fooCmd,
barCmd.Name(): barCmd,
}
func setupCommonFlags() {
for _, fs := range subcommands {
fs.StringVar(
&required,
"required",
"",
"required for all commands",
)
}
}
func main() {
setupCommonFlags()
cmd := subcommands[os.Args[1]]
if cmd == nil {
log.Fatalf("[ERROR] unknown subcommand '%s', see help for more details.", os.Args[1])
}
cmd.Parse(os.Args[2:])
fmt.Println(cmd.Name())
if required == "" {
fmt.Println("-required is required for all commands")
}
}

Put the global flags before the subcommand:
go run . -required=x foo.
Use flag.Args() instead of os.Args:
package main
import (
"flag"
"fmt"
"log"
"os"
)
var (
required = flag.String(
"required",
"",
"required for all commands",
)
fooCmd = flag.NewFlagSet("foo", flag.ExitOnError)
barCmd = flag.NewFlagSet("bar", flag.ExitOnError)
)
func main() {
flag.Parse()
if *required == "" {
fmt.Println("-required is required for all commands")
}
args := flag.Args() // everything after the -required flag, e.g. [foo, -foo-flag-1, -foo-flag-2, ...]
switch args[0] {
case "foo":
fooCmd.Parse(args[1:])
fmt.Println("foo")
case "bar":
barCmd.Parse(args[1:])
fmt.Println("bar")
default:
log.Fatalf("[ERROR] unknown subcommand '%s', see help for more details.", args[0])
}
}
If you want to keep all flags together, after the subcommand, write a helper function that adds common flags to each FlagSet:
var (
fooCmd = flag.NewFlagSet("foo", flag.ExitOnError)
barCmd = flag.NewFlagSet("bar", flag.ExitOnError)
)
type globalOpts struct {
required string
}
func main() {
var opts globalOpts
addGlobalFlags(fooCmd, &opts)
addGlobalFlags(barCmd, &opts)
if opts.required == "" {
fmt.Println("-required is required for all commands")
}
// ...
}
func addGlobalFlags(fs *flag.FlagSet, opts *globalOpts) {
fs.StringVar(
&opts.required,
"required",
"",
"required for all commands",
)
}
Perhaps you can also combine the two approaches to make the global flags work in any position.

Maybe you would be interested in using https://github.com/spf13/cobra - it supports exactly this usecase and many others.

Related

Subcommand flags (options)

I'm trying to create a CLI application that will accept few arguments based on the subcommand by using the standard flag package. This is the code I'm trying to use:
package main
import (
"flag"
"fmt"
"os"
)
func main() {
fmt.Printf("start application\n")
fooCmd := flag.NewFlagSet("foo", flag.ExitOnError)
fooName := fooCmd.String("name", "", "name")
barCmd := flag.NewFlagSet("bar", flag.ExitOnError)
barLevel := barCmd.Int("level", 0, "level")
if len(os.Args) < 2 {
fmt.Println("expected 'foo' or 'bar' subcommands")
os.Exit(1)
}
switch os.Args[1] {
case "foo":
fooCmd.Parse(os.Args[2:])
fmt.Println(" name:", *fooName)
case "bar":
barCmd.Parse(os.Args[2:])
fmt.Println(" level:", *barLevel)
default:
fmt.Println("expected 'foo' or 'bar' subcommands")
os.Exit(1)
}
}
and by calling it with:
$ ./main foo -name=test
It's working as expected.
The issue is that I want to have another flag (let's say --loglevel=Debug) which should be called for any of those subcommands (foo/bar), something like:
$ ./main foo -name=test -loglevel=debug
One option would be to create the same flag (loglevel) for any of those subcommands, but I just wonder is there any other way to achieve this without duplicating the code?
In my case, I have about 6 subcommands and 4 "general" flags.
That is why I prefer using a third-party library, rather than the default flags package.
For, instance, with alecthomas/kong:
func TestPropagatedFlags(t *testing.T) {
var cli struct {
Flag1 string
Command1 struct {
Flag2 bool
Command2 struct{} `kong:"cmd"`
} `kong:"cmd"`
}
parser := mustNew(t, &cli)
_, err := parser.Parse([]string{"command-1", "command-2", "--flag-2", "--flag-1=moo"})
require.NoError(t, err)
require.Equal(t, "moo", cli.Flag1)
require.Equal(t, true, cli.Command1.Flag2)
}
Flag1 is global, and would apply to any subcommand.

Testing urfave/cli based applications with go

I'm writing a small CLI application in Golang using urfave/cli framework and I'd like to write tests for it, but I can't find any useful information on how to test CLI applications, specifically written with the urfave/cli library. I have a lot of flags in the application and some of them are mutually exclusive and I'd like a proper test to stay on top of them - does anyone have an idea how to do it the right way?
EDIT:
Consider the following minimal example of application with several flags and restrictions around them. How would you test these flags usage (requirements, exclusivity, etc.) and how they influence the functions when they're set or not?
package main
import (
"errors"
"fmt"
"os"
"github.com/urfave/cli"
)
func doSomething(flag1 string, flag2 string, flag3 bool, flag4 bool) error {
err := errors.New("something")
return err
}
func main() {
app := cli.NewApp()
app.Name = "greet"
app.Usage = "fight the loneliness!"
var flag1, flag2 string
var flag3, flag4 bool
app.Flags = []cli.Flag{
cli.StringFlag{
Name: "flag1",
Value: "",
Usage: "flag1",
Destination: &flag1,
},
cli.StringFlag{
Name: "flag2",
Value: "",
Usage: "flag2",
Destination: &flag2,
},
cli.BoolFlag{
Name: "flag3",
Usage: "flag3",
Destination: &flag3,
},
cli.BoolFlag{
Name: "flag4",
Usage: "flag4",
Destination: &flag4,
},
}
app.Action = func(c *cli.Context) error {
if flag1 != "" && c.NumFlags() > 1 {
fmt.Println("--flag1 flag cannot be used with any other flags")
cli.ShowAppHelp(c)
os.Exit(1)
}
if flag1 == "" && flag2 == "" || c.NumFlags() < 1 {
fmt.Println("--flag2 is required")
cli.ShowAppHelp(c)
os.Exit(1)
}
if flag3 && flag4 {
fmt.Println("--flag3 and --flag4 flags are mutually exclusive")
cli.ShowAppHelp(c)
os.Exit(1)
}
err := doSomething(flag1, flag2, flag3, flag4)
return err
}
}
As Adrian correctly wrote
the same way you test anything else
Given a slightly modified example of the sample code of the project
package main
import (
"fmt"
"log"
"os"
"github.com/urfave/cli"
)
func Friend(c *cli.Context) error {
fmt.Println("Hello friend!")
return nil
}
func main() {
app := cli.NewApp()
app.Name = "greet"
app.Usage = "fight the loneliness!"
app.Action = Friend
err := app.Run(os.Args)
if err != nil {
log.Fatal(err)
}
}
Since this code actually prints something instead of returning a value you can evaluate, you could use a testable example
func ExampleFriend(){
// Yeah, technically, we can save the error check with the code above
// but this illustrates how you can make sure the output
// is not what the testable example expects.
if err := Friend(nil){
fmt.Printf("Friend: %s",err)
}
// Output:
// Hello friend!
}
Note that Action expects an ActionFunc. Where you define that ActionFunc is pretty much your thing. It could even come from a different package. So it is your design on how good your application will be testable.
Edit The signature of the value Action expects will change in the future, at least according to the docs. I already find it questionable to use interface{} to be able to pass nil to Action, and then check and type assert for ActionFunc, where a no-op ActionFunc would actually serve the same purpose, but removing an error return value really makes me scratch my head. I strongly recommend to have a look at alecthomas/kingpin for smaller to medium size applications or spf13/cobra, which is suitable even for the most complex of cli applications.

Check if Flag Was Provided in Go

With the flag package, is there a good way to distinguish if a string flag was passed?
For example, when the flag is not passed, I want to set it to a dynamic default value. However, I want to set it to empty if the flag was provided but with a value of "".
Current I am doing the following:
flagHost = flag.String(flagHostFlagKey, "", "...")
...
setHostname := false
for _, arg := range os.Args {
if arg == "-"+flagHostFlagKey {
setHostname = true
}
}
if !setHostname {
...
Which seems to work fine, but is kind of ugly. Is there a better way while staying with the standard flag package?
Use the flag.Visit()
Description:
Visit visits the command-line flags in lexicographical order, calling fn for each. It visits only those flags that have been set.
use:
func isFlagPassed(name string) bool {
found := false
flag.Visit(func(f *flag.Flag) {
if f.Name == name {
found = true
}
})
return found
}
The built-in flag types don't support distinguishing default values and explicit assignment to the default value. However, the flag package is quite flexible, and allows you to roll your own type that does, using the flag.Value interface.
Here's a full example that contains a string flag which records if it's been assigned to.
package main
import (
"flag"
"fmt"
)
type stringFlag struct {
set bool
value string
}
func (sf *stringFlag) Set(x string) error {
sf.value = x
sf.set = true
return nil
}
func (sf *stringFlag) String() string {
return sf.value
}
var filename stringFlag
func init() {
flag.Var(&filename, "filename", "the filename")
}
func main() {
flag.Parse()
if !filename.set {
fmt.Println("--filename not set")
} else {
fmt.Printf("--filename set to %q\n", filename.value)
}
}
Here's some example runs:
$ go run a.go -filename=abc
--filename set to "abc"
$ go run a.go -filename=
--filename set to ""
$ go run a.go
--filename not set
The issue with using a custom flag type (the stringFlag example in this thread) is you'll slightly upset the PrintDefaults output (i.e. --help). For example, with a string username flag and a stringFlag servername flag, --help looks like this:
-server value
server:port (default localhost:1234)
-username string
username (default "kimmi")
Note these are both string arguments as far as the user is concerned, but presented differently as a stringFlag is not a string.
flag's Flagset has an internal map that includes the flags that were declared ('formal') and those actually set ('actual'). The former is available via Lookup(), though alas the latter is not exposed, or you could just write:
var servername = flag.String("server", "localhost:8129", "server:port")
flag.Parse()
if f := flag.CommandLine.LookupActual("server"); f != nil {
fmt.Printf("server set to %#v\n", f)
} else {
fmt.Printf("server not set\n")
}
Seems like the best you can do, if you want consistent PrintDefaults() output, is to use Visit to extract your own view of 'actual' (VisitAll does the same thing with 'formal'):
var servername = flag.String("server", "localhost:8129", "server:port")
flag.Parse()
flagset := make(map[string]bool)
flag.Visit(func(f *flag.Flag) { flagset[f.Name]=true } )
if flagset["server"] {
fmt.Printf("server set via flags\n")
} else {
fmt.Printf("server not explicitly set, using default\n")
}
To use a dynamic default value for a flag, create the flag with the default set to the dynamic value:
func main() {
flagHost = flag.String(flagHostFlagKey, computedHostFlag(), "...")
flag.Parse()
// *flagHost equals the return value from computedHostFlag() if
// the flag is not specified on the command line.
...
}
With this approach, it's not necessary to detect if the flag is specified on the command line. Also, help shows the correct default.
If the computed value depends on other flags or is expensive to calculate, then use the approach suggested in one of the other answers.
Face with same problem, but have even complex case with bool flag, in this case computedHostFlag() not working, since you can provide to flag creation only true or false. "type stringFlag struct" solution also not the best, since ruin idea of default values.
Solve it in this way: create two sets of flags, with different default values, after parse - just check - if flag in first flagset have the same value that flag from second flagset - that it means that flag value was provided by user from command line. If they different - than this mean that flag was set by default.
package main
import (
"fmt"
"flag"
)
func main() {
args := []string{"-foo="}
flagSet1 := flag.NewFlagSet("flagSet1", flag.ContinueOnError)
foo1 := flagSet1.String("foo", "-", ``)
boolFoo1 := flagSet1.Bool("boolfoo", false, ``)
flagSet1.Parse(args)
flagSet2 := flag.NewFlagSet("flagSet2", flag.ContinueOnError)
foo2 := flagSet2.String("foo", "+", ``)
boolFoo2 := flagSet2.Bool("boolfoo", true, ``)
flagSet2.Parse(args)
if *foo1 != *foo2 {
fmt.Println("foo flag set by default")
} else {
fmt.Println("foo flag provided by user")
}
if *boolFoo1 != *boolFoo2 {
fmt.Println("boolfoo flag set by default")
} else {
fmt.Println("boolfoo flag provided by user")
}
}
playground: https://play.golang.org/p/BVceE_pN5PO , for real CLI execution, you can do something like that: https://play.golang.org/p/WNvDaaPj585
Same as https://stackoverflow.com/a/35809400/3567989 but with a pointer to a string instead of a custom struct. The *string is nil if unset, non-nil if set.
package main
import (
"flag"
"fmt"
)
type stringPtrFlag struct {
ptr **string
}
func (f stringPtrFlag) String() string {
if *f.ptr == nil {
return ""
}
return **f.ptr
}
func (f stringPtrFlag) Set(s string) error {
*f.ptr = &s
return nil
}
var filename *string
func init() {
flag.Var(stringPtrFlag{&filename}, "filename", "the filename")
}
func main() {
flag.Parse()
if filename == nil {
fmt.Println("--filename not set")
} else {
fmt.Printf("--filename set to %q\n", *filename)
}
}
I think a more reliable way is to check whether any flag in the command-line parameters (os.Args[1:]) is prefixed by "prefix" + str, so the function:
func isInSlice(str string, list []string, prefix string) bool {
for _, v := range list {
if strings.HasPrefix(v, prefix + str) {
return true
}
}
return false
}
I found that we have the Lookup() method:
func isFlagPassed(name string) bool {
rs := flag.Lookup(name)
return rs != nil
}
Full docs
The FlagSet does not have a function LookupActual() in my environment (go version go1.13.4 windows/amd64), and the internal map actual mentioned in Ben L's answer can not be accessed directly.
I have an approach to check if a flag is set using reflect:
import "reflect"
fs := flag.NewFlagSet("the flags", flag.ExitOnError)
flag_name := "host"
host := fs.String(flag_name, "localhost", "specify the host address")
// other flags
fs.Parse(os.Args[1:])
if reflect.Indirect(reflect.ValueOf(fs)).FieldByName("actual").MapIndex(reflect.ValueOf(flag_name)).IsValid() {
fmt.Printf("the host flag is set with value %v", *host)
} else {
fmt.Printf("the host flag is not set")
}

GO: find/scan for structs/functions

Is it possible in GO to find structs or functions by criteria such as name, tag or interface? i.e something along the lines of command line tasks/verbs? i.e:
func cmd1() {
...
}
func cmd2() {
...
}
...
func cmdN() {
}
func main() {
// Inspect os.Args and call cmd{X}() based on args.
...
}
I don't mind what the exact mechanism is and if the final targets are functions or structs - the goal is to get something working by convention without any boilerplate code.
You could use reflection
package main
import (
"flag"
"fmt"
"reflect"
)
var cmd command
type command struct{}
func (c command) execute(name string) {
v := reflect.ValueOf(c)
cmd := v.MethodByName(name)
if !cmd.IsValid() {
fmt.Println(name + " not a command")
return
}
cmd.Call([]reflect.Value{})
}
func (c command) Cmd1() {
fmt.Println("command 1")
}
func (c command) Cmd2() {
fmt.Println("command 2")
}
func (c command) Cmd3() {
fmt.Println("command 3")
}
func main() {
flag.Parse()
cmd.execute(flag.Arg(0))
}
or you could use a map.
package main
import (
"flag"
"fmt"
)
func cmd1() {
fmt.Println("command 1")
}
func cmd2() {
fmt.Println("command 2")
}
func cmd3() {
fmt.Println("command 3")
}
var funcs = map[string]func(){
"cmd1": cmd1,
"cmd2": cmd2,
"cmd3": cmd3,
}
func main() {
flag.Parse()
if f, ok := funcs[flag.Arg(0)]; ok {
f()
} else {
fmt.Println(flag.Arg(0) + " command not found")
}
}
I used a similar approach in "How to test a collection of functions by reflection in Go?"
The idea is to list and find all the functions needed, in my case, functions for a certain struct type:
stype := reflect.ValueOf(s)
for _, fname := range funcNames {
sfunc := stype.MethodByName(fname)
// no parameter => empty slice of Value
ret := sfunc.Call([]reflect.Value{})

Flag command line parsing in golang

I'm not sure I understand the reasoning behind this example (taken from here), nor what it is trying to communicate about the Go language:
package main
import (
"flag"
"fmt"
)
func main() {
f := flag.NewFlagSet("flag", flag.ExitOnError)
f.Bool("bool", false, "this is bool flag")
f.Int("int", 0, "this is int flag")
visitor := func(a *flag.Flag) {
fmt.Println(">", a.Name, "value=", a.Value)
}
fmt.Println("Visit()")
f.Visit(visitor)
fmt.Println("VisitAll()")
f.VisitAll(visitor)
// set flags
f.Parse([]string{"-bool", "-int", "100"})
fmt.Println("Visit() after Parse()")
f.Visit(visitor)
fmt.Println("VisitAll() after Parse()")
f.VisitAll(visitor)
}
Something along the lines of the setup they have but then adding a
int_val := f.get("int")
to get the named argument would seem more useful. I'm completely new to Go, so just trying to get acquainted with the language.
This is complicated example of using flag package. Typically flags set up this way:
package main
import "flag"
// note, that variables are pointers
var strFlag = flag.String("long-string", "", "Description")
var boolFlag = flag.Bool("bool", false, "Description of flag")
func init() {
// example with short version for long flag
flag.StringVar(strFlag, "s", "", "Description")
}
func main() {
flag.Parse()
println(*strFlag, *boolFlag)
}

Resources