I would like to run my program like this:
go run launch.go http://example.com --m=2 --strat=par
"http://example.com" gets interpreted as the first command line argument, which is ok, but the flags are not parsed after that and stay at the default value. If I put it like this:
go run launch.go --m=2 --strat=par http://example.com
then "--m=2" is interpreted as the first argument (which should be the URL).
I could also just remove the os.Args completely, but then I would have only optional flags and I want one (the URL) to be mandatory.
Here's my code:
package main
import (
"fmt"
"webcrawler/crawler"
"webcrawler/model"
"webcrawler/urlutils"
"os"
"flag"
)
func main() {
if len(os.Args) < 2 {
log.Fatal("Url must be provided as first argument")
}
strategy := flag.String("strat", "par", "par for parallel OR seq for sequential crawling strategy")
routineMultiplier := flag.Int("m", 1, "Goroutine multiplier. Default 1x logical CPUs. Only works in parallel strategy")
page := model.NewBasePage(os.Args[1])
urlutils.BASE_URL = os.Args[1]
flag.Parse()
pages := crawler.Crawl(&page, *strategy, *routineMultiplier)
fmt.Printf("Crawled: %d\n", len(pages))
}
I am pretty sure that this should be possible, but I can't figure out how.
EDIT:
Thanks justinas for the hint with the flag.Args(). I now adapted it like this and it works:
...
flag.Parse()
args := flag.Args()
if len(args) != 1 {
log.Fatal("Only one argument (URL) allowed.")
}
page := model.NewBasePage(args[0])
...
os.Args doesn't really know anything about the flag package and contains all command-line arguments. Try flag.Args() (after calling flag.Parse(), of course).
As a followup, to parse flags that follow a command like
runme init -m thisis
You can create your own flagset to skip the first value like
var myValue string
mySet := flag.NewFlagSet("",flag.ExitOnError)
mySet.StringVar(&myValue,"m","mmmmm","something")
mySet.Parse(os.Args[2:])
This tripped me up too, and since I call flag.String/flag.Int64/etc in a couple of places in my app, I didn't want to have to pass around a new flag.FlagSet all over the place.
// If a commandline app works like this: ./app subcommand -flag -flag2
// `flag.Parse` won't parse anything after `subcommand`.
// To still be able to use `flag.String/flag.Int64` etc without creating
// a new `flag.FlagSet`, we need this hack to find the first arg that has a dash
// so we know when to start parsing
firstArgWithDash := 1
for i := 1; i < len(os.Args); i++ {
firstArgWithDash = i
if len(os.Args[i]) > 0 && os.Args[i][0] == '-' {
break
}
}
flag.CommandLine.Parse(os.Args[firstArgWithDash:])
The reason I went with this is because flag.Parse just calls flag.CommandLine.Parse(os.Args[1:]) under the hood anyway.
You can check if the Arg starts with "--" or "-" and avoid using that Arg in a loop.
For example:
for _, file := range os.Args[1:] {
if strings.HasPrefix(file, "--") {
continue
}
//do stuff
}
Related
I want to test different (correct/incorrect) command line arguments passed to my CLI program, but I am not sure how to achieve this with go/testing package because I am getting flag redefined error. Looks like it happens because flag.Parse() can be called only once. What is the proper approach to test different command line arguments passed into the go program? Is there is any way to define something like setup()/teardown() or run every case in isolation (but in the same file)?
Here is my code:
Function to test:
func (p *Params) Parse() (*Params, error) {
param1Ptr := flag.String("param1", "default", "param1 desc")
param2Ptr := flag.String("param2", "default", "param1 desc")
...
...
flag.Parse()
...
}
Test file:
package main
import (
"os"
"testing"
)
func TestParam1(t *testing.T) {
os.Args = []string{"cmd", "-param1", "incorrect", "-param2", "correct"}
params := Params{}
_, err := params.Parse()
...
...
}
func TestParam2(t *testing.T) {
os.Args = []string{"cmd", "-param1", "correct", "-param2", "incorrect"}
params := Params{}
_, err := params.Parse()
...
...
}
Don't use the global FlagSet object in the flags package. Create your own FlagSet as a field of Params: https://golang.org/pkg/flag/#FlagSet
All that flag.String et al do is pass through the function call to a global FlagSet object in the flag package (specifically flag.CommandLine is the variable). This is easy to use but not a generally good practice. Using your own flagset would avoid the issues you described as well as other potential side effects from using global variables.
Clear the global FlagSet before each test using:
flag.CommandLine = flag.NewFlagSet(os.Args[0], flag.ExitOnError)
See How to unset flags Visited on command line in GoLang for Tests
I'm facing with a weird golang issue. The following code will clarify:
package main
import (
"os/exec"
"io"
"fmt"
"os"
)
var (
pw io.WriteCloser
pr io.ReadCloser
)
func main() {
term := exec.Command("/bin/sh")
// Get stdin writer pipe
pw, _ = term.StdinPipe()
pr, _ = term.StdoutPipe()
term.Start()
run("cd ~")
pwd := run("pwd");
// Do something with pwd output
...
term.Wait()
}
func run(c string) string {
io.WriteString(pw, fmt.Sprintln(c))
buf := make([]byte, 32 * 1024)
pr.Read(buf)
return string(buf)
}
I'd like to run some commands in a shell env and read their output. There's no problem on write/run command but it seems that there're some limitations while reading:
you can't know if a command doesn't output anything or not;
there's no way to check if stdout is ready to be read or not.
The pr.Read(dest) method will block the code flow until something is read from stdout. As said, the goal is to read sequentially (without using a go routine and/or an infinite loop). This means that if we send a cd command the func end is never reached.
Setting the non-block flag through unix.SetNonblock on stdout file descriptor seems to solve the above issue but you can't know prior if it's ready or not and an error saying "resource temporary not available" is returned from .Read call.
As Cerise Limón mentioned go functions whould be the way to go here, since these sorts of interactive scripting exercises are traditionally done with expect.
You can wrap the the parrellel execution into a library to it might still look like sequencial code, so this might be helpful: https://github.com/ThomasRooney/gexpect
From the readme:
child, err := gexpect.Spawn("python")
if err != nil { panic(err) }
child.Expect(">>>")
child.SendLine("print 'Hello World'")
child.Interact()
child.Close()
I have been trying to run a command and parse the output in golang. Here is a sample of what I am trying to do:
package main
import (
"fmt"
"os/exec"
)
func main() {
out,err := exec.Command("ls -ltr").Output()
if err != nil {
fmt.Println("Error: %s", err)
}
fmt.Printf("%s",out)
}
Now, when I am trying to run "ls -ltr", I get this error:
Error: %s exec: "ls -ltr": executable file not found in $PATH
So, basically go is looking for whole "ls -ltr" in PATH. And it's not there obviously. Is there any way I can pass a flag to any argument?TIA.
You pass arguments to the program by passing more arguments to the function - it's variadic:
out,err := exec.Command("ls","-ltr").Output()
https://golang.org/pkg/os/exec/#Command
This is a pretty common convention with exec-style functions which you will see in most languages. The other common pattern is builders.
Sometimes the layout of arguments you need to pass won't be known at compile-time (though it's not a good idea to send arbitrary commands to the system - stay safe!). If you want to pass an unknown number of arguments, you can use an array with some special syntax:
// Populate myArguments however you like
myArguments := []string{"bar","baz"}
// Pass myArguments with "..." to use variadic behaviour
out,err := exec.Command("foo", myArguments...).Output()
Sometimes I want to pass an argument on the command line with no name, say a command like wc, which takes a filename as input:
wc filename.txt
With the flag package, it looks like every flag has to be given a name, with a default value if unspecified.
filename := flag.String("filename", "foo.txt", "Which file to count the words for")
However I don't want a default value, I want the program to exit with an error code if an argument is not specified. How would I add a required argument to a Go binary?
I would also like to be able to parse arguments with type information, so just checking the Args() directly doesn't quite do it.
You just have to check flag.NArg().
From https://golang.org/pkg/flag/#NArg:
NArg is the number of arguments remaining after flags have been processed.
flag.Parse()
if flag.NArg() == 0 {
flag.Usage()
os.Exit(1)
}
You can also use the flag.Narg() function to ensure you have the required number of positional arguments, though I don't know what it gives you over len(flag.Args())
if flag.NArg() < minArgs {
// do something
...
}
In case anyone is unsatisfied with the standard flag package behavior of stopping the parse as soon as it sees the first positional arg, you can use these small library functions to parse flags in a way that flags can come before or after positional args.
// ParseFlags parses the command line args, allowing flags to be
// specified after positional args.
func ParseFlags() error {
return ParseFlagSet(flag.CommandLine, os.Args[1:])
}
// ParseFlagSet works like flagset.Parse(), except positional arguments are not
// required to come after flag arguments.
func ParseFlagSet(flagset *flag.FlagSet, args []string) error {
var positionalArgs []string
for {
if err := flagset.Parse(args); err != nil {
return err
}
// Consume all the flags that were parsed as flags.
args = args[len(args)-flagset.NArg():]
if len(args) == 0 {
break
}
// There's at least one flag remaining and it must be a positional arg since
// we consumed all args that were parsed as flags. Consume just the first
// one, and retry parsing, since subsequent args may be flags.
positionalArgs = append(positionalArgs, args[0])
args = args[1:]
}
// Parse just the positional args so that flagset.Args()/flagset.NArgs()
// return the expected value.
// Note: This should never return an error.
return flagset.Parse(positionalArgs)
}
How do I do “cursor-up” in Go? (Clear-to-end-of-line would also be good to know). (All platforms).
To elaborate and show the context, I’m writing a test program in Go that requires the input of some parameters (via console) that are stored in a text file and used as defaults for the next usage. I want to have some very rudimentary console “editing” features.
Currently it is fairly primitive because I don’t want to go deeply into console editing, I just want something fairly basic but also not too basic.
In the example below from my test program, the String variable “sPrompt” contains the prompt for the input, and to the right it shows the default and then there are backspace characters to position the cursor so that the default is not overwritten – like I said, very basic.
When the operator enters the input, if an error, I'd like to display an error message, and then in either case move the cursor up to the line just displayed/entered and if an error, then display the original line, or if correct, display just the prompt and the new parameter.
I did read somewhere that ReadLine() should be avoided, but it appears to do the job.
Example:
func fInputString(sPrompt string, asValid []string, sDefault string)
(sInput string, tEnd bool) {
oBufReader := bufio.NewReader(os.Stdin)
for {
print("\n" + sPrompt)
vLine, _, _ := oBufReader.ReadLine()
sInput = strings.ToLower(string(vLine))
if sInput == "end" {
return "", true
}
if sInput == "" {
sInput = sDefault
}
// check for valid input //
for _, sVal := range asValid {
if sInput == sVal {
return sInput, false
}
}
}
}
This is how sPrompt is constructed (not meant to be optimized):
if sDefault != "" {
for len(sPrompt) < 67 {
sPrompt += " "
}
sPrompt += sDefault
for iBack := 20 + len(sDefault); iBack > 0; iBack-- {
sPrompt += "\b"
}
}
With tput strings that control the cursor around the screen:
tput sc to save the cursor position
tput rc to restore the cursor position
Then using these strings in the Go function:
package main
import (
"fmt"
"time"
)
const sc = "\u001B7"
const rc = "\u001B8"
func main() {
fmt.Print(sc)
for i := 1; i <= 10; i++ {
fmt.Print(rc + sc)
fmt.Println(i, "one")
fmt.Println(i, "two")
fmt.Println(i, "three")
fmt.Println(i, "four")
fmt.Println(i, "five")
time.Sleep(time.Second)
}
}
Should work in most terminals.
Make your own little shell
You should not reinvent the wheel and use a library which does exactly what you want.
A popular option is the readline library which is apparently available for Windows
as well. This is used, for example, by bash and ZSH. There are some Go wrappers for it:
https://github.com/shavac/readline
https://github.com/bobappleyard/readline
I personally would recommend bobappleyard/readline as it is better documented
and has a nicer API (less C-like). There does not seem to be a special build tag for
Windows, so you might have to write it for yourself but that should be not that hard.
termbox and its (pure) Go implementation termbox-go which was already pointed out by #bgp does not seem to be good for simply reading a line as it seems to be more intended
for full screen console applications. Also, you would have to code the up/down matching
and history yourself.
.Readline()
The doc is right by saying that you should not use this as it does not handle anything for
you. For example, reading from a stream that emits partial data, you have no guarantee that you will get a full line from Readline. Use ReadSlice('\n') for that.