When I am running below code
package main
import (
"flag"
"fmt"
)
func main() {
i := flag.Int("i", 0, "Any integer value")
b := flag.Bool("b", false, "Any boolean value")
s := flag.String("s", "Hello", "Any string value")
flag.Parse()
fmt.Println("-i", *i)
fmt.Println("-b", *b)
fmt.Println("-s", *s)
}
go run main.go -i 33 -b true -s hi
-i 33
-b true
-s Hello
go run main.go -i 33 -s hi
-i 33
-b false
-s hi
go run main.go -i 33 -s hi -b true
-i 33
-b true
-s hi
go run main.go -i 33 -b true -s hi
-i 33
-b true
-s Hello
Why "-s" command line argument not working when it passed at the end
Thanks in advance
It is because of the -b boolean flag. A boolean flag tests for the existence of flag, it does not test for the argument to the flag. That is:
go run main.go -b
will output -b true, and
go run main.go
will output -b false.
go run main.go -b false
will output -b true, because the -b flag is given. false is not a recognized argument, so it stops processing there.
If you want to use true/false, you have to use this format:
go run main.go -i 33 -b=false -s hi
This should also work (here, -b is true):
go run main.go -i 33 -b -s hi
Related
Why could not get the values that following bool type arg in the command line?
Here's my test code:
import (
"flag"
"fmt"
)
var stringVal string
var intVal int
var boolValue bool
func init() {
flag.StringVar(&stringVal, "s", "", "string value")
flag.BoolVar(&boolValue, "b", false, "bool value")
flag.IntVar(&intVal, "i", -1, "int value")
}
func main() {
flag.Parse()
fmt.Println("stringVal:", stringVal, ", boolValue:", boolValue, ", intVal:", intVal)
}
Run it as:
go run flag.go -s test -b true -i 10
Got: stringVal: test , boolValue: true , intVal: -1
go run flag.go -s test -b false -i 10
Got: stringVal: test , boolValue: true , intVal: -1
go run flag.go -b false -s test -i 10
Got: stringVal: , boolValue: true , intVal: -1
go run flag.go -s test -i 10 -b false
Got: stringVal: test , boolValue: true , intVal: 10
Go version: 1.16
Boolean flags are set based on the presence or absence of the flag. They typically default to false, and the presence of the flag is used to modify the default behavior.
For example...
to display the long listing of a directory with ls, you use ls -l, not ls -l true
to make rm safer by having it interactively prompt for deletes, you use rm -i, not rm -i true
to display human-readable output with df, you use df -h, not df -h true
The true and false you're placing after the -b are being provided to your program as arguments, not as flags, and their presence interrupts further processing of flags.
Add the following to the end of your main function:
fmt.Println("Remaining args", flag.Args())
Given an invocation such as go run flag.go -b false -s test -i 10, you'll see that flag processing stopped at false, and that the remaining arguments are passed to your program as non-flag arguments:
$ go run flag.go -b false -s test -i 10
stringVal: , boolValue: true , intVal: -1
Remaining args [false -s test -i 10]
As an aside, the general philosophy behind "boolean" flags is that you give your program some default behavior, and provide two flags: One which modifies that default, and an opposite flag that negates the modification, restoring the default. When processing flags, the last flag will win.
For example, rm provides -i to make the command interactive, and -f to negate the -i flag.
This allows you to set alias rm="rm -i" in your shell, meaning all invocations of rm will have the -i flag applied for safety, prompting you before removing files. However, if you want to run a non-interactive rm, you can still type rm -f, which expands to be rm -i -f, and the last flag (-f) wins. Otherwise, you'd have destroy the alias, run rm and then restore the alias every time you wanted a non-interactive invocation.
A boolean flag tests the existence of the flag, not the value of it. If you want to set a value, use:
go run flag.go -s test -b=false -i 10
I am trying to download some files with snakemake. The files (http://snpeff.sourceforge.net/SnpSift.html#dbNSFP) I would like to download are on a google site/drive and my usual wget approach does not work. I found a bash function that does the job (https://www.zachpfeffer.com/single-post/wget-a-Google-Drive-file):
function gdrive_download () { CONFIRM=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id=$1" -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p') wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$CONFIRM&id=$1" -O $2 rm -rf /tmp/cookies.txt }
gdrive_download 120aPYqveqPx6jtssMEnLoqY0kCgVdR2fgMpb8FhFNHo test.txt
I have tested this function with my ids in a plain bash script and was able to download all the files. To add a bit to the complexity, I must use a workplace template, and incorporate the function into it.
rule dl:
params:
url = 'ftp://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_{genome}/{afile}'
output:
'data/{genome}/{afile}'
params:
id1 = '0B7Ms5xMSFMYlOTV5RllpRjNHU2s',
f1 = 'dbNSFP.txt.gz'
shell:
"""CONFIRM=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id={{params.id1}}" -O- | sed -rn "s/.*confirm=([0-9A-Za-z_]+).*/\1\n/p") && wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$CONFIRM&id={{params.id1}}" -O {{params.f1}} && rm -rf /tmp/cookies.txt"""
#'wget -c {params.url} -O {output}'
rule checksum:
input:
i = 'data/{genome}/{afile}'
output:
o = temp('tmp/{genome}/{afile}.md5')
shell:
'md5sum {input} > {output}'
rule file_size:
input:
i = 'data/{genome}/{afile}'
output:
o = temp('tmp/{genome}/{afile}.size')
shell:
'du -csh --apparent-size {input} > {output}'
rule file_info:
"""md5 checksum and file size"""
input:
md5 = 'tmp/{genome}/{afile}.md5',
s = 'tmp/{genome}/{afile}.size'
output:
o = temp('tmp/{genome}/info/{afile}.csv')
run:
with open(input.md5) as f:
md5, fp = f.readline().strip().split()
with open(input.s) as f:
size = f.readline().split()[0]
with open(output.o, 'w') as fout:
print('filepath,size,md5', file=fout)
print(f"{fp},{size},{md5}", file=fout)
rule manifest:
input:
expand('tmp/{genome}/info/{suffix}.csv', genome=('GRCh37','GRCh38'), suffix=('dbNSFP.txt.gz', 'dbNSFP.txt.gz.tbi'))
#expand('tmp/{genome}/info/SnpSift{suffix}.csv', genome=('GRCh37','GRCh38'), suffix=('dbNSFP.txt.gz', 'dbNSFP.txt.gz.tbi'))
output:
o = 'MANIFEST.csv'
run:
pd.concat([pd.read_csv(afile) for afile in input]).to_csv(output.o, index=False)
There are four downloadable files for which I have ids (I only show one in params), however I don't know how to call the bash functions as written by ZPfeffer for all the ids I have with snakemake. Additionally, when I run this script, there are several errors, the most pressing being
sed: -e expression #1, char 31: unterminated `s' command
I am far from a snakemake expert, any assistance on how to modify my script to a) call the functions with 4 different ids, b) remove the sed error, and c) verify whether this is the correct url format (currently url = 'https://docs.google.com/uc?export/{afile}) will be greatly appreciated.
You would want to use raw string literal so that snakemake doesn't escape special characters, such as backslash in sed command. For example (notice r in front of shell command):
rule foo:
shell:
r"sed d\s\"
You could use --printshellcmds or -p to see how exactly shell: commands get resolved by snakemake.
Here is how I "solved" it:
import pandas as pd
rule dl:
output:
'data/{genome}/{afile}'
shell:
"sh download_snpsift.sh"
rule checksum:
input:
i = 'data/{genome}/{afile}'
output:
o = temp('tmp/{genome}/{afile}.md5')
shell:
'md5sum {input} > {output}'
rule file_size:
input:
i = 'data/{genome}/{afile}'
output:
o = temp('tmp/{genome}/{afile}.size')
shell:
'du -csh --apparent-size {input} > {output}'
rule file_info:
"""md5 checksum and file size"""
input:
md5 = 'tmp/{genome}/{afile}.md5',
s = 'tmp/{genome}/{afile}.size'
output:
o = temp('tmp/{genome}/info/{afile}.csv')
run:
with open(input.md5) as f:
md5, fp = f.readline().strip().split()
with open(input.s) as f:
size = f.readline().split()[0]
with open(output.o, 'w') as fout:
print('filepath,size,md5', file=fout)
print(f"{fp},{size},{md5}", file=fout)
rule manifest:
input:
expand('tmp/{genome}/info/{suffix}.csv', genome=('GRCh37','GRCh38'), suffix=('dbNSFP.txt.gz', 'dbNSFP.txt.gz.tbi'))
output:
o = 'MANIFEST.csv'
run:
pd.concat([pd.read_csv(afile) for afile in input]).to_csv(output.o, index=False)
And here is the bash script.
function gdrive_download () {
CONFIRM=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id=$1" -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$CONFIRM&id=$1" -O $2
rm -rf /tmp/cookies.txt
}
gdrive_download 0B7Ms5xMSFMYlSTY5dDJjcHVRZ3M data/GRCh37/dbNSFP.txt.gz
gdrive_download 0B7Ms5xMSFMYlOTV5RllpRjNHU2s data/GRCh37/dbNSFP.txt.gz.tbi
gdrive_download 0B7Ms5xMSFMYlbTZodjlGUDZnTGc data/GRCh38/dbNSFP.txt.gz
gdrive_download 0B7Ms5xMSFMYlNVBJdFA5cFZRYkE data/GRCh38/dbNSFP.txt.gz.tbi
I try to use one tool and I need to use a wildcard present on input.
This is an example:
aDict = {"120":"121" } #tumor : normal
rule all:
input: expand("{case}.mutect2.vcf",case=aDict.keys())
def get_files_somatic(wildcards):
case = wildcards.case
control = aDict[wildcards.case]
return [case + ".sorted.bam", control + ".sorted.bam"]
rule gatk_Mutect2:
input:
get_files_somatic,
output:
"{case}.mutect2.vcf"
params:
genome="ref/hg19.fa",
target= "chr12",
name_tumor='{case}'
log:
"logs/{case}.mutect2.log"
threads: 8
shell:
" gatk-launch Mutect2 -R {params.genome} -I {input[0]} -tumor {params.name_tumor} -I {input[1]} -normal {wildcards.control}"
" -L {params.target} -O {output}"
I Have this error:
'Wildcards' object has no attribute 'control'
So I have a function with case and control. I'm not able to extract code.
The wildcards are derived from the output file/pattern. That is why you only have the wildcard called case. You have to derive the control from that. Try replacing your shell statement with this:
run:
control = aDict[wildcards.case]
shell(
"gatk-launch Mutect2 -R {params.genome} -I {input[0]} "
"-tumor {params.name_tumor} -I {input[1]} -normal {control} "
"-L {input.target2} -O {output}"
)
You could define control in params. Also {input.target2} in shell command would result in error. May be it's supposed to be params.target?
rule gatk_Mutect2:
input:
get_files_somatic,
output:
"{case}.mutect2.vcf"
params:
genome="ref/hg19.fa",
target= "chr12",
name_tumor='{case}',
control = lambda wildcards: aDict[wildcards.case]
shell:
"""
gatk-launch Mutect2 -R {params.genome} -I {input[0]} -tumor {params.name_tumor} \\
-I {input[1]} -normal {params.control} -L {params.target} -O {output}
"""
I'm passing argument from console. There are some flags too. Like:
go run test.go "-IP=10.10.10.10" "-db=kite" "-wv=45" "-cv=75" "A = value1" "B = value2" "C = 100" "D := ((A-B)/A)*C" "D ?"
Here, -IP, -db, -wv, -wc these four are flags and others are passing as normal argument as I know.
Number of flags can be variable.
How can I know how many flags are passed to my program. In this case 4 flags are passed.
If you use the standard flag package to parse command-line flags, you can call the NFlag function to get the number of flags:
package main
import "fmt"
import "flag"
func main() {
flag.Bool("a", true, "A value");
flag.Bool("b", true, "B value");
flag.Parse();
fmt.Println(flag.NFlag())
}
Test:
$ go run test.go
0
$ go run test.go -a
1
$ go run test.go -a -b
2
Is it possible to increment a minor version number automatically each time a Go app is compiled?
I would like to set a version number inside my program, with an autoincrementing section:
$ myapp -version
MyApp version 0.5.132
Being 0.5 the version number I set, and 132 a value that increments automatically each time the binary is compiled.
Is this possible in Go?
The Go linker (go tool link) has an option to set the value of an uninitialised string variable:
-X importpath.name=value
Set the value of the string variable in importpath named name to
value.
Note that before Go 1.5 this option took two separate arguments.
Now it takes one argument split on the first = sign.
As part of your build process, you could set a version string variable using this. You can pass this through the go tool using -ldflags. For example, given the following source file:
package main
import "fmt"
var xyz string
func main() {
fmt.Println(xyz)
}
Then:
$ go run -ldflags "-X main.xyz=abc" main.go
abc
In order to set main.minversion to the build date and time when building:
go build -ldflags "-X main.minversion=`date -u +.%Y%m%d.%H%M%S`" service.go
If you compile without initializing main.minversion in this way, it will contain the empty string.
Use ldflags to set variables in main package:
With file main.go:
package main
import "fmt"
var (
version string
build string
)
func main() {
fmt.Println("version=", version)
fmt.Println("build=", build)
}
Then run:
go run \
-ldflags "-X main.version=1.0.0 -X main.build=12082019" \
main.go
Build:
go build -o mybinary \
-ldflags "-X main.version=1.0.0 -X 'main.build=$(date)'" \
main.go
Use ldflags to set variable in a non-main package:
With file config.go:
package config
import "fmt"
var (
Version string
)
func LogVersion() {
fmt.Println("version=", Version)
}
You will also need file main.go:
package main
import (
"fmt"
"github.com/user/repo/config"
}
func main() {
config.LogVersion()
}
Build your binary first:
go build -o mybinary main.go
Find the full path of variable name you want to set:
go tool nm <path_to_binary> | grep Version
Run and build the binary again but with the ldflags:
go run \
-ldflags "-X github.com/user/repo/config.Version=1.0.0" \
main.go --version
go build -o mybinary \
-ldflags "-X github.com/user/repo/config.Version=1.0.0" \
main.go
Inspired by https://github.com/golang/go/wiki/GcToolchainTricks#including-build-information-in-the-executable
Also if you are using goreleaser then read this https://goreleaser.com/environment/#using-the-mainversion :
Default wise GoReleaser sets three ldflags:
main.version: Current Git tag
main.commit: Current git commit SHA
main.date: Date according RFC3339
If you want to see this in action: https://github.com/hoto/fuzzy-repo-finder/blob/master/pkg/config/config.go
Additionally I would like to post a small example how to use git and a makefile:
--- Makefile ----
# This how we want to name the binary output
BINARY=gomake
# These are the values we want to pass for VERSION and BUILD
# git tag 1.0.1
# git commit -am "One more change after the tags"
VERSION=`git describe --tags`
BUILD=`date +%FT%T%z`
# Setup the -ldflags option for go build here, interpolate the variable values
LDFLAGS_f1=-ldflags "-w -s -X main.Version=${VERSION} -X main.Build=${BUILD} -X main.Entry=f1"
LDFLAGS_f2=-ldflags "-w -s -X main.Version=${VERSION} -X main.Build=${BUILD} -X main.Entry=f2"
# Builds the project
build:
go build ${LDFLAGS_f1} -o ${BINARY}_f1
go build ${LDFLAGS_f2} -o ${BINARY}_f2
# Installs our project: copies binaries
install:
go install ${LDFLAGS_f1}
# Cleans our project: deletes binaries
clean:
if [ -f ${BINARY} ] ; then rm ${BINARY} ; fi
.PHONY: clean install
The make file will create two executables. One is executing function one, the other will take function two as main entry:
package main
import (
"fmt"
)
var (
Version string
Build string
Entry string
funcs = map[string]func() {
"f1":functionOne,"f2":functionTwo,
}
)
func functionOne() {
fmt.Println("This is function one")
}
func functionTwo() {
fmt.Println("This is function two")
}
func main() {
fmt.Println("Version: ", Version)
fmt.Println("Build Time: ", Build)
funcs[Entry]()
}
Then just run:
make
You will get:
mab#h2470988:~/projects/go/gomake/3/gomake$ ls -al
total 2020
drwxrwxr-x 3 mab mab 4096 Sep 7 22:41 .
drwxrwxr-x 3 mab mab 4096 Aug 16 10:00 ..
drwxrwxr-x 8 mab mab 4096 Aug 17 16:40 .git
-rwxrwxr-x 1 mab mab 1023488 Sep 7 22:41 gomake_f1
-rwxrwxr-x 1 mab mab 1023488 Sep 7 22:41 gomake_f2
-rw-rw-r-- 1 mab mab 399 Aug 16 10:21 main.go
-rw-rw-r-- 1 mab mab 810 Sep 7 22:41 Makefile
mab#h2470988:~/projects/go/gomake/3/gomake$ ./gomake_f1
Version: 1.0.1-1-gfb51187
Build Time: 2016-09-07T22:41:38+0200
This is function one
mab#h2470988:~/projects/go/gomake/3/gomake$ ./gomake_f2
Version: 1.0.1-1-gfb51187
Build Time: 2016-09-07T22:41:39+0200
This is function two
I had trouble using the -ldflags parameter when building my mixed command-line app and library project, so I ended up using a Makefile target to generate a Go source file containing my app's version and the build date:
BUILD_DATE := `date +%Y-%m-%d\ %H:%M`
VERSIONFILE := cmd/myapp/version.go
gensrc:
rm -f $(VERSIONFILE)
#echo "package main" > $(VERSIONFILE)
#echo "const (" >> $(VERSIONFILE)
#echo " VERSION = \"1.0\"" >> $(VERSIONFILE)
#echo " BUILD_DATE = \"$(BUILD_DATE)\"" >> $(VERSIONFILE)
#echo ")" >> $(VERSIONFILE)
In my init() method, I do this:
flag.Usage = func() {
fmt.Fprintf(os.Stderr, "%s version %s\n", os.Args[0], VERSION)
fmt.Fprintf(os.Stderr, "built %s\n", BUILD_DATE)
fmt.Fprintln(os.Stderr, "usage:")
flag.PrintDefaults()
}
If you wanted an atomically-increasing build number instead of a build date, however, you would probably need to create a local file that contained the last build number. Your Makefile would read the file contents into a variable, increment it, insert it in the version.go file instead of the date, and write the new build number back to the file.
On Windows OS given the program below
package main
import "fmt"
var (
version string
date string
)
func main() {
fmt.Printf("version=%s, date=%s", version, date)
}
You can build using
go build -ldflags "-X main.version=0.0.1 -X main.date=%date:~10,4%-%date:~4,2%-%date:~7,2%T%time:~0,2%:%time:~3,2%:%time:~6,2%"
Date format assumes your environment echo %date% is Fri 07/22/2016 and echo %time% is 16:21:52.88
Then the output will be: version=0.0.1, date=2016-07-22T16:21:52
to use multi -ldflags:
$ go build -ldflags "-X name1=value1 -X name2=value2" -o path/to/output
Building on the other answers, with recent go versions it's also possible to write a buildid to an ELF section - though that's not so easily readable from within the program.
I write the same value to both, using something like the following:
BuildInfo:= "BUILD #x, branch # rev built yymmdd hh:mm:ss"
// note the nested quotes "''" required to get a string with
// spaces passed correctly to the underlying tool
ldFl := fmt.Sprintf("-X 'main.buildId=%s' -s -w '-buildid=%s'", BuildInfo, BuildInfo)
args := []string{
"build",
"-ldflags", ldFl,
"-trimpath",
"-gcflags", "-dwarf=false",
}
buildpath:="path/to/my/cmd"
args=append(args,buildpath)
buildCmd:=exec.Command("go", args...)
I use this with mage, a build tool written in go. You don't need the extra flags above, but I chose those to strip as much information as possible from release binaries.
(off topic: Mage requires a bit more upfront work than something like Make, but is much easier to extend/maintain than a make-based build system - plus you don't have to switch mental gears between go and some other syntax.)