I have a Go project, where I wanted to generate a Go report card (https://goreportcard.com/)
One of the things that this report card is that it runs
gofmt -s
On all files.
My repo contains around 25 Go files, the only flag that is raised is this one, on six files:
Line 1: warning: file is not gofmted with -s (gofmt)
I've been googling around on gofmt, but I really can't find what this actually means.
Here is an example of a file that raises the error:
package services
import (
"github.com/heyjoakim/devops-21/models"
log "github.com/sirupsen/logrus"
)
var d = GetDBInstance()
// GetUserID returns user ID for username
func GetUserID(username string) (uint, error) {
var user models.User
getUserIDErr := d.db.First(&user, "username = ?", username).Error
if getUserIDErr != nil {
log.WithFields(log.Fields{
"err": getUserIDErr,
"username": username,
}).Error("Error in GetUserID")
}
return user.UserID, getUserIDErr
}
and here is a file that does not raise the error:
package services
import (
"strconv"
"github.com/heyjoakim/devops-21/models"
log "github.com/sirupsen/logrus"
)
func UpdateLatest(latest int) {
var c models.Config
err := d.db.First(&c, "key = ?", "latest").Error
if err != nil {
log.WithField("err", err).Error("Latest does not exist: DB err")
c.ID = 0
c.Key = "latest"
c.Value = strconv.Itoa(latest)
d.db.Create(&c)
} else {
err := d.db.Model(&models.Config{}).Where("key = ?", "latest").Update("Value", latest).Error
if err != nil {
log.WithField("err", err).Error("UpdateLatest: DB err")
}
}
}
I really don't see why one raises some error on line 1, and the other doesn't?
What does this flag mean?
The command gofmt -s myfile.go prints the formatted file to stdout. The -s flag applies simplifications to the formatted file.
Run gofmt -s -d myfile.go to view the differences between the original file and the formatted file.
Run gofmt -s -w myfile.go to update the file to the desired formatting.
Replace myfile.go with . to operate on all files in the directory.
The documentation for the gofmt command is here.
Your files are not formatted. Run gofmt -s -w . in the directory to fix the files.
Related
As I'm new to golang I want to create flag using cobra package :
Currently my flag is working for below condition :
cmd.Flags().StringVarP(&flag, "flag", "f", "", "print names")
cmd -f "value"
cmd
but if I'm using
cmd -flag
then it is showing below error
flag needs an argument: 'f' in -f
In this case how to handle the situation as I want to all 3 conditions to work?
I created a sample program for your scenario by setting no option default values for flags using Cobra.
Please refer this link
OR
You can also refer this link for other scenarios
package main
import (
"fmt"
"github.com/spf13/cobra"
)
func main() {
var rootCmd = &cobra.Command{}
var flag string
rootCmd.Flags().StringVarP(&flag, "flag", "f", "yep", "times to echo the input")
rootCmd.Flags().Lookup("flag").NoOptDefVal = "user" //use this line
//rootCmd.Execute()
err := rootCmd.Execute()
if err != nil {
fmt.Println("Error :", err)
}
fmt.Println("Output :", flag)
}
Output:
D:\cobra>go run main.go rootCmd --flag
Output : user
D:\cobra>go run main.go rootCmd --flag=ms
Output : ms
D:\cobra>go run main.go rootCmd
Output : yep
I try to use REST over gRPC using google api annotations.
Unfortunately, I'm facing a protoc issue telling me annotations.proto is not exists or had errors.
I tried several fixes in vain.
I even tried to reinstall a complete stack in case I did something wrong.
I will detail you as much as possible the full lines and files I have set from my fresh install.
From a fresh go install VM I type these shell lines :
$ mkdir sources/golang
$ echo 'export GOPATH=$HOME/sources/golang' >> $HOME/.zshrc
$ source ~/.zshrc
$ cd sources/golang
$ mkdir src
$ cd src
$ export PATH=$PATH:$GOPATH/bin
$ go get -u google.golang.org/grpc
$ go get -u github.com/golang/protobuf/protoc-gen-go
$ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
$ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger
$ mkdir -p test/proto/test
$ cd test/proto/test
$ vim test.proto
In My test.proto file, I wrote very basic lines :
syntax = "proto3";
package main;
import "google/api/annotations.proto";
service Tester {
rpc Test (Request) returns (Reply) {
option (google.api.http) = { get: "/v1/test" };
}
}
message Request {
string Message = 1;
}
message Reply {
string Message = 1;
}
Then
$ cd $GOPATH/src
$ vim main.go
In my main.go, very basic too :
package main
import (
tc "test/proto/test"
"context"
"fmt"
"log"
"net"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
)
func main() {
err := StartServer("tcp", "127.0.0.1:50051")
if err != nil {
fmt.Printf("Error!! %s", err)
}
}
type Server struct {
tc.UnimplementedTesterServer
}
func (s *Server) Test(ctx context.Context, in *tc.Request) (*tc.Reply, error) {
return &tc.Reply{Message: ""}, nil
}
func StartServer(protocol string, port string) error {
lis, err := net.Listen(protocol, port)
if err != nil {
fmt.Printf("failed to listen: %v", err)
}
s := grpc.NewServer()
tc.RegisterTesterServer(s, &Server{})
error := s.Serve(lis)
if error != nil {
fmt.Printf("failed to serve: %v", error)
return error
}
return nil
}
Finally, I try to compile my proto files :
$ protoc --proto_path=.:$GOPATH/src --go_out=plugins=grpc:. proto/*/*.proto
And I systematically have the following error :
proto/test/test.proto:5:1: Import "github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis/google/api/annotations.proto" was not found or had errors.
When I see files get by go get ..., they are placed under $GOPATH/pkg/mod/
For example the googleapis' annotations.proto file is under : $GOPATH/pkg/mod/github.com/grpc-ecosystem/grpc-gateway#v1.12.1/third_party/googleapis/google/api/annotations.proto
Maybe is it the cause?
Mixing up your protobuf commands and your go commands is making this more complicated than it needs to be. Just focus on the protobuf to handle the error.
You are importing "google/api/annotations.proto"
Your proto_path is --proto_path=.:$GOPATH/src
So that means that when you execute protoc you should have the file located at ./google/api/annotations.proto or $GOPATH/src/google/api/annotations.proto
I faced the same problem and I was trying to automate a solution that would work on any Linux with as little dependencies as possible.
The official docs for grpc_gateway states:
You will need to provide the required third party protobuf files to the protoc compiler. They are included in this repo under the third_party/googleapis folder, and we recommend copying them into your protoc generation file structure. If you've structured your proto files according to something like the Buf style guide, you could copy the files into a top-level ./google folder.
In other words: download it somehow and make it work.
Also, since this quote is also a quote in the original source, it is not very visible which made me skip it the first 2 times I read it looking for a solution, which is bad.
So, as I argued on the comments of #SeanF answer, using go get is not a maintainable option since it saves the project in a folder whose name contains the latest version, which would cause headaches to maintain when the version changes.
So the best option is actually to clone the grpc-gateway project:
git clone https://github.com/grpc-ecosystem/grpc-gateway
protoc -I grpc-gateway/ \
-I grpc-gateway/third_party/googleapis \
--go_out=plugins=grpc:. proto/*/*.proto
I also wrote a gist with my solution which depends only on docker and bash and, thus, should be stable:
https://gist.github.com/VinGarcia/43dfa71b412c16b9365c224d3760af5e
There is a problem with it, where I am using go.mod but discarding it after every execution, which is not good. But it works well enough for now and it might be helpful to some people.
Regards
I use cobra to create CLI command tool.
everything is looking OK except the error handling
what I want that if command was sent by mistake (wrong args or wrong input) return std.err instead of std.out
to simplify the secnario I've created this which demonstrate my use-case
package main
import (
"errors"
"fmt"
"os"
"github.com/spf13/cobra"
)
var (
RootCmd = &cobra.Command{
Use: "myApp",
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("ROOT verbose = %d, args = %v\n", args)
},
}
provideCmd = &cobra.Command{
Use: "provide",
Run: nil,
}
appCmd = &cobra.Command{
Use: "apps",
RunE: func(cmd *cobra.Command, args []string) error {
name := args[0]
if name != "myapp" {
err := errors.New("app name doesnt exist")
return err
}
return nil
},
SilenceUsage: true,
}
)
func init() {
// Add the application command to app command
provideCmd.AddCommand(appCmd)
// add provide command to root command
RootCmd.AddCommand(provideCmd)
}
func main() {
if err := RootCmd.Execute(); err != nil {
fmt.Println(err)
os.Exit(-1)
}
}
Now if I compile the binary and run exec.Command against the binary everything is working as expected. but if I want to test the error scenario like mycli provide apps apps1
I want to see that returned in std.err and not at std.out
When I execute mycli provide apps myapp everything should be OK
but if I run mycli provide apps myapp2 I want to get std.err and not std.out , which is not the case here ...what am I missing here ?
https://play.golang.org/p/B00z4eZ7Sj-
Your sample already prints the error both to stdout and stderr.
By default the cobra package prints any errors it encounters to stderr, unless you specifically change that.
So running
./main provide apps something 2> ./stderr.txt creates a text file with the following content (this is what cobra writes to stderr without your intervention):
Error: app name doesnt exist
And running ./main provide apps something > ./stdout.txt - creates a text file with the following content (you printed that yourself with fmt.Println(err), the second line from the bottom in your code):
app name doesnt exist
Which means default behaviour prints errors both to stdout and stderr.
As Devin has advised you, changing the last line to os.Stderr.WriteString(err) or
fmt.Fprintln(os.Stderr, err) (the one I would use) will make your project to print everything to stderr only, which means printing errors twice:
Error: app name doesnt exist
app name doesnt exist
It might be useful to know that cobra allows you some control of error printing behaviour. For example, you can tell a cobra command which stream to print to:
command.SetOutput(os.Stdout) // Defaults to os.Stderr
you could also prevent printing of errors:
command.SilenceErrors = true
or prevent printing of usage text:
command.SilenceUsage = true
i try to create integrity protection of my application , this is my actual code :
package main
import (
"os"
"io"
"crypto/sha256"
"fmt"
)
var OriginalSign string
func checkSUM() string {
hasher := sha256.New()
f, err := os.Open(os.Args[0])
if err != nil {
os.Exit(0)
}
defer f.Close()
if _, err = io.Copy(hasher, f); err != nil {
os.Exit(0)
}
return fmt.Sprintf("%x", hasher.Sum(nil))
}
func main() {
signature := checkSUM()
fmt.Println(OriginalSign)
fmt.Println(signature)
if signature != OriginalSign {
fmt.Println("binary is compromised")
}
}
i compiled with this command :
C:\Users\admin\go\src\localhost\lic>go build -ldflags="-s -w -X main.OriginalSig
n=8636cdeef255e52c6fd3f391fd7d75fbaf7c6e830e0e7ac66a645093c7efcbc7" -o checksum.
exe checksum.go
C:\Users\admin\go\src\localhost\lic>checksum.exe
8636cdeef255e52c6fd3f391fd7d75fbaf7c6e830e0e7ac66a645093c7efcbc7
d29440d3467f6176a6af0dcb61ea696cb318db3a6f1680b5b8f7890e165d8d7e
binary is compromised
how i can do this corectly in go ? i need to know signature of final binary file and check if is compromited.
I can't see how to hook into tool buildid in a program but it can (kind of) detect changes to a binary
buildid does seem to store a "contentid" of the binary which is the essence of the original question
Here's a bash script that shows this (sorry I don't do MS Windows)
#
# delete any old binaries
rm -f t
# do an initial build
go build t.go
# show it works
./t
# get the original buildid
ORIG=$(go tool buildid t)
# now tamper with it!
perl -p -i -e 's/testing/porkpie/' t
# run again, show the tamper
./t
# now regenerate the buildid
go tool buildid -w t
# get the buildid after the regeneration
LATER=$(go tool buildid t)
# show the original and the post-tampering buildid - they are different
echo "$ORIG"
echo "$LATER"
Here's the do nothing t.go
package main
import (
"fmt"
)
func main() {
fmt.Println("testing 123")
}
Here's the output
testing 123
porkpie 123
koB1H61TwQSHTQGiI4PP/-o93sSzqt1ltMhBJn4pR/2wvL4J9vF4vGUGjdbsyd/y-0uRBmxfJdrbAfsE1lr
koB1H61TwQSHTQGiI4PP/-o93sSzqt1ltMhBJn4pR/2wvL4J9vF4vGUGjdbsyd/UeLetY1pBF54B_4Y8-Nj
So the go tool buildid can store a hash in with the binary and (kind of) detect tampering. But I couldn't work out how to get the contentid from a normal call inside a normal program
I have a project that using package flag to read the argv(parameter), it'll print the default settings when no parameter is given:
func initFlag() {
path := flag.String("F", "store_server.conf", "config file path")
v := flag.Bool("V", false, "print version")
flag.Parse()
if flag.NFlag() == 0 {
flag.PrintDefaults()
os.Exit(0)
}
fmt.Println(*path, *v)
}
func main() {
initFlag() // initialize flag and load configure file
select{}
}
Here comes the execution results:
vinllen# ~$ go run main.go
-F string
config file path (default "store_server.conf")
-V print version
But when my code includes other package like glog, the PrintDefaults function will show more settings including glog flag:
-F string
config file path (default "store_server.conf")
-V print version
-alsologtostderr
log to standard error as well as files
-log_backtrace_at value
when logging hits line file:N, emit a stack trace
-log_dir string
If non-empty, write log files in this directory
-logtostderr
log to standard error instead of files
-stderrthreshold value
logs at or above this threshold go to stderr
-v value
log level for V logs
-vmodule value
comma-separated list of pattern=N settings for file-filtered logging
The only two settings I needed are -F and -V, how to delete the others?
You need to use a new FlagSet instead of the default one:
package main
import (
"flag"
"github.com/golang/glog"
"os"
)
func initFlag() {
flags := flag.NewFlagSet("myFlagSet", 0)
path := flags.String("F", "store_server.conf", "config file path")
v := flags.Bool("V", false, "print version")
flags.Parse(os.Args[1:])
if flags.NFlag() == 0 {
if len(os.Args) == 0 {
flags.PrintDefaults()
os.Exit(0)
}
}
glog.Info(*path, *v)
}
func main() {
initFlag() // initialize flag and load configure file
select {}
}
The second argument to NewFlagSet is the error handling. See here for more details.
When you call flag.someMethod() directly, it uses a default, shared, flag set. If you create a new one, it will be empty.