I am relatively new to the packages Cobra and Logrus in Golang and there is one little thing I would to ask for help when it comes to integrating them. I am working on a new Golang Cobra CLI which contains several commands/sub-commands. I'd like to use Logrus for logging inside my subcommands but I cannot find a good way to add new fields to the logger and pass them all to my subcommands.
This is what I have so far (I simplified my code for better explanation of my purpose):
├──pkg/
├──logger/
| ├──logger.go
├──cmd/
├── root.go
├──scmd/
| ├──scmd.go
// root.go
package cmd
func ExecuteRoot() {
rootCmd := getRootCmd()
err := rootCmd.Execute()
if err != nil {
os.Exit(1)
}
}
func getRootCmd() *cobra.Command {
rootCmd := &cobra.Command{
Use: "mycli",
PersistentPreRun: enableLogs,
}
rootCmd.AddCommand(scmd.Getscmd())
return rootCmd
}
func enableLogs(cmd *cobra.Command, args []string) {
logger.ConfigureLogger(cmd.Flag("log-level").Value.String())
}
// logger.go
package logger
import (
"github.com/sirupsen/logrus"
)
var MyCli *logrus.Entry
func ConfigureLogger(d string) {
lv, err := logrus.ParseLevel(d)
if err != nil {
logrus.Fatal(err)
}
logrus.SetLevel(lv)
logrus.SetFormatter(&logrus.TextFormatter{
DisableQuote: true,
ForceColors: true,
})
MyCli = logrus.WithFields(logrus.Fields{"foo1": "bar1", "foo2": "bar2"})
}
// scmd.go
package scmd
import (
"pkg/logger"
"github.com/spf13/cobra"
)
func Getscmd() *cobra.Command {
serviceUpdateCmd := &cobra.Command{
Use: "scmd",
Run: runServiceUpdateCmd,
}
return serviceUpdateCmd
}
func runServiceUpdateCmd(cmd *cobra.Command, args []string) {
logger.MyCli.Info("start scmd")
// here what the command does ...
}
If I run it, I get what I am expecting: My subcommands (in this case scmd) logs at the level I set my flag log-level and the fields defined in my package logger are passed ("foo1": "bar1", "foo2": "bar2"). However, I feel it is not the right way and could become problematic when creating unit test as I am using the global variable var MyCli from my logger package. Moreover, I should import it and use it as logger.MyCli in every Info, error, warn, etc line I want to log.
My question is whether there is a better approach to pass fields in Logrus on all subcommands I create or the way I explained above is my only option.
Any help or idea to improve my code is really welcome.
Thanks!
Related
I have this method used in a lambda:
import (
"os"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
)
func InitLogger() *zap.Logger {
config := zap.NewProductionEncoderConfig()
config.EncodeTime = zapcore.RFC3339TimeEncoder
consoleEncoder := zapcore.NewJSONEncoder(config)
core := zapcore.NewTee(zapcore.NewCore(consoleEncoder, zapcore.AddSync(os.Stdout), zapcore.InfoLevel))
return zap.New(core).With()
}
And in my lambda Handler i have:
var (
log *zap.Logger
)
func init() {
log = u.InitLogger()
}
func handler(r events.APIGatewayProxyRequest) (*events.APIGatewayProxyResponse, error) {
out, err := exec.Command("uuidgen").Output()
uuid := strings.ReplaceAll(string(out), "\n", "")
if err != nil {
log.Error(err.Error())
}
log.Info("PRINT_1", zap.Any("uuid", uuid), zap.Any("Request", r.Body))
}
I have a question, is possible add the UUID to all logs without adding one by one?, because in each log that I need print something, I need add zap.Any("uuid", uuid)
The problem is that I need pass as parameter to all methods the UUID to print it in the log info, or error.
You will have to slightly re-arrange your code since you're only creating the UUID in the handler, which implies it's request-specific whilst the logger is global...
But the gist, specific to the library, is that you've got to create a child logger (which you are, in fact, already doing: you just need to pass the fields there). Any subsequent log writes to the child logger will include those fields.
For example:
func main() {
logger := InitLogger(zap.String("foo", "bar"))
logger.Info("First message with our `foo` key")
logger.Info("Second message with our `foo` key")
}
func InitLogger(fields ...zap.Field) *zap.Logger {
config := zap.NewProductionEncoderConfig()
config.EncodeTime = zapcore.RFC3339TimeEncoder
consoleEncoder := zapcore.NewJSONEncoder(config)
core := zapcore.NewTee(zapcore.NewCore(consoleEncoder, zapcore.AddSync(os.Stdout), zapcore.InfoLevel))
return zap.New(core).With(fields...)
}
Output:
{"level":"info","ts":"2022-11-24T18:30:45+01:00","msg":"First message with our `foo` key","foo":"bar"}
{"level":"info","ts":"2022-11-24T18:30:45+01:00","msg":"Second message with our `foo` key","foo":"bar"}
I've the following project structure:
myGithubProject/
|---- cmd
|----- command
|----- hello.go
|---- internal
|----- template
|----- template.go
|----- log
|----- logger.go
main.go
The log and the template are on the same level (under internal package)
In logger.go Im using logrus as logger with some config sand I want to use the logger.go object inside the template package.
How should I do that in a clean way ?
Currently I use it with import logger inside my template.go file,
And under internal package I’ve 6 more packages which needs this logger. and each of them is depend on it.
(On the log package), is there a nice in go to handle it ?
update:
in case I've more things that I need to pass (like logger) what will be the approach/ pattern here ? maybe using dependency injection ? interface ? other clean approach...
I need some best practice full example how can I use logger inside the hello.go file and also in template.go.
this is my project
cliProject/main.go
package main
import (
"cliProject/cmd"
"cliProject/internal/logs"
)
func main() {
cmd.Execute()
logs.Logger.Error("starting")
}
**cliProject/cmd/root.go**
package cmd
import (
"fmt"
"github.com/spf13/cobra"
)
var rootCmd = &cobra.Command{
Use: "cliProject",
Short: "A brief description of your application",
}
func Execute() {
if err := rootCmd.Execute(); err != nil {
fmt.Println(err)
}
}
**cliProject/cmd/serve.go**
package cmd
import (
"cliProject/internal/logs"
"fmt"
"github.com/spf13/cobra"
)
// serveCmd represents the serve command
var serveCmd = &cobra.Command{
Use: "serve",
Short: "A brief description of your command",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("serve called")
startServe()
stoppingServe()
},
}
func init() {
rootCmd.AddCommand(serveCmd)
serveCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}
func startServe() {
logs.Logger.Error("starting from function serve")
}
func stoppingServe() {
logs.Logger.Error("stoping from function serve")
}
**cliProject/cmd/start.go**
package cmd
import (
"cliProject/internal/logs"
"github.com/spf13/cobra"
)
// startCmd represents the start command
var startCmd = &cobra.Command{
Use: "start",
Short: "Start command",
Run: func(cmd *cobra.Command, args []string) {
// Print the logs via logger from internal
logs.Logger.Println("start called inline")
// example of many functions which should use the logs...
start()
stopping()
},
}
func init() {
logs.NewLogger()
rootCmd.AddCommand(startCmd)
startCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}
func start() {
logs.Logger.Error("starting from function start")
}
func stopping() {
logs.Logger.Error("stoping from function start")
}
**cliProject/internal/logs/logger.go**
package logs
import (
"github.com/sirupsen/logrus"
"github.com/x-cray/logrus-prefixed-formatter"
"os"
)
var Logger *logrus.Logger
func NewLogger() *logrus.Logger {
var level logrus.Level
level = LogLevel("info")
logger := &logrus.Logger{
Out: os.Stdout,
Level: level,
Formatter: &prefixed.TextFormatter{
DisableColors: true,
TimestampFormat: "2009-06-03 11:04:075",
},
}
Logger = logger
return Logger
}
func LogLevel(lvl string) logrus.Level {
switch lvl {
case "info":
return logrus.InfoLevel
case "error":
return logrus.ErrorLevel
default:
panic("Not supported")
}
}
this is how it looks
And under internal package I’ve 6 more packages which needs this logger. and each of them is depend on it. (On the log package), is there a nice in go to handle it ?
A good general principle would be to respect the application's choice (whether to log or not) instead of setting policy.
Let Go pkgs in your internal directory be support packages that
will only return error in case of problems
won't log (console or otherwise)
won't panic
Let your application (packages in your cmd directory) decide what the appropriate behavior in case of an error (log / graceful shutdown / recover to 100% integrity)
This would streamline development by having logging at a particular layer only. Note: remember to give the application give enough context to determine action
internal/process/process.go
package process
import (
"errors"
)
var (
ErrNotFound = errors.New("Not Found")
ErrConnFail = errors.New("Connection Failed")
)
// function Process is a dummy function that returns error for certain arguments received
func Process(i int) error {
switch i {
case 6:
return ErrNotFound
case 7:
return ErrConnFail
default:
return nil
}
}
cmd/servi/main.go
package main
import (
"log"
p "../../internal/process"
)
func main() {
// sample: generic logging on any failure
err := p.Process(6)
if err != nil {
log.Println("FAIL", err)
}
// sample: this application decides how to handle error based on context
err = p.Process(7)
if err != nil {
switch err {
case p.ErrNotFound:
log.Println("DOESN'T EXIST. TRY ANOTHER")
case p.ErrConnFail:
log.Println("UNABLE TO CONNECT; WILL RETRY LATER")
}
}
}
in case I've more things that I need to pass (like logger) what will be the approach/ pattern here
Dependency injection is always a good first choice. Consider others only when the simplest implementation is insufficient.
The code below 'wires' the template and logger packages together using dependency injection and first-class function passing.
internal/logs/logger.go
package logger
import (
"github.com/sirupsen/logrus"
"github.com/x-cray/logrus-prefixed-formatter"
"os"
)
var Logger *logrus.Logger
func NewLogger() *logrus.Logger {
var level logrus.Level
level = LogLevel("info")
logger := &logrus.Logger{
Out: os.Stdout,
Level: level,
Formatter: &prefixed.TextFormatter{
DisableColors: true,
TimestampFormat: "2009-06-03 11:04:075",
},
}
Logger = logger
return Logger
}
func LogLevel(lvl string) logrus.Level {
switch lvl {
case "info":
return logrus.InfoLevel
case "error":
return logrus.ErrorLevel
default:
panic("Not supported")
}
}
internal/template/template.go
package template
import (
"fmt"
"github.com/sirupsen/logrus"
)
type Template struct {
Name string
logger *logrus.Logger
}
// factory function New accepts a logging function and some data
func New(logger *logrus.Logger, data string) *Template {
return &Template{data, logger}
}
// dummy function DoSomething should do something and log using the given logger
func (t *Template) DoSomething() {
t.logger.Info(fmt.Sprintf("%s, %s", t.Name, "did something"))
}
cmd/servi2/main.go
package main
import (
"../../internal/logs"
"../../internal/template"
)
func main() {
// wire our template and logger together
loggingFunc := logger.NewLogger()
t := template.New(loggingFunc, "Penguin Template")
// use the stuff we've set up
t.DoSomething()
}
Hope this helps. Cheers,
There are a few possibilities, each with their own tradeoffs.
Pass in dependencies explicitly
Pass in a context with all dependencies
Use a struct for context to methods
Use a package global and import
All of them have their place in different circumstances, and all have different tradeoffs:
This is very clear, but can get very messy and clutters your functions with lots of dependencies. It makes tests easy to mock if that is your thing.
This is my least favourite option, as it is seductive at first but quickly grows to a god object which mixes lots of unrelated concerns. Avoid.
This can be very useful in many cases, for example many people approach db access this way. Also easy to mock if required. This allows you to set/share dependencies without changing code at point of use - basically invert control in a neater way than passing in explicit parameters.
This has the virtue of clarity and orthogonality. It will require you to add separate setup for say tests vs production, initialising the package to the proper state prior to using it. Some dislike it for that reason.
I prefer the package global approach for something like logging, as long as a very simple signature is used. I don’t tend to test log output, or change logger often. Consider what you really need from logs, and whether it might be best just to use the built in log package, and perhaps try one of these approaches to see which suit you.
Examples in pseudo code for brevity:
// 1. Pass in dependencies explicitly
func MyFunc(log LoggerInterface, param, param)
// 2. Pass in a context with all dependencies
func MyFunc(c *ContextInterface, param, param)
// 3. Use a struct for context to methods
func (controller *MyController) MyFunc(param, param) {
controller.Logger.Printf("myfunc: doing foo:%s to bar:%s",foo, bar)
}
// 4. Use a package global and import
package log
var DefaultLogger PrintfLogger
func Printf(format, args) {DefaultLogger.Printf(format, args)}
// usage:
import "xxx/log"
log.Printf("myfunc: doing foo:%s to bar:%s",foo, bar)
I prefer this option 4 for logging and config, and even db access as it is explicit, flexible, and allows easily switching out another logger - just change the imports, no interfaces to change. The calculation on which is best depends on circumstances though - if you were writing a library for widespread use you might prefer to allow setting a logger at the struct level.
I'd usually require setup with an explicit setup on app startup, and always avoid using init functions as they are confusing and difficult to test.
I've always explicitly passed a *logrus.Logger into functions (or occasionally objects) that needed it. That avoids weird dependency cycles, makes it explicit that logging is part of the things this function does, and makes it easier to reuse the function or module elsewhere. The initial log object is created and configured in my main function (likely after some command-line argument handling).
The documentation in Cobra and Viper are confusing me. I did cobra init fooproject and then inside the project dir I did cobra add bar. I have a PersistentFlag that is named foo and here is the init function from the root command.
func Execute() {
if err := RootCmd.Execute(); err != nil {
fmt.Println(err)
os.Exit(-1)
}
fmt.Println(cfgFile)
fmt.Println("fooString is: ", fooString)
}
func init() {
cobra.OnInitialize(initConfig)
// Here you will define your flags and configuration settings.
// Cobra supports Persistent Flags, which, if defined here,
// will be global for your application.
RootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $HOME/.fooproject.yaml)")
RootCmd.PersistentFlags().StringVar(&fooString, "foo", "", "loaded from config")
viper.BindPFlag("foo", RootCmd.PersistentFlags().Lookup("foo"))
// Cobra also supports local flags, which will only run
// when this action is called directly.
RootCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}
My configuration file looks like this...
---
foo: aFooString
And when I call go run main.go I see this...
A longer description that spans multiple lines and likely contains
examples and usage of using your application. For example:
Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.
Usage:
fooproject [command]
Available Commands:
bar A brief description of your command
help Help about any command
Flags:
--config string config file (default is $HOME/.fooproject.yaml)
--foo string loaded from config
-h, --help help for fooproject
-t, --toggle Help message for toggle
Use "fooproject [command] --help" for more information about a command.
fooString is:
When I call go run main.go bar I see this...
Using config file: my/gopath/github.com/user/fooproject/.fooproject.yaml
bar called
fooString is:
So it is using the configuration file, but neither one of them seems to be reading it. Maybe I am misunderstanding the way that Cobra and Viper work. Any ideas?
To combine spf13/cobra and spf13/viper, first define the flag with Cobra:
RootCmd.PersistentFlags().StringP("foo", "", "loaded from config")
Bind it with Viper:
viper.BindPFlag("foo", RootCmd.PersistentFlags().Lookup("foo"))
And get the variable via the Viper's method:
fmt.Println("fooString is: ", viper.GetString("foo"))
Since Viper values are somewhat inferior to pflags (e.g. no support for custom data types), I was not satisfied with answer "use Viper to retrieve values", and ended up writing small helper type to put values back in pflags.
type viperPFlagBinding struct {
configName string
flagValue pflag.Value
}
type viperPFlagHelper struct {
bindings []viperPFlagBinding
}
func (vch *viperPFlagHelper) BindPFlag(configName string, flag *pflag.Flag) (err error) {
err = viper.BindPFlag(configName, flag)
if err == nil {
vch.bindings = append(vch.bindings, viperPFlagBinding{configName, flag.Value})
}
return
}
func (vch *viperPFlagHelper) setPFlagsFromViper() {
for _, v := range vch.bindings {
v.flagValue.Set(viper.GetString(v.configName))
}
}
func main() {
var rootCmd = &cobra.Command{}
var viperPFlagHelper viperPFlagHelper
rootCmd.PersistentFlags().StringVar(&config.Password, "password", "", "API server password (remote HTTPS mode only)")
viperPFlagHelper.BindPFlag("password", rootCmd.Flag("password"))
rootCmd.PersistentPreRunE = func(cmd *cobra.Command, args []string) error {
err := viper.ReadInConfig()
if err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); !ok {
return err
}
}
viperPFlagHelper.setPFlagsFromViper()
return nil
}
}
For those facing the same issue, the problem is on this call:
cobra.OnInitialize(initConfig)
The function initConfig is not executed directly but append to an array of initializers https://github.com/spf13/cobra/blob/master/cobra.go#L80
So, the values stored in your config file will be loaded when the Run field is executed:
rootCmd = &cobra.Command{
Use: "example",
Short: "example cmd",
Run: func(cmd *cobra.Command, args []string) { // OnInitialize is called first
fmt.Println(viper.AllKeys())
},
}
#WGH 's answer is correct, you can do something with the persistent flag in rootCmd.PersistenPreRun, which will be available in all other sub commands.
var rootCmd = &cobra.Command{
PersistentPreRun: func(_ *cobra.Command, _ []string) {
if persistentflag {
// do something with persistentflag
}
},
Hi I am new to Go and I am writing a simple app which gets some configuration from the env variables. I do this in the init function as shown below.
type envVars struct {
Host string `env:"APP_HOST"`
Username string `env:"APP_USERNAME"`
Password string `env:"APP_PASSWORD"`
}
var envConfig envVars
func init() {
if err := env.Parse(&envConfig); err != nil {
log.Fatal(err)
}
}
I wrote test to verify of the env variables are being read correctly. But the problem is that my program's init func gets called even before my test's init func. Is there any way I can do some sort of setup before my program's init func gets called.
func init() {
os.Setenv("APP_HOST", "http://localhost:9999")
os.Setenv("APP_USERNAME", "john")
os.Setenv("APP_PASSWORD", "doe")
}
func TestEnvConfig(t *testing.T) {
assert.NotNil(t, envConfig)
assert.Equal(t, "http://localhost:9999", envConfig.Host)
}
You can use the TestMain func to control what happens before and after your tests.
For example:
func TestMain(m *testing.M) {
// Write code here to run before tests
// Run tests
exitVal := m.Run()
// Write code here to run after tests
// Exit with exit value from tests
os.Exit(exitVal)
}
func TestYourFunc(t *testing.T) {
// Test code
}
You can add a Test_parse_params(t *testing.T) function before your real tests. Look like this:
type envVars struct {
Host string `env:"APP_HOST"`
Username string `env:"APP_USERNAME"`
Password string `env:"APP_PASSWORD"`
}
var envConfig envVars
//parse command params
func Test_parse_params(t *testing.T) {
if err := env.Parse(&envConfig); err != nil {
log.Fatal(err)
}
}
func Test_real_test(t *testing.T) {
....
}
No, you shouldn't expect init() run in some order, (in fact it based on file loaded order, but still, you should not count on it).
The simple way is, if you want to test it, use a shell script to run you test, or something like Makefile.
Shell example:
set +e
export APP_HOST=http://localhost:9999
export APP_USERNAME=john
export APP_PASSWORD=doe
go test .
unset APP_HOST
unset APP_USERNAME
unset APP_PASSWORD
or a single line command:
APP_HOST=http://localhost:9999 APP_USERNAME=john APP_PASSWORD=doe go test .
Edit:
Other solution: move out the read env from init func.
func init(){
envInit()
}
func envInit(){
if err := env.Parse(&envConfig); err != nil {
log.Fatal(err)
}
}
Then you can call again envInit in your test to make sure it works.
Less than ideal, but this works for me.
Inside of the package that you're testing:
func init() {
if len(os.Args) > 1 && os.Args[1][:5] == "-test" {
log.Println("testing")//special test setup goes goes here
return // ...or just skip the setup entirely
}
//...
}
I am developing an web app with Go. So far so good, but now I am integrating Wercker as a CI tool and started caring about testing. But my app relies heavily on Cobra/Viper configuration/flags/environment_variables scheme, and I do not know how to properly init Viper values before running my test suite. Any help would be much appreciated.
When I use Cobra/Viper or any other combination of CLI helpers, my way of doing this is to have the CLI tool run a function whose sole purpose will be to get arguments and pass them to another method who will do the actual work.
Here is a short (and dumb) example using Cobra :
package main
import (
"fmt"
"os"
"github.com/spf13/cobra"
)
func main() {
var Cmd = &cobra.Command{
Use: "boom",
Short: "Explode all the things!",
Run: Boom,
}
if err := Cmd.Execute(); err != nil {
fmt.Println(err)
os.Exit(-1)
}
}
func Boom(cmd *cobra.Command, args []string) {
boom(args...)
}
func boom(args ...string) {
for _, arg := range args {
println("boom " + arg)
}
}
Here, the Boom function is hard to test, but the boom one is easy.
You can see another (non-dumb) example of this here (and the correspond test here).
i have found an easy way to test commands with multiple level sub commands, it is not professional but it worked well.
assume we have a command like this
RootCmd = &cobra.Command{
Use: "cliName",
Short: "Desc",
}
SubCmd = &cobra.Command{
Use: "subName",
Short: "Desc",
}
subOfSubCmd = &cobra.Command{
Use: "subOfSub",
Short: "Desc",
Run: Exec
}
//commands relationship
RootCmd.AddCommand(SubCmd)
SubCmd.AddCommand(subOfSubCmd)
When testing the subOfSubCmd we can do this way:
func TestCmd(t *testing.T) {
convey.Convey("test cmd", t, func() {
args := []string{"subName", "subOfSub"}
RootCmd.SetArgs(args)
RootCmd.Execute()
})
}