I use logrus in all my go apps and recently I started using a context logger. Now I want to "build up" a context during the execution path of my app. See example below, which illustrates what I want.
package main
import (
"github.com/Sirupsen/logrus"
)
func main() {
logrus.Info("normal logger")
cl := logrus.WithFields(
logrus.Fields{
"extra_field_one": "extra_value_one",
})
// some code here
// here I want to add an additional field to to contextlogger cl.
// How do I do that?
}
EDIT
As ymonad mentioned, it's possible by overwriting the contextLogger. Also found out that you can add one additional field:
cl = cl.WithField("key", "value")
You can just call cl.WithFields()
package main
import "github.com/Sirupsen/logrus"
func main() {
cl := logrus.WithFields(
logrus.Fields{
"extra_field_one": "extra_value_one",
})
cl = cl.WithFields(
logrus.Fields{
"extra_field_two": "extra_value_two",
})
cl.Info("hello world")
}
Output is:
INFO[0000] hello world extra_field_one="extra_value_one" extra_field_two="extra_value_two"
Related
As the title asks, is there any way to import a go file from inside of a function?
I was thinking of using a loop for my discordgo(https://pkg.go.dev/github.com/bwmarrin/discordgo) program.
ex :
package main
import (
...
)
func main() {
client, _ := disocrdgo.New("Bot " + mytoken)
...
events, _ := ioutil.ReadDir("./events")
for event, _ := range events {
x, _ := import "MyApp/src/events/" + event // <---
client.AddHandler(x.executor) // using type struct where executor is the function i want to use from the imported file
}
...
}
I feel obligated to precise it so :
Thanks for your SERIOUS answers.
Imports are a compiler concept, so you cannot import packages at runtime (the source code doesn't even exist on the machine running your program, usually).
You can use the registry pattern to get close to what you're looking for.
In the events package, create a function that stores handlers. Call that function in the init function for each event handler package.
In the events package, create another function that adds the stored handlers to a client.
In the main package, import all event handler packages you need and call that second function.
This is more or less how the sql and image packages in the standard library work. See sql.Register and image.RegisterFormat.
// events/registry.go
package events
var handlers = map[string]interface{}{}
func Register(name string, h interface{}) {
handlers[name] = h
}
func ConfigureClient(client *discordgo.Session) {
for _, h := range handlers {
client.AddHandler(h)
}
}
// events/foo/foo.go
package foo
import "MyApp/src/events"
func init() {
events.Register("foo", executor{})
}
type executor struct{}
// events/bar/bar.go
package bar
import "MyApp/src/events"
func init() {
events.Register("bar", executor{})
}
type executor struct{}
// main.go
package main
import (
_ "MyApp/src/events/foo"
_ "MyApp/src/events/bar"
// ...
)
func main() {
client, _ := discordgo.New("Bot " + mytoken)
events.ConfigureClient(client)
}
Today I try to program with context,code as follow:
package main
func main(){
ctx := context.Background()
ctx = context.WithValue(ctx,"appid","test111")
b.dosomething()
}
package b
func dosomething(ctx context.Context){
fmt.Println(ctx.Value("appid").(string))
}
Then my program has crashed.I think it's due to that these ctx is in different package
I suggest you to use context only in a lifetime of a single task and pass the same context through functions.
Also you should understand where to use context and where just to pass arguments to functions.
Another suggestion is to use custom types for setting and getting values from context.
According to all above, you program should look like this:
package main
import (
"context"
"fmt"
)
type KeyMsg string
func main() {
ctx := context.WithValue(context.Background(), KeyMsg("msg"), "hello")
DoSomething(ctx)
}
// DoSomething accepts context value, retrieves message by KeyMsg and prints it.
func DoSomething(ctx context.Context) {
msg, ok := ctx.Value(KeyMsg("msg")).(string)
if !ok {
return
}
fmt.Println("got msg:", msg)
}
You can move function DoSomething into another package and just call it as packagename.DoSomething it will change nothing.
I've the following project structure:
myGithubProject/
|---- cmd
|----- command
|----- hello.go
|---- internal
|----- template
|----- template.go
|----- log
|----- logger.go
main.go
The log and the template are on the same level (under internal package)
In logger.go Im using logrus as logger with some config sand I want to use the logger.go object inside the template package.
How should I do that in a clean way ?
Currently I use it with import logger inside my template.go file,
And under internal package I’ve 6 more packages which needs this logger. and each of them is depend on it.
(On the log package), is there a nice in go to handle it ?
update:
in case I've more things that I need to pass (like logger) what will be the approach/ pattern here ? maybe using dependency injection ? interface ? other clean approach...
I need some best practice full example how can I use logger inside the hello.go file and also in template.go.
this is my project
cliProject/main.go
package main
import (
"cliProject/cmd"
"cliProject/internal/logs"
)
func main() {
cmd.Execute()
logs.Logger.Error("starting")
}
**cliProject/cmd/root.go**
package cmd
import (
"fmt"
"github.com/spf13/cobra"
)
var rootCmd = &cobra.Command{
Use: "cliProject",
Short: "A brief description of your application",
}
func Execute() {
if err := rootCmd.Execute(); err != nil {
fmt.Println(err)
}
}
**cliProject/cmd/serve.go**
package cmd
import (
"cliProject/internal/logs"
"fmt"
"github.com/spf13/cobra"
)
// serveCmd represents the serve command
var serveCmd = &cobra.Command{
Use: "serve",
Short: "A brief description of your command",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("serve called")
startServe()
stoppingServe()
},
}
func init() {
rootCmd.AddCommand(serveCmd)
serveCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}
func startServe() {
logs.Logger.Error("starting from function serve")
}
func stoppingServe() {
logs.Logger.Error("stoping from function serve")
}
**cliProject/cmd/start.go**
package cmd
import (
"cliProject/internal/logs"
"github.com/spf13/cobra"
)
// startCmd represents the start command
var startCmd = &cobra.Command{
Use: "start",
Short: "Start command",
Run: func(cmd *cobra.Command, args []string) {
// Print the logs via logger from internal
logs.Logger.Println("start called inline")
// example of many functions which should use the logs...
start()
stopping()
},
}
func init() {
logs.NewLogger()
rootCmd.AddCommand(startCmd)
startCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}
func start() {
logs.Logger.Error("starting from function start")
}
func stopping() {
logs.Logger.Error("stoping from function start")
}
**cliProject/internal/logs/logger.go**
package logs
import (
"github.com/sirupsen/logrus"
"github.com/x-cray/logrus-prefixed-formatter"
"os"
)
var Logger *logrus.Logger
func NewLogger() *logrus.Logger {
var level logrus.Level
level = LogLevel("info")
logger := &logrus.Logger{
Out: os.Stdout,
Level: level,
Formatter: &prefixed.TextFormatter{
DisableColors: true,
TimestampFormat: "2009-06-03 11:04:075",
},
}
Logger = logger
return Logger
}
func LogLevel(lvl string) logrus.Level {
switch lvl {
case "info":
return logrus.InfoLevel
case "error":
return logrus.ErrorLevel
default:
panic("Not supported")
}
}
this is how it looks
And under internal package I’ve 6 more packages which needs this logger. and each of them is depend on it. (On the log package), is there a nice in go to handle it ?
A good general principle would be to respect the application's choice (whether to log or not) instead of setting policy.
Let Go pkgs in your internal directory be support packages that
will only return error in case of problems
won't log (console or otherwise)
won't panic
Let your application (packages in your cmd directory) decide what the appropriate behavior in case of an error (log / graceful shutdown / recover to 100% integrity)
This would streamline development by having logging at a particular layer only. Note: remember to give the application give enough context to determine action
internal/process/process.go
package process
import (
"errors"
)
var (
ErrNotFound = errors.New("Not Found")
ErrConnFail = errors.New("Connection Failed")
)
// function Process is a dummy function that returns error for certain arguments received
func Process(i int) error {
switch i {
case 6:
return ErrNotFound
case 7:
return ErrConnFail
default:
return nil
}
}
cmd/servi/main.go
package main
import (
"log"
p "../../internal/process"
)
func main() {
// sample: generic logging on any failure
err := p.Process(6)
if err != nil {
log.Println("FAIL", err)
}
// sample: this application decides how to handle error based on context
err = p.Process(7)
if err != nil {
switch err {
case p.ErrNotFound:
log.Println("DOESN'T EXIST. TRY ANOTHER")
case p.ErrConnFail:
log.Println("UNABLE TO CONNECT; WILL RETRY LATER")
}
}
}
in case I've more things that I need to pass (like logger) what will be the approach/ pattern here
Dependency injection is always a good first choice. Consider others only when the simplest implementation is insufficient.
The code below 'wires' the template and logger packages together using dependency injection and first-class function passing.
internal/logs/logger.go
package logger
import (
"github.com/sirupsen/logrus"
"github.com/x-cray/logrus-prefixed-formatter"
"os"
)
var Logger *logrus.Logger
func NewLogger() *logrus.Logger {
var level logrus.Level
level = LogLevel("info")
logger := &logrus.Logger{
Out: os.Stdout,
Level: level,
Formatter: &prefixed.TextFormatter{
DisableColors: true,
TimestampFormat: "2009-06-03 11:04:075",
},
}
Logger = logger
return Logger
}
func LogLevel(lvl string) logrus.Level {
switch lvl {
case "info":
return logrus.InfoLevel
case "error":
return logrus.ErrorLevel
default:
panic("Not supported")
}
}
internal/template/template.go
package template
import (
"fmt"
"github.com/sirupsen/logrus"
)
type Template struct {
Name string
logger *logrus.Logger
}
// factory function New accepts a logging function and some data
func New(logger *logrus.Logger, data string) *Template {
return &Template{data, logger}
}
// dummy function DoSomething should do something and log using the given logger
func (t *Template) DoSomething() {
t.logger.Info(fmt.Sprintf("%s, %s", t.Name, "did something"))
}
cmd/servi2/main.go
package main
import (
"../../internal/logs"
"../../internal/template"
)
func main() {
// wire our template and logger together
loggingFunc := logger.NewLogger()
t := template.New(loggingFunc, "Penguin Template")
// use the stuff we've set up
t.DoSomething()
}
Hope this helps. Cheers,
There are a few possibilities, each with their own tradeoffs.
Pass in dependencies explicitly
Pass in a context with all dependencies
Use a struct for context to methods
Use a package global and import
All of them have their place in different circumstances, and all have different tradeoffs:
This is very clear, but can get very messy and clutters your functions with lots of dependencies. It makes tests easy to mock if that is your thing.
This is my least favourite option, as it is seductive at first but quickly grows to a god object which mixes lots of unrelated concerns. Avoid.
This can be very useful in many cases, for example many people approach db access this way. Also easy to mock if required. This allows you to set/share dependencies without changing code at point of use - basically invert control in a neater way than passing in explicit parameters.
This has the virtue of clarity and orthogonality. It will require you to add separate setup for say tests vs production, initialising the package to the proper state prior to using it. Some dislike it for that reason.
I prefer the package global approach for something like logging, as long as a very simple signature is used. I don’t tend to test log output, or change logger often. Consider what you really need from logs, and whether it might be best just to use the built in log package, and perhaps try one of these approaches to see which suit you.
Examples in pseudo code for brevity:
// 1. Pass in dependencies explicitly
func MyFunc(log LoggerInterface, param, param)
// 2. Pass in a context with all dependencies
func MyFunc(c *ContextInterface, param, param)
// 3. Use a struct for context to methods
func (controller *MyController) MyFunc(param, param) {
controller.Logger.Printf("myfunc: doing foo:%s to bar:%s",foo, bar)
}
// 4. Use a package global and import
package log
var DefaultLogger PrintfLogger
func Printf(format, args) {DefaultLogger.Printf(format, args)}
// usage:
import "xxx/log"
log.Printf("myfunc: doing foo:%s to bar:%s",foo, bar)
I prefer this option 4 for logging and config, and even db access as it is explicit, flexible, and allows easily switching out another logger - just change the imports, no interfaces to change. The calculation on which is best depends on circumstances though - if you were writing a library for widespread use you might prefer to allow setting a logger at the struct level.
I'd usually require setup with an explicit setup on app startup, and always avoid using init functions as they are confusing and difficult to test.
I've always explicitly passed a *logrus.Logger into functions (or occasionally objects) that needed it. That avoids weird dependency cycles, makes it explicit that logging is part of the things this function does, and makes it easier to reuse the function or module elsewhere. The initial log object is created and configured in my main function (likely after some command-line argument handling).
I’ve application which should use log in state of debug. i.e. all the logs that I want to
provide is like log.debug
I’ve read about it and find the following
https://github.com/Sirupsen/logrus
https://github.com/uber-go/zap
My question is how should I “tell” to the program that now run at debug mode an then
all the logs will be printed since this I believe should come from outside …
example will be very helpful since Im new to golfing .
Ok, a really simple example of the approach I suggested in the comment:
package main
import (
"os"
"github.com/sirupsen/logrus"
)
func init() {
lvl, ok := os.LookupEnv("LOG_LEVEL")
// LOG_LEVEL not set, let's default to debug
if !ok {
lvl = "debug"
}
// parse string, this is built-in feature of logrus
ll, err := logrus.ParseLevel(lvl)
if err != nil {
ll = logrus.DebugLevel
}
// set global log level
logrus.SetLevel(ll)
}
func main() {
logrus.Debug("Will only be visible if the loglevel permits it")
}
The original comment:
Check codebases that are out there. In go, the common way to do that is to load configuration via environment variables (eg LOG_LEVEL, and use os.Getenv or a config package to load the value). That's why logrus for example allows you to set log levels via ints or strings.
Please, please: before you ask a question, read the basic info about the packages you use. Even the github repo for logrus' main README contains an example setting the log level to a specific level:
https://github.com/sirupsen/logrus#example
If the only thing you need is check level before printing you can create you own thin wrapper for standard logger.
This will help to better understand how they work. Feel free to ask any questions.
package main
import (
"log"
)
type MyLog struct {
PrintDebug bool
}
func (m *MyLog) Debug(args ...interface{}) {
if m.PrintDebug {
m.Print(args...)
}
}
func (m *MyLog) Print(args ...interface{}) {
log.Print(args...)
}
func main() {
ml := MyLog{}
ml.Debug("PrintDebig = false, so no oitput")
ml.Print("this will be printed anyway")
ml.PrintDebug = true
ml.Debug("Hello, playground")
}
https://play.golang.org/p/gKxQtC9NqX
I have not yet able to find some good examples in using Gauge, Counter and Histogram in prometheus. Any help on this will do. I tried using the the documentation but I was not able to successfully create a working app.
You could find examples form the prometheus/client_golang. To get you started, you can just get the packages:
$ go get github.com/prometheus/client_golang/prometheus
$ go get github.com/prometheus/client_golang/prometheus/push
The you can just run the following example by setting your correct pushgateway address, which is http://localhost:9091/ in this example:
package main
import (
"fmt"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/push"
)
func ExamplePusher_Push() {
completionTime := prometheus.NewGauge(prometheus.GaugeOpts{
Name: "db_backup_last_completion_timestamp_seconds",
Help: "The timestamp of the last successful completion of a DB backup.",
})
completionTime.SetToCurrentTime()
if err := push.New("http://localhost:9091/", "db_backup").
Collector(completionTime).
Grouping("db", "customers").
Push(); err != nil {
fmt.Println("Could not push completion time to Pushgateway:", err)
}
}
func main() {
ExamplePusher_Push()
}
Run your script:
$ go run pushExample.go
After running your code, you should see the metrics at your gateway (http://localhost:9091/). The interface looks like the following:
I have found this
`
package main
import (
"net/http"
"github.com/prometheus/client_golang/prometheus"
)
var (
cpuTemp = prometheus.NewGauge(prometheus.GaugeOpts{
Name: "cpu_temperature_celsius",
Help: "Current temperature of the CPU.",
})
hdFailures = prometheus.NewCounter(prometheus.CounterOpts{
Name: "hd_errors_total",
Help: "Number of hard-disk errors.",
})
)
func init() {
prometheus.MustRegister(cpuTemp)
prometheus.MustRegister(hdFailures)
}
func main() {
cpuTemp.Set(65.3)
hdFailures.Inc()
http.Handle("/metrics", prometheus.Handler())
http.ListenAndServe(":8080", nil)
}
`
This might be useful to some.
Prometheus is a pull-based system, if you want push-based monitoring, you need to use a gateway of some sort. A minimal example (without actually doing anything useful like starting an HTTP listener, or actually doing anything to a metric) follows:
import (
"github.com/prometheus/client_golang/prometheus"
"net/http"
)
var responseMetric = prometheus.NewHistogram(
prometheus.HistogramOpts{
Name: "request_duration_milliseconds",
Help: "Request latency distribution",
Buckets: prometheus.ExponentialBuckets(10.0, 1.13, 40),
})
func main() {
prometheus.MustRegister(responseMetric)
http.Handle("/metrics", prometheus.Handler())
// Any other setup, then an http.ListenAndServe here
}
You then need to configure Prometheus to scrape the /metrics page your binary provides.