I am using logrus library for structured logging in my Go project.
I have configured my logrus as below:
// Global variable for logging
var gLog = &Logger{moduleName: ModuleName, logrus: logrus.New()}
type Logger struct {
moduleName string
logrus *logrus.Logger
}
func SetupGlobalLogger(logPrefix string, logMode string) error {
if logMode == "file" {
logFilePath := fmt.Sprintf("var/%s.log", vite.Environment())
file, err := os.OpenFile(logFilePath, logFileFlags, logFilePermission)
if err != nil {
return err
}
gLog.logrus.SetOutput(file)
// redirect logs written using standard log library to same place as logrus
log.SetOutput(gLog.logrus.Writer())
log.Println(vite.MarkInfo, "redirect log to file:", logFilePath)
}
return nil
}
In this Go project, there are several places where standard log library statements like log.Println() are used.
I want to redirect those log messages to logrus.
For that I am using following statement in above code.
log.SetOutput(gLog.logrus.Writer())
My question is: Is this thread safe?
If one thread/go-routine is executing log.Println() while another executing gLog.logrus.Info() or something on logrus, will that be fine?
Is Logrus Thread-safe?
Yes:
Thread safety
By default, Logger is protected by a mutex for concurrent writes.
Except when it isn't.
Most of the non-thread-safe cases are bugs which will occur only in rare situations. Whether any of them matter to you depends, of course, on your use case.
Related
I try to add hook with WithOptions but there was nothing printed for catching some of log events:
logger.WithOptions(zap.Hooks(func(entry zapcore.Entry) error {
fmt.Println("test hooks test hooks")
return nil
}))
From the documentation:
func (log *Logger) WithOptions(opts ...Option) *Logger
WithOptions clones the current Logger, applies the supplied Options, and returns the resulting Logger. It's safe to use concurrently.
Notice that it clones a new logger instead of modifying the logger. So, you should reassign the logger variable (or define a new variable) like this:
logger = logger.WithOptions(zap.Hooks(func(entry zapcore.Entry) error {
fmt.Println("test hooks test hooks")
return nil
}))
I'm used to work with java and log4j, in which I could dynamically change the log level for a single class without any additional (coding) effort.
Now, when I work with golang (zap for logging) – I seek the same functionality, but cannot find it.
Is there an easy way to change log level dynamically for a file or a package or a part of the code?
zap supports log.Logger wrapping via constructors like NewStdLog, so if you want to keep using zap then the following technique will work:
If you don't want to use a third-party loggers, the following technique is quite simple, using go's standard library log.Logger:
Define some default app loggers:
var (
pkgname = "mypkgname"
Info = log.New(os.Stdout, "[INFO:"+pkgname+"] ", log.LstdFlags)
Debug = log.New(ioutil.Discard, "[DBUG:"+pkgname+"] ", log.LstdFlags|log.Lshortfile)
Trace = log.New(ioutil.Discard, "[TRCE:"+pkgname+"] ", log.LstdFlags|log.Lmicroseconds|log.Lshortfile)
)
so by default Info will write to Stdout - but Debug & Trace will go be "off" by default.
One can then, safely (i.e. goroutine-safe) turn these log.Loggers on/off either at start time:
func init() {
if v, ok := os.LookupEnv("DEBUG"); ok && v != "" {
Info.Println("ENV VAR 'DEBUG' set: enabling debug-level logging")
Debug.SetOutput(os.Stderr)
}
}
or during runtime:
func httpServiceHandler(r *req) {
if r.TraceingOn {
Trace.SetOutput(os.Stderr)
} else {
Trace.SetOutput(ioutil.Discard)
}
}
Logging at pkg level is just like any other log method:
Debug.Printf("http request: %+v", r)
And if there's an expensive log event that you don't want to generate - if say the logger is set to discard - you can safely check the state of the logger like so:
if Trace.Writer() != ioutil.Discard {
// do expensive logging here
// e.g. bs, _ = ioutil.ReadAll(resp.Body); Trace.Println("RESP: RAW-http response:", string(bs))
}
Update: I think now that there is no universal answer to this question. We can return both errors using the technique explained in the answer. I think that the most important thing here is not to forget the case when we have two errors and somehow handle it.
Notes: There are many questions on SO about how to return an error from deferred function. This is not a question here.
(In Go) What is the proper way to return an error from a deferred function when the function is already returning an error. For example
func errorMaker() (err error) {
defer func() {
err = errors.New("Deferred error")
}()
err = errors.New("Some error")
return
}
func main() {
err := errorMaker()
fmt.Printf("Error: %v\n", err)
}
In the code above the error returned by the deferred function overwrites the error returned by the function. What is the canonical way to return both errors? If another programmer uses my function what result might she expect from the function when the function returns 'two errors'?
Should I use Error wrapping for this?
Additional notes:
As #Volker says in his comment I write some application specific handling for this error. Because I know what should be done based on nature of the errors.
I think my question is - if I want to return all errors from the function what is the best way to combine them in my scenario?
Disclaimer: I don't know if the following advice can be seen as "standard" or "widely-accepted".
Should I use Error wrapping for this?
Short answer: yes (I would do so).
Go 1.12 and earlier
What I do when I need my errors to convey some specific meaning, without foregoing the error interface, I create a wrapper that implements the error interface - Error() string -. This wrapper contains all extra information I need.
If the caller is aware of the existence of those extra info, it can unwrap the error with a cast and find those info.
With the added benefit that unaware callers can just handle the error as a generic error.
type MyError struct {
DeferredError error
}
// Implements 'error' interface
func (e MyError) Error() string {
// format to string
}
func someFunc() error {
// might return an instance of MyError
}
...
// Caller code
err := someFunc()
if err != nil {
if myErr, ok := err.(*MyError); ok {
// here you can access the wrapped info
fmt.Println(myErr.DeferredError)
} else {
// otherwise handle the error generically
}
}
Go 1.13 onwards
With Go.13 you can use errors.As to unwrap an error. From the official docs:
[The method] As finds the first error in err's chain that matches target, and if so, sets target to that error value and returns true. The chain consists of err itself followed by the sequence of errors obtained by repeatedly calling Unwrap.
var myErr *MyError
if errors.As(err, &myErr) {
// here you can access the wrapped info
fmt.Println(myErr.DeferredError)
} else {
// otherwise handle the error generically
}
As the docs say the myErr variable is populated as a side-effect of calling As.
Currently running a go service on GCP however in the logs viewer every message is treated as an error.
Is there a generally advised way of logging to stderr and stdout depending on the log level. Ie errors to stderr and anything else to stdout.
I'm currently using the logrus package and have come across this implementation. Other ways i see achieving this while still using the same package is to pass the logger to each package that needs it or to create a global log object, neither of which i am too keen on.
https://github.com/microsoft/fabrikate/pull/252/commits/bd24d62d7c2b851ad6e7b36653eb0a6dc364474b#diff-ed0770fdbf87b0c6d536e33a99a8df9c
You can use Stackdriver library package for GoLang:
go get -u cloud.google.com/go/logging
Then you can use StandardLogger:
// Sample stdlogging writes log.Logger logs to the Stackdriver Logging.
package main
import (
"context"
"log"
"cloud.google.com/go/logging"
)
func main() {
ctx := context.Background()
// Sets your Google Cloud Platform project ID.
projectID := "YOUR_PROJECT_ID"
// Creates a client.
client, err := logging.NewClient(ctx, projectID)
if err != nil {
log.Fatalf("Failed to create client: %v", err)
}
defer client.Close()
// Sets the name of the log to write to.
logName := "my-log"
logger := client.Logger(logName).StandardLogger(logging.Info)
// Logs "hello world", log entry is visible at
// Stackdriver Logs.
logger.Println("hello world")
}
Here you can find documentation on Google Cloud website
Update:
Alternatively you could give a try GCP formatter for logrus
This will not tie your app to Google Cloud Platform. However, it does not mean that on another platform you will not need to change your code to format output.
Using StackDriver library is the recommended solution for Google Cloud.
We use https://github.com/rs/zerolog with the following method called in our Init() method to setup the logging options at global level:
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
"os"
)
// initializeLogging sets up the logging configuration for the service.
// Invoke this method in your Init() method.
func initializeLogging() {
// Set logging options for production development
if os.Getenv("ENV") != "DEV" {
// change the level field name to ensure these are parsed correctly in Stackdriver
zerolog.LevelFieldName = "severity"
// UNIX Time is faster and smaller than most timestamps
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
} else {
// Set logging options for local development
log.Logger = log.Output(zerolog.ConsoleWriter{Out: os.Stderr})
zerolog.SetGlobalLevel(zerolog.DebugLevel)
}
// Example log
log.Info().Msg("This is how you log at Info level")
}
I recommend the https://github.com/apsystole/log. You can swap either log and logrus imports for it. It is a small zero-dependency module, hence much lighter than logrus.
I am using logrus in a Go app. I believe this question is applicable to any other logging package (which doesn't offer external file based configuration) as well.
logrus provides functions to setup various configuration, e.g. SetOutput, SetLevel etc.
Like any other application I need to do logging from multiple source files/packages, it seems you need to setup these options in each file with logrus.
Is there any way to setup these options once somewhere in a central place to be shared all over the application. That way if I have to make logging level change I can do it in one place and applies to all the components of the app.
You don't need to set these options in each file with Logrus.
You can import Logrus as log:
import log "github.com/Sirupsen/logrus"
Then functions like log.SetOutput() are just functions and modify the global logger and apply to any file that includes this import.
You can create a package global log variable:
var log = logrus.New()
Then functions like log.SetOutput() are methods and modify your package global. This is awkward IMO if you have multiple packages in your program, because each of them has a different logger with different settings (but maybe that's good for some use cases). I also don't like this approach because it confuses goimports (which will want to insert log into your imports list).
Or you can create your own wrapper (which is what I do). I have my own log package with its own logger var:
var logger = logrus.New()
Then I make top-level functions to wrap Logrus:
func Info(args ...interface{}) {
logger.Info(args...)
}
func Debug(args ...interface{}) {
logger.Debug(args...)
}
This is slightly tedious, but allows me to add functions specific to my program:
func WithConn(conn net.Conn) *logrus.Entry {
var addr string = "unknown"
if conn != nil {
addr = conn.RemoteAddr().String()
}
return logger.WithField("addr", addr)
}
func WithRequest(req *http.Request) *logrus.Entry {
return logger.WithFields(RequestFields(req))
}
So I can then do things like:
log.WithConn(c).Info("Connected")
(I plan in the future to wrap logrus.Entry into my own type so that I can chain these better; currently I can't call log.WithConn(c).WithRequest(r).Error(...) because I can't add WithRequest() to logrus.Entry.)
This is the solution that I arrived at for my application that allows adding of fields to the logging context. It does have a small performance impact due to the copying of context base fields.
package logging
import (
log "github.com/Sirupsen/logrus"
)
func NewContextLogger(c log.Fields) func(f log.Fields) *log.Entry {
return func(f log.Fields) *log.Entry {
for k, v := range c {
f[k] = v
}
return log.WithFields(f)
}
}
package main
import (
"logging"
)
func main {
app.Logger = logging.NewContextLogger(log.Fields{
"module": "app",
"id": event.Id,
})
app.Logger(log.Fields{
"startTime": event.StartTime,
"endTime": event.EndTime,
"title": event.Name,
}).Info("Starting process")
}