How and when is the Go sdk logger flushed? - go

Question

I'm trying to determine if the default/sdk logger log.PrintYYY() functions are flushed at some point in time, on exit, on panic, etc. I'm unsure if I need to find a way to flush the writer that the logger is hooked up to, especially when setting the output writer with SetOutput(...). Of course, the writer interface doesn't have a flush() method, so not really sure how that might get done.
How and when is the Go sdk logger flushed?

Answer 1

The log package is not responsible for flushing the underlying io.Writer. It would be possible for the log package to perform a type-assertion to see if the current io.Writer has a Flush() method, and if so then call it, but there is no guarantee that if multiple io.Writers are "chained", the data will eventually be flushed to the bottommost layer.
And the primary reason why the log package doesn't flush in my opinion is performance. We use buffered writers so we don't have to reach the underlying layer each time a single byte (or byte slice) is written to it, but we can cache the recently written data, and when we reach a certain size (or a certain time), we can write the "batch" at once, efficiently.
If the log package would flush after each log statement, that would render buffered IO useless. It might not matter in case of small apps, but if you have a high traffic web server, issuing a flush after each log statement (of which there may be many in each request handing) would cause a serious performance drawback.
Then yes, there is the issue if the app is terminated, the last log statements might not make it to the underlying layer. The proper solution is to do a graceful shutdown: implement signal handling, and when your app is about to terminate, properly flush and close the underlying io.Writer of the logger you use. For details, see:
Is it possible to capture a Ctrl+C signal and run a cleanup function, in a "defer" fashion?
Is there something like finally() in Go just opposite to what init()?
Are deferred functions called when SIGINT is received in Go?
If–for simplicity only–you still need a logger that flushes after each log statement, you can achieve that easily. This is because the log.Logger type guarantees that each log message is delivered to the destination io.Writer with a single Writer.Write() call:
Each logging operation makes a single call to the Writer's Write method. A Logger can be used simultaneously from multiple goroutines; it guarantees to serialize access to the Writer.
So basically all you need to do is create a "wrapper" io.Writer whose Write() method does a flush after "forwarding" the Write() call to its underlying writer.
This is how it could look like:
type myWriter struct {
io.Writer
}
func (m *myWriter) Write(p []byte) (n int, err error) {
n, err = m.Writer.Write(p)
if flusher, ok := m.Writer.(interface{ Flush() }); ok {
flusher.Flush()
} else if syncer := m.Writer.(interface{ Sync() error }); ok {
// Preserve original error
if err2 := syncer.Sync(); err2 != nil && err == nil {
err = err2
}
}
return
}
This implementation checks for both the Flush() method and os.File's Sync() method, and calls if they are "present".
This is how this can be used so that logging statements always flush:
f, err := os.Create("log.txt")
if err != nil {
panic(err)
}
defer f.Close()
log.SetOutput(&myWriter{Writer: f})
log.Println("hi")
See related questions:
Go: Create io.Writer inteface for logging to mongodb database
net/http set custom logger

Answer 2

Logger shouldn't know how to flush data. You have to flush output writer which you specified on logger creation (if it has such capability).
See example from github discussion
package main
import (
"bufio"
"flag"
"log"
"os"
"strings"
)
func main() {
var flush bool
flag.BoolVar(&flush, "flush", false, "if set, will flush the buffered io before exiting")
flag.Parse()
br := bufio.NewWriter(os.Stdout)
logger := log.New(br, "", log.Ldate)
logger.Printf("%s\n", strings.Repeat("This is a test\n", 5))
if flush {
br.Flush()
}
logger.Fatalf("exiting now!")
}
You can read entire discussion related to your issue on github
Alternatively, you can look at 3rd party loggers which have flush. Check out zap logger which have method logger.Sync()