log printf and Write connection - go

I cannot for the life of me from below golang code, why when you call write function at the bottom,
func write(message string) {
log.Printf("%v\n", message)
}
why log.Printf calls below method
func (fl fileLog) Write(data []byte) (int, error ) {
fmt.Println("does this ever get called?")
f, err := os.OpenFile(string(fl), os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0600)
if err != nil {
return 0, err
}
defer f.Close()
return f.Write(data)
}
Logger's printf definition is below
func (*Logger) Printf ΒΆ
func (l *Logger) Printf(format string, v ...interface{})
-- Full Code below, someone please explain to me why this is so please--
package main
import (
"fmt"
stlog "log"
"os"
)
var log *stlog.Logger
type fileLog string
func (fl fileLog) Write(data []byte) (int, error ) {
fmt.Println("does this ever get called?")
f, err := os.OpenFile(string(fl), os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0600)
if err != nil {
return 0, err
}
defer f.Close()
return f.Write(data)
}
func registerHandlers() {
var msgRaw = "how are you"
write(msgRaw)
}
func run(destination string) {
log = stlog.New(fileLog(destination), "", stlog.LstdFlags)
}
func main() {
fmt.Println("here we go")
run("../test.log")
registerHandlers()
}
func write(message string) {
log.Printf("%v\n", message)
}

The logger writes the log messages to an io.Writer, which is an interface defined as:
type Writer interface {
Write([]byte) (int,error)
}
The fileLog type implements the io.Writer interface, and you set it as the output of the new logger. So whenever the logger tries to write a log, it calls the Write function of its writer, which is fileLog.Write.

Related

Correct way to pass interfaces to methods

Preference: I'm new to Golang and keen to improve.
So I'm trying to learn from Dave Cheney's talk here: https://youtu.be/NwEuRO_w8HE?t=812 where we pass interfaces to methods to make code clearer and more generic.
I've implemented an example below, where I have a struct that can be outputted either to the std.out or to a file. However, I feel like it's a bit redundant just making empty structs (called "print" and "save" in ,my example), just so I can attach methods. How can I improve this?
package main
import (
"fmt"
"io"
"io/ioutil"
"log"
)
type file struct {
data string
}
func (f *file) output(w io.Writer) error {
len, err := w.Write([]byte(f.data))
if err != nil {
return err
} else {
log.Println("Bytes Out: ", len)
return nil
}
}
type print struct{}
func (print *print) Write(p []byte) (n int, err error) {
return fmt.Printf("%s\n", p)
}
type save struct{}
func (s *save) Write(p []byte) (n int, err error) {
err = ioutil.WriteFile("./dat1", []byte(p), 0644)
if err != nil {
return -1, err
}
return len(p), nil
}
func main() {
f := file{"test"}
s := save{}
p := print{}
f.output(&s)
f.output(&p)
}
Your intentions aren't clear, but to answer your original question: how to pass functions as interface values?
You could create a single adapter-like type that implements io.Writer, and when its Write() method is called, it forwards to a function of your choice. A typical example of this is the http.HandlerFunc: it's a function type that implements http.Handler, so a function can be passed when an http.Handler implementation is required.
For an io.Writer adapter, it could look like this:
type WriteFunc func(p []byte) (n int, err error)
func (w WriteFunc) Write(p []byte) (n int, err error) {
return w(p)
}
And if you have functions like these:
func PrintWrite(p []byte) (n int, err error) {
return fmt.Printf("%s\n", p)
}
func SaveWrite(p []byte) (n int, err error) {
err = ioutil.WriteFile("./dat1", []byte(p), 0644)
if err != nil {
return -1, err
}
return len(p), nil
}
You can use them as io.Writer like this:
f.output(WriteFunc(PrintWrite))
f.output(WriteFunc(SaveWrite))
Try it on the Go Playground.
You don't need print or save types here. You have a method, output, that takes an io.Writer; the whole point of that is that you can pass it any writer. Both stdout and files are writers, so everything else here is unnecessary. You could just:
func main() {
f := file{"test"}
fp,err := os.Create("./dat1")
if err != nil {
panic(err)
}
f.output(fp) // This writes to a file
f.output(os.Stdout) // This writes to stdout
}

Replacement for *os.File of Go default log

I'm trying to write a wrapper for Go's built-in logger.
This is to have compatibility.
package main
import (
"log"
"os"
)
var(
mylog *log.Logger
)
func main() {
mylog = log.New(os.Stdout, "", 0)
mylog.Printf("test")
}
Instead of using os.Stdout, I want to create one something. Similar to os.Stdout but prints with prefix like below.
package main
import(
"log"
"mylibrary"
)
var(
mylog *log.Logger
)
func main() {
mylog = log.New(mylibrary.Prefix, "", 0)
mylog.Printf("test")
}
Basically, I still want to have *log.Logger while having custom log. Can someone give me a hint how I can make this works?
Currently, I'm using following to do that. But I bet there's a better way.
func NewIoWriter(f func(string)) *io.PipeWriter {
r, w := io.Pipe()
go func() {
scanner := bufio.NewScanner(r)
for scanner.Scan() {
f(scanner.Text())
}
if err := scanner.Err(); err != nil {
f(err.Error())
}
r.Close()
}()
runtime.SetFinalizer(w, (*io.PipeWriter).Close)
return w
}
What would be the better way to make it work?
Thank you
How about something like this:
type myLogWriter struct {
logFunc func(string)
line string
}
func (w *myLogWriter) Write(b []byte) (int, error) {
l := len(b)
for len(b) != 0 {
i := bytes.Index(b, []byte{'\n'})
if i == -1 {
w.line += string(b)
break
} else {
w.logFunc(w.line + string(b[:i]))
b = b[i+1:]
w.line = ""
}
}
return l, nil
}
func NewLogWriter(f func(string)) *myLogWriter {
return &myLogWriter{
logFunc: f,
}
}
See https://play.golang.org/p/L6PG1gCK1er.

How to retype a function in Golang?

In my current Golang project, I use buffering of logs before sending them to Elasticsearch within my log library. I want to introduce something like C atexit() function to flush all pending logs in case of an unexpected exit.
I have found atexit library but it was insufficient in my case because it does not allow passing of arguments to handler functions. I decided to write my version and ended up with roughly similar structure. I have module atexit:
package atexit
import (
"fmt"
"os"
"reflect"
)
var handlers []interface{}
var params []interface{}
func runHandler(handler interface{}, p interface{}) {
defer func() {
if err := recover(); err != nil {
fmt.Fprintln(os.Stderr, "error: atexit handler error:", err)
}
}()
f := reflect.ValueOf(handler)
s := reflectSlice(p)
fmt.Printf("%#v", s)
f.Call(s)
}
func reflectSlice(slice interface{}) []reflect.Value {
s := reflect.ValueOf(slice)
if s.Kind() != reflect.Slice {
panic("InterfaceSlice() given a non-slice type")
}
ret := make([]reflect.Value, s.Len())
for i:=0; i<s.Len(); i++ {
ret[i] = reflect.ValueOf(s.Index(i))
}
return ret
}
func runHandlers() {
for i, handler := range handlers {
runHandler(handler, params[i])
}
}
func Exit(code int) {
runHandlers()
os.Exit(code)
}
func Register(handler interface{}, p interface{}) {
f := reflect.TypeOf(handler)
if f.Kind() != reflect.Func {
panic("Register() given a non-function type")
}
handlers = append(handlers, handler)
params = append(params, p)
}
and I am calling it from the main program:
package main
import (
"fmt"
"./atexit"
"encoding/json"
"reflect"
)
type Batch struct {
Index string `json:"index"`
Type string `json:"_Type"`
Content interface{} `json:"Content"`
}
func flush(b ...interface{}) {
for _, entry := range(b) {
fmt.Printf("%#v\n", entry)
fmt.Println(reflect.TypeOf(reflect.ValueOf(entry)))
fmt.Println(reflect.TypeOf(entry))
a, err := json.Marshal(entry)
if err != nil {
fmt.Println("error!")
}
fmt.Println(string(a))
}
}
func handler(v ...interface{}) {
fmt.Println("Exiting")
for _, batch := range(v) {
fmt.Printf("%#v\n", batch)
}
}
func main() {
type ColorGroup struct {
ID int
Name string
Colors []string
}
group := ColorGroup{
ID: 1,
Name: "Reds",
Colors: []string{"Crimson", "Red", "Ruby", "Maroon"},
}
fmt.Printf("%#v\n", group)
r, err := json.Marshal(group)
if err != nil {
fmt.Println("error:", err)
}
fmt.Println(string(r))
b := []Batch{Batch{"index", "type", "content1"},Batch{"index", "type", "content2"}}
atexit.Register(handler, b)
atexit.Register(flush, []ColorGroup{group})
atexit.Exit(0)
}
As you can see, by calling reflect.ValueOf() I get structure reflect.Value which is then passed to callback function. The problem seems to be that this structure does not contain metadata about json export or is not handled correctly with json.Marshal(), which then outputs empty json. Is there any way I can pass correct []interface{} structure to the callback function or some similar mechanism which would do roughly what I'm trying to accomplish? Please note that I want a general callback mechanism which should be independent of the type passed to it. So far it seems to be impossible or at least very limited by f.Call(s), which is called in runHandler() and takes reflect.Value as its parameter.
You should not be using reflection for this. You should make your handlers be of type func(), then just pass in an anonymous function (possibly a closure) that handles any necessary arguments directly. Example:
atexit.Register(func() {
handler(b)
})

I want to generate four log files for warning, debug, error, and info. I want to control which file should be print at run time

package Loggers
import (
"io"
"os"
"log"
)
var logCloser io.Closer
func MyLogger() {
logFile, err := os.OpenFile("C:/Git/goclassec/src/gclassec/Logs/log.txt", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666)
if err != nil {
panic(err)
}
logCloser = logFile
log.SetOutput(logFile)
}
func CloseMyLogger() {
logCloser.Close()
}
You can use log.New to create different loggers and wrap it inside your MyLogger
This is the New format
New(out io.Writer, prefix string, flag int)
Here is a sample implementation
Code
package loggers
import (
"log"
"os"
)
//MyLogger custom logger
type MyLogger struct {
ErrorL *log.Logger
InfoL *log.Logger
DebugL *log.Logger
}
//New returns logger
//TODO : deal with error
func New() *MyLogger {
ml := new(MyLogger)
if logFile, err := os.OpenFile("C:/Git/goclassec/src/gclassec/Logs/log_error.txt", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666); err == nil {
ml.ErrorL = log.New(logFile, "ERROR", log.LUTC)
}
if logFile, err := os.OpenFile("C:/Git/goclassec/src/gclassec/Logs/log_info.txt", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666); err == nil {
ml.InfoL = log.New(logFile, "INFO", log.LUTC)
}
if logFile, err := os.OpenFile("C:/Git/goclassec/src/gclassec/Logs/log_debug.txt", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666); err == nil {
ml.DebugL = log.New(logFile, "DEBUG", log.LUTC)
}
return ml
}
func (ml MyLogger) Error(data ...interface{}) {
ml.ErrorL.Println(data...)
}
func (ml MyLogger) Info(data ...interface{}) {
ml.InfoL.Println(data...)
}
func (ml MyLogger) Debug(data ...interface{}) {
ml.DebugL.Println(data...)
}
Usage
logger := loggers.New()
logger.Error("Error occured",err)
logger.Info("Doing","something","again","blah")

Proper pattern to encapsulate log setup in golang

When trying to move log setup code into a separate function I ran into inability to hide the destination file object from the main function. In the following INCORRECT simplified example the attempt is made to setup log writing to both Stderr and a file via a single function call:
package main
import (
"io"
"log"
"os"
)
func SetupLogging() {
logFile, err := os.OpenFile("test.log", os.O_APPEND|os.O_CREATE, 0666)
if err != nil {
log.Panicln(err)
}
defer logFile.Close()
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
}
func main() {
SetupLogging()
log.Println("Test message")
}
Clearly is does not work because defer closes the log file at the end of the SetupLogging function.
A working example below adds extra code and IMHO looses some clarity if repeated in a larger application as a pattern:
package main
import (
"io"
"log"
"os"
)
func SetupLogging() *os.File {
logFile, err := os.OpenFile("test.log", os.O_APPEND|os.O_CREATE, 0666)
if err != nil {
log.Panicln(err)
}
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
return logFile
}
func main() {
logf := SetupLogging()
defer logf.Close()
log.Println("Test message")
}
Is there a different way to fully encapsulate open file management into a function, yet still nicely release the handle?
I have now successfully used the below approach for about a year in multiple projects. The idea is to return a function from the setup call. That resulting function contains the destruction logic. Here is an example:
package main
import (
"fmt"
"io"
"log"
"os"
)
func LogSetupAndDestruct() func() {
logFile, err := os.OpenFile("test.log", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666)
if err != nil {
log.Panicln(err)
}
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
return func() {
e := logFile.Close()
if e != nil {
fmt.Fprintf(os.Stderr, "Problem closing the log file: %s\n", e)
}
}
}
func main() {
defer LogSetupAndDestruct()()
log.Println("Test message")
}
It is using a closure around the cleanup logic being deferred.
A somewhat more elaborate public example of using this approach is in the Viper code: here is the return from a test initializer, and here it is used to encapsulate the cleanup logic and objects
The proper way of doing this is passing the handle in main to SetupLogging:
func SetupLogging(lf *os.File) {
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
log.Println("Started")
}
func main() {
logFile, err := os.OpenFile("test.log", os.O_APPEND|os.O_CREATE, 0666)
if err != nil {
log.Panicln(err)
}
defer logFile.Close()
SetupLogging(logFile)
log.Println("Test message")
}
Another option is to use runtime.SetFinalizer, but it's not always guaranteed to run before main exits.
func SetupLogging() {
logFile, err := os.OpenFile("test.log", os.O_APPEND|os.O_CREATE, 0666)
if err != nil {
log.Panicln(err)
}
runtime.SetFinalizer(logFile, func(h *os.File) {
h.Close()
})
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
}
You can do this using channels, here is my approach
type InfoLog struct {
InfoChan chan string
CloseChan chan struct{} //empty signal
log *log.Logger
file *os.File
}
func NewInfoLog(file *os.File) *InfoLog {
return &InfoLog{
InfoChan: make(chan string),
CloseChan: make(chan struct{}),
log: log.New(file, "TAG", log.Ldate|log.Ltime),
file: file,
}
}
func (i *InfoLog) listen() {
for {
select {
case infoMsg := <-i.InfoChan:
i.log.Println(infoMsg)
case <-i.CloseChan:
i.file.Close()
close(i.InfoChan)
}
}
}
then in main
func main() {
infoLog := NewInfoLog(ANY_OPEN_FILE_HERE)
go infoLog.listen()
infoLog.InfoChan <- "msg"
infoLog.InfoChan <- "msg"
infoLog.InfoChan <- "msg"
infoLog.CloseChan <- struct{}{}
// exits normaly
}
you can see an asynchronous log system i have made for a complete example: https://github.com/sescobb27/ciudad-gourmet/blob/master/services/log_service.go
in case where multiple "teardown" processes are needed, great solution to this is using google context package (https://blog.golang.org/context). advantage is that you can teardown all currently executing procedures with single context. smth like this:
package main
import (
"fmt"
"io"
"log"
"os"
"golang.org/x/net/context"
)
func LogSetup(ctx context.Context) error {
logFile, err := os.OpenFile("test.log", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666)
if err != nil {
return err
}
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
// here we could f.ex. execute:
// sendLogOutputToExternalService(ctx)
// and it could have it's own teardown procedure
// which would be called on main context's expiration
go func() {
for _ = range ctx.Done() {
err := logFile.Close()
if err = nil {
fmt.Fprintf(os.Stderr, "Problem closing the log file: %s\n", e)
}
}()
return nil
}
func main() {
var stopAll func()
mainContext, stopAll = context.WithCancel(context.Background())
defer stopAll()
err := LogSetup(mainContext)
if err!=nil {
log.Fatal("error while initializing logging")
}
log.Println("Test message")
}

Resources