package logger
import (
"bytes"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
ctrl "sigs.k8s.io/controller-runtime"
)
var _ = Describe("Logger", func() {
It("Test Default Log Level", func() {
buf := &bytes.Buffer{}
testLog := ctrl.Log.WithName("setup")
SetLogger()
testLog.Info("This is a test")
Expect(buf.String(),"This is a test")
})
})
And this is the SetLogger function, which is used also in production:
package logger
import (
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
...
)
func SetLogger() {
opts := zap.Options{
Development: developmentFlag,
StacktraceLevel: stacktraceLevel,
Level: isLevelEnabler,
Encoder: logFmtEncoder,
}
ctrl.SetLogger(zap.New(zap.UseFlagOptions(&opts)))
}
How I can change the output of the testLog.Info to buffer?
If you are only interested in testing the log message, you can use a hook.
In particular zap.Hooks function constructs a zap.Option from a variable number of hooks. A hook is just a func(entry zapcore.Entry) error which you can use to intercept the entry and write its message to the buffer.
To set this zap.Option into your sigs.k8s.io logger, you set it to the ZapOpts field:
opts := k8szap.Options{
// ...
ZapOpts: []zap.Option{
zap.Hooks(func(entry zapcore.Entry) error {
buf.WriteString(entry.Message)
return nil
}),
},
}
So since you need access to the buffer, you can pass it as argument to the SetLogger function:
func SetLogger(buf *bytes.Buffer) {
opts := zap.Options{
Development: developmentFlag,
StacktraceLevel: stacktraceLevel,
Level: isLevelEnabler,
Encoder: logFmtEncoder,
// here 'zap' selector is 'go.uber.org/zap'
ZapOpts: []zap.Option{
zap.Hooks(func(entry zapcore.Entry) error {
buf.WriteString(entry.Message)
return nil
}),
},
}
// here I call 'k8szap' selector the package 'sigs.k8s.io/controller-runtime/pkg/log/zap'
ctrl.SetLogger(k8szap.New(k8szap.UseFlagOptions(&opts)))
}
And then in your test function:
It("Test Default Log Level", func() {
buf := &bytes.Buffer{}
testLog := ctrl.Log.WithName("setup")
// pass buffer to SetLogger
SetLogger(buf)
testLog.Info("This is a test")
Expect(buf.String(), "This is a test")
})
Minimal example (it may timeout when downloading the packages in the playground): https://play.golang.org/p/oBN3SHFKVC8
Related
Followup question to: How do I guarantee that the request happened correctly when mocking an API?
main.go
package main
import (
"net/http"
)
func SomeFeature(host, a string) {
if a == "foo" {
resp, err := http.Get(host + "/foo")
}
if a == "bar" {
resp, err := http.Get(host + "/baz"))
}
// baz is missing, the test should error!
}
main_test.go
package main
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestSomeFeature(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(200)
}))
testCases := []struct {
name string
variable string
}{
{
name: "test 1",
variable: "foo",
},
{
name: "test 2",
variable: "bar",
},
{
name: "test 3",
variable: "baz",
},
}
for _, tc := range testCases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
SomeFeature(server.URL, tc.variable)
// assert that the http call happened somehow?
})
}
}
GO Playground: https://go.dev/play/p/EFanSSzgnbk
How to do I assert that each test case send a request to the mocked server?
How can I assert that a request wasn't sent?
All while keeping the tests parallel/concurrent?
You could create a new server for each test case.
Or you can use channels, specifically a map of channels where the key is the test case's identifier, e.g.
getChans := map[string]chan struct{}{}
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
key := strings.Split(r.URL.Path, "/")[1] // extract channel key from path
go func() { getChans[key] <- struct{}{} }()
w.WriteHeader(200)
}))
Add a channel key field to the test case. This will be added to the host's URL and the handler will then extract the key, as demonstrated above, to get the correct channel. Also add a field to indicate whether http.Get should be called or not:
testCases := []struct {
name string
chkey string
variable string
shouldGet bool
}{
{
name: "test 1",
chkey: "key1"
variable: "foo",
shouldGet: true,
},
// ...
}
Before running the test case add the test-case-specific channel to the map:
getChans[tc.chkey] = make(chan struct{})
Then use the channel key field in the test case as part of the host's URL path:
err := SomeFeature(server.URL+"/"+tc.chkey, tc.variable)
if err != nil {
t.Error("SomeFeature should not error")
}
And to check whether or not http.Get was called use select with some acceptable timeout:
select {
case <-getChans[tc.chkey]:
if !tc.shouldGet {
t.Error(tc.name + " get called")
}
case <-time.Tick(3 * time.Second):
if tc.shouldGet {
t.Error(tc.name + " get not called")
}
}
https://go.dev/play/p/7By3ArkbI_o
How to get function name printed in logs from Uber Zap logging ?
This is the PR request with which they seemed to have added the functionality to output function names in log.
I am using golang version 1.15 and go.uber.org/zap v1.16.0
This is my code:
package main
import (
"go.uber.org/zap"
)
var logger *zap.Logger
func main() {
logger := NewLogger()
logger.Info("Test msg Main")
TestFunc(logger)
}
func TestFunc(logger *zap.Logger) {
logger.Info("Test msg TestFunc")
}
func NewLogger() *zap.Logger {
config := zap.NewDevelopmentConfig()
opts := []zap.Option{
zap.AddCallerSkip(1), // traverse call depth for more useful log lines
zap.AddCaller(),
}
logger, _ = config.Build(opts...)
return logger
}
This is the output I get with/without the addition of AddCaller() option
2021-03-01T15:00:02.927-0800 INFO runtime/proc.go:204 Test msg Main
2021-03-01T15:00:02.927-0800 INFO cmd/main.go:12 Test msg TestFunc
I am expecting something like
2021-03-01T15:00:02.927-0800 INFO runtime/proc.go:204 main Test msg Main
2021-03-01T15:00:02.927-0800 INFO cmd/main.go:12 TestFunc Test msg TestFunc
By default, the provided encoder presets (NewDevelopmentEncoderConfig used by NewDevelopmentConfig and NewProductionEncoderConfig used by NewProductionConfig) do not enable function name logging.
To enable function name, you need to enable caller (true by default) and set a non-empty value for config.EncoderConfig.FunctionKey.
Source: EncoderConfig
type EncoderConfig struct {
// Set the keys used for each log entry. If any key is empty, that portion
// of the entry is omitted.
...
CallerKey string `json:"callerKey" yaml:"callerKey"`
FunctionKey string `json:"functionKey" yaml:"functionKey"` // this needs to be set
StacktraceKey string `json:"stacktraceKey" yaml:"stacktraceKey"`
...
}
Example Console Logger:
func main() {
config := zap.NewDevelopmentConfig()
// if you're using console encoding, the FunctionKey value can be any
// non-empty string because console encoding does not print the key.
config.EncoderConfig.FunctionKey = "F"
logger, _ := config.Build()
logger.Info("Test Logging")
// Output: 2021-03-03T11:41:47.728+0800 INFO example/main.go:11 main.main Test Logging
}
Example JSON Logger:
func main() {
config := zap.NewProductionConfig()
// the FunctionKey value matters because it will become the JSON field
config.EncoderConfig.FunctionKey = "func"
logger, _ := config.Build()
log(logger)
// Output: {"level":"info","ts":1614743088.538128,"caller":"example/main.go:15","func":"main.log","msg":"Test Logging"}
}
func log(logger *zap.Logger) {
logger.Info("Test Logging")
}
I am using github.com/sirupsen/logrus for logging in my golang scripts, however I want to get the filename and the line number which is logging the message. I am able to get that using the below code:
package main
import (
"fmt"
"os"
"runtime"
"strings"
"github.com/sirupsen/logrus"
)
func GetLogger() (*logrus.Logger, *os.File) {
log := logrus.New()
log.SetReportCaller(true)
file, err := os.OpenFile("info.log", os.O_CREATE|os.O_APPEND, 0644)
if err != nil {
log.Fatal(err)
}
log.Out = file
log.Formatter = &logrus.TextFormatter{
CallerPrettyfier: func(f *runtime.Frame) (string, string) {
repopath := fmt.Sprintf("%s/src/github.com/bob", os.Getenv("GOPATH"))
filename := strings.Replace(f.File, repopath, "", -1)
return fmt.Sprintf("%s()", f.Function), fmt.Sprintf("%s:%d", filename, f.Line)
},
}
return log, file
}
However this gives log in the below format:
time="2020-04-02T11:43:19+05:30" level=info msg=Hello func="main.main()" file="D:/.../main.go:13"
But I want the log in format as below:
Apr 02 00:00:00 INFO main.go:20 : Hello this is a log line
How can a custom formatter be written to get this?
This option is included in the library itself since the end of 2018.
Just set 'SetReportCaller' to true.
Here's an example:
package main
import (
log "github.com/sirupsen/logrus"
)
func main() {
// Add this line for logging filename and line number!
log.SetReportCaller(true)
log.Println("hello world")
}
The output:
INFO[0000]/home/trex/go/src/awesomeProject/main.go:11 main.main() hello world
FYI
log.SetReportCaller(true)
log.SetFormatter(&log.JSONFormatter{
CallerPrettyfier: func(frame *runtime.Frame) (function string, file string) {
fileName := path.Base(frame.File) + ":" + strconv.Itoa(frame.Line)
//return frame.Function, fileName
return "", fileName
},
})
output:
{"file":"inspParse.go:290","level":"info","msg":"(3, 24), ","time":"2021-08-30T16:41:38+08:00"}
you can take advantage of codes below
package main
import (
"bytes"
"fmt"
"github.com/sirupsen/logrus"
"io"
"os"
"strings"
)
type MyFormatter struct {}
var levelList = [] string{
"PANIC",
"FATAL",
"ERROR",
"WARN",
"INFO",
"DEBUG",
"TRACE",
}
func (mf *MyFormatter) Format(entry *logrus.Entry) ([]byte, error){
var b *bytes.Buffer
if entry.Buffer != nil {
b = entry.Buffer
} else {
b = &bytes.Buffer{}
}
level := levelList[int(entry.Level)]
strList := strings.Split(entry.Caller.File, "/")
fileName := strList[len(strList)-1]
b.WriteString(fmt.Sprintf("%s - %s - [line:%d] - %s - %s\n",
entry.Time.Format("2006-01-02 15:04:05,678"), fileName,
entry.Caller.Line, level, entry.Message))
return b.Bytes(), nil
}
func MakeLogger(filename string, display bool) *logrus.Logger {
f, err := os.OpenFile(filename, os.O_CREATE|os.O_RDWR|os.O_APPEND, 0644)
if err != nil {
panic(err.Error())
}
logger := logrus.New()
if display {
logger.SetOutput(io.MultiWriter(os.Stdout, f))
} else {
logger.SetOutput(io.MultiWriter(f))
}
logger.SetReportCaller(true)
logger.SetFormatter(&MyFormatter{})
return logger
}
func main() {
logger := MakeLogger("/tmp/test.log", true)
logger.Info("hello world!")
}
Result:
/tmp/test.log
2021-11-24 00:49:10,678 - main.go - [line:58] - INFO - hello world!
The package you're using github.com/sirupsen/logrus produces structured log output: that is key/value pairs. It looks like you want just a plain text logger.
The standard logger import "log", produces output quite like what you want: log.New(out, "INFO", .Ldate|log.Ltime|log.Lshortfile). (See https://play.golang.org/p/LKitIwjPuVH on the playground)
Here's example output:
INFO 2009/11/10 23:00:00 prog.go:10: hello
In go1.14, the extra flag log.Lmsgprefix moves the INFO to before the message, if that's preferable (and you can wait).
If the standard library logger doesn't do what you want (and you're not prepared to live with it), why not just copy it and edit it, essentially making your own log package? It's around 400 lines of straightforward code, and by the time you remove the parts you don't want, it'll be a lot less.
The source is here: https://golang.org/src/log/log.go
Showing file, function name, and line number with logrus:
func Error(err error, msg ...interface{}) {
if pc, file, line, ok := runtime.Caller(1); ok {
file = file[strings.LastIndex(file, "/")+1:]
funcName := runtime.FuncForPC(pc).Name()
logrus.WithFields(
logrus.Fields{
"err": err,
"src": fmt.Sprintf("%s:%s:%d", file, funcName, line),
}).Error(msg...)
}
}
Call this in every log event.
I'm trying to build a customized zap logger with 1) customized *zap.Config and 2) lumberjack, but can't find proper example to apply both configurations.
Since config.Build does not accept WriteSync as an input. Do you know how to achieve this?
func genBaseLoggerZap() Logger {
ex, err := os.Executable()
if err != nil {
Fatalf("Failed to get os.Executable, err: %v", err)
}
zlManager.outputPath = path.Join(filepath.Dir(ex), zlManager.outputPath)
// Want to add sync here..
zapcore.AddSync(&lumberjack.Logger{
Filename: zlManager.outputPath + "123",
MaxSize: 500,
MaxBackups: 10,
MaxAge: 28,
})
return genLoggerZap(BaseLogger, genDefaultConfig())
}
// genLoggerZap creates a zapLogger with given ModuleID and Config.
func genLoggerZap(mi ModuleID, cfg *zap.Config) Logger {
logger, err := cfg.Build()
if err != nil {
Fatalf("Failed to generate zap logger, err: %v", err)
}
newLogger := &zapLogger{mi, cfg, logger.Sugar()}
newLogger.register()
return newLogger
}
You can add custom log destinations using the zap.RegisterSink function and the Config.OutputPaths field. RegisterSink maps URL schemes to Sink constructors, and OutputPaths configures log destinations (encoded as URLs).
Conveniently, *lumberjack.Logger implements almost all of the zap.Sink interface already. Only the Sync method is missing, which can be easily added with a thin wrapper type.
package main
import (
"net/url"
"go.uber.org/zap"
lumberjack "gopkg.in/natefinch/lumberjack.v2"
)
type lumberjackSink struct {
*lumberjack.Logger
}
// Sync implements zap.Sink. The remaining methods are implemented
// by the embedded *lumberjack.Logger.
func (lumberjackSink) Sync() error { return nil }
func main() {
zap.RegisterSink("lumberjack", func(u *url.URL) (zap.Sink, error) {
return lumberjackSink{
Logger: &lumberjack.Logger{
Filename: u.Opaque,
// Use query parameters or hardcoded values for remaining
// fields.
},
}, nil
})
config := zap.NewProductionConfig()
// Add a URL with the "lumberjack" scheme.
config.OutputPaths = append(config.OutputPaths, "lumberjack:foo.log")
log, _ := config.Build()
log.Info("test", zap.String("foo", "bar"))
}
EDIT: Adrian's suggestion makes sense, so I moved my code into a function and called the function from my cobra block:
package cmd
import (
"fmt"
"log"
"os"
"io"
"github.com/spf13/cobra"
"github.com/spf13/viper"
input "github.com/tcnksm/go-input"
)
var configureCmd = &cobra.Command{
Use: "configure",
Short: "Configure your TFE credentials",
Long: `Prompts for your TFE API credentials, then writes them to
a configuration file (defaults to ~/.tgc.yaml`,
Run: func(cmd *cobra.Command, args []string) {
CreateConfigFileFromPrompts(os.Stdin, os.Stdout)
},
}
func CreateConfigFileFromPrompts(stdin io.Reader, stdout io.Writer) {
ui := &input.UI{
Writer: stdout,
Reader: stdin,
}
tfeURL, err := ui.Ask("TFE URL:", &input.Options{
Default: "https://app.terraform.io",
Required: true,
Loop: true,
})
if err != nil {
log.Fatal(err)
}
viper.Set("tfe_url", tfeURL)
tfeAPIToken, err := ui.Ask(fmt.Sprintf("TFE API Token (Create one at %s/app/settings/tokens)", tfeURL), &input.Options{
Default: "",
Required: true,
Loop: true,
Mask: true,
MaskDefault: true,
})
if err != nil {
log.Fatal(err)
}
viper.Set("tfe_api_token", tfeAPIToken)
configPath := ConfigPath()
viper.SetConfigFile(configPath)
err = viper.WriteConfig()
if err != nil {
log.Fatal("Failed to write to: ", configPath, " Error was: ", err)
}
fmt.Println("Saved to", configPath)
}
So what can I pass to this method to test that the output is as expected?
package cmd
import (
"strings"
"testing"
)
func TestCreateConfigFileFromPrompts(t *testing.T) {
// How do I pass the stdin and out to the method?
// Then how do I test their contents?
// CreateConfigFileFromPrompts()
}
func TestCreateConfigFileFromPrompts(t *testing.T) {
var in bytes.Buffer
var gotOut, wantOut bytes.Buffer
// The reader should read to the \n each of two times.
in.Write([]byte("example-url.com\nexampletoken\n"))
// wantOut could just be []byte, but for symmetry's sake I've used another buffer
wantOut.Write([]byte("TFE URL:TFE API Token (Create one at example-url.com/app/settings/tokens)"))
// I don't know enough about Viper to manage ConfigPath()
// but it seems youll have to do it here somehow.
configFilePath := "test/file/location"
CreateConfigFileFromPrompts(&in, &gotOut)
// verify that correct prompts were sent to the writer
if !bytes.Equal(gotOut.Bytes(), wantOut.Bytes()) {
t.Errorf("Prompts = %s, want %s", gotOut.Bytes(), wantOut.Bytes())
}
// May not need/want to test viper's writing of the config file here, or at all, but if so:
var fileGot, fileWant []byte
fileWant = []byte("Correct Config file contents:\n URL:example-url.com\nTOKEN:exampletoken")
fileGot, err := ioutil.ReadFile(configFilePath)
if err != nil {
t.Errorf("Error reading config file %s", configFilePath)
}
if !bytes.Equal(fileGot, fileWant) {
t.Errorf("ConfigFile: %s not created correctly got = %s, want %s", configFilePath, fileGot, fileWant)
}
}
As highlighted by #zdebra in comments to his answer, the go-input package is panicing and giving you the error: Reader must be a file. If you are married to using that package, you can avoid the problem by disabling the masking option on the ui.Ask for your second input:
tfeAPIToken, err := ui.Ask(fmt.Sprintf("TFE API Token (Create one at %s/app/settings/tokens)", tfeURL), &input.Options{
Default: "",
Required: true,
Loop: true,
//Mask: true, // if this is set to True, the input must be a file for some reason
//MaskDefault: true,
})
The reader and the writer need to be set up before the tested function is called. After is called, the result is written into the writer where it should be verified.
package cmd
import (
"strings"
"testing"
)
func TestCreateConfigFileFromPrompts(t *testing.T) {
in := strings.NewReader("<your input>") // you can use anything that satisfies io.Reader interface here
out := new(strings.Builder) // you could use anything that satisfies io.Writer interface here like bytes.Buffer
CreateConfigFileFromPrompts(in, out)
// here you verify the output written into the out
expectedOutput := "<your expected output>"
if out.String() != expectedOutput {
t.Errorf("expected %s to be equal to %s", out.String(), expectedOutput)
}
}