I am trying to use a global variable defined in parent package in the internal package. There are no errors but none of the logs from internal package are printed. My assumption on order of init is: parent.go global variables and then init() from internal package; which seems right when tested with fmt.Println(). But looks like the call to ReplaceGlobals is dropped during compilation. Is there a way to fix it so that I can use global variable from internal package?
Here is the zap package I am using: zap. The API in question is discussed here. Here is the code snippet.
~/parent/parent.go
package main
import (
"log"
"go.uber.org/zap"
"parent/internal"
)
var (
logger = GetMyLogger()
undo = zap.ReplaceGlobals(logger.Desugar())
)
func GetMyLogger() *zap.SugaredLogger {
xx, err := zap.NewProduction()
if err != nil {
log.Fatal("error")
}
sugar := xx.Sugar()
return sugar
}
func main() {
logger.Infof("logger in main")
internal.New()
}
Then in ~/parent/internal/internal.go
package internal
import (
"go.uber.org/zap"
)
var logger *zap.SugaredLogger
func init() {
logger = zap.L().Sugar()
logger.Infof("init() from internal: %v\n", logger)
logger = zap.S()
logger.Infof("init() from internal: %v\n", logger)
}
func New() {
logger.Infof("New() from internal")
}
Package initialization order is well defined, see Spec: Package initialization. Basically you can be sure that if you refer to a package, that package's initialization will be done first, recursively.
Since your internal package does not refer to your main, you have no guarantee if main is initialized before anything is executed from internal.
An easy solution in your case is to move the logger to your internal package, and have main refer / use that.
If you have more packages involved and you want to use the same logger from other packages, you may have a designated package holding the logger, and have everyone refer / use that.
Related
I have a init() function defined in "config/config.go"
config.go
package config
import(
log "github.com/sirupsen/logrus"
)
func init() {
log.SetReportCaller(true)
}
I have another go file called auth.go in auth package
package auth
import(
log "github.com/sirupsen/logrus"
)
func auth(username string, pwd string) {
//some auth code
log.Info("Auth success")
}
When log.Info() is called in auth.go the log prints as below
2018-11-09T16:38:27+05:30 auth/auth.go:36 level=info msg="Auth success"
What I am confused here is that, how "log" in auth.go is aware of the settings done in config.go. log.SetReportCaller() is in config.go but even when auth.go is logged it takes settings of log.SetReportCaller() done in config.go
Since log.SetReportCaller() is not set in auth.go expected log should be as below without showing line number of caller method.
2018-11-09T16:38:27+05:30 level=info msg="Auth success"
main.go
package main
import (
"path/to/auth"
log "github.com/sirupsen/logrus"
"net/http"
)
func main() {
r := server.Route()
log.Info("Listening on 8080")
http.Handle("/", r)
log.Fatal(http.ListenAndServe(":8080", nil))
}
auth/router.go
package auth
import (
"github.com/gorilla/mux"
"github.com/rs/cors"
"net/http"
)
func Route() http.Handler {
r := mux.NewRouter()
// UI handlers
r.HandleFunc("/", IndexPageHandler)
r.HandleFunc("/login", LoginHandler).Methods("POST")
handler := cors.Default().Handler(r)
return
}
auth/login.go
package auth
import (
"fmt"
"path/to/config"
log "github.com/sirupsen/logrus"
"net/http"
)
func LoginHandler(w http.ResponseWriter, r *http.Request) {
username := r.FormValue("username")
password := r.FormValue("password")
status, userName, mail, _ := auth(username, password)
//some code goes here
}
Kindly explain how this is happening
Check this diagram to understand how init() work: diagram
Initialization order are as follows,
If a package imports other packages, the imported packages are initialised first.
Package level variables are initialised then.
init function of current package is called next. A package can have multiple init functions (either in a single file or distributed across multiple files) and they are called in the order in which they are presented to the compiler.
You will found an example explaining this here.
I am suspecting in dependency tree, there is a common ancestor file that import both config package and auth package.
Here is official doc on initialization: Package initialization
UPD: As you have added respective codes, let's visualize what is happening here. Look at the picture bellow,
What happening:
Your main package start initialing. But it is has imported auth package. So to complete initialization, auth package must be initialized.
auth package start initializing. But it has imported config package. So to complete initialization, config package must be initialized.
config package complete initialization(log.SetReportCaller(true) is called).
auth package complete initialization.
main package complete initialization.
main() function starts executing...
The behavior you're asking about actually has nothing to do with how init() functions work. SetReportCaller sets a global variable in github.com/sirupsen/logrus, not in your config or auth packages. So it doesn't matter where you call that function or where you call log.Info et al; the setting affects all calls to logrus regardless of call origin.
I am currently trying to export all shared components from multiple services to one new package and then reimporting this package to the services.
I quickly came accross the following problem:
In my utility package I create a function:
package utils
import "github.com/rs/zerolog"
func CreateLogger(logLevel string, w io.Writer) Logger {
...
return zerolog.New(w).With().Timestamp().Logger()
}
So now this package references "github.com/rs/zerolog"
Now in my services I want to use this method like this:
import "github.com/rs/zerolog"
var (
logger zerolog.Logger
)
func New() {
logger = utils.CreateLogger("info", os.Stdout)
}
Now this also references "github.com/rs/zerolog", but throws the error:
cannot use utils.CreateLogger("info", os.Stdout) (type "MY_REPO_PATH/vendor/VENDORED_REPO_PATH/vendor/github.com/rs/zerolog".Logger) as type "MY_REPO_PATH/vendor/github.com/rs/zerolog".Logger in assignment
So I understand why this is happening and I found a solution by wrapping the zerologger in my utils into a struct like this:
type Logger struct {
Logger zerolog.Logger
}
Now this works, but it is inconvenient because I have to call logger.Logger.Info()... instead of logger.Info() in my code.
It looks to me like this is not the perfect solution to this problem and I would like to know if there is a better way to solve it.
Best Regards,
Jonathan
I'm new to Go and I've created an application which is working as expected.
My application structure is like following:
myproj
Gopkg.toml
Gopkg.lock
src
server
main.go
utils
file1.go
logger.go
handler
handler1.go
handler2.go
Now inside the main.go file I've created a logger like the following:
File server-> main.go
import (
"handler"
"utils"
"github.com/sirupsen/logrus"
)
var logger *logrus.Logger
fun init(){
logger = utils.InitLogs()
}
func main(){
logger.info("my message")
…
handler.run()
}
Everything is working as expected!
Now I want to use the logger inside the handler1&2 files (from diff package inside my local project)
To do this, I did the following steps.
Inside the handler init the logger (exactly as I did in the main file) and it’s working
File handler-> handler1.go
import (
"utils"
"github.com/sirupsen/logrus"
)
var logger *logrus.Logger
fun init(){
//Init the logger again
logger = utils.InitLogs()
}
func run(){
//here Im not using logger otherwise maybe I can move it as parameter…
}
func build(){
//Here Im using the logger
logger.info("Hi")
}
While this is working but I've created two instance of logger, first on main and second on handler1 which Im not sure is best
My questions are:
I want to reuse the logger inside the handler class/module, is a better way to do it in Go?
This is diff package (inside my local proj) and I'm not sure that I'm that if Im not re-use the same logger object from the main method, this is the right way to do it...
(lower prio question) Is my project structure okay for Go? I used this as reference which is similar...also this which is a bit different
If you want to share your logger instance across your project, one way is to just export it as a global variable from your utils package as Armin suggested in his answer:
package utils
...
var Logger *logrus.Logger
func init() {
Logger = InitLogs()
}
I'd bet you can then get away with not having to export InitLogs() by changing it to initLogs()
Then, elsewhere in your code, you can import utils and use that logger instance:
import "utils"
...
func something() {
utils.Logger.Info("Hi")
}
Alternatively, if it makes sense to keep all your configuration in one place, you can declare your logger pointer as a field of a config struct and initialize it along with the rest of your program configuration (if you have any).
For example, say you have the following in your utils package:
package utils
...
type MyAppConfig struct {
// whatever config parameters your app needs,
// like DB connections, etc.
Logger *logrus.Logger
}
// pass in whatever configuration parameters you need,
// like DB URL, etc.
func InitConfig() (*MyAppConfig, error) {
// set up other configuration
config := &MyAppConfig{
// other config
Logger: InitLogs(),
}
return config, nil
}
func (c *MyAppConfig) DoSomethingImportant() {
c.Logger.Info("Hello")
}
Meanwhile, you can use this anywhere else, like in your CLI interface:
package main
import "utils"
...
func main() {
// input, or CLI parameters...
// pass in other CLI parameters, if any
// (of course you'd have to change the function signature in
// the previous file above)
config, err := utils.NewConfig()
if err != nil {
// handle error
}
config.DoSomethingImportant()
// or since Logger is exported:
config.Logger.Info("hello")
}
If you find you end up with lots of global things like the Logger you need to configure, the config struct is the more scalable approach IMO. As an added bonus, using a config type makes it easier to do dependency injection in unit tests than having the global variables used directly.
On the other hand, if you only have to worry about the one global Logger, then the config struct might be overkill, especially if it's not something you'll need to instrument when designing tests. It's a subjective call depending on your circumstances.
To answer your second question...
Project Layout
If /server has a main package with a command, move it under cmd/server.
Also, the way you currently have it, it would be a bit confusing for others to depend on your project. Since your repository appears to begin at src/, here's what it will look like when someone else tries to import your utils package:
File structure:
<top of someone else's GOPATH>/
src/
myproject/src/utils/
...
To import:
import "myproject/src/utils"
... and actually, since you import "utils" in your code right now, I don't think your code will compile in someone else's GOPATH because they don't have a $GOPATH/src/utils.
Solution: Your repository should not include src/. The idea is that you setup your $GOPATH and you put different repositories/packages inside of it. For example (assuming you host your code on Github):
<top of gopath>/
src/
github.com/you/myproject/ <---- your repo starts here
cmd/server/
main.go
utils
file1.go
logger.go
handler
handler1.go
handler2.go
This would make your packages importable to you and to others like so:
import "github.com/you/myproject/utils"
Also, this project structure allow others to include your project inside their $GOPATH or in their vendor/ interchangeably.
It seems you want to use the exact logger from main.go. You should just export it and use it in other packages:
server/main.go
var Logger *logrus.Logger // Notice capital L
func main() {
Logger = utils.InitLogs() // Init logger once
...
}
handler/handler1.go
import "server"
func run(){
//here Im not using logger otherwise maybe I can move it as parameter…
}
func build(){
// No need to initialize logger as its done in main func.
//Here Im using the logger
server.Logger.info("in xyz")
}
And your project structure is not standard. See this.
I am using logrus in a Go app. I believe this question is applicable to any other logging package (which doesn't offer external file based configuration) as well.
logrus provides functions to setup various configuration, e.g. SetOutput, SetLevel etc.
Like any other application I need to do logging from multiple source files/packages, it seems you need to setup these options in each file with logrus.
Is there any way to setup these options once somewhere in a central place to be shared all over the application. That way if I have to make logging level change I can do it in one place and applies to all the components of the app.
You don't need to set these options in each file with Logrus.
You can import Logrus as log:
import log "github.com/Sirupsen/logrus"
Then functions like log.SetOutput() are just functions and modify the global logger and apply to any file that includes this import.
You can create a package global log variable:
var log = logrus.New()
Then functions like log.SetOutput() are methods and modify your package global. This is awkward IMO if you have multiple packages in your program, because each of them has a different logger with different settings (but maybe that's good for some use cases). I also don't like this approach because it confuses goimports (which will want to insert log into your imports list).
Or you can create your own wrapper (which is what I do). I have my own log package with its own logger var:
var logger = logrus.New()
Then I make top-level functions to wrap Logrus:
func Info(args ...interface{}) {
logger.Info(args...)
}
func Debug(args ...interface{}) {
logger.Debug(args...)
}
This is slightly tedious, but allows me to add functions specific to my program:
func WithConn(conn net.Conn) *logrus.Entry {
var addr string = "unknown"
if conn != nil {
addr = conn.RemoteAddr().String()
}
return logger.WithField("addr", addr)
}
func WithRequest(req *http.Request) *logrus.Entry {
return logger.WithFields(RequestFields(req))
}
So I can then do things like:
log.WithConn(c).Info("Connected")
(I plan in the future to wrap logrus.Entry into my own type so that I can chain these better; currently I can't call log.WithConn(c).WithRequest(r).Error(...) because I can't add WithRequest() to logrus.Entry.)
This is the solution that I arrived at for my application that allows adding of fields to the logging context. It does have a small performance impact due to the copying of context base fields.
package logging
import (
log "github.com/Sirupsen/logrus"
)
func NewContextLogger(c log.Fields) func(f log.Fields) *log.Entry {
return func(f log.Fields) *log.Entry {
for k, v := range c {
f[k] = v
}
return log.WithFields(f)
}
}
package main
import (
"logging"
)
func main {
app.Logger = logging.NewContextLogger(log.Fields{
"module": "app",
"id": event.Id,
})
app.Logger(log.Fields{
"startTime": event.StartTime,
"endTime": event.EndTime,
"title": event.Name,
}).Info("Starting process")
}
We parse flags in main.go which is in main package, of course. Then we have another package where we want to read some flag's value.
flags.Args() work fine, it will return all non-flag values.
But I cannot figure out to how read already parsed value for a flag in a package other than main.
Is it possible?
Thanks
Amer
I had the same requirement recently and I wanted a solution that avoided calling flag.Parse repeatedly in init functions.
Perusing the flag package I found Lookup(name string) which returns a Flag which has a Value. Every built in Value implements the flag.Getter interface. The call chain looks like this:
flag.Lookup("httplog").Value.(flag.Getter).Get().(bool)
If you mistype the flag name or use the wrong type you get a runtime error. I wrapped the lookup in a function that I call directly where needed since the lookup and get methods are fast and the function is not called often. So the main package declares the flag.
// main.go
package main
import "flag"
var httplog = flag.Bool("httplog", false, "Log every HTTP request and response.")
func main() {
flag.Parse()
// ...
}
And the utility package, which is decoupled from main except for the flag name, reads the flag value.
// httpdiag.go
package utility
import "flag"
func logging() bool {
return flag.Lookup("httplog").Value.(flag.Getter).Get().(bool)
}
You can define the var storing the flag in the separate package, as an exported variable, then call the flag parsing in the main package to use that variable, like this:
mypackage/const.go
var (
MyExportedVar string
)
mainpackage/main.go
func init() {
flag.StringVar(&mypackage.MyExportedVar, "flagName", "defaultValue", "usage")
flag.Parse()
}
This way, everybody can access that flag, including that package itself.
Note:
this only works for exported variables.
You can define the flag in that package and call flag.Parse() in func init(), flag.Parse can be called multiple times.
However, if you want to access the flag's value from multiple packages you have to expose it or create an exposed function to check it.
for example:
// pkgA
var A = flag.Bool("a", false, "why a?")
func init() {
flag.Parse()
}
// main package
func main() {
flag.Parse()
if *pkgA.A {
// stuff
}
}
Also you can use FlagSet.Parse(os.Args) if you want to reparse the args.
When you parse your flags, parse them into global variables which start with an initial Capital so they are public, eg
package main
var Species = flag.String("species", "gopher", "the species we are studying")
func main() {
flag.Parse()
}
Then in your other package you can refer to them as
package other
import "/path/to/package/main"
func Whatever() {
fmt.Println(main.Species)
}