Below is the signature of the WithValue func.
func WithValue(parent Context, key, val interface{}) Context
I took a look at Go official documentation but they didn't mention how a value would get marshalled when the context is passed over to a remote service.
For example, I can pass in anything as val such as a complex struct containing exported and un-exported fields. I wonder if my un-exported fields would ever reach the remote service. If they do, how does it work behind the scene? The whole context object gets marshalled into []byte regardless of fields?
One concrete sample of a complex object is the internal implementation of a Span in opentracing.
// Implements the `Span` interface. Created via tracerImpl (see
// `basictracer.New()`).
type spanImpl struct {
tracer *tracerImpl
event func(SpanEvent)
sync.Mutex // protects the fields below
raw RawSpan
// The number of logs dropped because of MaxLogsPerSpan.
numDroppedLogs int
}
The above Span can be passed around from service to service by creating a derived context using the func below.
// ContextWithSpan returns a new `context.Context` that holds a reference to
// the span. If span is nil, a new context without an active span is returned.
func ContextWithSpan(ctx context.Context, span Span) context.Context {
if span != nil {
if tracerWithHook, ok := span.Tracer().(TracerContextWithSpanExtension); ok {
ctx = tracerWithHook.ContextWithSpanHook(ctx, span)
}
}
return context.WithValue(ctx, activeSpanKey, span)
}
What is the high-level protocols/rules that Go uses to pass around complex values stored in Go context? Would the func(SpanEvent) and the mutex really travel over the wire to the next service?
I'd be very grateful if you could explain the expected behavior or point me in the direction of some articles I've not found.
I'm using github.com/google/wire for dependency injection in an open source example project that I'm working on.
I have the following interfaces in a package named interfaces:
type LoginService interface {
Login(email, password) (*LoginResult, error)
}
type JWTService interface {
Generate(user *models.User) (*JWTGenerateResult, error)
Validate(tokenString string) (*JWTValidateResult, error)
}
type UserDao interface {
ByEmail(email string) (*models.User, error)
}
I have implementations that look like this:
type LoginServiceImpl struct {
jwt interfaces.JWTService
dao interfaces.UserDao
logger *zap.Logger
}
func NewLoginService(jwt interfaces.JWTService, dao interfaces.UserDao, \
logger *zap.Logger) *LoginServiceImpl {
return &LoginServiceImpl{jwt: jwt, dao: dao, logger: logger }
}
type JWTServiceImpl struct {
key [32]byte
logger *zap.Logger
}
func NewJWTService(key [32]byte, logger *zap.Logger) (*JWTServiceImpl, error) {
r := JWTServiceImpl {
key: key,
logger: logger,
}
if !r.safe() {
return nil, fmt.Errorf("unable to create JWT service, unsafe key: %s", err)
}
return &r, nil
}
type UserDaoImpl struct {
db: *gorm.DB
logger: *zap.Logger
}
func NewUserDao(db *gorm.DB, logger *zap.Logger) *UserDao {
return &UserDaoImpl{ db: db, logger: logger }
}
I'll exclude other factory functions and implementations here because they all look very similar. They may return an error or be infallible.
I have one other interesting factory for creating the database connection, which I'll just show the interface and not the implementation:
func Connect(config interfaces.MySQLConfig) (*gorm.DB, error) { /* ... */ }
Now, onto the problem. In my command-line entry-point, I'm creating a logger:
logger, err := zap.NewDevelopment()
For each of the factory methods above, I need to provide a logger and not the same logger instance, rather as if these methods were called as follows:
logger, err := zap.NewDevelopment()
// check err
db, err := database.Connect(config)
// check err
userDao := dao.NewUserDao(db, logger.Named("dao.user"))
jwtService, err := service.NewJWTService(jwtKey)
// check err
loginService := service.NewLoginService(jwtService, userDao, logger.Named("service.login"))
My wire.ProviderSet construction looks like this:
wire.NewSet(
wire.Bind(new(interfaces.LoginService), new(*service.LoginServiceImpl)),
wire.Bind(new(interfaces.JWTService), new(*service.JWTServiceImpl)),
wire.Bind(new(interfaces.UserDao), new(*dao.UserDaoImpl)),
service.NewLoginService,
service.NewJWTService,
dao.NewUserDao,
database.Connect,
)
I've read through the user guide, the tutorial, and best practices, and I can't seem to find a way to route a unique zap.Logger to each of these factory methods, and routing a random [32]byte for the JWT service.
Since my root logger is not created at compile time, and since each of these factory methods needs its own unique logger, how do I tell wire to bind these instances to the corresponding factory methods? I'm having a tough time seeing how to route custom instances of the same type to disparate factory methods.
In summary:
Wire seems to favor doing everything at compile-time, storing the dependency injection configuration in a static package-level variable. For most of my use-case, this is okay.
For the rest of my use-case, I need to create some instances manually before running the dependency injection and the ability to route various *zap.Logger instances to each service that needs it.
Essentially, I need to have wire do services.NewUserDao(Connect(mysqlConfig), logger.Named("dao.user"), but I don't know how to express this in wire and merge variables at runtime with wire's compile-time approach.
How do I do this in wire?
I had to change what I was doing somewhat, as is recommended in the documentation:
If you need to inject a common type like string, create a new string type to avoid conflicts with other providers. For example:
type MySQLConnectionString string
Adding Custom Types
The documentation is admittedly very terse, but what I ended up doing is creating a bunch of types:
type JWTKey [32]byte
type JWTServiceLogger *zap.Logger
type LoginServiceLogger *zap.Logger
type UserDaoLogger *zap.Logger
Updating Producer Functions
I updated my producer methods to accept these types, but did not have to update my structs:
// LoginServiceImpl implements interfaces.LoginService
var _ interfaces.LoginService = (*LoginServiceImpl)(nil)
type LoginServiceImpl struct {
dao interfaces.UserDao
jwt interfaces.JWTService
logger *zap.Logger
}
func NewLoginService(dao interfaces.UserDao, jwt interfaces.JWTService,
logger LoginServiceLogger) *LoginServiceImpl {
return &LoginServiceImpl {
dao: dao,
jwt: jwt,
logger: logger,
}
}
This above part made sense; giving distinct types meant that wire had less to figure out.
Creating an Injector
Next, I had to create the dummy injector and then use wire to generate the corresponding wire_gen.go. This was not easy and very unintuitive. When following the documentation, things kept breaking and giving me very unhelpful error messages.
I have a cmd/ package and my CLI entrypoint lives in cmd/serve/root.go, which is run as ./api serve from the command-line. I created my injector function in cmd/serve/injectors.go, note that // +build wireinject and the following newline are required to inform Go that this file is used for code generation and not code itself.
I ultimately arrived at the following code after much trial and error:
// +build wireinject
package serve
import /*...*/
func initializeLoginService(
config interfaces.MySQLConfig,
jwtKey service.JWTKey,
loginServiceLogger service.LoginServiceLogger,
jwtServiceLogger service.JWTServiceLogger,
userDaoLogger service.UserDaoLogger,
databaseLogger database.DatabaseLogger,
) (interfaces.LoginService, error) {
wire.Build(
// bind interfaces to implementations
wire.Bind(new(interfaces.LoginService), new(*service.LoginServiceImpl)),
wire.Bind(new(interfaces.JWTService), new(*service.JWTServiceImpl)),
wire.Bind(new(interfaces.UserDao), new(*dao.UserDao)),
// services
service.NewLoginService,
service.NewJWTService,
// daos
dao.NewUserDao,
// database
database.Connect,
)
return nil, nil
}
The wire.Bind calls inform wire which implementation to use for a given interface so it will know that service.NewLoginService which returns a *LoginServiceImpl should be used as the interfaces.LoginService.
The rest of the entities in the call to wire.Build are just factory functions.
Passing Values to an Injector
One of the the issues I ran into was that I was trying to pass values into wire.Build like the documentation describes:
Occasionally, it is useful to bind a basic value (usually nil) to a type. Instead of having injectors depend on a throwaway provider function, you can add a value expression to a provider set.
type Foo struct {
X int
}
func injectFoo() Foo {
wire.Build(wire.Value(Foo{X: 42}))
return Foo{}
}
...
It's important to note that the expression will be copied to the injector's package; references to variables will be evaluated during the injector package's initialization. Wire will emit an error if the expression calls any functions or receives from any channels.
This is what confused me; it sounded like you could only really use constant values when trying to run an injector, but there are two lines in the docs in the "injectors" section:
Like providers, injectors can be parameterized on inputs (which then get sent to providers) and can return errors. Arguments to wire.Build are the same as wire.NewSet: they form a provider set. This is the provider set that gets used during code generation for that injector.
These lines are accompanied by this code:
func initializeBaz(ctx context.Context) (foobarbaz.Baz, error) {
wire.Build(foobarbaz.MegaSet)
return foobarbaz.Baz{}, nil
}
This is what I missed and what caused me to lose a lot of time on this. context.Context doesn't seem to be passed anywhere in this code, and it's a common type so I just kind of shrugged it off and didn't learn from it.
I defined my injector function to take arguments for the JWT key, the MySQL config, and the logger types:
func initializeLoginService(
config interfaces.MySQLConfig,
jwtKey service.JWTKey,
loginServiceLogger service.LoginServiceLogger,
jwtServiceLogger service.JWTServiceLogger,
userDaoLogger service.UserDaoLogger,
databaseLogger database.DatabaseLogger,
) (interfaces.LoginService, error) {
// ...
return nil, nil
}
Then, I attempted to inject them into wire.Build:
wire.Build(
// ...
wire.Value(config),
wire.Value(jwtKey),
wire.Value(loginServiceLogger),
// ...
)
When I attempted to run wire, it complained that these types were defined twice. I was very confused by this behavior, but ultimately learned that wire automatically sends all function parameters into wire.Build.
Once again: wire automatically sends all injector function parameters into wire.Build.
This was not intuitive to me, but I learned the hard way that it's the way wire works.
Summary
wire does not provide a way for it to distinguish values of the same type within its dependency injection system. Thus, you need to wrap these simple types with type definitions to let wire know how to route them, so instead of [32]byte, type JWTKey [32]byte.
To inject live values into your wire.Build call, simply change your injector function signature to include those values in the function parameters and wire will automatically inject them into wire.Build.
Run cd pkg/my/package && wire to create wire_gen.go in that directory for your defined injectors. Once this is done, future calls to go generate will automatically update wire_gen.go as changes occur.
I have wire_gen.go files checked-in to my version control system (VCS) which is Git, which feels weird due to these being generated build artifacts, but this seems to be the way that this is typically done. It might be more advantageous to exclude wire_gen.go, but if you do this, you'll need to find every package which includes a file with a // +build wireinject header, run wire in that directory, and then go generate just to be sure.
Hopefully this clears up the way that wire works with actual values: make them type safe with type wrappers, and simply pass them to your injector function, and wire does the rest.
I am using echo to build a web server in go. What is the best way to pass objects around middlewares and a handler apart from using context.Set method?
For example, let's consider the following case:
func mw(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Conext) error {
var value TypeX = // load value of type TypeX based on query parameters
c.Set("key", value)
return next(c)
}
}
func h1(c echo.context) error {
value := c.Get("key").(TypeX)
return c.JSON(http.StatusOk, value.H1())
}
func h2(c echo.context) error {
value := c.Get("key").(TypeX)
return c.JSON(http.StatusOk, value.H2())
}
func registerRoute(e *echo.Echo) {
e.Get("/test", h)
}
Is there any way to get rid of get/set methods? It seems like an unclean way of doing this operation. I am open to complete refactoring, creating new structures/interfaces to make this happen.
To take this example further, lets say my apis can fall into bucket1, bucket2, bucket3. Value would call the method b1, b2, b3 based on which bucked it falls in. These buckets can be identified by adding them as middleware mwB1, mwB2, mwB3 each of which calls the respective method. Hence, all my apis would first call the middleware mw; then one of mwB1, mwB2, mwB3 and finally the actual handler, something like, e.Get("/test", h, mw, mwB2). I don't want to load value everywhere and get/set seems unclean to me (unless that is the standard way of doing this).
I'm writing some code that uses a library called Vault. In this library we have a Client. My code makes use of this Client but I want to be able to easily test the code that uses it. I use only a couple methods from the library so I ended up creating an interface:
type VaultClient interface {
Logical() *api.Logical
SetToken(v string)
NewLifetimeWatcher(i *api.LifetimeWatcherInput) (*api.LifetimeWatcher, error)
}
Now if my code is pointed at this interface everything is easily testable.. Except let's look at the Logical() method. It returns a struct here. My issue is that this Logical struct also has methods on it that allow you to Read, Write, ex:
func (c *Logical) Read(path string) (*Secret, error) {
return c.ReadWithData(path, nil)
}
and these are being used in my project as well to do something like:
{{ VaultClient defined above }}.Logical().Write("something", something)
Here is the issue. The Logical returned from the call to .Logical() has a .Write and .Read method that I can't reach to mock. I don't want all the logic within those methods to run in my tests.
Ideally I'd like to be able to do something similar to what I did above and create an interface for Logical as well. I'm relatively new to Golang, but I'm struggling with the best approach here. From what I can tell that's not possible. Embedding doesn't work like inheritance so it seems like I have to return a Logical. That leaves my code unable to be tested as simply as I would like because all the logic within a Logical's methods can't be mocked.
I'm sort of at a loss here. I have scoured Google for an answer to this but nobody ever talks about this scenario. They only go as far as I went with the initial interface for the client.
Is this a common scenario? Other libraries I've used don't return structs like Logical. Instead they typically just return a bland struct that holds data and has no methods.
package usecasevaultclient
// usecase.go
type VaultClient interface {
Logical() *api.Logical
SetToken(v string)
NewLifetimeWatcher(i *api.LifetimeWatcherInput) (*api.LifetimeWatcher, error)
}
type vaultClient struct {
repo RepoVaultClient
}
// create new injection
func NewVaultClient(repo RepoVaultClient) VaultClient {
return &vaultClient{repo}
}
func(u *vaultClient) Logical() *api.Logical {
// do your logic and call the repo of
u.repo.ReadData()
u.repo.WriteData()
}
func(u *vaultClient) SetToken(v string) {}
func(u *vaultClient) NewLifetimeWatcher(i *api.LifetimeWatcherInput) (*api.LifetimeWatcher, error)
// interfaces.go
type RepoVaultClient interface {
ReadData() error
WriteData() error
}
// repo_vaultclient_mock.go
import "github.com/stretchr/testify/mock"
type MockRepoVaultClient struct {
mock.Mock
}
func (m *MockRepoVaultClient) ReadData() error {
args := m.Called()
return args.Error(0)
}
func (m *MockRepoVaultClient) WriteData() error {
args := m.Called()
return args.Error(0)
}
// vaultClient_test.go
func TestLogicalShouldBeSuccess(t *testing.T) {
mockRepoVaultClient = &MockRepoVaultClient{}
useCase := NewVaultClient(mockRepoVaultClient)
mockRepoVaultClient.On("ReadData").Return(nil)
mockRepoVaultClient.On("WriteData").Return(nil)
// your logics gonna make this response as actual what u implemented
response := useCase.Logical()
assert.Equal(t, expected, response)
}
if you want to test the interface of Logical you need to mock the ReadData and WriteData , with testify/mock so u can defined the respond of return of those methods and you can compare it after you called the new injection of your interface
I want to create an interface to make it easy to add new storage backends.
package main
// Storage is an interface to describe storage backends
type Storage interface {
New() (newStorage Storage)
}
// File is a type of storage that satisfies the interface Storage
type File struct {
}
// New returns a new File
func (File) New() (newFile Storage) {
newFile = File{}
return newFile
}
// S3 is a type of storage that satisfies the interface Storage
type S3 struct {
}
// New returns a new S3
func (S3) New() (newS3 S3) {
newS3 = S3{}
return newS3
}
func main() {
// List of backends to choose from
var myStorage map[string]Storage
myStorage["file"] = File{}
myStorage["s3"] = S3{}
// Using one of the backends on demand
myStorage["file"].New()
myStorage["s3"].New()
}
But it seems not possible to define and satisfy a function that should return an object that satisfies the interface itself as well.
File.New() returns an object of type Storage which satisfies Storage.
S3.New() returns an object of type S3.
S3 should satisfies the interface Storage as well but I get this:
./main.go:32: cannot use S3 literal (type S3) as type Storage in assignment:
S3 does not implement Storage (wrong type for New method)
have New() S3
want New() Storage
What am I doing wrong?
I hope I am missing something basic here.
This code does not make sense at all. You are either implementing a factory pattern which is tied to a struct which is of the type the factory is going to produce or you are reinventing the wheel in a wrong way by reimplementing the already existing new keyword and tie it to a struct which is nil the time you would use it.
You can either get rid of the helper function and simply use
s := new(S3)
f := new (File)
Or you could use a static Factory function like:
// Do NOT tie your Factory to your type
function New() S3 {
return S3{}
}
Or, which seems to better suit your use case, create a factory interface, implement it and have its New() function return a Storage instance:
type StorageFactory interface {
New() Storage
}
type S3Factory struct {}
function (f *S3Factory) New() Storage {
return S3{}
}
There are various ways of registering your factory. You could use a global var and init
import "example.com/foo/storage/s3"
type FactoryGetter func() StorageFactory
type FactoryRegistry map[string] FactoryGetter
// Registry will be updated by an init function in the storage provider packages
var Registry FactoryRegistry
func init(){
Registry = make(map[string] FactoryGetter)
}
// For the sake of shortness, a const. Make it abflag, for example
const storageProvider = "s3"
func main(){
f := Registry[storageProvider]()
s := f.New()
s.List()
}
And somewhere in the S3 package
func init() {
Registry["s3"] = function(){ return S3Factory{}}
}
You could even think of making the Factories taking params.
I like what you're doing here and I've actually worked on projects that involved very similar design challenges, so I hope my suggestions can help you out some.
In order to satisfy the interface, you'd need to update your code from...
// New returns a new S3
func (S3) New() (newS3 S3) {
newS3 = S3{}
return newS3
}
to this
// New returns a new S3
func (S3) New() (newS3 Storage) {
newS3 = S3{}
return newS3
}
This means you will receive an instance of Storage back, so to speak. If you want to then access anything from S3 without having to use type assertion, it would be best to expose that S3 function/method in the interface.
So let's say you want a way to List your objects in your S3 client. A good approach to supporting this would be to update Storage interface to include List, and update S3 so it has its own implementation of List:
// Storage is an interface to describe storage backends
type Storage interface {
New() (newStorage Storage)
List() ([]entry, error) // or however you would prefer to trigger List
}
...
// New returns a new S3
func (S3) List() ([] entry, error) {
// initialize "entry" slice
// do work, looping through pages or something
// return entry slice and error if one exists
}
When it comes time to add support for Google Cloud Storage, Rackspace Cloud Files, Backblaze B2, or any other object storage provider, each of them will also need to implement List() ([] entry, error) as well - which is good! Once you've used this List function in the way you need, adding more clients/providers will be more like developing plugins than actually writing/architecting code (since your design is complete by that point).
The real key with satisfying interfaces is to have the signature match exactly and think of interfaces as a list of common functions/methods that you'd want every storage provider type to handle in order to meet your goals.
If you have any questions or if anything I've written is unclear, please comment and I'll be happy to clarify or adjust my post :)