Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 4 years ago.
Improve this question
i am confusing that, is below snippet perfect?
import "sync"
import "sync/atomic"
var initialized uint32
var instance *singleton
var instance *singleton
var once sync.Once
func GetInstance() *singleton {
once.Do(func() {
instance = &singleton{}
})
return instance
}
atomic.StoreUint32(&initialized, 1) will flush instance to all CPUs?
i think i need add an atomic store and load for instance, like below snippet
var instance *singleton
var once sync.Once
func GetInstance() *singleton {
once.Do(func() {
atomic.StorePointer(&instance, &singleton{})
})
return atomic.LoadPointer(&instance)
}
i think Once.Do is only guarantee execute function f one time.
and atomic.StoreUint32(&o.done, 1) is only memory barrier for o.done.
it doesn't ensure instance is global visible
func (o *Once) Do(f func()) {
if atomic.LoadUint32(&o.done) == 1 {
return
}
// Slow-path.
o.m.Lock()
defer o.m.Unlock()
if o.done == 0 {
defer atomic.StoreUint32(&o.done, 1)
f()
}
}
Lets break down your question to two pieces:
Singletons
Atomics and the Go memory model
Singletons
Go has package level variables. These are instantiated before anything has the chance to get moving, therefore if you good with these things being created as soon as the package is used, you get a singleton for free.
package somepack
var(
connection = createConn()
)
func Connection() SomeConnection {
return connection
}
connection will be created once and therefore Connection() will return the same instance of it safely.
Sometimes developers reach for a singleton when they want "lazy" instantiation. This is a good idea if the resource is expensive to create and not always needed. This is where sync.Once is useful.
var (
connection SomeConnection // Not instantiated
connectionOnce sync.Once
)
func Connection() SomeConnection {
connectionOnce.Do(func(){
connection = createConn()
})
return connection
}
Notice I'm not doing anything special with the assignment (e.g., atomic.Store()). This is because sync.Once takes care of all the locking required for this to be safe.
Atomics and the Go memory model
A good resource to start with is the published docs for this: The Go Memory Model
Your concern of "flushing" to the different CPUs is valid (despite some of the comments) because each CPU has its own cache with its own state. C++ (among other languages like Rust) developers tend to care about this because they get to. Go developers don't get to care AS much because Go only has "happens before". Rust in fact has some nice docs on it.
That being said, you normally don't need to worry about it. A mutex (and sync.Once) will force the state of the memory on each CPU to be what you would expect.
Related
I am newer to golang, so I have some courses that I bought from udemy to help break me into the language. One of them I found very helpful for a general understanding as I took on a project in the language.
In the class that I took, all of the sql related functions were in the sqlc folder with the structure less broken out:
sqlc
generatedcode
store
One of those files is a querier that is generated by sqlc that contains an interface with all of the methods that were generated. Here is the general idea of what it currently looks like: https://github.com/techschool/simplebank/tree/master/db/sqlc
package db
import (
"context"
"github.com/google/uuid"
)
type Querier interface {
AddAccountBalance(ctx context.Context, arg AddAccountBalanceParams) (Account, error)
CreateAccount(ctx context.Context, arg CreateAccountParams) (Account, error)
...
}
var _ Querier = (*Queries)(nil)
Would it be possible to wrap both what sqlc generates AND any queries that a developer creates (dynamic queries) into a single querier? I'm also trying to have it so that the sqlc generated code is in its own folder. The structure I am aiming for is:
sql
sqlc
generatedcode
store - (wraps it all together)
dynamicsqlfiles
This should clear up what a I mean by store: https://github.com/techschool/simplebank/blob/master/db/sqlc/store.go
package db
import (
"context"
"database/sql"
"fmt"
)
// Store defines all functions to execute db queries and transactions
type Store interface {
Querier
TransferTx(ctx context.Context, arg TransferTxParams) (TransferTxResult, error)
}
// SQLStore provides all functions to execute SQL queries and transactions
type SQLStore struct {
db *sql.DB
*Queries
}
// NewStore creates a new store
func NewStore(db *sql.DB) Store {
return &SQLStore{
db: db,
Queries: New(db),
}
}
I'm trying to run everything through that store (both generated and my functions), so I can make a call similar to the CreateUser function in this file (server.store.): https://github.com/techschool/simplebank/blob/master/api/user.go
arg := db.CreateUserParams{
Username: req.Username,
HashedPassword: hashedPassword,
FullName: req.FullName,
Email: req.Email,
}
user, err := server.store.CreateUser(ctx, arg)
if err != nil {
if pqErr, ok := err.(*pq.Error); ok {
switch pqErr.Code.Name() {
case "unique_violation":
ctx.JSON(http.StatusForbidden, errorResponse(err))
return
}
}
ctx.JSON(http.StatusInternalServerError, errorResponse(err))
return
}
I've tried creating something that houses another querier interface that embeds the generated one, then creating my own db.go that uses the generated DBTX interface but has its own Queries struct, and New function. It always gives me an error that the Queries struct I created aren't implementing the functions I made, despite having it implemented in one of the custom methods I made.
I deleted that branch, and have been clicking through the simplebank project linked above to see if I can find another way this could be done, or if I missed something. If it can't be done, that's okay. I'm just using this as a good opportunity to learn a little more about the language, and keep some code separated if possible.
UPDATE:
There were only a few pieces I had to change, but I modified the store.go to look more like:
// sdb is imported, but points to the generated Querier
// Store provides all functions to execute db queries and transactions
type Store interface {
sdb.Querier
DynamicQuerier
}
// SQLStore provides all functions to execute SQL queries and transactions
type SQLStore struct {
db *sql.DB
*sdb.Queries
*dynamicQueries
}
// NewStore creates a new Store
func NewStore(db *sql.DB) Store {
return &SQLStore{
db: db,
Queries: sdb.New(db),
dynamicQueries: New(db),
}
}
Then just created a new Querier and struct for the methods I would be creating. Gave them their own New function, and tied it together in the above. Before, I was trying to figure out a way to reuse as much of the generated code as possible, which I think was the issue.
Why I wanted the Interface:
I wanted a structure that separated the files I would be working in more from the files that I would never touch (generated). This is the new structure:
I like how the generated code put everything in the Querier interface, then checked that anything implementing it satisfied all of the function requirements. So I wanted to replicate that for the dynamic portion which I would be creating on my own.
It might be complicating it a bit more than it would 'NEED' to be, but it also provides an additional set of error checking that is nice to have. And in this case, even while maybe not necessary, it ended up being doable.
Would it be possible to wrap both what sqlc generates AND any queries that a developer creates (dynamic queries) into a single querier?
If I'm understanding your question correctly I think that you are looking for something like the below (playground):
package main
import (
"context"
"database/sql"
)
// Sample SQL C Code
type DBTX interface {
ExecContext(context.Context, string, ...interface{}) (sql.Result, error)
PrepareContext(context.Context, string) (*sql.Stmt, error)
QueryContext(context.Context, string, ...interface{}) (*sql.Rows, error)
QueryRowContext(context.Context, string, ...interface{}) *sql.Row
}
type Queries struct {
db DBTX
}
func (q *Queries) DeleteAccount(ctx context.Context, id int64) error {
// _, err := q.db.ExecContext(ctx, deleteAccount, id)
// return err
return nil // Pretend that this always works
}
type Querier interface {
DeleteAccount(ctx context.Context, id int64) error
}
//
// Your custom "dynamic" queries
//
type myDynamicQueries struct {
db DBTX
}
func (m *myDynamicQueries) GetDynamicResult(ctx context.Context) error {
// _, err := q.db.ExecContext(ctx, deleteAccount, id)
// return err
return nil // Pretend that this always works
}
type myDynamicQuerier interface {
GetDynamicResult(ctx context.Context) error
}
// Combine things
type allDatabase struct {
*Queries // Note: You could embed this directly into myDynamicQueries instead of having a seperate struct if that is your preference
*myDynamicQueries
}
type DatabaseFunctions interface {
Querier
myDynamicQuerier
}
func main() {
// Basic example
var db DatabaseFunctions
db = getDatabase()
db.DeleteAccount(context.Background(), 0)
db.GetDynamicResult(context.Background())
}
// getDatabase - Perform whatever is needed to connect to database...
func getDatabase() allDatabase {
sqlc := &Queries{db: nil} // In reality you would use New() to do this!
myDyn := &myDynamicQueries{db: nil} // Again it's often cleaner to use a function
return allDatabase{Queries: sqlc, myDynamicQueries: myDyn}
}
The above is all in one file for simplicity but could easily pull from multiple packages e.g.
type allDatabase struct {
*generatedcode.Queries
*store.myDynamicQueries
}
If this does not answer your question then please show one of your failed attempts (so we can see where you are going wrong).
One general comment - do you really need the interface? A common recommendation is "Accept interfaces, return structs". While this may not always apply I suspect you may be introducing interfaces where they are not really necessary and this may add unnecessary complexity.
I thought that the Store, which was housing both Queriers, was tying it all together. Can you explain a little with the example above (in the question post) why it's not necessary? How does SQLStore get access to all of the Querier interface functions?
The struct SQLStore is what is "tying it all together". As per the Go spec:
Given a struct type S and a named type T, promoted methods are included in the method set of the struct as follows:
If S contains an embedded field T, the method sets of S and *S both include promoted methods with receiver T. The method set of *S also includes promoted methods with receiver *T.
If S contains an embedded field *T, the method sets of S and *S both include promoted methods with receiver T or *T.
So an object of type SQLStore:
type SQLStore struct {
db *sql.DB
*sdb.Queries
*dynamicQueries
}
var foo SQLStore // Assume that we are actually providing values for all fields
Will implement all of the methods of sdb.Queries and, also, those in dynamicQueries (you can also access the sql.DB members via foo.db.XXX). This means that you can call foo.AddAccountBalance() and foo.MyGenericQuery() (assuming that is in dynamicQueries!) etc.
The spec says "In its most basic form an interface specifies a (possibly empty) list of methods". So you can think of an interface as a list of functions that must be implemented by whatever implementation (e.g. struct) you assign to the interface (the interface itself does not implement anything directly).
This example might help you understand.
Hopefully that helps a little (as I'm not sure which aspect you don't understand I'm not really sure what to focus on).
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
OK, I'm not quite getting it....
I have 2 modules I crafted with identical functions (in different files of course):
package mod1
func MyFunc() string {
return "mod1.Myfunc"
}
func Func2() string {
return "mod1.Func2"
}
package mod2
func MyFunc() string {
return "mod2.MyFunc"
}
func Func2() string {
return "mod2.Func2"
}
I have an interface defined correctly, (I think) in a third package:
package types
type MyType interface {
MyFunc() string
Func2() string
}
I have code which can pick whether I want to use mod1 or mod2, but I'm not quite understanding what I should have this code return:
func mypicker() ????{
}
Then in main, I want to somehow call either mod1.MyFunc() or mod2.MyFunc() based on
mypicker, without knowing which it is.... something like this:
func main() {
p := mypicker()
fmt.Print(p.MyFunc())
// and later
fmt.Print(p.Func2())
}
I read that interfaces are like void *, but clearly I'm not getting the complete picture.
Pointers to docs, code, anything useful would be great.
Interfaces should be used with types, not just plain functions. You can start by reading the Tour of Go sequence on interfaces. Here's an example close to your question's original code:
Given the interface:
type MyType interface {
MyFunc() string
Func2() string
}
You'd have a type:
type MyType1 struct{}
func (mt MyType1) MyFunc() string {
return "MyType1.MyFunc"
}
func (mt MyType1) Func2() string {
return "MyType1.Func2"
}
And similarly:
type MyType2 struct{}
func (mt MyType2) MyFunc() string {
return "MyType2.MyFunc"
}
func (mt MyType2) Func2() string {
return "MyType2.Func2"
}
And now, if you have some function that takes your MyType interface:
func Foo(m MyType) {
fmt.Println(m.Func2())
fmt.Println(m.MyFunc())
}
You could call it with either of your types that implements that interface:
m1 := MyType1{}
Foo(m1)
m2 := MyType2{}
Foo(m2)
Here's a Go Playground link where you can try this in action.
As for "picking a type", perhaps you mean something like this:
var mi MyType
if (... some condition ...) {
mi = m1
} else {
mi = m2
}
// Now you can do with mi whatever its interfaces permits,
// like calling mi.Func2(), etc.
Regarding the "picking one of two packages" part of the question:
Interfaces are implemented by types; they're orthogonal to packages and modules. In other words, an interface and types that implement it can all be in the same package, or in different packages, or in different modules.
You have to be careful with terminology. Go modules and Go packages are very different, even though both can be contained by directories. Basically, a directory is a package if it has at least one Go file in it and no go.mod file. If a directory has a go.mod file in it then it's recognized as a module. Generally, a whole project can be a single module with the go.mod file at the root of the project and that's sufficient. Assuming this is your case, move forward thinking that every sub-directory is just a package within that single module.
An interface doesn't really have to do with modules or packages, it has to do with types. The reason being is that an interface defines behavior, meaning it defines what methods are required for a type to accurately implement that interface. In your case, you defined both functions declared in your interface BUT they are NOT METHODS because they are top-level functions only attached to the package. In order for a function to be a method, it must be "attached" to a type. Then, that type becomes a valid implementation of that interface.
This...
package mod1
func MyFunc() string {
return "mod1.Myfunc"
}
func Func2() string {
return "mod1.Func2"
}
Needs to become this...
package mod1
type MyTypeImpl struct {}
func (m MyTypeImpl) MyFunc() string {
return "mod1.Myfunc"
}
func (m MyTypeImpl) Func2() string {
return "mod1.Func2"
}
The naming could be improved greatly but the point is that the above function declaration syntax is how you "attach" a function to a type, making it a method, which allows that MyTypeImpl struct to now be a valid implementation of your MyType interface.
Now you can call the interface methods without regards to which underlying type is actually the implementation:
var iType MyType
iType = MyTypeImpl{}
iType.MyFunc()
Notice that in that last line, it does not matter that we used MyTypeImpl to implement the interface. Once the implementation is assigned to a variable with the interface type, we just work with the interface and forget the underlying implementation. When we call iType.MyFunc(), Go will call the proper method from the underlying implementation.
If we had 100 different structs that implemented the MyType interface as MyTypeImpl does, they could all work for the right side of that iType = MyTypeImpl{} line. That's the point of an interface, to define it once and then use it without regard to what underlying struct is actually implementing it.
I have a piece of code which is used to load configuration from file and parse it into a struct, I use this configuration quite often and hence I pass it around in the method parameters. Now I as my method parameters are increasing, I am looking at dependency injection and have settle with wire.
Now I have created a provider to load the configuration and an injector to provide the config struct. However each time I call the injection my file is read again, I want that the file is read once and the injection provided as many times as required without any additional loading.
Here is my provider:
// ProvideConfig config provider ...
func ProvideConfig() *config.FileConfig {
var cfp string
flag.StringVar(&cfp, "config", "config.json", "absolute path")
flag.Parse()
return config.Loadconfig(cfp)
}
Injector:
// GetConfig injector ...
func GetConfig() ConfigResource {
wire.Build(ProvideConfig, NewConfigResource)
return ConfigResource{}
}
Now when I call:
injection.GetConfig()
I see that ProvideConfig is called always. I can have a check in the provide config method the determine if the config is already loaded, I am not sure if there is a better way, something like a single instance loader which is built into the wire. I tried looking into the docs but could not find anything relevant.
As far as I'm aware, there's no built in way in wire to specify that a provider is a singleton / should only be called once.
This is accomplished in the usual way in Go, by using sync.Once. Your provider function can be a closure that does the expensive operation only once using sync.Once.Do. This is idiomatic in Go, and doesn't require any special provision from every library that wants to provide "single" loading.
Here's an example without wire:
type Value struct {
id int
msg string
}
type ValueProvider func() *Value
// consumer takes a function that provides a new *Value and consumes
// the *Value provided by it.
func consumer(vp ValueProvider) {
v := vp()
fmt.Printf("Consuming %+v\n", *v)
}
// MakeSingleLoader returns a ValueProvider that creates a value once using an
// expensive operation, and then keeps returning the same value.
func MakeSingleLoader() ValueProvider {
var v *Value
var once sync.Once
return func() *Value {
once.Do(func() {
v = ExpensiveOperation()
})
return v
}
}
// ExpensiveOperation emulates an expensive operation that can take a while
// to run.
func ExpensiveOperation() *Value {
return &Value{id: 1, msg: "hello"}
}
func main() {
sl := MakeSingleLoader()
consumer(sl)
consumer(sl)
consumer(sl)
}
If you're OK with the "singleton" value being a global, this code can be simplified a bit. Otherwise, it only calls ExpensiveOperation once, and keeps the value cached in a local inaccessible outside MakeSingleLoader.
Assuming I want to return an instance of a "stateful" component to a user, what is the typical way I can cleanup/join background work within that instance? And are there any patterns to avoid viral propagation of explicit cleanup functions all the way to the root code?
For example, let's assume I am returning a client to a database to the user. In this client, I have a loop that periodically polls the server for updates. Now any time this exists within an ownership DAG (like as a member variable in another struct, or as a list in another struct). Requiring an explicit Close() will bubble up virally throughout the call stack. As each upwards link in the DAG will require a Close() as well. All the way to the function that owns the root instance (eg. main() will be required to call Close() on the root server instance, which will require an implementation of Close() so it cleans up background behind itself, etc). Something like the below
type DbClient struct { ... }
func Cleanup(client DbClient) { ... }
type Component struct {
client DbClient
...
}
func Cleanup(component Component) { ... }
type Server struct {
component Component
...
}
func Cleanup(server Server) { ... }
Is there any other way to handle these cases? Or is an explicit Close() function the recommendation for such stateful components?
I guess the problem you mentiond: "upwards link in the DAG will require a Close()" & "all the way to the func that owns root instance.
Go has struct embedding feature. Go favors composition over inheritance.
There's an important way in which embedding differs from subclassing. When we embed a type, the methods of that type become methods of the outer type, but when they are invoked the receiver of the method is the inner type, not the outer one.
package main
import "fmt"
type DbClient struct{}
func (client *DbClient) Cleanup() {
fmt.Println("Closed called on client")
}
type Component struct {
*DbClient
}
type Server struct {
*Component
}
func main() {
client := DbClient{}
component := Component{&client}
server := Server{&component}
server.Cleanup()
}
This question already has an answer here:
Is there a way to create a function using unexported type as parameter in Golang?
(1 answer)
Closed 3 months ago.
I have a package where I have unexported struct and exported New function to create it and exported function that runs on this struct (as advised for example here: Return an unexported type from a function).
If I run the function in the same place the New is called I can run the package function but I am unable to send this entity to another function.
what is the best way to achieve this behavior without the need to have all my code in one function
this for example works:
client := package.New()
client.Foo()
but this cannot work:
client := package.New()
hello(client)
func hello(client interface{}) {
client.Foo()
}
What your hello function essentially needs is something that has a Foo function. That's why go has interfaces. There's nothing wrong with returning an unexported type (in fact, it's common and often the right thing to do). What I'd do is this:
package foobar
// whatever thing that has a Foo function
type FClient interface {
Foo()
}
func Hello(client FClient) {
client.Foo() // will work
}
The reason why you'd do it like this is to be able to unit-test this code:
package foobar_test
import (
"testing"
)
type testFC struct {
callCount uint64
}
// implement interface
func (t testFC) Foo() {
testFC.callCount++
}
func TestHello(t *testing.T) {
client := testFC{}
Hello(client)
if client.callCount != 1 {
t.Fail("dependency not called")
}
}
Of course, for more complex dependencies you'd use tools like mockgen or stuff like that, but you get the idea. By definition, a UNIT test focuses on a single UNIT of code. The last thing you'd need to do to test a package unit is to instantiate a type from another package. You should be able to mock everything your code depends on. The best way to do so is interfaces.