I want to make sure that my update function is executed only by one thread at a time for a given value.
func update1(int id){
...
makeUpdate(id)
...
}
func update2(int id){
...
makeUpdate(id)
...
}
So, how should I write my makeUpdate() function that the myUpdate block is executed only once for a given id value? That means if update1 is updating the record with the id "15" and update2 with the id "20", the block access should not be synchronized.
As comments suggest - you need to protect data access not functional access.
Easiest way to achieve this, is to make a struct type with a lock - and attach the critical functional update as a method e.g.
type MyData struct {
l sync.Mutex
// add any other task related attributes here too
}
// makeUpdate *MUST* use a pointer to our struct (i.e. 'm *MyData')
// as Mutex logic breaks if copied (so no 'm MyData')
func (m *MyData) makeUpdate(id int) {
m.l.Lock()
defer m.l.Unlock()
fmt.Printf("better makeUpdate(%d)\n", id)
// do critical stuff here
// don't dilly-dally - lock is still being used - so return quickly
}
Try this out in the playground.
Related
I am newer to golang, so I have some courses that I bought from udemy to help break me into the language. One of them I found very helpful for a general understanding as I took on a project in the language.
In the class that I took, all of the sql related functions were in the sqlc folder with the structure less broken out:
sqlc
generatedcode
store
One of those files is a querier that is generated by sqlc that contains an interface with all of the methods that were generated. Here is the general idea of what it currently looks like: https://github.com/techschool/simplebank/tree/master/db/sqlc
package db
import (
"context"
"github.com/google/uuid"
)
type Querier interface {
AddAccountBalance(ctx context.Context, arg AddAccountBalanceParams) (Account, error)
CreateAccount(ctx context.Context, arg CreateAccountParams) (Account, error)
...
}
var _ Querier = (*Queries)(nil)
Would it be possible to wrap both what sqlc generates AND any queries that a developer creates (dynamic queries) into a single querier? I'm also trying to have it so that the sqlc generated code is in its own folder. The structure I am aiming for is:
sql
sqlc
generatedcode
store - (wraps it all together)
dynamicsqlfiles
This should clear up what a I mean by store: https://github.com/techschool/simplebank/blob/master/db/sqlc/store.go
package db
import (
"context"
"database/sql"
"fmt"
)
// Store defines all functions to execute db queries and transactions
type Store interface {
Querier
TransferTx(ctx context.Context, arg TransferTxParams) (TransferTxResult, error)
}
// SQLStore provides all functions to execute SQL queries and transactions
type SQLStore struct {
db *sql.DB
*Queries
}
// NewStore creates a new store
func NewStore(db *sql.DB) Store {
return &SQLStore{
db: db,
Queries: New(db),
}
}
I'm trying to run everything through that store (both generated and my functions), so I can make a call similar to the CreateUser function in this file (server.store.): https://github.com/techschool/simplebank/blob/master/api/user.go
arg := db.CreateUserParams{
Username: req.Username,
HashedPassword: hashedPassword,
FullName: req.FullName,
Email: req.Email,
}
user, err := server.store.CreateUser(ctx, arg)
if err != nil {
if pqErr, ok := err.(*pq.Error); ok {
switch pqErr.Code.Name() {
case "unique_violation":
ctx.JSON(http.StatusForbidden, errorResponse(err))
return
}
}
ctx.JSON(http.StatusInternalServerError, errorResponse(err))
return
}
I've tried creating something that houses another querier interface that embeds the generated one, then creating my own db.go that uses the generated DBTX interface but has its own Queries struct, and New function. It always gives me an error that the Queries struct I created aren't implementing the functions I made, despite having it implemented in one of the custom methods I made.
I deleted that branch, and have been clicking through the simplebank project linked above to see if I can find another way this could be done, or if I missed something. If it can't be done, that's okay. I'm just using this as a good opportunity to learn a little more about the language, and keep some code separated if possible.
UPDATE:
There were only a few pieces I had to change, but I modified the store.go to look more like:
// sdb is imported, but points to the generated Querier
// Store provides all functions to execute db queries and transactions
type Store interface {
sdb.Querier
DynamicQuerier
}
// SQLStore provides all functions to execute SQL queries and transactions
type SQLStore struct {
db *sql.DB
*sdb.Queries
*dynamicQueries
}
// NewStore creates a new Store
func NewStore(db *sql.DB) Store {
return &SQLStore{
db: db,
Queries: sdb.New(db),
dynamicQueries: New(db),
}
}
Then just created a new Querier and struct for the methods I would be creating. Gave them their own New function, and tied it together in the above. Before, I was trying to figure out a way to reuse as much of the generated code as possible, which I think was the issue.
Why I wanted the Interface:
I wanted a structure that separated the files I would be working in more from the files that I would never touch (generated). This is the new structure:
I like how the generated code put everything in the Querier interface, then checked that anything implementing it satisfied all of the function requirements. So I wanted to replicate that for the dynamic portion which I would be creating on my own.
It might be complicating it a bit more than it would 'NEED' to be, but it also provides an additional set of error checking that is nice to have. And in this case, even while maybe not necessary, it ended up being doable.
Would it be possible to wrap both what sqlc generates AND any queries that a developer creates (dynamic queries) into a single querier?
If I'm understanding your question correctly I think that you are looking for something like the below (playground):
package main
import (
"context"
"database/sql"
)
// Sample SQL C Code
type DBTX interface {
ExecContext(context.Context, string, ...interface{}) (sql.Result, error)
PrepareContext(context.Context, string) (*sql.Stmt, error)
QueryContext(context.Context, string, ...interface{}) (*sql.Rows, error)
QueryRowContext(context.Context, string, ...interface{}) *sql.Row
}
type Queries struct {
db DBTX
}
func (q *Queries) DeleteAccount(ctx context.Context, id int64) error {
// _, err := q.db.ExecContext(ctx, deleteAccount, id)
// return err
return nil // Pretend that this always works
}
type Querier interface {
DeleteAccount(ctx context.Context, id int64) error
}
//
// Your custom "dynamic" queries
//
type myDynamicQueries struct {
db DBTX
}
func (m *myDynamicQueries) GetDynamicResult(ctx context.Context) error {
// _, err := q.db.ExecContext(ctx, deleteAccount, id)
// return err
return nil // Pretend that this always works
}
type myDynamicQuerier interface {
GetDynamicResult(ctx context.Context) error
}
// Combine things
type allDatabase struct {
*Queries // Note: You could embed this directly into myDynamicQueries instead of having a seperate struct if that is your preference
*myDynamicQueries
}
type DatabaseFunctions interface {
Querier
myDynamicQuerier
}
func main() {
// Basic example
var db DatabaseFunctions
db = getDatabase()
db.DeleteAccount(context.Background(), 0)
db.GetDynamicResult(context.Background())
}
// getDatabase - Perform whatever is needed to connect to database...
func getDatabase() allDatabase {
sqlc := &Queries{db: nil} // In reality you would use New() to do this!
myDyn := &myDynamicQueries{db: nil} // Again it's often cleaner to use a function
return allDatabase{Queries: sqlc, myDynamicQueries: myDyn}
}
The above is all in one file for simplicity but could easily pull from multiple packages e.g.
type allDatabase struct {
*generatedcode.Queries
*store.myDynamicQueries
}
If this does not answer your question then please show one of your failed attempts (so we can see where you are going wrong).
One general comment - do you really need the interface? A common recommendation is "Accept interfaces, return structs". While this may not always apply I suspect you may be introducing interfaces where they are not really necessary and this may add unnecessary complexity.
I thought that the Store, which was housing both Queriers, was tying it all together. Can you explain a little with the example above (in the question post) why it's not necessary? How does SQLStore get access to all of the Querier interface functions?
The struct SQLStore is what is "tying it all together". As per the Go spec:
Given a struct type S and a named type T, promoted methods are included in the method set of the struct as follows:
If S contains an embedded field T, the method sets of S and *S both include promoted methods with receiver T. The method set of *S also includes promoted methods with receiver *T.
If S contains an embedded field *T, the method sets of S and *S both include promoted methods with receiver T or *T.
So an object of type SQLStore:
type SQLStore struct {
db *sql.DB
*sdb.Queries
*dynamicQueries
}
var foo SQLStore // Assume that we are actually providing values for all fields
Will implement all of the methods of sdb.Queries and, also, those in dynamicQueries (you can also access the sql.DB members via foo.db.XXX). This means that you can call foo.AddAccountBalance() and foo.MyGenericQuery() (assuming that is in dynamicQueries!) etc.
The spec says "In its most basic form an interface specifies a (possibly empty) list of methods". So you can think of an interface as a list of functions that must be implemented by whatever implementation (e.g. struct) you assign to the interface (the interface itself does not implement anything directly).
This example might help you understand.
Hopefully that helps a little (as I'm not sure which aspect you don't understand I'm not really sure what to focus on).
I have a piece of code which is used to load configuration from file and parse it into a struct, I use this configuration quite often and hence I pass it around in the method parameters. Now I as my method parameters are increasing, I am looking at dependency injection and have settle with wire.
Now I have created a provider to load the configuration and an injector to provide the config struct. However each time I call the injection my file is read again, I want that the file is read once and the injection provided as many times as required without any additional loading.
Here is my provider:
// ProvideConfig config provider ...
func ProvideConfig() *config.FileConfig {
var cfp string
flag.StringVar(&cfp, "config", "config.json", "absolute path")
flag.Parse()
return config.Loadconfig(cfp)
}
Injector:
// GetConfig injector ...
func GetConfig() ConfigResource {
wire.Build(ProvideConfig, NewConfigResource)
return ConfigResource{}
}
Now when I call:
injection.GetConfig()
I see that ProvideConfig is called always. I can have a check in the provide config method the determine if the config is already loaded, I am not sure if there is a better way, something like a single instance loader which is built into the wire. I tried looking into the docs but could not find anything relevant.
As far as I'm aware, there's no built in way in wire to specify that a provider is a singleton / should only be called once.
This is accomplished in the usual way in Go, by using sync.Once. Your provider function can be a closure that does the expensive operation only once using sync.Once.Do. This is idiomatic in Go, and doesn't require any special provision from every library that wants to provide "single" loading.
Here's an example without wire:
type Value struct {
id int
msg string
}
type ValueProvider func() *Value
// consumer takes a function that provides a new *Value and consumes
// the *Value provided by it.
func consumer(vp ValueProvider) {
v := vp()
fmt.Printf("Consuming %+v\n", *v)
}
// MakeSingleLoader returns a ValueProvider that creates a value once using an
// expensive operation, and then keeps returning the same value.
func MakeSingleLoader() ValueProvider {
var v *Value
var once sync.Once
return func() *Value {
once.Do(func() {
v = ExpensiveOperation()
})
return v
}
}
// ExpensiveOperation emulates an expensive operation that can take a while
// to run.
func ExpensiveOperation() *Value {
return &Value{id: 1, msg: "hello"}
}
func main() {
sl := MakeSingleLoader()
consumer(sl)
consumer(sl)
consumer(sl)
}
If you're OK with the "singleton" value being a global, this code can be simplified a bit. Otherwise, it only calls ExpensiveOperation once, and keeps the value cached in a local inaccessible outside MakeSingleLoader.
So a struct holds data that could get mutated. Is there some trick or technique in Golang that can tell a func that it must accept a new instance of a struct? In other words, try to best avoid reusing data that may have been mutated before the fact or may get mutated during func lifecycle. (I could avoid mutating stuff, but other devs on my team might not get the memo).
To illustrate:
type CMRequest struct {
Endpoint string
Method string
}
func (cmreq CMRequest) Run() (res *goreq.Response) {
/// this could mutate cmreq
}
obviously Run() could mutate the data in cmreq, so I am wondering if there is a good pattern to force the creation of fresh data every time? The only thing I can think of is to keep the struct private, and do something like this:
type cmrequest struct {
Endpoint string
Method string
}
func (cmreq cmrequest) Run() (res *goreq.Response) {
/// this could mutate cmreq
}
and then expose a helper func:
func MakeRequestAndUnmarshalBody(d CMRequestNoReuse) (*goreq.Response) {
// check that d has a unique memory location?
cmreq := NewCPRequest(d)
res := cmreq.Run()
return res
}
so the helper func would be public, and it would create a new instance of the struct every time? is there any other way to go about it? I still can't force the user to pass in new data every time, although I could check to see if the memory location of d CMRequestNoReuse is unique?
Actually, no, in your example, this doesn't mutate data in CMRequest instance:
func (cmreq CMRequest) Run() (res *goreq.Response) {
/// this could mutate cmreq
}
When you pass object by value, you actually get copy of it, not reference to it.
To mutate, you need to pass by pointer. I.e. Run method would look like:
func (cmreq *CMRequest) Run() (res *goreq.Response) {
/// this actually can mutate cmreq
}
Then you actually have access to original object via pointer and can mutate it.
Here is example in go playground. But note, if one of the fields of struct is pointer -- you still can mutate it.
I am having trouble in accessing the one struct's properties (named Params) in different file.
please consider x.go where i invoke a function(CreateTodo)
type Params struct {
Title string `json:"title"`
IsCompleted int `json:is_completed`
Status string `json:status`
}
var data = &Params{Title:"booking hotel", IsCompleted :0,Status:"not started"}
isCreated := todoModel.CreateTodo(data) // assume todoModel is imported
now CreateTodo is a method on a struct (named Todo) in different file lets say y.go
type Todo struct {
Id int `json:todo_id`
Title string `json:"title"`
IsCompleted int `json:is_completed`
Status string `json:status`
}
func (mytodo Todo)CreateTodo(data interface{}) bool{
// want to access the properties of data here
fmt.Println(data.Title)
return true
}
Now I just want to use properties of data in CreateTodo function in y.go.
But i am not able to do so and getting following error
data.Title undefined (type interface {} is interface with no methods)
I am sure issue is around accepting struct as an empty interface but i am not able to figure out.
Please help here.Thanks
So you have one of two options, depending on your model:
#1
Switch to data *Params instead of data interface{} as suggested in another answer but it looks like you are expecting different types in this function, if so; check option #2 below.
#2
Use Type switches as follows:
func (t Todo) CreateTodo(data interface{}) bool {
switch x := data.(type) {
case Params:
fmt.Println(x.Title)
return true
// Other expected types
default:
// Unexpected type
return false
}
}
P.S. Be careful with your json tags: it should be json:"tagName". Notice the ""! Check go vet.
You could just type the function parameter:
func (mytodo Todo)CreateTodo(data *Params) bool{
// want to access the properties of data here
fmt.Println(data.Title)
return true
}
See: https://play.golang.org/p/9N8ixBaSHdP
If you want to operate on a Params (or *Params), you must do that.
If you want to operate on an opaque type hidden behind an interface{}, you must do that.
In short, you cannot peek behind the curtain without peeking behind the curtain. Either expose the actual type Params, so that you can look at it, or keep all the code that does look at it elsewhere. The "keep the code elsewhere" is where interface really shines, because it allows you to declare that something otherwise-opaque has behaviors and ask for those behaviors to happen:
type Titler interface {
GetTitle() string
}
If Params has a GetTitle function, it becomes a Titler.
You can now define your CreateTodo as a function that takes a Titler, and then you can pass &data to this function.
This structure is overall quite klunky and it seems much more likely that Todo itself should be an embeddable struct instead, but see a more complete example starting from a stripped-down version of your sample code here, in the Go Playground.
I want to create before save and after save method hooks for my Go structs how can I achieve this?
type Person struct {
FirstName string
LastName string
}
func (p *Person) Save() {
// call beforeSave()
// Save person data
// call afterSave()
}
func (p *Person) Update() {
// call beforeUpdate()
// Update person data
// call afterUpdate()
}
type Order struct {
Number bson.ObjectId
Items []Item
}
func (o *Order) Save() {
// call beforeSave()
// Save order data
// call afterSave()
}
func (o *Order) Update() {
// call beforeUpdate()
// Update order data
// call afterUpdate()
}
For any struct I create as a model I want it to have a beforeSave() and afterSave() hook called automatically and be able to further override if necessary.
Packages like gorm use callback hooks heavily. But if you are writing your own engine (for some specific logic) using interfaces can help greatly (sample).
https://medium.com/#matryer/the-http-handler-wrapper-technique-in-golang-updated-bc7fbcffa702
I would only add to that article above, which I think well answers your question, that you can go further using pipelines (methods that return the associated variable they are bound to). It's particularly interesting as in the above article, as relates to being able to interpose anything between two processes, and you can use this to create tees and journalling/logging systems, and it can go a long way towards compensating for the kinda primitive error handling in Go.
I designed a number of related byte slice processing libraries that were able to be easily chained without needing to break a a line, although obviously more than about 3-4 in a line can get confusing and you can find yourself wanting to pass things in a DAG style pattern, at which point I think channels would make sense.