Refactoring Golang avoiding manual fields updating between similar structs - go

I'm using GraphQL and go-pg.
I have many entities like these:
type Player struct {
ID int
CreatedAt time.Time `pg:"default:now(),notnull"`
TeamID int `pg:",notnull"`
Team *Team
Type int
Score int64 `pg:",notnull"`
Note *string
// and others...
}
type PlayerInput struct {
TeamID int
Type int
Score int64
Note *string
// and others...
}
I have many times functions like these:
func (db *postgres) Update(context context.Context, id int, input types.PlayerInput) (*types.Player, error) {
var actualPlayer types.Player
newPlayer := graphqlToDB(&input)
tx, err := db.Begin()
//handle err
err = tx.Model(&actualPlayer).Where("id = ?", id).For("UPDATE").Select()
// handle err and rollback
actualPlayer.TeamID = newPlayer.TeamID
actualPlayer.Type = newPlayer.Type
actualPlayer.Score = newPlayer.Score
actualPlayer.Note = newPlayer.Note
// and others...
_, err = tx.Model(&actualPlayer).WherePK().Update()
// handle err and rollback
err = tx.Commit()
//handle err
return &actualPlayer, nil
}
func graphqlToDB(input *types.PlayerInput) *types.Player {
var output = &types.Player{
TeamID: input.TeamID,
Type: input.Type,
Score: input.Score,
Note: input.Note,
// and others...
}
if input.Type == "example" {
output.Score = 10000000
}
return output
}
I have this code for each entity in my project and I would like to limit/avoid redundant code, specially:
transformation from Graphql input type every time
newPlayer := graphqlToDB(&input)
manual updating of these (and other) fields every time
actualPlayer.TeamID = newPlayer.TeamID
actualPlayer.Type = newPlayer.Type
actualPlayer.Score = newPlayer.Score
actualPlayer.Note = newPlayer.Note
opening and closing DB transaction every time
tx, err := db.Begin()
Am I asking for the moon?

I don't think there's an abnormal amount of redundancy in this code.
transformation from Graphql input type every time
Transforming structs from external to internal models is a common pattern, and helps with separation of concerns. Furthermore, you already have the graphqlToDB function that allows you to reuse the 10 lines of code in its body. That's probably as good as it can get.
manual updating of these (and other) fields every time
In the specific piece of code you showed here, actualPlayer is of type types.Player and graphqlToDB function returns a *types.Player object.
So you could simply write actualPlayer := graphqlToDB(&input) and then pass the pointer around, like tx.Model(actualPlayer).
This saves remapping newPlayer to actualPlayer
opening and closing DB transaction every time
If you need to hit the DB transactionally every time, then you need to open the transaction every time (and then commit/rollback). There's no redundancy in this. Refactoring might just result in loss of readability.

Related

How can I dynamically populate a struct?

I want to dynamically populate my internal struct, for an atomic insert. I am new to go so pointers and referencing them is something that I am still learning. I can not figure out why this for each loop is putting the same fields in twice. I tried removing the '&' then I get a cannot use type as *type error, I checked to make sure my loop was hitting every object in the tradeArray, and it is. It looks like it is overwriting the object before it with the last one it loops over. How can I fix this?
func createTrade(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var tradeArray []Trade
if err := json.NewDecoder(r.Body).Decode(&tradeArray); err != nil {
e := Error{Message: "Bad Request - Improper Types Passed"}
w.WriteHeader(http.StatusBadRequest)
_ = json.NewEncoder(w).Encode(e)
return
}
for _, trade := range tradeArray {
internal := InternalTrade{
Id: strconv.Itoa(rand.Intn(1000000)),
Trade: &trade,
}
submit := TradeSubmitted{
TradeId: internal.Id,
ClientTradeId: trade.ClientTradeId ,
}
submitArray = append(submitArray, submit)
trades = append(trades, internal)
}
if err := json.NewEncoder(w).Encode(submitArray); err != nil {
e := Error{Message:"Internal Server Error"}
w.WriteHeader(http.StatusInternalServerError)
_ = json.NewEncoder(w).Encode(e)
return
}
}
edit: I was able to fix my problem by creating a new variable to hold the trade and referencing that variable in the struct creation. I am not sure how this is different that what I was doing above with just referencing the "trade" if someone could explain that I would greatly appreciate it.
for _, trade := range tradeArray {
p := trade
internal := InternalTrade{
Id: strconv.Itoa(rand.Intn(1000000)),
Trade: &p,
}
submit := TradeSubmitted{
TradeId: internal.Id,
ClientTradeId: trade.ClientTradeId ,
}
submitArray = append(submitArray, submit)
trades = append(trades, internal)
}
Let's look at just these parts:
var tradeArray []Trade
// code that fills in `tradeArray` -- correct, and omitted here
for _, trade := range tradeArray {
internal := InternalTrade{
Id: strconv.Itoa(rand.Intn(1000000)),
Trade: &trade,
}
submit := TradeSubmitted{
TradeId: internal.Id,
ClientTradeId: trade.ClientTradeId ,
}
submitArray = append(submitArray, submit)
trades = append(trades, internal)
}
This for loop, as you have seen, doesn't work the way you want. Here's a variant of it that's kind of similar, except that the variable trade has scope that extends beyond the for loop:
var trade Trade
for i := range tradeArray {
trade = tradeArray[i]
internal := InternalTrade{
Id: strconv.Itoa(rand.Intn(1000000)),
Trade: &trade,
}
// do correct stuff with `internal`
}
Note that each internal object points to a single, shared trade variable, whose value gets overwritten on each trip through the loop. The result is that they all point to the one from the last trip through the loop.
Your fix is itself OK: each trip through the loop, you make a new (different) p variable, and use &p, so that each internal.Trade has a different pointer to a different copy. You could also just do trade := trade inside the loop, to create a new unique trade variable. However, in this particular case, it may make the most sense to rewrite the loop this way:
for i := range tradeArray {
internal := InternalTrade{
Id: strconv.Itoa(rand.Intn(1000000)),
Trade: &tradeArray[i],
}
// do correct stuff with `internal`
}
That is, you already have len(tradeArray) different Trade objects: the slice header tradeArray gives you access to each tradeArray[i] instance, stored in the underlying array. You can just point to those directly.
There are various advantages and disadvantages to this approach. The big advantage is that you don't re-copy each trade at all: you just use the ones from the array that the slice header covers, that was allocated inside the json Decode function somewhere. The big disadvantage is that this underlying array cannot be garbage-collected as long as you retain any pointer to any of its elements. That disadvantage may have no cost at all, depending on the structure of the remaining code, but if it is a disadvantage, consider declaring tradeArray as:
var tradeArray []*Trade
so that the json Decode function allocates each one separately, and you can point to them one at a time without forcing the retention of the entire collection.

GoRoutines and passing struct to original context

I have a configuration that defines a number of instances (SomeConfigItems) which have a thing() created for each of them.
That thing is a struct returned by an included package, which contains, among other things, a Price (float64) and a nested struct. The nested struct maintains a map of trades.
The problem is that I am able to loop through the thing.Streams.Trades and see all trades happening in real time from my main()'s for{} loop. I am not able to see an updated thing.Price even though it is set in the Handler on occasion.
I am having a hard time understanding how the nested structs can contain data but not Price. I feel as though I am missing something with scoping, goroutines, or possibly pointers for instantiation of new objects.
Any help would be appreciated, I will continue reading in the meantime. I've reduced the code to what seems relevant.
main.go:
package main
import thing
var Things []thing.Handler
for _, name := range SomeConfigItems {
handler := thing.New(name)
Things = append(Things, handler)
}
for {
for _, t := range Things {
log.Info("Price: ", t.Price) // This is set to 0 every iteration, but I can actively data in thing.Streams.Trades
}
}
thing.go:
package thing
import streams
type Handler struct {
Name string
Price float64
Streams streams.Streams
}
func New(name string) (h Handler, err error) {
stream, err := streams.New(strings.ToLower(name))
h = Handler{
Name: name,
Price: "0.0"
Streams: stream,
}
go h.handler()
return h, err
}
func (bot *Handler) handler() {
var currentPrice float64
for {
currentPrice = external.GetPrice(bot.Name).Price // Validated that this returns a float64
bot.Price = currentPrice // Verified that this is updated immediately after in this context.
// Unable to see Price updated from outer context.
}
}
streams.go:
package streams
type Streams struct {
Trades
}
type State struct {
Price string `json:"p"`
Quantity string `json:"q"`
}
type Trades struct {
Trades map[float64]float64
TradeMutex sync.Mutex
Updates chan State
}
func New(name string) (s Streams, err error) {
p := newTradeStream(name)
s = Streams{
Trades: p,
}
return s, err
}
func newTradeStream(name string) (ts Trades) {
ts = Trades{}
ts.Trades = make(map[float64]float64, MaxDepth)
ts.Updates = make(chan State, 500)
// ... Other watchdog code
return ts
}
Note:
I am added some debug logging in multiple locations. From within the Bot Handler, the price was printed (successfully), then updated, and then printed (successfully) again -- Showing no gap in the setting of Price from within the handler() function.
When adding the same type of debugging to the main() for{} loop, I tried setting an incrementing counter and assigning the value of thing.Price -- Printing thing.Price on each loop results in 0, even if I set the price (and validate it gets set) in the same loop, it is back to 0 on the next iteration.
This behavior is why I think that I am missing something very fundamental.
In Go, arguments are passed to functions by value -- meaning what the function gets is a copy of the value, not a reference to the variable. The same is true of the function receiver, and also the return list.
It's not the most elegant description, but for the sake of explanation, let's call this the "function wall." If the value being passed one way or the other is a pointer, the function still gets a copy, but it's a copy of a memory address, and so the pointer can be used to change the value of the variable on the other side of the wall. If it is a reference type, which uses a pointer in the implementation of the type, then again a change to the thing being pointed to can cross that wall. But otherwise the change does not cross the wall, which is one reason so many Go functions are written to return values instead of just modifying values.
Here's a runnable example:
package main
import (
"fmt"
)
type Car struct {
Color string
}
func (c Car) Change() { // c was passed by value, it's a copy
c.Color = "Red"
}
func main() {
ride := Car{"Blue"}
ride.Change()
fmt.Println(ride.Color)
}
Prints "Blue"
But two small changes:
func (c *Car) Change() { // here
c.Color = "Red"
}
func main() {
ride := &Car{"Blue"} // and here
ride.Change()
fmt.Println(ride.Color)
}
And now it prints "Red". Struct is not a reference type. So if you want modifications to a struct to cross the wall without using the return list to do it, use a pointer. Of course this only applies to values being passed via argument, return list, or receiver; and not to variables that are in scope on both sides of the wall; or to modifying the underlying value behind a reference type.
See also "Pointers Versus Values" in Effective Go, and "Go Data Structures" by Russ Cox.

Query result is memory address

I'm new to go and still confused about pointers but I have followed the instructions for querying multiple rows but the result I get back is series of memory addresses instead of actual values.
This same structure, minus the rows.Next() works just fine for a single user so I'm confused as to the origin of the problem here.
Ultimately I'm trying to use the results of the function in a template but I'm trying to figure out the structure of it so I can range it in my HTML.
For example, if I try to run the code below, I get something like: &{[0xc... 0xc... 0xc...]}
type User struct {
Id int `json:"int"`
Name string `json:"name"`
Role string `json:"role"`
}
type Users struct {
Users []*User
}
func getUsers(company string) *Users {
users := Users{}
rows, err := db.Query("SELECT Id, Name, Role...")
// Check err
defer rows.Close()
for rows.Next() {
user := &User{}
err = rows.Scan(&user.Id, &user.Name, &user.Role)
// Check err
users.Users = append(users.Users, user)
}
err = rows.Err()
// Check err
return &users
}
This is how I'm attempting to use the function
func userView(w http.ResponseWriter, r *http.Request) {
res := getUsers("test") // Should return 3 results
fmt.Println(res.Users)
}
The problem isn't in your fetching of the data, it's in your display of the data. fmt.Println() prints memory addresses when given pointers--so it's behaving exactly as expected.
If you instead do:
fmt.Printf("%+v", res.Users)
you'll get a different result, probably closer to what you expect.
If you're planning to use a template, then you should do so--your template should be able to access the fields of each User just fine.
But the short answer is: Your testing method is invalid.
Type Users is a slice of pointers. If you print the return value of getUsers it looks like a bunch of memory addresses. This is OK.
If you want to print something more meaningful, write a String() method for Users in which you dereference each pointer and build a string containing struct fields.

Using empty interfaces in go

I am trying to understand the code that is used at my company. I am new to go lang, and I have already gone through the tutorial on their official website. However, I am having a hard time wrapping my head around empty interfaces, i.e. interface{}. From various sources online, I figured out that the empty interface can hold any type. But, I am having a hard time figuring out the codebase, especially some of the functions. I will not be posting the entire thing here, but just the minimal functions in which it has been used. Please bear with me!
Function (I am trying to understand):
func (this *RequestHandler) CreateAppHandler(rw http.ResponseWriter, r *http.Request) *foo.ResponseError {
var data *views.Data = &views.Data{Attributes: &domain.Application{}}
var request *views.Request = &views.Request{Data: data}
if err := json.NewDecoder(r.Body).Decode(request); err != nil {
logrus.Error(err)
return foo.NewResponsePropogateError(foo.STATUS_400, err)
}
requestApp := request.Data.Attributes.(*domain.Application)
requestApp.CreatedBy = user
Setting some context, RequestHandler is a struct defined in the same package as this code. domain and views are seperate packages. Application is a struct in the package domain. The following two structs are part of the package views:
type Data struct {
Id string `json:"id"`
Type string `json:"type"`
Attributes interface{} `json:"attributes"`
}
type Request struct {
Data *Data `json:"data"`
}
The following are part of the package json:
func NewDecoder(r io.Reader) *Decoder {
return &Decoder{r: r}
}
func (dec *Decoder) Decode(v interface{}) error {
if dec.err != nil {
return dec.err
}
if err := dec.tokenPrepareForDecode(); err != nil {
return err
}
if !dec.tokenValueAllowed() {
return &SyntaxError{msg: "not at beginning of value"}
}
// Read whole value into buffer.
n, err := dec.readValue()
if err != nil {
return err
}
dec.d.init(dec.buf[dec.scanp : dec.scanp+n])
dec.scanp += n
// Don't save err from unmarshal into dec.err:
// the connection is still usable since we read a complete JSON
// object from it before the error happened.
err = dec.d.unmarshal(v)
// fixup token streaming state
dec.tokenValueEnd()
return err
}
type Decoder struct {
r io.Reader
buf []byte
d decodeState
scanp int // start of unread data in buf
scan scanner
err error
tokenState int
tokenStack []int
}
Now, I understood that, in the struct Data in package views, Application is being set as a type for the empty interface. After that, a pointer to Request in the same package is created which points to the variable data.
I have the following doubts:
What exactly does this keyword mean in Go? What is the purpose of writing this * RequestHandler?
Initialization of a structure in Go can be done while assigning it to a variable by specifying the values of all it's members. However, here, for the struct Data, only the empty interface value is assigned and the values for the other two fields are not assigned?
What is the advantage of assigning the Application struct to an empty interface? Does it mean I can use the struct members using the interface variable directly?
Can someone help me figure out the meaning of this statement? json.NewDecoder(r.Body).Decode(request)?
While I know this is too much, but I am having a hard time figuring out the meaning of interfaces in Go. Please help!
this is not a keyword in go; any variable name can be used there. That is called the receiver. A function declared in that way must be called like thing.func(params), where "thing" is an expression of the type of the receiver. Within the function, the receiver is set to the value of thing.
A struct literal does not have to contain values for all the fields (or any of them). Any fields not explicitly set will have the zero value for their types.
As you said, an empty interface can take on a value of any type. To use a value of type interface{}, you would use type assertion or a type switch to determine the type of the value, or you could use reflection to use the value without having to have code for the specific type.
What specifically about that statement do you not understand? json is the name of a package in which the function NewDecoder is declared. That function is called, and then the Decode function (which is implemented by the type of the return value of NewDecoder) is called on that return value.
You may want to take a look at Effective Go and/or The Go Programming Language Specification for more information.

What happens if I concurrently access a single go map? [duplicate]

When you use a map in a program with concurrent access, is there any need to use a mutex in functions to read values?
Multiple readers, no writers is okay:
https://groups.google.com/d/msg/golang-nuts/HpLWnGTp-n8/hyUYmnWJqiQJ
One writer, no readers is okay. (Maps wouldn't be much good otherwise.)
Otherwise, if there is at least one writer and at least one more either writer or reader, then all readers and writers must use synchronization to access the map. A mutex works fine for this.
sync.Map has merged to Go master as of April 27, 2017.
This is the concurrent Map we have all been waiting for.
https://github.com/golang/go/blob/master/src/sync/map.go
https://godoc.org/sync#Map
I answered your question in this reddit thread few days ago:
In Go, maps are not thread-safe. Also, data requires locking even for
reading if, for example, there could be another goroutine that is
writing the same data (concurrently, that is).
Judging by your clarification in the comments, that there are going to be setter functions too, the answer to your question is yes, you will have to protect your reads with a mutex; you can use a RWMutex. For an example you can look at the source of the implementation of a table data structure (uses a map behind the scenes) which I wrote (actually the one linked in the reddit thread).
You could use concurrent-map to handle the concurrency pains for you.
// Create a new map.
map := cmap.NewConcurrentMap()
// Add item to map, adds "bar" under key "foo"
map.Add("foo", "bar")
// Retrieve item from map.
tmp, ok := map.Get("foo")
// Checks if item exists
if ok == true {
// Map stores items as interface{}, hence we'll have to cast.
bar := tmp.(string)
}
// Removes item under key "foo"
map.Remove("foo")
if you only have one writer, then you can probably get away with using an atomic Value. The following is adapted from https://golang.org/pkg/sync/atomic/#example_Value_readMostly (the original uses locks to protect writing, so supports multiple writers)
type Map map[string]string
var m Value
m.Store(make(Map))
read := func(key string) (val string) { // read from multiple go routines
m1 := m.Load().(Map)
return m1[key]
}
insert := func(key, val string) { // update from one go routine
m1 := m.Load().(Map) // load current value of the data structure
m2 := make(Map) // create a new map
for k, v := range m1 {
m2[k] = v // copy all data from the current object to the new one
}
m2[key] = val // do the update that we need (can delete/add/change)
m.Store(m2) // atomically replace the current object with the new one
// At this point all new readers start working with the new version.
// The old version will be garbage collected once the existing readers
// (if any) are done with it.
}
Why no made use of Go concurrency model instead, there is a simple example...
type DataManager struct {
/** This contain connection to know dataStore **/
m_dataStores map[string]DataStore
/** That channel is use to access the dataStores map **/
m_dataStoreChan chan map[string]interface{}
}
func newDataManager() *DataManager {
dataManager := new(DataManager)
dataManager.m_dataStores = make(map[string]DataStore)
dataManager.m_dataStoreChan = make(chan map[string]interface{}, 0)
// Concurrency...
go func() {
for {
select {
case op := <-dataManager.m_dataStoreChan:
if op["op"] == "getDataStore" {
storeId := op["storeId"].(string)
op["store"].(chan DataStore) <- dataManager.m_dataStores[storeId]
} else if op["op"] == "getDataStores" {
stores := make([]DataStore, 0)
for _, store := range dataManager.m_dataStores {
stores = append(stores, store)
}
op["stores"].(chan []DataStore) <- stores
} else if op["op"] == "setDataStore" {
store := op["store"].(DataStore)
dataManager.m_dataStores[store.GetId()] = store
} else if op["op"] == "removeDataStore" {
storeId := op["storeId"].(string)
delete(dataManager.m_dataStores, storeId)
}
}
}
}()
return dataManager
}
/**
* Access Map functions...
*/
func (this *DataManager) getDataStore(id string) DataStore {
arguments := make(map[string]interface{})
arguments["op"] = "getDataStore"
arguments["storeId"] = id
result := make(chan DataStore)
arguments["store"] = result
this.m_dataStoreChan <- arguments
return <-result
}
func (this *DataManager) getDataStores() []DataStore {
arguments := make(map[string]interface{})
arguments["op"] = "getDataStores"
result := make(chan []DataStore)
arguments["stores"] = result
this.m_dataStoreChan <- arguments
return <-result
}
func (this *DataManager) setDataStore(store DataStore) {
arguments := make(map[string]interface{})
arguments["op"] = "setDataStore"
arguments["store"] = store
this.m_dataStoreChan <- arguments
}
func (this *DataManager) removeDataStore(id string) {
arguments := make(map[string]interface{})
arguments["storeId"] = id
arguments["op"] = "removeDataStore"
this.m_dataStoreChan <- arguments
}

Resources