I am currently learning Go.
Following this link. On initCache function it accepts a evictionAlgo type parameter without * prefix so meaning it's not the pointer. See below the code I am referring.
type cache struct {
storage map[string]string
evictionAlgo evictionAlgo
capacity int
maxCapacity int
}
func initCache(e evictionAlgo) *cache {
storage := make(map[string]string)
return &cache{
storage: storage,
evictionAlgo: e,
capacity: 0,
maxCapacity: 2,
}
}
I got really confused. So my question is when or what scenario to consider when using a dereference pointer? And when should I not use dereference.
From my experience, you only have two situations where you would use a pointer as input or output. For output, I don't think it's ever really needed. My use case for it, it just as a convenience. Normally if you are returning an empty struct, you might do it like this:
package user
import "os"
type cache struct { dir string }
func newCache() (cache, error) {
c, e := os.UserCacheDir()
if e != nil {
return cache{}, e
}
return cache{c}, nil
}
but if you instead change to pointer, then you can use nil for the zero value:
func newCache() (*cache, error) {
c, e := os.UserCacheDir()
if e != nil { return nil, e }
return &cache{c}, nil
}
Regarding input, you should only need a pointer, if you need to change a field:
package user
type cache struct { dir string }
func (c *cache) set() {
c.dir = "north"
}
I thought you could change a slice or map field without a pointer receiver, but upon trying it only seems to work with pointer receiver as well.
Related
I have a map (entities) where the key is a string and the value is a struct. Building that struct is an expensive operation because it has to go to the database to load the data.
type entityStateManagers struct {
entities map[string]*entityStateManager
mainLock *sync.Mutex
locks map[string]*sync.Mutex
}
The function below returns that struct for the provided key. If the struct is not in memory it will load it from the database.
I didn't want to have only one lock for everything because that would completely block access to the map while expensiveLoad(entityId) is running.
I tried having a separate struct (locks) with one lock per map key. With that, only access to entity_123 will have to wait for expensiveLoad(entity_123), for example.
I still have to do more testing, but it seems to be working.
func (handler entityStateManagers) getStateManager(entityId string) (*entityStateManager, error) {
handler.mainLock.Lock()
if handler.locks[entityId] == nil {
handler.locks[entityId] = &sync.Mutex{}
}
handler.mainLock.Unlock()
handler.locks[entityId].Lock()
defer handler.locks[entityId].Unlock()
if handler.entities[entityId] == nil {
handler.entities[entityId] = expensiveLoad(entityId)
}
return handler.entities[entityId], nil
}
Is this a reasonable approach? Is there something I am missing? Is there a better way for doing that?
i would say like this but i can be wrong, keeping two maps with same keys is useles, also pointers to mutexes are ehh.
type Wrapper struct {
*entityStateManager
sync.Mutex
}
type entityStateManagers struct {
entities map[string]*Wrapper
mainLock sync.Mutex
}
func (handler entityStateManagers) getStateManager(entityId string) (*entityStateManager, error) {
handler.mainLock.Lock()
val, ok := handler.entities[entityId]
if !ok {
val = &Wrapper{}
handler.entities[entityId] = val
}
val.Lock()
handler.mainLock.Unlock()
val.entityStateManager = expensiveLoad(entityId)
defer val.Unlock()
return val.entityStateManager, nil
}
So, I have struct P. I need to unmarshal some json data into P but sometimes it comes embedded struct, Embedded. In either case, I unmarshal the json from the API and need to format the "Formatted" field. It seems in the Embedded case my unmarshaller doesn't get called.
I have the following code:
package main
import (
"encoding/json"
"fmt"
)
type P struct {
Name string `json:"name"`
Formatted string `json:"formatted"`
}
type Embedded struct {
A struct {
B struct {
*P
} `json:"b"`
} `json:"a"`
}
func (p *P) UnmarshalJSON(b []byte) error {
type Alias P
a := &struct {
*Alias
}{
Alias: (*Alias)(p),
}
if err := json.Unmarshal(b, &a); err != nil {
return err
}
a.Formatted = fmt.Sprintf("Hi, my name is %v", a.Name)
return nil
}
func simple() {
b := []byte(`{"name":"bob"}`)
p := &P{}
if err := json.Unmarshal(b, &p); err != nil {
panic(err)
}
fmt.Printf("normal: %+v\n", p)
}
func embedded() {
b := []byte(`{"a":{"b":{"name":"bob"}}}`)
e := &Embedded{}
if err := json.Unmarshal(b, &e); err != nil {
panic(err)
}
fmt.Printf("embedded: %+v\n", e.A.B.P)
}
func main() {
simple()
embedded()
}
(I realize I can get rid of the custom unmarshaller and create a method to format the name but wanted to see if this way was possible.)
I don't know enough to explain all the reasons, I will just list what works and what doesn't. Someone more knowledgeable can fill you in on the reasons behind it.
The following works when B is a *struct, not sure why.
type Embedded struct {
A struct {
B *struct {
P
} `json:"b"`
} `json:"a"`
}
The following also works. I'm guessing that using an anonymous struct had some effect in the last one since a *struct is not required here.
type embedP struct {
P
}
type Embedded struct {
A struct {
B embedP `json:"b"`
} `json:"a"`
}
The following works if *P is initialised.
type embedP struct {
*P
}
type intermediate struct {
B embedP `json:"b"`
}
type Embedded struct {
A intermediate `json:"a"`
}
e := &Embedded{A:intermediate{embedP{P:&P{}}}}
But the same thing doesn't work with anonymous structs.
type Embedded struct {
A struct {
B struct {
*P
} `json:"b"`
} `json:"a"`
}
e := &Embedded{A : struct{B struct{*P}`json:"b"`}{B: struct{*P}{&P{}}}}
Play link
Other improvements
If p := &P{} is already a pointer you don't need to pass &p in json.Unmarshal. json.Unmarshal(b, p) would suffice. Same with e := &Embedded{}.
To extent #John's answer, take a look at the source code of json decoder, especially method indirect(v reflect.Value, decodingNull bool) line 442-483.
// indirect walks down v allocating pointers as needed,
// until it gets to a non-pointer.
// if it encounters an Unmarshaler, indirect stops and returns that.
// if decodingNull is true, indirect stops at the last pointer so it can be set to nil.
The method returns, json.Unmarshaler, encoding.TextUnmarshaler and the value of v. In current implementation, inside the method, basically the following steps were executed
If argument v is not a pointer, it will return immediately without checking whether v implements json.Unmarshaler/encoding.TextUnmarshaler or not. The method assigns nil for both unmarshaller regardless B implements custom unmarshaller or not.
If argument v is a pointer, it will check whether v implements json.Unmarshaler/encoding.TextUnmarshaler or not. In this case, if v is nil, a new value will be assigned to v.
If Embedded is defined as
type Embedded struct {
A struct {
B struct {
*P
} `json:"b"`
} `json:"a"`
}
when, decoding "b":{"name":"bob"} to field B, since B is not a pointer, (1) is applicable. As the result, custom unmarshaller is returned as nil, thus never being called. The json decoder uses default unmarshaller to decode json value to B's fields.
If Embedded is defined as
type Embedded struct {
A struct {
*B struct {
P
} `json:"b"`
} `json:"a"`
}
since field B is a pointer, (2) is applicable. The decoder allocates new struct{*P} to B, detects that B implements custom unmarshaller, then call it as expected. The following declaration
type Embedded struct {
A struct {
*B struct {
*P
} `json:"b"`
} `json:"a"`
}
also works, if P is preallocated, i.e.
//...
e := Embedded{}
e.A.B = &struct{ *P }{P: &P{}}
//...
If it's not preallocated, in (2) the decoder will assign &struct{*P}{} to B, then call the custom unmarshaller with B.P == nil. As the result, json value can't be captured by B.P during unmarshall.
Note:
I'm not sure whether it is desired behavior or not, and I can't find a clear documentation related to embedded struct.
I have a list of objects (olievere/Elastic SearchResult.Hits to be exact). Each of these has a json.RawMessage object and I'm looking to create an abstractable method that takes in an interface slice of any struct, Unmarshal's each individual hits' json.RawMessage into said struct, and appends it to the passed in []interface.
This func is not supposed to have any logic or insight into the desired business layer struct, and the DB Call is interfaced pretty heavily, and as such has no visibility into the Elastic package mentioned above. Example of what I was attempting to do...
foo.go
import (bar, package)
type TestStruct struct {
Slice []*package.Struct // package.Struct has a value of Source which is a
// json.RawMessage
}
func GetData() bar.Test {
return &TestStruct{*package.GetData()}
}
func (result TestStruct) UnmarshalStruct(v []interface{}) {
for _, singleStruct := range result.Slice {
append(json.Unmarshal(singleStruct, &v))
}
Second File
bar.go
type Handler interface {
GetData() Test
}
type Test interface {
UnmarshalStruct
}
type OtherType struct {
foo string
bar string
}
func RetrieveData() []OtherType {
handler := New(Handler)
test := handler.GetData()
var typeSlice []OtherType
test.UnmarshalStruct(&typeSlice)
}
I'm looking to hand something of type []OtherType, or any other new struct I decide to create, to UnmarshalStruct, and have it return to me that same struct, just full of data
As an example, I have two different types of data I'll be searching for from Elastic. I will be receiving a list of ONE of the following two objects.
{ 'foo': '',
'id':
}
And in a different index
{ 'bar': '',
'baz': '',
'eee': ''
}
Each of these will naturally require two different structs.
However, I desire a single method to be able to decode either of these lists. I'll be given below, and using the same function I want to be able to convert this to a bar struct, and the other type to a foo struct.
{ 'source': [
{ 'bar': '',
'baz': '',
'eee': ''
},
{ 'bar': '',
'baz': '',
'eee': ''
},
{ 'bar': '',
'baz': '',
'eee': ''
}
]
}
There's really no way to do what you want without reflection. I would personally structure this differently, so that you unmarshal into more generic types, like a map[string]string, or as #ThunderCat shows, get rid of the intermediary state and unamrshal directly into the correct types. But it can be done...
(I moved the json.RawMessage directly into TestStruct to get rid of one level of indirection and make the example more clear)
type TestStruct struct {
Slice []json.RawMessage
}
func (t TestStruct) UnmarshalStruct(v interface{}) error {
// get the a Value for the underlying slice
slice := reflect.ValueOf(v).Elem()
// make sure we have adequate capacity
slice.Set(reflect.MakeSlice(slice.Type(), len(t.Slice), len(t.Slice)))
for i, val := range t.Slice {
err := json.Unmarshal(val, slice.Index(i).Addr().Interface())
if err != nil {
return err
}
}
return nil
}
You can then call it like so
var others []OtherType
err := ts.UnmarshalStruct(&others)
if err != nil {
log.Fatal(err)
}
http://play.golang.org/p/dgly2OOXDG
If I understand correctly, you want to unmarshal data to slices of two types:
type A struct {
Foo string `json:"foo"`
ID string `json:"id"`
}
type B struct {
Bar string `json:"bar"`
Baz string `json:"baz"`
Eee string `json:"eee"`
}
from a SearchHit Source.
The JSON package can do most of the work for you:
func executeQuery(q Query, v interface{}) error {
// Get a SearchHit. I am making this up.
// I have no idea how the package works.
searchHit, err := getHit(q)
if err != nil {
return err
}
// This is the important part. Convert the raw message to
// a slice of bytes and decode to the caller's slice.
return json.Unmarshal([]byte(*searchHit.Source), v)
}
You can call this function to decode to a slice of the types or a slice of pointers to the types.
// Slice of type
var s1 []TypeA
if err := executeQuery(q1, &s1); err != nil {
// handle error
}
// Slice of pointer to type
var s2 []*TypeB
if err := error(q2, &s2); err != nil {
// handle error
}
I know that this is not a direct answer to the question, but this is how this scenario is typically handled.
I don't believe this is easy to do. In the Raw Message Example in the godocs they use a value within the json, "Space" in their example, to determine which struct type to unmarshal into.
For this to work, the function would have to have some way of getting every struct that has been defined ever for the program, and then it would have to examine each json object and compare it to each struct using reflection to figure out which type it most likely is. And what if there are multiple structs that "could be it"? Then conflict resolution complicates things.
In short, I don't think you can do this.
Due to error handling in Go, I often end up with multiple values functions. So far, the way I have managed this has been very messy and I am looking for best practices to write cleaner code.
Let's say I have the following function:
type Item struct {
Value int
Name string
}
func Get(value int) (Item, error) {
// some code
return item, nil
}
How can I assign a new variable to item.Value elegantly. Before introducing the error handling, my function just returned item and I could simply do this:
val := Get(1).Value
Now I do this:
item, _ := Get(1)
val := item.Value
Isn't there a way to access directly the first returned variable?
In case of a multi-value return function you can't refer to fields or methods of a specific value of the result when calling the function.
And if one of them is an error, it's there for a reason (which is the function might fail) and you should not bypass it because if you do, your subsequent code might also fail miserably (e.g. resulting in runtime panic).
However there might be situations where you know the code will not fail in any circumstances. In these cases you can provide a helper function (or method) which will discard the error (or raise a runtime panic if it still occurs).
This can be the case if you provide the input values for a function from code, and you know they work.
Great examples of this are the template and regexp packages: if you provide a valid template or regexp at compile time, you can be sure they can always be parsed without errors at runtime. For this reason the template package provides the Must(t *Template, err error) *Template function and the regexp package provides the MustCompile(str string) *Regexp function: they don't return errors because their intended use is where the input is guaranteed to be valid.
Examples:
// "text" is a valid template, parsing it will not fail
var t = template.Must(template.New("name").Parse("text"))
// `^[a-z]+\[[0-9]+\]$` is a valid regexp, always compiles
var validID = regexp.MustCompile(`^[a-z]+\[[0-9]+\]$`)
Back to your case
IF you can be certain Get() will not produce error for certain input values, you can create a helper Must() function which would not return the error but raise a runtime panic if it still occurs:
func Must(i Item, err error) Item {
if err != nil {
panic(err)
}
return i
}
But you should not use this in all cases, just when you're sure it succeeds. Usage:
val := Must(Get(1)).Value
Go 1.18 generics update: Go 1.18 adds generics support, it is now possible to write a generic Must() function:
func Must[T any](v T, err error) T {
if err != nil {
panic(err)
}
return v
}
This is available in github.com/icza/gog, as gog.Must() (disclosure: I'm the author).
Alternative / Simplification
You can even simplify it further if you incorporate the Get() call into your helper function, let's call it MustGet:
func MustGet(value int) Item {
i, err := Get(value)
if err != nil {
panic(err)
}
return i
}
Usage:
val := MustGet(1).Value
See some interesting / related questions:
How to pass multiple return values to a variadic function?
Return map like 'ok' in Golang on normal functions
Yes, there is.
Surprising, huh? You can get a specific value from a multiple return using a simple mute function:
package main
import "fmt"
import "strings"
func µ(a ...interface{}) []interface{} {
return a
}
type A struct {
B string
C func()(string)
}
func main() {
a := A {
B:strings.TrimSpace(µ(E())[1].(string)),
C:µ(G())[0].(func()(string)),
}
fmt.Printf ("%s says %s\n", a.B, a.C())
}
func E() (bool, string) {
return false, "F"
}
func G() (func()(string), bool) {
return func() string { return "Hello" }, true
}
https://play.golang.org/p/IwqmoKwVm-
Notice how you select the value number just like you would from a slice/array and then the type to get the actual value.
You can read more about the science behind that from this article. Credits to the author.
No, but that is a good thing since you should always handle your errors.
There are techniques that you can employ to defer error handling, see Errors are values by Rob Pike.
ew := &errWriter{w: fd}
ew.write(p0[a:b])
ew.write(p1[c:d])
ew.write(p2[e:f])
// and so on
if ew.err != nil {
return ew.err
}
In this example from the blog post he illustrates how you could create an errWriter type that defers error handling till you are done calling write.
No, you cannot directly access the first value.
I suppose a hack for this would be to return an array of values instead of "item" and "err", and then just do
item, _ := Get(1)[0]
but I would not recommend this.
How about this way?
package main
import (
"fmt"
"errors"
)
type Item struct {
Value int
Name string
}
var items []Item = []Item{{Value:0, Name:"zero"},
{Value:1, Name:"one"},
{Value:2, Name:"two"}}
func main() {
var err error
v := Get(3, &err).Value
if err != nil {
fmt.Println(err)
return
}
fmt.Println(v)
}
func Get(value int, err *error) Item {
if value > (len(items) - 1) {
*err = errors.New("error")
return Item{}
} else {
return items[value]
}
}
Here's a generic helper function with assumption checking:
func assumeNoError(value interface{}, err error) interface{} {
if err != nil {
panic("error encountered when none assumed:" + err.Error())
}
return value
}
Since this returns as an interface{}, you'll generally need to cast it back to your function's return type.
For example, the OP's example called Get(1), which returns (Item, error).
item := assumeNoError(Get(1)).(Item)
The trick that makes this possible: Multi-values returned from one function call can be passed in as multi-variable arguments to another function.
As a special case, if the return values of a function or method g are equal in number and individually assignable to the parameters of another function or method f, then the call f(g(parameters_of_g)) will invoke f after binding the return values of g to the parameters of f in order.
This answer borrows heavily from existing answers, but none had provided a simple, generic solution of this form.
In Go, http form data (e.g. from a POST or PUT request) can be accessed as a map of the form map[string][]string. I'm having a hard time converting this to structs in a generalizable way.
For example, I want to load a map like:
m := map[string][]string {
"Age": []string{"20"},
"Name": []string{"John Smith"},
}
Into a model like:
type Person struct {
Age int
Name string
}
So I'm trying to write a function with the signature LoadModel(obj interface{}, m map[string][]string) []error that will load the form data into an interface{} that I can type cast back to a Person. Using reflection so that I can use it on any struct type with any fields, not just a Person, and so that I can convert the string from the http data to an int, boolean, etc as necessary.
Using the answer to this question in golang, using reflect, how do you set the value of a struct field? I can set the value of a person using reflect, e.g.:
p := Person{25, "John"}
reflect.ValueOf(&p).Elem().Field(1).SetString("Dave")
But then I'd have to copy the load function for every type of struct I have. When I try it for an interface{} it doesn't work.
pi := (interface{})(p)
reflect.ValueOf(&pi).Elem().Field(1).SetString("Dave")
// panic: reflect: call of reflect.Value.Field on interface Value
How can I do this in the general case? Or even better, is there a more idiomatic Go way to accomplish what I'm trying to do?
You need to make switches for the general case, and load the different field types accordingly. This is basic part.
It gets harder when you have slices in the struct (then you have to load them up to the number of elements in the form field), or you have nested structs.
I have written a package that does this. Please see:
http://www.gorillatoolkit.org/pkg/schema
For fun, I tried it out. Note that I cheated a little bit (see comments), but you should get the picture. There is usually a cost to use reflection vs statically typed assignments (like nemo's answer), so be sure to weigh that in your decision (I haven't benchmarked it though).
Also, obvious disclaimer, I haven't tested all edge cases, etc, etc. Don't just copy paste this in production code :)
So here goes:
package main
import (
"fmt"
"reflect"
"strconv"
)
type Person struct {
Age int
Name string
Salary float64
}
// I cheated a little bit, made the map's value a string instead of a slice.
// Could've used just the index 0 instead, or fill an array of structs (obj).
// Either way, this shows the reflection steps.
//
// Note: no error returned from this example, I just log to stdout. Could definitely
// return an array of errors, and should catch a panic since this is possible
// with the reflect package.
func LoadModel(obj interface{}, m map[string]string) {
defer func() {
if e := recover(); e != nil {
fmt.Printf("Panic! %v\n", e)
}
}()
val := reflect.ValueOf(obj)
if val.Kind() == reflect.Ptr {
val = val.Elem()
}
// Loop over map, try to match the key to a field
for k, v := range m {
if f := val.FieldByName(k); f.IsValid() {
// Is it assignable?
if f.CanSet() {
// Assign the map's value to this field, converting to the right data type.
switch f.Type().Kind() {
// Only a few kinds, just to show the basic idea...
case reflect.Int:
if i, e := strconv.ParseInt(v, 0, 0); e == nil {
f.SetInt(i)
} else {
fmt.Printf("Could not set int value of %s: %s\n", k, e)
}
case reflect.Float64:
if fl, e := strconv.ParseFloat(v, 0); e == nil {
f.SetFloat(fl)
} else {
fmt.Printf("Could not set float64 value of %s: %s\n", k, e)
}
case reflect.String:
f.SetString(v)
default:
fmt.Printf("Unsupported format %v for field %s\n", f.Type().Kind(), k)
}
} else {
fmt.Printf("Key '%s' cannot be set\n", k)
}
} else {
// Key does not map to a field in obj
fmt.Printf("Key '%s' does not have a corresponding field in obj %+v\n", k, obj)
}
}
}
func main() {
m := map[string]string{
"Age": "36",
"Name": "Johnny",
"Salary": "1400.33",
"Ignored": "True",
}
p := new(Person)
LoadModel(p, m)
fmt.Printf("After LoadModel: Person=%+v\n", p)
}
I'd propose to use a specific interface instead of interface{} in your LoadModel
which your type has to implement in order to be loaded.
For example:
type Loadable interface{
LoadValue(name string, value []string)
}
func LoadModel(loadable Loadable, data map[string][]string) {
for key, value := range data {
loadable.LoadValue(key, value)
}
}
And your Person implements Loadable by implementing LoadModel like this:
type Person struct {
Age int
Name string
}
func (p *Person) LoadValue(name string, value []string) {
switch name {
case "Age":
p.Age, err = strconv.Atoi(value[0])
// etc.
}
}
This is the way, the encoding/binary package or the encoding/json package work, for example.