Using reflect to update value by reference when argument is not a pointer in go - go

I've had difficulty learning the basics of reflect, pointers and interface in go, so here's another entry level question I can't seem to figure out.
This code does what I want it to do - I'm using reflect to add another record to a slice that's typed as an interface.
package main
import (
"reflect"
"log"
)
type Person struct {
Name string
}
func Add(slice interface{}) {
s := reflect.ValueOf(slice).Elem()
// in my actual code, p is declared via the use of reflect.New([Type])
p := Person{Name:"Sam"}
s.Set(reflect.Append(s,reflect.ValueOf(p)))
}
func main() {
p := []Person{}
Add(&p)
log.Println(p)
}
If I changed the Add and main function to this, things don't work the way I want it to.
func Add(slice interface{}) {
s := reflect.ValueOf(&slice).Elem()
p := Person{Name:"Sam"}
s.Set(reflect.Append(reflect.ValueOf(slice),reflect.ValueOf(p)))
log.Println(s)
}
func main() {
p := []Person{}
Add(p)
log.Println(p)
}
That is, the log.Println(p) at the end doesn't show a slice with the record Sam in it like the way I had hoped. So my question is whether it's possible for me to have Add() receive a slice that is not a pointer, and for me to still write some code in Add() that will produce the outcome shown in my first scenario?
A lot of my recent questions dance around this kind of subject, so it's still taking me a while to figure out how to use the reflect package effectively.

No, it's not possible to append to a slice in a function without passing in a pointer to the slice. This isn't related to reflection, but to how variables are passed in to functions. Here's the same code, modified to not use reflection:
package main
import (
"log"
)
type Person struct {
Name string
}
func AddWithPtr(slicep interface{}) {
sp := slicep.(*[]Person)
// This modifies p1 itself, since *sp IS p1
*sp = append(*sp, Person{"Sam"})
}
func Add(slice interface{}) {
// s is now a copy of p2
s := slice.([]Person)
sp := &s
// This modifies a copy of p2 (i.e. s), not p2 itself
*sp = append(*sp, Person{"Sam"})
}
func main() {
p1 := []Person{}
// This passes a reference to p1
AddWithPtr(&p1)
log.Println("Add with pointer: ", p1)
p2 := []Person{}
// This passes a copy of p2
Add(p2)
log.Println("Add without pointer:", p2)
}
(Above, when it says 'copy' of the slice, it doesn't mean the copy of the underlying data - just the slice)
When you pass in a slice, the function effectively gets a new slice that refers to the same data as the original. Appending to the slice in the function increases the length of the new slice, but doesn't change the length of the original slice that was passed in. That's why the original slice remains unchanged.

Related

Should I encapsulate slice and maps in Go? If so, how to do it?

I want to create a structure which will be accessible in other packages, but I don't want to allow modify this structure. In other languages this is archived by making all fields private and exposing only public getters.
Solution with getters works fine for all data types except slices and maps because returned slices and maps aren't copied by default so they can be modified. The only solution that I managed to figure out is to create new map/slice and assign all items in a loop but this introduce a lot of repetitive and ugly code, especially for large nested structures.
package main
import (
"fmt"
)
type OtherStruct struct {
prop string
}
type Struct struct {
prop map[string]OtherStruct
}
func (s Struct) Prop() map[string]OtherStruct {
return s.prop
}
func (s Struct) Prop2() map[string]*OtherStruct {
prop := make(map[string]*OtherStruct, 0)
for k := range s.prop {
v := s.prop[k]
prop[k] = &v
}
return prop
}
func main() {
var s Struct;
// Simple getter
s = Struct{make(map[string]OtherStruct, 0)}
p1 := s.Prop()
fmt.Println(s) // &{map[]}
p1["something"] = OtherStruct{"test"}
fmt.Println(s) // {map[something:{test}]}
// Getter which copies map
s = Struct{make(map[string]OtherStruct, 0)}
p2 := s.Prop2()
fmt.Println(s) // &{map[]}
p2["something"] = &OtherStruct{"test"}
fmt.Println(s) // &{map[]}
}
Is there any better way to encapsulate slices/maps in Go? Or maybe I shouldn't use encapsulation at all in Go and use different approach?
Returning slice or map values is idiomatic Go. The user of your package will know how those data structures work in Go.
In your example, the user of Struct should know that adding a new entry into the returned map would reflect for any other user of the same map.
// Simple getter
s = Struct{make(map[string]OtherStruct, 0)}
p1 := s.Prop()
fmt.Println(s) // &{map[]}
p1["something"] = OtherStruct{"test"}
fmt.Println(s) // {map[something:{test}]}
You should worry about such things only in case of concurrency. That is, when multiple goroutines are accessing, and possibly changing, the elements inside your slice or map.

Can't set field of a struct that is typed as an interface{}

I've been struggling with the reflect package. This code below does what I expect:
package main
import (
"reflect"
"log"
)
type Car struct {
Model string
}
type Person struct {
Name string
Cars []Car
}
func ModifyIt(parent interface{},fieldName string, val interface{}) {
slice := reflect.ValueOf(parent).Elem()
nth := slice.Index(0)
//row := nth.Interface() // this line causes errors
row := nth.Interface().(Person)
elem := reflect.ValueOf(&row).Elem()
field := elem.FieldByName(fieldName)
log.Println(field.CanSet())
}
func main() {
p := []Person{Person{Name:"john"}}
c := []Car{Car{"corolla"},Car{"jetta"}}
ModifyIt(&p,"Cars",&c)
}
However, if I replace the line row := nth.Interface().(Person) with row := nth.Interface(), that is I remove the type assertion, then I get the error:
panic: reflect: call of reflect.Value.FieldByName on interface Value
on line "field := elem.FieldByName(fieldName)
I've tried a bunch of other things the last few hours like trying to do reflect.TypeOf(), reflect.Indirect() etc... on some of the other variables but with no success.
I've read some other questions like these:
reflect: call of reflect.Value.FieldByName on ptr Value
Set a struct field with field type of a interface
Golang reflection: Can't set fields of interface wrapping a struct
They seem to suggest that I don't have a good understanding of how pointers or interfaces work.
So my question is, how do I go about setting the field of a struct when the struct is typed as an interface?
UPDATE
I posted a solution as an answer, but I have no confidence in whether it is the proper or safe way of doing things. I hope someone can explain, or post a better solution.
Try this:
func ModifyIt(slice interface{}, fieldName string, newVal interface{}) {
// Create a value for the slice.
v := reflect.ValueOf(slice)
// Get the first element of the slice.
e := v.Index(0)
// Get the field of the slice element that we want to set.
f := e.FieldByName(fieldName)
// Set the value!
f.Set(reflect.ValueOf(newVal))
}
Call it like this:
p := []Person{Person{Name: "john"}}
c := []Car{Car{"corolla"}, Car{"jetta"}}
ModifyIt(p, "Cars", c)
Note that the call passes the slices directly instead of using pointers to slices. The pointers are not needed and add extra complexity.
Run it on the Playground.
Out of sheer luck, I finally got something to work.
I pieced together a bunch of random things I read with very little rhyme or reason. I even tried reading the Laws of Reflection on the Golang site, but I don't think I have a good grasp of how it relates to why I couldn't set variables typed as interface{}. In general, I still don't understand what I did.
My solution below is littered with comments to indicate my confusion, and lack of confidence in whether I did things properly or safely.
package main
import (
"reflect"
"log"
)
type Car struct {
Model string
}
type Person struct {
Name string
Cars []Car
}
func ModifyIt(parent interface{},fieldName string, val interface{}) {
log.Println(parent)
slice := reflect.ValueOf(parent).Elem()
nth := slice.Index(0)
row := nth.Interface()
log.Println(nth.CanSet()) // I can set this nth item
// I think I have a to make a copy, don't fully understand why this is necessary
newitem := reflect.New(reflect.ValueOf(row).Type())
newelem := newitem.Elem()
field := newelem.FieldByName(fieldName)
// I need to copy the values over from the old nth row to this new item
for c:=0; c<nth.NumField(); c++ {
newelem.Field(c).Set(reflect.Indirect(nth.Field(c)))
}
// now I can finally set the field for some reason I don't understand
field.Set(reflect.ValueOf(val).Elem())
// now that newitem has new contents in the field object, I need to overwrite the nth item with new item
// I don't know why I'm doing it, but I'll do it
// I also don't fully understand why I have to use Indirect sometimes, and not other times...it seems interchangeable with ValueOf(something).Elem(), I'm confused....
nth.Set(reflect.Indirect(newitem))
}
func main() {
p := []Person{Person{Name:"john"}}
c := []Car{Car{"corolla"},Car{"jetta"}}
ModifyIt(&p,"Cars",&c)
// now parent is up to date, although I have no idea how I got here.
log.Println(p)
}
If anyone can post a better answer that clears up my confusion, that will be great. I've been having a really hard time learning golang.

GoRoutines and passing struct to original context

I have a configuration that defines a number of instances (SomeConfigItems) which have a thing() created for each of them.
That thing is a struct returned by an included package, which contains, among other things, a Price (float64) and a nested struct. The nested struct maintains a map of trades.
The problem is that I am able to loop through the thing.Streams.Trades and see all trades happening in real time from my main()'s for{} loop. I am not able to see an updated thing.Price even though it is set in the Handler on occasion.
I am having a hard time understanding how the nested structs can contain data but not Price. I feel as though I am missing something with scoping, goroutines, or possibly pointers for instantiation of new objects.
Any help would be appreciated, I will continue reading in the meantime. I've reduced the code to what seems relevant.
main.go:
package main
import thing
var Things []thing.Handler
for _, name := range SomeConfigItems {
handler := thing.New(name)
Things = append(Things, handler)
}
for {
for _, t := range Things {
log.Info("Price: ", t.Price) // This is set to 0 every iteration, but I can actively data in thing.Streams.Trades
}
}
thing.go:
package thing
import streams
type Handler struct {
Name string
Price float64
Streams streams.Streams
}
func New(name string) (h Handler, err error) {
stream, err := streams.New(strings.ToLower(name))
h = Handler{
Name: name,
Price: "0.0"
Streams: stream,
}
go h.handler()
return h, err
}
func (bot *Handler) handler() {
var currentPrice float64
for {
currentPrice = external.GetPrice(bot.Name).Price // Validated that this returns a float64
bot.Price = currentPrice // Verified that this is updated immediately after in this context.
// Unable to see Price updated from outer context.
}
}
streams.go:
package streams
type Streams struct {
Trades
}
type State struct {
Price string `json:"p"`
Quantity string `json:"q"`
}
type Trades struct {
Trades map[float64]float64
TradeMutex sync.Mutex
Updates chan State
}
func New(name string) (s Streams, err error) {
p := newTradeStream(name)
s = Streams{
Trades: p,
}
return s, err
}
func newTradeStream(name string) (ts Trades) {
ts = Trades{}
ts.Trades = make(map[float64]float64, MaxDepth)
ts.Updates = make(chan State, 500)
// ... Other watchdog code
return ts
}
Note:
I am added some debug logging in multiple locations. From within the Bot Handler, the price was printed (successfully), then updated, and then printed (successfully) again -- Showing no gap in the setting of Price from within the handler() function.
When adding the same type of debugging to the main() for{} loop, I tried setting an incrementing counter and assigning the value of thing.Price -- Printing thing.Price on each loop results in 0, even if I set the price (and validate it gets set) in the same loop, it is back to 0 on the next iteration.
This behavior is why I think that I am missing something very fundamental.
In Go, arguments are passed to functions by value -- meaning what the function gets is a copy of the value, not a reference to the variable. The same is true of the function receiver, and also the return list.
It's not the most elegant description, but for the sake of explanation, let's call this the "function wall." If the value being passed one way or the other is a pointer, the function still gets a copy, but it's a copy of a memory address, and so the pointer can be used to change the value of the variable on the other side of the wall. If it is a reference type, which uses a pointer in the implementation of the type, then again a change to the thing being pointed to can cross that wall. But otherwise the change does not cross the wall, which is one reason so many Go functions are written to return values instead of just modifying values.
Here's a runnable example:
package main
import (
"fmt"
)
type Car struct {
Color string
}
func (c Car) Change() { // c was passed by value, it's a copy
c.Color = "Red"
}
func main() {
ride := Car{"Blue"}
ride.Change()
fmt.Println(ride.Color)
}
Prints "Blue"
But two small changes:
func (c *Car) Change() { // here
c.Color = "Red"
}
func main() {
ride := &Car{"Blue"} // and here
ride.Change()
fmt.Println(ride.Color)
}
And now it prints "Red". Struct is not a reference type. So if you want modifications to a struct to cross the wall without using the return list to do it, use a pointer. Of course this only applies to values being passed via argument, return list, or receiver; and not to variables that are in scope on both sides of the wall; or to modifying the underlying value behind a reference type.
See also "Pointers Versus Values" in Effective Go, and "Go Data Structures" by Russ Cox.

How to pass slice of struct as pointer to a function and modify it?

I have a slice of struct []student, and I want to modify its content with function.
type student struct {
name string
age int
}
students := []student{
{"Doraemon", 30},
{"King Kong", 25},
}
Thus, I decided to pass it as a pointer. May I know how to pass the slice as a reference to a function?
func addAge (s *[]student) error { //this code has error
//everyone add 2 years old
for i, e := range *s {
s[i].age = s[i].age + 2
}
//make the first student much older
s[0].age = s[0].age + 5
return nil
}
I keep playing with Go Playground, but it gives many complains, such as
cannot range over s (type *[]student)
invalid operation: s[i] (type *[]student does not support indexing)
invalid indirect of s
...
How to precisely pass the reference of a slice of struct to a function? How to range the slice of struct? And how to change the value of the struct (modify the same struct in THE slice)?
I keep getting error while playing with s *[]student, range *s, s []student, s *[]*student ... so hard to get it correct...
sorry for my NEWBIE question, still learning GO... trying hard
Slices are passed by reference, so as long as you are modifying the existing slice content you should not explicitly pass a pointer.
package main
import (
"fmt"
)
type student struct {
name string
age int
}
func main() {
students := []student{
{"Doraemon", 30},
{"King Kong", 25},
}
err := addAge (students)
fmt.Println(students)
if err != nil {
fmt.Println("error")
}
}
func addAge (s []student) error {
for i, _ := range s {
s[i].age = 3
}
return nil
}
Now, for your addAdditinalStudent function you should actually use the append function. Plus, have in mind
..., since the slice header is always updated by a call to
append, you need to save the returned slice after the call. In fact,
the compiler won't let you call append without saving the result.
Slices#append
// add student
students = append(students, student{"Test", 33})
Go Playground
in Go you can pass items by value ([]student) or by reference ([]*student). When you want to operate on the values of a struct{} you should pass it to a function with its reference (the pointer).
So you can do something like this:
type student struct {
name string
age int
}
func addTwoYearsToAll(students []*student){
for _, s := range students {
s.age += 2
}
}
This way you're working with the same exact items you build when appending to the slice. Playground example.
Also take a look at Are Golang function parameter passed as copy-on-write?

How to change pointer slice in golang

I'm trying to get a better understanding of go. I created a little exercise for myself: pass a pointer slice to a function and modify it.
This is what I came up with:
package main
import (
"fmt"
"unsafe"
)
func main() {
var data *[]byte
fillData(data)
fmt.Println((*data)[0:5])
}
func fillData(data *[]byte) {
b := []byte("hello")
fmt.Println(b[0:5])
fmt.Println(string(b[0:5]))
data = (*[]byte)(unsafe.Pointer(&b[0]))
}
But it gives an invalid memory address or nil pointer dereference error. I know I wouldn't use something like this in real code but I was just curious how to pass a slice and modify it without returning it.
https://play.golang.org/p/_K5ltKKKNV
When you try to update data in fillData, you make two errors. First, you update the pointer rather than what it's pointed to. Second, data is a nil pointer, so writing through that pointer will cause a nil pointer error.
Here's one possible way to write the code. data starts as a zero'ed slice, and gets updated inside fillData. This will copy the slice information (len, cap, and pointer to array) from b to *data which means that data will share information with b (importantly, including sharing the underlying array).
package main
import "fmt"
func main() {
var data []byte
fillData(&data)
fmt.Println(data, data[0:5])
}
func fillData(data *[]byte) {
b := []byte("hello")
*data = b[0:1]
}
Another way would be to have data being a pointer, and updating it. Then you have to pass a double pointer into fillData. That would look like this:
package main
import "fmt"
func main() {
var data *[]byte
fillData(&data)
fmt.Println((*data)[0:5])
}
func fillData(data **[]byte) {
b := []byte("hello")
*data = &b
}
Finally, the best way to write this code isn't to use pointers at all, and just return the slice. Unlike C or C++, it's rarely needed to use "output" parameters to functions. That's because go allows multiple return values.
package main
import "fmt"
func main() {
data := getData()
fmt.Println(data, data[0:5])
}
func getData() []byte {
return []byte("hello")[:1]
}

Resources