I have a configuration that defines a number of instances (SomeConfigItems) which have a thing() created for each of them.
That thing is a struct returned by an included package, which contains, among other things, a Price (float64) and a nested struct. The nested struct maintains a map of trades.
The problem is that I am able to loop through the thing.Streams.Trades and see all trades happening in real time from my main()'s for{} loop. I am not able to see an updated thing.Price even though it is set in the Handler on occasion.
I am having a hard time understanding how the nested structs can contain data but not Price. I feel as though I am missing something with scoping, goroutines, or possibly pointers for instantiation of new objects.
Any help would be appreciated, I will continue reading in the meantime. I've reduced the code to what seems relevant.
main.go:
package main
import thing
var Things []thing.Handler
for _, name := range SomeConfigItems {
handler := thing.New(name)
Things = append(Things, handler)
}
for {
for _, t := range Things {
log.Info("Price: ", t.Price) // This is set to 0 every iteration, but I can actively data in thing.Streams.Trades
}
}
thing.go:
package thing
import streams
type Handler struct {
Name string
Price float64
Streams streams.Streams
}
func New(name string) (h Handler, err error) {
stream, err := streams.New(strings.ToLower(name))
h = Handler{
Name: name,
Price: "0.0"
Streams: stream,
}
go h.handler()
return h, err
}
func (bot *Handler) handler() {
var currentPrice float64
for {
currentPrice = external.GetPrice(bot.Name).Price // Validated that this returns a float64
bot.Price = currentPrice // Verified that this is updated immediately after in this context.
// Unable to see Price updated from outer context.
}
}
streams.go:
package streams
type Streams struct {
Trades
}
type State struct {
Price string `json:"p"`
Quantity string `json:"q"`
}
type Trades struct {
Trades map[float64]float64
TradeMutex sync.Mutex
Updates chan State
}
func New(name string) (s Streams, err error) {
p := newTradeStream(name)
s = Streams{
Trades: p,
}
return s, err
}
func newTradeStream(name string) (ts Trades) {
ts = Trades{}
ts.Trades = make(map[float64]float64, MaxDepth)
ts.Updates = make(chan State, 500)
// ... Other watchdog code
return ts
}
Note:
I am added some debug logging in multiple locations. From within the Bot Handler, the price was printed (successfully), then updated, and then printed (successfully) again -- Showing no gap in the setting of Price from within the handler() function.
When adding the same type of debugging to the main() for{} loop, I tried setting an incrementing counter and assigning the value of thing.Price -- Printing thing.Price on each loop results in 0, even if I set the price (and validate it gets set) in the same loop, it is back to 0 on the next iteration.
This behavior is why I think that I am missing something very fundamental.
In Go, arguments are passed to functions by value -- meaning what the function gets is a copy of the value, not a reference to the variable. The same is true of the function receiver, and also the return list.
It's not the most elegant description, but for the sake of explanation, let's call this the "function wall." If the value being passed one way or the other is a pointer, the function still gets a copy, but it's a copy of a memory address, and so the pointer can be used to change the value of the variable on the other side of the wall. If it is a reference type, which uses a pointer in the implementation of the type, then again a change to the thing being pointed to can cross that wall. But otherwise the change does not cross the wall, which is one reason so many Go functions are written to return values instead of just modifying values.
Here's a runnable example:
package main
import (
"fmt"
)
type Car struct {
Color string
}
func (c Car) Change() { // c was passed by value, it's a copy
c.Color = "Red"
}
func main() {
ride := Car{"Blue"}
ride.Change()
fmt.Println(ride.Color)
}
Prints "Blue"
But two small changes:
func (c *Car) Change() { // here
c.Color = "Red"
}
func main() {
ride := &Car{"Blue"} // and here
ride.Change()
fmt.Println(ride.Color)
}
And now it prints "Red". Struct is not a reference type. So if you want modifications to a struct to cross the wall without using the return list to do it, use a pointer. Of course this only applies to values being passed via argument, return list, or receiver; and not to variables that are in scope on both sides of the wall; or to modifying the underlying value behind a reference type.
See also "Pointers Versus Values" in Effective Go, and "Go Data Structures" by Russ Cox.
Related
This question already has an answer here:
Golang static identifier resolution
(1 answer)
Closed 4 months ago.
How can I determine the type of a variable through static analysis?
Suppose I have the following code:
func doSomething(x interface{}) {}
func main() {
p := Person()
doSomething(p)
}
And I want to analyze doSomething(person), is it possible to get the type of Person through static analysis?
What if there were multiple levels of assignment?
p1 := Person()
p2 := p1
doSomething(p2)
or
parent := Parent()
p := Parent.Child() // type Person
doSomething(p)
The use case is that I have a generic function that is commonly used throughout the (very large) codebase, and would like to introduce a new type safe version of this function. To do this, I hope to automatically determine the "type" of the function and refactor it accordingly:
// old
DB.InsertRow(person)
// new
Person.InsertRow(person)
Finding the type of an expression through static analysis is non-trivial, and sometimes not possible, for details see Golang static identifier resolution.
The use case is that I have a generic function that is commonly used throughout the (very large) codebase, and would like to introduce a new type safe version of this function. To do this, I hope to automatically determine the "type" of the function and refactor it accordingly:
// old
DB.InsertRow(person)
// new
Person.InsertRow(person)
Just for refactoring purposes, I don't think it is worth the hassle to implement it.
What you may do is change the signature of DB.InsertRow() temporarily to accept only a specific type such as int or your custom type you're sure is not used anywhere (e.g. type tempFoo struct{}).
To what end? Doing so, the compiler will do the hard work for you. You will see error messages showing exactly the types your codebase is trying to pass to DB.InsertRow(), so I'd say mission accomplished.
For example this code compiles:
func doSomething(x interface{}) {}
func main() {
doSomething(image.Pt(1, 2))
doSomething("abc")
doSomething(image.Rect) // image.Rect is a function which we don't call,
// so we're passing a value of a function type here
}
If we change doSomething():
func doSomething(x int) {}
We get the types we're seeking for from the compiler:
./prog.go:10:14: cannot use image.Pt(1, 2) (value of type image.Point) as type int in argument to doSomething
./prog.go:11:14: cannot use "abc" (untyped string constant) as int value in argument to doSomething
./prog.go:12:14: cannot use image.Rect (value of type func(x0 int, y0 int, x1 int, y1 int) image.Rectangle) as type int in argument to doSomething
Using the advice from Golang static identifier resolution to use golang.org/x/tools/go/types, I found that this was pretty straight forward to do with the golang.org/x/tools/go/analysis package, which has the types info available alongside the parsed ast.
This was my solution:
package rewriter
import (
"go/ast"
"golang.org/x/tools/go/analysis"
"golang.org/x/tools/go/analysis/passes/inspect"
"golang.org/x/tools/go/ast/inspector"
)
func run(pass *analysis.Pass) (interface{}, error) {
inspect := pass.ResultOf[inspect.Analyzer].(*inspector.Inspector)
nodeFilter := []ast.Node{
(*ast.CallExpr)(nil),
}
inspect.Nodes(nodeFilter, func(node ast.Node, push bool) bool {
callExpr, ok := node.(*ast.CallExpr)
if !ok {
return true
}
funcExpr, ok := callExpr.Fun.(*ast.SelectorExpr)
if !ok {
return true
}
// check method name
if funcExpr.Sel.Name != "doSomething" {
return true
}
for _, arg := range callExpr.Args {
// lookup type of the arg
argType := pass.TypesInfo.Types[arg].Type
if argType.String() == "*rewriter.Person" {
// do whatever you want here
}
}
return false
})
return nil, nil
}
One can augment this to look at the receiver of the method and add refactoring logic as needed (using analysis.Diagnostic).
If I have function like this
func TestMethod ( d interface{} ) {
}
If I am calling this as
TestMethod("syz")
Is this pass by value or pass by pointer ?
To summarise some of the discussion in the comments and answer the question:
In go everything in Go is passed by value. In this case the value is an interface type, which is represented as a pointer to the data and a pointer to the type of the interface.
This can be verified by running the following snippet (https://play.golang.org/p/9xTsetTDfZq):
func main() {
var s string = "syz"
read(s)
}
//go:noinline
func read(i interface{}) {
println(i)
}
which will return (0x999c0,0x41a788), one pointer to the data and one pointer to the type of interface.
Updated: Answer and comments above are correct. Just a lite bit of extra information.
Some theory
Passing by reference enables function members, methods, properties,
indexers, operators, and constructors to change the value of the
parameters and have that change persist in the calling environment.
Little code sniped to check how function calls work in GO for pointers
package main_test
import (
"testing"
)
func MyMethod(d interface{}) {
// assume that we received a pointer to string
// here we reassign pointer
newStr := "bar"
d = &newStr
}
func TestValueVsReference(t *testing.T) {
data := "foo"
dataRef := &data
// sending poiner to sting into function that reassigns that pointer in its body
MyMethod(dataRef)
// check is pointer we sent changed
if *dataRef != "foo" {
t.Errorf("want %q, got %q", "bar", *dataRef)
}
// no error, our outer pointer was not changed inside function
// confirms that pointer was sent as value
}
I've had difficulty learning the basics of reflect, pointers and interface in go, so here's another entry level question I can't seem to figure out.
This code does what I want it to do - I'm using reflect to add another record to a slice that's typed as an interface.
package main
import (
"reflect"
"log"
)
type Person struct {
Name string
}
func Add(slice interface{}) {
s := reflect.ValueOf(slice).Elem()
// in my actual code, p is declared via the use of reflect.New([Type])
p := Person{Name:"Sam"}
s.Set(reflect.Append(s,reflect.ValueOf(p)))
}
func main() {
p := []Person{}
Add(&p)
log.Println(p)
}
If I changed the Add and main function to this, things don't work the way I want it to.
func Add(slice interface{}) {
s := reflect.ValueOf(&slice).Elem()
p := Person{Name:"Sam"}
s.Set(reflect.Append(reflect.ValueOf(slice),reflect.ValueOf(p)))
log.Println(s)
}
func main() {
p := []Person{}
Add(p)
log.Println(p)
}
That is, the log.Println(p) at the end doesn't show a slice with the record Sam in it like the way I had hoped. So my question is whether it's possible for me to have Add() receive a slice that is not a pointer, and for me to still write some code in Add() that will produce the outcome shown in my first scenario?
A lot of my recent questions dance around this kind of subject, so it's still taking me a while to figure out how to use the reflect package effectively.
No, it's not possible to append to a slice in a function without passing in a pointer to the slice. This isn't related to reflection, but to how variables are passed in to functions. Here's the same code, modified to not use reflection:
package main
import (
"log"
)
type Person struct {
Name string
}
func AddWithPtr(slicep interface{}) {
sp := slicep.(*[]Person)
// This modifies p1 itself, since *sp IS p1
*sp = append(*sp, Person{"Sam"})
}
func Add(slice interface{}) {
// s is now a copy of p2
s := slice.([]Person)
sp := &s
// This modifies a copy of p2 (i.e. s), not p2 itself
*sp = append(*sp, Person{"Sam"})
}
func main() {
p1 := []Person{}
// This passes a reference to p1
AddWithPtr(&p1)
log.Println("Add with pointer: ", p1)
p2 := []Person{}
// This passes a copy of p2
Add(p2)
log.Println("Add without pointer:", p2)
}
(Above, when it says 'copy' of the slice, it doesn't mean the copy of the underlying data - just the slice)
When you pass in a slice, the function effectively gets a new slice that refers to the same data as the original. Appending to the slice in the function increases the length of the new slice, but doesn't change the length of the original slice that was passed in. That's why the original slice remains unchanged.
I am passing a pointer to a string, to a method which takes an interface (I have multiple versions of the method, with different receivers, so I am trying to work with empty interfaces, so that I don't end up with a ton of boilerplate madness. Essentially, I want to populate the string with the first value in the slice. I am able to see the value get populated inside the function, but then for some reason, in my application which calls it, tha value doesn't change. I suspect this is some kind of pointer arithmetic problem, but could really use some help!
I have the following interface :
type HeadInterface interface{
Head(interface{})
}
And then I have the following functions :
func Head(slice HeadInterface, result interface{}){
slice.Head(result)
}
func (slice StringSlice) Head(result interface{}){
result = reflect.ValueOf(slice[0])
fmt.Println(result)
}
and... here is my call to the function from an application which calls the mehtod...
func main(){
test := x.StringSlice{"Phil", "Jessica", "Andrea"}
// empty result string for population within the function
var result string = ""
// Calling the function (it is a call to 'x.Head' because I lazily just called th import 'x')
x.Head(test, &result)
// I would have thought I would have gotten "Phil" here, but instead, it is still empty, despite the Println in the function, calling it "phil.
fmt.Println(result)
}
*NOTE : I am aware that getting the first element doesn't need to be this complicated, and could be slice[0] as a straight assertion, but this is more of an exercise in reusable code, and also in trying to get a grasp of pointers, so please don't point out that solution - I would get much more use out of a solution to my actual problem here * :)
As you said in your NOTE, I'm pretty sure this doesn't have to be this complicated, but to make it work in your context:
package main
import (
"fmt"
"reflect"
)
type HeadInterface interface {
Head(interface{})
}
func Head(slice HeadInterface, result interface{}) {
slice.Head(result)
}
type StringSlice []string
func (slice StringSlice) Head(result interface{}) {
switch result := result.(type) {
case *string:
*result = reflect.ValueOf(slice[0]).String()
fmt.Println("inside Head:", *result)
default:
panic("can't handle this type!")
}
}
func main() {
test := StringSlice{"Phil", "Jessica", "Andrea"}
// empty result string for population within the function
var result string = ""
// Calling the function (it is a call to 'x.Head' because I lazily just called th import 'x')
Head(test, &result)
// I would have thought I would have gotten "Phil" here, but instead, it is still empty, despite the Println in the function, calling it "phil.
fmt.Println("outside:", result)
}
The hard part about working with interface{} is that it's hard to be specific about a type's behavior given that interface{} is the most un-specific type. To modify a variable that you pass as a pointer to a function, you have to use the asterisk (dereference) (for example *result) on the variable in order to change the value it points to, not the pointer itself. But to use the asterisk, you have to know it's actually a pointer (something interface{} doesn't tell you) so that's why I used the type switch to be sure it's a pointer to a string.
I have a slice of struct []student, and I want to modify its content with function.
type student struct {
name string
age int
}
students := []student{
{"Doraemon", 30},
{"King Kong", 25},
}
Thus, I decided to pass it as a pointer. May I know how to pass the slice as a reference to a function?
func addAge (s *[]student) error { //this code has error
//everyone add 2 years old
for i, e := range *s {
s[i].age = s[i].age + 2
}
//make the first student much older
s[0].age = s[0].age + 5
return nil
}
I keep playing with Go Playground, but it gives many complains, such as
cannot range over s (type *[]student)
invalid operation: s[i] (type *[]student does not support indexing)
invalid indirect of s
...
How to precisely pass the reference of a slice of struct to a function? How to range the slice of struct? And how to change the value of the struct (modify the same struct in THE slice)?
I keep getting error while playing with s *[]student, range *s, s []student, s *[]*student ... so hard to get it correct...
sorry for my NEWBIE question, still learning GO... trying hard
Slices are passed by reference, so as long as you are modifying the existing slice content you should not explicitly pass a pointer.
package main
import (
"fmt"
)
type student struct {
name string
age int
}
func main() {
students := []student{
{"Doraemon", 30},
{"King Kong", 25},
}
err := addAge (students)
fmt.Println(students)
if err != nil {
fmt.Println("error")
}
}
func addAge (s []student) error {
for i, _ := range s {
s[i].age = 3
}
return nil
}
Now, for your addAdditinalStudent function you should actually use the append function. Plus, have in mind
..., since the slice header is always updated by a call to
append, you need to save the returned slice after the call. In fact,
the compiler won't let you call append without saving the result.
Slices#append
// add student
students = append(students, student{"Test", 33})
Go Playground
in Go you can pass items by value ([]student) or by reference ([]*student). When you want to operate on the values of a struct{} you should pass it to a function with its reference (the pointer).
So you can do something like this:
type student struct {
name string
age int
}
func addTwoYearsToAll(students []*student){
for _, s := range students {
s.age += 2
}
}
This way you're working with the same exact items you build when appending to the slice. Playground example.
Also take a look at Are Golang function parameter passed as copy-on-write?