I'd like to iterate over the fields in a struct and prompt for string values to string fields, doing this recursively for fields that are pointers to structs.
Currently this is what I've tried, but I get an error when trying to set this value in the pointer's string field.
package main
import (
"fmt"
"reflect"
)
type Table struct {
PK *Field
}
type Field struct {
Name string
}
func main() {
PopulateStruct(&Table{})
}
func PopulateStruct(a interface{}) interface {} {
typeOf := reflect.TypeOf(a)
valueOf := reflect.ValueOf(a)
for i := 0; i < typeOf.Elem().NumField(); i++ {
switch typeOf.Elem().Field(i).Type.Kind() {
case reflect.String:
fmt.Print(typeOf.Elem().Field(i).Name)
var s string
fmt.Scanf("%s", &s)
valueOf.Elem().Field(i).SetString(s)
case reflect.Ptr:
ptr := reflect.New(valueOf.Elem().Field(i).Type())
PopulateStruct(ptr.Elem().Interface())
valueOf.Elem().Field(i).Set(ptr)
}
}
}
Expecting the return value to include an initialised struct with the pointers string field set.
Getting an error when setting the pointer's string field.
panic: reflect: call of reflect.Value.Field on zero Value
I dropped your code as-is into the Go Playground and it doesn't build because PopulateStruct is declared as returning interface{} but does not actually return anything. Removing the declared return type produces the panic you mention.
This is because at entry to the outer PopulateStruct call, you have a valid pointer, pointing to a zero-valued Table. A zero-valued Table has one element: a nil pointer in it of type *Field. Your loop therefore runs once and finds a reflect.Ptr, as you expected. Adding more diagnostic print messages helps see what's happening:
fmt.Printf("PopulateStruct: I have typeOf=%v, valueOf=%v\n", typeOf, valueOf)
for i := 0; i < typeOf.Elem().NumField(); i++ {
switch typeOf.Elem().Field(i).Type.Kind() {
// ... snipped some code ...
case reflect.Ptr:
ptr := reflect.New(valueOf.Elem().Field(i).Type())
fmt.Println("after allocating ptr, we have:", ptr.Type(), ptr,
"but its Elem is:", ptr.Elem().Type(), ptr.Elem())
This prints:
PopulateStruct: I have typeOf=*main.Table, valueOf=&{<nil>}
after allocating ptr, we have: **main.Field 0x40c138 but its Elem is: *main.Field <nil>
Given the way PopulateStruct itself is constructed, we must actually allocate a real Field instance now, before calling PopulateStruct. We can do this with:
p2 := ptr.Elem()
ptr.Elem().Set(reflect.New(p2.Type().Elem()))
(code borrowed from json.Unmarshal). Now we can fill in this Field, which has one field named Name of type String.
The overall strategy here is not that great, in my opinion: filling-in probably should take a generic pointer, not specifically a pointer-to-struct pointer. You can then emulate the indirect function in the json unmarshaller. However, the addition of these two lines—creating the target object and making the allocated pointer point to it—suffices to make your existing code run.
(Alternatively, you could just create and return a whole instance from scratch, in which case all you need is the type—but I'm assuming you have a pattern in which only some fields are nil.)
Here's the complete Go Playground example. I made a few other changes as there's nothing to scan from when using the playground.
Related
If I have function like this
func TestMethod ( d interface{} ) {
}
If I am calling this as
TestMethod("syz")
Is this pass by value or pass by pointer ?
To summarise some of the discussion in the comments and answer the question:
In go everything in Go is passed by value. In this case the value is an interface type, which is represented as a pointer to the data and a pointer to the type of the interface.
This can be verified by running the following snippet (https://play.golang.org/p/9xTsetTDfZq):
func main() {
var s string = "syz"
read(s)
}
//go:noinline
func read(i interface{}) {
println(i)
}
which will return (0x999c0,0x41a788), one pointer to the data and one pointer to the type of interface.
Updated: Answer and comments above are correct. Just a lite bit of extra information.
Some theory
Passing by reference enables function members, methods, properties,
indexers, operators, and constructors to change the value of the
parameters and have that change persist in the calling environment.
Little code sniped to check how function calls work in GO for pointers
package main_test
import (
"testing"
)
func MyMethod(d interface{}) {
// assume that we received a pointer to string
// here we reassign pointer
newStr := "bar"
d = &newStr
}
func TestValueVsReference(t *testing.T) {
data := "foo"
dataRef := &data
// sending poiner to sting into function that reassigns that pointer in its body
MyMethod(dataRef)
// check is pointer we sent changed
if *dataRef != "foo" {
t.Errorf("want %q, got %q", "bar", *dataRef)
}
// no error, our outer pointer was not changed inside function
// confirms that pointer was sent as value
}
I've been struggling with the reflect package. This code below does what I expect:
package main
import (
"reflect"
"log"
)
type Car struct {
Model string
}
type Person struct {
Name string
Cars []Car
}
func ModifyIt(parent interface{},fieldName string, val interface{}) {
slice := reflect.ValueOf(parent).Elem()
nth := slice.Index(0)
//row := nth.Interface() // this line causes errors
row := nth.Interface().(Person)
elem := reflect.ValueOf(&row).Elem()
field := elem.FieldByName(fieldName)
log.Println(field.CanSet())
}
func main() {
p := []Person{Person{Name:"john"}}
c := []Car{Car{"corolla"},Car{"jetta"}}
ModifyIt(&p,"Cars",&c)
}
However, if I replace the line row := nth.Interface().(Person) with row := nth.Interface(), that is I remove the type assertion, then I get the error:
panic: reflect: call of reflect.Value.FieldByName on interface Value
on line "field := elem.FieldByName(fieldName)
I've tried a bunch of other things the last few hours like trying to do reflect.TypeOf(), reflect.Indirect() etc... on some of the other variables but with no success.
I've read some other questions like these:
reflect: call of reflect.Value.FieldByName on ptr Value
Set a struct field with field type of a interface
Golang reflection: Can't set fields of interface wrapping a struct
They seem to suggest that I don't have a good understanding of how pointers or interfaces work.
So my question is, how do I go about setting the field of a struct when the struct is typed as an interface?
UPDATE
I posted a solution as an answer, but I have no confidence in whether it is the proper or safe way of doing things. I hope someone can explain, or post a better solution.
Try this:
func ModifyIt(slice interface{}, fieldName string, newVal interface{}) {
// Create a value for the slice.
v := reflect.ValueOf(slice)
// Get the first element of the slice.
e := v.Index(0)
// Get the field of the slice element that we want to set.
f := e.FieldByName(fieldName)
// Set the value!
f.Set(reflect.ValueOf(newVal))
}
Call it like this:
p := []Person{Person{Name: "john"}}
c := []Car{Car{"corolla"}, Car{"jetta"}}
ModifyIt(p, "Cars", c)
Note that the call passes the slices directly instead of using pointers to slices. The pointers are not needed and add extra complexity.
Run it on the Playground.
Out of sheer luck, I finally got something to work.
I pieced together a bunch of random things I read with very little rhyme or reason. I even tried reading the Laws of Reflection on the Golang site, but I don't think I have a good grasp of how it relates to why I couldn't set variables typed as interface{}. In general, I still don't understand what I did.
My solution below is littered with comments to indicate my confusion, and lack of confidence in whether I did things properly or safely.
package main
import (
"reflect"
"log"
)
type Car struct {
Model string
}
type Person struct {
Name string
Cars []Car
}
func ModifyIt(parent interface{},fieldName string, val interface{}) {
log.Println(parent)
slice := reflect.ValueOf(parent).Elem()
nth := slice.Index(0)
row := nth.Interface()
log.Println(nth.CanSet()) // I can set this nth item
// I think I have a to make a copy, don't fully understand why this is necessary
newitem := reflect.New(reflect.ValueOf(row).Type())
newelem := newitem.Elem()
field := newelem.FieldByName(fieldName)
// I need to copy the values over from the old nth row to this new item
for c:=0; c<nth.NumField(); c++ {
newelem.Field(c).Set(reflect.Indirect(nth.Field(c)))
}
// now I can finally set the field for some reason I don't understand
field.Set(reflect.ValueOf(val).Elem())
// now that newitem has new contents in the field object, I need to overwrite the nth item with new item
// I don't know why I'm doing it, but I'll do it
// I also don't fully understand why I have to use Indirect sometimes, and not other times...it seems interchangeable with ValueOf(something).Elem(), I'm confused....
nth.Set(reflect.Indirect(newitem))
}
func main() {
p := []Person{Person{Name:"john"}}
c := []Car{Car{"corolla"},Car{"jetta"}}
ModifyIt(&p,"Cars",&c)
// now parent is up to date, although I have no idea how I got here.
log.Println(p)
}
If anyone can post a better answer that clears up my confusion, that will be great. I've been having a really hard time learning golang.
I am passing a pointer to a string, to a method which takes an interface (I have multiple versions of the method, with different receivers, so I am trying to work with empty interfaces, so that I don't end up with a ton of boilerplate madness. Essentially, I want to populate the string with the first value in the slice. I am able to see the value get populated inside the function, but then for some reason, in my application which calls it, tha value doesn't change. I suspect this is some kind of pointer arithmetic problem, but could really use some help!
I have the following interface :
type HeadInterface interface{
Head(interface{})
}
And then I have the following functions :
func Head(slice HeadInterface, result interface{}){
slice.Head(result)
}
func (slice StringSlice) Head(result interface{}){
result = reflect.ValueOf(slice[0])
fmt.Println(result)
}
and... here is my call to the function from an application which calls the mehtod...
func main(){
test := x.StringSlice{"Phil", "Jessica", "Andrea"}
// empty result string for population within the function
var result string = ""
// Calling the function (it is a call to 'x.Head' because I lazily just called th import 'x')
x.Head(test, &result)
// I would have thought I would have gotten "Phil" here, but instead, it is still empty, despite the Println in the function, calling it "phil.
fmt.Println(result)
}
*NOTE : I am aware that getting the first element doesn't need to be this complicated, and could be slice[0] as a straight assertion, but this is more of an exercise in reusable code, and also in trying to get a grasp of pointers, so please don't point out that solution - I would get much more use out of a solution to my actual problem here * :)
As you said in your NOTE, I'm pretty sure this doesn't have to be this complicated, but to make it work in your context:
package main
import (
"fmt"
"reflect"
)
type HeadInterface interface {
Head(interface{})
}
func Head(slice HeadInterface, result interface{}) {
slice.Head(result)
}
type StringSlice []string
func (slice StringSlice) Head(result interface{}) {
switch result := result.(type) {
case *string:
*result = reflect.ValueOf(slice[0]).String()
fmt.Println("inside Head:", *result)
default:
panic("can't handle this type!")
}
}
func main() {
test := StringSlice{"Phil", "Jessica", "Andrea"}
// empty result string for population within the function
var result string = ""
// Calling the function (it is a call to 'x.Head' because I lazily just called th import 'x')
Head(test, &result)
// I would have thought I would have gotten "Phil" here, but instead, it is still empty, despite the Println in the function, calling it "phil.
fmt.Println("outside:", result)
}
The hard part about working with interface{} is that it's hard to be specific about a type's behavior given that interface{} is the most un-specific type. To modify a variable that you pass as a pointer to a function, you have to use the asterisk (dereference) (for example *result) on the variable in order to change the value it points to, not the pointer itself. But to use the asterisk, you have to know it's actually a pointer (something interface{} doesn't tell you) so that's why I used the type switch to be sure it's a pointer to a string.
Here is a piece of code play.google.org that runs without any problem:
package main
import (
"fmt"
)
func PrintAnonymous(v struct {
i int
s string
}) {
fmt.Printf("%d: %s\n", v.i, v.s)
}
func PrintAnonymous2(v struct{}) {
fmt.Println("Whatever")
}
func main() {
value := struct {
i int
s string
}{
0, "Hello, world!",
}
PrintAnonymous(value)
PrintAnonymous2(struct{}{})
}
However, if the PrintAnonymous() function exists in another package (let's say, temp), the code will not work:
cannot use value (type struct { i int; s string })
as type struct { i int; s string } in argument to temp.PrintAnonymous
My question are:
Is there a way to call a (public) function with anonymous struct as a parameter (a.k.a. PrintAnonymous() above)?
A function with empty struct as a parameter (a.k.a. PrintAnonymous2() above) can be called even if it exists in another package. Is this a special case?
Well, I know that I can always name the struct to solve the problem, I'm just curious about this, and wonder why it seems that this is not allowed.
The fields of your anonymous struct type are unexported. This means you cannot create values of this struct and specify values for the fields from another package. Spec: Composite literals:
It is an error to specify an element for a non-exported field of a struct belonging to a different package.
If you change the struct definition to export the fields, then it will work because all fields can be assigned to by other packages (see Siu Ching Pong -Asuka Kenji-'s answer).
This is the case with the empty struct (with no fields) too: the empty struct has no fields, thus it has no unexported fields, so you're allowed to pass a value of that.
You can call the function with unmodified struct (with unexported fields) via reflection though. You can obtain the reflect.Type of the PrintAnonymous() function, and you can use Type.In() to get the reflect.Type of its first parameter. That is the anonymous struct we want to pass a value for. And you can construct a value of that type using reflect.New(). This will be a reflect.Value, wrapping a pointer to the zero value of the anonymous struct. Sorry, you can't have a struct value with fields having non-zero values (for reason mentioned above).
This is how it could look like:
v := reflect.ValueOf(somepackage.PrintAnonymous)
paramt := v.Type().In(0)
v.Call([]reflect.Value{reflect.New(paramt).Elem()})
This will print:
0:
0 is zero value for int, and "" empty string for string.
For deeper inside into the type system and structs with unexported fields, see related questions:
Identify non builtin-types using reflect
How to clone a structure with unexported field?
Interestingly (this is a bug, see linked issue below), using reflection, you can use a value of your own anonymous struct type (with matching, unexported fields), and in this case we can use values other than the zero value of the struct fields:
value := struct {
i int
s string
}{
1, "Hello, world!",
}
v.Call([]reflect.Value{reflect.ValueOf(value)})
Above runs (without panic):
1: Hello, world!
The reason why this is allowed is due to a bug in the compiler. See the example code below:
s := struct{ i int }{2}
t := reflect.TypeOf(s)
fmt.Printf("Name: %q, PkgPath: %q\n", t.Name(), t.PkgPath())
fmt.Printf("Name: %q, PkgPath: %q\n", t.Field(0).Name, t.Field(0).PkgPath)
t2 := reflect.TypeOf(subplay.PrintAnonymous).In(0)
fmt.Printf("Name: %q, PkgPath: %q\n", t2.Name(), t2.PkgPath())
fmt.Printf("Name: %q, PkgPath: %q\n", t2.Field(0).Name, t2.Field(0).PkgPath)
Output is:
Name: "", PkgPath: ""
Name: "i", PkgPath: "main"
Name: "", PkgPath: ""
Name: "i", PkgPath: "main"
As you can see the unexported field i in both anonymous struct types (in main package and in somepackage as parameter to PrintAnonymous() function) –falsely– report the same package, and thus their type will be equal:
fmt.Println(t == t2) // Prints true
Note: I consider this a flaw: if this is allowed using reflection, then this should be possible without using reflection too. If without reflection the compile-time error is justified, then using reflection should result in runtime panic. I opened an issue for this, you can follow it here: issue #16616. Fix currently aims Go 1.8.
Oh! I just realized that the field names are in lowercase, and thus not public! Changing the first letter of the field names to uppercase solves the problem:
package main
import (
"temp"
)
func main() {
value := struct {
I int
S string
}{
0, "Hello, world!",
}
temp.PrintAnonymous(value)
temp.PrintAnonymous2(struct{}{})
}
package temp
import (
"fmt"
)
func PrintAnonymous(v struct{I int; S string}) {
fmt.Printf("%d: %s\n", v.I, v.S)
}
func PrintAnonymous2(v struct{}) {
fmt.Println("Whatever")
}
I have a struct type with a *int64 field.
type SomeType struct {
SomeField *int64
}
At some point in my code, I want to declare a literal of this (say, when I know said value should be 0, or pointing to a 0, you know what I mean)
instance := SomeType{
SomeField: &0,
}
...except this doesn't work
./main.go:xx: cannot use &0 (type *int) as type *int64 in field value
So I try this
instance := SomeType{
SomeField: &int64(0),
}
...but this also doesn't work
./main.go:xx: cannot take the address of int64(0)
How do I do this? The only solution I can come up with is using a placeholder variable
var placeholder int64
placeholder = 0
instance := SomeType{
SomeField: &placeholder,
}
Note: the &0 syntax works fine when it's a *int instead of an *int64. Edit: no it does not. Sorry about this.
Edit:
Aparently there was too much ambiguity to my question. I'm looking for a way to literally state a *int64. This could be used inside a constructor, or to state literal struct values, or even as arguments to other functions. But helper functions or using a different type are not solutions I'm looking for.
The Go Language Specification (Address operators) does not allow to take the address of a numeric constant (not of an untyped nor of a typed constant).
The operand must be addressable, that is, either a variable, pointer indirection, or slice indexing operation; or a field selector of an addressable struct operand; or an array indexing operation of an addressable array. As an exception to the addressability requirement, x [in the expression of &x] may also be a (possibly parenthesized) composite literal.
For reasoning why this isn't allowed, see related question: Find address of constant in go. A similar question (similarly not allowed to take its address): How can I store reference to the result of an operation in Go?
0) Generic solution (from Go 1.18)
Generics are added in Go 1.18. This means we can create a single, generic Ptr() function that returns a pointer to whatever value we pass to it. Hopefully it'll get added to the standard library. Until then, you can use github.com/icza/gog, the gog.Ptr() function (disclosure: I'm the author).
This is how it can look like:
func Ptr[T any](v T) *T {
return &v
}
Testing it:
i := Ptr(2)
log.Printf("%T %v", i, *i)
s := Ptr("abc")
log.Printf("%T %v", s, *s)
x := Ptr[any](nil)
log.Printf("%T %v", x, *x)
Which will output (try it on the Go Playground):
2009/11/10 23:00:00 *int 2
2009/11/10 23:00:00 *string abc
2009/11/10 23:00:00 *interface {} <nil>
Your other options (prior to Go 1.18) (try all on the Go Playground):
1) With new()
You can simply use the builtin new() function to allocate a new zero-valued int64 and get its address:
instance := SomeType{
SomeField: new(int64),
}
But note that this can only be used to allocate and obtain a pointer to the zero value of any type.
2) With helper variable
Simplest and recommended for non-zero elements is to use a helper variable whose address can be taken:
helper := int64(2)
instance2 := SomeType{
SomeField: &helper,
}
3) With helper function
Note: Helper functions to acquire a pointer to a non-zero value are available in my github.com/icza/gox library, in the gox package, so you don't have to add these to all your projects where you need it.
Or if you need this many times, you can create a helper function which allocates and returns an *int64:
func create(x int64) *int64 {
return &x
}
And using it:
instance3 := SomeType{
SomeField: create(3),
}
Note that we actually didn't allocate anything, the Go compiler did that when we returned the address of the function argument. The Go compiler performs escape analysis, and allocates local variables on the heap (instead of the stack) if they may escape the function. For details, see Is returning a slice of a local array in a Go function safe?
4) With a one-liner anonymous function
instance4 := SomeType{
SomeField: func() *int64 { i := int64(4); return &i }(),
}
Or as a (shorter) alternative:
instance4 := SomeType{
SomeField: func(i int64) *int64 { return &i }(4),
}
5) With slice literal, indexing and taking address
If you would want *SomeField to be other than 0, then you need something addressable.
You can still do that, but that's ugly:
instance5 := SomeType{
SomeField: &[]int64{5}[0],
}
fmt.Println(*instance2.SomeField) // Prints 5
What happens here is an []int64 slice is created with a literal, having one element (5). And it is indexed (0th element) and the address of the 0th element is taken. In the background an array of [1]int64 will also be allocated and used as the backing array for the slice. So there is a lot of boilerplate here.
6) With a helper struct literal
Let's examine the exception to the addressability requirements:
As an exception to the addressability requirement, x [in the expression of &x] may also be a (possibly parenthesized) composite literal.
This means that taking the address of a composite literal, e.g. a struct literal is ok. If we do so, we will have the struct value allocated and a pointer obtained to it. But if so, another requirement will become available to us: "field selector of an addressable struct operand". So if the struct literal contains a field of type int64, we can also take the address of that field!
Let's see this option in action. We will use this wrapper struct type:
type intwrapper struct {
x int64
}
And now we can do:
instance6 := SomeType{
SomeField: &(&intwrapper{6}).x,
}
Note that this
&(&intwrapper{6}).x
means the following:
& ( (&intwrapper{6}).x )
But we can omit the "outer" parenthesis as the address operator & is applied to the result of the selector expression.
Also note that in the background the following will happen (this is also a valid syntax):
&(*(&intwrapper{6})).x
7) With helper anonymous struct literal
The principle is the same as with case #6, but we can also use an anonymous struct literal, so no helper/wrapper struct type definition needed:
instance7 := SomeType{
SomeField: &(&struct{ x int64 }{7}).x,
}
Use a function which return an address of an int64 variable to solve the problem.
In the below code we use function f which accepts an integer and
returns a pointer value which holds the address of the integer. By using this method we can easily solve the above problem.
type myStr struct {
url *int64
}
func main() {
f := func(s int64) *int64 {
return &s
}
myStr{
url: f(12345),
}
}
There is another elegant way to achieve this which doesn't produce much boilerplate code and doesn't look ugly in my opinion. In case I need a struct with pointers to primitives instead of values, to make sure that zero-valued struct members aren't used across the project, I will create a function with those primitives as arguments.
You can define a function which creates your struct and then pass primitives to this function and then use pointers to function arguments.
type Config struct {
Code *uint8
Name *string
}
func NewConfig(code uint8, name string) *Config {
return &Config{
Code: &code,
Name: &name,
}
}
func UseConfig() {
config := NewConfig(1, "test")
// ...
}
// in case there are many values, modern IDE will highlight argument names for you, so you don't have to remember
func UseConfig2() {
config := NewConfig(
1,
"test",
)
// ...
}
If you don't mind using third party libraries, there's the lo package which uses generics (go 1.18+) which has the .ToPtr() function
ptr := lo.ToPtr("hello world")
// *string{"hello world"}