Iterate through the fields of a struct in Go
I have read above thread and now I'm trying to extend it by processing multiple items at a time
package main
import (
"fmt"
"reflect"
)
type BaseStats struct {
value1 int
value2 byte
value3 int
value4 byte
}
func StatsCreate(stats BaseStats) {
v := reflect.ValueOf(stats)
val := make([]interface{}, v.NumField())
for i := 0; i< v.NumField(); i+=2 {
val[i+0] = v.Field(i+0).Interface().(int)
val[i+1] = v.Field(i+1).Interface().(byte)
fmt.Printf("%v %v", val[i+0], val[i+1])
}
}
func main() {
StatsCreate(BaseStats{20, '%', 400, '$'})
}
Error I had before: panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
Go requires you to make the first letter of any variable or function that you wish to use outside of the package (aka EXPORTED). Because you're trying to pass variables to another package, in this case the reflect package, but they're not exportable. It's very common that all struct field names are capitalized as best practice.
The capitalized->exported paradigm is one of the few things I don't like about the language, but that's the way it is.
Related
If for example I have a struct such as:
type Example struct {
Foo string
Bar string
Baz struct{
A int
B string
}
Qux []string
}
What would be the best approach to convert it into a flat slice of strings representing the dot path to each struct field?
I want an output that looks like the following slice:
["Example.Foo", "Example.Bar", "Example.Baz.A", "Example.Baz.B", "Example.Qux.0", "Example.Qux.1"]
The exact structs will be known at compile time. Also, the conversion from struct to a flat list is in a hot path so performance will be an important consideration.
Any hints would be appreciated!
You have to code it yourself, with reflection.
This is a demonstrative function that prints the output you provided:
package main
import (
"fmt"
"reflect"
"strconv"
)
type Example struct {
Foo string
Bar string
Baz struct{
A int
B string
}
Qux []string
}
func main() {
example := Example{Qux: []string{"a", "b"}}
t := reflect.ValueOf(example)
prefix := t.Type().Name()
fmt.Println(ToPathSlice(t, prefix, make([]string, 0)))
}
func ToPathSlice(t reflect.Value, name string, dst []string) []string {
switch t.Kind() {
case reflect.Ptr, reflect.Interface:
return ToPathSlice(t.Elem(), name, dst)
case reflect.Struct:
for i := 0; i < t.NumField(); i++ {
fname := t.Type().Field(i).Name
dst = ToPathSlice(t.Field(i), name+"."+fname, dst)
}
case reflect.Slice, reflect.Array:
for i := 0; i < t.Len(); i++ {
dst = ToPathSlice(t.Index(i), name+"."+strconv.Itoa(i), dst)
}
default:
return append(dst, name)
}
return dst
}
Will print:
[Example.Foo Example.Bar Example.Baz.A Example.Baz.B Example.Qux.0 Example.Qux.1]
Note that:
reflection comes at a performance penalty; if you are concerned about this, you should profile the relevant code path to see if it's a deal breaker
the above code is contrived, for example it doesn't handle maps, it doesn't handle nil, etc.; you can expand it yourself
in your desired output, the indices of slice/array fields is printed. Slices don't have inherent length as arrays. In order to know the length of slices, you have to work with reflect.Value. This IMO makes the code more awkward. If you can accept not printing indices for slices, then you can work with reflect.Type.
Playground: https://play.golang.org/p/isNFSfFiXOP
Is there a way to use reflect to access unexported fields in Go 1.8?
This no longer seems to work: https://stackoverflow.com/a/17982725/555493
Note that reflect.DeepEqual works just fine (that is, it can access unexported fields) but I can't make heads or tails of that function. Here's a go playarea that shows it in action: https://play.golang.org/p/vyEvay6eVG. The src code is below
import (
"fmt"
"reflect"
)
type Foo struct {
private string
}
func main() {
x := Foo{"hello"}
y := Foo{"goodbye"}
z := Foo{"hello"}
fmt.Println(reflect.DeepEqual(x,y)) //false
fmt.Println(reflect.DeepEqual(x,z)) //true
}
If the struct is addressable, you can use unsafe.Pointer to access the field (read or write) it, like this:
rs := reflect.ValueOf(&MyStruct).Elem()
rf := rs.Field(n)
// rf can't be read or set.
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
// Now rf can be read and set.
See full example on the playground.
This use of unsafe.Pointer is valid according to the documentation and running go vet returns no errors.
If the struct is not addressable this trick won't work, but you can create an addressable copy like this:
rs = reflect.ValueOf(MyStruct)
rs2 := reflect.New(rs.Type()).Elem()
rs2.Set(rs)
rf = rs2.Field(0)
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
// Now rf can be read. Setting will succeed but only affects the temporary copy.
See full example on the playground.
Based on cpcallen's work:
import (
"reflect"
"unsafe"
)
func GetUnexportedField(field reflect.Value) interface{} {
return reflect.NewAt(field.Type(), unsafe.Pointer(field.UnsafeAddr())).Elem().Interface()
}
func SetUnexportedField(field reflect.Value, value interface{}) {
reflect.NewAt(field.Type(), unsafe.Pointer(field.UnsafeAddr())).
Elem().
Set(reflect.ValueOf(value))
}
reflect.NewAt might be confusing to read at first. It returns a reflect.Value representing a pointer to a value of the specified field.Type(), using unsafe.Pointer(field.UnsafeAddr()) as that pointer. In this context reflect.NewAt is different than reflect.New, which would return a pointer to a freshly initialized value.
Example:
type Foo struct {
unexportedField string
}
GetUnexportedField(reflect.ValueOf(&Foo{}).Elem().FieldByName("unexportedField"))
https://play.golang.org/p/IgjlQPYdKFR
reflect.DeepEqual() can do it because it has access to unexported features of the reflect package, in this case namely for the valueInterface() function, which takes a safe argument, which denies access to unexported field values via the Value.Interface() method if safe=true. reflect.DeepEqual() will (might) call that passing safe=false.
You can still do it, but you cannot use Value.Interface() for unexported fields. Instead you have to use type-specific methods, such as Value.String() for string, Value.Float() for floats, Value.Int() for ints etc. These will return you a copy of the value (which is enough to inspect it), but will not allow you to modify the field's value (which might be "partly" possible if Value.Interface() would work and the field type would be a pointer type).
If a field happens to be an interface type, you may use Value.Elem() to get to the value contained / wrapped by the interface value.
To demonstrate:
type Foo struct {
s string
i int
j interface{}
}
func main() {
x := Foo{"hello", 2, 3.0}
v := reflect.ValueOf(x)
s := v.FieldByName("s")
fmt.Printf("%T %v\n", s.String(), s.String())
i := v.FieldByName("i")
fmt.Printf("%T %v\n", i.Int(), i.Int())
j := v.FieldByName("j").Elem()
fmt.Printf("%T %v\n", j.Float(), j.Float())
}
Output (try it on the Go Playground):
string hello
int64 2
float64 3
package main
import (
"fmt"
"reflect"
"strings"
"unsafe"
)
type Person1 struct {
W3ID string
Name string
}
type Address1 struct {
city string
country string
}
type User1 struct {
name string
age int
address Address1
manager Person1
developer Person1
tech Person1
}
func showDetails(load, email interface{}) {
if reflect.ValueOf(load).Kind() == reflect.Struct {
typ := reflect.TypeOf(load)
value := reflect.ValueOf(load)
//#1 For struct, not addressable create a copy With Element.
value2 := reflect.New(value.Type()).Elem()
//#2 Value2 is addressable and can be set
value2.Set(value)
for i := 0; i < typ.NumField(); i++ {
if value.Field(i).Kind() == reflect.Struct {
rf := value2.Field(i)
/* #nosec G103 */
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
irf := rf.Interface()
typrf := reflect.TypeOf(irf)
nameP := typrf.String()
if strings.Contains(nameP, "Person") {
//fmt.Println(nameP, "FOUND !!!!!!! ")
for j := 0; j < typrf.NumField(); j++ {
re := rf.Field(j)
nameW := typrf.Field(j).Name
if strings.Contains(nameW, "W3ID") {
valueW := re.Interface()
fetchEmail := valueW.(string)
if fetchEmail == email {
fmt.Println(fetchEmail, " MATCH!!!!")
}
}
}
}
showDetails(irf, email)
} else {
// fmt.Printf("%d.Type:%T || Value:%#v\n",
// (i + 1), value.Field(i), value.Field(i))
}
}
}
}
func main() {
iD := "tsumi#in.org.com"
load := User1{
name: "John Doe",
age: 34,
address: Address1{
city: "New York",
country: "USA",
},
manager: Person1{
W3ID: "jBult#in.org.com",
Name: "Bualt",
},
developer: Person1{
W3ID: "tsumi#in.org.com",
Name: "Sumi",
},
tech: Person1{
W3ID: "lPaul#in.org.com",
Name: "Paul",
},
}
showDetails(load, iD)
}
I have some code that looks to be working but does nothing in the end:
http://play.golang.org/p/TfAWWy4-R8
Have a struct that has fields of type struct. The inner struct has all string fields.
Using reflect in a loop, want to get all struct fields from outer struct. Next, populate all string values in inner struct. the example code is getting text from the tags and parsing it on "," to get strings values for inner loop.
This is the main part that should create the inner struct and add the parsed data to the string values.
t := reflect.TypeOf(Alias{})
alias = reflect.New(t)
for i := 0; i < alias.Elem().NumField(); i++ {
p := alias.Elem().Field(i)
p.SetString(params[i])
}
It looks like it is working when you look at the output from example, but after printing a value from outer struct it seems to be empty:
fmt.Println("Final01 = ", Attr.Final01.Command) // prints empty string
So not sure how to get values into the inner struct Alias{}
Thanks for any help.
Here's the working program. Explanation below.
package main
import "fmt"
import "strings"
import "reflect"
type Alias struct {
Name string
DevicePath string
GuiPath string
Setpoint string
Command string
Status string
FunctionCmds string
}
type Manifold struct {
Final01 Alias "Final01,/Gaspanel/Shared/Final01,,,wOpen,rIsOpen,"
Dump01 Alias "Dump01,/Gaspanel/Shared/Dump01,,,wOpen,rIsOpen,"
N2Vent01 Alias "N2Vent01,/Gaspanel/Shared/N2Vent01,,,wOpen,rIsOpen,"
N2Vent201 Alias "N2Vent201,/Gaspanel/Shared/N2Vent201,,,wOpen,rIsOpen,"
PurgeValve Alias "PurgeValve,/Gaspanel/Shared/Purge01,,,wOpen,rIsOpen,"
}
func MapTagedAliasToChamber(chamber string, struc interface{}) []string {
attributeStruct := reflect.ValueOf(struc).Elem()
typeAttributeStruct := attributeStruct.Type()
attributes := make([]string, attributeStruct.NumField(), attributeStruct.NumField())
for i := 0; i < attributeStruct.NumField(); i++ {
alias := attributeStruct.Field(i)
tag := string(typeAttributeStruct.Field(i).Tag)
name := typeAttributeStruct.Field(i).Name
params := strings.Split(tag, ",")
alias = reflect.New(reflect.TypeOf(Alias{})).Elem()
for i := 0; i < alias.NumField(); i++ {
alias.Field(i).SetString(params[i])
}
attributeStruct.Field(i).Set(alias)
fmt.Printf("%d: %s %s = %v\n", i, name, alias.Type(), alias.Interface())
}
return attributes
}
func main() {
Attr := Manifold{}
MapTagedAliasToChamber("A", &Attr)
fmt.Println("Final01 = ", Attr.Final01.Command)
}
The problem was on line 38 of your original program, where you created a new reflect.Value named alias representing a value of type *Alias, then filled it with your information but never wrote it back into your Manifold struct.
Additionally I suggest that you stick to the standard struct-tag format which can be parsed and used more easily through (reflect.StructTag).Get(key string). And don't use strings where you don't need them e.g. rIsOpen sounds like a boolean value to me.
They seem to be the same:
package main
import "fmt"
type S struct {
i int
}
func main() {
var s1 *S = new(S)
fmt.Println(s1)
var s2 *S = &S{}
fmt.Println(s2) // Prints the same thing.
}
Update:
Hm. I just realized that there's no obvious way to initialize S.i using new. Is there a way to do that? new(S{i:1}) does not seem to work :/
From Effective Go:
As a limiting case, if a composite literal contains no fields at all, it creates a zero value for the type. The expressions new(File) and &File{} are equivalent.
Not only do they give the same resulting value, but if we allocate something both ways and look at their values...
// Adapted from http://tour.golang.org/#30
package main
import "fmt"
type Vertex struct {
X, Y int
}
func main() {
v := &Vertex{}
v2 := new(Vertex)
fmt.Printf("%p %p", v, v2)
}
...we'll see that they are in fact allocated in consecutive memory slots. Typical output: 0x10328100 0x10328108. I'm not sure if this is an implementation detail or part of the specification, but it does demonstrate that they're both being allocated from the same pool.
Play around with the code here.
As for initializing with new, according to the language spec: The built-in function new takes a type T and returns a value of type *T. The memory [pointed to] is initialized as described in the section on initial values. Because functions in go can't be overloaded, and this isn't a variadic function, there's no way to pass in any initialization data. Instead, go will initialize it with whatever version of 0 makes sense for the type and any member fields, as appropriate.
Case 1: package main
import (
"fmt"
)
type Drink struct {
Name string
Flavour string
}
func main() {
a := new(Drink)
a.Name = "Maaza"
a.Flavour = "Mango"
b := a
fmt.Println(&a)
fmt.Println(&b)
b.Name = "Frooti"
fmt.Println(a.Name)
}//This will output Frooti for a.Name, even though the addresses for a and b are different.
Case 2:
package main
import (
"fmt"
)
type Drink struct {
Name string
Flavour string
}
func main() {
a := Drink{
Name: "Maaza",
Flavour: "Mango",
}
b := a
fmt.Println(&a)
fmt.Println(&b)
b.Name = "Froti"
fmt.Println(a.Name)
}//This will output Maaza for a.Name. To get Frooti in this case assign b:=&a.
having a rough time working with struct fields using reflect package. in particular, have not figured out how to set the field value.
type t struct { fi int; fs string }
var r t = t{ 123, "jblow" }
var i64 int64 = 456
getting Name of field i - this seems to work
var field = reflect.TypeOf(r).Field(i).Name
getting value of field i as a) interface{}, b) int - this seems to work
var iface interface{} = reflect.ValueOf(r).Field(i).Interface()
var i int = int(reflect.ValueOf(r).Field(i).Int())
setting value of field i - try one - panic
reflect.ValueOf(r).Field(i).SetInt( i64 )
panic: reflect.Value·SetInt using value obtained using unexported field
assuming it did not like field names "id" and "name", so renamed to "Id" and "Name"
a) is this assumption correct?
b) if correct, thought not necessary since in same file / package
setting value of field i - try two (with field names capitalized ) - panic
reflect.ValueOf(r).Field(i).SetInt( 465 )
reflect.ValueOf(r).Field(i).SetInt( i64 )
panic: reflect.Value·SetInt using unaddressable value
Instructions below by #peterSO are thorough and high quality
Four. this works:
reflect.ValueOf(&r).Elem().Field(i).SetInt( i64 )
he documents as well that the field names must be exportable (begin with capital letter)
The Go json package marshals and unmarshals JSON from and to Go structures.
Here's a step-by-step example which sets the value of a struct field while carefully avoiding errors.
The Go reflect package has a CanAddr function.
func (v Value) CanAddr() bool
CanAddr returns true if the value's
address can be obtained with Addr.
Such values are called addressable. A
value is addressable if it is an
element of a slice, an element of an
addressable array, a field of an
addressable struct, or the result of
dereferencing a pointer. If CanAddr
returns false, calling Addr will
panic.
The Go reflect package has a CanSet function, which, if true, implies that CanAddr is also true.
func (v Value) CanSet() bool
CanSet returns true if the value of v
can be changed. A Value can be changed
only if it is addressable and was not
obtained by the use of unexported
struct fields. If CanSet returns
false, calling Set or any
type-specific setter (e.g., SetBool,
SetInt64) will panic.
We need to make sure we can Set the struct field. For example,
package main
import (
"fmt"
"reflect"
)
func main() {
type t struct {
N int
}
var n = t{42}
// N at start
fmt.Println(n.N)
// pointer to struct - addressable
ps := reflect.ValueOf(&n)
// struct
s := ps.Elem()
if s.Kind() == reflect.Struct {
// exported field
f := s.FieldByName("N")
if f.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if f.CanSet() {
// change value of N
if f.Kind() == reflect.Int {
x := int64(7)
if !f.OverflowInt(x) {
f.SetInt(x)
}
}
}
}
}
// N at end
fmt.Println(n.N)
}
Output:
42
7
If we can be certain that all the error checks are unnecessary, the example simplifies to,
package main
import (
"fmt"
"reflect"
)
func main() {
type t struct {
N int
}
var n = t{42}
fmt.Println(n.N)
reflect.ValueOf(&n).Elem().FieldByName("N").SetInt(7)
fmt.Println(n.N)
}
BTW, Go is available as open source code. A good way to learn about reflection is to see how the core Go developers use it. For example, the Go fmt and json packages. The package documentation has links to the source code files under the heading Package files.
This seems to work:
package main
import (
"fmt"
"reflect"
)
type Foo struct {
Number int
Text string
}
func main() {
foo := Foo{123, "Hello"}
fmt.Println(int(reflect.ValueOf(foo).Field(0).Int()))
reflect.ValueOf(&foo).Elem().Field(0).SetInt(321)
fmt.Println(int(reflect.ValueOf(foo).Field(0).Int()))
}
Prints:
123
321