If for example I have a struct such as:
type Example struct {
Foo string
Bar string
Baz struct{
A int
B string
}
Qux []string
}
What would be the best approach to convert it into a flat slice of strings representing the dot path to each struct field?
I want an output that looks like the following slice:
["Example.Foo", "Example.Bar", "Example.Baz.A", "Example.Baz.B", "Example.Qux.0", "Example.Qux.1"]
The exact structs will be known at compile time. Also, the conversion from struct to a flat list is in a hot path so performance will be an important consideration.
Any hints would be appreciated!
You have to code it yourself, with reflection.
This is a demonstrative function that prints the output you provided:
package main
import (
"fmt"
"reflect"
"strconv"
)
type Example struct {
Foo string
Bar string
Baz struct{
A int
B string
}
Qux []string
}
func main() {
example := Example{Qux: []string{"a", "b"}}
t := reflect.ValueOf(example)
prefix := t.Type().Name()
fmt.Println(ToPathSlice(t, prefix, make([]string, 0)))
}
func ToPathSlice(t reflect.Value, name string, dst []string) []string {
switch t.Kind() {
case reflect.Ptr, reflect.Interface:
return ToPathSlice(t.Elem(), name, dst)
case reflect.Struct:
for i := 0; i < t.NumField(); i++ {
fname := t.Type().Field(i).Name
dst = ToPathSlice(t.Field(i), name+"."+fname, dst)
}
case reflect.Slice, reflect.Array:
for i := 0; i < t.Len(); i++ {
dst = ToPathSlice(t.Index(i), name+"."+strconv.Itoa(i), dst)
}
default:
return append(dst, name)
}
return dst
}
Will print:
[Example.Foo Example.Bar Example.Baz.A Example.Baz.B Example.Qux.0 Example.Qux.1]
Note that:
reflection comes at a performance penalty; if you are concerned about this, you should profile the relevant code path to see if it's a deal breaker
the above code is contrived, for example it doesn't handle maps, it doesn't handle nil, etc.; you can expand it yourself
in your desired output, the indices of slice/array fields is printed. Slices don't have inherent length as arrays. In order to know the length of slices, you have to work with reflect.Value. This IMO makes the code more awkward. If you can accept not printing indices for slices, then you can work with reflect.Type.
Playground: https://play.golang.org/p/isNFSfFiXOP
Related
Iterate through the fields of a struct in Go
I have read above thread and now I'm trying to extend it by processing multiple items at a time
package main
import (
"fmt"
"reflect"
)
type BaseStats struct {
value1 int
value2 byte
value3 int
value4 byte
}
func StatsCreate(stats BaseStats) {
v := reflect.ValueOf(stats)
val := make([]interface{}, v.NumField())
for i := 0; i< v.NumField(); i+=2 {
val[i+0] = v.Field(i+0).Interface().(int)
val[i+1] = v.Field(i+1).Interface().(byte)
fmt.Printf("%v %v", val[i+0], val[i+1])
}
}
func main() {
StatsCreate(BaseStats{20, '%', 400, '$'})
}
Error I had before: panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
Go requires you to make the first letter of any variable or function that you wish to use outside of the package (aka EXPORTED). Because you're trying to pass variables to another package, in this case the reflect package, but they're not exportable. It's very common that all struct field names are capitalized as best practice.
The capitalized->exported paradigm is one of the few things I don't like about the language, but that's the way it is.
Is there a way to copy a generic struct (i.e. a struct whose property names are unknown) and skip a single, known property?
Here is what I know:
The parameter to my function--I will call the parameter myData-- is of type interface{}.
myData is a struct.
myData has a known property path.
myData has anywhere from 0 to 6 or so other properties, none of which are known a priori.
Once I remove that path property, then the “leftover” is one of say 30 possible struct types.
So I want to strip path out of myData (or more accurately make a copy that omits path) so that various bits of generated code that try to coerce the struct to one of its possible types will be able to succeed.
I have found examples of copying a struct by reflection, but they typically create an empty struct of the same underlying type, then fill it in. So is it even possible to delete a property as I have outlined...?
You can use reflect.StructOf to dynamically create structs from a list of fields.
package main
import (
"fmt"
"reflect"
)
type A struct {
Foo string
Bar int
Baz bool // to be skipped
}
type B struct {
Foo string
Bar int
}
func main() {
av := reflect.ValueOf(A{"hello", 123, true})
fields := make([]reflect.StructField, 0)
values := make([]reflect.Value, 0)
for i := 0; i < av.NumField(); i++ {
f := av.Type().Field(i)
if f.Name != "Baz" {
fields = append(fields, f)
values = append(values, av.Field(i))
}
}
typ := reflect.StructOf(fields)
val := reflect.New(typ).Elem()
for i := 0; i < len(fields); i++ {
val.Field(i).Set(values[i])
}
btyp := reflect.TypeOf(B{})
bval := val.Convert(btyp)
b, ok := bval.Interface().(B)
fmt.Println(b, ok)
}
Is there a way to use reflect to access unexported fields in Go 1.8?
This no longer seems to work: https://stackoverflow.com/a/17982725/555493
Note that reflect.DeepEqual works just fine (that is, it can access unexported fields) but I can't make heads or tails of that function. Here's a go playarea that shows it in action: https://play.golang.org/p/vyEvay6eVG. The src code is below
import (
"fmt"
"reflect"
)
type Foo struct {
private string
}
func main() {
x := Foo{"hello"}
y := Foo{"goodbye"}
z := Foo{"hello"}
fmt.Println(reflect.DeepEqual(x,y)) //false
fmt.Println(reflect.DeepEqual(x,z)) //true
}
If the struct is addressable, you can use unsafe.Pointer to access the field (read or write) it, like this:
rs := reflect.ValueOf(&MyStruct).Elem()
rf := rs.Field(n)
// rf can't be read or set.
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
// Now rf can be read and set.
See full example on the playground.
This use of unsafe.Pointer is valid according to the documentation and running go vet returns no errors.
If the struct is not addressable this trick won't work, but you can create an addressable copy like this:
rs = reflect.ValueOf(MyStruct)
rs2 := reflect.New(rs.Type()).Elem()
rs2.Set(rs)
rf = rs2.Field(0)
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
// Now rf can be read. Setting will succeed but only affects the temporary copy.
See full example on the playground.
Based on cpcallen's work:
import (
"reflect"
"unsafe"
)
func GetUnexportedField(field reflect.Value) interface{} {
return reflect.NewAt(field.Type(), unsafe.Pointer(field.UnsafeAddr())).Elem().Interface()
}
func SetUnexportedField(field reflect.Value, value interface{}) {
reflect.NewAt(field.Type(), unsafe.Pointer(field.UnsafeAddr())).
Elem().
Set(reflect.ValueOf(value))
}
reflect.NewAt might be confusing to read at first. It returns a reflect.Value representing a pointer to a value of the specified field.Type(), using unsafe.Pointer(field.UnsafeAddr()) as that pointer. In this context reflect.NewAt is different than reflect.New, which would return a pointer to a freshly initialized value.
Example:
type Foo struct {
unexportedField string
}
GetUnexportedField(reflect.ValueOf(&Foo{}).Elem().FieldByName("unexportedField"))
https://play.golang.org/p/IgjlQPYdKFR
reflect.DeepEqual() can do it because it has access to unexported features of the reflect package, in this case namely for the valueInterface() function, which takes a safe argument, which denies access to unexported field values via the Value.Interface() method if safe=true. reflect.DeepEqual() will (might) call that passing safe=false.
You can still do it, but you cannot use Value.Interface() for unexported fields. Instead you have to use type-specific methods, such as Value.String() for string, Value.Float() for floats, Value.Int() for ints etc. These will return you a copy of the value (which is enough to inspect it), but will not allow you to modify the field's value (which might be "partly" possible if Value.Interface() would work and the field type would be a pointer type).
If a field happens to be an interface type, you may use Value.Elem() to get to the value contained / wrapped by the interface value.
To demonstrate:
type Foo struct {
s string
i int
j interface{}
}
func main() {
x := Foo{"hello", 2, 3.0}
v := reflect.ValueOf(x)
s := v.FieldByName("s")
fmt.Printf("%T %v\n", s.String(), s.String())
i := v.FieldByName("i")
fmt.Printf("%T %v\n", i.Int(), i.Int())
j := v.FieldByName("j").Elem()
fmt.Printf("%T %v\n", j.Float(), j.Float())
}
Output (try it on the Go Playground):
string hello
int64 2
float64 3
package main
import (
"fmt"
"reflect"
"strings"
"unsafe"
)
type Person1 struct {
W3ID string
Name string
}
type Address1 struct {
city string
country string
}
type User1 struct {
name string
age int
address Address1
manager Person1
developer Person1
tech Person1
}
func showDetails(load, email interface{}) {
if reflect.ValueOf(load).Kind() == reflect.Struct {
typ := reflect.TypeOf(load)
value := reflect.ValueOf(load)
//#1 For struct, not addressable create a copy With Element.
value2 := reflect.New(value.Type()).Elem()
//#2 Value2 is addressable and can be set
value2.Set(value)
for i := 0; i < typ.NumField(); i++ {
if value.Field(i).Kind() == reflect.Struct {
rf := value2.Field(i)
/* #nosec G103 */
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
irf := rf.Interface()
typrf := reflect.TypeOf(irf)
nameP := typrf.String()
if strings.Contains(nameP, "Person") {
//fmt.Println(nameP, "FOUND !!!!!!! ")
for j := 0; j < typrf.NumField(); j++ {
re := rf.Field(j)
nameW := typrf.Field(j).Name
if strings.Contains(nameW, "W3ID") {
valueW := re.Interface()
fetchEmail := valueW.(string)
if fetchEmail == email {
fmt.Println(fetchEmail, " MATCH!!!!")
}
}
}
}
showDetails(irf, email)
} else {
// fmt.Printf("%d.Type:%T || Value:%#v\n",
// (i + 1), value.Field(i), value.Field(i))
}
}
}
}
func main() {
iD := "tsumi#in.org.com"
load := User1{
name: "John Doe",
age: 34,
address: Address1{
city: "New York",
country: "USA",
},
manager: Person1{
W3ID: "jBult#in.org.com",
Name: "Bualt",
},
developer: Person1{
W3ID: "tsumi#in.org.com",
Name: "Sumi",
},
tech: Person1{
W3ID: "lPaul#in.org.com",
Name: "Paul",
},
}
showDetails(load, iD)
}
I read about the interfaces a lot and I think I understand how it works. I read about the interface{} type and use it to take an argument of function. It is clear. My question (and what I don't understand) is what is my benefit if I am using it. It is possible I didn't get it entirely but for example I have this:
package main
import (
"fmt"
)
func PrintAll(vals []interface{}) {
for _, val := range vals {
fmt.Println(val)
}
}
func main() {
names := []string{"stanley", "david", "oscar"}
vals := make([]interface{}, len(names))
for i, v := range names {
vals[i] = v
}
PrintAll(vals)
}
Why is it better than this:
package main
import (
"fmt"
)
func PrintAll(vals []string) {
for _, val := range vals {
fmt.Println(val)
}
}
func main() {
names := []string{"stanley", "david", "oscar"}
PrintAll(names)
}
If you're always want to print string values, then the first using []interface{} is not better at all, it's worse as you lose some compile-time checking: it won't warn you if you pass a slice which contains values other than strings.
If you want to print values other than strings, then the second with []string wouldn't even compile.
For example the first also handles this:
PrintAll([]interface{}{"one", 2, 3.3})
While the 2nd would give you a compile-time error:
cannot use []interface {} literal (type []interface {}) as type []string in argument to PrintAll
The 2nd gives you compile-time guarantee that only a slice of type []string is passed; should you attempt to pass anything other will result in compile-time error.
Also see related question: Why are interfaces needed in Golang?
I have some code that looks to be working but does nothing in the end:
http://play.golang.org/p/TfAWWy4-R8
Have a struct that has fields of type struct. The inner struct has all string fields.
Using reflect in a loop, want to get all struct fields from outer struct. Next, populate all string values in inner struct. the example code is getting text from the tags and parsing it on "," to get strings values for inner loop.
This is the main part that should create the inner struct and add the parsed data to the string values.
t := reflect.TypeOf(Alias{})
alias = reflect.New(t)
for i := 0; i < alias.Elem().NumField(); i++ {
p := alias.Elem().Field(i)
p.SetString(params[i])
}
It looks like it is working when you look at the output from example, but after printing a value from outer struct it seems to be empty:
fmt.Println("Final01 = ", Attr.Final01.Command) // prints empty string
So not sure how to get values into the inner struct Alias{}
Thanks for any help.
Here's the working program. Explanation below.
package main
import "fmt"
import "strings"
import "reflect"
type Alias struct {
Name string
DevicePath string
GuiPath string
Setpoint string
Command string
Status string
FunctionCmds string
}
type Manifold struct {
Final01 Alias "Final01,/Gaspanel/Shared/Final01,,,wOpen,rIsOpen,"
Dump01 Alias "Dump01,/Gaspanel/Shared/Dump01,,,wOpen,rIsOpen,"
N2Vent01 Alias "N2Vent01,/Gaspanel/Shared/N2Vent01,,,wOpen,rIsOpen,"
N2Vent201 Alias "N2Vent201,/Gaspanel/Shared/N2Vent201,,,wOpen,rIsOpen,"
PurgeValve Alias "PurgeValve,/Gaspanel/Shared/Purge01,,,wOpen,rIsOpen,"
}
func MapTagedAliasToChamber(chamber string, struc interface{}) []string {
attributeStruct := reflect.ValueOf(struc).Elem()
typeAttributeStruct := attributeStruct.Type()
attributes := make([]string, attributeStruct.NumField(), attributeStruct.NumField())
for i := 0; i < attributeStruct.NumField(); i++ {
alias := attributeStruct.Field(i)
tag := string(typeAttributeStruct.Field(i).Tag)
name := typeAttributeStruct.Field(i).Name
params := strings.Split(tag, ",")
alias = reflect.New(reflect.TypeOf(Alias{})).Elem()
for i := 0; i < alias.NumField(); i++ {
alias.Field(i).SetString(params[i])
}
attributeStruct.Field(i).Set(alias)
fmt.Printf("%d: %s %s = %v\n", i, name, alias.Type(), alias.Interface())
}
return attributes
}
func main() {
Attr := Manifold{}
MapTagedAliasToChamber("A", &Attr)
fmt.Println("Final01 = ", Attr.Final01.Command)
}
The problem was on line 38 of your original program, where you created a new reflect.Value named alias representing a value of type *Alias, then filled it with your information but never wrote it back into your Manifold struct.
Additionally I suggest that you stick to the standard struct-tag format which can be parsed and used more easily through (reflect.StructTag).Get(key string). And don't use strings where you don't need them e.g. rIsOpen sounds like a boolean value to me.