Is there a way to copy a generic struct (i.e. a struct whose property names are unknown) and skip a single, known property?
Here is what I know:
The parameter to my function--I will call the parameter myData-- is of type interface{}.
myData is a struct.
myData has a known property path.
myData has anywhere from 0 to 6 or so other properties, none of which are known a priori.
Once I remove that path property, then the “leftover” is one of say 30 possible struct types.
So I want to strip path out of myData (or more accurately make a copy that omits path) so that various bits of generated code that try to coerce the struct to one of its possible types will be able to succeed.
I have found examples of copying a struct by reflection, but they typically create an empty struct of the same underlying type, then fill it in. So is it even possible to delete a property as I have outlined...?
You can use reflect.StructOf to dynamically create structs from a list of fields.
package main
import (
"fmt"
"reflect"
)
type A struct {
Foo string
Bar int
Baz bool // to be skipped
}
type B struct {
Foo string
Bar int
}
func main() {
av := reflect.ValueOf(A{"hello", 123, true})
fields := make([]reflect.StructField, 0)
values := make([]reflect.Value, 0)
for i := 0; i < av.NumField(); i++ {
f := av.Type().Field(i)
if f.Name != "Baz" {
fields = append(fields, f)
values = append(values, av.Field(i))
}
}
typ := reflect.StructOf(fields)
val := reflect.New(typ).Elem()
for i := 0; i < len(fields); i++ {
val.Field(i).Set(values[i])
}
btyp := reflect.TypeOf(B{})
bval := val.Convert(btyp)
b, ok := bval.Interface().(B)
fmt.Println(b, ok)
}
Related
If for example I have a struct such as:
type Example struct {
Foo string
Bar string
Baz struct{
A int
B string
}
Qux []string
}
What would be the best approach to convert it into a flat slice of strings representing the dot path to each struct field?
I want an output that looks like the following slice:
["Example.Foo", "Example.Bar", "Example.Baz.A", "Example.Baz.B", "Example.Qux.0", "Example.Qux.1"]
The exact structs will be known at compile time. Also, the conversion from struct to a flat list is in a hot path so performance will be an important consideration.
Any hints would be appreciated!
You have to code it yourself, with reflection.
This is a demonstrative function that prints the output you provided:
package main
import (
"fmt"
"reflect"
"strconv"
)
type Example struct {
Foo string
Bar string
Baz struct{
A int
B string
}
Qux []string
}
func main() {
example := Example{Qux: []string{"a", "b"}}
t := reflect.ValueOf(example)
prefix := t.Type().Name()
fmt.Println(ToPathSlice(t, prefix, make([]string, 0)))
}
func ToPathSlice(t reflect.Value, name string, dst []string) []string {
switch t.Kind() {
case reflect.Ptr, reflect.Interface:
return ToPathSlice(t.Elem(), name, dst)
case reflect.Struct:
for i := 0; i < t.NumField(); i++ {
fname := t.Type().Field(i).Name
dst = ToPathSlice(t.Field(i), name+"."+fname, dst)
}
case reflect.Slice, reflect.Array:
for i := 0; i < t.Len(); i++ {
dst = ToPathSlice(t.Index(i), name+"."+strconv.Itoa(i), dst)
}
default:
return append(dst, name)
}
return dst
}
Will print:
[Example.Foo Example.Bar Example.Baz.A Example.Baz.B Example.Qux.0 Example.Qux.1]
Note that:
reflection comes at a performance penalty; if you are concerned about this, you should profile the relevant code path to see if it's a deal breaker
the above code is contrived, for example it doesn't handle maps, it doesn't handle nil, etc.; you can expand it yourself
in your desired output, the indices of slice/array fields is printed. Slices don't have inherent length as arrays. In order to know the length of slices, you have to work with reflect.Value. This IMO makes the code more awkward. If you can accept not printing indices for slices, then you can work with reflect.Type.
Playground: https://play.golang.org/p/isNFSfFiXOP
Suppose I have some type, and I want to instantiate a variable of this type, with every value non-nil.
type Event struct {
HappenedAtMs int64
ReceivedAtMs int64
FieldA *FieldAType
FieldB []*FieldBType
Here is what I am currently trying:
eventFields := reflect.TypeOf(Event{})
event := Event{}
for i := 0; i < eventFields.NumField(); i++ {
nonEmptyType := reflect.New(eventFields.Field(i).Type).Elem()
reflect.ValueOf(&event).Elem().Field(i).Set(nonEmptyType)
}
However, upon running this code, all the fields in the event variable are still set to nil. How can I achieve what I want?
The reflect package needs a pointer to the struct for it to be able to set its fields. The fields also need to be exported which you can check against using the CanSet method.
To initialize a pointer type with reflect you can simply do reflect.New(T.Elem()). To initialize a map, a slice, a func, or a chan type to non-nil you can use the MakeMap, MakeSlice, MakeFunc, and MakeChan functions respectively. To initialize an interface type to non-nil you can create an anonymous struct type, using reflect.StructOf, with a single embedded field of the target interface type, by embedding the interface type the struct type automatically satisfies the interface and an instance of it can be used to set the field to non-nil.
event := Event{}
rv := reflect.ValueOf(&event).Elem()
for i := 0; i < rv.NumField(); i++ {
if f := rv.Field(i); isNilable(f) && f.IsNil() && f.CanSet() {
switch f.Kind() {
case reflect.Ptr:
f.Set(reflect.New(f.Type().Elem()))
case reflect.Slice:
f.Set(reflect.MakeSlice(f.Type(), 0, 0))
case reflect.Interface:
sf := reflect.StructField{
Name: f.Type().Name(),
Type: f.Type(),
Anonymous: true,
}
rt := reflect.StructOf([]reflect.StructField{sf})
f.Set(reflect.New(rt).Elem())
// TODO handle the rest of nilable types
}
}
}
https://play.golang.com/p/nQqvUIROqF-
Is there a way to use reflect to access unexported fields in Go 1.8?
This no longer seems to work: https://stackoverflow.com/a/17982725/555493
Note that reflect.DeepEqual works just fine (that is, it can access unexported fields) but I can't make heads or tails of that function. Here's a go playarea that shows it in action: https://play.golang.org/p/vyEvay6eVG. The src code is below
import (
"fmt"
"reflect"
)
type Foo struct {
private string
}
func main() {
x := Foo{"hello"}
y := Foo{"goodbye"}
z := Foo{"hello"}
fmt.Println(reflect.DeepEqual(x,y)) //false
fmt.Println(reflect.DeepEqual(x,z)) //true
}
If the struct is addressable, you can use unsafe.Pointer to access the field (read or write) it, like this:
rs := reflect.ValueOf(&MyStruct).Elem()
rf := rs.Field(n)
// rf can't be read or set.
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
// Now rf can be read and set.
See full example on the playground.
This use of unsafe.Pointer is valid according to the documentation and running go vet returns no errors.
If the struct is not addressable this trick won't work, but you can create an addressable copy like this:
rs = reflect.ValueOf(MyStruct)
rs2 := reflect.New(rs.Type()).Elem()
rs2.Set(rs)
rf = rs2.Field(0)
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
// Now rf can be read. Setting will succeed but only affects the temporary copy.
See full example on the playground.
Based on cpcallen's work:
import (
"reflect"
"unsafe"
)
func GetUnexportedField(field reflect.Value) interface{} {
return reflect.NewAt(field.Type(), unsafe.Pointer(field.UnsafeAddr())).Elem().Interface()
}
func SetUnexportedField(field reflect.Value, value interface{}) {
reflect.NewAt(field.Type(), unsafe.Pointer(field.UnsafeAddr())).
Elem().
Set(reflect.ValueOf(value))
}
reflect.NewAt might be confusing to read at first. It returns a reflect.Value representing a pointer to a value of the specified field.Type(), using unsafe.Pointer(field.UnsafeAddr()) as that pointer. In this context reflect.NewAt is different than reflect.New, which would return a pointer to a freshly initialized value.
Example:
type Foo struct {
unexportedField string
}
GetUnexportedField(reflect.ValueOf(&Foo{}).Elem().FieldByName("unexportedField"))
https://play.golang.org/p/IgjlQPYdKFR
reflect.DeepEqual() can do it because it has access to unexported features of the reflect package, in this case namely for the valueInterface() function, which takes a safe argument, which denies access to unexported field values via the Value.Interface() method if safe=true. reflect.DeepEqual() will (might) call that passing safe=false.
You can still do it, but you cannot use Value.Interface() for unexported fields. Instead you have to use type-specific methods, such as Value.String() for string, Value.Float() for floats, Value.Int() for ints etc. These will return you a copy of the value (which is enough to inspect it), but will not allow you to modify the field's value (which might be "partly" possible if Value.Interface() would work and the field type would be a pointer type).
If a field happens to be an interface type, you may use Value.Elem() to get to the value contained / wrapped by the interface value.
To demonstrate:
type Foo struct {
s string
i int
j interface{}
}
func main() {
x := Foo{"hello", 2, 3.0}
v := reflect.ValueOf(x)
s := v.FieldByName("s")
fmt.Printf("%T %v\n", s.String(), s.String())
i := v.FieldByName("i")
fmt.Printf("%T %v\n", i.Int(), i.Int())
j := v.FieldByName("j").Elem()
fmt.Printf("%T %v\n", j.Float(), j.Float())
}
Output (try it on the Go Playground):
string hello
int64 2
float64 3
package main
import (
"fmt"
"reflect"
"strings"
"unsafe"
)
type Person1 struct {
W3ID string
Name string
}
type Address1 struct {
city string
country string
}
type User1 struct {
name string
age int
address Address1
manager Person1
developer Person1
tech Person1
}
func showDetails(load, email interface{}) {
if reflect.ValueOf(load).Kind() == reflect.Struct {
typ := reflect.TypeOf(load)
value := reflect.ValueOf(load)
//#1 For struct, not addressable create a copy With Element.
value2 := reflect.New(value.Type()).Elem()
//#2 Value2 is addressable and can be set
value2.Set(value)
for i := 0; i < typ.NumField(); i++ {
if value.Field(i).Kind() == reflect.Struct {
rf := value2.Field(i)
/* #nosec G103 */
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
irf := rf.Interface()
typrf := reflect.TypeOf(irf)
nameP := typrf.String()
if strings.Contains(nameP, "Person") {
//fmt.Println(nameP, "FOUND !!!!!!! ")
for j := 0; j < typrf.NumField(); j++ {
re := rf.Field(j)
nameW := typrf.Field(j).Name
if strings.Contains(nameW, "W3ID") {
valueW := re.Interface()
fetchEmail := valueW.(string)
if fetchEmail == email {
fmt.Println(fetchEmail, " MATCH!!!!")
}
}
}
}
showDetails(irf, email)
} else {
// fmt.Printf("%d.Type:%T || Value:%#v\n",
// (i + 1), value.Field(i), value.Field(i))
}
}
}
}
func main() {
iD := "tsumi#in.org.com"
load := User1{
name: "John Doe",
age: 34,
address: Address1{
city: "New York",
country: "USA",
},
manager: Person1{
W3ID: "jBult#in.org.com",
Name: "Bualt",
},
developer: Person1{
W3ID: "tsumi#in.org.com",
Name: "Sumi",
},
tech: Person1{
W3ID: "lPaul#in.org.com",
Name: "Paul",
},
}
showDetails(load, iD)
}
I have some code that looks to be working but does nothing in the end:
http://play.golang.org/p/TfAWWy4-R8
Have a struct that has fields of type struct. The inner struct has all string fields.
Using reflect in a loop, want to get all struct fields from outer struct. Next, populate all string values in inner struct. the example code is getting text from the tags and parsing it on "," to get strings values for inner loop.
This is the main part that should create the inner struct and add the parsed data to the string values.
t := reflect.TypeOf(Alias{})
alias = reflect.New(t)
for i := 0; i < alias.Elem().NumField(); i++ {
p := alias.Elem().Field(i)
p.SetString(params[i])
}
It looks like it is working when you look at the output from example, but after printing a value from outer struct it seems to be empty:
fmt.Println("Final01 = ", Attr.Final01.Command) // prints empty string
So not sure how to get values into the inner struct Alias{}
Thanks for any help.
Here's the working program. Explanation below.
package main
import "fmt"
import "strings"
import "reflect"
type Alias struct {
Name string
DevicePath string
GuiPath string
Setpoint string
Command string
Status string
FunctionCmds string
}
type Manifold struct {
Final01 Alias "Final01,/Gaspanel/Shared/Final01,,,wOpen,rIsOpen,"
Dump01 Alias "Dump01,/Gaspanel/Shared/Dump01,,,wOpen,rIsOpen,"
N2Vent01 Alias "N2Vent01,/Gaspanel/Shared/N2Vent01,,,wOpen,rIsOpen,"
N2Vent201 Alias "N2Vent201,/Gaspanel/Shared/N2Vent201,,,wOpen,rIsOpen,"
PurgeValve Alias "PurgeValve,/Gaspanel/Shared/Purge01,,,wOpen,rIsOpen,"
}
func MapTagedAliasToChamber(chamber string, struc interface{}) []string {
attributeStruct := reflect.ValueOf(struc).Elem()
typeAttributeStruct := attributeStruct.Type()
attributes := make([]string, attributeStruct.NumField(), attributeStruct.NumField())
for i := 0; i < attributeStruct.NumField(); i++ {
alias := attributeStruct.Field(i)
tag := string(typeAttributeStruct.Field(i).Tag)
name := typeAttributeStruct.Field(i).Name
params := strings.Split(tag, ",")
alias = reflect.New(reflect.TypeOf(Alias{})).Elem()
for i := 0; i < alias.NumField(); i++ {
alias.Field(i).SetString(params[i])
}
attributeStruct.Field(i).Set(alias)
fmt.Printf("%d: %s %s = %v\n", i, name, alias.Type(), alias.Interface())
}
return attributes
}
func main() {
Attr := Manifold{}
MapTagedAliasToChamber("A", &Attr)
fmt.Println("Final01 = ", Attr.Final01.Command)
}
The problem was on line 38 of your original program, where you created a new reflect.Value named alias representing a value of type *Alias, then filled it with your information but never wrote it back into your Manifold struct.
Additionally I suggest that you stick to the standard struct-tag format which can be parsed and used more easily through (reflect.StructTag).Get(key string). And don't use strings where you don't need them e.g. rIsOpen sounds like a boolean value to me.
Given a struct like so:
type B struct {
X string
Y string
}
type D struct {
B
Z string
}
I want to reflect on D and get to the fields X, Y, Z.
Intuitively, before attempting the solution, I was assuming I would be able to traverse the struct D and get all fields using reflection (X, Y, Z) and won't have to deal with B.
But as you can see, I only see the embedded struct B using reflection and not its fields.
http://play.golang.org/p/qZQD5GdTA8
Is there a way I can make B fully transparent when reflecting on D?
Why do I want this?
Imaging a common struct (B in the example here), that is used in multiple other structs by using embedding. Using reflection, the attempt is to copy D into another similar struct in a different package. The destination struct for copying will have all attributes flatly laid out (no embedding there). So there is a mismatch from the source to the destination (embedding vs no embedding) but all the attributes flatly laid out are the same. I don't want to create custom solutions for each struct.
The 'transparency' you expected is just syntactic sugar and has nothing to do with the data representation. If you want to have a function that flattens your data structure, you would have to write it by yourself.
For example (On play):
func DeepFields(iface interface{}) []reflect.Value {
fields := make([]reflect.Value, 0)
ifv := reflect.ValueOf(iface)
ift := reflect.TypeOf(iface)
for i := 0; i < ift.NumField(); i++ {
v := ifv.Field(i)
switch v.Kind() {
case reflect.Struct:
fields = append(fields, DeepFields(v.Interface())...)
default:
fields = append(fields, v)
}
}
return fields
}
Use the following code to collect all promoted field names as keys in map m:
func collectFieldNames(t reflect.Type, m map[string]struct{}) {
// Return if not struct or pointer to struct.
if t.Kind() == reflect.Ptr {
t = t.Elem()
}
if t.Kind() != reflect.Struct {
return
}
// Iterate through fields collecting names in map.
for i := 0; i < t.NumField(); i++ {
sf := t.Field(i)
m[sf.Name] = struct{}{}
// Recurse into anonymous fields.
if sf.Anonymous {
collectFieldNames(sf.Type, m)
}
}
}
Use it like this:
m := make(map[string]struct{})
collectFieldNames(reflect.TypeOf((*D)(nil)), m)
for name := range m {
fmt.Println(name)
}
Run it on the playground.
This program prints X, Y an Z as requested in the question, but also B because B is also a field name.
This function in this answer can be improved:
Don't blow up on recursive type definitions.
Do not include names repeated at the same level in the hierarchy.
The typeField function in encoding/json/encode.go handles both of these issues.