Recursively parse over struct from dict keys - go

I am trying to recursively parse a form data dictionary into an embedded struct.
It is kindof working, but eventually only one field gets set. I figured that the field keeps being overridden by the old struct, but need some help finding the solution.
Entrypoint of the function:
func FormDataToStruct(data map[string]string, s any) {
// Parse the form data into the struct
v := reflect.ValueOf(s)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
if v.Kind() != reflect.Struct {
panic("not a struct")
}
formParse(data, v, s)
}
Set the values on the topmost struct:
func formParse(data map[string]string, v reflect.Value, s any) {
for key, value := range data {
if hasField(v, key) {
println("Found field: " + key)
var val, err = TransformValue(s, key, value)
if err != nil {
panic(err)
}
SetValue(s, key, val)
delete(data, key)
}
}
// Set values of inner structs
for key, value := range data {
keys := strings.Split(key, "_")
// Parse the inner struct recursively with another function
recurseKeys(keys, value, v, s, s)
}
}
Struct looks like this:
type Me struct {
Name string
Age int
Friend struct {
Name string
Age int
}
}
Form data to parse looks like this:
map[name:John age:20 friend_name:Jane friend_age:20]
func recurseKeys(keys []string, value string, v reflect.Value, s any, parent any) {
if len(keys) == 1 {
// We are at the end of the keys
// Set the value
var val, err = TransformValue(s, keys[0], value)
if err != nil {
panic(err)
}
SetValue(s, keys[0], val)
return
}
// We are not at the end of the keys
// We need to iterate over the struct
for i := 0; i < v.NumField(); i++ {
if strings.EqualFold(v.Type().Field(i).Name, keys[0]) {
// We found the field
// Recurse with the next key
newS := reflect.New(v.Field(i).Type())
recurseKeys(keys[1:], value, v.Field(i), newS.Interface(), parent)
// Check if the field on the old struct is a pointer, if it is, we need to set the pointer
// If it is not a pointer, we need to set the value
if v.Field(i).Kind() == reflect.Ptr {
// Check if newS is a pointer
if newS.Kind() == reflect.Ptr {
v.Field(i).Set(newS)
} else {
v.Field(i).Set(newS.Elem())
}
} else {
// Check if newS is a pointer
if newS.Kind() == reflect.Ptr {
v.Field(i).Set(newS.Elem())
} else {
v.Field(i).Set(newS)
}
}
}
}
}
Running the above form data through the struct would result in the following output:
println(meInstance)
// {John 20 {Jane 0}}
Any help is very much appreciated!
EDIT
Go playground with minimal reproducible example:
https://go.dev/play/p/d5pIK3uQrUL

The issue with this code is that it creates a new struct and replaces it with the existing object. Therefore, you will always see the last assigned value.
The fixed version of the code will be like below:
func recurseKeys(keys []string, value string, v reflect.Value, s any, parent any) {
if len(keys) == 1 {
// We are at the end of the keys
// Set the value
var val, err = TransformValue(s, keys[0], value)
if err != nil {
panic(err)
}
SetValue(s, keys[0], val)
return
}
// We are not at the end of the keys
// We need to iterate over the struct
for i := 0; i < v.NumField(); i++ {
if strings.EqualFold(v.Type().Field(i).Name, keys[0]) {
// We found the field
// Recurse with the next key
if v.Field(i).IsZero() {
var newS reflect.Value
if v.Field(i).Kind() == reflect.Ptr {
newS = reflect.New(v.Field(i).Type().Elem()).Elem()
} else {
newS = reflect.New(v.Field(i).Type())
}
// Check if the field on the old struct is a pointer, if it is, we need to set the pointer
// If it is not a pointer, we need to set the value
if v.Field(i).Kind() == reflect.Ptr {
v.Field(i).Set(newS.Addr())
} else {
v.Field(i).Set(newS.Elem())
}
}
if v.Field(i).Kind() == reflect.Ptr {
recurseKeys(keys[1:], value, v.Field(i), v.Field(i).Interface(), parent)
} else {
recurseKeys(keys[1:], value, v.Field(i), v.Field(i).Addr().Interface(), parent)
}
}
}
}
This code accepts struct pointers as well.
Performance improvement tip: You may want to consider scanning the target object and creating a map of name -> Reflect value to reduce the number of loops.
Maintenance tip: It's better to consider using struct tags instead of reflecting the struct Variable name directly.

Related

What is the right way to conditionally assign multiple properties to a struct

I'm working on a resolver function for a GraphQL query for a BE I'm writing in Go. In the resolver, I have user data that I want to update, using an input value containing several possible update properties.
In JavaScript, this can be done quickly through destructuring (pseudo):
const mergedObj = {...oldProps, ...newProps}
For now, my resolver function looks like this (using gqlgen for GraphQL Go resolvers):
func (r *mutationResolver) ModifyUser(ctx context.Context, input *model.ModifyUserInput) (*model.User, error) {
id := input.ID
us, ok := r.Resolver.UserStore[id]
if !ok {
return nil, fmt.Errorf("not found")
}
if input.FirstName != nil {
us.FirstName = *input.FirstName
}
if input.LastName != nil {
us.LastName = *input.LastName
}
if input.ProfileImage != nil {
us.ProfileImage = input.ProfileImage
}
if input.Password != nil {
us.Password = *input.Password
}
if input.Email != nil {
us.Email = *input.Email
}
if input.InTomorrow != nil {
us.InTomorrow = input.InTomorrow
}
if input.DefaultDaysIn != nil {
us.DefaultDaysIn = input.DefaultDaysIn
}
r.Resolver.UserStore[id] = us
return &us, nil
}
This feels quite boilerplatey. Would it make sense in this situation to iterate through struct keys? Or is there another pattern I'm missing?
Use a function to reduce the boilerplate:
func mergef[T any](a, b *T) {
if b != nil {
*a = *b
}
}
...
mergef(&us.FirstName, input.FirstName)
mergef(&us.LastName, input.LastName)
...
Use the reflect package to reduce more boilerplate:
// merge sets fields in struct pointed to by d to
// dereferenced fields in struct pointed to by s.
//
// Argument s must point to a struct with pointer type
// fields.
// Argument d must point to a struct with fields that
// correspond to the fields in s: there must be a field
// in d with the same name as a field in s; the type of
// the field in s must be a pointer to the type of the field
// in d.
func merge(d, s any) {
sv := reflect.ValueOf(s).Elem()
dv := reflect.ValueOf(d).Elem()
for i := 0; i < sv.NumField(); i++ {
sf := sv.Field(i)
if sf.IsNil() {
continue
}
df := dv.FieldByName(sv.Type().Field(i).Name)
df.Set(sf.Elem())
}
}
Employ the function like this:
merge(us, input)

How to update the values of an interface within a function with the type defined at runtime?

Let's say I have a struct User
type User struct {
Name string `owm:"newNameFromAPI"`
}
The code below initialises the struct and passes it to a function
func main() {
dest := User{
Name: "Sebastien",
}
deepUpdate(dest, "owm")
}
The function uses reflect in order to iterate over all the fields of the struct and update them accordingly (several chunks of code were removed for clarity)
func deepUpdate(destinationInterface interface{}, selector string) {
// Here I'm not supposed to know that the type is `User`
// And if I just use dest := destinationInterface, the value won't update
dest := destinationInterface.(User)
// Pointer to struct - addressable
destVal := reflect.ValueOf(&dest)
destType := reflect.TypeOf(&dest)
// psindirect := reflect.Indirect(destVal) // ? ValueOf(<Ptr Value>) Requires to be adressed via reflect.Indirect() to access Field - https://stackoverflow.com/a/50098755/9077800
// Struct
destValElem := destVal.Elem()
destTypeElem := destType.Elem()
// Iterate over all fields of the struct
for i := 0; i < destValElem.NumField(); i++ {
// // for i := 0; i < destTypeElem.NumField(); i++ {
destValField := destValElem.Field(i)
destTypeField := destTypeElem.Field(i)
switch destValField.Kind() {
// Field is a struct
case reflect.Struct:
// Recursion
fmt.Println("IS A STRUCT")
default:
// Get the complete tag
tag := destTypeField.Tag
if len(tag) == 0 {
continue
}
// Get the value of our custom tag key
tagVal := tag.Get(selector)
if len(tagVal) == 0 {
continue
}
// Access the value in our source data, thanks to a dot notation access - example: "user.profile.firstname"
sourceValue := "John" // tee.Get(sourceInterface, tagVal)
// Exported field
f := destValField
if f.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if f.CanSet() {
// Change value of Name
fv := reflect.ValueOf(sourceValue)
destValField.Set(fv)
}
}
fmt.Println("NOT STRUCT")
}
}
fmt.Println(dest.Name)
}
The problem is the following line, because I'm not supposed to know that the destinationInterface is to be casted to User.
How can I dynamically cast the interface to some unknown type defined at runtime, or any other way to get the same output of the updated Name "John" instead of "Sebastien"?
dest := destinationInterface.(User)
Here is the complete code running on the golang playgound
https://play.golang.org/p/sYvz-Fwp97P
You don't have to know the type of dest. The example is not recursive, but it can be easily upgraded.
package main
import (
"fmt"
"reflect"
)
type User struct {
Name string `owm:"newNameFromAPI"`
}
func main() {
dest := User{
Name: "Sebastien",
}
fmt.Println("Before:", dest.Name)
deepUpdate(&dest, "owm")
fmt.Println("After:", dest.Name)
}
func deepUpdate(dest interface{}, selector string) {
//Get the reflect value of dest
rv := reflect.ValueOf(dest)
//Dereference every pointers
for rv.Kind() == reflect.Ptr {
rv = reflect.Indirect(rv)
}
//Check if its a struct, should use panic or return error
if reflect.TypeOf(rv.Interface()).Kind() != reflect.Struct {
fmt.Println("NOT A STRUCT")
return
}
//Loop over the fields
for i := 0; i < rv.NumField(); i++ {
//Get the tag value
tag := rv.Type().Field(i).Tag.Get(selector)
if tag == "" {
continue
}
//Get the source
sourceValue := "John"
//Assign the source to the dest's corresponding field
if rv.Field(i).CanSet() {
rv.Field(i).Set(reflect.ValueOf(sourceValue))
}
}
}
The only thing is that you have to use the same type for sourceValue that the corresponding field is.
Working example: https://goplay.space/#D0CmTaS5AiP

Golang search specific item inside Struct array

I'm trying to find the best way for searching inside an Stuct array for getting a specific item with the id of the element.
type Device struct {
Addr net.Addr
AssignedId int
Data string
}
type RegistredDevices struct {
AllDevices []Device
}
Right now i do that
var currentDevice models.Device
for _, device := range registredDevices.AllDevices {
if device.AssignedId == id{
currentDevice = device
}
}
I expected to do something better like that for searching, but i don't know what do to if item can't be find. What should be the return ? Can i return nil or i need to return an empty Device ?
func (registerDevice *RegistredDevices) GetById(id int) Device{
for _, device := range registerDevice.AllDevices {
if device.AssignedId == id{
return device
}
else{
return ?????
}
}
}
var currentDevice = registredDevices.GetById(1)
To signal that an item wasn't found you could either return a pointer to an item (which would be nil if not found), or use two return values with an error or a boolean.
For example map lookups return a value, ok pair. Example from the spec:
An index expression on a map a of type map[K]V used in an assignment or initialization of the special form
v, ok = a[x]
v, ok := a[x]
var v, ok = a[x]
yields an additional untyped boolean value. The value of ok is true if the key x is present in the map, and false otherwise.
In your case it would be:
func (registerDevice *RegistredDevices) GetById(id int) (Device, bool) {
for _, device := range registerDevice.AllDevices {
if device.AssignedId == id {
return device, true
}
}
return Device{}, false
}
And then:
if currentDevice, ok := registredDevices.GetById(1); ok {
// found. use currentDevice
} else {
// not found
}

How do you set fields in a nested struct to their zero value?

Let's say I have an instance of struct Thing1 that I want to json.Marshal
type Thing1 struct {
A string `json:"a,omitempty"`
B int `json:"b,omitempty"`
C Thing2 `json:"c,omitempty"`
}
type Thing2 struct {
D bool `json:"d,omitempty"`
E int `json:"e,omitempty"`
}
...
thing1 := Thing1{
A: "test",
B: 42,
C: Thing2{D: true, E: 43},
}
How would you write a function that takes an instance of any struct and a list of fields to redact and returns a clone (or just mutates) of the incoming object, but with the redacted fields set to their zero values?
redact(thing1, []string{"B", "D"})
thing1 == Thing1{
A: "test",
B: 0,
C: Thing2{D: false, E: 43},
}
I can't use json:"-" as a field tag because the current ones in place are required for the query language I am using (Dgraph).
edit: not in the example, but objects inside arrays should also be redacted if applicable
Use reflect to manipulate the value of struct's field. Below is a proof of concept from what I have written in the comment. Since this is just a poc, you might need to adjust/modify the code to follow your needs.
This function mutate the original data. Code is self explanatory.
func redact(target interface{}, fieldsToModify []string) {
// if target is not pointer, then immediately return
// modifying struct's field requires addresable object
addrValue := reflect.ValueOf(target)
if addrValue.Kind() != reflect.Ptr {
return
}
// if target is not struct then immediatelly return
// this might need to be modified as per your needs
targetValue := addrValue.Elem()
targetType := targetValue.Type()
if targetType.Kind() != reflect.Struct {
return
}
// loop the fields
for i := 0; i < targetType.NumField(); i++ {
fType := targetType.Field(i)
fValue := targetValue.Field(i)
// if the field type is struct, then call redact() recursively
if fValue.Kind() == reflect.Struct {
redact(fValue.Addr().Interface(), fieldsToModify)
continue
}
// if the field is slice, loop then call redact() recursively
if fValue.Kind() == reflect.Array || fValue.Kind() == reflect.Slice {
for i := 0; i < fValue.Len(); i++ {
redact(fValue.Index(i).Addr().Interface(), fieldsToModify)
}
continue
}
// loop the fieldsToModify
for _, fieldToModify := range fieldsToModify {
if fieldToModify == fType.Name && fValue.CanSet() {
fValue.Set(reflect.Zero(fType.Type))
}
}
}
}
The redact() function pointer data in first parameter, since modifying fields require addresable object.
type Thing2 struct {
D bool `json:"d,omitempty"`
E int `json:"e,omitempty"`
}
type Thing1 struct {
A string `json:"a,omitempty"`
B int `json:"b,omitempty"`
C Thing2 `json:"c,omitempty"`
H []Thing2 `json:"h,omitempty"`
}
thing1 := Thing1{
A: "test",
B: 42,
C: Thing2{D: true, E: 43},
H: []Thing2{Thing2{D: true, E: 43}},
}
fmt.Printf("before: %#v \n", thing1)
// before: main.Thing1{A:"test", B:42, C:main.Thing2{D:true, E:43}, H:[]main.Thing2{main.Thing2{D:true, E:43}}}
redact(&thing1, []string{"B", "D"})
fmt.Printf("after: %#v \n", thing1)
// after: main.Thing1{A:"test", B:0, C:main.Thing2{D:false, E:43}, H:[]main.Thing2{main.Thing2{D:false, E:43}}}
Playground: https://play.golang.org/p/wy39DGdSVV7
Here's how to do it with the reflect package:
func redact(x interface{}, names []string) error {
// Starting value must be a pointer.
v := reflect.ValueOf(x)
if v.Kind() != reflect.Ptr {
return errors.New("not pointer")
}
// Create map for easy lookup.
m := make(map[string]bool)
for _, name := range names {
m[name] = true
}
redactValue(v, m)
return nil
}
func redactValue(v reflect.Value, names map[string]bool) {
switch v.Kind() {
case reflect.Ptr:
if v.IsZero() {
return
}
redactValue(v.Elem(), names)
case reflect.Interface:
if v.IsZero() {
return
}
iv := v.Elem()
switch iv.Kind() {
case reflect.Slice, reflect.Ptr:
redactValue(iv, names)
case reflect.Struct, reflect.Array:
// Copy required for modification.
copy := reflect.New(iv.Type()).Elem()
copy.Set(iv)
redactValue(copy, names)
v.Set(copy)
}
case reflect.Struct:
t := v.Type()
for i := 0; i < t.NumField(); i++ {
sf := t.Field(i)
ft := sf.Type
fv := v.Field(i)
if names[sf.Name] {
// Clobber the field.
fv.Set(reflect.Zero(ft))
continue
}
redactValue(fv, names)
}
case reflect.Slice, reflect.Array:
for i := 0; i < v.Len(); i++ {
redactValue(v.Index(i), names)
}
}
}
Run it on the playground.
This answer handles structs, slices, arrays, pointers and interfaces.

Find struct fields recursively

If I have a struct like this:
var Foo struct {
Bar struct {
blah *bool
}
}
And I send the struct to a function that takes an interface as a parameter, is there an easy way to use reflection to find the field "blah" by name using inVal.FieldByName("blah")?
Here's one way to do it:
func findField(v interface{}, name string) reflect.Value {
// create queue of values to search. Start with the function arg.
queue := []reflect.Value{reflect.ValueOf(v)}
for len(queue) > 0 {
v := queue[0]
queue = queue[1:]
// dereference pointers
for v.Kind() == reflect.Ptr {
v = v.Elem()
}
// ignore if this is not a struct
if v.Kind() != reflect.Struct {
continue
}
// iterate through fields looking for match on name
t := v.Type()
for i := 0; i < v.NumField(); i++ {
if t.Field(i).Name == name {
// found it!
return v.Field(i)
}
// push field to queue
queue = append(queue, v.Field(i))
}
}
return reflect.Value{}
}
playground example

Resources