I am trying to iterate over a map of interfaces in golang, it has the below structure, I am able to use for loop to iterate to a single level but couldn't go deep to get values of the interface.
Yaml
steps:
execute:
- mvn : 1.9.3
goals: 'clean install'
concurrent: false
- mvn : 1.9.3
goals: 'dependency-check:check'
concurrent: false
Go
// reading a yaml file
// trying to use "gopkg.in/yaml.v2" package
data, err := ioutil.ReadFile(fileName)
m := make(map[interface{}]interface{})
err = yaml.Unmarshal([]byte(data), &m)
for k, v := range m {
// Trying to explore the data here
fmt.Println("k:", k, "v:", v)
}
Output of
fmt.Printf("--- m:\n%v\n\n", m)
looks like below
map[steps:map[execute:[map[concurrent:false goals:clean install mvn:1.9.3] map[concurrent:false goals:dependency-check:check mvn:1.9.3]]]]
My attempt
for k, v := range m {
fmt.Println("k:", k, "v:", v)
}
Assuming that you have a tree of map[interface{}]interface{} and []interface{}, use the following code to walk the tree:
func walk(v interface{}) {
switch v := v.(type) {
case []interface{}:
for i, v := range v {
fmt.Println("index:", i)
walk(v)
}
case map[interface{}]interface{}:
for k, v := range v {
fmt.Println("key:", k)
walk(v)
}
default:
fmt.Println(v)
}
}
This code recurses down the structure and uses type switches to find the slices and maps.
Use it like this:
data, err := ioutil.ReadFile(fileName)
var m map[interface{}]interface{}
err = yaml.Unmarshal([]byte(data), &m)
walk(m)
Run it on the playground
To access the Execute object and iterate over it you have to do a lot of type assertions like the answer given by #vooker.
Actually it's not recommended to use map[interface{}]interface{} for unmarshaling process. It should be unmarshalled to a concrete type as a struct.
From the YAML structure, it could be translated as a struct type like this:
type data struct {
Steps struct {
Execute []execute `yaml:"execute"`
} `yaml:"steps"`
}
type execute struct {
MVN string `yaml:"mvn"`
Goals string `yaml:"goals"`
Concurrent bool `yaml:"concurrent"`
}
And from the structure, it is easier to access the unmarshalled data from the YAML. For example, if you want to iterate over the "execute" object you could do something like this:
var result data
yaml.Unmarshal([]byte(str), &result)
fmt.Println(result)
// iterate over the slice of object
for _, e := range result.Steps.Execute {
fmt.Println()
fmt.Println("mvn:", e.MVN)
fmt.Println("goals:", e.Goals)
fmt.Println("concurrent:", e.Concurrent)
}
Try it on the playground
I don't know what you mean with that "value of m" as that doesn't look like any format I've seen in Go so I'll talk to a few situations: When the interfaces are probably types you know what they are vs when the interfaces could be anything and you aren't sure.
If there are only a couple of types that you know went into the map, you can do a type switch and handle each type individually. This would give you the opportunity to reference subfields and print them as well. If you always print the same info for the same type of objects, you could look into adding a String() string function to your structs which will make them implement the Stringer interface and then you can print the object and your String() func will get called even if it's boxed as an interface.
If you're working with a library that's filling the map, or there's simply too big a diversity of the types in the map for a type switch to be reasonable, then you'll either want a generic solution such as a library like spew or a custom solution written with reflection.
Related
I'm trying to test around an SQL query wherein one of the arguments is a gosnowflake.Array (essentially a wrapper to a slice) using the go-sqlmock package. Normally, something like this requires me to create a value converter, which I have included:
func (opt arrayConverterOption[T]) ConvertValue(v any) (driver.Value, error) {
casted, ok := v.(*[]T)
if ok {
Expect(*casted).Should(HaveLen(len(opt.Expected)))
for i, c := range *casted {
Expect(c).Should(Equal(opt.Expected[i]))
}
} else {
fmt.Printf("Type: %T\n", v)
return v, nil
}
return "TEST_RESULT", nil
}
Now, this function is called for every argument submitted to the query. I use it to test the correctness of the values in the slice or pass the argument through if it isn't one. The problem I'm having is that, when I create a arrayConverterOption[string] and give it a gosnowflake.Array(["A", "B", "C"]) as an argument, the type assertion fails because gosnowflake.Array returns an internal dynamic type, *stringArray, which is defined as a *[]string.
So you can see my dilemma here. On the one hand, I can't convert v because it's an interface{} and I can't alias v because the inner type is not *[]string, but *stringArray. So then, what should I do here?
I didn't find a way to do this without resulting to reflection. However, with reflction I did manage it:
var casted []T
var ok bool
value := reflect.ValueOf(v)
if value.Kind() == reflect.Pointer {
if inner := value.Elem(); inner.Kind() == reflect.Slice {
r := inner.Convert(reflect.TypeOf([]T{})).Interface()
casted, ok = r.([]T)
}
}
So, this code checks specifically for anything that is a pointer to a slice, which my dynamic type is. Then it uses reflection to convert the inner object to the slice type I was expecting. After that, I call Interface() on the result to get the interface{} from the reflected value and then cast it to a []T. This succeeds. If it doesn't then I'm not working with one of those dynamically typed slices and I can handle the type normally.
I decided that now that generics have been introduced into Go that something like map/reduce should be possible. So, I took a naive stab at it and I get the error:
./prog.go:18:36: cannot use thing (variable of type int) as type I in argument to mapper
Which doesn't explain if the problem is fundamental or I am simply doing something wrong syntactically. Can generic map/reduce be implemented in Go?
package main
import "fmt"
func main() {
things := []int{1, 2, 3, 4}
results := Map(things, func(t int) int {
return t + 1
})
fmt.Printf("%v", results)
}
func Map[I interface{}, O interface{}](things []I, mapper func(thing I) O) []O {
results := make([]O, 0, len(things))
for thing := range things {
results = append(results, mapper(thing))
}
return results
}
You have incorrect use of range. A single variable extracted from range will be the index (type int), not the value (type I, which is only coincidentally int in this case).
Try
for _, thing := range things{...}
This can be done quite easily. You have an error in your code, though right here:
for thing := range things {
You are iterating over the index values (int), not the values of type I. You're also specifying 2 constraints (types I and O) both set to be interface{}. You can just use any instead (it's shorthand for interface{})
So simply write:
func Map[T any, O any](things []T, mapper func(thing T) O) []O {
result := make([]O, 0, len(things))
for _, thing := range things {
result = append(result, mapper(thing))
}
return result
}
Demo
This is quite closely related to some code I reviewed on codereview exchange here. After going through the code, and writing snippets with a ton of suggestions, I decided to just create a package and throw it up on github instead. You can find the repo here.
In it, there's some examples that may come in handy, or help you work through some other quirks WRT generics in golang. I wsa specifically thinking about this bit, where you can filter a generic map type using callbacks like so:
// given the sMap type
type sMap[K comparable, V any] struct {
mu *sync.RWMutex
m map[K]V
}
// Filter returns a map containing the elements that matched the filter callback argument
func (s *sMap[K, V]) Filter(cb func(K, V) bool) map[K]V {
s.mu.RLock()
defer s.mu.RUnlock()
ret := make(map[K]V, len(s.m))
for k, v := range s.m {
if cb(k, v) {
ret[k] = v
}
}
return ret
}
I have a function that accepts an interface{} I then do a switch, case on the type and if it's a slice I want to iterate over the elements. The issue I'm having is I can't have multiple options in the case selector, for example I can't seem to have []int, []float32 and then do a range over the values.
What I want to do is something like this
func digestCollection(obj interface{}) ([]byte, error) {
switch v := obj.(type) {
case []int64, []float64:
for _, values := range v {
// do something with v whether its an int or float
}
}
}
But I get an error saying I can't iterate over interface.
In a type switch, if there is a single type case, then v is of that type:
switch v:=obj.(type) {
case []int64:
// Here, v is []int64
case []float64:
// here, v is []float64
}
However if there are multiple cases, or if it is the default case, then the type of v is the type of obj:
switch v:=obj.(type) {
case []int64,[]float64:
// Here, type of v is type of obj
because v cannot have a definite type if it is either an int array or a float64 array. The code generated for the two would be different.
You can try using reflection to go through the array, or write two loops, one for int and one for float64.
With reference to this:
https://play.golang.org/p/0kYRHO5f7kE
If I have 20+ different fields, if one of the fields in the Struct is empty, don't update it. Only update the ones with values in them.
What's the best way forward? I've seen passing as variadic input to another function but how best can I do this elegantly?
you can use this library to convert your struct fields into map of interfaces (can be done by yourself using reflect from stdlib) then loop over it
pipe := redisClient.TxPipeline()
m := structs.Map(server)
for k, v := range m {
pipe.HMSet(username, k, v)
}
cmder, err := pipe.Exec()
if err != nil {
return nil, err
}
the driver for redis used is go-redis
I have run into this problem a few times when wanting to use keys of maps in a similar way but the values in the maps are different. I thought I could write a function that takes the key type I want with interface{} as the value type but it doesn't work.
func main() {
mapOne := map[string]int
mapTwo := map[string]double
mapThree := map[string]SomeStruct
useKeys(mapOne)
}
func useKeys(m map[string]interface{}) {
//something with keys here
}
Not sure if there is an elegant way to do this I just feel waist full rewriting simple things for different values.
Though maps and slices in go are generic themselves, they are not covariant (nor could they be, since interfaces aren't generics). It's part of working with a language that doesn't have generics, you will have to repeat some things.
If you really just need to get the keys of any old map, you can use reflection to do so:
func useKeys(m interface{}) {
v := reflect.ValueOf(m)
if v.Kind() != reflect.Map {
fmt.Println("not a map!")
return
}
keys := v.MapKeys()
fmt.Println(keys)
}
Go 1.18
You can write a function with type parameters (generic) for this:
func useKeys[V any](m map[string]V) V {
return m["foo"]
}
And use it as:
func main() {
m1 := map[string]int{"foo": 1}
m2 := map[string]float64{"foo": 4.5}
m3 := map[string]*SomeStruct{}
fmt.Println(useKeys(m1))
fmt.Println(useKeys(m2))
fmt.Println(useKeys(m3))
}
As you can see, the type parameter V unifies with the map value, so that you can explicitly force callers of useKeys to pass maps whose keys are string only.
You can see this on the GoTip Playground: https://gotipplay.golang.org/p/epFA2_9u5l5