I am dealing with data that is of the type map[string]interface{}. it can have unlimited number of nested objects inside (map[string]interface{}) types.
EDIT: This data comes from mongodb. I can't really apply golang's struct here because the attributes vary from document to document. All I want to do is get the most deeply nested object, add a new attribute to it and make sure the entire data object is updated after.
data["person"] = map[string]interface{}{
"peter": map[string]interface{}{
"scores": map[string]interface{}{
"calculus": 88,
"algebra": 99,
"golang": 89,
},
},
}
This data is coming from a remote API and I have no idea of the properties inside. All I want to add is add new attribute inside the last object (in this case "scores"), and lets say with this new attribute ("physics") the data would look like this
data["person"] = map[string]interface{}{
"peter": map[string]interface{}{
"scores": map[string]interface{}{
"calculus": 88,
"algebra": 99,
"golang": 89,
"physics": 95,
},
},
}
I am not sure how I could get that attribute added to the very last object.
I did recursive type checking and was able to get each field and print its value. But because maps are not referential I cannot add a value to the original map when I reach the map with the values that are not complex types.
package main
import "fmt"
func main() {
data := make(map[string]interface{})
data["person"] = map[string]interface{}{
"peter": map[string]interface{}{
"scores": map[string]interface{}{
"calculus": 88,
"algebra": 99,
"golang": 89,
},
},
}
parseMap(data)
}
func parseMap(aMap map[string]interface{}) interface{} {
var retVal interface{}
for _, val := range aMap {
switch val.(type) {
case map[string]interface{}:
retVal = parseMap(val.(map[string]interface{}))
//case []interface{}:
// retVal = parseArray(val.([]interface{}))
default:
//here i would have done aMap["physics"] = 95 if I could access the original map by reference, but that is not possible
retVal = aMap
}
}
return retVal
}
According the comments on the question, the goal is to set a value in the most deeply nested map.
Use the following function to find a map at the greatest nesting level. If there is more than one map at the greatest nesting level, this function returns an arbitrary one of those maps.
func findDeepest(m map[string]interface{}) (int, map[string]interface{}) {
depth := 0
candidate := m
for _, v := range m {
if v, ok := v.(map[string]interface{}); ok {
d, c := findDeepest(v)
if d+1 > depth {
depth = d + 1
candidate = c
}
}
}
return depth, candidate
}
Use it like this to set a value in the deeply nested map:
_, m := findDeepest(data)
m["physics"] = 95
Run it on the playground.
Try to avoid working with the raw map[string]interface{} type as much as you can. The Go encoding/json file can deal with string-keyed maps just fine, and hopefully the remote API has some sort of specification for what you're dealing with. (You know that you're expecting a person top-level key and scores in a specific point in the hierarchy, for example.)
I'm assuming the remote API is JSON-over-HTTP. You might model its structure as
type Input struct {
Person map[string]Person `json:"person"`
}
type Person struct {
Scores map[string]int `json:"scores"`
}
Once you've json.Unmarshal()ed data into this structure, you can directly set
data.Person["peter"].Scores["physics"] = 95
and then json.Marshal() the result again. https://play.golang.org/p/qoAVFodSvK2 has a complete example.
If you really wanted to directly manipulate the map[string]interface{} structure, I'd suggest splitting each "level" into a separate function call
func ParseTopLevel(data map[string]interface{}) {
switch peter := data["peter"].(type) {
case map[string]interface{}:
ParsePeter(peter)
}
}
map types are passed by reference, so when you get to the bottom of the stack you can directly set scores["physics"] = 95. (In your original code, I'd be surprised if you can't directly set aMap["physics"] as you propose, though it's rather imprecise on what gets set; compare https://play.golang.org/p/VuTjcjezwwU.)
Related
This question already has answers here:
Access struct property by name
(5 answers)
Golang dynamic access to a struct property
(2 answers)
How to access to a struct parameter value from a variable in Golang
(1 answer)
Closed 9 months ago.
Came from javascript background, and just started with Golang. I am learning all the new terms in Golang, and creating new question because I cannot find the answer I need (probably due to lack of knowledge of terms to search for)
I created a custom type, created an array of types, and I want to create a function where I can retrieve all the values of a specific key, and return an array of all the values (brands in this example)
type Car struct {
brand string
units int
}
....
var cars []Car
var singleCar Car
//So i have a loop here and inside the for-loop, i create many single cars
singleCar = Car {
brand: "Mercedes",
units: 20
}
//and i append the singleCar into cars
cars = append(cars, singleCar)
Now what I want to do is to create a function that I can retrieve all the brands, and I tried doing the following. I intend to have key as a dynamic value, so I can search by specific key, e.g. brand, model, capacity etc.
func getUniqueByKey(v []Car, key string) []string {
var combined []string
for i := range v {
combined = append(combined, v[i][key])
//this line returns error -
//invalid operation: cannot index v[i] (map index expression of type Car)compilerNonIndexableOperand
}
return combined
//This is suppose to return ["Mercedes", "Honda", "Ferrari"]
}
The above function is suppose to work if i use getUniqueByKey(cars, "brand") where in this example, brand is the key. But I do not know the syntaxes so it's returning error.
Seems like you're trying to get a property using a slice accessor, which doesn't work in Go. You'd need to write a function for each property. Here's an example with the brands:
func getUniqueBrands(v []Car) []string {
var combined []string
tempMap := make(map[string]bool)
for _, c := range v {
if _, p := tempMap[c.brand]; !p {
tempMap[c.brand] = true
combined = append(combined, c.brand)
}
}
return combined
}
Also, note the for loop being used to get the value of Car here. Go's range can be used to iterate over just indices or both indices and values. The index is discarded by assigning to _.
I would recommend re-using this code with an added switch-case block to get the result you want. If you need to return multiple types, use interface{} and type assertion.
Maybe you could marshal your struct into json data then convert it to a map. Example code:
package main
import (
"encoding/json"
"fmt"
)
type RandomStruct struct {
FieldA string
FieldB int
FieldC string
RandomFieldD bool
RandomFieldE interface{}
}
func main() {
fieldName := "FieldC"
randomStruct := RandomStruct{
FieldA: "a",
FieldB: 5,
FieldC: "c",
RandomFieldD: false,
RandomFieldE: map[string]string{"innerFieldA": "??"},
}
randomStructs := make([]RandomStruct, 0)
randomStructs = append(randomStructs, randomStruct, randomStruct, randomStruct)
res := FetchRandomFieldAndConcat(randomStructs, fieldName)
fmt.Println(res)
}
func FetchRandomFieldAndConcat(randomStructs []RandomStruct, fieldName string) []interface{} {
res := make([]interface{}, 0)
for _, randomStruct := range randomStructs {
jsonData, _ := json.Marshal(randomStruct)
jsonMap := make(map[string]interface{})
err := json.Unmarshal(jsonData, &jsonMap)
if err != nil {
fmt.Println(err)
// panic(err)
}
value, exists := jsonMap[fieldName]
if exists {
res = append(res, value)
}
}
return res
}
I am using a 3rd party package, which allows you to create structures of a certain non-exported type through an exported function.
package squirrel
type expr struct {
sql string
args []interface{}
}
func Expr(sql string, args ...interface{}) expr {
return expr{sql: sql, args: args}
}
Because of the way some other function of this library accepts data, I ended up with such a map:
m := map[string]interface{} {
"col1": 123,
"col2": "a_string",
"col3": Expr("now()"),
}
but because of a different function in this library I need to filter out all squirrel.expr from this map.
Obviously, I wasn't able to assert the type directly, by doing so:
filtered := make(map[string]interface{})
for k, v := range m {
switch v.(type) {
case squirrel.expr:
continue
default:
filtered[k] = v
}
}
Is there another way to achieve the same result?
You may use reflection to compare the type of values to the type of squirrel.expr. Type here means the reflect.Type descriptors, acquired by reflect.TypeOf().
For example:
m := map[string]interface{}{
"col1": 123,
"col2": "a_string",
"col3": squirrel.Expr("now()"),
}
fmt.Println(m)
exprType := reflect.TypeOf(squirrel.Expr(""))
filtered := make(map[string]interface{})
for k, v := range m {
if reflect.TypeOf(v) == exprType {
continue
}
filtered[k] = v
}
fmt.Println(filtered)
This will output:
map[col1:123 col2:a_string col3:{now() []}]
map[col1:123 col2:a_string]
Note:
We obtained the reflect.Type descriptor of the values we want to filter out by passing the return value of a squirrel.Expr() call (which is of type squirrel.expr). This is fine in this case, but if it is unfeasible to call this function just to get the type (e.g. the call has side effects which must be avoided), we can avoid that. We can use reflection to obtain the reflect.Type descriptor of the squirrel.Expr function itself, and get the type descriptor of its return type. This is how it could be done:
exprType := reflect.TypeOf(squirrel.Expr).Out(0)
I have a slice of struct []student, and I want to modify its content with function.
type student struct {
name string
age int
}
students := []student{
{"Doraemon", 30},
{"King Kong", 25},
}
Thus, I decided to pass it as a pointer. May I know how to pass the slice as a reference to a function?
func addAge (s *[]student) error { //this code has error
//everyone add 2 years old
for i, e := range *s {
s[i].age = s[i].age + 2
}
//make the first student much older
s[0].age = s[0].age + 5
return nil
}
I keep playing with Go Playground, but it gives many complains, such as
cannot range over s (type *[]student)
invalid operation: s[i] (type *[]student does not support indexing)
invalid indirect of s
...
How to precisely pass the reference of a slice of struct to a function? How to range the slice of struct? And how to change the value of the struct (modify the same struct in THE slice)?
I keep getting error while playing with s *[]student, range *s, s []student, s *[]*student ... so hard to get it correct...
sorry for my NEWBIE question, still learning GO... trying hard
Slices are passed by reference, so as long as you are modifying the existing slice content you should not explicitly pass a pointer.
package main
import (
"fmt"
)
type student struct {
name string
age int
}
func main() {
students := []student{
{"Doraemon", 30},
{"King Kong", 25},
}
err := addAge (students)
fmt.Println(students)
if err != nil {
fmt.Println("error")
}
}
func addAge (s []student) error {
for i, _ := range s {
s[i].age = 3
}
return nil
}
Now, for your addAdditinalStudent function you should actually use the append function. Plus, have in mind
..., since the slice header is always updated by a call to
append, you need to save the returned slice after the call. In fact,
the compiler won't let you call append without saving the result.
Slices#append
// add student
students = append(students, student{"Test", 33})
Go Playground
in Go you can pass items by value ([]student) or by reference ([]*student). When you want to operate on the values of a struct{} you should pass it to a function with its reference (the pointer).
So you can do something like this:
type student struct {
name string
age int
}
func addTwoYearsToAll(students []*student){
for _, s := range students {
s.age += 2
}
}
This way you're working with the same exact items you build when appending to the slice. Playground example.
Also take a look at Are Golang function parameter passed as copy-on-write?
I don't understand the behavior of the following piece of code. In creating a list of matching structs as a slice of struct pointers, the code always prints the last element of original array (which actually wasn't a match)—it prints 12 and 12. However, if I change matches to be []Widget instead of []*Widget, then it will print 10 and 11.
Why is this?
package main
import (
"fmt"
)
func main() {
type Widget struct {
id int
attrs []string
}
widgets := []Widget{
Widget{
id: 10,
attrs: []string{"blah", "foo"},
},
Widget{
id: 11,
attrs: []string{"foo", "bar"},
},
Widget{
id: 12,
attrs: []string{"xyz"},
},
}
var matches []*Widget
for _, w := range widgets {
for _, a := range w.attrs {
if a == "foo" {
matches = append(matches, &w)
break
}
}
}
for _, m := range matches {
fmt.Println(m.id)
}
}
That's because when you use the pointers you are adding &w to the array.
Note that w is actually the local variable used in the loop, so that's not the address you want to add to the matches array.
(even though the value of the variable w changes through the loop, its address stays the same)
When the loop ends, w ends up with the last value so that's why it prints 12 two times.
You need to add the address of the element that matched instead.
If you do this:
matches = append(matches, &widgets[i])
Then it'd work fine with pointers as well.
Modified Go playground for you to test it:
https://play.golang.org/p/YE-cokyEHu
I am trying to create a generic function that can handle actions on slices in Go... for instance, append an item of any type to a slice of that same type. This is simply a generic purpose for a more complex solution, but overall the issue boils down to this example:
package main
type car struct {
make string
color string
}
type submarine struct {
name string
length int
}
func genericAppender(thingList interface{}, thing interface{}) []interface{} {
return append(thingList, thing)
}
func main() {
cars := make([]car, 0, 10)
cars[0] = car{make: "ford", color: "red"}
cars[1] = car{make: "chevy", color: "blue"}
subs := make([]submarine, 0, 10)
subs[0] = submarine{name: "sally", length: 100}
subs[1] = submarine{name: "matilda", length: 200}
newCar := car{make: "bmw", color: "white"}
genericAppender(&cars, newCar)
}
The code playground is at this location
The above errors as follows:
prog.go:14: first argument to append must be slice; have interface {}
After this change you're still getting a runtime error (index out of range) however the problem is that thingList is not of type []interface{} but rather interface{} so you can't append to it. Here's an updated version of your code on playground that does a type assertion to convert it to an []interface{} in line with the append. In reality you need to do that on a separate line and check for errors.
https://play.golang.org/p/YMed0VDZrv
So to put some code here;
func genericAppender(thingList interface{}, thing interface{}) []interface{} {
return append(thingList.([]interface{}), thing)
}
will solve the basic problem you're facing. As noted, you still get runtime errors when indexing into the slice. Also, you could change the argument to avoid this by making it;
func genericAppender(thingList []interface{}, thing interface{}) []interface{} {
return append(thingList, thing)
}
Here's a complete example of the second type; https://play.golang.org/p/dIuW_UG7XY
Note I also corrected the runtime error. When you use make with 3 args they are, in this order, type, length, capacity. This means the length of the array is 0 so when you try to assign to indexes 0 and 1 it was causing a panic for IndexOutoFRange. Instead I removed the middle argument so it's make([]interface{}, 10) meaning the length is initially set to 10 so you can assign to those indexes.
In the answer above if you do the following then it throws error. This is what the original question was about:
//genericAppender(subs, newCar). // Throws "cannot use subs (type []submarine) as type []interface {} in argument to genericAppender"
The trick is to convert your slice of specific type into a generic []interface{}.
func convertToGeneric(thingList interface{}) []interface{} {
input := reflect.ValueOf(thingList)
length := input.Len()
out := make([]interface{},length)
for i:=0 ;i < length; i++ {
out[i] = input.Index(i).Interface()
}
return out
}
This you can call the function like this:
genericAppender(convertToGeneric(subs), newCar)
You can check modified working code here: https://play.golang.org/p/0_Zmme3c8lT
With Go 1.19 (Q4 2022), no need for interface, or "convert your slice of specific type into a generic []interface{}"
CL 363434 comes with a new slices packages:
// Package slices defines various functions useful with slices of any type.
// Unless otherwise specified, these functions all apply to the elements
// of a slice at index 0 <= i < len(s).
package slices
import "constraints"
// Grow increases the slice's capacity, if necessary, to guarantee space for
// another n elements. After Grow(n), at least n elements can be appended
// to the slice without another allocation. If n is negative or too large to
// allocate the memory, Grow panics.
func Grow[S ~[]T, T any](s S, n int) S {
return append(s, make(S, n)...)[:len(s)]
}
// Equal reports whether two slices are equal: the same length and all
// elements equal. If the lengths are different, Equal returns false.
// Otherwise, the elements are compared in index order, and the
// comparison stops at the first unequal pair.
// Floating point NaNs are not considered equal.
func Equal[T comparable](s1, s2 []T) bool {
if len(s1) != len(s2) {
return false
}
for i, v1 := range s1 {
v2 := s2[i]
if v1 != v2 {
return false
}
}
return true
}
// ...
Ian Lance Taylor confirms in issue 45955:
This package is now available at golang.org/x/exp/slices.
Per this thread, it will not be put into standard library until the 1.19 release.
We may of course adjust it based on anything we learn about having it in x/exp.