Using "dynamic" key to extract value from map [duplicate] - go

This question already has answers here:
Access struct property by name
(5 answers)
Golang dynamic access to a struct property
(2 answers)
How to access to a struct parameter value from a variable in Golang
(1 answer)
Closed 9 months ago.
Came from javascript background, and just started with Golang. I am learning all the new terms in Golang, and creating new question because I cannot find the answer I need (probably due to lack of knowledge of terms to search for)
I created a custom type, created an array of types, and I want to create a function where I can retrieve all the values of a specific key, and return an array of all the values (brands in this example)
type Car struct {
brand string
units int
}
....
var cars []Car
var singleCar Car
//So i have a loop here and inside the for-loop, i create many single cars
singleCar = Car {
brand: "Mercedes",
units: 20
}
//and i append the singleCar into cars
cars = append(cars, singleCar)
Now what I want to do is to create a function that I can retrieve all the brands, and I tried doing the following. I intend to have key as a dynamic value, so I can search by specific key, e.g. brand, model, capacity etc.
func getUniqueByKey(v []Car, key string) []string {
var combined []string
for i := range v {
combined = append(combined, v[i][key])
//this line returns error -
//invalid operation: cannot index v[i] (map index expression of type Car)compilerNonIndexableOperand
}
return combined
//This is suppose to return ["Mercedes", "Honda", "Ferrari"]
}
The above function is suppose to work if i use getUniqueByKey(cars, "brand") where in this example, brand is the key. But I do not know the syntaxes so it's returning error.

Seems like you're trying to get a property using a slice accessor, which doesn't work in Go. You'd need to write a function for each property. Here's an example with the brands:
func getUniqueBrands(v []Car) []string {
var combined []string
tempMap := make(map[string]bool)
for _, c := range v {
if _, p := tempMap[c.brand]; !p {
tempMap[c.brand] = true
combined = append(combined, c.brand)
}
}
return combined
}
Also, note the for loop being used to get the value of Car here. Go's range can be used to iterate over just indices or both indices and values. The index is discarded by assigning to _.
I would recommend re-using this code with an added switch-case block to get the result you want. If you need to return multiple types, use interface{} and type assertion.

Maybe you could marshal your struct into json data then convert it to a map. Example code:
package main
import (
"encoding/json"
"fmt"
)
type RandomStruct struct {
FieldA string
FieldB int
FieldC string
RandomFieldD bool
RandomFieldE interface{}
}
func main() {
fieldName := "FieldC"
randomStruct := RandomStruct{
FieldA: "a",
FieldB: 5,
FieldC: "c",
RandomFieldD: false,
RandomFieldE: map[string]string{"innerFieldA": "??"},
}
randomStructs := make([]RandomStruct, 0)
randomStructs = append(randomStructs, randomStruct, randomStruct, randomStruct)
res := FetchRandomFieldAndConcat(randomStructs, fieldName)
fmt.Println(res)
}
func FetchRandomFieldAndConcat(randomStructs []RandomStruct, fieldName string) []interface{} {
res := make([]interface{}, 0)
for _, randomStruct := range randomStructs {
jsonData, _ := json.Marshal(randomStruct)
jsonMap := make(map[string]interface{})
err := json.Unmarshal(jsonData, &jsonMap)
if err != nil {
fmt.Println(err)
// panic(err)
}
value, exists := jsonMap[fieldName]
if exists {
res = append(res, value)
}
}
return res
}

Related

Get the type of value using cty in hclwrite

am looking for a way to find the type of variable using go-cty package in hclwrite.
My aim is to generate a variables file like below
variable "test_var" {
val1 = bool
val2 = string
val3 = number
}
reference: https://developer.hashicorp.com/terraform/language/values/variables
I am using the below code to generate this.
vars := hclwrite.NewEmptyFile()
vars_root_body := vars.Body()
vars_file, vars_create_err := os.Create("variables.tf")
logErrors(vars_create_err)
vars_block := vars_root_body.AppendNewBlock("variable",[]string{"test_var"})
vars_block_body := vars_block.Body()
vars_block_body.SetAttributeValue("val", cty.Value{})
_, vars_write_err := vars_file.Write(vars.Bytes())
logErrors(vars_write_err)
defer vars_file.Close()
the above code generates this
variable "test_var" {
val = null
}
I want to fetch the type of that variable and set the attribute value based on that type, as show in the reference link above. I tried lot of ways but didn't get anything. Can someone please help me on this?
I tried the above code and lot of other ways like
cty.SetValEmpty(cty.Bool)
but it didn't work.
The expected syntax for a variable block in Terraform includes an argument named type, not an argument named val. From your example I assume that you are intending to populate type.
The type constraint syntax that Terraform uses is not directly part of HCL and so there isn't any built-in way to generate that syntax in only one step. However, type constraint are built from HCL's identifier and function call syntaxes, and hclwrite does have some functions for helping to generate those as individual parts:
TokensForIdentifier
TokensForFunctionCall
f := hclwrite.NewEmptyFile()
rootBody := f.Body()
varBlock := rootBody.AppendNewBlock("variable", []string{"example"})
varBody := varBlock.Body()
varBody.SetAttributeRaw(
"type",
hclwrite.TokensForFunctionCall(
"set",
hclwrite.TokensForIdentifier("string"),
),
)
fmt.Printf("%s", f.Bytes())
The above will generate the following:
variable "example" {
type = set(string)
}
If you already have a cty.Value value then you can obtain its type using the Type method. However, as mentioned above there isn't any ready-to-use function for converting a type into a type expression, so if you want to be able to generate a type constraint for any value then you'd need to write a function for this yourself, wrapping the TokensForFunctionCall and TokensForIdentifier functions. For example:
package main
import (
"fmt"
"sort"
"github.com/hashicorp/hcl/v2/hclwrite"
"github.com/zclconf/go-cty/cty"
)
func main() {
f := hclwrite.NewEmptyFile()
rootBody := f.Body()
varBlock := rootBody.AppendNewBlock("variable", []string{"example"})
varBody := varBlock.Body()
varBody.SetAttributeRaw(
"type",
typeExprTokens(cty.Set(cty.String)),
)
fmt.Printf("%s", f.Bytes())
}
func typeExprTokens(ty cty.Type) hclwrite.Tokens {
switch ty {
case cty.String:
return hclwrite.TokensForIdentifier("string")
case cty.Bool:
return hclwrite.TokensForIdentifier("bool")
case cty.Number:
return hclwrite.TokensForIdentifier("number")
case cty.DynamicPseudoType:
return hclwrite.TokensForIdentifier("any")
}
if ty.IsCollectionType() {
etyTokens := typeExprTokens(ty.ElementType())
switch {
case ty.IsListType():
return hclwrite.TokensForFunctionCall("list", etyTokens)
case ty.IsSetType():
return hclwrite.TokensForFunctionCall("set", etyTokens)
case ty.IsMapType():
return hclwrite.TokensForFunctionCall("map", etyTokens)
default:
// Should never happen because the above is exhaustive
panic("unsupported collection type")
}
}
if ty.IsObjectType() {
atys := ty.AttributeTypes()
names := make([]string, 0, len(atys))
for name := range atys {
names = append(names, name)
}
sort.Strings(names)
items := make([]hclwrite.ObjectAttrTokens, len(names))
for i, name := range names {
items[i] = hclwrite.ObjectAttrTokens{
Name: hclwrite.TokensForIdentifier(name),
Value: typeExprTokens(atys[name]),
}
}
return hclwrite.TokensForObject(items)
}
if ty.IsTupleType() {
etys := ty.TupleElementTypes()
items := make([]hclwrite.Tokens, len(etys))
for i, ety := range etys {
items[i] = typeExprTokens(ety)
}
return hclwrite.TokensForTuple(items)
}
panic(fmt.Errorf("unsupported type %#v", ty))
}
This program will generate the same output as the previous example. You can change func main to pass a different type to typeExprTokens to see how it behaves with some different types.

Append to golang slice passed as empty interface

How to append to empty interface (that has been verified to be a *[]struct)?
func main() {
var mySlice []myStruct // myStruct can be any struct (dynamic)
decode(&mySlice, "...")
}
func decode(dest interface{}, src string) {
// assume dest has been verified to be *[]struct
var modelType reflect.Type = getStructType(dest)
rows, fields := getRows(src)
for _, row := range rows {
// create new struct of type modelType and assign all fields
model := reflect.New(modelType)
for field := fields {
fieldValue := getRowValue(row, field)
model.Elem().FieldByName(field).Set(fieldValue)
}
castedModelRow := model.Elem().Interface()
// append model to dest; how to do this?
// dest = append(dest, castedModelRow)
}
}
Things I've tried:
This simply panics: reflect: call of reflect.Append on ptr Value (as we pass &mySlice instead of mySlice)
dest = reflect.Append(reflect.ValueOf(dest), reflect.ValueOf(castedModelRow))
This works but doesn't set the value back to dest... in main func, len(mySlice) remains 0 after decode function is called.
func decode(dest interface{}, src string) {
...
result := reflect.MakeSlice(reflect.SliceOf(modelType), rowCount, rowCount)
for _, row : range rows {
...
result = reflect.Append(result, reflect.ValueOf(castedModelRow))
}
dest = reflect.ValueOf(result)
}
Here's how to fix the second decode function shown in the question. The statement
dest = reflect.ValueOf(result)
modifies local variable dest, not the caller's value. Use the following statement to modify the caller's slice:
reflect.ValueOf(dest).Elem().Set(result)
The code in the question appends decoded elements after the elements created in reflect.MakeSlice. The resulting slice has len(rows) zero values followed by len(rows) decoded values. Fix by changing
result = reflect.Append(result, reflect.ValueOf(castedModelRow))
to:
result.Index(i).Set(model)
Here's the update version of the second decode function in the question:
func decode(dest interface{}, src string) {
var modelType reflect.Type = getStructType(dest)
rows, fields := getRows(src)
result := reflect.MakeSlice(reflect.SliceOf(modelType), len(rows), len(rows))
for i, row := range rows {
model := reflect.New(modelType).Elem()
for _, field := range fields {
fieldValue := getRowValue(row, field)
model.FieldByName(field).Set(fieldValue)
}
result.Index(i).Set(model)
}
reflect.ValueOf(dest).Elem().Set(result)
}
Run it on the Playground.
You were very close with your original solution. You had to de-reference the pointer before calling the append operation. This solution would be helpful if your dest already had some existing elements and you don't want to lose them by creating a newSlice.
tempDest := reflect.ValueOf(dest).Elem()
tempDest = reflect.Append(tempDest, reflect.ValueOf(model.Interface()))
Similar to how #I Love Reflection pointed out, you finally need to set the new slice back to the pointer.
reflect.ValueOf(dest).Elem().Set(tempDest)
Overall Decode:
var modelType reflect.Type = getStructType(dest)
rows, fields := getRows(src)
tempDest := reflect.ValueOf(dest).Elem()
for _, row := range rows {
model := reflect.New(modelType).Elem()
for _, field := range fields {
fieldValue := getRowValue(row, field)
model.FieldByName(field).Set(fieldValue)
}
tempDest = reflect.Append(tempDest, reflect.ValueOf(model.Interface()))
}
reflect.ValueOf(dest).Elem().Set(tempDest)

Go: A function that would consume maps with different types of values

In my code, I need a function that would return an ordered slice of keys from a map.
m1 := make(map[string]string)
m2 := make(map[string]int)
And now I need to call a function passing both types of maps:
keys1 := sortedKeys(m1)
keys2 := sortedKeys(m1)
Problem: I have to write two functions because the function should consume maps of two different types. At the same time, the body of the function will be the same in both cases.
Question: How can I use a single implementation for two maps? Or is there any other way of solving the problem in an elegant way?
My first idea was to use map[string]interface{} as an argument type, but you can't assign neither map[string]string, nor map[string]int to it.
My code:
func sortedKeys(m map[string]string) []string {
var keys []string
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
return keys
}
I would have to repeat the same code but for map[string]int.
You can use interface{} and use reflection for achieving this.
You can write two functions for the same but it is just not scalable, say, you are supporting string and int now but you wish to support int64, float64, bool or struct in the future. Having a common function using map[string]interface{} and using reflection is the way to go.
Suggested Code :
package main
import (
"fmt"
"reflect"
)
func main() {
m1 := make(map[string]string)
m2 := make(map[string]int)
m1["a"] = "b"
m1["b"] = "c"
m2["a"] = 1
m2["b"] = 2
fmt.Println(sortedKeys(m1))
fmt.Println(sortedKeys(m2))
}
// Returns slice of values in the type which is sent to it
func sortedKeys(m interface{}) interface{} {
if m == nil {
return nil
}
if reflect.TypeOf(m).Kind() != reflect.Map {
return nil
}
mapIter := reflect.ValueOf(m).MapRange()
mapVal := reflect.ValueOf(m).Interface()
typ := reflect.TypeOf(mapVal).Elem()
outputSlice := reflect.MakeSlice(reflect.SliceOf(typ), 0, 0)
for mapIter.Next() {
outputSlice = reflect.Append(outputSlice, mapIter.Value())
}
return outputSlice.Interface()
}
Output :
[b c]
[1 2]
https://play.golang.org/p/2fkpydH9idG

how to access nested Json key values in Golang

Team,
new to Programming.
I have data available after unmarshaling the Json as shown below, which has nested Key values. flat key values I am able to access, how do I access nested key values.
Here is the byte slice data shown below after unmarshaling —>
tables:[map[name:basic__snatpool_members] map[name:net__snatpool_members] map[name:optimizations__hosts] map[columnNames:[name] name:pool__hosts rows:[map[row:[ry.hj.com]]]] traffic_group:/Common/traffic-group-1
Flat key values I am able to access by using the following code
p.TrafficGroup = m[“traffic_group”].(string)
here is the complete function
func dataToIapp(name string, d *schema.ResourceData) bigip.Iapp {
var p bigip.Iapp
var obj interface{}
jsonblob := []byte(d.Get("jsonfile").(string))
err := json.Unmarshal(jsonblob, &obj)
if err != nil {
fmt.Println("error", err)
}
m := obj.(map[string]interface{}) // Important: to access property
p.Name = m[“name”].(string)
p.Partition = m[“partition”].(string)
p.InheritedDevicegroup = m[“inherited_devicegroup”].(string)
}
Note: This may not work with your JSON structure. I inferred what it would be based on your question but without the actual structure, I cannot guarantee this to work without modification.
If you want to access them in a map, you need to assert that the interface pulled from the first map is actually a map. So you would need to do this:
tmp := m["tables"]
tables, ok := tmp.(map[string]string)
if !ok {
//error handling here
}
r.Name = tables["name"].(string)
But instead of accessing the unmarshaled JSON as a map[string]interface{}, why don't you create structs that match your JSON output?
type JSONRoot struct {
Name string `json:"name"`
Partition string `json:"partition"`
InheritedDevicegroup string `json:"inherited_devicegroup"`
Tables map[string]string `json:"tables"` //Ideally, this would be a map of structs
}
Then in your code:
func dataToIapp(name string, d *schema.ResourceData) bigip.Iapp {
var p bigip.Iapp
var obj &JSONRoot{}
jsonblob := []byte(d.Get("jsonfile").(string))
err := json.Unmarshal(jsonblob, &obj)
if err != nil {
fmt.Println("error", err)
}
p.Name = obj.Name
p.Partition = obj.Partition
p.InheritedDevicegroup = obj.InheritedDevicegroup
p.Name = obj.Tables["name"]
}
JSON objects are unmarshaled into map[string]interface{}, JSON arrays into []interface{}, same applies for nested objects/arrays.
So for example if a key/index maps to a nested object you need to type assert the value to map[string]interface{} and if the key/index maps to an array of objects you first need to assert the value to []interface{} and then each element to map[string]interface{}.
e.g. (for brevity this code is not guarding against panic)
tables := obj.(map[string]interface{})["tables"]
table1 := tables.([]interface{})[0]
name := table1.(map[string]interface{})["name"]
namestr := name.(string)
However, if it's the case that the json you are parsing is not dynamic but instead has a specific structure you should define a struct type that mirrors that structure and unmarshal the json into that.
All you have to do is repeatedly accessing the map via type-switching or assertion:
for _, table := range m["tables"] {
switch val := table {
case string:
fmt.Println("table is string")
case int:
fmt.Println("table is integer")
// This is your case, since JSON is unmarshaled to type []interface{} and map[string]interface{}
case []interface{}:
fmt.Println("table is a slice of interface{}")
for _, tb := range value {
if m, ok := tb.(map[string]interface{}); ok {
// Now it's accessible
fmt.Println(m["name"])
}
}
default:
fmt.Println("unknown type")
}
}
You might want to handle errors better than this.
To read more, check out my writing from a while ago https://medium.com/code-zen/dynamically-creating-instances-from-key-value-pair-map-and-json-in-go-feef83ab9db2.

How to pass slice of struct as pointer to a function and modify it?

I have a slice of struct []student, and I want to modify its content with function.
type student struct {
name string
age int
}
students := []student{
{"Doraemon", 30},
{"King Kong", 25},
}
Thus, I decided to pass it as a pointer. May I know how to pass the slice as a reference to a function?
func addAge (s *[]student) error { //this code has error
//everyone add 2 years old
for i, e := range *s {
s[i].age = s[i].age + 2
}
//make the first student much older
s[0].age = s[0].age + 5
return nil
}
I keep playing with Go Playground, but it gives many complains, such as
cannot range over s (type *[]student)
invalid operation: s[i] (type *[]student does not support indexing)
invalid indirect of s
...
How to precisely pass the reference of a slice of struct to a function? How to range the slice of struct? And how to change the value of the struct (modify the same struct in THE slice)?
I keep getting error while playing with s *[]student, range *s, s []student, s *[]*student ... so hard to get it correct...
sorry for my NEWBIE question, still learning GO... trying hard
Slices are passed by reference, so as long as you are modifying the existing slice content you should not explicitly pass a pointer.
package main
import (
"fmt"
)
type student struct {
name string
age int
}
func main() {
students := []student{
{"Doraemon", 30},
{"King Kong", 25},
}
err := addAge (students)
fmt.Println(students)
if err != nil {
fmt.Println("error")
}
}
func addAge (s []student) error {
for i, _ := range s {
s[i].age = 3
}
return nil
}
Now, for your addAdditinalStudent function you should actually use the append function. Plus, have in mind
..., since the slice header is always updated by a call to
append, you need to save the returned slice after the call. In fact,
the compiler won't let you call append without saving the result.
Slices#append
// add student
students = append(students, student{"Test", 33})
Go Playground
in Go you can pass items by value ([]student) or by reference ([]*student). When you want to operate on the values of a struct{} you should pass it to a function with its reference (the pointer).
So you can do something like this:
type student struct {
name string
age int
}
func addTwoYearsToAll(students []*student){
for _, s := range students {
s.age += 2
}
}
This way you're working with the same exact items you build when appending to the slice. Playground example.
Also take a look at Are Golang function parameter passed as copy-on-write?

Resources