Function doesn't recognize Yaml nested struct - go

I have the following struct YAML:
type YamlConfig struct {
Items struct {
RiskyRoles []struct {
Name string `yaml:"name"`
Rules []struct{
Verbs []string `yaml:"verbs"`
ResourceOperator string `yaml:"resourcesOperator"`
Resources []string `yaml:"resources"`
}
} `yaml:"RiskyRoles"`
} `yaml:"Items"`
}
I have a function that parse a YAML file to an object and then I want to send the Rules struct object to a function named DoingStuff(..):
yamlFile, err := ioutil.ReadFile("actionItems.yaml")
if err != nil {
fmt.Printf("Error reading YAML file: %s\n", err)
} else{
var yamlConfig YamlConfig
err = yaml.Unmarshal(yamlFile, &yamlConfig)
if err != nil {
fmt.Printf("Error parsing YAML file: %s\n", err)
}
for _, yamlRole := range yamlConfig.Items.RiskyRoles{
DoingStuff(yamlRole.Rules)
}
}
But inside the function DoingStuff, the struct object Rules is not recognized:
func DoingStuff(yamlRules []struct{}) {
// Not recognize ****
for _, rule := range yamlRules {
fmt.Print(rule.ResourceOperator)
}
}
How can I convert it to:
Rules []struct{
Verbs []string `yaml:"verbs"`
ResourceOperator string `yaml:"resourcesOperator"`
Resources []string `yaml:"resources"`
}
Should I re-declare this struct again?
Or cast using interfaces ?
EDIT:
I added new struct and used it inside the YamlConfig struct but the parse failed to parse the Rules:
type RulesStruct struct {
Rules []struct{
Verbs []string `yaml:"verbs"`
ResourceOperator string `yaml:"resourcesOperator"`
Resources []string `yaml:"resources"`
}
}
type YamlConfig struct {
Items struct {
RiskyRoles []struct {
Name string `yaml:"name"`
Message string `yaml:"message"`
Priority string `yaml:"priority"`
Rules []RulesStruct
} `yaml:"RiskyRoles"`
} `yaml:"Items"`
}

Thanks to #mkporiva help, I changed the structs like that:
type RulesStruct struct {
Verbs []string `yaml:"verbs"`
ResourceOperator string `yaml:"resourcesOperator"`
Resources []string `yaml:"resources"`
}
type YamlConfig struct {
Items struct {
RiskyRoles []struct {
Name string `yaml:"name"`
Message string `yaml:"message"`
Priority string `yaml:"priority"`
Rules []RulesStruct
} `yaml:"RiskyRoles"`
} `yaml:"Items"`
}
Now it works fine.

Related

Unmarshal YAML into complex object which may be either struct or string

Trying to unmarshal YAML into complex object such as map[string]map[interface{}]string.
The problem is that I want to be able to differentiate an interface{} part between string and Source which is a struct.
type Source struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
LogoID string `yaml:"logoId"`
URL string `yaml:"url"`
}
type UNFT struct {
ItemMeta map[string]map[interface{}]string `yaml:"item_meta"`
// could be
// ItemMeta map[string]map[string]string `yaml:"item_meta"`
// or
// ItemMeta map[string]map[Source]string `yaml:"item_meta"`
}
Obviously YAML does not know how to unmarshal into Source struct so I have to implement Unmarshaler interface:
type Unmarshaler interface {
UnmarshalYAML(value *Node) error
}
But I don't quite understand the big picture of unmarshaling process. In general I assume that I have to manually traverse *yaml.Node and call func UnmarshalYAML(value *Node) error on every node.
package main
import (
"fmt"
"gopkg.in/yaml.v3"
)
type Source struct {
ID string `json:"id"`
Name string `json:"name"`
LogoID string `json:"logoId"`
URL string `json:"url"`
}
var data = `
unf:
item_meta:
source:
!struct
? id: "data-watch"
name: "DataWatch"
logoid: "data-watch"
url: "https"
: "product_any('SS')"
public_usage:
"": "source_any('SDF')"
"provider": "source_any('ANO')"`
type UNFT struct {
ItemMeta map[string]map[interface{}]string `yaml:"item_meta"`
}
type MetaConverterConfigT struct {
UNFT UNFT `yaml:"unf"`
}
func main() {
cfg := MetaConverterConfigT{}
err := yaml.Unmarshal([]byte(data), &cfg)
if err != nil {
fmt.Println("%w", err)
}
fmt.Println(cfg)
}
func (s *UNFT) UnmarshalYAML(n *yaml.Node) error {
var cfg map[string]map[interface{}]string
if err := n.Decode(&cfg); err != nil {
fmt.Println("%w", err)
}
return nil
}
Go playground
type MetaKey struct {
String string
Source Source
}
func (k *MetaKey) UnmarshalYAML(n *yaml.Node) error {
if n.Tag == "!!str" {
return n.Decode(&k.String)
}
if n.Tag == "!!map" {
return n.Decode(&k.Source)
}
return fmt.Errorf("unsupported MetaKey type")
}
// ...
type UNFT struct {
ItemMeta map[string]map[MetaKey]string `yaml:"item_meta"`
}
https://go.dev/play/p/Nhtab4l-ANT
If you need the map type to remain as is, i.e. without adding the custom key type, then you can implement the unmarshaler on UNFT as well and just do a re-mapping with any:
type UNFT struct {
ItemMeta map[string]map[any]string `yaml:"item_meta"`
}
func (u *UNFT) UnmarshalYAML(n *yaml.Node) error {
var obj struct {
ItemMeta map[string]map[MetaKey]string `yaml:"item_meta"`
}
if err := n.Decode(&obj); err != nil {
return err
}
u.ItemMeta = make(map[string]map[any]string, len(obj.ItemMeta))
for k, v := range obj.ItemMeta {
m := make(map[any]string, len(v))
for k, v := range v {
if k.Source != (Source{}) {
m[k.Source] = v
} else {
m[k.String] = v
}
}
u.ItemMeta[k] = m
}
return nil
}
https://go.dev/play/p/uwboGKf3qnD

why the "switch t := policy.Parameter.(type)" does not work with viper.Unmarshal()

I want to unmarshal several types from JSON and use the interface to represent the actual struct that it is different. But when I send the struct as interface{} it converts it to a map. The animal.json is:
"{"type":"cat","policies":[{"name":"kitty","parameter":{"duration":600,"percent":90}}]}"
package main
import (
"reflect"
log "github.com/sirupsen/logrus"
"github.com/spf13/viper"
)
func main() {
var err error
animal := New()
viper.SetConfigType("json")
viper.SetConfigName("animal")
viper.AddConfigPath("~/Desktop/")
viper.AddConfigPath(".")
if err := viper.ReadInConfig(); err != nil {
return
}
if err = viper.Unmarshal(&animal); err != nil {
return
}
for _, policy := range animal.Policies {
log.Info(policy.Name)
log.Info(policy.Parameter)
//INFO[0000] map[duration:600 percent:90]
log.Info(reflect.TypeOf(policy.Parameter))
//INFO[0000] map[string]interface {}, Why is it not an interface{} and how do I get it?
switch t := policy.Parameter.(type) {
//why does the switch not work?
case *CatParameter:
log.Info("cat", t)
case *DogParameter:
log.Info("dog", t)
}
}
}
func New() *Animal {
var animal Animal
animal.Type = "cat"
return &animal
}
type Animal struct {
Type string `json:"type" form:"type"`
Policies []Policy `json:"policies" form:"policies"`
}
type CatParameter struct {
Duration int `json:"duration"`
Percent int `json:"percent"`
}
type DogParameter struct {
Percent int `json:"percent"`
Duration int `json:"duration"`
Operation string `json:"operation"`
}
type Policy struct {
Name string `json:"name"`
Parameter interface{} `json:"parameter"`
}
It's json unmarshal feature
If you use an interface{} as a decoder, the default json object for interface{} is map[string]interface{}
You can see it here:
https://godoc.org/encoding/json#Unmarshal
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
So in t := policy.Parameter.(type), the t is map[string]interface{}
For solving your problem, you can try to define another field to distinguish CatParameter or DogParameter
Maybe:
type Policy struct {
Name string `json:"name"`
Parameter Parameter `json:"parameter"`
}
type Parameter struct {
Name string `json:"name"` // cat or dog
Percent int `json:"percent,omitempty"`
Duration int `json:"duration,omitempty"`
Operation string `json:"operation,omitempty"`
}

Parsing Nested JSON string

I am trying to parse a nested json string
I did get it to work by using multiple structs, but I am wondering if I can parse the JSON without using an extra struct.
type Events struct {
Events []Event `json:"events"`
}
type Event struct {
Name string `json:"name"`
Url string `json:"url"`
Dates struct {
Start struct {
LocalDate string
LocalTime string
}
}
}
type Embed struct {
TM Events `json:"_embedded"`
}
func TMGetEventsByCategory(location string, category string) {
parameters := "city=" + location + "&classificationName=" + category + "&apikey=" + api_key
tmUrl := tmBaseUrl + parameters
resp, err := http.Get(tmUrl)
var embed Embed
var tm Event
if err != nil {
log.Printf("The HTTP request failed with error %s\n", err)
} else {
data, _ := ioutil.ReadAll(resp.Body)
err := json.Unmarshal(data, &embed)
json.Unmarshal(data, &tm)
}
}
JSON Data looks like this:
{
"_embedded": {
"events": [],
},
"OtherStuff": {
}
}
Is it possible to get rid of the Embed struct and read straight to the events part of the json string?
What you're looking for here is json.RawMessage. It can help delay parsing of certain values, and in you case map[string]json.RawMessage should represent the top-level object where you can selectively parse values. Here's a simplified example you can adjust to your case:
package main
import (
"encoding/json"
"fmt"
)
type Event struct {
Name string `json:"name"`
Url string `json:"url"`
}
func main() {
bb := []byte(`
{
"event": {"name": "joe", "url": "event://101"},
"otherstuff": 15.2,
"anotherstuff": 100
}`)
var m map[string]json.RawMessage
if err := json.Unmarshal(bb, &m); err != nil {
panic(err)
}
if eventRaw, ok := m["event"]; ok {
var event Event
if err := json.Unmarshal(eventRaw, &event); err != nil {
panic(err)
}
fmt.Println("Parsed Event:", event)
} else {
fmt.Println("Can't find 'event' key in JSON")
}
}
In your case look for the _embedded key and then Unmarshal its value to Events
yes of course
type Embed struct {
TM []struct {
Name string `json:"name"`
Url string `json:"url"`
Dates struct {
Start struct {
LocalDate string
LocalTime string
}
} // add tag here if you want
} `json:"_embedded"`
}

reusing structs in another struct in golang

I have two structs in golang as below
type Data struct {
Name string
Description string
HasMore bool
}
type DataWithItems struct {
Name string
Description string
HasMore bool
Items []Items
}
At most DataWithItems struct can be rewritten as
type DataWithItems struct {
Info Data
Items []Items
}
But the above make it difficult when decoding a json object into DataWithItems. I know this can be solved with inheritance in other programming languages but Is there a way I can solve this in Go?
You can "embed" the one struct into the other:
type Items string
type Data struct {
Name string
Description string
HasMore bool
}
type DataWithItems struct {
Data // Notice that this is just the type name
Items []Items
}
func main() {
d := DataWithItems{}
d.Data.Name = "some-name"
d.Data.Description = "some-description"
d.Data.HasMore = true
d.Items = []Items{"some-item-1", "some-item-2"}
result, err := json.Marshal(d)
if err != nil {
panic(err)
}
println(string(result))
}
this prints
{"Name":"some-name","Description":"some-description","HasMore":true,"Items":["some-item-1","some-item-2"]}
Just use one struct - DataWithItems and sometimes leave items blank

How to parse each HCL dictionary item by golang?

I have tried to parse HCL config using golang, but it's not working.
type cfg_dict struct {
name string `hcl:",key"`
type string `hcl:"type"`
}
type hcl_config struct {
config_items cfg_dict `hcl:"config"`
}
func main() {
hcl_example = `config "cfg1" {
type = "string"
}`
hcl_opts := &hcl_config{}
hcl_tree, err := hcl.Parse(hcl_example)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
if err := hcl.DecodeObject(&hcl_opts, hcl_tree); err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println(hcl_opts)
}
When I tried to run this test code after built, it shows empty value.
&{[]}
Is there any problem what I have to fix?
Fields on the struct you are attempting to unmarshal from HCL need to be exported. To export the fields make the first character in the field name upper case.
type cfg_dict struct {
Name string `hcl:",key"`
Type string `hcl:"type"`
}
type hcl_config struct {
Config_items cfg_dict `hcl:"config"`
}

Resources