I am fairly new to Golang, please excuse my newbyness.
I am currently using yaml.v2 package (https://github.com/go-yaml/yaml) to unmarshal YAML data into structs.
Consider the following example code:
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"log"
)
type Container struct {
First string
Second struct {
Nested1 string
Nested2 string
Nested3 string
Nested4 int
}
}
var data = `
first: first value
second:
nested1: GET
nested2: /bin/bash
nested3: /usr/local/bin/customscript
nested4: 8080
first: second value
second:
nested1: POST
nested2: /bin/ksh
nested3: /usr/local/bin/customscript2
nested4: 8081
`
func main() {
container := Container{}
err := yaml.Unmarshal([]byte(data), &container)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("---values found:\n%+v\n\n", container)
}
The result:
---values found: {First:second value Second:{Nested1:POST Nested2:/bin/ksh Nested3:/usr/local/bin/customscript2 Nested4:8081}}
This is as expected, the unmarshal function finds one occurrence of the YAML data.
What I would like to do is write a simple while/each/for loop that loops through the data variable and marshals all the occurrences into seperate Container structs. How could I achieve this?
A simple change to accomplish what you want is to have the data in the yaml be items in an array, then Unmarshal into a slice of Container
var data = `
- first: first value
second:
nested1: GET
nested2: /bin/bash
nested3: /usr/local/bin/customscript
nested4: 8080
- first: second value
second:
nested1: POST
nested2: /bin/ksh
nested3: /usr/local/bin/customscript2
nested4: 8081
`
func main() {
container := []Container{}
err := yaml.Unmarshal([]byte(data), &container)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("---values found:\n%+v\n\n", container)
}
---values found:
[{First:first value Second:{Nested1:GET Nested2:/bin/bash Nested3:/usr/local/bin/customscript Nested4:8080}} {First:second value Second:{Nested1:POST Nested2:/bin/ksh Nested3:/usr/local/bin/customscript2 Nested4:8081}}]
Related
I really searched a while here, but didn't found an adequate answer:
I am trying to unmarshall yaml dict keys onto a property of a struct rather than the key of a map.
Given this yaml
commands:
php:
service: php
bin: /bin/php
node:
service: node
bin: /bin/node
I am able to unmarshall this into a struct like this:
type Config struct {
Commands map[string]struct {
Service string
Bin string
}
}
But how am I able to unmarshall it into a struct like this:
type Config struct {
Commands []struct {
Name string // <-- this should be key from the yaml (i.e. php or node)
Service string
Bin string
}
}
Thx in advance for the help
You can write a custom unmarshaler, like this (on Go playground):
package main
import (
"fmt"
"gopkg.in/yaml.v3"
)
var input []byte = []byte(`
commands:
php:
service: php
bin: /bin/php
node:
service: node
bin: /bin/node
`)
type Command struct {
Service string
Bin string
}
type NamedCommand struct {
Command
Name string
}
type NamedCommands []NamedCommand
type Config struct {
Commands NamedCommands
}
func (p *NamedCommands) UnmarshalYAML(value *yaml.Node) error {
if value.Kind != yaml.MappingNode {
return fmt.Errorf("`commands` must contain YAML mapping, has %v", value.Kind)
}
*p = make([]NamedCommand, len(value.Content)/2)
for i := 0; i < len(value.Content); i += 2 {
var res = &(*p)[i/2]
if err := value.Content[i].Decode(&res.Name); err != nil {
return err
}
if err := value.Content[i+1].Decode(&res.Command); err != nil {
return err
}
}
return nil
}
func main() {
var f Config
var err error
if err = yaml.Unmarshal(input, &f); err != nil {
panic(err)
}
for _, cmd := range f.Commands {
fmt.Printf("%+v\n", cmd)
}
}
I have split the command data into Command and NamedCommand to make the code simpler, since you can just call Decode giving the embedded Command struct for the values. If everything was in the same struct, you'd need to manually map keys to struct fields.
I'm getting time to learn Go, and I'm having a problem dealing with yaml file.
this is my yaml file
---
endpoints:
service1:
url: "https://service1.com"
frequency: 2
interval: 1
service2:
url: "https://service2.com"
frequency: 3
interval: 2
My go code
package main
import (
"fmt"
"io/ioutil"
"reflect"
"gopkg.in/yaml.v3"
)
// Config define estrutura do arquivo de configuração
type Config struct {
Endpoint map[string]interface{} `yaml:"endpoints"`
}
func main() {
yamlFile, err := ioutil.ReadFile("config.yml")
if err != nil {
fmt.Printf("Error reading YAML file: %s\n", err)
return
}
var yamlConfig Config
err = yaml.Unmarshal(yamlFile, &yamlConfig)
if err != nil {
fmt.Printf("Error parsing YAML file: %s\n", err)
}
for k := range yamlConfig.Endpoint {
nm := reflect.ValueOf(yamlConfig.Endpoint[k])
for _, key := range nm.MapKeys() {
strct := nm.MapIndex(key)
fmt.Println(key.Interface(), strct.Interface())
}
}
}
// PingEndpoint acessa os endpoint informados
func PingEndpoint(url string, frequency, interval int) {
// do something
}
Is there a better way to define the Config structure without using interface? And is really necessary use reflect do get properties of a service1 or exist a better whey to do that?
In general, an interface{} is used in this situation if you don't know the structure. In this case, the structure appears to be fixed:
type Service struct {
URL string `yaml:"url"`
Frequency int `yaml:"frequency"`
Interval int `yaml:"interval"`
}
type Config struct {
Endpoint map[string]Service `yaml:"endpoints"`
}
For your second question, you no longer need to deal with unknown fields after you do this, but even if you had interface{}, you can use type assertions (type yaml library unmarshals yaml into a map[interface{}]interface{}):
for k := range yamlConfig.Endpoint {
if mp, ok:=yamlConfig.Endpoint[k].(map[interface{}]interface{}); ok {
for key, value:=range mp {
}
}
}
I'm trying to parse this yaml and I want to get the values of the run entry (test1 or test2) without success, here is my working example.
im a bit get lost with the map inside map :( ,
this is given yaml which I couldent change ...
any idea how could I got those values
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
version: "3.2"
run-parameters:
before:
run-parameters:
run: test1
after:
run-parameters:
run: test2
`)
type FTD struct {
Version string `yaml:"version,omitempty"`
BuildParams *RunParams `yaml:"run-parameters,omitempty"`
}
type RunParams struct {
BeforeExec map[string]interface{} `yaml:"before,omitempty"`
AfterExec map[string]interface{} `yaml:"after,omitempty"`
}
func main() {
runners := &FTD{}
// parse mta yaml
err := yaml.Unmarshal(runContent, runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
for k, v := range runners.BuildParams.BeforeExec {
fmt.Println(k, v.(interface{}))
}
}
This is working example
https://play.golang.org/p/qTqUJy3md0c
also I've tried with
this is working
run := runners.BuildParams.BeforeExec["run-parameters"].(map[interface{}]interface{})["run"]
fmt.Println("run: ", run)
what I've tried is this which works but what happens if the run value is empty or no entry at all,this will cause a dump how can I overcome this ?
what I've tried is this which works but what happens if the run value is empty or no entry at all,this will cause a dump how can I overcome this ?
You can do
runParams, ok := runners.BuildParams.BeforeExec["run-parameters"]
if !ok {
// handle lack of "run-parameters" in BeforeExec
}
runParamsMap, ok := runParams.(map[interface{}]interface{})
if !ok {
// handle "run-parameters" not being a map
}
run, ok := runParamsMap["run"]
if !ok {
// handle lack of "run" inside "run-parameters"
}
runStr, ok := run.(string)
if !ok {
// handle "run" not being a string
}
fmt.Println("run: ", runStr)
This is quite verbose so you could use something like https://github.com/jmoiron/jsonq, where you can specify a "path" to the desired value nested inside several levels of maps. Despite the "json" in the name, this library works with map[string]interface{} and not json files. But note that the library you use for yaml unmarshalling results in map[interface{}]interface{} instead of map[string]interface{} and you will have to use a different one in order for it to work with jsonq.
run, err := jsonq.NewQuery(runners.BuildParams.BeforeExec).String("run-parameters", "run")
if err != nil {
// handle all possible errors in one place
}
fmt.Println("run: ", run)
Given the following YAML:
array.test: ["val1", "val2", "val3"]
I Unmarshal it using gopkg.in/yaml.v2 into a map[string]interface{}. Then I get a single key whose value is an array of 3 values.
When I then Marshal it again to YAML, the resulting YAML looks like this:
array.test:
- val1
- val2
- val3
The array was actually marshaled as a sequence instead of an array.
This is the entire GoLang code:
func main(){
data := `array.test: ["val1", "val2", "val3"]`
conf := make(map[string]interface{})
yaml.Unmarshal([]byte(data), conf)
data2, _ := yaml.Marshal(conf)
fmt.Printf("%s\n", string(data2))
}
How can I overcome this issue?
This one helped me in the same case as you.
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var data = `
a: Easy!
b:
c: 2
d.test: ["d1", "d2"]
`
// Note: struct fields must be public in order for unmarshal to
// correctly populate the data.
type T struct {
A string
B struct {
RenamedC int `yaml:"c"`
DTest []string `yaml:"d.test,flow"`
}
}
func main() {
// if we use struct containing yaml encoding for yaml formated string
t := T{}
err := yaml.Unmarshal([]byte(data), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t after unmarshal:\n%v\n\n", t)
d, err := yaml.Marshal(&t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t after marshal:\n%s\n\n", string(d))
}
Ref: https://github.com/go-yaml/yaml
Use flow in struct field tag format, to indicate you desire this behavior. But, of course, this requires unmarshaling to a struct, not to a map.
Flow tag allows you to choose the representation of an array in yaml
package main
import (
"fmt"
"gopkg.in/yaml.v2"
)
type Conf struct {
Test []string `yaml:"array.test,flow"`
}
func main(){
data := `array.test: ["val1", "val2", "val3"]`
var conf Conf
yaml.Unmarshal([]byte(data), &conf)
data2, _ := yaml.Marshal(conf)
fmt.Printf("%s\n", string(data2))
}
I am currently unable to unmarshall the data correctly from a map into a structure. Following is a code snippet (Brief Code at playground):
Request you to kindly provide the reason of getting default values on unmarshlling back the data.
package main
import (
"fmt"
"encoding/json"
"os"
)
func main() {
fmt.Println("Hello, playground")
type PDPOffer struct {
cart_value int `json:"cart_value"`
discount_amount_default int `json:"discount_amount_default"`
max_discount string `json:"max_discount"`
}
a:= map[string]interface{} {
"cart_value" : 1,
"max_discount" : 2,
}
var pdf PDPOffer
b, err := json.Marshal(a)
if err != nil {
fmt.Println("error:", err)
}
os.Stdout.Write(b)//working
err1 := json.Unmarshal(b, &pdf)
if err1 != nil {
fmt.Println("error:", err)
}
fmt.Printf("%+v", pdf)//displaying just the defualt values????????
}
json.Marshal and json.Unmarshal can only work on exported struct fields. Your fields aren't exported and aren't visible to the json code.
The reason for your unmarshal not working is that you need to expose fields of the struct and for doing that you need to start field name with Capital letters. Something has below:
type PDPOffer struct {
Cart_value int `json:"cart_value"`
Discount_amount_default int `json:"discount_amount_default"`
Max_discount string `json:"max_discount"`
}
Also, you were trying to marshal an int value into a string for "max_discount", you need to store it as a string in your map that you're marshaling:
a := map[string]interface{}{
"cart_value": 1,
"max_discount": "2",
}
The error handling had a bug checking for err1 != nil then printing err which was hiding the message error: json: cannot unmarshal number into Go value of type string
Working example with all the fixes: http://play.golang.org/p/L8VC-531nS