I have next yaml, if I validate it in online yaml website, it said it's valid:
- {"foo": "1", "bar": "2"}
Then, I write a code to parse the value 1 and 2 from this yaml as next:
test.go:
package main
import "gopkg.in/yaml.v2"
import "fmt"
type Config struct {
Foo string
Bar string
}
type Configs struct {
Cfgs []Config `foobar`
}
func main() {
//var data = `
// foobar:
// - foo: 1
// bar: 2
//`
var data = `
- foo: 1
bar: 2
`
source := []byte("foobar:" + data)
var configs Configs
err := yaml.Unmarshal(source, &configs)
if err != nil {
fmt.Println("error: %v", err)
}
fmt.Printf("--- configs:\n%v\n\n", configs)
fmt.Println(configs.Cfgs[0].Foo)
fmt.Println(configs.Cfgs[0].Bar)
}
It works ok as next:
shubuntu1#shubuntu1:~/20210810$ go run test.go
--- configs:
{[{1 2}]}
1
2
What's the problem?
You could see I made a workaround here as next to add special foobar key before the original yaml, then I could use type Configs struct to unmarshal it:
From
- foo: 1
bar: 2
to
foobar:
- foo: 1
bar: 2
So, if I don't use the workaround to add a prefix foobar:, how could I directly parse - {"foo": 1, "bar": 2}?
Since your YAML data is an array, unmarshal it to an array of Config structure.
package main
import (
"fmt"
"gopkg.in/yaml.v2"
)
type Config struct {
Foo string
Bar string
}
func main() {
var configs []Config
var data = `
- foo: 1
bar: 2
`
err := yaml.Unmarshal([]byte(data), &configs)
if err != nil {
panic(err)
}
fmt.Println(configs)
}
Output:
[{1 2}]
Try on - Go Playground
Since the question has already been answered, I thought I might add something to the discussion:
var data = `
- foo: 1
bar: 2
`
The data variable here, the way you wrote it, will include the indentation at the start of each line.
The extra indentation here WILL be passed into yaml.Unmarshal() alongside all the actual yaml data, which can mess things up since gofmt formats your code to use TAB as indentation and TAB indentation is forbidden in yaml (https://stackoverflow.com/a/19976827/7509248).
If you use tab as indentation in yaml, an error like this will be thrown when trying to unmarshal it:
yaml: line 1: found a tab character that violates indentation
Preferably, you want to load data from a separate file to avoid such issue.
package main
import (
"fmt"
"io/ioutil"
"gopkg.in/yaml.v2"
)
type Config struct {
Foo string
Bar string
}
func main() {
var configs []Config
source, err := ioutil.ReadFile("config.yaml")
if err != nil {
fmt.Printf("failed reading config file: %v\n", err)
}
err = yaml.Unmarshal(source, &configs)
if err != nil {
fmt.Printf("error: %v\n", err)
}
fmt.Printf("config:\n%+v\n", configs)
}
config.yaml:
- foo: 1
bar: 2
output:
config:
[{Foo:1 Bar:2}]
Related
I am trying to parse a yaml file into go using the "gopkg.in/yaml.v3" package.
The problem I haven't been able to solve is parsing a file starting with a -. For example:
---
- type: people
info:
- { name: John, last: Doe }
...
So when I try to parse this
package main
import (
"fmt"
"io/ioutil"
"log"
"gopkg.in/yaml.v3"
)
type YamlFile struct {
type string `yaml:"type"`
}
func main() {
d := YamlFile{}
src, err := ioutil.ReadFile("test1.yaml")
if err != nil {
log.Println(err)
}
err = yaml.Unmarshal(src, &d)
if err != nil {
log.Printf("error: %v", err)
}
fmt.Println(d)
}
output: error: yaml: unmarshal errors: line 2: cannot unmarshal !!seq into main.YamlFile
The above code works when the - is removed from the file
---
type: people
info:
- { name: John, last: Doe }
...
However I cannot reformat the file so I need to know what I am doing wrong trying to parse with the - in place. Thanks for any pointers in the right direction.
The - indicates that it's a list/array. Therefore you must unmarshal into a slice or array in Go.
Change d := YamlFile{} to d := []YamlFile{}, and you will no longer get that error.
But also note that you'll always get an empty result with the struct you've defined, because it has no exported fields.
Try instead:
type YamlFile struct {
Type string `yaml:"type"`
}
See it on the playground.
I am trying to unmasrhal the following flux HelmRelease file.
apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
annotations:
fluxcd.io/automated: 'false'
fluxcd.io/tag.ats: glob:*
name: ats
namespace: myns
spec:
chart:
git: git#github.com:reponame/project.git
path: charts/path1/path1/myapp
ref: master
releaseName: foobar
values:
allowAllEgress: true
recycleApp: true
hooks:
slackChannel: https://hooks.slack.com/services/something/somethingelse/
Here are my models
type HelmReleaseValues struct {
AllowAllEgress bool `yaml:"allowAllEgress"`
RecycleApp bool `yaml:"recycleApp"`
Hooks `yaml:"hooks"`
}
type Hooks struct {
SlackChannel string `yaml:"slackChannel"`
}
type Values struct {
HelmReleaseValues `yaml:"values"`
ReleaseName string `yaml:"releaseName"`
Chart `yaml:"chart"`
}
type Spec struct {
Values `yaml:"spec"`
}
The problem is that the fields allowAllEgress and recycleApp are getting unmarshalled.
However the Hooks field in my struct turns out to be empty.
What am I doing wrong in the struct modelling / tagging?
edit: here is my code
package main
import (
"fmt"
"io/ioutil"
"os"
"github.com/davecgh/go-spew/spew"
"gopkg.in/yaml.v3"
)
const ExitCodeCmdErr = 1
func main() {
rawYaml := parseHelmReleaseFile("myfile.yaml")
spew.Dump(rawYaml)
}
func parseHelmReleaseFile(fileName string) Spec {
var v Spec
yamlFile, err := ioutil.ReadFile(fileName)
if err != nil {
fmt.Printf("yaml file err #%v ", err)
os.Exit(ExitCodeCmdErr)
}
err = yaml.Unmarshal(yamlFile, &v)
if err != nil {
fmt.Printf("Unmarshal: %v", err)
os.Exit(ExitCodeCmdErr)
}
return v
}
I am running the program and grepping for the output (the actual helm release file is huge)
▶ go clean && gb .
~/Desktop/yamltutorial
./foobar | grep -i hooks -A 3
--
Hooks: (main.Hooks) {
SlackChannel: (string) ""
}
},
You did not have Chart struct
type Chart struct {
Git string `yaml:"git"`
Path string `yaml:"path"`
Ref string `yaml:"ref"`
}
Added that and got the following output
{Values:{HelmReleaseValues:{AllowAllEgress:true RecycleApp:true Hooks:{SlackChannel:https://hooks.slack.com/services/something/somethingelse/}} ReleaseName:foobar Chart:{Git:git#github.com:reponame/project.git Path:charts/path1/path1/myapp Ref:master}}}
Playground file with complete code.
https://play.golang.org/p/vCnjApr6gI9
Given the following YAML:
array.test: ["val1", "val2", "val3"]
I Unmarshal it using gopkg.in/yaml.v2 into a map[string]interface{}. Then I get a single key whose value is an array of 3 values.
When I then Marshal it again to YAML, the resulting YAML looks like this:
array.test:
- val1
- val2
- val3
The array was actually marshaled as a sequence instead of an array.
This is the entire GoLang code:
func main(){
data := `array.test: ["val1", "val2", "val3"]`
conf := make(map[string]interface{})
yaml.Unmarshal([]byte(data), conf)
data2, _ := yaml.Marshal(conf)
fmt.Printf("%s\n", string(data2))
}
How can I overcome this issue?
This one helped me in the same case as you.
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var data = `
a: Easy!
b:
c: 2
d.test: ["d1", "d2"]
`
// Note: struct fields must be public in order for unmarshal to
// correctly populate the data.
type T struct {
A string
B struct {
RenamedC int `yaml:"c"`
DTest []string `yaml:"d.test,flow"`
}
}
func main() {
// if we use struct containing yaml encoding for yaml formated string
t := T{}
err := yaml.Unmarshal([]byte(data), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t after unmarshal:\n%v\n\n", t)
d, err := yaml.Marshal(&t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t after marshal:\n%s\n\n", string(d))
}
Ref: https://github.com/go-yaml/yaml
Use flow in struct field tag format, to indicate you desire this behavior. But, of course, this requires unmarshaling to a struct, not to a map.
Flow tag allows you to choose the representation of an array in yaml
package main
import (
"fmt"
"gopkg.in/yaml.v2"
)
type Conf struct {
Test []string `yaml:"array.test,flow"`
}
func main(){
data := `array.test: ["val1", "val2", "val3"]`
var conf Conf
yaml.Unmarshal([]byte(data), &conf)
data2, _ := yaml.Marshal(conf)
fmt.Printf("%s\n", string(data2))
}
How does one parse multiple yamls in a file similar to how kubectl does it?
example.yaml
---
a: Easy!
b:
c: 0
d: [1, 2]
---
a: Peasy!
b:
c: 1000
d: [3, 4]
There's a difference in behavior between gopkg.in/yaml.v2 and gopkg.in/yaml.v3:
V2: https://play.golang.org/p/XScWhdPHukO
V3: https://play.golang.org/p/OfFY4qH5wW2
Both implementations produce an incorrect result IMHO but V3 is apparently slightly worse.
There's a workaround. If you slightly change code in the accepted answer, it works correctly and in the same fashion with both versions of yaml package: https://play.golang.org/p/r4ogBVcRLCb
Current gopkg.in/yaml.v3 Deocder produces quite correct result, you just need to pay attention on creating new structure for each document and check it was parsed correctly (with nil check), and handle EOF error correctly:
package main
import "fmt"
import "gopkg.in/yaml.v3"
import "os"
import "errors"
import "io"
type Spec struct {
Name string `yaml:"name"`
}
func main() {
f, err := os.Open("spec.yaml")
if err != nil {
panic(err)
}
d := yaml.NewDecoder(f)
for {
// create new spec here
spec := new(Spec)
// pass a reference to spec reference
err := d.Decode(&spec)
// check it was parsed
if spec == nil {
continue
}
// break the loop in case of EOF
if errors.Is(err, io.EOF) {
break
}
if err != nil {
panic(err)
}
fmt.Printf("name is '%s'\n", spec.Name)
}
}
Test file spec.yaml:
---
name: "doc first"
---
name: "second"
---
---
name: "skip 3, now 4"
---
Solution I found using gopkg.in/yaml.v2:
package main
import (
"bytes"
"fmt"
"io/ioutil"
"log"
"path/filepath"
"gopkg.in/yaml.v2"
)
type T struct {
A string
B struct {
RenamedC int `yaml:"c"`
D []int `yaml:",flow"`
}
}
func main() {
filename, _ := filepath.Abs("./example.yaml")
yamlFile, err := ioutil.ReadFile(filename)
if err != nil {
panic(err)
}
r := bytes.NewReader(yamlFile)
dec := yaml.NewDecoder(r)
var t T
for dec.Decode(&t) == nil {
fmt.Printf("a :%v\nb :%v\n", t.A, t.B)
}
}
This Go Yaml Interpretation Example,
https://gist.github.com/suntong001/74c85d15b19ef4b14b0e, can unmarshal a simple YAML like this:
foo: 1
bar:
- one
- two
Now I want to interpretation an array of the exact same data structure, what's the correct way to do?
Below is my code so far, and it is not working as expected. Please help.
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
type Config struct {
Foo string
Bar []string
}
type Configs struct {
Cfgs []Config
}
var data = `
- foo: 1
bar:
- one
- two
- three
- foo: 2
bar:
- one1
- two2
- three3
`
func main() {
var config Configs
/*
filename := os.Args[1]
source, err := ioutil.ReadFile(filename)
if err != nil {
panic(err)
}
*/
source := []byte(data)
err := yaml.Unmarshal(source, &config)
if err != nil {
log.Fatalf("error: %v", err)
}
//fmt.Printf("Value: %#v\n", config.Bar[0])
fmt.Printf("--- config:\n%v\n\n", config)
}
UPDATE:
To make it work, Jfly pointed out, simply replace my var config Configs with var config []Config, and it does the trick. Now I think if change my data definition to the following, my above code would work.
foobars:
- foo: 1
bar:
- one
- two
- three
- foo: 2
bar:
- one1
- two2
- three3
Nops, it doesn't. Please help.
The contents of the example yaml file are sequence of objects, so do it like this:
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
type Config struct {
Foo string
Bar []string
}
var data = `
- foo: 1
bar:
- one
- two
- three
- foo: 2
bar:
- one1
- two2
- three3
`
func main() {
var config []Config
/*
filename := os.Args[1]
source, err := ioutil.ReadFile(filename)
if err != nil {
panic(err)
}
*/
source := []byte(data)
err := yaml.Unmarshal(source, &config)
if err != nil {
log.Fatalf("error: %v", err)
}
//fmt.Printf("Value: %#v\n", config.Bar[0])
fmt.Printf("--- config:\n%v\n\n", config)
}
As to your updated question, your code almost works, just give a hint to the yaml parser like this:
type Configs struct {
Cfgs []Config `foobars`
}