Parse yaml error for map - go

I've the following program in which I need to parse yaml
with the following structure
https://codebeautify.org/yaml-validator/cbabd352
this is valid yaml and I use byte to make it more simple
maybe the indentation was changed during the copy paste to the question but you can see in the link that the yaml is valid
The yaml have an api_version
and runners, for each runner (key which is name) I've a list of commands
and I need to print those commands for function1 and function4 ,what am I doing wrong here ?
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
api_ver: 1
runners:
- name: function1
type:
- command: spawn child process
- command: build
- command: gulp
- name: function2
type:
- command: run function 1
- name: function3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Result struct {
Version string `yaml:"api_ver"`
Runners []Runners `yaml:"runners"`
}
type Runners struct {
Name string `yaml:"name"`
Type []Command `yaml:"type"`
}
type Command struct {
Command string `yaml:"command"`
}
func main() {
var runners []Result
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
fmt.Printf("%+v", runners[0])
}
The error which I got cannot unmarshal !!map into []main.Result
I cannot change the yaml and it should be exactly like this
https://codebeautify.org/yaml-validator/cbabd352
This is the code
https://play.golang.org/p/zidjOA6-gc7

The yaml that you have provided contains error in token. Validate the yaml used in your code here https://codebeautify.org/yaml-validator/cbaabb32
After that Create a variable of struct type result not an array. Because the yaml that you are using is creating a struct with Runners array and api_version.
This
var runners []Result
should be
var runners Result
Since because the struct is not a slice. To fetch the list of command for a name of function used in yaml. You need to loop over the runners array to find the name of function and get the value of commands for that function.
Below is the working code:
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
api_ver: 1
runners:
- name: function1
type:
- command: spawn child process
- command: build
- command: gulp
- name: function2
type:
- command: run function 1
- name: function3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Result struct {
Version string `yaml:"api_ver"`
Runners []Runners `yaml:"runners"`
}
type Runners struct {
Name string `yaml:"name"`
Type []Command `yaml:"type"`
}
type Command struct {
Command string `yaml:"command"`
}
func main() {
var runners Result
// parse mta yaml
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
commandList := getCommandList("function1", runners.Runners)
fmt.Printf("%+v", commandList)
}
func getCommandList(name string, runners []Runners) []Command {
var commandList []Command
for _, value := range runners {
if value.Name == name {
commandList = value.Type
}
}
return commandList
}
Playground example

Related

Unable to unmarshal array field in golang

I am trying to unmasrhal the following flux HelmRelease file.
apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
annotations:
fluxcd.io/automated: 'false'
fluxcd.io/tag.ats: glob:*
name: ats
namespace: myns
spec:
chart:
git: git#github.com:reponame/project.git
path: charts/path1/path1/myapp
ref: master
releaseName: foobar
values:
allowAllEgress: true
recycleApp: true
hooks:
slackChannel: https://hooks.slack.com/services/something/somethingelse/
Here are my models
type HelmReleaseValues struct {
AllowAllEgress bool `yaml:"allowAllEgress"`
RecycleApp bool `yaml:"recycleApp"`
Hooks `yaml:"hooks"`
}
type Hooks struct {
SlackChannel string `yaml:"slackChannel"`
}
type Values struct {
HelmReleaseValues `yaml:"values"`
ReleaseName string `yaml:"releaseName"`
Chart `yaml:"chart"`
}
type Spec struct {
Values `yaml:"spec"`
}
The problem is that the fields allowAllEgress and recycleApp are getting unmarshalled.
However the Hooks field in my struct turns out to be empty.
What am I doing wrong in the struct modelling / tagging?
edit: here is my code
package main
import (
"fmt"
"io/ioutil"
"os"
"github.com/davecgh/go-spew/spew"
"gopkg.in/yaml.v3"
)
const ExitCodeCmdErr = 1
func main() {
rawYaml := parseHelmReleaseFile("myfile.yaml")
spew.Dump(rawYaml)
}
func parseHelmReleaseFile(fileName string) Spec {
var v Spec
yamlFile, err := ioutil.ReadFile(fileName)
if err != nil {
fmt.Printf("yaml file err #%v ", err)
os.Exit(ExitCodeCmdErr)
}
err = yaml.Unmarshal(yamlFile, &v)
if err != nil {
fmt.Printf("Unmarshal: %v", err)
os.Exit(ExitCodeCmdErr)
}
return v
}
I am running the program and grepping for the output (the actual helm release file is huge)
▶ go clean && gb .
~/Desktop/yamltutorial
./foobar | grep -i hooks -A 3
--
Hooks: (main.Hooks) {
SlackChannel: (string) ""
}
},
You did not have Chart struct
type Chart struct {
Git string `yaml:"git"`
Path string `yaml:"path"`
Ref string `yaml:"ref"`
}
Added that and got the following output
{Values:{HelmReleaseValues:{AllowAllEgress:true RecycleApp:true Hooks:{SlackChannel:https://hooks.slack.com/services/something/somethingelse/}} ReleaseName:foobar Chart:{Git:git#github.com:reponame/project.git Path:charts/path1/path1/myapp Ref:master}}}
Playground file with complete code.
https://play.golang.org/p/vCnjApr6gI9

Match array according to value

I'm using the following code to parse yaml and should get output as runners object and the function buildshould change the data structure and provide output according to below struct
type Exec struct {
NameVal string
Executer []string
}
This is what I have tried but I'm not sure how to replace
the hard-code values inside the function runner from the value I'm getting inside the yaml
return []Exec{
{"#mytest",
[]string{"spawn child process", "build", "gulp"}},
}
with the data from the parsed runner
This is all what I have tried any idea how it could be done?
package main
import (
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
api_ver: 1
runners:
- name: function1
data: mytest
type:
- command: spawn child process
- command: build
- command: gulp
- name: function2
data: mytest2
type:
- command: webpack
- name: function3
data: mytest3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Result struct {
Version string `yaml:"api_ver"`
Runners []Runners `yaml:"runners"`
}
type Runners struct {
Name string `yaml:"name"`
Type []Command `yaml:"type"`
}
type Command struct {
Command string `yaml:"command"`
}
func main() {
var runners Result
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
//Here Im calling to the function with the parsed structured data which need to return the list of Exec
build("function1", runners)
}
type Exec struct {
NameVal string
Executer []string
}
func build(name string, runners Result) []Exec {
for _, runner := range runners.Runners {
if name == runner.Name {
return []Exec{
// this just for example, nameVal and Command
{"# mytest",
[]string{"spawn child process", "build", "gulp"}},
}
}
}
}
Assign the name of runners object to the struct Exec field for name and append the command list to the []string type field with the commands of the function that matched the name as:
func build(name string, runners Result) []Exec {
exec := make([]Exec, len(runners.Runners))
for i, runner := range runners.Runners {
if name == runner.Name {
exec[i].NameVal = runner.Name
for _, cmd := range runner.Type {
exec[i].Executer = append(exec[i].Executer, cmd.Command)
}
fmt.Printf("%+v", exec)
return exec
}
}
return exec
}
Working code on Playground

Error while parsing yaml in golang

I’ve a yaml like following which I need to parse using go.
When I tried to run the code with the parse I got an error.
Below is the code:
var runContent= []byte(`
- runners:
- name: function1
type: func1
- command: spawn child process
- command: build
- command: gulp
- name: function1
type: func2
- command: run function 1
- name: function3
type: func3
- command: ruby build
- name: function4
type: func4
- command: go build
`)
These are the types:
type Runners struct {
runners string `yaml:"runners"`
name string `yaml:”name”`
Type: string `yaml: ”type”`
command [] Command
}
type Command struct {
command string `yaml: ”command”`
}
runners := Runners{}
err = yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
When I try to parse it I got an error invalid map , what could be missing here ?
The code you have posted contains multiple errors including the struct field Type. The yaml provided in your code is not valid. This will lead to err when unmarshalling the yaml into struct.
On unmarshalling yaml in go, It is required that:
The type of the decoded values should be compatible with the
respective values in out. If one or more values cannot be decoded due
to a type mismatches, decoding continues partially until the end of
the YAML content, and a *yaml.TypeError is returned with details for
all missed values.
Along with that:
Struct fields are only unmarshalled if they are exported (have an
upper case first letter), and are unmarshalled using the field name
lowercased as the default key.
Also there is an error in defining the yaml tags, which contains space. Custom keys may be defined via the "yaml" name in the field tag: the content preceding the first comma is used as the key.
type Runners struct {
runners string `yaml:"runners"` // fields should be exportable
name string `yaml:”name”`
Type: string `yaml: ”type”` // tags name should not have space in them.
command [] Command
}
To make the struct exportable convert the struct and fields into uppercase starting letter and remove space in yaml tag names:
type Runners struct {
Runners string `yaml:"runners"`
Name string `yaml:"name"`
Type string `yaml:"type"`
Command []Command
}
type Command struct {
Command string `yaml:"command"`
}
Modify the code as below to make it work.
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
- runners:
- name: function1
type:
- command: spawn child process
- command: build
- command: gulp
- name: function1
type:
- command: run function 1
- name: function3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Runners []struct {
Runners []struct {
Type []struct {
Command string `yaml:"command"`
} `yaml:"type"`
Name string `yaml:"name"`
} `yaml:"runners"`
}
func main() {
runners := Runners{}
// parse mta yaml
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
fmt.Println(runners)
}
Playground example
Validate your yaml online here https://codebeautify.org/yaml-validator/cb92c85b

Parse yaml return empty object

I have the following yaml which I need to parse to struct.
In the builds property I got empty value while debug, what am I missing here?
I use "gopkg.in/yaml.v2"
- name: srv
type: java
path: srv
builds:
- name: db
properties:
JBR_CONFIG_RESOURCE_CONFIG: '[META-INF/context.xml:
{"service_name" : "~{h-container}"}]'
TEST2: aaaa
The struct is
type Runs struct {
Name string
Type string
Path string `yaml:"path,omitempty"`
Builds []Builds `yaml:”builds,omitempty"`
}
type Builds struct {
Name string `yaml:"name,omitempty"`
Properties Properties `yaml:"properties,omitempty"`
}
type Properties map[string]string
Properly formated yaml is the first thing that you should consider.
If u wanna have one Runs you should have your yaml formated something like that
name: srv
builds:
-
name: db
properties:
JBR_CONFIG_RESOURCE_CONFIG: "[META-INF/context.xml:
{\"service_name\" : \"~{h-container}\"}]"
TEST2: aaaa
path: srv
type: java
But then i u wanna have more of this object you need to group them in one parameter. It can look like this
runs:
-
name: srv
builds:
-
name: db
properties:
JBR_CONFIG_RESOURCE_CONFIG: "[META-INF/context.xml:
{\"service_name\" : \"~{h-container}\"}]"
TEST2: aaaa
path: srv
type: java
-
name: srv2
builds:
-
name: db2
properties:
JBR_CONFIG_RESOURCE_CONFIG: "[META-INF/context.xml:
{\"service_name\" : \"~{h-container}\"}]"
TEST2: aaaa2
path: srv2
type: java2
And then in your code could look like this
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"io/ioutil"
"log"
"os"
)
type Document struct {
Runs []Runs `yaml:"runs,omitempty"`
}
type Runs struct {
Name string `yaml:"name,omitempty"`
Type string `yaml:"type,omitempty"`
Path string `yaml:"path,omitempty"`
Builds []Builds `yaml:"builds,omitempty"`
}
type Builds struct {
Name string `yaml:"name,omitempty"`
Properties map[string]string `yaml:"properties,omitempty"`
}
func main() {
var document Document
reader, err := os.Open("demo.yml")
if err != nil {
log.Fatal(err)
}
buf, _ := ioutil.ReadAll(reader)
yaml.Unmarshal(buf, &document)
if err := yaml.Unmarshal(buf, &document); err != nil {
fmt.Print(err)
os.Exit(1)
}
fmt.Println(document)
}
Make sure your yaml file is formatted correctly. Check with this tool.
The following piece of code worked fine.
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"log"
)
type Runs struct {
Name string
Type string
Path string `yaml:"path,omitempty"`
Builds []Builds `yaml:”builds,omitempty"`
}
type Builds struct {
Name string `yaml:"name,omitempty"`
Properties Properties `yaml:"properties,omitempty"`
}
type Properties map[string]string
func main() {
data := `builds:
-
name: db
properties:
JBR_CONFIG_RESOURCE_CONFIG: "[META-INF/context.xml: {\"service_name\" : \"~{h-container}\"}]"
TEST2: aaaa
name: srv
path: srv
type: java
`
runs := Runs{}
err := yaml.Unmarshal([]byte(data), &runs)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t:\n%v\n\n", runs)
d, err := yaml.Marshal(&runs)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t dump:\n%s\n\n", string(d))
}
I hope this helps!

How to build repetitive struct

I have a program which parses yamls file to an objects (structs).
I use the following repo to do it
https://github.com/go-yaml/yaml
for example in the file I have:
dependency :
- name: ui
type: runner
cwd: /ui
install:
- name: api
group: test
And I use the following struct for it
type Dependency struct {
Name string
Type string
CWD string
Install []Install
//here I have the issue
Requires ?
}
type Install struct {
Name string
Group string
}
Now I have two few issue with a bit complex struct.
This is the entry which could be inside the Dependency struct and this is how it look in the yaml file
requires:
- name: db
- type: mongo
but it also can be
requires:
- name: db
- name: rst
- name: test
- name: test2
Since it have multiple name properties how should I got build this struct
In addition I've field in the yaml
_type-version: "1.0.0"
when I put it inside struct like following I got error since I use -
type TypeVer struct{
_Type-version string
}
How to overcome this?
The yaml package actually allows you to remap the the name for the properties, you can use this to handle your _type-version Property.
And your initial question: Just define Requires the same as Install:
package main
import (
"fmt"
"log"
"github.com/go-yaml/yaml"
)
type File struct {
TypeVersion string `yaml:"_type-version"`
Dependency []Dependency
}
type Dependency struct {
Name string
Type string
CWD string
Install []Install
Requires []Requires
}
type Install struct {
Name string
Group string
}
type Requires struct {
Name string
Type string
}
var data = `
_type-version: "1.0.0"
dependency:
- name: ui
type: runner
cwd: /ui
install:
- name: api
group: test
requires:
- name: db
- type: mongo
- name: rst
- name: test
- name: test2
`
func main() {
f := File{}
err := yaml.Unmarshal([]byte(data), &f)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t:\n%v\n\n", f)
d, err := yaml.Marshal(&f)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t dump:\n%s\n\n", string(d))
}

Resources