I'm using the following code to parse yaml and should get output as runners object and the function buildshould change the data structure and provide output according to below struct
type Exec struct {
NameVal string
Executer []string
}
This is what I have tried but I'm not sure how to replace
the hard-code values inside the function runner from the value I'm getting inside the yaml
return []Exec{
{"#mytest",
[]string{"spawn child process", "build", "gulp"}},
}
with the data from the parsed runner
This is all what I have tried any idea how it could be done?
package main
import (
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
api_ver: 1
runners:
- name: function1
data: mytest
type:
- command: spawn child process
- command: build
- command: gulp
- name: function2
data: mytest2
type:
- command: webpack
- name: function3
data: mytest3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Result struct {
Version string `yaml:"api_ver"`
Runners []Runners `yaml:"runners"`
}
type Runners struct {
Name string `yaml:"name"`
Type []Command `yaml:"type"`
}
type Command struct {
Command string `yaml:"command"`
}
func main() {
var runners Result
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
//Here Im calling to the function with the parsed structured data which need to return the list of Exec
build("function1", runners)
}
type Exec struct {
NameVal string
Executer []string
}
func build(name string, runners Result) []Exec {
for _, runner := range runners.Runners {
if name == runner.Name {
return []Exec{
// this just for example, nameVal and Command
{"# mytest",
[]string{"spawn child process", "build", "gulp"}},
}
}
}
}
Assign the name of runners object to the struct Exec field for name and append the command list to the []string type field with the commands of the function that matched the name as:
func build(name string, runners Result) []Exec {
exec := make([]Exec, len(runners.Runners))
for i, runner := range runners.Runners {
if name == runner.Name {
exec[i].NameVal = runner.Name
for _, cmd := range runner.Type {
exec[i].Executer = append(exec[i].Executer, cmd.Command)
}
fmt.Printf("%+v", exec)
return exec
}
}
return exec
}
Working code on Playground
Related
I really searched a while here, but didn't found an adequate answer:
I am trying to unmarshall yaml dict keys onto a property of a struct rather than the key of a map.
Given this yaml
commands:
php:
service: php
bin: /bin/php
node:
service: node
bin: /bin/node
I am able to unmarshall this into a struct like this:
type Config struct {
Commands map[string]struct {
Service string
Bin string
}
}
But how am I able to unmarshall it into a struct like this:
type Config struct {
Commands []struct {
Name string // <-- this should be key from the yaml (i.e. php or node)
Service string
Bin string
}
}
Thx in advance for the help
You can write a custom unmarshaler, like this (on Go playground):
package main
import (
"fmt"
"gopkg.in/yaml.v3"
)
var input []byte = []byte(`
commands:
php:
service: php
bin: /bin/php
node:
service: node
bin: /bin/node
`)
type Command struct {
Service string
Bin string
}
type NamedCommand struct {
Command
Name string
}
type NamedCommands []NamedCommand
type Config struct {
Commands NamedCommands
}
func (p *NamedCommands) UnmarshalYAML(value *yaml.Node) error {
if value.Kind != yaml.MappingNode {
return fmt.Errorf("`commands` must contain YAML mapping, has %v", value.Kind)
}
*p = make([]NamedCommand, len(value.Content)/2)
for i := 0; i < len(value.Content); i += 2 {
var res = &(*p)[i/2]
if err := value.Content[i].Decode(&res.Name); err != nil {
return err
}
if err := value.Content[i+1].Decode(&res.Command); err != nil {
return err
}
}
return nil
}
func main() {
var f Config
var err error
if err = yaml.Unmarshal(input, &f); err != nil {
panic(err)
}
for _, cmd := range f.Commands {
fmt.Printf("%+v\n", cmd)
}
}
I have split the command data into Command and NamedCommand to make the code simpler, since you can just call Decode giving the embedded Command struct for the values. If everything was in the same struct, you'd need to manually map keys to struct fields.
I am trying to unmasrhal the following flux HelmRelease file.
apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
annotations:
fluxcd.io/automated: 'false'
fluxcd.io/tag.ats: glob:*
name: ats
namespace: myns
spec:
chart:
git: git#github.com:reponame/project.git
path: charts/path1/path1/myapp
ref: master
releaseName: foobar
values:
allowAllEgress: true
recycleApp: true
hooks:
slackChannel: https://hooks.slack.com/services/something/somethingelse/
Here are my models
type HelmReleaseValues struct {
AllowAllEgress bool `yaml:"allowAllEgress"`
RecycleApp bool `yaml:"recycleApp"`
Hooks `yaml:"hooks"`
}
type Hooks struct {
SlackChannel string `yaml:"slackChannel"`
}
type Values struct {
HelmReleaseValues `yaml:"values"`
ReleaseName string `yaml:"releaseName"`
Chart `yaml:"chart"`
}
type Spec struct {
Values `yaml:"spec"`
}
The problem is that the fields allowAllEgress and recycleApp are getting unmarshalled.
However the Hooks field in my struct turns out to be empty.
What am I doing wrong in the struct modelling / tagging?
edit: here is my code
package main
import (
"fmt"
"io/ioutil"
"os"
"github.com/davecgh/go-spew/spew"
"gopkg.in/yaml.v3"
)
const ExitCodeCmdErr = 1
func main() {
rawYaml := parseHelmReleaseFile("myfile.yaml")
spew.Dump(rawYaml)
}
func parseHelmReleaseFile(fileName string) Spec {
var v Spec
yamlFile, err := ioutil.ReadFile(fileName)
if err != nil {
fmt.Printf("yaml file err #%v ", err)
os.Exit(ExitCodeCmdErr)
}
err = yaml.Unmarshal(yamlFile, &v)
if err != nil {
fmt.Printf("Unmarshal: %v", err)
os.Exit(ExitCodeCmdErr)
}
return v
}
I am running the program and grepping for the output (the actual helm release file is huge)
▶ go clean && gb .
~/Desktop/yamltutorial
./foobar | grep -i hooks -A 3
--
Hooks: (main.Hooks) {
SlackChannel: (string) ""
}
},
You did not have Chart struct
type Chart struct {
Git string `yaml:"git"`
Path string `yaml:"path"`
Ref string `yaml:"ref"`
}
Added that and got the following output
{Values:{HelmReleaseValues:{AllowAllEgress:true RecycleApp:true Hooks:{SlackChannel:https://hooks.slack.com/services/something/somethingelse/}} ReleaseName:foobar Chart:{Git:git#github.com:reponame/project.git Path:charts/path1/path1/myapp Ref:master}}}
Playground file with complete code.
https://play.golang.org/p/vCnjApr6gI9
I've the following program in which I need to parse yaml
with the following structure
https://codebeautify.org/yaml-validator/cbabd352
this is valid yaml and I use byte to make it more simple
maybe the indentation was changed during the copy paste to the question but you can see in the link that the yaml is valid
The yaml have an api_version
and runners, for each runner (key which is name) I've a list of commands
and I need to print those commands for function1 and function4 ,what am I doing wrong here ?
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
api_ver: 1
runners:
- name: function1
type:
- command: spawn child process
- command: build
- command: gulp
- name: function2
type:
- command: run function 1
- name: function3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Result struct {
Version string `yaml:"api_ver"`
Runners []Runners `yaml:"runners"`
}
type Runners struct {
Name string `yaml:"name"`
Type []Command `yaml:"type"`
}
type Command struct {
Command string `yaml:"command"`
}
func main() {
var runners []Result
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
fmt.Printf("%+v", runners[0])
}
The error which I got cannot unmarshal !!map into []main.Result
I cannot change the yaml and it should be exactly like this
https://codebeautify.org/yaml-validator/cbabd352
This is the code
https://play.golang.org/p/zidjOA6-gc7
The yaml that you have provided contains error in token. Validate the yaml used in your code here https://codebeautify.org/yaml-validator/cbaabb32
After that Create a variable of struct type result not an array. Because the yaml that you are using is creating a struct with Runners array and api_version.
This
var runners []Result
should be
var runners Result
Since because the struct is not a slice. To fetch the list of command for a name of function used in yaml. You need to loop over the runners array to find the name of function and get the value of commands for that function.
Below is the working code:
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
api_ver: 1
runners:
- name: function1
type:
- command: spawn child process
- command: build
- command: gulp
- name: function2
type:
- command: run function 1
- name: function3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Result struct {
Version string `yaml:"api_ver"`
Runners []Runners `yaml:"runners"`
}
type Runners struct {
Name string `yaml:"name"`
Type []Command `yaml:"type"`
}
type Command struct {
Command string `yaml:"command"`
}
func main() {
var runners Result
// parse mta yaml
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
commandList := getCommandList("function1", runners.Runners)
fmt.Printf("%+v", commandList)
}
func getCommandList(name string, runners []Runners) []Command {
var commandList []Command
for _, value := range runners {
if value.Name == name {
commandList = value.Type
}
}
return commandList
}
Playground example
I’ve a yaml like following which I need to parse using go.
When I tried to run the code with the parse I got an error.
Below is the code:
var runContent= []byte(`
- runners:
- name: function1
type: func1
- command: spawn child process
- command: build
- command: gulp
- name: function1
type: func2
- command: run function 1
- name: function3
type: func3
- command: ruby build
- name: function4
type: func4
- command: go build
`)
These are the types:
type Runners struct {
runners string `yaml:"runners"`
name string `yaml:”name”`
Type: string `yaml: ”type”`
command [] Command
}
type Command struct {
command string `yaml: ”command”`
}
runners := Runners{}
err = yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
When I try to parse it I got an error invalid map , what could be missing here ?
The code you have posted contains multiple errors including the struct field Type. The yaml provided in your code is not valid. This will lead to err when unmarshalling the yaml into struct.
On unmarshalling yaml in go, It is required that:
The type of the decoded values should be compatible with the
respective values in out. If one or more values cannot be decoded due
to a type mismatches, decoding continues partially until the end of
the YAML content, and a *yaml.TypeError is returned with details for
all missed values.
Along with that:
Struct fields are only unmarshalled if they are exported (have an
upper case first letter), and are unmarshalled using the field name
lowercased as the default key.
Also there is an error in defining the yaml tags, which contains space. Custom keys may be defined via the "yaml" name in the field tag: the content preceding the first comma is used as the key.
type Runners struct {
runners string `yaml:"runners"` // fields should be exportable
name string `yaml:”name”`
Type: string `yaml: ”type”` // tags name should not have space in them.
command [] Command
}
To make the struct exportable convert the struct and fields into uppercase starting letter and remove space in yaml tag names:
type Runners struct {
Runners string `yaml:"runners"`
Name string `yaml:"name"`
Type string `yaml:"type"`
Command []Command
}
type Command struct {
Command string `yaml:"command"`
}
Modify the code as below to make it work.
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
- runners:
- name: function1
type:
- command: spawn child process
- command: build
- command: gulp
- name: function1
type:
- command: run function 1
- name: function3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Runners []struct {
Runners []struct {
Type []struct {
Command string `yaml:"command"`
} `yaml:"type"`
Name string `yaml:"name"`
} `yaml:"runners"`
}
func main() {
runners := Runners{}
// parse mta yaml
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
fmt.Println(runners)
}
Playground example
Validate your yaml online here https://codebeautify.org/yaml-validator/cb92c85b
I use the VSCode generation for test file of my project,
currenlty it generate the folloing structure
tests := []struct {
name string
args args
wantOut ZTR
}{
name: "test123",
args: args{
ztrFile: "./testdata/ztrfile.yaml",
},
wantOut: “ZTR.Modules",
}
The test should cover parse of yaml and testing the properties
Here it calles to parse file
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if gotOut := parseFile(tt.args.ztrFile); !reflect.DeepEqual(gotOut, tt.wantOut) {
t.Errorf("parseFile() = %v, want %v", gotOut, tt.wantOut)
}
})
This is the struct
type Modules struct {
Name string
Type string
cwd string `yaml:”cwd,omitempty"`
}
Not sure what I need to put here to make it work, I try to play with the types but Im getting errors
{
name: "test123",
args: args{
mtaFile: "./testdata/ztrfile.yaml",
},
wantOut: “ZTR.Modules",
}
The errors I got is
message: 'cannot use "test123" (type string) as type struct { name string; args args; wantOut ZTR } in array or slice literal'
at: '41,3'
source: ''
code: 'undefined'
Your tests declaration is incorrect. You need to provide a slice of structs, but you're providing just keys/values:
tests := []struct {
name string
args args
wantOut ZTR
}{
name: "test123",
args: args{
mtaFile: "./testdata/ztrfile.yaml",
},
wantOut: “ZTR.Modules",
}
should be:
tests := []struct {
name string
args args
wantOut ZTR
}{
{
name: "test123",
args: args{
mtaFile: "./testdata/ztrfile.yaml",
},
wantOut: “ZTR.Modules",
},
}