I want to unmarshal my ~/.kube/config file into a go struct.
I am using the following approach
func ListContexts(pathToKubeConfig string) ([]string, error) {
type Contexts struct {
Ctx []string `yaml:"contexts"`
//ApiVersion string `yaml:"apiVersion"`
}
var ctx []string
var c Contexts
file, err := ioutil.ReadFile(pathToKubeConfig)
if err != nil {
return nil, err
}
err = yaml.Unmarshal(file, &c)
fmt.Printf("%#v\n", c.Ctx)
return ctx, nil
}
}
As is widely known, a kubeconfig file has the following struct:
apiVersion: v1
. . .
contexts:
- context:
cluster: cluster1
user: user1
name: context1
- context:
cluster: cluster2
user: user2
name: context2
My approach is printing:
[]string(nil)
Since context is a yaml array, why my mapping into a string array not working?
When I uncomment the ApiVersion field of my struct and try to print it, it works.
context is an array, but not a string array. Either use []map[string]interface{} for context, or define the context as a struct, and use its array:
type context struct {
Cluster string `yaml:"cluster"`
...
}
type contexts struct {
Contexts []context `yaml:"contexts"`
}
Related
I really searched a while here, but didn't found an adequate answer:
I am trying to unmarshall yaml dict keys onto a property of a struct rather than the key of a map.
Given this yaml
commands:
php:
service: php
bin: /bin/php
node:
service: node
bin: /bin/node
I am able to unmarshall this into a struct like this:
type Config struct {
Commands map[string]struct {
Service string
Bin string
}
}
But how am I able to unmarshall it into a struct like this:
type Config struct {
Commands []struct {
Name string // <-- this should be key from the yaml (i.e. php or node)
Service string
Bin string
}
}
Thx in advance for the help
You can write a custom unmarshaler, like this (on Go playground):
package main
import (
"fmt"
"gopkg.in/yaml.v3"
)
var input []byte = []byte(`
commands:
php:
service: php
bin: /bin/php
node:
service: node
bin: /bin/node
`)
type Command struct {
Service string
Bin string
}
type NamedCommand struct {
Command
Name string
}
type NamedCommands []NamedCommand
type Config struct {
Commands NamedCommands
}
func (p *NamedCommands) UnmarshalYAML(value *yaml.Node) error {
if value.Kind != yaml.MappingNode {
return fmt.Errorf("`commands` must contain YAML mapping, has %v", value.Kind)
}
*p = make([]NamedCommand, len(value.Content)/2)
for i := 0; i < len(value.Content); i += 2 {
var res = &(*p)[i/2]
if err := value.Content[i].Decode(&res.Name); err != nil {
return err
}
if err := value.Content[i+1].Decode(&res.Command); err != nil {
return err
}
}
return nil
}
func main() {
var f Config
var err error
if err = yaml.Unmarshal(input, &f); err != nil {
panic(err)
}
for _, cmd := range f.Commands {
fmt.Printf("%+v\n", cmd)
}
}
I have split the command data into Command and NamedCommand to make the code simpler, since you can just call Decode giving the embedded Command struct for the values. If everything was in the same struct, you'd need to manually map keys to struct fields.
I'm trying to unmarshal the following YAML data into Go structures.
The data is the in the following format:
fetchers:
- type: "aws"
config:
omega: "lul"
- type: "kubernetes"
config:
foo: "bar"
Based of the type field, I want to determine wether to unmarshal the config field into awsConfig or kubernetesConfig struct.
My current code looks like this (using "gopkg.in/yaml.v2"):
type kubernetesConfig struct {
foo string `yaml:"foo"`
}
type awsConfig struct {
omega string `yaml:"omega"`
}
var c struct {
Fetchers []struct {
Type string `yaml:"type"`
Config interface{} `yaml:"config"`
} `yaml:"fetchers"`
}
err := yaml.Unmarshal(data, &c)
if err != nil {
log.Fatal(err)
}
for _, val := range c.Fetchers {
switch val.Type {
case "kubernetes":
conf := val.Config.(kubernetesConfig)
fmt.Println(conf.foo)
case "aws":
conf := val.Config.(awsConfig)
fmt.Println(conf.omega)
default:
log.Fatalf("No matching type, was type %v", val.Type)
}
}
Code in playground: https://go.dev/play/p/klxOoHMCtnG
Currently it gets unmarshalled as map[interface {}]interface {}, which can't be converted to one of the structs above.
Error:
panic: interface conversion: interface {} is map[interface {}]interface {}, not main.awsConfig \
Do I have to implemented the Unmarshaler Interface of the YAML package with a custom UnmarshalYAML function to get this done?
Found the solution by implementing Unmarshaler Interface:
type Fetcher struct {
Type string `yaml:"type"`
Config interface{} `yaml:"config"`
}
// Interface compliance
var _ yaml.Unmarshaler = &Fetcher{}
func (f *Fetcher) UnmarshalYAML(unmarshal func(interface{}) error) error {
var t struct {
Type string `yaml:"type"`
}
err := unmarshal(&t)
if err != nil {
return err
}
f.Type = t.Type
switch t.Type {
case "kubernetes":
var c struct {
Config kubernetesConfig `yaml:"config"`
}
err := unmarshal(&c)
if err != nil {
return err
}
f.Config = c.Config
case "aws":
var c struct {
Config awsConfig `yaml:"config"`
}
err := unmarshal(&c)
if err != nil {
return err
}
f.Config = c.Config
}
return nil
}
This type of task - where you want to delay the unmarshaling - is very similar to how json.RawMessage works with examples like this.
The yaml package does not have a similar mechanism for RawMessage - but this technique can easily be replicated as outlined here:
type RawMessage struct {
unmarshal func(interface{}) error
}
func (msg *RawMessage) UnmarshalYAML(unmarshal func(interface{}) error) error {
msg.unmarshal = unmarshal
return nil
}
// call this method later - when we know what concrete type to use
func (msg *RawMessage) Unmarshal(v interace{}) error {
return msg.unmarshal(v)
}
So to leverage this in your case:
var fs struct {
Configs []struct {
Type string `yaml:"type"`
Config RawMessage `yaml:"config"` // delay unmarshaling
} `yaml:"fetchers"`
}
err = yaml.Unmarshal([]byte(data), &fs)
if err != nil {
return
}
and based on the config "Type" (aws or kubernetes), you can finally unmarshal the RawMessage into the correct concrete type:
aws := awsConfig{} // concrete type
err = c.Config.Unmarshal(&aws)
or:
k8s := kubernetesConfig{} // concrete type
err = c.Config.Unmarshal(&k8s)
Working example here: https://go.dev/play/p/wsykOXNWk3H
The YAML file:
namespaces:
- namespace: default
aliasname: k8s
components:
- component: comp1
replicas: 1
port: 8080
- component: comp2
replicas: 1
port: 9999
- namespace: ns2
components:
- component: comp1
replicas: 1
From the YAML file above, I want to create structs like the following:
type Namespaces struct {
NamespaceName string `yaml:"namespace"`
Aliasname string `yaml:"aliasname,omitempty"`
ListOfComponents []Components `yaml:"components"`
ComponentMap map[string]Components
}
type Components struct {
ComponentName string `yaml:"component"`
NumReplicas int `yaml:"replicas"`
Port int `yaml:"port"`
}
type Config struct {
ListOfNamespaces []Namespaces `yaml:"namespaces"`
NamespaceMap map[string]Namespaces
}
The fields Namespacemap and Componentmap should be able to be retrieved when accessing the config and namespace object respectively. I created a method to convert the list of the namespaces and components into maps, but when I call config.Namespacemap or Namespace.ComponentMap, it returns an empty map.
Basically I would like to know: How do we add extra fields to type structs? I would like to access new variables like a map from the config struct.
Update:
Thanks blami for guiding me, but when I try to write the same for Components, it doesn't give me the whole namespacemap with the componentmap included:
type Components struct {
ComponentName string `yaml:"component"`
NumReplicas int `yaml:"replicas"`
Port int `yaml:"port"`
}
type Namespaces struct {
NamespaceName string `yaml:"namespace"`
Aliasname string `yaml:"aliasname"`
ComponentMap map[string]Components `yaml:"components"`
}
func (n *Namespaces) UnmarshalYAML(unmarshal func(interface{}) error) error {
type origNamespace struct {
ListOfComponents []Components `yaml:"components"`
}
var on origNamespace
err1 := unmarshal(&on)
if err1 != nil {
return err1
}
n.ComponentMap = make(map[string]Components)
for _, i := range on.ListOfComponents {
n.ComponentMap[i.ComponentName] = i
}
return nil
}
When I run the config.NamespaceMap it gives the following
map[:{NamespaceName: K8sNamespace: ComponentMap:map[comp1:{ComponentName:comp1 NumShards:0 NumReplicas:1 Port:0 EpochLength:0}]}]
If you want to do such transformation you will need to write a customized UnmarshalYAML() receiver on types Config and Namespace. Here is basic working example of doing so (only for "namespaces"):
type Config struct {
Namespaces map[string]Namespace `yaml:"namespaces"`
}
// Replace go-yaml built-in unmarshaller with custom that will transform list to map.
// Note this code is meant only as demonstration!
func (c *Config) UnmarshalYAML(unmarshal func(interface{}) error) error {
type origConfig struct {
Namespaces []Namespace `yaml:"namespaces"`
}
var o origConfig
err := unmarshal(&o)
if err != nil {
return err
}
// Assign namespaces in list to map; namespaces with same name will be overwritten
c.Namespaces = make(map[string]Namespace)
for _, n := range o.Namespaces {
c.Namespaces[n.Namespace] = n
}
return nil
}
// You can use "Config" type as usual
func main() {
var config Config
err := yaml.Unmarshal(<your_yaml>, &config)
if err != nil {
log.Panic(err)
}
fmt.Println(config.Namespaces)
// map[default:{default k8s} ns2:{ns2 }]
fmt.Printf("%v\n", config.Namespaces["default"])
// {default k8s}
}
As noted in code example this will cause some problems (e.g. what to do if namespace names are same?).
Playground link: https://go.dev/play/p/IKg8kmRnknq
I the following issue: i receive information in yaml form, it can come in 2 configurations-
asingle character:
Character:
Name: Stackoverflow
Type: Awesome
Or in a slice of multiple characters with an optional field of slice of friends:
Character:
- Name: Stackoverflow
Type: Awesome
Friends: [Ben, John]
- Name: Facebook
Type: Ok
Because I don't know which configuration will arrive I tried to unmarshal the yaml to interface{} and then cast it to one of the structs but it didn't worked out:
type CharacterConfig struct{
Name string `yaml:"Name"`
Type string `yaml:"Type"`
Friends []string `yaml:"Friends,omitempty"`
var d interface{}
err := yaml.Unmarshal([]byte(yamlData), &d)
if err != nil {
panic(err)
}
res, ok := d.(CharacterConfig)
if ok {
fmt.Println("CharacterConfig are ok")
}
res, ok := d.([]CharacterConfig)
if ok {
fmt.Println("[]CharacterConfig are ok")
}
But I don't receive any of the prints... when I debug is I can see the unmarshal into interface worked, but non of the castings.
I know I can just unmarshal straight into the structs, but I don't understand wht what i did didn't work.
Unmarshaling into interface{} will not magically guess you want the result in your custom CharacterConfig struct. Unmarshaling YAML into interface{} will use map[interface{}]interface{} to represent objects, and []interface{} to represent lists / arrays, recursively.
So the type assertions above yield ok = false because the values stored in them are maps or slices of interfaces{}. You may type assert those types, and the assertion will succeed, but you won't be any closer in getting those values as your structs.
Also note that your input YAML needs another wrapper layer:
type Character struct {
CC CharacterConfig `yaml:"Character"`
}
type CharacterList struct {
CCs []CharacterConfig `yaml:"Character"`
}
So an easy way would be to first try to unmarshal into a value of type Character, and if that succeeds, you're done. If not, try to unmarshal again, this time into CharacterList.
Here's an example code that does that:
func unmarshal(data []byte) {
var c Character
err := yaml.Unmarshal(data, &c)
if err == nil {
fmt.Printf("It's Character: %+v\n", c)
return
}
var cl CharacterList
err = yaml.Unmarshal(data, &cl)
if err != nil {
panic(err)
}
fmt.Printf("It's CharacterList: %+v\n", cl)
}
Testing it like this:
func main() {
unmarshal([]byte(src))
unmarshal([]byte(src2))
}
const src = `Character:
Name: Stackoverflow
Type: Awesome`
const src2 = `Character:
- Name: Stackoverflow
Type: Awesome
Friends: [Ben, John]
- Name: Facebook
Type: Ok`
Output will be (try it on the Go Playground):
It's Character: {CC:{Name:Stackoverflow Type:Awesome Friends:[]}}
It's CharacterList: {CCs:[{Name:Stackoverflow Type:Awesome Friends:[Ben John]} {Name:Facebook Type:Ok Friends:[]}]}
Similar to what you did but without interface:
type CharacterConfig struct{
Name string `yaml:"Name"`
Type string `yaml:"Type"`
Friends []string `yaml:"Friends,omitempty"`
}
var a CharacterConfig
var b []CharacterConfig
err := yaml.Unmarshal([]byte(yamlData), &a)
if err == nil {
// a is valid format
}
err = yaml.Unmarshal([]byte(yamlData), &b)
if err == nil {
// b is valid format
}
I have a struct defined for my Yaml file like so:
type Service struct {
ServiceName string `yaml:"service_name"`
PipelineType PipelineType `yaml:"pipeline_type"`
}
In the file, this struct comes in as a string:
service_name: service
pipeline_type: app
My type is defined like this:
// PipelineType Pipeline Types
type PipelineType struct {
Value string
}
var (
AppPipeline = PipelineType{"app"}
...
)
Because of the type declaration, I'm getting the following error (expected):
line 4: cannot unmarshal !!str `app` into main.PipelineType
Is there a way to tell GoYAML, or create some form of consturctor to convert the value using string(PipelineType) or something similar?
Alternatively, is there a "GoYAML" friendly way to do this?
try this:
type Service struct {
ServiceName string `yaml:"service_name"`
PipelineType PipelineType `yaml:"pipeline_type"`
}
func (s *Service) myYml() *Service {
yamlFile, err := ioutil.ReadFile("service.yaml")
if err != nil {
log.Println(err.Error())
}
err = yaml.Unmarshal(yamlFile, s)
if err != nil {
log.Println(err.Error())
}
return s
}
get it:
var s Service
s.myYml()