How to iterate over yml sections with structs? - go

I use viper. I'm trying to get info from structs with yml-config.
type Config struct {
Account User `mapstructure:"user"`
}
type User struct {
Name string `mapstructure:"name"`
Contacts []Contact `mapstructure:"contact"`
}
type Contact struct {
Type string `mapstructure:"type"`
Value string `mapstructure:"value"`
}
func Init() *Config {
conf := new(Config)
viper.SetConfigType("yaml")
viper.ReadInConfig()
...
viper.Unmarshal(conf)
return conf
}
...
config := Init()
...
for _, contact := range config.Account.Contacts {
type := contact.type
vlaue := contact.value
}
And config.yml
user:
name: John
contacts:
email:
type: email
value: test#test.com
skype:
type: skype
value: skypeacc
Can I get structure items like this? I could not get contact data like that. Is it possible?

If I'm getting right what you want to achieve, and based on the for loop you provided;
What you actually need is a YAML sequence, which is an array. So your final YAML file should look like;
user:
name: John
contacts:
- type: email
value: test#test.com
- type: skype
value: skypeacc
- type: email
value: joe#example.com
Also, you have a typo in your tag on Contacts slice. It should match the YAML key;
type User struct {
Name string `mapstructure:"name"`
Contacts []Contact `mapstructure:"contacts"`
}
If you wish to keep the original YAML file structure you have to provide a tag (and the corresponding struct field) for each YAML key, so looping over it wouldn't be possible out of the box, because email and skype are parsed as struct fields. An example of the struct for the original YAML file would be;
type Config struct {
Account User `mapstructure:"user"`
}
type User struct {
Name string `mapstructure:"name"`
Contacts Contacts `mapstructure:"contacts"`
}
type Contacts struct {
Email Contact `mapstructure:"email"`
Skype Contact `mapstructure:"skype"`
}
type Contact struct {
Type string `mapstructure:"type"`
Value string `mapstructure:"value"`
}

I think the only significant problem is that in your data structures you've declared Contacts as a list, but in your YAML file it's a dictionary. If you structure your input file like this:
user:
name: John
contacts:
- type: email
value: test#test.com
- type: skype
value: skypeacc
Then you can read it like this:
package main
import (
"fmt"
"github.com/spf13/viper"
)
type Config struct {
User User
}
type User struct {
Name string
Contacts []Contact
}
type Contact struct {
Type string
Value string
}
func main() {
var cfg Config
viper.SetConfigName("config")
viper.AddConfigPath(".")
err := viper.ReadInConfig()
if err != nil {
panic(err)
}
viper.Unmarshal(&cfg)
fmt.Println("user: ", cfg.User.Name)
for _, contact := range cfg.User.Contacts {
fmt.Println(" ", contact.Type, ": ", contact.Value)
}
}
The above code is runnable as-is; you should just be able to drop it
into a file and build it. When I run the above example, I get as
output:
user: John
email : test#test.com
skype : skypeacc

Related

how to escape ! while unmarshalling the yamlfile in golang

I have yaml file which contain special character ! in the key-value pair as shown below:
name: Amit Kumar
age: !kk
address:
street: !123 Some Random Street
city: Chennai
state: TN
zip: 763098
phoneNumbers:
- type: home
number: 0123456789
- type: work
number: 0987654321
While unmarshall this yaml file, street: !123 Some Random Street or age: !dd, any word that begin with ! are removed.
For example:
person.age or person.address will result in
age: kk
address: map[city:Chennai state:TN street:Some Random Street zip:763098]
expected:
age: !kk
address: map[city:Chennai state:TN street:!123 Some Random Street zip:763098]
Any suggestion on retaining the special character while unmarshalling the yaml file?
Go file:
package main
import (
"fmt"
"gopkg.in/yaml.v3"
"io/ioutil"
)
type Address struct {
Street string `yaml:"street"`
City string `yaml:"city"`
State string `yaml:"state"`
Zip string `yaml:"zip"`
}
type PhoneNumber struct {
Type string `yaml:"type"`
Number string `yaml:"number"`
}
type Person struct {
Name string `yaml:"name"`
Age string `yaml:"age"`
Address interface{} `yaml:"address"`
PhoneNumber []PhoneNumber `yaml:"phoneNumbers"`
}
func main() {
// Read the file
data, err := ioutil.ReadFile("testyamltemplate.yaml")
if err != nil {
fmt.Println(err)
return
}
// Create a struct to hold the YAML data
var person Person
// Unmarshal the YAML data into the struct
err = yaml.Unmarshal(data, &person)
if err != nil {
fmt.Println(err)
return
}
// Print the data
fmt.Println(person.Age)
}
Unquoted leading exclamation mark is treated by YAML specification as a start of a tag.
For prevent such interpretation wrap the whole value with double or single quotes:
age: "!kk"
street: "!123 Some Random Street"

Why is the unmarshaled YAML data empty?

I have a problem unmarshaling a simple slice of YAML data:
package main
import (
"fmt"
"gopkg.in/yaml.v2"
)
type myDataStruct struct {
HTTP []struct {
Name string
Target string
}
}
func main() {
yamlData := `
HTTP:
- name: one
target: http://wazaa
- name: two
target: http://wazii
`
var myData myDataStruct
err := yaml.Unmarshal([]byte(yamlData), &myData)
if err != nil {
fmt.Printf("error: %v", err)
return
}
fmt.Print(myData)
}
Playground: https://play.golang.org/p/Srb2DJVVZqN
The result is {[]} and my limited knowledge of Go does not help to understand why?
If you don't specify the mapping between labels used in the YAML source and Go struct fields, by default they will only be matched if only changing the first letter to lower matches.
E.g. the struct field Name will match name, but not NAME.
Specify the mapping for the HTTP field:
type myDataStruct struct {
HTTP []struct {
Name string
Target string
} `yaml:"HTTP"`
}
With this change it works and outputs (try it on the Go Playground):
{[{one http://wazaa} {two http://wazii}]}
It's good practice to provide mappings for all fields, so it'll continue to work if you rename the fields:
type myDataStruct struct {
HTTP []struct {
Name string `yaml:"name"`
Target string `yaml:"target"`
} `yaml:"HTTP"`
}
Although in your case the default matching works for the Name and Target fields without providing the mappings.

Unmarshal DynamoDBAttributeValues into a struct with different attributes

I read some data from dynamodb. This is what I get
{
Item: {
rating: {
N: "6"
},
pk: {
S: "artist-1"
},
gender: {
S: "woman"
},
sk: {
S: "Alexandra A"
}
}
}
Now I have a struct which looks like this:
type Artist struct {
ArtistID string `json:"id"`
Gender string `json:"gender"`
Name string `json:"name"`
Rating float64 `json:"rating"`
}
Now I do
artist := model.Artist{}
err = dynamodbattribute.UnmarshalMap(result.Item, &artist)
Now I can access e.g. gender with artist.gender. So this is fine, but I can't do this for ArtistId because it's called pk in my dynamodb and I use 'id' for my struct. What is a clean way to solve this? I don't want to replace my 'id' with 'pk' in the struct.
UnmarshalMap() didn't support any given tag Unmarshal. If you don't want to change 'id' with 'pk' in the struct then you have to set value of pk in id key before UnmarshalMap in map manually.
func UnmarshalMap(m map[string]*dynamodb.AttributeValue, out interface{}) error {
m["id"] = m["pk"]
dynamodbattribute.UnmarshalMap(m, out)
}
Tt's better to create a own generic Unmarshal function for this special case tweak and then call UnmarshalMap() inside.
artist := model.Artist{}
err = UnmarshalMap(result.Item, &artist)

Parse a dynamic yaml file

I have a yaml file that is currently written as:
keys:
- key: secret/dog
values:
- username: shiba
- password: inu
- key: secret/cat
values:
- dbhost: localhost
- words: meow
However, this yaml file often changes, so new entries could be added with different values each time:
keys:
- key: secret/dog
values:
- username: shiba
- password: inu
- key: secret/cat
values:
- dbhost: localhost
- words: meow
- key: secret/mouse
values:
- color: white
- key: secret/clouds
values:
- type: fluffy
I know from go using the gopkg.in/yaml.v2 package, i can parse through the yaml file, if all the values are the same, for example:
type Secrets struct {
Keys []struct {
Key string `json:"key"`
Values []struct {
Username string `json:"username"`
Password string `json:"password"`
} `json:"values"`
} `json:"keys"`
}
func main() {
var secret Secrets
reader, err := os.Open("demo.yml")
if err != nil {
log.Fatal(err)
}
buf, _ := ioutil.ReadAll(reader)
yaml.Unmarshal(buf, &secret)
fmt.Printf("%+v\n", secret.Keys[1].Key)
}
In the example above, it would only work for the secret/dog key, but not the others.
How can I do this in Go, when new values are added to my yaml file often?
Thanks
If you don't now exact struct you should probably make your struct looking like this
type Secrets struct {
Keys []struct {
Key string `json:"key"`
Values []map[string]string `json:"values"`
} `json:"keys"`
}
It will parse your whole yaml and get all values, but it will be an array so you loosing type hinting on object. Other way arround would be advance endoding https://blog.gopheracademy.com/advent-2016/advanced-encoding-decoding/ but you will need to add new object every time new key/value pair will appear.

Copy one struct to another where structs have same members and different types

I have two struct having the same members, I want to copy one struct to another, see the pseudo code below:
type Common struct {
Gender int
From string
To string
}
type Foo struct {
Id string
Name string
Extra Common
}
type Bar struct {
Id string
Name string
Extra Common
}
Then I have foo of struct Foo, and bar of struct Bar, Is there any way to copy bar from foo?
Use a conversion to change the type. The following code uses a conversion to copy a value of type Foo to a value of type Bar:
foo := Foo{Id: "123", Name: "Joe"}
bar := Bar(foo)
playground example
The conversion only works when the underlying types are identical except for struct tags.
https://github.com/jinzhu/copier (same author of gorm) is also quite a good one, I have nested structs and all I do is:
copier.Copy(&employees, &user)
works great
If you would like to copy or clone to a different struct, I would suggest using deepcopier
It provides nice features like skipping, custom mapping, and forcing. below is an example from github:
Install:
go get -u github.com/ulule/deepcopier
Example:
package main
import (
"fmt"
"github.com/ulule/deepcopier"
)
// Model
type User struct {
// Basic string field
Name string
// Deepcopier supports https://golang.org/pkg/database/sql/driver/#Valuer
Email sql.NullString
}
func (u *User) MethodThatTakesContext(ctx map[string]interface{}) string {
// do whatever you want
return "hello from this method"
}
// Resource
type UserResource struct {
//copy from field "Name"
DisplayName string `deepcopier:"field:Name"`
//this will be skipped in copy
SkipMe string `deepcopier:"skip"`
//this should call method named MethodThatTakesContext
MethodThatTakesContext string `deepcopier:"context"`
Email string `deepcopier:"force"`
}
func main() {
user := &User{
Name: "gilles",
Email: sql.NullString{
Valid: true,
String: "gilles#example.com",
},
}
resource := &UserResource{}
deepcopier.Copy(user).To(resource)
//copied from User's Name field
fmt.Println(resource.DisplayName)//output: gilles
fmt.Println(resource.Email) //output: gilles#example.com
fmt.Println(resource.MethodThatTakesContext) //output: hello from this method
}
Also, some other way you could achieve this is by encoding the source object to JSON and then decode it back to the destination object.
This is another possible answer
type Common struct {
Gender int
From string
To string
}
type Foo struct {
Id string
Name string
Extra Common
}
type Bar struct {
Id string
Name string
Extra Common
}
foo:=Foo{
Id:"123",
Name:"damitha",
Extra: struct {
Gender int
From string
To string
}{Gender:1 , From:"xx", To:"yy" },
}
bar:=*(*Bar)(unsafe.Pointer(&foo))
fmt.Printf("%+v\n",bar)
If you would like to copy or clone to a different struct, I would suggest using deepcopier
It provides nice features like skipping, custom mapping, and forcing.
You can achieve nested struct copy following way.
Install:
go get -u github.com/ulule/deepcopier
Example:
package main
import (
"fmt"
"strconv"
"github.com/ulule/deepcopier"
)
//FieldStruct - Field Struct
type FieldStruct struct {
Name string `deepcopier:"field:TargetName"`
Type string `deepcopier:"field:TargetType"`
}
//SourceStruct - Source Struct
type SourceStruct struct {
Name string `deepcopier:"field:TargetName"`
Age int `deepcopier:"field:TargetAge"`
StringArray []string `deepcopier:"field:TargetStringArray"`
StringToInt string `deepcopier:"context"`
Field FieldStruct
Fields []FieldStruct
}
//TargetFieldStruct - Field Struct
type TargetFieldStruct struct {
TargetName string
TargetType string
}
//TargetStruct - Target Struct
type TargetStruct struct {
TargetName string
TargetAge int
TargetStringArray []string
TargetInt int
TargetField TargetFieldStruct
TargetFields []TargetFieldStruct
}
//write methods
//TargetInt - StringToInt
func (s *SourceStruct) TargetInt() int {
i, _ := strconv.Atoi(s.StringToInt)
return i
}
func main() {
s := &SourceStruct{
Name: "Name",
Age: 12,
StringArray: []string{"1", "2"},
StringToInt: "123",
Field: FieldStruct{
Name: "Field",
Type: "String",
},
Fields: []FieldStruct{
FieldStruct{
Name: "Field1",
Type: "String1",
},
FieldStruct{
Name: "Field2",
Type: "String2",
},
},
}
t := &TargetStruct{}
//coping data into inner struct
deepcopier.Copy(&t.TargetField).From(&s.Field)
// copied array of Struct
for i := range s.Fields {
// init a struct
t.TargetFields = append(t.TargetFields, TargetFieldStruct{})
// coping the data
deepcopier.Copy(&t.TargetFields[i]).From(&s.Fields[i])
}
//Top level copy
deepcopier.Copy(t).From(s)
fmt.Println(t)
}
Output:
&{Name 12 [1 2] 123 {Field String} [{Field1 String1} {Field2 String2}]}

Resources