I have yaml file which contain special character ! in the key-value pair as shown below:
name: Amit Kumar
age: !kk
address:
street: !123 Some Random Street
city: Chennai
state: TN
zip: 763098
phoneNumbers:
- type: home
number: 0123456789
- type: work
number: 0987654321
While unmarshall this yaml file, street: !123 Some Random Street or age: !dd, any word that begin with ! are removed.
For example:
person.age or person.address will result in
age: kk
address: map[city:Chennai state:TN street:Some Random Street zip:763098]
expected:
age: !kk
address: map[city:Chennai state:TN street:!123 Some Random Street zip:763098]
Any suggestion on retaining the special character while unmarshalling the yaml file?
Go file:
package main
import (
"fmt"
"gopkg.in/yaml.v3"
"io/ioutil"
)
type Address struct {
Street string `yaml:"street"`
City string `yaml:"city"`
State string `yaml:"state"`
Zip string `yaml:"zip"`
}
type PhoneNumber struct {
Type string `yaml:"type"`
Number string `yaml:"number"`
}
type Person struct {
Name string `yaml:"name"`
Age string `yaml:"age"`
Address interface{} `yaml:"address"`
PhoneNumber []PhoneNumber `yaml:"phoneNumbers"`
}
func main() {
// Read the file
data, err := ioutil.ReadFile("testyamltemplate.yaml")
if err != nil {
fmt.Println(err)
return
}
// Create a struct to hold the YAML data
var person Person
// Unmarshal the YAML data into the struct
err = yaml.Unmarshal(data, &person)
if err != nil {
fmt.Println(err)
return
}
// Print the data
fmt.Println(person.Age)
}
Unquoted leading exclamation mark is treated by YAML specification as a start of a tag.
For prevent such interpretation wrap the whole value with double or single quotes:
age: "!kk"
street: "!123 Some Random Street"
Related
I use viper. I'm trying to get info from structs with yml-config.
type Config struct {
Account User `mapstructure:"user"`
}
type User struct {
Name string `mapstructure:"name"`
Contacts []Contact `mapstructure:"contact"`
}
type Contact struct {
Type string `mapstructure:"type"`
Value string `mapstructure:"value"`
}
func Init() *Config {
conf := new(Config)
viper.SetConfigType("yaml")
viper.ReadInConfig()
...
viper.Unmarshal(conf)
return conf
}
...
config := Init()
...
for _, contact := range config.Account.Contacts {
type := contact.type
vlaue := contact.value
}
And config.yml
user:
name: John
contacts:
email:
type: email
value: test#test.com
skype:
type: skype
value: skypeacc
Can I get structure items like this? I could not get contact data like that. Is it possible?
If I'm getting right what you want to achieve, and based on the for loop you provided;
What you actually need is a YAML sequence, which is an array. So your final YAML file should look like;
user:
name: John
contacts:
- type: email
value: test#test.com
- type: skype
value: skypeacc
- type: email
value: joe#example.com
Also, you have a typo in your tag on Contacts slice. It should match the YAML key;
type User struct {
Name string `mapstructure:"name"`
Contacts []Contact `mapstructure:"contacts"`
}
If you wish to keep the original YAML file structure you have to provide a tag (and the corresponding struct field) for each YAML key, so looping over it wouldn't be possible out of the box, because email and skype are parsed as struct fields. An example of the struct for the original YAML file would be;
type Config struct {
Account User `mapstructure:"user"`
}
type User struct {
Name string `mapstructure:"name"`
Contacts Contacts `mapstructure:"contacts"`
}
type Contacts struct {
Email Contact `mapstructure:"email"`
Skype Contact `mapstructure:"skype"`
}
type Contact struct {
Type string `mapstructure:"type"`
Value string `mapstructure:"value"`
}
I think the only significant problem is that in your data structures you've declared Contacts as a list, but in your YAML file it's a dictionary. If you structure your input file like this:
user:
name: John
contacts:
- type: email
value: test#test.com
- type: skype
value: skypeacc
Then you can read it like this:
package main
import (
"fmt"
"github.com/spf13/viper"
)
type Config struct {
User User
}
type User struct {
Name string
Contacts []Contact
}
type Contact struct {
Type string
Value string
}
func main() {
var cfg Config
viper.SetConfigName("config")
viper.AddConfigPath(".")
err := viper.ReadInConfig()
if err != nil {
panic(err)
}
viper.Unmarshal(&cfg)
fmt.Println("user: ", cfg.User.Name)
for _, contact := range cfg.User.Contacts {
fmt.Println(" ", contact.Type, ": ", contact.Value)
}
}
The above code is runnable as-is; you should just be able to drop it
into a file and build it. When I run the above example, I get as
output:
user: John
email : test#test.com
skype : skypeacc
I have a problem unmarshaling a simple slice of YAML data:
package main
import (
"fmt"
"gopkg.in/yaml.v2"
)
type myDataStruct struct {
HTTP []struct {
Name string
Target string
}
}
func main() {
yamlData := `
HTTP:
- name: one
target: http://wazaa
- name: two
target: http://wazii
`
var myData myDataStruct
err := yaml.Unmarshal([]byte(yamlData), &myData)
if err != nil {
fmt.Printf("error: %v", err)
return
}
fmt.Print(myData)
}
Playground: https://play.golang.org/p/Srb2DJVVZqN
The result is {[]} and my limited knowledge of Go does not help to understand why?
If you don't specify the mapping between labels used in the YAML source and Go struct fields, by default they will only be matched if only changing the first letter to lower matches.
E.g. the struct field Name will match name, but not NAME.
Specify the mapping for the HTTP field:
type myDataStruct struct {
HTTP []struct {
Name string
Target string
} `yaml:"HTTP"`
}
With this change it works and outputs (try it on the Go Playground):
{[{one http://wazaa} {two http://wazii}]}
It's good practice to provide mappings for all fields, so it'll continue to work if you rename the fields:
type myDataStruct struct {
HTTP []struct {
Name string `yaml:"name"`
Target string `yaml:"target"`
} `yaml:"HTTP"`
}
Although in your case the default matching works for the Name and Target fields without providing the mappings.
I have two struct having the same members, I want to copy one struct to another, see the pseudo code below:
type Common struct {
Gender int
From string
To string
}
type Foo struct {
Id string
Name string
Extra Common
}
type Bar struct {
Id string
Name string
Extra Common
}
Then I have foo of struct Foo, and bar of struct Bar, Is there any way to copy bar from foo?
Use a conversion to change the type. The following code uses a conversion to copy a value of type Foo to a value of type Bar:
foo := Foo{Id: "123", Name: "Joe"}
bar := Bar(foo)
playground example
The conversion only works when the underlying types are identical except for struct tags.
https://github.com/jinzhu/copier (same author of gorm) is also quite a good one, I have nested structs and all I do is:
copier.Copy(&employees, &user)
works great
If you would like to copy or clone to a different struct, I would suggest using deepcopier
It provides nice features like skipping, custom mapping, and forcing. below is an example from github:
Install:
go get -u github.com/ulule/deepcopier
Example:
package main
import (
"fmt"
"github.com/ulule/deepcopier"
)
// Model
type User struct {
// Basic string field
Name string
// Deepcopier supports https://golang.org/pkg/database/sql/driver/#Valuer
Email sql.NullString
}
func (u *User) MethodThatTakesContext(ctx map[string]interface{}) string {
// do whatever you want
return "hello from this method"
}
// Resource
type UserResource struct {
//copy from field "Name"
DisplayName string `deepcopier:"field:Name"`
//this will be skipped in copy
SkipMe string `deepcopier:"skip"`
//this should call method named MethodThatTakesContext
MethodThatTakesContext string `deepcopier:"context"`
Email string `deepcopier:"force"`
}
func main() {
user := &User{
Name: "gilles",
Email: sql.NullString{
Valid: true,
String: "gilles#example.com",
},
}
resource := &UserResource{}
deepcopier.Copy(user).To(resource)
//copied from User's Name field
fmt.Println(resource.DisplayName)//output: gilles
fmt.Println(resource.Email) //output: gilles#example.com
fmt.Println(resource.MethodThatTakesContext) //output: hello from this method
}
Also, some other way you could achieve this is by encoding the source object to JSON and then decode it back to the destination object.
This is another possible answer
type Common struct {
Gender int
From string
To string
}
type Foo struct {
Id string
Name string
Extra Common
}
type Bar struct {
Id string
Name string
Extra Common
}
foo:=Foo{
Id:"123",
Name:"damitha",
Extra: struct {
Gender int
From string
To string
}{Gender:1 , From:"xx", To:"yy" },
}
bar:=*(*Bar)(unsafe.Pointer(&foo))
fmt.Printf("%+v\n",bar)
If you would like to copy or clone to a different struct, I would suggest using deepcopier
It provides nice features like skipping, custom mapping, and forcing.
You can achieve nested struct copy following way.
Install:
go get -u github.com/ulule/deepcopier
Example:
package main
import (
"fmt"
"strconv"
"github.com/ulule/deepcopier"
)
//FieldStruct - Field Struct
type FieldStruct struct {
Name string `deepcopier:"field:TargetName"`
Type string `deepcopier:"field:TargetType"`
}
//SourceStruct - Source Struct
type SourceStruct struct {
Name string `deepcopier:"field:TargetName"`
Age int `deepcopier:"field:TargetAge"`
StringArray []string `deepcopier:"field:TargetStringArray"`
StringToInt string `deepcopier:"context"`
Field FieldStruct
Fields []FieldStruct
}
//TargetFieldStruct - Field Struct
type TargetFieldStruct struct {
TargetName string
TargetType string
}
//TargetStruct - Target Struct
type TargetStruct struct {
TargetName string
TargetAge int
TargetStringArray []string
TargetInt int
TargetField TargetFieldStruct
TargetFields []TargetFieldStruct
}
//write methods
//TargetInt - StringToInt
func (s *SourceStruct) TargetInt() int {
i, _ := strconv.Atoi(s.StringToInt)
return i
}
func main() {
s := &SourceStruct{
Name: "Name",
Age: 12,
StringArray: []string{"1", "2"},
StringToInt: "123",
Field: FieldStruct{
Name: "Field",
Type: "String",
},
Fields: []FieldStruct{
FieldStruct{
Name: "Field1",
Type: "String1",
},
FieldStruct{
Name: "Field2",
Type: "String2",
},
},
}
t := &TargetStruct{}
//coping data into inner struct
deepcopier.Copy(&t.TargetField).From(&s.Field)
// copied array of Struct
for i := range s.Fields {
// init a struct
t.TargetFields = append(t.TargetFields, TargetFieldStruct{})
// coping the data
deepcopier.Copy(&t.TargetFields[i]).From(&s.Fields[i])
}
//Top level copy
deepcopier.Copy(t).From(s)
fmt.Println(t)
}
Output:
&{Name 12 [1 2] 123 {Field String} [{Field1 String1} {Field2 String2}]}
Take a look at this snip found at here
import (
"encoding/xml"
"fmt"
"os"
)
func main() {
type Address struct {
City, State string
}
type Person struct {
XMLName xml.Name `xml:"person"`
Id int `xml:"id,attr"`
FirstName string `xml:"name>first"`
LastName string `xml:"name>last"`
Age int `xml:"age"`
Height float32 `xml:"height,omitempty"`
Married bool
Address
Comment string `xml:",comment"`
}
v := &Person{Id: 13, FirstName: "John", LastName: "Doe", Age: 42}
v.Comment = " Need more details. "
v.Address = Address{"Hanga Roa", "Easter Island"}
enc := xml.NewEncoder(os.Stdout)
enc.Indent(" ", " ")
if err := enc.Encode(v); err != nil {
fmt.Printf("error: %v\n", err)
}
}
I can understand in the struct Person, It has a var called Id, which is of type int, but what about the stuff xml:"person" after int? What does it mean? Thanks.
It's a struct tag. Libraries use these to annotate struct fields with extra information; in this case, the module encoding/xml uses these struct tags to denote which tags correspond to the struct fields.
which mean that variable will present in the name of Person example
type sample struct {
dateofbirth string `xml:"dob"`
}
In the above example, the field 'dateofbirth' will present in the name of 'dob' in the XML.
you will see this notation often in go struct.
I'm starting out with Google go language, and I'm trying to write a simple program to obtain a Json object from Facebook's graph API and unmarshal it.
According to the documentation, http://golang.org/doc/articles/json_and_go.html, I should provide a matching data structure, for example if the json object is:
{
Name: "Alice"
Body: "Hello",
Time: 1294706395881547000,
}
The data structure will look like this:
type Message struct {
Name string
Body string
Time int64
}
In my case, my json look like this:
{
"id": "350685531728",
"name": "Facebook for Android",
"description": "Keep up with friends, wherever you are.",
"category": "Utilities",
"subcategory": "Communication",
"link": "http://www.facebook.com/apps/application.php?id=350685531728",
"namespace": "fbandroid",
"icon_url": "http://static.ak.fbcdn.net/rsrc.php/v2/yo/r/OKB7Z2hkREe.png",
"logo_url": "http://photos-b.ak.fbcdn.net/photos-ak-snc7/v43/207/227200684078827/app_115_227200684078827_474764570.png",
"company": "Facebook"
}
So I provided a matching data structure:
type Graph struct {
id string
name string
description string
category string
subcategory string
link string
namespace string
icon_url string
logo_url string
company string
}
This is the code:
package main
import "fmt"
import "net/http"
import "io/ioutil"
import "encoding/json"
type Graph struct {
id string
name string
description string
category string
subcategory string
link string
namespace string
icon_url string
logo_url string
company string
}
func main() {
fmt.Println("Hello, playground")
resp, err := http.Get("http://graph.facebook.com/android")
if err != nil {
fmt.Println("http.Get err: ",err)// handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
fmt.Println("resp.Body: ", resp.Body)
fmt.Println("body: ",string(body))
fmt.Println("err: ",err)
var g Graph
err = json.Unmarshal(body, &g)
if err == nil {
fmt.Println("graph: ", g)
} else {
fmt.Println("graph error: ",err) // <---error at this line
}
}
But I'm still getting the following error:
graph error: json: cannot unmarshal object key "id" into unexported field id of type main.Graph
Any idea?
Thanks!
All of the given fields are lowercase, and therefore unexported. In order to unmarshal into the structure, you'll need to make use of tags.
For example, a valid structure in your case would be
type Graph struct {
Id string `json:"id"`
Name string `json:"name"`
Description string `json:"description"`
Category string `json:"category"`
Subcategory string `json:"subcategory"`
Link string `json:"link"`
Namespace string `json:"namespace"`
Icon_url string `json:"icon_url"`
Logo_url string `json:"logo_url"`
Company string `json:"company"`
}
However, because of the way that encoding/json behaves, (no link, sorry, I'm in a bit of a rush,) you may be able to get by without the tags entirely. Just make sure that your field names are exported.
The field id is unexported because it starts with a lower case letter. See Exported identifiers.