YAML files can contain fields with "multi-line string" data. Example below:
Data:
Foo: |
enemies=aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten
How can I properly encode and decode these in Golang and what should be the type of the Data field, map[string][]byte?
I tried:
import (
yamlv2 "gopkg.in/yaml.v2"
)
type data struct {
Data map[string][]byte
}
func decode(bytes []byte) (*data, error) {
d := &data{}
err := yamlv2.Unmarshal(bytes, d)
if err != nil {
return nil, err
}
return d, nil
}
Also see in Playground.
You would parse a multi-line string the same way as a normal string:
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"log"
)
var yamlString = `
Data:
Foo: |
enemies=aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten`
type data struct {
Data map[string]string `yaml:"Data"`
}
func main() {
t := data{}
err := yaml.Unmarshal([]byte(yamlString), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
if foo, ok := t.Data["Foo"]; ok {
fmt.Printf("--- t:\n%v\n\n", foo)
}
}
Related
I am getting the following error while using sequence of array in yaml file. I am not sure what fix required to be applied as I am new to 'Go' and learning the program.
2021/07/28 14:16:07 yaml: unmarshal errors:
line 3: cannot unmarshal !!seq into string
Yaml:
ENV:
attributes:
- foo
- boo
- coo
code:
package test
import (
"fmt"
"gopkg.in/yaml.v3"
"io/ioutil"
"log"
"testing"
)
type ENV struct {
Attributes string `yaml:"attributes"`
}
func TestTerraformAzureCosmosDBExample(t *testing.T) {
yFile, err := ioutil.ReadFile("config.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]ENV)
err = yaml.Unmarshal(yFile, &data)
if err != nil {
log.Fatal(err)
}
for k, v := range data {
fmt.Printf(`key: %v, value: %v`, k,v)
}
}
Expected:
foo
boo
coo
Actual:
C:\Pfoo\boo>go test -v
=== RUN TestTerraformAzureCosmosDBExample
2021/07/28 14:16:07 yaml: unmarshal errors:
line 3: cannot unmarshal !!seq into string
exit status 1
FAIL foo_test 0.199s
As mkopriva, said in the comments, you should change to []string, so the struct would be
type ENV struct {
Attributes []string `yaml:"attributes"`
}
And why is that ? the yaml, reconizes the
ENV: attributes: - foo - boo - coo as a array.
What you can do to turn into one string, is use:
strings.Join(String Slice,{ something to separate, you can let this as "")`
the imports are : "strings", and strings.Join returns a string.
As per suggestion of mkopriva changing Attributes field to string[] instead of string:
package test
import (
"fmt"
"gopkg.in/yaml.v3"
"io/ioutil"
"log"
"testing"
)
type ENV struct {
Attributes []string `yaml:"attributes"`
}
func TestTerraformAzureCosmosDBExample(t *testing.T) {
yFile, err := ioutil.ReadFile("config.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]ENV)
err = yaml.Unmarshal(yFile, &data)
if err != nil {
log.Fatal(err)
}
for k, v := range data {
fmt.Printf(`key: %v, value: %v`, k, v)
}
}
Here is an example I did. Hope it helps
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"io/ioutil"
"log"
)
type Image struct {
Name string `yaml:"name"`
Image string `yaml:"image"`
}
type Specification struct {
Spec struct {
Container []Image `yaml:"containers,flow"`
}
}
func main() {
container := Specification{}
content, err := ioutil.ReadFile("final-result.yml")
if err != nil {
log.Fatal(err.Error())
return
}
err = yaml.Unmarshal(content, &container)
if err != nil {
log.Fatalf("error: %v", err)
}
log.Println(container)
for _, ct := range container.Spec.Container {
fmt.Println(ct.Name)
}
}
spec:
containers:
- name: nginx
image: nginx:1.14.2
- name: php
image: php:1.4.8
- name: golang
image: golang:1.4.3
Output
nginx
php
golang
Language: Go
I am practicing how to read and print the yaml file data but unable to do so and the code is getting through. Can someone help here?
Yaml file:
ENV:
foo: test
boo: test-123-tet222
code:
package test
import (
"testing"
"fmt"
"io/ioutil"
"log"
"gopkg.in/yaml.v3"
)
type config struct {
foo string
boo string
}
func TestTerraformAzureCosmosDBExample(t *testing.T) {
yFile, err := ioutil.ReadFile("config.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]config)
err2 := yaml.Unmarshal(yFile, &data)
if err2 != nil {
log.Fatal(err2)
}
for k, v := range data {
fmt.Printf(k, v)
}
}
Expected Output:
foo: test
boo: test-123-tet222
Actual Output:
C:\foo\boo>go test -v
=== RUN TestTerraformAzureCosmosDBExample
ENV%!(EXTRA test.config={ })--- PASS: TestTerraformAzureCosmosDBExample (0.00s)
PASS
ok foobo_test 0.179s
Your config struct is missing yaml tags. Edit it as follows. Also the Printf method expects a formatter string, edit that too as follows.
import (
"fmt"
"gopkg.in/yaml.v3"
"io/ioutil"
"log"
"testing"
)
type config struct {
Foo string `yaml:"foo"`
Boo string `yaml:"boo"`
}
func TestTerraformAzureCosmosDBExample(t *testing.T) {
yFile, err := ioutil.ReadFile("config.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]config)
err = yaml.Unmarshal(yFile, &data)
if err != nil {
log.Fatal(err)
}
for k, v := range data {
fmt.Printf(`key: %v, value: %v`, k,v)
}
}
gives the output:
key: ENV, value: {test test-123-tet222}--- PASS: TestTerraformAzureCosmosDBExample (0.00s)
You are iterating over data that is a map[string]config.
That object has one key, ENV and the value for that key is the config object you are looking for.
Try with:
fmt.Printf("Foo: %S\n", data["ENV"].Foo)
fmt.Printf("Boo: %S\n", data["ENV"].Boo)
I have an issue with reading a YAML file. I think it's something in the file structure but I can't figure out what.
YAML file:
conf:
hits:5
time:5000000
code:
type conf struct {
hits int64 `yaml:"hits"`
time int64 `yaml:"time"`
}
func (c *conf) getConf() *conf {
yamlFile, err := ioutil.ReadFile("conf.yaml")
if err != nil {
log.Printf("yamlFile.Get err #%v ", err)
}
err = yaml.Unmarshal(yamlFile, c)
if err != nil {
log.Fatalf("Unmarshal: %v", err)
}
return c
}
your yaml file must be
hits: 5
time: 5000000
your code should look like this:
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"io/ioutil"
"log"
)
type conf struct {
Hits int64 `yaml:"hits"`
Time int64 `yaml:"time"`
}
func (c *conf) getConf() *conf {
yamlFile, err := ioutil.ReadFile("conf.yaml")
if err != nil {
log.Printf("yamlFile.Get err #%v ", err)
}
err = yaml.Unmarshal(yamlFile, c)
if err != nil {
log.Fatalf("Unmarshal: %v", err)
}
return c
}
func main() {
var c conf
c.getConf()
fmt.Println(c)
}
the main error was capital letter for your struct.
Example
Using an upgraded version 3 of yaml package.
An example conf.yaml file:
conf:
hits: 5
time: 5000000
camelCase: sometext
The main.go file:
package main
import (
"fmt"
"io/ioutil"
"log"
"gopkg.in/yaml.v3"
)
type myData struct {
Conf struct {
Hits int64
Time int64
CamelCase string `yaml:"camelCase"`
}
}
func readConf(filename string) (*myData, error) {
buf, err := ioutil.ReadFile(filename)
if err != nil {
return nil, err
}
c := &myData{}
err = yaml.Unmarshal(buf, c)
if err != nil {
return nil, fmt.Errorf("in file %q: %w", filename, err)
}
return c, err
}
func main() {
c, err := readConf("conf.yaml")
if err != nil {
log.Fatal(err)
}
fmt.Printf("%#v", c)
}
Running instructions (in case it's the first time you step out of stdlib):
cat conf.yaml
go mod init example.com/whatever
go get gopkg.in/yaml.v3
cat go.sum
go run .
About The yaml:"field"
The tags (like yaml:"field") are optional for all-lowercase yaml key identifiers. For demonstration the above example parses an extra camel case identifier which does require such a tag.
Corner Case: JSON+YAML
Confusingly, the useful lowercasing behavior of yaml package is not seen in the standard json package. Do you have a data structure which is sometimes encoded to JSON and sometimes to YAML? If so, consider specifying both JSON tags and YAML tags on literally every field. Verbose, but reduces mistakes. Example below.
type myData struct {
Conf conf `yaml:"conf" json:"conf"`
}
type conf struct {
Hits int64 `yaml:"hits" json:"hits"`
Time int64 `yaml:"time" json:"time"`
CamelCase string `yaml:"camelCase" json:"camelCase"`
}
package main
import (
"gopkg.in/yaml.v2"
"io/ioutil"
"log"
)
type someConf struct {
AccessKeyID string `yaml:"access_key_id"`
//...
}
func getConf(file string, cnf interface{}) error {
yamlFile, err := ioutil.ReadFile(file)
if err == nil {
err = yaml.Unmarshal(yamlFile, cnf)
}
return err
}
func main() {
cfg := &awsConf{}
if err := getConf("conf.yml", cfg); err != nil {
log.Panicln(err)
}
}
shortest getConf ever
I have a Go template that should resolve to a struct. How can I convert the bytes.Bufferresult from template execute function back to the struct. Playground
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
"text/template"
)
type Data struct {
Age int
Username string
SubData SubData
}
type SubData struct {
Name string
}
func main() {
s := SubData{Name: "J. Jr"}
d := Data{Age: 26, Username: "HelloWorld", SubData: s}
tmpl := "{{ .SubData }}"
t := template.New("My template")
t, _ = t.Parse(string(tmpl))
buffer := new(bytes.Buffer)
t.Execute(buffer, d)
fmt.Println(buffer)
// writing
enc := gob.NewEncoder(buffer)
err := enc.Encode(s)
if err != nil {
log.Fatal("encode error:", err)
}
// reading
buffer = bytes.NewBuffer(buffer.Bytes())
e := new(SubData)
dec := gob.NewDecoder(buffer)
err = dec.Decode(e)
if err != nil {
log.Fatal("decode error:", err)
}
fmt.Println(e, err)
}
You cannot. This is plain simply impossible.
But why on earth would anybody want to do something like this? Why don't you just send your Data directly via gob and decode it directly? Why creating a textual representation which you gob?
How can I fix the error?
http://play.golang.org/p/0UMnUZOUHw
// JSON to CSV in Golang
package main
import (
"encoding/csv"
"encoding/json"
"fmt"
"io/ioutil"
"os"
)
type Json struct {
RecordID int64 `json:"recordId"`
DOJ string `json:"Date of joining"`
EmpID string `json:"Employee ID"`
}
func main() {
// reading data from JSON File
data, err := ioutil.ReadFile("./people.json")
if err != nil {
fmt.Println(err)
}
// Unmarshal JSON data
var d []Json
err = json.Unmarshal([]byte(data), &d)
if err != nil {
fmt.Println(err)
}
// Create a csv file
f, _ := os.Create("./people.csv")
defer f.Close()
// Write Unmarshaled json data to CSV file
w := csv.NewWriter(f)
// How to proceed further?
/* I am getting errors if i use below
for _,obj := range d {
var record []interface{}
record = append(record, obj.RecordID)
record = append(record, obj.DOJ)
record = append(record, obj.EmpID)
w.Write(record)
}
*/
}
Error:
cannot use record (type []interface {}) as type []string in function argument
For example,
package main
import (
"encoding/csv"
"encoding/json"
"fmt"
"io/ioutil"
"os"
"strconv"
)
type Json struct {
RecordID int64 `json:"recordId"`
DOJ string `json:"Date of joining"`
EmpID string `json:"Employee ID"`
}
func main() {
// reading data from JSON File
data, err := ioutil.ReadFile("./people.json")
if err != nil {
fmt.Println(err)
}
// Unmarshal JSON data
var d []Json
err = json.Unmarshal([]byte(data), &d)
if err != nil {
fmt.Println(err)
}
// Create a csv file
f, err := os.Create("./people.csv")
if err != nil {
fmt.Println(err)
}
defer f.Close()
// Write Unmarshaled json data to CSV file
w := csv.NewWriter(f)
for _, obj := range d {
var record []string
record = append(record, strconv.FormatInt(obj.RecordID, 10))
record = append(record, obj.DOJ)
record = append(record, obj.EmpID)
w.Write(record)
}
w.Flush()
}
csv Writer's Write does not expect an []interface{} it expects a []string.