How to read a YAML file - go

I have an issue with reading a YAML file. I think it's something in the file structure but I can't figure out what.
YAML file:
conf:
hits:5
time:5000000
code:
type conf struct {
hits int64 `yaml:"hits"`
time int64 `yaml:"time"`
}
func (c *conf) getConf() *conf {
yamlFile, err := ioutil.ReadFile("conf.yaml")
if err != nil {
log.Printf("yamlFile.Get err #%v ", err)
}
err = yaml.Unmarshal(yamlFile, c)
if err != nil {
log.Fatalf("Unmarshal: %v", err)
}
return c
}

your yaml file must be
hits: 5
time: 5000000
your code should look like this:
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"io/ioutil"
"log"
)
type conf struct {
Hits int64 `yaml:"hits"`
Time int64 `yaml:"time"`
}
func (c *conf) getConf() *conf {
yamlFile, err := ioutil.ReadFile("conf.yaml")
if err != nil {
log.Printf("yamlFile.Get err #%v ", err)
}
err = yaml.Unmarshal(yamlFile, c)
if err != nil {
log.Fatalf("Unmarshal: %v", err)
}
return c
}
func main() {
var c conf
c.getConf()
fmt.Println(c)
}
the main error was capital letter for your struct.

Example
Using an upgraded version 3 of yaml package.
An example conf.yaml file:
conf:
hits: 5
time: 5000000
camelCase: sometext
The main.go file:
package main
import (
"fmt"
"io/ioutil"
"log"
"gopkg.in/yaml.v3"
)
type myData struct {
Conf struct {
Hits int64
Time int64
CamelCase string `yaml:"camelCase"`
}
}
func readConf(filename string) (*myData, error) {
buf, err := ioutil.ReadFile(filename)
if err != nil {
return nil, err
}
c := &myData{}
err = yaml.Unmarshal(buf, c)
if err != nil {
return nil, fmt.Errorf("in file %q: %w", filename, err)
}
return c, err
}
func main() {
c, err := readConf("conf.yaml")
if err != nil {
log.Fatal(err)
}
fmt.Printf("%#v", c)
}
Running instructions (in case it's the first time you step out of stdlib):
cat conf.yaml
go mod init example.com/whatever
go get gopkg.in/yaml.v3
cat go.sum
go run .
About The yaml:"field"
The tags (like yaml:"field") are optional for all-lowercase yaml key identifiers. For demonstration the above example parses an extra camel case identifier which does require such a tag.
Corner Case: JSON+YAML
Confusingly, the useful lowercasing behavior of yaml package is not seen in the standard json package. Do you have a data structure which is sometimes encoded to JSON and sometimes to YAML? If so, consider specifying both JSON tags and YAML tags on literally every field. Verbose, but reduces mistakes. Example below.
type myData struct {
Conf conf `yaml:"conf" json:"conf"`
}
type conf struct {
Hits int64 `yaml:"hits" json:"hits"`
Time int64 `yaml:"time" json:"time"`
CamelCase string `yaml:"camelCase" json:"camelCase"`
}

package main
import (
"gopkg.in/yaml.v2"
"io/ioutil"
"log"
)
type someConf struct {
AccessKeyID string `yaml:"access_key_id"`
//...
}
func getConf(file string, cnf interface{}) error {
yamlFile, err := ioutil.ReadFile(file)
if err == nil {
err = yaml.Unmarshal(yamlFile, cnf)
}
return err
}
func main() {
cfg := &awsConf{}
if err := getConf("conf.yml", cfg); err != nil {
log.Panicln(err)
}
}
shortest getConf ever

Related

cannot unmarshal !!seq into string in Go

I am getting the following error while using sequence of array in yaml file. I am not sure what fix required to be applied as I am new to 'Go' and learning the program.
2021/07/28 14:16:07 yaml: unmarshal errors:
line 3: cannot unmarshal !!seq into string
Yaml:
ENV:
attributes:
- foo
- boo
- coo
code:
package test
import (
"fmt"
"gopkg.in/yaml.v3"
"io/ioutil"
"log"
"testing"
)
type ENV struct {
Attributes string `yaml:"attributes"`
}
func TestTerraformAzureCosmosDBExample(t *testing.T) {
yFile, err := ioutil.ReadFile("config.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]ENV)
err = yaml.Unmarshal(yFile, &data)
if err != nil {
log.Fatal(err)
}
for k, v := range data {
fmt.Printf(`key: %v, value: %v`, k,v)
}
}
Expected:
foo
boo
coo
Actual:
C:\Pfoo\boo>go test -v
=== RUN TestTerraformAzureCosmosDBExample
2021/07/28 14:16:07 yaml: unmarshal errors:
line 3: cannot unmarshal !!seq into string
exit status 1
FAIL foo_test 0.199s
As mkopriva, said in the comments, you should change to []string, so the struct would be
type ENV struct {
Attributes []string `yaml:"attributes"`
}
And why is that ? the yaml, reconizes the
ENV: attributes: - foo - boo - coo as a array.
What you can do to turn into one string, is use:
strings.Join(String Slice,{ something to separate, you can let this as "")`
the imports are : "strings", and strings.Join returns a string.
As per suggestion of mkopriva changing Attributes field to string[] instead of string:
package test
import (
"fmt"
"gopkg.in/yaml.v3"
"io/ioutil"
"log"
"testing"
)
type ENV struct {
Attributes []string `yaml:"attributes"`
}
func TestTerraformAzureCosmosDBExample(t *testing.T) {
yFile, err := ioutil.ReadFile("config.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]ENV)
err = yaml.Unmarshal(yFile, &data)
if err != nil {
log.Fatal(err)
}
for k, v := range data {
fmt.Printf(`key: %v, value: %v`, k, v)
}
}
Here is an example I did. Hope it helps
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"io/ioutil"
"log"
)
type Image struct {
Name string `yaml:"name"`
Image string `yaml:"image"`
}
type Specification struct {
Spec struct {
Container []Image `yaml:"containers,flow"`
}
}
func main() {
container := Specification{}
content, err := ioutil.ReadFile("final-result.yml")
if err != nil {
log.Fatal(err.Error())
return
}
err = yaml.Unmarshal(content, &container)
if err != nil {
log.Fatalf("error: %v", err)
}
log.Println(container)
for _, ct := range container.Spec.Container {
fmt.Println(ct.Name)
}
}
spec:
containers:
- name: nginx
image: nginx:1.14.2
- name: php
image: php:1.4.8
- name: golang
image: golang:1.4.3
Output
nginx
php
golang

Unable to read and print yaml file data

Language: Go
I am practicing how to read and print the yaml file data but unable to do so and the code is getting through. Can someone help here?
Yaml file:
ENV:
foo: test
boo: test-123-tet222
code:
package test
import (
"testing"
"fmt"
"io/ioutil"
"log"
"gopkg.in/yaml.v3"
)
type config struct {
foo string
boo string
}
func TestTerraformAzureCosmosDBExample(t *testing.T) {
yFile, err := ioutil.ReadFile("config.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]config)
err2 := yaml.Unmarshal(yFile, &data)
if err2 != nil {
log.Fatal(err2)
}
for k, v := range data {
fmt.Printf(k, v)
}
}
Expected Output:
foo: test
boo: test-123-tet222
Actual Output:
C:\foo\boo>go test -v
=== RUN TestTerraformAzureCosmosDBExample
ENV%!(EXTRA test.config={ })--- PASS: TestTerraformAzureCosmosDBExample (0.00s)
PASS
ok foobo_test 0.179s
Your config struct is missing yaml tags. Edit it as follows. Also the Printf method expects a formatter string, edit that too as follows.
import (
"fmt"
"gopkg.in/yaml.v3"
"io/ioutil"
"log"
"testing"
)
type config struct {
Foo string `yaml:"foo"`
Boo string `yaml:"boo"`
}
func TestTerraformAzureCosmosDBExample(t *testing.T) {
yFile, err := ioutil.ReadFile("config.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]config)
err = yaml.Unmarshal(yFile, &data)
if err != nil {
log.Fatal(err)
}
for k, v := range data {
fmt.Printf(`key: %v, value: %v`, k,v)
}
}
gives the output:
key: ENV, value: {test test-123-tet222}--- PASS: TestTerraformAzureCosmosDBExample (0.00s)
You are iterating over data that is a map[string]config.
That object has one key, ENV and the value for that key is the config object you are looking for.
Try with:
fmt.Printf("Foo: %S\n", data["ENV"].Foo)
fmt.Printf("Boo: %S\n", data["ENV"].Boo)

Generate .pb file without using protoc in golang

I'm trying to generate .pb.go file using service.proto as file input in Go.
Is there a way to do it without using protoc binary (like directly using package github.com/golang/protobuf/protoc-gen-go)?
If you have a detail.proto like this:
message AppDetails {
optional string version = 4;
}
You can parse it into a message like this:
package main
import (
"fmt"
"github.com/golang/protobuf/proto"
"github.com/jhump/protoreflect/desc/protoparse"
"github.com/jhump/protoreflect/dynamic"
)
func parse(file, msg string) (*dynamic.Message, error) {
var p protoparse.Parser
fd, err := p.ParseFiles(file)
if err != nil {
return nil, err
}
md := fd[0].FindMessage(msg)
return dynamic.NewMessage(md), nil
}
func main() {
b := []byte("\"\vhello world")
m, err := parse("detail.proto", "AppDetails")
if err != nil {
panic(err)
}
if err := proto.Unmarshal(b, m); err != nil {
panic(err)
}
fmt.Println(m) // version:"hello world"
}
However you may notice, this package is still using the old Protobuf V1. I did
find a Pull Request for V2:
https://github.com/jhump/protoreflect/pull/354

Decode/Encode multi-line string fields from YAML using golang

YAML files can contain fields with "multi-line string" data. Example below:
Data:
Foo: |
enemies=aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten
How can I properly encode and decode these in Golang and what should be the type of the Data field, map[string][]byte?
I tried:
import (
yamlv2 "gopkg.in/yaml.v2"
)
type data struct {
Data map[string][]byte
}
func decode(bytes []byte) (*data, error) {
d := &data{}
err := yamlv2.Unmarshal(bytes, d)
if err != nil {
return nil, err
}
return d, nil
}
Also see in Playground.
You would parse a multi-line string the same way as a normal string:
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"log"
)
var yamlString = `
Data:
Foo: |
enemies=aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten`
type data struct {
Data map[string]string `yaml:"Data"`
}
func main() {
t := data{}
err := yaml.Unmarshal([]byte(yamlString), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
if foo, ok := t.Data["Foo"]; ok {
fmt.Printf("--- t:\n%v\n\n", foo)
}
}

Specify names for parsed templates

I am trying to dynamically parse files using walk in a folder and I want to be able to set the path of the file "path/file.html". But my issue is if I have a file in a folder "path/folder/files.html" I can't do it because when I ExecuteTemplate the file name will be the same "files.html". Is it possible to name each template as I ParseFiles?
Im ok with doing a file one at a time if trying to do them all at once wont work.
// Parse file and send to responsewriter
func View(w http.ResponseWriter, path string) {
temp, err := template.ParseFiles("application/views/"+path+".html")
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
} else {
temp.ExecuteTemplate(w, path, nil)
}
}
Walk the filesystem using filepath.Walk and a consumer method that will create templates with the full file paths as names:
package main
import (
"fmt"
"html/template"
"os"
"path/filepath"
)
func consumer(p string, i os.FileInfo, e error) error {
t := template.New(p)
fmt.Println(t.Name())
return nil
}
func main() {
filepath.Walk("/path/to/template/root", filepath.WalkFunc(consumer))
}
You can try template.Lookup, the whole process looks like:
var (
templates *template.Template
)
func loadTemplate() {
funcMap := template.FuncMap{
"safe":func(s string) template.HTML {
return template.HTML(s)
},
}
var err error
templates, err = utils.BuildTemplate("/theme/path/", funcMap)
if err != nil {
log.Printf("Can't read template file %v,", err)
}
}
func homeHandler(w http.ResponseWriter, r *http.Request) {
//lookup the theme your want to use
templ = templates.Lookup("theme.html")
err := templ.Execute(w, data)
if err != nil {
log.Println(err)
}
}
func main() {
loadTemplate()
}
BuildTemplate looks like:
func BuildTemplate(dir string, funcMap template.FuncMap) (*template.Template, error) {
fs, err := ioutil.ReadDir(dir)
if err != nil {
fmt.Printf("Can't read template folder: %s\n", dir)
return nil, err
}
files := make([]string, len(fs))
for i, f := range (fs) {
files[i] = path.Join(dir, f.Name())
}
return template.Must(template.New("Template").Funcs(funcMap).ParseFiles(files...)), nil
}

Resources