Unable to extract key/value pairs from a map obtained through YAML - go

I use the goldmark-meta package to read a YAML file.
The contents I'm interested in look like this
in the YAML:
burger:
- a: ay
- b: bee
- c: see
I want to get access to both keys and values
of the returned interface, and I'm stuck.
Iterating through the return gives me a
list of the key/value pairs, but I don't
know how to obtain any info when I don't
know the key names in advance.
This program prints out the following:
func (c *config) burger() string {
// c.pageFm is type map[string]interface{}
b, ok := c.pageFm["burger"].([]interface{})
if !ok {
// No burger entry in yaml
return ""
}
debug("Burger list has %v items:\n%v", len(b), b)
debug("Type: %v", reflect.TypeOf(b))
for i := 0; i < len(b); i++ {
debug("%v", b[i])
}
return ""
}
Burger list has 3 items:
[map[a:ay] map[b:bee] map[c:see]]
Type: []interface {}
map[a:ay]
map[b:bee]
map[c:see]
How do I obtain the key and value strings?

In your YAML data, you have a key (burger) the value of which is a list of maps (and each map has a single key). We can iterate over the items in burger like this:
b, ok := c.pageFm["burger"].([]interface{})
if !ok {
return ""
}
for _, item := range burger {
...
}
For each item, we can iterate over available keys and values:
for _, item := range burger {
for k, v := range item.(map[interface{}]interface{}) {
...
}
}
We can convert keys and values from interface{} into string using fmt.Sprintf:
for _, item := range burger {
for k, v := range item.(map[interface{}]interface{}) {
k_str := fmt.Sprintf("%v", k)
v_str := fmt.Sprintf("%v", v)
fmt.Printf("key %s value %s\n", k_str, v_str)
}
}
Starting with the sample code from goldmark-meta, I put together this example:
package main
import (
"bytes"
"fmt"
"github.com/yuin/goldmark"
meta "github.com/yuin/goldmark-meta"
"github.com/yuin/goldmark/parser"
)
func main() {
markdown := goldmark.New(
goldmark.WithExtensions(
meta.Meta,
),
)
source := `---
burger:
- a: ay
- b: bee
- c: see
---
# Hello goldmark-meta
`
var buf bytes.Buffer
context := parser.NewContext()
if err := markdown.Convert([]byte(source), &buf, parser.WithContext(context)); err != nil {
panic(err)
}
metaData := meta.Get(context)
burger := metaData["burger"].([]interface{})
for _, item := range burger {
for k, v := range item.(map[interface{}]interface{}) {
k_str := fmt.Sprintf("%v", k)
v_str := fmt.Sprintf("%v", v)
fmt.Printf("key %s value %s\n", k_str, v_str)
}
}
}
Which outputs:
key a value ay
key b value bee
key c value see

Related

How to convert numerical value to json in GoLang

I have a Go container that calculates the total marks scored, and works perfectly.
For this to be able to be read by my proxy file, i need the return value to be json.
I am trying this however it isnt working:
// Find the total grade
marksSum := 0
for _, mark := range marks {
marksSum += mark
}
j, _ := json.Marshal(markSum)
return j
Any help is much appreciated!
You can create a struct of how you want to structure your JSON object.(variable names should start with a capital letters)
type Response struct {
Error error `json:"error"`
Input_text string `json:"string"`
Answer int `json:"answer"`
}
Then just create a response using the above struct.
func main() {
marks := []int{1, 2, 3, 4}
marksSum := 0
input := ""
for _, mark := range marks {
input = fmt.Sprintf("%s %d", input, mark)
marksSum += mark
}
resp := &Response{
Error: nil,
Input_text: input,
Answer: marksSum,
}
j, err := json.Marshal(resp)
if err != nil {
fmt.Printf("Errr : %v", err)
return
}
fmt.Println(string(j))
}
https://go.dev/play/p/iC484GS7GKS

Golang comparing two yaml files and updating them

Im a newbie in golang. I am trying to compare two yaml files and update the 2nd file's value if there is any new value in 1st yaml for that particular key.
So the files are of format: These are sample yaml files. Real yaml files have much more nested complicated maps with different datatypes for each key.
1st yaml:
name: john
city: washington
2nd yaml:
name: peter
city: washington
Final result for 2nd yaml file should be:
name: john
city: washington
Tried creating a map string interface for both yaml files using unmarshal. But having trouble how to compare both maps. Was trying to loop over each key of map and search for that key in 2nd yaml map. If key exists update the value in 2nd yaml map. But i am not able to implement that. Any suggestions/better ideas?
Edit: Updated code
package main
import (
"fmt"
"io/ioutil"
"log"
"github.com/imdario/mergo"
"gopkg.in/yaml.v3"
)
func main() {
yfile, err := ioutil.ReadFile("C:/Users/212764682/lifecycle/userconfig.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]interface{})
err2 := yaml.Unmarshal(yfile, &data)
if err2 != nil {
log.Fatal(err2)
} else {
yfile1, err3 := ioutil.ReadFile("C:/Users/212764682/lifecycle/conf.yaml")
yfile2, err4 := ioutil.ReadFile("C:/Users/212764682/lifecycle/prof.yaml")
if err3 != nil && err4 != nil {
log.Fatal(err3)
log.Fatal(err4)
} else {
dat := make(map[string]interface{})
dat2 := make(map[string]interface{})
err5 := yaml.Unmarshal(yfile1, &dat)
err6 := yaml.Unmarshal(yfile2, &dat2)
_ = err5
_ = err6
for key1, element1 := range data {
for key2, element2 := range dat {
if key1 == key2 {
if element1 == element2 {
} else {
element2 = element1
}
} else {
dat[key1] = data[key1]
}
}
}
}
}
}
So im want to compare each key of data with dat. If that key exists in dat, check for value in data. If value different in dat, update with value of data in dat for that key. Also, if any key of data dosent exist in dat, then append that key in dat. But not able to implement it correctly.
You can try to compare the map and then update it if the key exists. Here's some example using your case.
package main
import (
"fmt"
"io/ioutil"
"log"
"gopkg.in/yaml.v3"
)
func main() {
// read file
yfile1, err := ioutil.ReadFile("file1.yaml")
if err != nil {
log.Fatal(err)
return
}
yfile2, err := ioutil.ReadFile("file2.yaml")
if err != nil {
log.Fatal(err)
return
}
// unmarshal ymal
data1 := make(map[string]interface{})
if err := yaml.Unmarshal(yfile1, &data1); err != nil {
log.Fatal(err)
return
}
data2 := make(map[string]interface{})
if err := yaml.Unmarshal(yfile2, &data2); err != nil {
log.Fatal(err)
return
}
// from this we can iterate the key in data2 then check whether it exists in data1
// if so then we can update the value in data2
// iterate key in data2
for key2 := range data2 {
// check whether key2 exists in data1
if val1, ok := data1[key2]; ok {
// update the value of key2 in data2
data2[key2] = val1
}
}
fmt.Printf("data2: %v", data2)
// output:
// data2: map[city:washington name:john]
// you can write the data2 into ymal
newYfile2, err := yaml.Marshal(&data2)
if err != nil {
log.Fatal(err)
return
}
// write to file
if err = ioutil.WriteFile("new_file2.yaml", newYfile2, 0644); err != nil {
log.Fatal(err)
return
}
}
Inside new_file2.yaml will be like this:
city: washington
name: john
One thing that you need to take a not is that map in Go doesn't maintain the order (AFAIK Go doesn't have built-in OrderedMap type per 7 May 2022) so the order key in the new file will be random
Additional note: for error handling, you better handle it right away (right after you got the error). Here's a good article about it Error handling and Go
There is a maps package, since 1.18 I believe. If you don't care about new keys being added to the destination map you can use its copy function.
func Copy[M ~map[K]V, K comparable, V any](dst, src M)
Copy copies all key/value pairs in src adding them to dst. When a key in src is already present in dst, the value in dst will be overwritten by the value associated with the key in src.
The source code of that function is very simple:
func Copy[M ~map[K]V, K comparable, V any](dst, src M) {
for k, v := range src {
dst[k] = v
}
}
You could also do it yourself. The below code is from the helm source code. Unlike the copy function above, it works recursivly:
func mergeMaps(a, b map[string]interface{}) map[string]interface{} {
out := make(map[string]interface{}, len(a))
for k, v := range a {
out[k] = v
}
for k, v := range b {
if v, ok := v.(map[string]interface{}); ok {
if bv, ok := out[k]; ok {
if bv, ok := bv.(map[string]interface{}); ok {
out[k] = mergeMaps(bv, v)
continue
}
}
}
out[k] = v
}
return out
}

How to cast interface{} to a map in GO

I am new to Go & I am trying to learn how to cast interface{} to a Map. Here is an example of what I am trying to implement.
Playground link : https://play.golang.org/p/3jhKlGKO46Z
Thanks for the help.
package main
import (
"fmt"
)
func main() {
Map := make(map[string]interface{})
test(Map)
for k,v := range Map {
fmt.Println("key : %v Value : %v", k, v)
}
}
func test(v interface{}) error{
data := make(map[int]interface{})
for i := 0; i < 10; i++ {
data[i] = i * 5
}
for key,val := range data {
// fmt.Println("key : %v Value : %v", key, val)
v[key] = val
}
return nil
Go supports type assertions for the interfaces. It provides the concrete value present in the interface.
You can achieve this with following code.
m, ok := v.(map[int]interface{})
if ok {
// use m
_ = m
}
If the asserted value is not of given type, ok will be false
If you avoid second return value, the program will panic for wrong assertions.
I strongly recommend you to go through https://tour.golang.org

Array seems to lose values after end of for loop

I'm a beginner at go (and not a good programmer) but I wanted to write a small program which would dump from a switch the list of mac addresses & interfaces name using snmp. I store the snmp values into an array of struct using multiple loops (the code here is to show the behavior).
During the first loop, I store Ports Vlan id & mac addresses into an array of struct (var allTableArray [30]allTable). At the end of this loop, I print the content of the array to be sure the mac addresses are in the array.
But when the second loop begins (to register bridge port number), the array seems empty (fmt.Printf("deux %x\n",allTableArray[i].macAddr) and fmt.Printf("trois %s\n",allTableArray[i].ptVlan1id)).
I don't understand why my array seems empty. Do you have any idea ?
package main
import (
"flag"
"fmt"
"os"
"time"
"strings"
"github.com/soniah/gosnmp"
"math/big"
)
type oidMacAddr struct {
oid string
macaddr string
}
type allTable struct {
ptVlan1id string
macAddr []byte
brPortNb *big.Int
ifIndex *big.Int
ifName string
}
var macAddrTable [30]oidMacAddr
func main() {
flag.Parse()
if len(flag.Args()) < 1 {
flag.Usage()
os.Exit(1)
}
target := flag.Args()[0]
showMacAddrTable(target)
}
func printValue(pdu gosnmp.SnmpPDU) error {
fmt.Printf("%s = ", pdu.Name)
//fmt.Println(reflect.TypeOf(pdu.Value.([]byte)))
switch pdu.Type {
case gosnmp.OctetString:
b := pdu.Value.([]byte)
fmt.Printf("STRING: %x\n", b)
default:
fmt.Printf("TYPE %d: %d\n", pdu.Type, gosnmp.ToBigInt(pdu.Value))
}
return nil
}
func showMacAddrTable(target string) () {
var allTableArray [30]allTable
ptVlan1Oid := ".1.3.6.1.2.1.17.4.3.1.1"
brPortOid := ".1.3.6.1.2.1.17.4.3.1.2"
brPortIfIndex := ".1.3.6.1.2.1.17.1.4.1.2"
ifIndexIfName := ".1.3.6.1.2.1.31.1.1.1.1"
community := "public"
gosnmp.Default.Target = target
gosnmp.Default.Community = community
gosnmp.Default.Timeout = time.Duration(10 * time.Second) // Timeout better suited to walking
err := gosnmp.Default.Connect()
if err != nil {
fmt.Printf("Connect err: %v\n", err)
os.Exit(1)
}
var essai []gosnmp.SnmpPDU
essai, err = gosnmp.Default.BulkWalkAll(ptVlan1Oid)
if err != nil {
fmt.Printf("Walk Error: %v\n", err)
os.Exit(1)
}
for i :=0 ; i < len(essai); i++ {
s := strings.TrimPrefix(essai[i].Name, ".1.3.6.1.2.1.17.4.3.1.1")
fmt.Printf("%s = ", s)
fmt.Printf("%x\n", essai[i].Value.([]byte))
bytes := essai[i].Value.([]byte)
macAddrTable[i] = oidMacAddr {s, string(bytes)}
allTableArray[i] = allTable {ptVlan1id: s, macAddr: bytes}
if(allTableArray[i].macAddr != nil){
fmt.Printf("%x\n",allTableArray[i].macAddr)
}
}
essai, err = gosnmp.Default.BulkWalkAll(brPortOid)
if err != nil {
fmt.Printf("Walk Error: %v\n", err)
os.Exit(1)
}
for i:=0 ; i < len(essai); i++ {
s := strings.TrimPrefix(essai[i].Name, ".1.3.6.1.2.1.17.4.3.1.2")
fmt.Printf("%s = ", s)
fmt.Printf("%d\n", essai[i].Value)
for j:=0 ; j < len(allTableArray); j++ {
if (s == allTableArray[j].ptVlan1id) {
allTableArray[j] = allTable {brPortNb: gosnmp.ToBigInt(essai[i].Value) }
}
}
fmt.Printf("deux %x\n",allTableArray[i].macAddr)
fmt.Printf("trois %s\n",allTableArray[i].ptVlan1id)
}
os.Exit(1)
}
Apparently this line
allTableArray[j] = allTable {brPortNb: gosnmp.ToBigInt(essai[i].Value) }
Update each member with a new allTable instance, where every field other than brPortNb is not defined thus becomes nil.
If what you were trying to do is to update each member's brPortNb field, you could have done so by accessing the field and assign the value to it instead of assigning a new allTable to every member.
allTableArray[j].brPortNb = gosnmp.ToBigInt(essai[i].Value)
Also, try simplifying your loops like this, provided len(essai) == len(allTableArray):
for i, v := range essai {
s := strings.TrimPrefix(v.Name, ".1.3.6.1.2.1.17.4.3.1.1")
bytes := v.Value.([]byte)
macAddrTable[i] = oidMacAddr { s, string(bytes) }
allTableArray[i] = allTable { ptVlan1id: s, macAddr: bytes }
s = strings.TrimPrefix(v.Name, ".1.3.6.1.2.1.17.4.3.1.2")
if s == allTableArray[i].ptVlan1id {
allTableArray[i].brPortNb = gosnmp.ToBigInt(v.Value)
}
}
Notice that by using for i, v := range essai syntax, you have access to both the index and the value without having to use essai[i] for the value.
Now your two loops can become just one, plus no embedded loops which are really hard to make sense of.
I Also recommend you work with slice instead of array. It's more flexible that way.

How can I output CSV with martini?

I would like to print CSV-data to the output with martini. Currently, I have always used r.JSON(200, somestruct) where r is a render.Render from github.com/martini-contrib.
Now I have an slice of structs and I would like to print them as CSV (stringify each field of a single struct and print one struct at one line).
Currently, I do it like this:
r.Data(200, []byte("id,Latitude,Longitude\n"))
for _, packet := range tour.Packets {
r.Data(200, []byte(strconv.FormatInt(packet.Id, 10)+","+strconv.FormatFloat(packet.Latitude, 'f', 6, 64)+","+strconv.FormatFloat(packet.Longitude, 'f', 6, 64)+"\n"))
}
But I don't like the way I do it for the following reasons:
It is downloaded directly and not printed to the screen.
I get http: multiple response.WriteHeader calls
I would prefer not to make this manually (the struct has much more fields, but all fields are either ìnt64, float64 or time.Time.
How can I implement the CSV export option in a simpler way?
Use the standard library. There is no general solution without reflection, but you can simplify it.
func handler(rw http.ResponseWriter) {
rw.Header().Add("Content-Type", "text/csv")
wr := csv.NewWriter(rw)
err := wr.Write([]string{"id", "Latitude", "Longitude"})
if err != nil {
...
}
for _, packet := range tour.Packets {
err := wr.Write([]string{
strconv.FormatInt(packet.Id, 10),
strconv.FormatFloat(packet.Latitude, 'f', 6, 64),
strconv.FormatFloat(packet.Longitude, 'f', 6, 64),
})
if err != nil {
...
}
}
}
If you need a general solution for any struct, it will require reflect.
See here.
// structToStringSlice takes a struct value and
// creates a string slice of all the values in that struct
func structToStringSlice(i interface{}) []string {
v := reflect.ValueOf(i)
n := v.NumField()
out := make([]string, n)
for i := 0; i < n; i++ {
field := v.Field(i)
switch field.Kind() {
case reflect.String:
out[i] = field.String()
case reflect.Int:
out[i] = strconv.FormatInt(field.Int(), 10)
// add cases here to support more field types.
}
}
return out
}
// writeToCSV prints a slice of structs as csv to a writer
func writeToCSV(w io.Writer, i interface{}) {
wr := csv.NewWriter(w)
v := reflect.ValueOf(i)
// Get slice's element type (some unknown struct type)
typ := v.Type().Elem()
numFields := typ.NumField()
fieldSet := make([]string, numFields)
for i := 0; i < numFields; i++ {
fieldSet[i] = typ.Field(i).Name
}
// Write header row
wr.Write(fieldSet)
// Write data rows
sliceLen := v.Len()
for i := 0; i < sliceLen; i++ {
wr.Write(structToStringSlice(v.Index(i).Interface()))
}
wr.Flush()
}
so then your example is just:
func handler(rw http.ResponseWriter) {
....
writeToCSV(rw, tour.Packets)
}
The function I've written will only work for int or string fields. You can easily extend this to more types by adding cases to the switch in structToStringSlice. See here for reflect docs on the other Kinds.

Resources