Get JSON paths to value from JSON - go

I'm trying to write a func to get all paths to values from a yaml file and I don't know know how that possible, here is my code:
func getpath(fileyaml) string {
if _, err := os.Stat(fileyaml); err == nil {
bfile, err := ioutil.ReadFile(fileyaml)
bjson, err := yaml.YAMLToJSON(bfile)
if err != nil {
fmt.Printf("YAMLToJSON err: %v\n", err)
}
json := string(bjson)
println json
paths := ? // don't know
return path
here my yaml file :
sentinel:
number: 3
server:
number: 7
config:
fere_size: 5
lcmea:
eza_ze: all
my function will convert it to a json: {"config":{"fere_size":5},"lcmea":{"eza_hooks":"all"},"sentinel":{"number":3},"server":{"number":7}}
the output that i want :
sentinel.number=3, server.number=3,config.fere_size=5,lcmea.eza_ze=all
how to parse this json in order to get this desired output?
I'm using "github.com/tidwall/gjson" to read the yaml and convert it to json

This is an example of how you could do it: https://play.golang.org/p/7yLq_PDLdXF
It is pretty naive and definitely could be improved, but it may give you an idea about how to parse the yaml file and then print the output in your desired format:
package main
import (
"fmt"
"log"
"strings"
"github.com/go-yaml/yaml"
)
var data = `
sentinel:
number: 3
server:
number: 7
config:
fere_size: 5
lcmea:
eza_ze: all
`
func main() {
m := make(map[string]map[string]interface{})
err := yaml.Unmarshal([]byte(data), &m)
if err != nil {
log.Fatalf("error: %v", err)
}
out := []string{}
for k, v := range m {
for j, i := range v {
out = append(out, fmt.Sprintf("%s.%v=%v", k, j, i))
}
}
fmt.Println(strings.Join(out, ", "))
}
It will return:
$ go run main.go
sentinel.number=3, server.number=7, config.fere_size=5, lcmea.eza_ze=all
It is using https://github.com/go-yaml/yaml, check more examples on the README.md

Related

Go: How to read in MRT (.bz2) file as byte and parse data

I am trying to read in an mrt (with .bz2 file extension) from archive.routeviews.org namely file - http://archive.routeviews.org/route-views.chile/bgpdata/2022.05/UPDATES/updates.20220501.0000.bz2.
I have found some code online that parses it using three different packages - FGBGP, go-mrt, goBGP. Here is the code:
package main
import (
"bufio"
"bytes"
"fmt"
"log"
"os"
mrt1 "github.com/cloudflare/fgbgp/mrt"
mrt2 "github.com/kaorimatz/go-mrt"
mrt3 "github.com/osrg/gobgp/pkg/packet/mrt"
)
func main() {
data, err := os.ReadFile("updates.20220501.0000")
if err != nil {
log.Fatal(err)
}
// or paste bytes instead data := []byte{}
fmt.Println("FGBGP")
rdr := bytes.NewBuffer(data)
r, err := mrt1.DecodeSingle(rdr)
for r != nil && err == nil {
fmt.Println(r)
r, err = mrt1.DecodeSingle(rdr)
}
fmt.Println("Go-mrt")
rdr2 := mrt2.NewReader(bytes.NewBuffer(data))
r2, err := rdr2.Next()
for r2 != nil && err == nil {
fmt.Println(r2)
r2, err = rdr2.Next()
}
fmt.Println("GoBGP")
sc := bufio.NewScanner(bytes.NewBuffer(data))
sc.Split(mrt3.SplitMrt)
for {
b := sc.Scan()
if !b {
break
}
mrtb := sc.Bytes()
hdr, err := mrt3.NewMRTHeader(0, mrt3.BGP4MP, mrt3.RIB_IPV4_UNICAST, 0)
if err != nil {
fmt.Println(err)
break
}
hdr.DecodeFromBytes(mrtb)
r3, err := mrt3.ParseMRTBody(hdr, mrtb[mrt3.MRT_COMMON_HEADER_LEN:])
if err != nil {
fmt.Println(err)
}
fmt.Println(r3)
}
}
When this is run, FGBGP does not input anything, go-mrt does seem to output the lines but each line seems to be missing things I would like to see such as AS PATH etc. And goBGP which was the one I was most interested in, throws an unsupported type error.
Here is an example output for go-mrt and goBGP:
go-mrt: &{{{0 63786960781 <nil>} 17 4} 27678 6447 0 1 200.16.114.34 200.16.114.60 0x140003f5900}
goBGP: <nil> unsupported type: 17

Golang comparing two yaml files and updating them

Im a newbie in golang. I am trying to compare two yaml files and update the 2nd file's value if there is any new value in 1st yaml for that particular key.
So the files are of format: These are sample yaml files. Real yaml files have much more nested complicated maps with different datatypes for each key.
1st yaml:
name: john
city: washington
2nd yaml:
name: peter
city: washington
Final result for 2nd yaml file should be:
name: john
city: washington
Tried creating a map string interface for both yaml files using unmarshal. But having trouble how to compare both maps. Was trying to loop over each key of map and search for that key in 2nd yaml map. If key exists update the value in 2nd yaml map. But i am not able to implement that. Any suggestions/better ideas?
Edit: Updated code
package main
import (
"fmt"
"io/ioutil"
"log"
"github.com/imdario/mergo"
"gopkg.in/yaml.v3"
)
func main() {
yfile, err := ioutil.ReadFile("C:/Users/212764682/lifecycle/userconfig.yaml")
if err != nil {
log.Fatal(err)
}
data := make(map[string]interface{})
err2 := yaml.Unmarshal(yfile, &data)
if err2 != nil {
log.Fatal(err2)
} else {
yfile1, err3 := ioutil.ReadFile("C:/Users/212764682/lifecycle/conf.yaml")
yfile2, err4 := ioutil.ReadFile("C:/Users/212764682/lifecycle/prof.yaml")
if err3 != nil && err4 != nil {
log.Fatal(err3)
log.Fatal(err4)
} else {
dat := make(map[string]interface{})
dat2 := make(map[string]interface{})
err5 := yaml.Unmarshal(yfile1, &dat)
err6 := yaml.Unmarshal(yfile2, &dat2)
_ = err5
_ = err6
for key1, element1 := range data {
for key2, element2 := range dat {
if key1 == key2 {
if element1 == element2 {
} else {
element2 = element1
}
} else {
dat[key1] = data[key1]
}
}
}
}
}
}
So im want to compare each key of data with dat. If that key exists in dat, check for value in data. If value different in dat, update with value of data in dat for that key. Also, if any key of data dosent exist in dat, then append that key in dat. But not able to implement it correctly.
You can try to compare the map and then update it if the key exists. Here's some example using your case.
package main
import (
"fmt"
"io/ioutil"
"log"
"gopkg.in/yaml.v3"
)
func main() {
// read file
yfile1, err := ioutil.ReadFile("file1.yaml")
if err != nil {
log.Fatal(err)
return
}
yfile2, err := ioutil.ReadFile("file2.yaml")
if err != nil {
log.Fatal(err)
return
}
// unmarshal ymal
data1 := make(map[string]interface{})
if err := yaml.Unmarshal(yfile1, &data1); err != nil {
log.Fatal(err)
return
}
data2 := make(map[string]interface{})
if err := yaml.Unmarshal(yfile2, &data2); err != nil {
log.Fatal(err)
return
}
// from this we can iterate the key in data2 then check whether it exists in data1
// if so then we can update the value in data2
// iterate key in data2
for key2 := range data2 {
// check whether key2 exists in data1
if val1, ok := data1[key2]; ok {
// update the value of key2 in data2
data2[key2] = val1
}
}
fmt.Printf("data2: %v", data2)
// output:
// data2: map[city:washington name:john]
// you can write the data2 into ymal
newYfile2, err := yaml.Marshal(&data2)
if err != nil {
log.Fatal(err)
return
}
// write to file
if err = ioutil.WriteFile("new_file2.yaml", newYfile2, 0644); err != nil {
log.Fatal(err)
return
}
}
Inside new_file2.yaml will be like this:
city: washington
name: john
One thing that you need to take a not is that map in Go doesn't maintain the order (AFAIK Go doesn't have built-in OrderedMap type per 7 May 2022) so the order key in the new file will be random
Additional note: for error handling, you better handle it right away (right after you got the error). Here's a good article about it Error handling and Go
There is a maps package, since 1.18 I believe. If you don't care about new keys being added to the destination map you can use its copy function.
func Copy[M ~map[K]V, K comparable, V any](dst, src M)
Copy copies all key/value pairs in src adding them to dst. When a key in src is already present in dst, the value in dst will be overwritten by the value associated with the key in src.
The source code of that function is very simple:
func Copy[M ~map[K]V, K comparable, V any](dst, src M) {
for k, v := range src {
dst[k] = v
}
}
You could also do it yourself. The below code is from the helm source code. Unlike the copy function above, it works recursivly:
func mergeMaps(a, b map[string]interface{}) map[string]interface{} {
out := make(map[string]interface{}, len(a))
for k, v := range a {
out[k] = v
}
for k, v := range b {
if v, ok := v.(map[string]interface{}); ok {
if bv, ok := out[k]; ok {
if bv, ok := bv.(map[string]interface{}); ok {
out[k] = mergeMaps(bv, v)
continue
}
}
}
out[k] = v
}
return out
}

Read Exif Metadata with Go

Does anyone have an example on using "github.com/gohugoio/hugo/resources/images/exif" to extract metadata from a local image using Go?
I looked through the docs and since I'm new to Go I'm not 100% sure if I'm doing things write. I do read the image, but I'm not sure what the next step would be.
fname := "image.jpg"
f, err := os.Open(fname)
if err != nil {
log.Fatal("Error: ", err)
}
(Edit 1)
Actually I think I found a solution:
d, err := exif.NewDecoder(exif.IncludeFields("File Type"))
x, err := d.Decode(f)
if err != nil {
log.Fatal("Error: ", err)
}
fmt.Println(x)
however, the question is how do I know what fields are available? File Type for example returns <nil>
Looks like Hugo uses github.com/rwcarlsen/goexif.
The documentation of the package on go.dev shows Exif.Walk can walk the name and tag for every non-nil EXIF field.
Eg, a small program:
package main
import (
"fmt"
"log"
"os"
"github.com/rwcarlsen/goexif/exif"
"github.com/rwcarlsen/goexif/tiff"
)
type Printer struct{}
func (p Printer) Walk(name exif.FieldName, tag *tiff.Tag) error {
fmt.Printf("%40s: %s\n", name, tag)
return nil
}
func main() {
if len(os.Args) < 2 {
log.Fatal("please give filename as argument")
}
fname := os.Args[1]
f, err := os.Open(fname)
if err != nil {
log.Fatal(err)
}
x, err := exif.Decode(f)
if err != nil {
log.Fatal(err)
}
var p Printer
x.Walk(p)
}
Example:
$ go run main.go IMG_123.JPG
ResolutionUnit: 2
YCbCrPositioning: 2
Make: "Canon"
Model: "Canon IXUS 255 HS"
ThumbJPEGInterchangeFormat: 5620
PixelYDimension: 3000
FocalPlaneResolutionUnit: 2
GPSVersionID: [2,3,0,0]
ExifVersion: "0230"
WhiteBalance: 1
DateTime: "2016:10:04 17:27:56"
CompressedBitsPerPixel: "5/1"
... etc ...
Orientation: 1
MeteringMode: 5
FocalLength: "4300/1000"
PixelXDimension: 4000
InteroperabilityIFDPointer: 4982
FocalPlaneXResolution: "4000000/244"
XResolution: "180/1"
ComponentsConfiguration: ""
ShutterSpeedValue: "96/32"
ApertureValue: "101/32"
ExposureBiasValue: "-1/3"
FocalPlaneYResolution: "3000000/183"
SceneCaptureType: 0

Checking if string from golang file exists in yaml file

I am rather new to working with yaml and golang. Currently, I am creating a golang program that parses an rpm package to check for subsystem dependencies. It extends the go-rpmutils library.
So far this is the code I have within my main function to handle conditions:
func main() {
// Parse the rpm
rpm, err := rpmutils.ReadRpm("file.rpm")
if err != nil {
panic(err)
}
// Get RPM Deps
dependencies, err := rpm.Header.GetStrings(rpmutils.REQUIRENAME)
if err != nil {
panic(err)
}
// Check for specific dep condition
for _, p := range dependencies {
if strings.HasPrefix(p, "prefix1") && p != "string-including-prefix1" {
fmt.Printf("\t%s\n", p)
defer os.Exit(1)
}
}
}
I am able to output the dependencies but want to set up several if else conditions for when specific subsystem dependencies exist.
In a separate yaml file, I have:
allowed-deps:
-dep1
-dep2
-dep3
third-party-deps:
-dep4
-dep5
-dep6
internal-deps:
-dep7
-dep8
-dep9
I'd like to compare the value of var p from the for loop with the values in the yaml file. So for example:
if p only equals values from allowed-deps, print "successfully built rpm" and do not prompt os.Exit(1)
if p equals any of the third-party-deps, print "err msg for third-party deps" and os.Exit(1)
if p equals any internal-deps, print "another err mssg" and os.Exit(1)
How can I go about doing this?
You can use a YAML package (like https://github.com/go-yaml/yaml), load your file into a variable and check it on every step in the ifs that you propose. I would use maps as it seems that you will be checking very frequently the sets.
Here you have a simple example that I made using that package so you can see how to unmarshal your file, convert into maps, and check the maps: https://play.golang.org/p/t1GhUPvAQNQ
package main
import (
"fmt"
"github.com/go-yaml/yaml"
)
const str = `
allowed-deps:
- dep1
- dep2
- dep3
third-party-deps:
- dep4
- dep5
- dep6
internal-deps:
- dep7
- dep8
- dep9
`
type MyYAML struct {
AllowedDeps []string `yaml:"allowed-deps"`
ThirdPartyDeps []string `yaml:"third-party-deps"`
InternalDeps []string `yaml:"internal-deps"`
}
func main() {
var a MyYAML
err := yaml.Unmarshal([]byte(str), &a)
if err != nil {
panic(err)
}
// Build a map for every section.
allowedDeps := map[string]struct{}{}
thirdPartyDeps := map[string]struct{}{}
internalDeps := map[string]struct{}{}
for _, dep := range a.AllowedDeps {
allowedDeps[dep] = struct{}{}
}
for _, dep := range a.ThirdPartyDeps {
thirdPartyDeps[dep] = struct{}{}
}
for _, dep := range a.InternalDeps {
internalDeps[dep] = struct{}{}
}
// Some checking examples.
if _, ok := allowedDeps["dep1"]; ok {
fmt.Println("dep1 found")
}
if _, ok := thirdPartyDeps["dep1"]; ok {
fmt.Println("dep1 found")
}
if _, ok := internalDeps["dep8"]; ok {
fmt.Println("dep8 found")
}
}

How can I write a string to a binary file?

such as, I write 'A' but in file it is '1000001' ,
how can I do ?
I have tried
buf := new(bytes.Buffer)
data := []int8{65, 80}
for _, i := range data {
binary.Write(buf, binary.LittleEndian, i)
fp.Write(buf.Bytes())
}
but I got string 'AP' in file not a binary code
I didn't really understand the question, but perhaps you want something like:
package main
import (
"fmt"
"log"
"os"
)
func main() {
f, err := os.OpenFile("out.txt", os.O_TRUNC|os.O_CREATE|os.O_WRONLY, 0600)
if err != nil {
log.Fatal(err)
}
for _, v := range "AP" {
fmt.Fprintf(f, "%b\n", v)
}
f.Close()
}
which gives:
$ cat out.txt
1000001
1010000

Resources