Match data from structs objects - go

in the following code (which I re-used from previous question) we need to match objects according to matching keys values
The keys are the value of
From model internalConfig
type
From external config
app-type
In case we have 'match' (in the value) we need to replace the data from the external config and update the internal config
e.g. if for zone2 app-type=app1 is equal to internal config type=app1 we need to change the content from external config to be
from
- name: TEST
type: app1
target: ./
to (since app1 is equal in internal and external config- we are updating the data from external to internal)
- name: TEST
type: AZURE
target: ./
given the above data the internal config should look after the match like
At the end it should look like
- name: TEST
type: AZURE
target: ./
- name: TEST3
type: HEROKU
target: ./
- name: TEST2
type: app3
target: ./
- name: TEST2
type: AZURE
target: ./
Im trying the following but I wasn’t able to match all the objects in case I’ve more then one entry to the same key
e.g. app1 “key” is in the first and the last entry, so my code is update only the first entry
- name: TEST
type: app1
target: ./
- name: TEST3
type: app2
target: ./
- name: TEST2
type: app3
target: ./
- name: TEST2
type: app1
target: ./
Any idea how could I resolve this ? this is what i've tried which works only on the first match
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"log"
)
type InternalConfig struct {
Models []Imodel `yaml:"models"`
}
type Imodel struct {
Name string `yaml:"name"`
Type string `yaml:"type"`
Target string `yaml:"target"`
}
var internalConfig = []byte(`
models:
- name: TEST
type: app1
target: ./
- name: TEST3
type: app2
target: ./
- name: TEST2
type: app3
target: ./
- name: TEST2
type: app1
target: ./
`)
type ExternalConfig struct {
Landscape Zone `yaml:"Landscape"`
}
type Zone struct {
Zone string `yaml:"zone"`
Models []Emodel `yaml:"models"`
}
type Emodel struct {
AppType string `yaml:"app-type"`
ServiceType string `yaml:"service-type"`
}
var externalConfig = []byte(`
Landscape:
zone: zone1
models:
- app-type: app1
service-type: GCP
- app-type: app2
service-type: AMAZON
zone: zone2
models:
- app-type: app1
service-type: AZURE
- app-type: app2
service-type: HEROKU
`)
func main() {
iCfgYaml := InternalConfig{}
if err := yaml.Unmarshal(internalConfig, &iCfgYaml); err != nil {
log.Fatalf("error in model InternalConfig: %v", err)
}
eCfgYaml := ExternalConfig{}
if err := yaml.Unmarshal(externalConfig, &eCfgYaml); err != nil {
log.Fatalf("error in model ExternalConfig: %v", err)
}
if err := UpdateInternalState(&iCfgYaml, eCfgYaml, "zone2"); err != nil {
log.Fatalf("Error updating internal config: %v", err)
}
fmt.Printf("DBG: %+v\n", iCfgYaml)
}
func UpdateInternalState(iCfg *InternalConfig, eCfg ExternalConfig, zone string) error {
ilm := make(map[string]int)
for i, v := range iCfg.Models {
ilm[v.Type] = i
}
for _, em := range eCfg.Landscape.Models {
if eCfg.Landscape.Zone == zone {
if i, ok := ilm[em.AppType]; ok {
iCfg.Models[i].Type = em.ServiceType
}
}
}
return nil
}
if you run the code above, this is the data you will get
{Models:[
{Name:TEST Type:app1 Target:./}
{Name:TEST3 Type:HEROKU Target:./}
{Name:TEST2 Type:app3 Target:./}
{Name:TEST2 Type:AZURE Target:./}]}
The first entry doesn't updated
https://play.golang.org/p/5f3Ad9Z7fqY
If needs more clarification please let me know

Related

How to use dapr nats io jetstream with Go

I am using dapr ,to use its pub sub configuration and using dapr go sdk to publish the messages .
With redis its working fine ,but when I am using nats.io jetstream its giving me error stream doesn't exists which ,even I am creating stream too
pubsub.yml I am using to run the dapr side car
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: order_pub_sub
namespace: default
spec:
type: pubsub.jetstream
version: v1
metadata:
- name: natsURL
value: "nats://localhost:4222"
- name: name
value: "connection name"
- name: durableName
value: "consumer durable name"
- name: queueGroupName
value: "queue group name"
- name: startSequence
value: 1
- name: startTime # in Unix format
value: 1630349391
- name: deliverAll
value: false
- name: flowControl
value: false
And this is the go code
//dependencies
import (
"context"
"log"
"math/rand"
"time"
"strconv"
dapr "github.com/dapr/go-sdk/client"
)
//code
var (
PUBSUB_NAME = "order_pub_sub"
TOPIC_NAME = "orders"
)
func main() {
for i := 0; i < 10; i++ {
time.Sleep(5000)
orderId := rand.Intn(1000-1) + 1
client, err := dapr.NewClient()
if err != nil {
panic(err)
}
defer client.Close()
ctx := context.Background()
//Using Dapr SDK to publish a topic
if err := client.PublishEvent(ctx, PUBSUB_NAME, TOPIC_NAME, []byte(strconv.Itoa(orderId)));
err != nil {
panic(err)
}
log.Println("Published data: " + strconv.Itoa(orderId))
}
}
I just want to know what will be the relevant names in nats ,like for topic and pubsub_name ,Or anyone able can help ?

Parse yaml error for map

I've the following program in which I need to parse yaml
with the following structure
https://codebeautify.org/yaml-validator/cbabd352
this is valid yaml and I use byte to make it more simple
maybe the indentation was changed during the copy paste to the question but you can see in the link that the yaml is valid
The yaml have an api_version
and runners, for each runner (key which is name) I've a list of commands
and I need to print those commands for function1 and function4 ,what am I doing wrong here ?
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
api_ver: 1
runners:
- name: function1
type:
- command: spawn child process
- command: build
- command: gulp
- name: function2
type:
- command: run function 1
- name: function3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Result struct {
Version string `yaml:"api_ver"`
Runners []Runners `yaml:"runners"`
}
type Runners struct {
Name string `yaml:"name"`
Type []Command `yaml:"type"`
}
type Command struct {
Command string `yaml:"command"`
}
func main() {
var runners []Result
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
fmt.Printf("%+v", runners[0])
}
The error which I got cannot unmarshal !!map into []main.Result
I cannot change the yaml and it should be exactly like this
https://codebeautify.org/yaml-validator/cbabd352
This is the code
https://play.golang.org/p/zidjOA6-gc7
The yaml that you have provided contains error in token. Validate the yaml used in your code here https://codebeautify.org/yaml-validator/cbaabb32
After that Create a variable of struct type result not an array. Because the yaml that you are using is creating a struct with Runners array and api_version.
This
var runners []Result
should be
var runners Result
Since because the struct is not a slice. To fetch the list of command for a name of function used in yaml. You need to loop over the runners array to find the name of function and get the value of commands for that function.
Below is the working code:
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v2"
)
var runContent = []byte(`
api_ver: 1
runners:
- name: function1
type:
- command: spawn child process
- command: build
- command: gulp
- name: function2
type:
- command: run function 1
- name: function3
type:
- command: ruby build
- name: function4
type:
- command: go build
`)
type Result struct {
Version string `yaml:"api_ver"`
Runners []Runners `yaml:"runners"`
}
type Runners struct {
Name string `yaml:"name"`
Type []Command `yaml:"type"`
}
type Command struct {
Command string `yaml:"command"`
}
func main() {
var runners Result
// parse mta yaml
err := yaml.Unmarshal(runContent, &runners)
if err != nil {
log.Fatalf("Error : %v", err)
}
commandList := getCommandList("function1", runners.Runners)
fmt.Printf("%+v", commandList)
}
func getCommandList(name string, runners []Runners) []Command {
var commandList []Command
for _, value := range runners {
if value.Name == name {
commandList = value.Type
}
}
return commandList
}
Playground example

Parse yaml return empty object

I have the following yaml which I need to parse to struct.
In the builds property I got empty value while debug, what am I missing here?
I use "gopkg.in/yaml.v2"
- name: srv
type: java
path: srv
builds:
- name: db
properties:
JBR_CONFIG_RESOURCE_CONFIG: '[META-INF/context.xml:
{"service_name" : "~{h-container}"}]'
TEST2: aaaa
The struct is
type Runs struct {
Name string
Type string
Path string `yaml:"path,omitempty"`
Builds []Builds `yaml:”builds,omitempty"`
}
type Builds struct {
Name string `yaml:"name,omitempty"`
Properties Properties `yaml:"properties,omitempty"`
}
type Properties map[string]string
Properly formated yaml is the first thing that you should consider.
If u wanna have one Runs you should have your yaml formated something like that
name: srv
builds:
-
name: db
properties:
JBR_CONFIG_RESOURCE_CONFIG: "[META-INF/context.xml:
{\"service_name\" : \"~{h-container}\"}]"
TEST2: aaaa
path: srv
type: java
But then i u wanna have more of this object you need to group them in one parameter. It can look like this
runs:
-
name: srv
builds:
-
name: db
properties:
JBR_CONFIG_RESOURCE_CONFIG: "[META-INF/context.xml:
{\"service_name\" : \"~{h-container}\"}]"
TEST2: aaaa
path: srv
type: java
-
name: srv2
builds:
-
name: db2
properties:
JBR_CONFIG_RESOURCE_CONFIG: "[META-INF/context.xml:
{\"service_name\" : \"~{h-container}\"}]"
TEST2: aaaa2
path: srv2
type: java2
And then in your code could look like this
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"io/ioutil"
"log"
"os"
)
type Document struct {
Runs []Runs `yaml:"runs,omitempty"`
}
type Runs struct {
Name string `yaml:"name,omitempty"`
Type string `yaml:"type,omitempty"`
Path string `yaml:"path,omitempty"`
Builds []Builds `yaml:"builds,omitempty"`
}
type Builds struct {
Name string `yaml:"name,omitempty"`
Properties map[string]string `yaml:"properties,omitempty"`
}
func main() {
var document Document
reader, err := os.Open("demo.yml")
if err != nil {
log.Fatal(err)
}
buf, _ := ioutil.ReadAll(reader)
yaml.Unmarshal(buf, &document)
if err := yaml.Unmarshal(buf, &document); err != nil {
fmt.Print(err)
os.Exit(1)
}
fmt.Println(document)
}
Make sure your yaml file is formatted correctly. Check with this tool.
The following piece of code worked fine.
package main
import (
"fmt"
"gopkg.in/yaml.v2"
"log"
)
type Runs struct {
Name string
Type string
Path string `yaml:"path,omitempty"`
Builds []Builds `yaml:”builds,omitempty"`
}
type Builds struct {
Name string `yaml:"name,omitempty"`
Properties Properties `yaml:"properties,omitempty"`
}
type Properties map[string]string
func main() {
data := `builds:
-
name: db
properties:
JBR_CONFIG_RESOURCE_CONFIG: "[META-INF/context.xml: {\"service_name\" : \"~{h-container}\"}]"
TEST2: aaaa
name: srv
path: srv
type: java
`
runs := Runs{}
err := yaml.Unmarshal([]byte(data), &runs)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t:\n%v\n\n", runs)
d, err := yaml.Marshal(&runs)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t dump:\n%s\n\n", string(d))
}
I hope this helps!

How to build repetitive struct

I have a program which parses yamls file to an objects (structs).
I use the following repo to do it
https://github.com/go-yaml/yaml
for example in the file I have:
dependency :
- name: ui
type: runner
cwd: /ui
install:
- name: api
group: test
And I use the following struct for it
type Dependency struct {
Name string
Type string
CWD string
Install []Install
//here I have the issue
Requires ?
}
type Install struct {
Name string
Group string
}
Now I have two few issue with a bit complex struct.
This is the entry which could be inside the Dependency struct and this is how it look in the yaml file
requires:
- name: db
- type: mongo
but it also can be
requires:
- name: db
- name: rst
- name: test
- name: test2
Since it have multiple name properties how should I got build this struct
In addition I've field in the yaml
_type-version: "1.0.0"
when I put it inside struct like following I got error since I use -
type TypeVer struct{
_Type-version string
}
How to overcome this?
The yaml package actually allows you to remap the the name for the properties, you can use this to handle your _type-version Property.
And your initial question: Just define Requires the same as Install:
package main
import (
"fmt"
"log"
"github.com/go-yaml/yaml"
)
type File struct {
TypeVersion string `yaml:"_type-version"`
Dependency []Dependency
}
type Dependency struct {
Name string
Type string
CWD string
Install []Install
Requires []Requires
}
type Install struct {
Name string
Group string
}
type Requires struct {
Name string
Type string
}
var data = `
_type-version: "1.0.0"
dependency:
- name: ui
type: runner
cwd: /ui
install:
- name: api
group: test
requires:
- name: db
- type: mongo
- name: rst
- name: test
- name: test2
`
func main() {
f := File{}
err := yaml.Unmarshal([]byte(data), &f)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t:\n%v\n\n", f)
d, err := yaml.Marshal(&f)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t dump:\n%s\n\n", string(d))
}

Only return specific fields from struct

I have a pair of structs like so:
type Datacenter struct {
Name string
Key string
Hosts []Host
}
type Host struct {
Name string
Port int
}
And then an example configuration file:
datacenters:
- name: dc1
key: test
hosts:
- name: dc1-host
port: 8200
- name: dc2
key: dc-test
hosts:
- name: dc2-host
port: 8200
I'm using viper to read the configuration file, here's the function:
func getDatacenters() []config.Datacenter {
err := viper.UnmarshalKey("datacenters", &datacenters)
if err != nil {
log.Error("Unable to read hosts key in config file: %s", err)
}
return datacenters
}
What I'd like to be able do now is specify an optional parameter, datacenter and if that's specified, to only return the keys from that datacenter. If the parameter is not specified, I'd like it to unmarshal and return the whole thing.
Is this possible?
EDIT: I should add, all I do with this so far is range over them:
for _, d := range datacenters {
for _, h := range d.Hosts {
}
}
So it may be there's a better way.

Resources