Create function that require other repeatbale logic as pre-requiste (clean code) - go

I've the following yaml file which I need to parse (the parse is working as expected) and need to provide a datafrom the yaml file content that should be exposed by the following decoupled functions
I need to provide the following functions (here is example of some of those functions, need more with the same pattern...)
getApps()
getServices()
GetApp(appname)
GetServiceForApp(appname)
This is the code (which works...)
var DMZ = []byte(`
applications:
- name: app1
type: php
src: /app1
host: us
use:
- redis
- mysql
- name: app2
type: rust
src: /app2
host: eu
use:
- mongo
- mysql
- name: app3
type: golang
src: /app3
host: us
use:
- postgress
- mysql
services:
- name: mongo
type: db
host: us
- name: mysql
type: db
host: eu
- name: postgress
type: db
host: us
- name: redis
type: db
host: us
`)
This is the structs
type DMZ struct {
Applications []*Applications `yaml:"applications,omitempty"`
Services []*Services `yaml:"services,omitempty"`
}
type Applications struct {
Name string
Type string
Src string `yaml:"src,omitempty"`
use []Use `yaml:"use,omitempty"`
}
type Services struct {
Name string
Type string
Host string `yaml:"host,omitempty"`
}
type Use struct {
Name string `yaml:"name,omitempty"`
host string `yaml:"host,omitempty"`
Type string `yaml:"type,omitempty"`
}
// Parse file
func Parse(yamlContent []byte) (out DMZ, err error) {
dmz := DMZ{}
err = yaml.Unmarshal([]byte(yamlContent), &dmz)
if err != nil {
logs.Error("Yaml file is not valid, Error: " + err.Error())
}
return dmz, err
}
As the Parse function is a per-requite to all the needed functions (which I listed above) I wonder how is the best to create them ,create simple function that every time call to the parse function and then do the logic (not a problem) but I wonder if there is better approach which follows the clean code principles for Golang , with 'interface / dependency injections' ?
UPDATE:
I want to avoid doing things like following, assume that I need to call to those function from different packages or even different GitHub repository how it's best to do it with Golang clean code.
func getApps(){
dmz := Parse()
....
}
func getServices(){
dmz := Parse()
....
}
func getApp(appname string){
dmz := Parse()
....
}
func GetServiceForApp(appname string){
dmz := Parse()
....
}
And I need more functions with the same pattern ...
What I need some Clean Code solution using interface/dependency injection like a best practice code example in Golang
If something is not clear please let me know :)

Parse the value inside pointer type struct value which will store the value inside the struct pointed by a pointer. Then Create method receives on All the methods from which you want to get the value of an app or service inside the struct.
Using pointer receiver in all methods you will be able to access the original struct which is updated when parsing the yaml.
Create a pointer to empty instance of the struct and pass it as a method receiver. Then access that struct in unmarshal to update yaml data in original struct. Access the updated struct in every function by accessing the value of original struct using pointer receiver to method.
package main
import (
"fmt"
"log"
yaml "gopkg.in/yaml.v2"
)
var dmz = []byte(`
applications:
- name: app1
type: php
src: /app1
host: us
use:
- redis
- mysql
- name: app2
type: rust
src: /app2
host: eu
use:
- mongo
- mysql
- name: app3
type: golang
src: /app3
host: us
use:
- postgress
- mysql
services:
- name: mongo
type: db
host: us
- name: mysql
type: db
host: eu
- name: postgress
type: db
host: us
- name: redis
type: db
host: us
`)
type DMZ struct {
Applications []*Applications `yaml:"applications,omitempty"`
Services []*Services `yaml:"services,omitempty"`
}
type Applications struct {
Name string
Type string
Src string `yaml:"src,omitempty"`
use []Use `yaml:"use,omitempty"`
}
type Services struct {
Name string
Type string
Host string `yaml:"host,omitempty"`
}
type Use struct {
Name string `yaml:"name,omitempty"`
host string `yaml:"host,omitempty"`
Type string `yaml:"type,omitempty"`
}
func main() {
dm := &DMZ{}
result, err := dm.Parse(dmz)
dm.getApp("app1")
if err != nil {
fmt.Println(err)
}
fmt.Println(result)
fmt.Println(dm.getApp("app2"))
fmt.Println(dm.GetServiceForApp("mongo"))
}
func (dmz *DMZ) getApps() []*Applications {
return dmz.Applications
}
func (dmz *DMZ) getServices() []*Services {
return dmz.Services
}
func (dmz *DMZ) getApp(appname string) *Applications {
for _, value := range dmz.Applications {
if appname == value.Name {
fmt.Println((*value).Name)
return value
}
}
return nil
}
func (dmz *DMZ) GetServiceForApp(appname string) *Services {
for _, value := range dmz.Services {
if appname == value.Name {
return value
}
}
return nil
}
// Parse file
func (dmz *DMZ) Parse(yamlContent []byte) (out *DMZ, err error) {
err = yaml.Unmarshal([]byte(yamlContent), &dmz)
if err != nil {
log.Fatal("Yaml file is not valid, Error: " + err.Error())
}
return dmz, err
}
Working code on Playground
If you want your code to be more clean then you can also skip the struct returned from parse function. Since we are passing a pointer type receiver and updating the original struct as:
package main
import (
"fmt"
"log"
yaml "gopkg.in/yaml.v2"
)
var dmz = []byte(`
applications:
- name: app1
type: php
src: /app1
host: us
use:
- redis
- mysql
- name: app2
type: rust
src: /app2
host: eu
use:
- mongo
- mysql
- name: app3
type: golang
src: /app3
host: us
use:
- postgress
- mysql
services:
- name: mongo
type: db
host: us
- name: mysql
type: db
host: eu
- name: postgress
type: db
host: us
- name: redis
type: db
host: us
`)
type DMZ struct {
Applications []*Applications `yaml:"applications,omitempty"`
Services []*Services `yaml:"services,omitempty"`
}
type Applications struct {
Name string
Type string
Src string `yaml:"src,omitempty"`
use []Use `yaml:"use,omitempty"`
}
type Services struct {
Name string
Type string
Host string `yaml:"host,omitempty"`
}
type Use struct {
Name string `yaml:"name,omitempty"`
host string `yaml:"host,omitempty"`
Type string `yaml:"type,omitempty"`
}
func main() {
dm := &DMZ{}
dm.Parse(dmz)
fmt.Println(dm.getApp("app2"))
fmt.Println(dm.GetServiceForApp("mongo"))
}
func (dmz *DMZ) getApps() []*Applications {
return dmz.Applications
}
func (dmz *DMZ) getServices() []*Services {
return dmz.Services
}
func (dmz *DMZ) getApp(appname string) *Applications {
for _, value := range dmz.Applications {
if appname == value.Name {
fmt.Println((*value).Name)
return value
}
}
return nil
}
func (dmz *DMZ) GetServiceForApp(appname string) *Services {
for _, value := range dmz.Services {
if appname == value.Name {
return value
}
}
return nil
}
// Parse file
func (dmz *DMZ) Parse(yamlContent []byte) {
if err := yaml.Unmarshal([]byte(yamlContent), &dmz); err != nil {
log.Fatal("Yaml file is not valid, Error: " + err.Error())
}
}
As we can notice that Parse function will become more clean since we are not returning anything from it we are just updating the original struct using method receiver which is a much better way to achieve what you have been trying to achieve.
You can also choose to implement an interface by defining the methods in struct as receiver for which you are trying to implement the interface as:
package main
import (
"fmt"
"log"
yaml "gopkg.in/yaml.v2"
)
type DI interface {
GetApps() []*Applications
GetServices() *Services
}
var dmz = []byte(`
applications:
- name: app1
type: php
src: /app1
host: us
use:
- redis
- mysql
- name: app2
type: rust
src: /app2
host: eu
use:
- mongo
- mysql
- name: app3
type: golang
src: /app3
host: us
use:
- postgress
- mysql
services:
- name: mongo
type: db
host: us
- name: mysql
type: db
host: eu
- name: postgress
type: db
host: us
- name: redis
type: db
host: us
`)
type DMZ struct {
Applications []*Applications `yaml:"applications,omitempty"`
Services []*Services `yaml:"services,omitempty"`
}
type Applications struct {
Name string
Type string
Src string `yaml:"src,omitempty"`
use []Use `yaml:"use,omitempty"`
}
type Services struct {
Name string
Type string
Host string `yaml:"host,omitempty"`
}
type Use struct {
Name string `yaml:"name,omitempty"`
host string `yaml:"host,omitempty"`
Type string `yaml:"type,omitempty"`
}
func main() {
dm := &DMZ{}
dm.Parse(dmz)
fmt.Println(dm.getApp("app2"))
fmt.Println(dm.GetServiceForApp("mongo"))
}
func (dmz *DMZ) GetApps() []*Applications {
return dmz.Applications
}
func (dmz *DMZ) GetServices() []*Services {
return dmz.Services
}
func (dmz *DMZ) getApp(appname string) *Applications {
for _, value := range dmz.Applications {
if appname == value.Name {
fmt.Println((*value).Name)
return value
}
}
return nil
}
func (dmz *DMZ) GetServiceForApp(appname string) *Services {
for _, value := range dmz.Services {
if appname == value.Name {
return value
}
}
return nil
}
// Parse file
func (dmz *DMZ) Parse(yamlContent []byte) {
if err := yaml.Unmarshal([]byte(yamlContent), &dmz); err != nil {
log.Fatal("Yaml file is not valid, Error: " + err.Error())
}
}
Note:
Generics are convenient but they come at a cost in complexity in the
type system and run-time. We haven't yet found a design that gives
value proportionate to the complexity, although we continue to think
about it. Meanwhile, Go's built-in maps and slices, plus the ability
to use the empty interface to construct containers (with explicit
unboxing) mean in many cases it is possible to write code that does
what generics would enable, if less smoothly.

You can define an interface and provide implementations in the struct
type DMZI interface {
GetApps() []Application
GetService() []Service
GetApp(name string) (Application, error)
GetServiceForApp(name string) ([]string, error)
}
type DMZ struct {
Application []Application `yaml:"applications,omitempty"`
Service []Service `yaml:"services,omitempty"`
}
func (dmz DMZ) GetApps() []Application {
return dmz.Application
}
func (dmz DMZ) GetService() []Service {
return dmz.Service
}
func (dmz DMZ) GetApp(name string) (Application, error) {
for _, app := range dmz.Application {
if app.Name == name {
return app, nil
}
}
return Application{}, fmt.Errorf("Did not find application with name %s", name)
}
func (dmz DMZ) GetServiceForApp(name string) ([]string, error) {
app, err := dmz.GetApp(name)
if err != nil {
return []string{}, err
}
return app.Use, nil
}
type Application struct {
Name string
Type string
Src string `yaml:"src,omitempty"`
Use []string `yaml:"use,omitempty"`
}
type Service struct {
Name string
Type string
Host string `yaml:"host,omitempty"`
}
// Parse file
func Parse(yamlContent []byte) (out DMZI, err error) {
dmz := DMZ{}
err = yaml.Unmarshal([]byte(yamlContent), &dmz)
if err != nil {
fmt.Println("Yaml file is not valid, Error: " + err.Error())
}
return dmz, err
}
So, you can call the methods on the returned interface e.g.
fmt.Printf("Apps : %+v\n", dmz.GetApps())
fmt.Printf("Service : %+v\n", dmz.GetService())
UPDATE
main method as requested in the comment
func main() {
dmz, err := Parse([]byte(ymlStr))
if err != nil {
panic(err)
}
fmt.Printf("Apps : %+v\n", dmz.GetApps())
fmt.Printf("Service : %+v\n", dmz.GetService())
}
Will print
Apps : [{Name:app1 Type:php Src:/app1 Use:[redis mysql]} {Name:app2 Type:rust Src:/app2 Use:[mongo mysql]} {Name:app3 Type:golang Src:/app3 Use:[postgress mysql]}]
Service : [{Name:mongo Type:db Host:us} {Name:mysql Type:db Host:eu} {Name:postgress Type:db Host:us} {Name:redis Type:db Host:us}]

I've changed your code a little and done what I think you are asking for, which is to always call "Parse" whenever you create a new instance of DMZ. This is how you would do this using Dargo. Look at the bottom to see how the code would bind DMZ into a ServiceLocator and how to get new instances of DMZ
type DMZ struct {
Applications []*Applications `yaml:"applications,omitempty"`
Services []*Services `yaml:"services,omitempty"`
}
// This method is called every time DMZ is created via the Dargo API
func (dmz *DMZ) DargoInitialize(ioc.Descriptor) error {
// get ur bytes from... wherever
Parse(dmz, []byte{})
}
type Applications struct {
Name string
Type string
Src string `yaml:"src,omitempty"`
use []Use `yaml:"use,omitempty"`
}
type Services struct {
Name string
Type string
Host string `yaml:"host,omitempty"`
}
type Use struct {
Name string `yaml:"name,omitempty"`
host string `yaml:"host,omitempty"`
Type string `yaml:"type,omitempty"`
}
// Parse file
func Parse(dmz *DMZ, yamlContent []byte) (out *DMZ, err error) {
err = yaml.Unmarshal([]byte(yamlContent), &dmz)
if err != nil {
logs.Error("Yaml file is not valid, Error: " + err.Error())
}
return dmz, err
}
func useDargo() error {
locator, err := ioc.CreateAndBind("example", func(binder ioc.Binder) error {
binder.Bind("DMZ", &DMZ{}).InScope(ioc.PerLookup)
})
if err != nil {
return err
}
// This is how you would get your instances of DMZ
locator.GetDService("DMZ")
}

Related

Unmarshal yaml map dict key to struct property

I really searched a while here, but didn't found an adequate answer:
I am trying to unmarshall yaml dict keys onto a property of a struct rather than the key of a map.
Given this yaml
commands:
php:
service: php
bin: /bin/php
node:
service: node
bin: /bin/node
I am able to unmarshall this into a struct like this:
type Config struct {
Commands map[string]struct {
Service string
Bin string
}
}
But how am I able to unmarshall it into a struct like this:
type Config struct {
Commands []struct {
Name string // <-- this should be key from the yaml (i.e. php or node)
Service string
Bin string
}
}
Thx in advance for the help
You can write a custom unmarshaler, like this (on Go playground):
package main
import (
"fmt"
"gopkg.in/yaml.v3"
)
var input []byte = []byte(`
commands:
php:
service: php
bin: /bin/php
node:
service: node
bin: /bin/node
`)
type Command struct {
Service string
Bin string
}
type NamedCommand struct {
Command
Name string
}
type NamedCommands []NamedCommand
type Config struct {
Commands NamedCommands
}
func (p *NamedCommands) UnmarshalYAML(value *yaml.Node) error {
if value.Kind != yaml.MappingNode {
return fmt.Errorf("`commands` must contain YAML mapping, has %v", value.Kind)
}
*p = make([]NamedCommand, len(value.Content)/2)
for i := 0; i < len(value.Content); i += 2 {
var res = &(*p)[i/2]
if err := value.Content[i].Decode(&res.Name); err != nil {
return err
}
if err := value.Content[i+1].Decode(&res.Command); err != nil {
return err
}
}
return nil
}
func main() {
var f Config
var err error
if err = yaml.Unmarshal(input, &f); err != nil {
panic(err)
}
for _, cmd := range f.Commands {
fmt.Printf("%+v\n", cmd)
}
}
I have split the command data into Command and NamedCommand to make the code simpler, since you can just call Decode giving the embedded Command struct for the values. If everything was in the same struct, you'd need to manually map keys to struct fields.

How to mimic a union type in Gorm?

I know that using custom types is a common question, but bear with me...
I would like to define a custom type 'ConnectionInfo' (see below):
type DataSource struct {
gorm.Model
Name string
Type DataSourceType `sql:"type:ENUM('POSTGRES')" gorm:"column:data_source_type"`
ConnectionInfo ConnectionInfo `gorm:"embedded"`
}
I would like to restrict ConnectionInfo to be one of a limited number of types, i.e.:
type ConnectionInfo interface {
PostgresConnectionInfo | MySQLConnectionInfo
}
How can I do this?
My progress thus far:
I defined a ConnectionInfo interface (I now know this is invalid in GORM, but how do I get around it?)
type ConnectionInfo interface {
IsConnectionInfoType() bool
}
I've then implemented this interface with two types (and implemented the scanner and valuer interfaces) like so:
type PostgresConnectionInfo struct {
Host string
Port int
Username string
Password string
DBName string
}
func (PostgresConnectionInfo) IsConnectionInfoType() bool {
return true
}
func (p *PostgresConnectionInfo) Scan(value interface{}) error {
bytes, ok := value.([]byte)
if !ok {
return fmt.Errorf("failed to unmarshal the following to a PostgresConnectionInfo value: %v", value)
}
result := PostgresConnectionInfo{}
if err := json.Unmarshal(bytes, &result); err != nil {
return err
}
*p = result
return nil
}
func (p PostgresConnectionInfo) Value() (driver.Value, error) {
return json.Marshal(p)
}
But of course I get the following error:
unsupported data type: /models.ConnectionInfo
Instead of using this union, you can approach this way GITHUB LINK. You can clone these repo and run the code. This is working.
package storage
import (
"database/sql/driver"
"encoding/json"
"fmt"
"log"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
type DataSourceType string
const (
POSTGRES DataSourceType = "POSTGRES"
MYSQL DataSourceType = "MYSQL"
)
type PostgresConnectionInfo struct {
Host string
Port int
Username string
Password string
DBName string
}
type MySQLConnectionInfo struct {
Host string
Port int
Username string
Password string
DBName string
}
type ConnectionInfo struct {
Postgres *PostgresConnectionInfo `gorm:"-" json:"postgres,omitempty"`
Mysql *MySQLConnectionInfo `gorm:"-" json:"mysql,omitempty"`
}
type DataSource struct {
gorm.Model
Name string
Type DataSourceType `sql:"type:ENUM('POSTGRES')" gorm:"column:data_source_type"`
ConnectionInfo ConnectionInfo `gorm:"type:json" `
}
func (a *ConnectionInfo) Scan(src any) error {
switch src := src.(type) {
case nil:
return nil
case []byte:
var res ConnectionInfo
err := json.Unmarshal(src, &res)
*a = res
return err
default:
return fmt.Errorf("scan: unable to scan type %T into struct", src)
}
}
func (a ConnectionInfo) Value() (driver.Value, error) {
ba, err := json.Marshal(a)
return ba, err
}
func GormTest2() {
db, err := gorm.Open(sqlite.Open("gorm.db"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Info),
})
if err != nil {
log.Fatal("could not open database")
}
err = db.AutoMigrate(&DataSource{})
if err != nil {
log.Fatal("could not migrate database")
}
createTestData1(db)
fetchData1(db)
}
func createTestData1(db *gorm.DB) {
ds := []DataSource{
{
Name: "Postgres",
Type: POSTGRES,
ConnectionInfo: ConnectionInfo{
Postgres: &PostgresConnectionInfo{
Host: "localhost",
Port: 333,
Username: "sdlfj",
Password: "sdfs",
DBName: "sdfsd",
},
},
},
{
Name: "Mysql",
Type: MYSQL,
ConnectionInfo: ConnectionInfo{
Mysql: &MySQLConnectionInfo{
Host: "localhost",
Port: 333,
Username: "sdlfj",
Password: "sdfs",
DBName: "sdfsd",
},
},
},
}
err := db.Create(&ds).Error
if err != nil {
log.Println("failed to create data")
}
}
func fetchData1(db *gorm.DB) {
var dsList []DataSource
if err := db.Find(&dsList).Error; err != nil {
log.Println("failed to load data")
}
log.Println(dsList)
}

Unmarshaling YAML into different struct based off YAML field

I'm trying to unmarshal the following YAML data into Go structures.
The data is the in the following format:
fetchers:
- type: "aws"
config:
omega: "lul"
- type: "kubernetes"
config:
foo: "bar"
Based of the type field, I want to determine wether to unmarshal the config field into awsConfig or kubernetesConfig struct.
My current code looks like this (using "gopkg.in/yaml.v2"):
type kubernetesConfig struct {
foo string `yaml:"foo"`
}
type awsConfig struct {
omega string `yaml:"omega"`
}
var c struct {
Fetchers []struct {
Type string `yaml:"type"`
Config interface{} `yaml:"config"`
} `yaml:"fetchers"`
}
err := yaml.Unmarshal(data, &c)
if err != nil {
log.Fatal(err)
}
for _, val := range c.Fetchers {
switch val.Type {
case "kubernetes":
conf := val.Config.(kubernetesConfig)
fmt.Println(conf.foo)
case "aws":
conf := val.Config.(awsConfig)
fmt.Println(conf.omega)
default:
log.Fatalf("No matching type, was type %v", val.Type)
}
}
Code in playground: https://go.dev/play/p/klxOoHMCtnG
Currently it gets unmarshalled as map[interface {}]interface {}, which can't be converted to one of the structs above.
Error:
panic: interface conversion: interface {} is map[interface {}]interface {}, not main.awsConfig \
Do I have to implemented the Unmarshaler Interface of the YAML package with a custom UnmarshalYAML function to get this done?
Found the solution by implementing Unmarshaler Interface:
type Fetcher struct {
Type string `yaml:"type"`
Config interface{} `yaml:"config"`
}
// Interface compliance
var _ yaml.Unmarshaler = &Fetcher{}
func (f *Fetcher) UnmarshalYAML(unmarshal func(interface{}) error) error {
var t struct {
Type string `yaml:"type"`
}
err := unmarshal(&t)
if err != nil {
return err
}
f.Type = t.Type
switch t.Type {
case "kubernetes":
var c struct {
Config kubernetesConfig `yaml:"config"`
}
err := unmarshal(&c)
if err != nil {
return err
}
f.Config = c.Config
case "aws":
var c struct {
Config awsConfig `yaml:"config"`
}
err := unmarshal(&c)
if err != nil {
return err
}
f.Config = c.Config
}
return nil
}
This type of task - where you want to delay the unmarshaling - is very similar to how json.RawMessage works with examples like this.
The yaml package does not have a similar mechanism for RawMessage - but this technique can easily be replicated as outlined here:
type RawMessage struct {
unmarshal func(interface{}) error
}
func (msg *RawMessage) UnmarshalYAML(unmarshal func(interface{}) error) error {
msg.unmarshal = unmarshal
return nil
}
// call this method later - when we know what concrete type to use
func (msg *RawMessage) Unmarshal(v interace{}) error {
return msg.unmarshal(v)
}
So to leverage this in your case:
var fs struct {
Configs []struct {
Type string `yaml:"type"`
Config RawMessage `yaml:"config"` // delay unmarshaling
} `yaml:"fetchers"`
}
err = yaml.Unmarshal([]byte(data), &fs)
if err != nil {
return
}
and based on the config "Type" (aws or kubernetes), you can finally unmarshal the RawMessage into the correct concrete type:
aws := awsConfig{} // concrete type
err = c.Config.Unmarshal(&aws)
or:
k8s := kubernetesConfig{} // concrete type
err = c.Config.Unmarshal(&k8s)
Working example here: https://go.dev/play/p/wsykOXNWk3H

I am trying to unmarshall the following yaml into the below structs, how do I convert the List field into a map field and access it using struct?

The YAML file:
namespaces:
- namespace: default
aliasname: k8s
components:
- component: comp1
replicas: 1
port: 8080
- component: comp2
replicas: 1
port: 9999
- namespace: ns2
components:
- component: comp1
replicas: 1
From the YAML file above, I want to create structs like the following:
type Namespaces struct {
NamespaceName string `yaml:"namespace"`
Aliasname string `yaml:"aliasname,omitempty"`
ListOfComponents []Components `yaml:"components"`
ComponentMap map[string]Components
}
type Components struct {
ComponentName string `yaml:"component"`
NumReplicas int `yaml:"replicas"`
Port int `yaml:"port"`
}
type Config struct {
ListOfNamespaces []Namespaces `yaml:"namespaces"`
NamespaceMap map[string]Namespaces
}
The fields Namespacemap and Componentmap should be able to be retrieved when accessing the config and namespace object respectively. I created a method to convert the list of the namespaces and components into maps, but when I call config.Namespacemap or Namespace.ComponentMap, it returns an empty map.
Basically I would like to know: How do we add extra fields to type structs? I would like to access new variables like a map from the config struct.
Update:
Thanks blami for guiding me, but when I try to write the same for Components, it doesn't give me the whole namespacemap with the componentmap included:
type Components struct {
ComponentName string `yaml:"component"`
NumReplicas int `yaml:"replicas"`
Port int `yaml:"port"`
}
type Namespaces struct {
NamespaceName string `yaml:"namespace"`
Aliasname string `yaml:"aliasname"`
ComponentMap map[string]Components `yaml:"components"`
}
func (n *Namespaces) UnmarshalYAML(unmarshal func(interface{}) error) error {
type origNamespace struct {
ListOfComponents []Components `yaml:"components"`
}
var on origNamespace
err1 := unmarshal(&on)
if err1 != nil {
return err1
}
n.ComponentMap = make(map[string]Components)
for _, i := range on.ListOfComponents {
n.ComponentMap[i.ComponentName] = i
}
return nil
}
When I run the config.NamespaceMap it gives the following
map[:{NamespaceName: K8sNamespace: ComponentMap:map[comp1:{ComponentName:comp1 NumShards:0 NumReplicas:1 Port:0 EpochLength:0}]}]
If you want to do such transformation you will need to write a customized UnmarshalYAML() receiver on types Config and Namespace. Here is basic working example of doing so (only for "namespaces"):
type Config struct {
Namespaces map[string]Namespace `yaml:"namespaces"`
}
// Replace go-yaml built-in unmarshaller with custom that will transform list to map.
// Note this code is meant only as demonstration!
func (c *Config) UnmarshalYAML(unmarshal func(interface{}) error) error {
type origConfig struct {
Namespaces []Namespace `yaml:"namespaces"`
}
var o origConfig
err := unmarshal(&o)
if err != nil {
return err
}
// Assign namespaces in list to map; namespaces with same name will be overwritten
c.Namespaces = make(map[string]Namespace)
for _, n := range o.Namespaces {
c.Namespaces[n.Namespace] = n
}
return nil
}
// You can use "Config" type as usual
func main() {
var config Config
err := yaml.Unmarshal(<your_yaml>, &config)
if err != nil {
log.Panic(err)
}
fmt.Println(config.Namespaces)
// map[default:{default k8s} ns2:{ns2 }]
fmt.Printf("%v\n", config.Namespaces["default"])
// {default k8s}
}
As noted in code example this will cause some problems (e.g. what to do if namespace names are same?).
Playground link: https://go.dev/play/p/IKg8kmRnknq

Using different levels of interfaces

I'm trying to organize my code using interfaces in Go.
I have 2 sources of data: FTP and API. In each source, I have several structs that make the logic vary depending on the case.
In this question, I will omit API and stick with FTP.
My problem comes from the impossibility to say: FTPAcq is also an Acquisition
If FetchMeters(), when I do ftp.Decrypt(nil) I would like ftp to be "compatible" with FTPAcq
Here is my code:
package main
import (
"github.com/dutchcoders/goftp"
log "github.com/sirupsen/logrus"
"os"
)
type Acquisition interface {
FetchMeters() ([]Meter, error)
Name() string
}
type FTPAcq interface {
Unzip(file string) string
Decrypt(file string) string
}
//type APIAcq interface {
// FetchMeter(meterID string) (Meter, error)
//}
func main() {
var acqs []Acquisition
ftp, err := NewFTPDriver(os.Getenv("FTP_USER"), os.Getenv("FTP_PASSWD"), os.Getenv("FTP_ADDR"), os.Getenv("FTP_PORT"))
if err != nil {
panic(err)
}
ftp1 := NewFTPDriverSGE(*ftp)
ftp2 := NewFTPDriverTA(*ftp)
acqs = append(acqs, ftp1, ftp2)
for _, acq := range acqs {
tmpMeters, err := acq.FetchMeters()
if err != nil {
log.Warn(acq.Name(), " got error :", err)
}
log.Info(tmpMeters)
}
}
type Meter struct {
ID string
OperationID string
Unit string
}
//FtpSGE is a implementation of acquisition Interface (see driver.go)
type FTP struct {
Username string
Password string
Url string
Port string
client *goftp.FTP
}
type FTPSGE struct {
FTP
}
type FTPTA struct {
FTP
}
func (f FTPSGE) Unzip(path string) []string {
return nil
}
func (f FTPTA) Unzip(path string) []string {
return nil
}
func (f FTPSGE) Decrypt(path string) []string {
return nil
}
func (f FTPTA) Decrypt(path string) []string {
return nil
}
func (ftp FTP) FetchMeters() ([]Meter, error) {
log.Info(ftp.Name(), " is running")
files := ftp.Download(nil)
files = ftp.Decrypt("") // I have several implementation of Decrypt
files = ftp.Unzip("") // I have several implementation of Unzip
log.Info(files)
return nil, nil
}
func (ftp FTP) Name() string {
panic("FTP ")
}
func (ftp FTP) Download(files []string) []string {
panic("implement me")
}
func NewFTPDriver(user, password, url, port string) (*FTP, error) {
var err error
ftp := &FTP{
Username: user,
Password: password,
Url: url,
Port: port,
}
if ftp.client, err = goftp.Connect(url + ":" + port); err != nil {
return nil, err
}
if err = ftp.client.Login(user, password); err != nil {
return nil, err
}
return ftp, nil
}
func NewFTPDriverSGE(f FTP) *FTPSGE {
return &FTPSGE{f}
}
func NewFTPDriverTA(f FTP) *FTPTA {
return &FTPTA{f}
}
In my case, I get:
ftp.Decrypt undefined (type FTP has no field or method Decrypt)
How should I do?
FTP does not implement FTPAcq. It implements only Acquisition. It doesn't even have Decrypt() as a method, interface or not.
FTPSGE and FTPTA implement FTPAcq, but they are not the same type as FTP.
I don't know what you're trying to accomplish, but perhaps something to try is embedding FTP in FTPSGE and FTPTA. This gives those 2 types the fields and methods of the embedded type and still allows you to define additional methods on those types (the methods for FTPAcq in your case).
For example
type FTPSGE {
FTP
}
// OR
type FTPSGE {
*FTP
}
Which you then create as so: x := FTPSGE{ftp1}. Keep in mind that this will create a copy of ftp1 inside x. If ftp1 is type FTP (not a pointer), the entire struct gets copied. If ftp1 is type *FTP (a pointer, which seems to be what you're using), the pointer is copied and x.FTP still points to the same data as ftp1.
This means FTPSGE will implement both Acquisition and FTPAcq.
You'll have to be careful of whether or not interfaces are implemented on the value or pointer: func (a A) Something() vs func (a *A) Somthing().
Here is some reading on methods, interfaces, and embedding.
https://www.ardanlabs.com/blog/2014/05/methods-interfaces-and-embedded-types.html
https://travix.io/type-embedding-in-go-ba40dd4264df
https://golang.org/doc/effective_go.html (and lots of other info)

Resources