using method of interface in golang from different file - go

I've method which I want to provide some interface to make it more easier to test
This is the function
File A
func readFile(s source) ([]byte, error) {
p := fs.GetPath()
file, err := ioutil.ReadFile(p + "/" + s.path + "/" + "rts.yaml")
if err != nil {
return yamlFile, fmt.Errorf("erro reading file : %s", err.Error())
}
return file, err
}
Now I add for it struct
type source struct{
path string
}
And the interface that the readFile is implementing
type fileReader interface {
readFile(path string) ([]byte, error)
}
And now I need to call this function from another file but Im getting error while doing this
File B
type source struct {
path string
}
a := source{}
yamlFile, err := readFile(a)
what am I missing here ?

Import the package containing the source struct in File A and then use that struct to initialize the variable after that pass the variable to the readFile function.
File B
import A
a := A.Source{}
Because source struct in File A is different from source struct in File B. And source struct of File A is implementing the interface that's why you need to import the source struct and then pass it into the function.
One this should be noticed that to make any struct or function exportable you should start the struct or fucntion name with upper case letter.
File A
// make struct exportable
type Source struct{
path string
}
implemented the interface which is different from
File B
type source struct{
path string
}
which does not implemented the interface.
Edited
File A
package main
import (
"fmt"
"io/ioutil"
"os"
)
type Source struct {
Path string
}
type fileReader interface {
readOneFile() ([]byte, error)
}
func(s Source) readOneFile() ([]byte, error) {
cwd, err := os.Getwd()
file, err := ioutil.ReadFile(fmt.Sprintf("%s/file.txt", cwd))
if err != nil {
return nil, fmt.Errorf("erro reading file : %s", err.Error())
}
return file, err
}
File B
package main
import (
"fmt"
)
func main() {
s := Source{}
data, err := s.readOneFile()
if err != nil {
fmt.Errorf("Error in reading the file")
}
fmt.Println(string(data))
}

Related

How to create a golang struct with given name of string

I am using protobuf to generate a lot of struct, and I want to use gorm.io to Db.AutoMigrate all the struct in the generate package into the database. But the function func (*gorm.DB).AutoMigrate(dst ...interface{}) error can only accept variadic struct instance as its parameters. The following code can retrieved all the name of structs is string format.
package main
import (
"fmt"
"go/ast"
"go/parser"
"go/token"
)
func main() {
// src is the input for which we want to inspect the AST.
src := `
package service
type Screen struct {
Width float64
Height float64
}
type Memory struct {
Unit float64
}
`
// Create the AST by parsing src.
fset := token.NewFileSet() // positions are relative to fset
f, err := parser.ParseFile(fset, "src.go", src, 0)
if err != nil {
panic(err)
}
for _, node := range f.Decls {
switch node.(type) {
case *ast.GenDecl:
genDecl := node.(*ast.GenDecl)
for _, spec := range genDecl.Specs {
switch spec.(type) {
case *ast.TypeSpec:
typeSpec := spec.(*ast.TypeSpec)
fmt.Printf("Struct: name=%s\n", typeSpec.Name.Name)
}
}
}
}
}
My Question is:
This code can got all the name of structs , but the gorm.io need initialize the struct DB.AutoMigrate(&User{}). How can I do create a empty struct that specified by the string of the name at the runtime. eg. the "Screen" ===> &Screen{}.

How to design classes for X number of config files which needs to be read individually in memory?

I am working with lot of config files. I need to read all those individual config file in their own struct and then make one giant Config struct which holds all other individual config struct in it.
Let's suppose if I am working with 3 config files.
ClientConfig deals with one config file.
DataMapConfig deals with second config file.
ProcessDataConfig deals with third config file.
I created separate class for each of those individual config file and have separate Readxxxxx method in them to read their own individual config and return struct back. Below is my config.go file which is called via Init method from main function after passing path and logger.
config.go
package config
import (
"encoding/json"
"fmt"
"io/ioutil"
"github.com/david/internal/utilities"
)
type Config struct {
ClientMapConfigs ClientConfig
DataMapConfigs DataMapConfig
ProcessDataConfigs ProcessDataConfig
}
func Init(path string, logger log.Logger) (*Config, error) {
var err error
clientConfig, err := ReadClientMapConfig(path, logger)
dataMapConfig, err := ReadDataMapConfig(path, logger)
processDataConfig, err := ReadProcessDataConfig(path, logger)
if err != nil {
return nil, err
}
return &Config{
ClientMapConfigs: *clientConfig,
DataMapConfigs: *dataMapConfig,
ProcessDataConfigs: *processDataConfig,
}, nil
}
clientconfig.go
package config
import (
"encoding/json"
"fmt"
"io/ioutil"
"github.com/david/internal/utilities"
)
type ClientConfig struct {
.....
.....
}
const (
ClientConfigFile = "clientConfigMap.json"
)
func ReadClientMapConfig(path string, logger log.Logger) (*ClientConfig, error) {
files, err := utilities.FindFiles(path, ClientConfigFile)
// read all the files
// do some validation on all those files
// deserialize them into ClientConfig struct
// return clientconfig object back
}
datamapconfig.go
Similar style I have for datamapconfig too. Exactly replica of clientconfig.go file but operating on different config file name and will return DataMapConfig struct back.
processdataConfig.go
Same thing as clientconfig.go file. Only difference is it will operate on different config file and return ProcessDataConfig struct back.
Problem Statement
I am looking for ideas where this above design can be improved? Is there any better way to do this in golang? Can we use interface or anything else which can improve the above design?
If I have let's say 10 different files instead of 3, then do I need to keep doing above same thing for remaining 7 files? If yes, then the code will look ugly. Any suggestions or ideas will greatly help me.
Update
Everything looks good but I have few questions as I am confuse on how can I achieve those with your current suggestion. On majority of my configs, your suggestion is perfect but there are two cases on two different configs where I am confuse on how to do it.
Case 1 After deserializing json into original struct which matches json format, I make another different struct after massaging that data and then I return that struct back.
Case 2 All my configs have one file but there are few configs which have multiple files in them and the number isn't fixed. So I pass regex file name and then I find all the files starting with that regex and then loop over all those files one by one. After deserializing each json file, I start populating another object and keep populating it until all files have been deserialized and then make a new struct with those objects and then return it.
Example of above scenarios:
Sample case 1
package config
import (
"encoding/json"
"fmt"
"io/ioutil"
"github.com/david/internal/utilities"
)
type CustomerManifest struct {
CustomerManifest map[int64]Site
}
type CustomerConfigs struct {
CustomerConfigurations []Site `json:"customerConfigurations"`
}
type Site struct {
....
....
}
const (
CustomerConfigFile = "abc.json"
)
func ReadCustomerConfig(path string, logger log.Logger) (*CustomerManifest, error) {
// I try to find all the files with my below utility method.
// Work with single file name and also with regex name
files, err := utilities.FindFiles(path, CustomerConfigFile)
if err != nil {
return nil, err
}
var customerConfig CustomerConfigs
// there is only file for this config so loop will run once
for _, file := range files {
body, err := ioutil.ReadFile(file)
if err != nil {
return nil, err
}
err = json.Unmarshal(body, &customerConfig)
if err != nil {
return nil, err
}
}
customerConfigIndex := BuildIndex(customerConfig, logger)
return &CustomerManifest{CustomerManifest: customerConfigIndex}, nil
}
func BuildIndex(customerConfig CustomerConfigs, logger log.Logger) map[int64]Site {
...
...
}
As you can see above in sample case 1, I am making CustomerManifest struct from CustomerConfigs struct and then return it instead of returning CustomerConfigs directly.
Sample case 2
package config
import (
"encoding/json"
"fmt"
"io/ioutil"
"github.com/david/internal/utilities"
)
type StateManifest struct {
NotionTemplates NotionTemplates
NotionIndex map[int64]NotionTemplates
}
type NotionMapConfigs struct {
NotionTemplates []NotionTemplates `json:"notionTemplates"`
...
}
const (
// there are many files starting with "state-", it's not fixed number
StateConfigFile = "state-*.json"
)
func ReadStateConfig(path string, logger log.Logger) (*StateManifest, error) {
// I try to find all the files with my below utility method.
// Work with single file name and also with regex name
files, err := utilities.FindFiles(path, StateConfigFile)
if err != nil {
return nil, err
}
var defaultTemp NotionTemplates
var idx = map[int64]NotionTemplates{}
// there are lot of config files for this config so loop will run multiple times
for _, file := range files {
var notionMapConfig NotionMapConfigs
body, err := ioutil.ReadFile(file)
if err != nil {
return nil, err
}
err = json.Unmarshal(body, &notionMapConfig)
if err != nil {
return nil, err
}
for _, tt := range notionMapConfig.NotionTemplates {
if tt.IsProcess {
defaultTemp = tt
} else if tt.ClientId > 0 {
idx[tt.ClientId] = tt
}
}
}
stateManifest := StateManifest{
NotionTemplates: defaultTemp,
NotionIndex: idx,
}
return &stateManifest, nil
}
As you can see above in my both the cases, I am making another different struct after deserializing is done and then I return that struct back but as of now in your current suggestion I think I won't be able to do this generically because for each config I do different type of massaging and then return those struct back. Is there any way to achieve above functionality with your current suggestion? Basically for each config if I want to do some massaging, then I should be able to do it and return new modified struct back but for some cases if I don't want to do any massaging then I can return direct deserialize json struct back. Can this be done generically?
Since there are config which has multiple files in them so that is why I was using my utilities.FindFiles method to give me all files basis on file name or regex name and then I loop over all those files to either return original struct back or return new struct back after massaging original struct data.
You can use a common function to load all the configuration files.
Assume you have config structures:
type Config1 struct {...}
type Config2 struct {...}
type Config3 struct {...}
You define configuration validators for those who need it:
func (c Config1) Validate() error {...}
func (c Config2) Validate() error {...}
Note that these implement a Validatable interface:
type Validatable interface {
Validate() error
}
There is one config type that includes all these configurations:
type Config struct {
C1 Config1
C2 Config2
C3 Config3
...
}
Then, you can define a simple configuration loader function:
func LoadConfig(fname string, out interface{}) error {
data, err:=ioutil.ReadFile(fname)
if err!=nil {
return err
}
if err:=json.Unmarshal(data,out); err!=nil {
return err
}
// Validate the config if necessary
if validator, ok:=out.(Validatable); ok {
if err:=validator.Validate(); err!=nil {
return err
}
}
return nil
}
Then, you can load the files:
var c Config
if err:=LoadConfig("file1",&c.C1); err!=nil {
return err
}
if err:=LoadConfig("file2",&c.C2); err!=nil {
return err
}
...
If there are multiple files loading different parts of the same struct, you can do:
LoadConfig("file1",&c.C3)
LoadConfig("file2",&c.C3)
...
You can simplify this further by defining a slice:
type cfgInfo struct {
fileName string
getCfg func(*Config) interface{}
}
var configs=[]cfgInfo {
{
fileName: "file1",
getCfg: func(c *Config) interface{} {return &c.C1},
},
{
fileName: "file2",
getCfg: func(c *Config) interface{} {return &c.C2},
},
{
fileName: "file3",
getCfg: func(c *Config) interface{} {return &c.C3},
},
...
}
func loadConfigs(cfg *Config) error {
for _,f:=range configs {
if err:=loadConfig(f.fileName,f.getCfg(cfg)); err!=nil {
return err
}
}
return nil
}
Then, loadConfigs would load all the configuration files into cfg.
func main() {
var cfg Config
if err:=loadConfigs(&cfg); err!=nil {
panic(err)
}
...
}
Any configuration that doesn't match this pattern can be dealt with using LoadConfig:
var customConfig1 CustomConfigStruct1
if err:=LoadConfig("customConfigFile1",&customConfig1); err!=nil {
panic(err)
}
cfg.CustomConfig1 = processCustomConfig1(customConfig1)
var customConfig2 CustomConfigStruct2
if err:=LoadConfig("customConfigFile2",&customConfig2); err!=nil {
panic(err)
}
cfg.CustomConfig2 = processCustomConfig2(customConfig2)

Is it possible to present an embedded file as a filename?

I use a function that requires a filename as a parameter (of type string). It works fine when providing the filename.
I would like to embed this file in my binary. I can then have the contents as []byte or string but that's not useful. I can also fave it as embed.FS but my understanding is that this is an abstraction that can be used by some functions only.
What I would need is the ability to present this embedded file as a filename (a string) that would then be used by the underlying function to open the (embedded) file.
Is this possible?
Filename that Key accept as string argument is only abstraction on ioutil.ReadFile, see auth.go
What you can do is implement ssh.Auth yourself, here is small example.
package main
import (
_ "embed"
"fmt"
"github.com/melbahja/goph"
"golang.org/x/crypto/ssh"
)
//go:embed id_rsa
var privateKey []byte
func main() {
auth, err := Auth(privateKey, []byte("foobar"))
fmt.Println(auth, err)
}
func Auth(privateKey, pass []byte) (goph.Auth, error) {
signer, err := Singer(privateKey, pass)
if err != nil {
return nil, err
}
return goph.Auth{
ssh.PublicKeys(signer),
}, nil
}
func Singer(privateKey, pass []byte) (ssh.Signer, error) {
if len(pass) != 0 {
return ssh.ParsePrivateKeyWithPassphrase(privateKey, pass)
}
return ssh.ParsePrivateKey(privateKey)
}

Can golang plugins be used for factory functions?

I have the following code in a golang plugin module:
plug.go
package main
import "fmt"
var (
Thing = New("first thing")
ThingFactory = thingFactory{}
)
type thing struct {
i int
s string
}
func New(s string) thing {
return thing{s: s}
}
func (t *thing) Say() string {
t.i++
return fmt.Sprintf("%s - %d", t.s, t.i)
}
type thingFactory struct{}
func (t thingFactory) Make(s string) thing {
return New(s)
}
it is compiled as a .so object and used in another program:
main.go
package main
import (
"fmt"
"plugin"
)
func main() {
p, err := plugin.Open("../plug/plug.so")
if err != nil {
panic(err)
}
symbol, err := p.Lookup("Thing")
if err != nil {
panic(err)
}
thing := symbol.(Sayer)
fmt.Println(thing.Say())
symbol, err = p.Lookup("ThingFactory") // <-problems start here
if err != nil {
panic(err)
}
factory := symbol.(GetSayer)
madeThing := factory.Make("how about me?")
fmt.Println(madeThing.Say())
fmt.Println(madeThing.Say())
}
type Sayer interface {
Say() string
}
type GetSayer interface {
Make(string) Sayer
}
I'm able to lookup the Thing, and call Say() on it, but the second interface conversion panics:
first thing - 1
panic: interface conversion: *main.thingFactory is not main.GetSayer: missing method Make
even though the runtime recognizes the first symbol as a Sayer it doesn't recognize that thingFactory obviously has a Make() method, which should return something that is also a Sayer.
Am I missing something obvious here?
The first problem is that in your plugin thingFactory (more precicely *thingfactory) does not have a method described in your main app's GetSayer interface:
Make(string) Sayer
You have:
Make(string) thing
So (first) you have to change thingFactory.Make() to this:
type Sayer interface {
Say() string
}
func (t thingFactory) Make(s string) Sayer {
th := New(s)
return &th
}
After this it still won't work. And the reason for this is because the plugin's Sayer type is not identical to your main app's Sayer type. But they must be the same in order to implement your main app's GetSayer interface.
One solution is to "outsource" the Sayer interface to its own package, and use this common, shared package both in the plugin and in the main app.
Let's create a new package, call it subplay:
package subplay
type Sayer interface {
Say() string
}
Import this package and use it in the plugin:
package main
import (
"fmt"
"path/to/subplay"
)
var (
Thing = New("first thing")
ThingFactory = thingFactory{}
)
type thing struct {
i int
s string
}
func New(s string) thing {
return thing{s: s}
}
func (t *thing) Say() string {
t.i++
return fmt.Sprintf("%s - %d", t.s, t.i)
}
type thingFactory struct{}
func (t thingFactory) Make(s string) subplay.Sayer {
th := New(s)
return &th
}
And also import and use it in the main app:
package main
import (
"fmt"
"path/to/subplay"
"plugin"
)
func main() {
p, err := plugin.Open("../plug/plug.so")
if err != nil {
panic(err)
}
symbol, err := p.Lookup("Thing")
if err != nil {
panic(err)
}
thing := symbol.(subplay.Sayer)
fmt.Println(thing.Say())
symbol, err = p.Lookup("ThingFactory")
if err != nil {
panic(err)
}
factory := symbol.(GetSayer)
madeThing := factory.Make("how about me?")
fmt.Println(madeThing.Say())
fmt.Println(madeThing.Say())
}
type GetSayer interface {
Make(string) subplay.Sayer
}
Now it will work, and output will be:
first thing - 1
how about me? - 1
how about me? - 2
See related questions:
go 1.8 plugin use custom interface
How do Go plugin dependencies work?
Your plugin Make method should return a Sayer object not thing
type Sayer interface {
Say() string
}
func (t *thingFactory) Make(s string) Sayer {
return New(s)
}

How can I implement my own File type?

I am having a function that opens a file and writes to it using the os.OpenFile function.
What I basically want to do is mock the File that is getting returned by it in order to write tests. Because I want to better understand Go, I'd like to do it without using third party packages.
Here is what I have tried:
My Package
package logger
import (
"fmt"
"time"
"sync"
"os"
"strings"
"path/filepath"
"io"
)
const timestampFormat = "2006-01-02 15:04:05.999999999"
type FileOpener interface {
OpenFile(name string, flag int, perm os.FileMode) (*os.File, error)
}
type RotateWriter struct {
fileOpener FileOpener
lock sync.Mutex
filename string
fp *os.File
}
type defaultFileOpener struct{}
func (fo defaultFileOpener) OpenFile(name string, flag int, perm os.FileMode) (*os.File, error) {
return os.OpenFile(name, flag, perm)
}
func CreateRotateWriter(filename string, fileOpener FileOpener) (RotateWriter) {
if (fileOpener == nil) {
return RotateWriter{filename: filename, fileOpener: defaultFileOpener{}}
}
return RotateWriter{filename: filename, fileOpener: fileOpener}
}
func (writer RotateWriter) Write(bytes []byte) (int, error) {
writer.lock.Lock()
defer writer.lock.Unlock()
extension := filepath.Ext(writer.filename)
filename := strings.TrimSuffix(writer.filename, extension)
// There is a specific constants that are used for formatting dates.
// For example 2006 is the YYYYY, 01 is MM and 02 is DD
// Check https://golang.org/src/time/format.go line 88 for reference
fullFilename := filename + time.Now().UTC().Format("-2006-01-02") + extension
// Open the file for the first time if we don't already did so
if writer.fp == nil {
fp, err := os.OpenFile(fullFilename, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0666)
if err != nil {
return 0, err
}
writer.fp = fp;
}
// We are now pointing to a different file. Close the previous one and open a new one
if fullFilename != writer.fp.Name() {
writer.fp.Close()
fp, err := os.OpenFile(fullFilename, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0666)
if err != nil {
return 0, err
}
writer.fp = fp;
}
return writer.fp.Write([]byte("[" + time.Now().UTC().Format(timestampFormat) + "] " + string(bytes)))
}
What I was hoping to do in my test package is something like this
type file interface {
io.Closer
io.Reader
io.ReaderAt
io.Seeker
Stat() (os.FileInfo, error)
}
type fileType struct{
fd int
name string
contents string // Where I'll keep the in-memory contents maybe
}
type defaultFileOpener struct{
}
func (fo defaultFileOpener) OpenFile(name string, flag int, perm os.FileMode) (*file, error){
return &fileType{1, name, ""}, nil //Cannot use &fileType{1, name, ""}(type *fileType) as type *file
}
func (f fileType) Write(bytes []byte) (int, error){
f.contents += string(bytes)
return len(bytes), nil
}
I'm most probably misunderstand something, is it even possible to create my own File type in go?
From the snippet it's not clear whether or not the *fileType implements all of the methods of the file interface, but if it doesn't you should first make sure that it does. Because if it doesn't you'll not be able to use it in your tests as you might intend.
Your file opener's OpenFile method should have the interface as its return type, not the pointer to the interface. That is:
func (fo defaultFileOpener) OpenFile(name string, flag int, perm os.FileMode) (file, error) {
This is because *file is not the same type as file, and a value of a type that implements the file interface (e.g. yours *fileType) cannot be used where the pointer to that interface is expected.
And returning a pointer to an interface is almost never what you actually want, maybe it would make sense to do that if you wanted switch the interface value for another using pointer indirection, but that does not seem to be the case here. If you scan through the standard library you'll have a hard time finding functions/methods that return pointers to interface types...
But let's say that that's what you want to do then you have to return a pointer to a value of the interface type, not a pointer to a type that implements the interface.
E.g.
var f file = &fileType{} // f is of type file
return &f, nil // &f is of type *file
Keep in mind that f's type has to be file for the return to work, if it's just *fileType it will not compile.
var f = &fileType{} // f is of type *fileType
return &f, nil // &f is of type **fileType

Resources