implement a generic argument method - go

I have code in Go, which includes many execution methods.
Each method receives it own struct of parameters.
I want a dispatcher to call each method with its related struct.
The dispatcher receives the name of the execution method, and a JSON of the parameters struct.
Then, it uses reflection to build the struct, and calls the method.
The problem is I get compilation error unless I create the execution methods using empty interface.
In the example below I have 2 execution methods: api1 is compiling, but is using the empty interface and explicit casting. api2 is what I want to do, but it is failing with compile error:
cannot use api2 (type func(Api2Parameters)) as type Api in assignment
How can I make api2 usage compile?
import (
"encoding/json"
"log"
"reflect"
)
type Api func(arguments interface{})
type ApiDetails struct {
Executor *Api
ParametersType reflect.Type
}
var Apis map[string]*ApiDetails
func RunApi(apiName string, data string) {
api := Apis[apiName]
parameters := reflect.New(api.ParametersType).Interface().(interface{})
_ = json.Unmarshal([]byte(data), parameters)
(*api.Executor)(parameters)
}
type Api1Parameters struct {
Count1 int
Id1 string
}
func api1(arguments interface{}) {
parameters, _ := arguments.(*Api1Parameters)
log.Printf("api1 parameters(%+v)", parameters)
}
type Api2Parameters struct {
Count2 int
Id2 string
}
func api2(arguments Api2Parameters) {
log.Printf("api2 parameters(%+v)", arguments)
}
func Test() {
// this assignment works fine
var api_1 Api = api1
Apis["api1"] = &ApiDetails{
Executor: &api_1,
ParametersType: reflect.TypeOf(Api1Parameters{}),
}
// this assignment produce compile error
var api_2 Api = api2
Apis["api2"] = &ApiDetails{
Executor: &api_2,
ParametersType: reflect.TypeOf(Api2Parameters{}),
}
RunApi("api1", `{"Count1":19, "Id1":"a"}`)
RunApi("api2", `{"Count2":34, "Id2":"b"}`)
}

Create a value using the argument type, unmarshal to that value and invoke the function:
var Apis = map[string]interface{}{
"api1": api1,
"api2": api2,
}
func RunApi(apiName string, data string) {
fv := reflect.ValueOf(Apis[apiName])
ft := fv.Type()
pin := reflect.New(ft.In(0))
_ = json.Unmarshal([]byte(data), pin.Interface())
fv.Call([]reflect.Value{pin.Elem()})
}
Run it on the playground.

If you want to keep this type-safe, you can push the JSON-to-args conversion a little deeper:
type Api func(string)
type ApiDetails struct {
// Don't use *Api, that's a function pointer-pointer
Executor Api
}
Then, use a closure to do the JSON translation:
Apis["api1"] = &ApiDetails{
Executor: func(data string) {
var args ArgsForAPI1
json.Unmarshal([]byte(string),&args)
api1Implem(args)
}
}
With this, you can do:
APIs["api1"].Executor( `{"Count1":19, "Id1":"a"}`)
You can change the RunApi to do something like this.
A second option is to do this using reflection:
type ApiDetails struct {
// Executor can accept any function signature
// Assume a function with a single parameter
Executor interface{}
ParametersType reflect.Type
}
Then, the RunApi function can use reflection to construct a struct and call Executor:
parameters := reflect.New(api.ParametersType)
_ = json.Unmarshal([]byte(data), parameters.Interface{})
reflect.ValueOf(api.Executor).Call([]reflect.Value{parameters})
This should work, but you'll only get runtime errors if you mess up.

Related

Unit tests mock an interface from struct property

I want to mock a property from a struct that uses an interface so I don't get a nil pointer when the function arrives at that point.
This is the code:
type Server struct{
parser: Parser
}
type Parser interface{
SetProvider(p *Provider)
}
func (s *Server) doSomething(){
s.anotherAction()
// ...
// here it crashes because I haven't assigned anything to parser in serverMock
s.parser.SetProvider(&Provider{
name: "foo"
})
}
And this the test code:
var serverMock = &Server{
parser:
}
// mock SetProvider
func (s *Server) SetProvider(p *Provider) {
// some action
}
TestMyCustomTest(){
res, err := serverMock.doSomething()
expected := struct{
hobby: "code",
}
assert.Equal(t, &expected, res)
}
As you see I haven't assigned anything to parser: from var serverMock because I still don't know what should I do to make it work. I cannot assign a mockInterface to that parser field because it'll tell me that the original Server struct requires the Parser interface not a mock one and I think I need the &Server pointer for var serverMock = &Server because it's using other actions like s.anotherAction().
If i understand your intention correctly, you want to test Server's doSomething method and it depends on Parser.SetProvider method. So, what you want, instead of mocking Server, is to create mock struct that satisfies Parser interface and create Server instance that uses it. Here is example of such test:
type mockParser struct {
// You can add field here
}
// mockParser implements SetProvider(p *Provider) method so that it satisfies Parser interface
func (parser *mockParser) SetProvider(provider *Provider) {
// You can do something here
}
TestMyCustomTest(t *testing.T){
parser := mockParser{}
srv := &Server{parser: &parser}
// write your test
assert.Equal(t, ...)
}

Golang plugin type assertion

I'm writing a simple app which loads plugin in a predefined format. Example plugin is the following:
package main
import (
"errors"
"fmt"
"strings"
)
var (
ok bool
InvConfig = errors.New("invalid config")
)
type Processor struct {
logEverything bool
}
func (p *Processor) Init(config map[string]interface{}) error {
p.logEverything, ok = config["log_everything"].(bool)
if !ok {
return InvConfig
}
return nil
}
func (p *Processor) Process(buf []byte) []byte {
if p.logEverything {
fmt.Printf("Shouter got data: %v\n", buf)
}
return []byte(strings.ToUpper(string(buf)))
}
func GetProcessor() *Processor {
return &Processor{}
}
I can't quite apprehend how to load such a structure in my main program. So, I declare an interface:
type Processor interface {
Init(map[string]interface{}) error
Process(buf []byte) []byte
}
Then I load "getter" function and try to cast it to a function returning interface to then call it:
p, err := plugin.Open(filepath)
if err != nil {
logrus.Fatalf("Error opening plugin %s: %v", pluginName, err)
}
procGetterInter, err := p.Lookup("GetProcessor")
if err != nil {
logrus.Fatalf("Error loading processor getter for plugin %s: %v", pluginName, err)
}
procGetter, ok := procGetterInter.(func() interface{})
if !ok {
logrus.Fatalf("Error casting processor getter for plugin %s: %T", pluginName, procGetterInter)
}
But the cast fails with an error:
Error casting processor getter for plugin simple_shout: func() *main.Processor
If I return an actual instance (not a pointer) from GetProcessor and try to cast the function to the one returning Processor, I get the same result:
Error casting processor getter for plugin simple_shout: func() main.Processor
How to get a struct instance from plugin (therefore load the function returning it) and type-assert it's an expected interface in my case?
UPD: If I remove everything from Processor interface (that is, it becomes just an empty interface):
type Processor interface {}
And try to cast procGetterInter to a function returning a pointer to Processor interface:
procGetter, ok := procGetterInter.(func() *Processor)
I still get the same error:
plugin.Symbol is func() *main.Processor, not func() *main.Processor (types from different scopes)
Why doesn't it cast even to pointer to an empty interface?
TL;DR:
Check out a full working demo here: https://github.com/jvmatl/go-plugindemo
The long, but (hopefully!) informative answer:
Plugins are tricky in several ways, and #icza's answer is totally correct, but to understand why it's correct and how it applies to your question, you need to understand that the flexible nature of go's interfaces does not apply to complex types.
You have probably already run across this in other contexts:
This is legal in Go:
var a interface{}
var b int
a = b // yep, an int meets the spec for interface{} !
But this is not:
var aa []interface{}
var bb []int
aa = bb // cannot use bb (type []int) as type []interface {} in assignment
Similarly, with functions, this is legal:
type Runner interface {
Run()
}
type UsainBolt struct{}
func (ub *UsainBolt) Run() {
fmt.Println("Catch me if you can!")
}
var a Runner
var b *UsainBolt
a = b // Yep, a (pointer to) Usain Bolt is a runner!
But this is not:
var aa func() Runner
var bb func() *UsainBolt
aa = bb // cannot use bb (type func() *UsainBolt) as type func() Runner in assignment
Now let's look at defined function types. This is where it gets really interesting:
type RunnerGetter func() Runner
var rg RunnerGetter
rg = getUsain // <-- Nope: doesn't compile: "cannot use getUsain (type func() *UsainBolt) as type RunnerGetter in assignment"
rg = getRunner // <-- This *assignment* is allowed: getRunner is assignable to a type RunnerGetter
var i interface{} = getRunner
rg = i.(RunnerGetter) // compiles, but panics at runtime: "interface conversion: interface {} is func() main.Runner, not main.RunnerGetter"
In other words, the language is ok with assigning func getRunner() Runner to a variable of type RunnerGetter, but the type assertion fails, because the type assertion is asking: is this thing actually a variable of type RunnerGetter? And the answer is no, it's a func() Runner which is close, but not quite right, so we panic.
But this works:
var rg RunnerGetter
var i interface{}
i = rg // after this assignment, i *is* a RunnerGetter
rg = i.(RunnerGetter) // so this assertion passes.
Ok, with all that background out of the way, the issue is that the symbol you lookup from your plugin must be exactly the same type as your type assertion says it is, not just close-enough-to-allow-assignment.
As #icza stated, you have a couple of options:
Option 1: Quick and Dirty, gets the job done
In your plugin
func GetGeneric() interface{} {
return &Processor{}
}
In your main: (error-handling skipped for clarity)
p, _ := plugin.Open(pluginFile) // load plugin
newIntf, _ := p.Lookup("Getgeneric") // find symbol
newProc, _ := newIntf.(func() interface{}) // assert symbol to generic constructor
shoutProc, _ := newProc().(processors.Processor) // call generic constructor, type assert the return value
// Now use your new plugin!
shoutProc.Init(map[string]interface{}{"log_everything": true})
output := shoutProc.Process([]byte("whisper"))
Option 2: Cleaner, better if you have many plugins
Declare the interface all you plugins have to meet in another package:
package processors
// Every plugin must be able to give me something that meets this interface
type Processor interface {
Init(map[string]interface{}) error
Process(buf []byte) []byte
}
In your plugin:
type ShoutProcessor struct {
configured bool
logEverything bool
}
func NewProcessor() processors.Processor {
return &ShoutProcessor{}
}
In your main:
p, _ := plugin.Open(pluginFile) // load plugin
newProcIntf, _ := p.Lookup("NewProcessor") // lookup constructor
newProc, _ := newProcIntf.(func() processors.Processor) // assert the type of the func
shoutProc := newProc() // call the constructor, get a new ShoutProcessor
// ready to rock and roll!
shoutProc.Init(map[string]interface{}{"log_everything": true})
output := shoutProc.Process([]byte("whisper"))
The function inside the plugin has a signature:
func GetProcessor() *Processor
You lookup this symbol as an interface{} and you try to type-assert a value of type
func() interface{}
These types do not match because these function types have different return types. Spec: Function types:
A function type denotes the set of all functions with the same parameter and result types.
So you may only type assert the same function type, but the problem is that you can't refer to identifiers declared in the plugin (the function's return type is a custom type defined in the plugin).
So a simple solution is to move the type declaration to another package, a common package that will be used by both the plugin and the main app (that loads the plugin).
Another solution is to declare your function to return an interface{} value so you can type assert this function, and you can call it, and you will obtain a value of type interface{}. Then your main app may define an interface type holding the methods you're interested in, and in the main app you can type assert to this interface type.
See details and examples here: go 1.8 plugin use custom interface
Also see related questions:
Is it possible to share a custom data type between a go plugin and an application?
Plugin symbol as function return

Typecasting one function into another in golang

I have the following code:
package vault
type Client interface {
GetHealth() error
}
func (c DefaultClient) GetHealth () error {
resp := &VaultHealthResponse{}
err := c.get(resp, "/v1/sys/health")
if err != nil {
return err
}
return nil;
}
Now, I want to use this function as part of this struct:
type DependencyHealthFunction func() error
type Dependency struct {
Name string `json:"name"`
Required bool `json:"required"`
Healthy bool `json:"healthy"`
Error error `json:"error,omitempty"`
HealthFunction DependencyHealthFunction
}
Basically, set the value of HealthFunction to GetHealth. Now, when I do the following:
func (config *Config) GetDependencies() *health.Dependency {
vaultDependency := health.Dependency{
Name: "Vault",
Required: true,
Healthy: true,
HealthFunction: vault.Client.GetHealth,
}
temp1 := &vaultDependency
return temp1;
}
This gives me an error and it says cannot use vault.Client.GetHealth (type func(vault.Client) error) as type health.DependencyHealthFunction in field value. How can I do this?
Edit: How DependencyHealthFunction is used?
As its part of Dependency struct, it's simply used as following: d.HealthFunction() where d is a variable of type *Dependency.
This is abstract:
HealthFunction: vault.Client.GetHealth,
If we were to call HealthFunction(), what code do you expect to run? vault.Client.GetHealth is just a promise that such a function exists; it isn't a function itself. Client is just an interface.
You need to create something that conforms to Client and pass its GetHealth. For example, if you had a existing DefaultClient such as:
defaultClient := DefaultClient{}
Then you could pass its function:
HealthFunction: defaultClient.GetHealth,
Now when you later call HealthFunction() it will be the same as calling defaultClient.GetHealth().
https://play.golang.org/p/9Lw7uc0GaE
I believe the issue is related to understanding how interfaces are treated in Go.
An interface simply defines a method or set of methods that a particular type must satisfy to be considered as "implementing" the interface.
For example:
import "fmt"
type Greeter interface {
SayHello() string
}
type EnglishGreeter struct{}
// Satisfaction of SayHello method
func (eg *EnglishGreeter) SayHello() string {
return "Hello"
}
type SpanishGreeter struct{}
func (sg *SpanishGreeter) SayHello() string {
return "Ola"
}
func GreetPerson(g Greeter) {
fmt.Println(g.SayHello())
}
func main() {
eg := &EnglishGreeter{}
sg := &SpanishGreeter{}
// greet person in english
GreetPerson(eg)
// greet person in spanish
GreetPerson(sg)
}
You can add this behavior into a custom struct by simply having a Greeter field inside the struct. ie
type FrontEntrance struct {
EntranceGreeter Greeter
}
fe := &FrontEntrance { EntranceGreeter: &EnglishGreeter{} }
// then call the SayHello() method like this
fe.EntranceGreeter.SayHello()
Interfaces in golang are useful for composing common expected behavior for types based on the methods that they satisfy.
Hope this helps.

Get struct from function type

I currently have the following code
func (r *Router) Get(path string, controller interface{}) {
...
t := reflect.TypeOf(controller)
...
}
That is called doing the following
Route.Get("/", controllers.Test.IsWorking)
The second argument is basically this
type Test struct {
*Controller
Name string
}
func (t Test) IsWorking() {
}
type Controller struct {
Method string
Request *http.Request
Response http.ResponseWriter
Data map[interface{}]interface{}
}
I want to get the struct the function refers to. create a new struct of that type and call the function so for example
controllers.Test.IsWorking
Create a Test struct and call IsWorking()
To make a new pointer to a newly allocated struct of the type in the interface, all you need is
newController := reflect.New(reflect.TypeOf(controller)).Interface()
Or to set a value on the new instance first:
newController := reflect.New(reflect.TypeOf(controller))
newController.Elem().FieldbyName("Method").Set(reflect.ValueOf("GET"))
If you want to create a pointer to a struct and call the IsWorking method, it looks like
t := reflect.New(reflect.TypeOf(Test{}))
t.MethodByName("IsWorking").Call(nil)

Getting websocket connection information in JSON-RPC method

I am using JSON-RPC over Websocket. And, in an RPC method (say, Multiply in the example below), I need to know which connection called this method. The part below that says "// Need Websocket connection information here". How do I do so?
package main
import (
"code.google.com/p/go.net/websocket"
"net/http"
"net/rpc"
"net/rpc/jsonrpc"
)
type Args struct {
A int
B int
}
type Arith int
func (t *Arith) Multiply(args *Args, reply *int) error {
*reply = args.A * args.B
// Need Websocket connection information here
return nil
}
func main() {
rpc.Register(new(Arith))
http.Handle("/conn", websocket.Handler(serve))
http.ListenAndServe("localhost:7000", nil)
}
func serve(ws *websocket.Conn) {
jsonrpc.ServeConn(ws)
}
This will be challenging because it violates the abstraction that RPC provides. Here's a strategy suggestion:
Google uses a context object for lots of their APIs: https://blog.golang.org/context. Part of that interface is a Value method for arbitrary data:
Value(key interface{}) interface{}
That will give you something like thread-local storage, which is often used for this purpose in other programming languages.
So how do you add a context object to the request? One way would be to create a new, custom ServerCodec:
type ServerCodec interface {
ReadRequestHeader(*Request) error
ReadRequestBody(interface{}) error
// WriteResponse must be safe for concurrent use by multiple goroutines.
WriteResponse(*Response, interface{}) error
Close() error
}
Your implementation can mostly mirror jsonrpc's:
var params [1]interface{}
params[0] = x
return json.Unmarshal(*c.req.Params, &params)
But before returning, you can use a bit of reflection and look for a field in params with name/type Context and then fill it. Something like:
ctx := createContextSomehow()
v := reflect.ValueOf(x)
if v.Kind() == reflect.Ptr {
v = v.Elem()
if v.Kind() == reflect.Struct {
ctxv := v.FieldByName("Context")
ctxv.Set(ctx)
}
}
Then change your request:
type Args struct {
A int
B int
}
Change it to:
type Args struct {
A int
B int
Context context.Context
}
Kinda clumsy but I think you could make that work.

Resources