I'm writing a simple app which loads plugin in a predefined format. Example plugin is the following:
package main
import (
"errors"
"fmt"
"strings"
)
var (
ok bool
InvConfig = errors.New("invalid config")
)
type Processor struct {
logEverything bool
}
func (p *Processor) Init(config map[string]interface{}) error {
p.logEverything, ok = config["log_everything"].(bool)
if !ok {
return InvConfig
}
return nil
}
func (p *Processor) Process(buf []byte) []byte {
if p.logEverything {
fmt.Printf("Shouter got data: %v\n", buf)
}
return []byte(strings.ToUpper(string(buf)))
}
func GetProcessor() *Processor {
return &Processor{}
}
I can't quite apprehend how to load such a structure in my main program. So, I declare an interface:
type Processor interface {
Init(map[string]interface{}) error
Process(buf []byte) []byte
}
Then I load "getter" function and try to cast it to a function returning interface to then call it:
p, err := plugin.Open(filepath)
if err != nil {
logrus.Fatalf("Error opening plugin %s: %v", pluginName, err)
}
procGetterInter, err := p.Lookup("GetProcessor")
if err != nil {
logrus.Fatalf("Error loading processor getter for plugin %s: %v", pluginName, err)
}
procGetter, ok := procGetterInter.(func() interface{})
if !ok {
logrus.Fatalf("Error casting processor getter for plugin %s: %T", pluginName, procGetterInter)
}
But the cast fails with an error:
Error casting processor getter for plugin simple_shout: func() *main.Processor
If I return an actual instance (not a pointer) from GetProcessor and try to cast the function to the one returning Processor, I get the same result:
Error casting processor getter for plugin simple_shout: func() main.Processor
How to get a struct instance from plugin (therefore load the function returning it) and type-assert it's an expected interface in my case?
UPD: If I remove everything from Processor interface (that is, it becomes just an empty interface):
type Processor interface {}
And try to cast procGetterInter to a function returning a pointer to Processor interface:
procGetter, ok := procGetterInter.(func() *Processor)
I still get the same error:
plugin.Symbol is func() *main.Processor, not func() *main.Processor (types from different scopes)
Why doesn't it cast even to pointer to an empty interface?
TL;DR:
Check out a full working demo here: https://github.com/jvmatl/go-plugindemo
The long, but (hopefully!) informative answer:
Plugins are tricky in several ways, and #icza's answer is totally correct, but to understand why it's correct and how it applies to your question, you need to understand that the flexible nature of go's interfaces does not apply to complex types.
You have probably already run across this in other contexts:
This is legal in Go:
var a interface{}
var b int
a = b // yep, an int meets the spec for interface{} !
But this is not:
var aa []interface{}
var bb []int
aa = bb // cannot use bb (type []int) as type []interface {} in assignment
Similarly, with functions, this is legal:
type Runner interface {
Run()
}
type UsainBolt struct{}
func (ub *UsainBolt) Run() {
fmt.Println("Catch me if you can!")
}
var a Runner
var b *UsainBolt
a = b // Yep, a (pointer to) Usain Bolt is a runner!
But this is not:
var aa func() Runner
var bb func() *UsainBolt
aa = bb // cannot use bb (type func() *UsainBolt) as type func() Runner in assignment
Now let's look at defined function types. This is where it gets really interesting:
type RunnerGetter func() Runner
var rg RunnerGetter
rg = getUsain // <-- Nope: doesn't compile: "cannot use getUsain (type func() *UsainBolt) as type RunnerGetter in assignment"
rg = getRunner // <-- This *assignment* is allowed: getRunner is assignable to a type RunnerGetter
var i interface{} = getRunner
rg = i.(RunnerGetter) // compiles, but panics at runtime: "interface conversion: interface {} is func() main.Runner, not main.RunnerGetter"
In other words, the language is ok with assigning func getRunner() Runner to a variable of type RunnerGetter, but the type assertion fails, because the type assertion is asking: is this thing actually a variable of type RunnerGetter? And the answer is no, it's a func() Runner which is close, but not quite right, so we panic.
But this works:
var rg RunnerGetter
var i interface{}
i = rg // after this assignment, i *is* a RunnerGetter
rg = i.(RunnerGetter) // so this assertion passes.
Ok, with all that background out of the way, the issue is that the symbol you lookup from your plugin must be exactly the same type as your type assertion says it is, not just close-enough-to-allow-assignment.
As #icza stated, you have a couple of options:
Option 1: Quick and Dirty, gets the job done
In your plugin
func GetGeneric() interface{} {
return &Processor{}
}
In your main: (error-handling skipped for clarity)
p, _ := plugin.Open(pluginFile) // load plugin
newIntf, _ := p.Lookup("Getgeneric") // find symbol
newProc, _ := newIntf.(func() interface{}) // assert symbol to generic constructor
shoutProc, _ := newProc().(processors.Processor) // call generic constructor, type assert the return value
// Now use your new plugin!
shoutProc.Init(map[string]interface{}{"log_everything": true})
output := shoutProc.Process([]byte("whisper"))
Option 2: Cleaner, better if you have many plugins
Declare the interface all you plugins have to meet in another package:
package processors
// Every plugin must be able to give me something that meets this interface
type Processor interface {
Init(map[string]interface{}) error
Process(buf []byte) []byte
}
In your plugin:
type ShoutProcessor struct {
configured bool
logEverything bool
}
func NewProcessor() processors.Processor {
return &ShoutProcessor{}
}
In your main:
p, _ := plugin.Open(pluginFile) // load plugin
newProcIntf, _ := p.Lookup("NewProcessor") // lookup constructor
newProc, _ := newProcIntf.(func() processors.Processor) // assert the type of the func
shoutProc := newProc() // call the constructor, get a new ShoutProcessor
// ready to rock and roll!
shoutProc.Init(map[string]interface{}{"log_everything": true})
output := shoutProc.Process([]byte("whisper"))
The function inside the plugin has a signature:
func GetProcessor() *Processor
You lookup this symbol as an interface{} and you try to type-assert a value of type
func() interface{}
These types do not match because these function types have different return types. Spec: Function types:
A function type denotes the set of all functions with the same parameter and result types.
So you may only type assert the same function type, but the problem is that you can't refer to identifiers declared in the plugin (the function's return type is a custom type defined in the plugin).
So a simple solution is to move the type declaration to another package, a common package that will be used by both the plugin and the main app (that loads the plugin).
Another solution is to declare your function to return an interface{} value so you can type assert this function, and you can call it, and you will obtain a value of type interface{}. Then your main app may define an interface type holding the methods you're interested in, and in the main app you can type assert to this interface type.
See details and examples here: go 1.8 plugin use custom interface
Also see related questions:
Is it possible to share a custom data type between a go plugin and an application?
Plugin symbol as function return
Related
Just started learning generics. I'm making a command processor and I honestly don't know how to word this so I'm just going to show an example problem:
var ErrInvalidCommand = errors.New("invalid command")
type TransactionalFn[T any] func(ctx context.Context, db T) error
func NewTransactionalCommand[T any](fn TransactionalFn[T]) *TransactionalCommand[T] {
return &TransactionalCommand[T]{
fn: fn,
}
}
type TransactionalCommand[T any] struct {
fn TransactionalFn[T]
}
func (cmd *TransactionalCommand[T]) StartTransaction() error {
return nil
}
func (cmd *TransactionalCommand[T]) Commit() error {
return nil
}
func (cmd *TransactionalCommand[T]) Rollback() error {
return nil
}
type CMD interface{}
type CommandManager struct{}
func (m *CommandManager) Handle(ctx context.Context, cmd CMD) error {
switch t := cmd.(type) {
case *TransactionalCommand[any]:
return m.handleTransactionalCommand(ctx, t)
default:
fmt.Printf("%T\n", cmd)
return ErrInvalidCommand
}
}
func (m *CommandManager) handleTransactionalCommand(ctx context.Context, cmd *TransactionalCommand[any]) error {
if err := cmd.StartTransaction(); err != nil {
return err
}
if err := cmd.fn(ctx, nil); err != nil {
if err := cmd.Rollback(); err != nil {
return err
}
}
if err := cmd.Commit(); err != nil {
return err
}
return nil
}
// tests
type db struct{}
func (*db) Do() {
fmt.Println("doing stuff")
}
func TestCMD(t *testing.T) {
ctx := context.Background()
fn := func(ctx context.Context, db *db) error {
fmt.Println("test cmd")
db.Do()
return nil
}
tFn := bus.NewTransactionalCommand(fn)
mng := &bus.CommandManager{}
err := mng.Handle(ctx, tFn)
if err != nil {
t.Fatal(err)
}
}
mng.handle returns ErrInvalidCommand so the test fails because cmd is *TransactionalCommand[*db] and not *TransactionalCommand[any]
Let me give another, more abstract example:
type A[T any] struct{}
func (*A[T]) DoA() { fmt.Println("do A") }
type B[T any] struct{}
func (*B[T]) DoB() { fmt.Println("do B") }
func Handle(s interface{}) {
switch x := s.(type) {
case *A[any]:
x.DoA()
case *B[any]:
x.DoB()
default:
fmt.Printf("%T\n", s)
}
}
func TestFuncSwitch(t *testing.T) {
i := &A[int]{}
Handle(i) // expected to print "do A"
}
Why doesn't this switch statement case *A[any] match *A[int]?
How to make CommandManager.Handle(...) accept generic Commands?
Why does the generic type switch fail to compile?
This is in fact the result of an intentional decision of the Go team. It turned out that allowing type switches on parametrized types can cause confusion
In an earlier version of this design, we permitted using type assertions and type switches on variables whose type was a type parameter, or whose type was based on a type parameter. We removed this facility because it is always possible to convert a value of any type to the empty interface type, and then use a type assertion or type switch on that. Also, it was sometimes confusing that in a constraint with a type set that uses approximation elements, a type assertion or type switch would use the actual type argument, not the underlying type of the type argument (the difference is explained in the section on identifying the matched predeclared type)
From the Type Parameters Proposal
Let me turn the emphasized statement into code. If the type constraint uses type approximation (note the tildes)...
func PrintStringOrInt[T ~string | ~int](v T)
...and if there also was a custom type with int as the underlying type...
type Seconds int
...and if PrintOrString() is called with a Seconds parameter...
PrintStringOrInt(Seconds(42))
...then the switch block would not enter the int case but go right into the default case, because Seconds is not an int. Developers might expect that case int: matches the type Seconds as well.
To allow a case statement to match both Seconds and int would require a new syntax, like, for example,
case ~int:
As of this writing, the discussion is still open, and maybe it will result in an entirely new option for switching on a type parameter (such as, switch type T).
More details, please refer to proposal: spec: generics: type switch on parametric types
Trick: convert the type into 'any'
Luckily, we do not need to wait for this proposal to get implemented in a future release. There is a super simple workaround available right now.
Instead of switching on v.(type), switch on any(v).(type).
switch any(v).(type) {
...
This trick converts v into an empty interface{} (a.k.a. any), for which the switch happily does the type matching.
Source: A tip and a trick when working with generics
*A[any] does not match *A[int] because any is a static type, not a wildcard. Therefore instantiating a generic struct with different types yields different types.
In order to correctly match a generic struct in a type switch, you must instantiate it with a type parameter:
func Handle[T any](s interface{}) {
switch x := s.(type) {
case *A[T]:
x.DoA()
case *B[T]:
x.DoB()
default:
panic("no match")
}
}
Though in absence of other function arguments to infer T, you will have to call Handle with explicit instantiation. T won't be inferred from the struct alone.
func main() {
i := &A[int]{}
Handle[int](i) // expected to print "do A"
}
Playground: https://go.dev/play/p/2e5E9LSWPmk
However when Handle is actually a method, as in your database code, this has the drawback of choosing the type parameter when instantiating the receiver.
In order to improve the code here you can make Handle a top-level function:
func Handle[T any](ctx context.Context, cmd CMD) error {
switch t := cmd.(type) {
case *TransactionalCommand[T]:
return handleTransactionalCommand(ctx, t)
default:
fmt.Printf("%T\n", cmd)
return ErrInvalidCommand
}
}
Then you have the problem of how to supply the argument db T to the command function. For this, you might:
simply pass an additional *db argument to Handle and handleTransactionalCommand, which also helps with type parameter inference. Call as Handle(ctx, &db{}, tFn). Playground: https://go.dev/play/p/6WESb86KN5D
pass an instance of CommandManager (like solution above but *db is wrapped). Much more verbose, as it requires explicit instantiation everywhere. Playground: https://go.dev/play/p/SpXczsUM5aW
use a parametrized interface instead (like below). So you don't even have to type-switch. Playground: https://go.dev/play/p/EgULEIL6AV5
type CMD[T any] interface {
Exec(ctx context.Context, db T) error
}
I have some questions about interfaces, especially when these interfaces are passed between functions.
I understand that interfaces are satisfied implicitly, meaning the following code is valid:
type itemX struct {}
func (x *itemX) Do() string {
return "itemX"
}
type Itf interface {
Do() string
}
func test(i Itf) string {
return i.Do()
}
func main() {
x := new(itemX)
str := test(x) // valid, since x implicitly satisfies Itf
}
However, it is not so clear what happens or what the type contract is like when I start passing interfaces between functions. An example:
// itemX, Itf, and test have the same declaration as the above snippet
func returnsItf(i Itf) Itf {
return i
}
func returnsTypeAssertedX(i Itf) Itf {
return i.(*itemX)
}
func takeItf(i Itf) {}
func takeX(x *itemX) {}
func main() {
x := new(itemX)
var i Itf = x
a := returnsItf(i) // returns type Itf
_ = takeItf(a) // no error
b := returnsTypeAssertedX(i)
_ = takeItf(b) // no error, since *itemX implements Itf
_ = takeX(b) // error, cannot use b (type Itf as *itemX)
}
There seems to be some hidden behavior when an interface is passed out as a function return. If the return is *itemX and type is Itf, the return is transformed into Itf before the function frame is terminated.
So, this implicit check (concrete -> interface if type is interface) is done twice per function call:
at the start of each function call,
and at the end.
Is my understanding of this implicit transformation correct?
An interface is a data type that has two members: The type of the underlying object, and a pointer to that object. So, wen you use a non-interface type in a context that needs an interface, the compiler constructs an interface type from that value, and uses that.
func returnsTypeAssertedX(i Itf) Itf {
return i.(*itemX)
}
In the above function, it first type-asserts that the passed in argument is of the required type, and then converts the underlying value of the argument back to an interface.
b := returnsTypeAssertedX(i)
_ = takeX(b)
The above will not work, because b is an interface{}, and takeX requires an *itemX. However, this would work:
takeX(b.(*itemX))
My code uses the interface with one function:
func InsertRow(rec []string) error
There are different types with different implementation of this interface. Now I would like to test this using "go test". In this case the implementation of InsertRow should do nothing:
func (t TestInserter) InsertRow(rec []string) error {
return nil
}
I can define a type internal in a test function. But now I would also like to define a dummy method for this type:
func TestInserter01(t *testing.T) {
type TestMyInserter struct {} <-- Test type
func (t TestMyInserter) InsertRow(rec []string) error { <-- Dummy function on this type.
return nil
}
... using InsertRow as a parameter in another function ...
}
But this produces compile errors:
expected operand, found ']'
expected ']', found 'return'
The same code works, if I define both the type and the method outside the test function.
Is it possible to hide the test implementation in the test function and do not define it outside the function? I need many of them, so that I would prefer to have them defined locally in the test function.
No, it's not possible. Method declarations may only be at the top level (outside of any function).
Spec: Declarations and scope:
Declaration = ConstDecl | TypeDecl | VarDecl .
TopLevelDecl = Declaration | FunctionDecl | MethodDecl .
See related: Is it possible to define an anonymous interface implementation in Go?
Note howerer that it's possible to supply "dynamic" implementations with a helper type. Meaning you'll provide the method implementations inside a function, and with the help of a helper type that implements the interface, you can get a "dynamic" implementation.
For example:
type Inserter interface {
InsertRow(rec []string) error
}
type helper func(rec []string) error
func (h helper) InsertRow(rec []string) error {
return h(rec)
}
func main() {
testInsert := func(rec []string) error {
return fmt.Errorf("rec: %v", rec)
}
var i Inserter = helper(testInsert)
err := i.InsertRow([]string{"one", "two"})
fmt.Println(err)
}
This will output (try it on the Go Playground):
rec: [one two]
A variant may be a struct holding fields of function types for the methods. It may be used to cover multiple methods:
type helper struct {
insertRow func(rec []string) error
}
func (h helper) InsertRow(rec []string) error {
return h.insertRow(rec)
}
func main() {
h := helper{
insertRow: func(rec []string) error {
return fmt.Errorf("rec: %v", rec)
},
}
var i Inserter = h
err := i.InsertRow([]string{"one", "two"})
fmt.Println(err)
}
This outputs the same. Try it on the Go Playground.
I'm running into a Go behaviour which I don't understand. My idea is to import a plugin which implements an interface that is out of both packages. If a struct is returned it works fine, but to be sure it implements the interface, I want to return an interface which fails.
Interface definition:
package iface
type IPlugin interface{
SayHello(string)
SayGoodby(string)
WhatsYourName() string
}
The main program looks like this:
package main
import (
"plugin"
"plugin_test/iface"
"errors"
"fmt"
)
//go:generate go build -buildmode=plugin -o ./pg/test.so ./pg/test.go
func main(){
path := "pg/test.so"
plug, err := plugin.Open(path)
if err != nil {
panic(err)
}
sym, err := plug.Lookup("Greeter")
if err != nil {
panic(err)
}
var pg iface.IPlugin
pg, ok := sym.(iface.IPlugin)
if !ok {
panic(errors.New("error binding plugin to interface"))
}
fmt.Printf("You're now connected to: %s \n", pg.WhatsYourName())
pg.SayHello("user")
pg.SayGoodby("user")
}
The plugin (stored in pg/test.go)
package main
import (
"fmt"
"plugin_test/iface"
)
type testpl struct {}
func(pl testpl) SayHello(s string){
fmt.Printf("Plugin says hello to %s \n", s)
}
func(pl testpl) SayGoodby(s string){
fmt.Printf("Plugin says goodby to %s \n", s)
}
func(pl testpl) WhatsYourName() string{
return "my name is Test-Plugin"
}
/* This function works */
func getPlugin() testpl{
return testpl{}
}
/* This function doesn't work */
func getPlugin() iface.IPlugin{
return testpl{}
}
/* This function also doesn't work */
func getPlugin() interface{}{
return testpl{}
}
var Greeter = getPlugin()
I tried every getPlugin function on its own.
The function returning testpl prints the expected output:
You're now connected to: my name is Test-Plugin
Plugin says hello to user
Plugin says goodby to user
The other functions end on sym.(iface.IPlugin)
panic: error binding plugin to interface
goroutine 1 [running]:
main.main()
/home/../../../main.go:27 +0x343
exit status 2
Can someone explain why this isn't possible? Wouldn't it be easier to create a plugin if it did't let you build your plugin in such a case?
What you want is possible, but there is something in the background that prevents it from working.
This is namely that you want to lookup a variable named Greeter from the plugin. Plugin.Lookup() will return a pointer to this variable! If it wouldn't, you could only inspect its value, but you couldn't change it.
You can verify this by simply printing the type of the value stored in sym:
fmt.Printf("%T\n", sym)
In your first case func getPlugin() testpl, output will be:
*main.testpl
In your second case func getPlugin() iface.IPlugin, output will be:
*iface.IPlugin
(Yes, it's a pointer to an interface!)
In your third case func getPlugin() interface{}, output will be:
*interface {}
So your first example works because the value stored in sym is of type *main.testpl, which also implements iface.IPlugin (because main.testpl implements it, so does the pointer type).
Back to your 2nd example: func getPlugin() iface.IPlugin
The value stored in sym is of type *iface.IPlugin. A value of pointer type to interface never satisfies any interfaces (except the empty interface), so attempting to type-assert iface.IPlugin from a value of type *iface.IPlugin will never succeed. You have to type assert *iface.IPlugin type, which you can dereference after to obtain a value of type iface.IPlugin. It could look like this:
pgPtr, ok := sym.(*iface.IPlugin)
if !ok {
panic(errors.New("error binding plugin to interface"))
}
pg := *pgPtr
And now everything works as expected!
To avoid such hassle and confusion, you may implement your plugin to expose a function which returns you the Greeter:
func Greeter() iface.IPlugin { return testpl{} }
And then get rid of the Greeter global var of course. If you do this, you may lookup the Greeter symbol which will be of type:
func() iface.IPlugin
The difference is that looking up a function does not require the plugin package to return a pointer to the value, while in case of a variable it does. A simple function type, no pointer-to-interface-kung-fu. Using it to obtain the greeter would be:
Greeter, err := p.Lookup("Greeter")
if err != nil {
panic(err)
}
greeterFunc, ok := GetFilter.(func() iface.IPlugin)
if !ok {
panic(errors.New("not of expected type"))
}
greeter := greeterFunc()
// And using it:
fmt.Printf("You're now connected to: %s \n", greeter.WhatsYourName())
greeter.SayHello("user")
greeter.SayGoodby("user")
See related / similar question: go 1.8 plugin use custom interface
I have the following code:
package vault
type Client interface {
GetHealth() error
}
func (c DefaultClient) GetHealth () error {
resp := &VaultHealthResponse{}
err := c.get(resp, "/v1/sys/health")
if err != nil {
return err
}
return nil;
}
Now, I want to use this function as part of this struct:
type DependencyHealthFunction func() error
type Dependency struct {
Name string `json:"name"`
Required bool `json:"required"`
Healthy bool `json:"healthy"`
Error error `json:"error,omitempty"`
HealthFunction DependencyHealthFunction
}
Basically, set the value of HealthFunction to GetHealth. Now, when I do the following:
func (config *Config) GetDependencies() *health.Dependency {
vaultDependency := health.Dependency{
Name: "Vault",
Required: true,
Healthy: true,
HealthFunction: vault.Client.GetHealth,
}
temp1 := &vaultDependency
return temp1;
}
This gives me an error and it says cannot use vault.Client.GetHealth (type func(vault.Client) error) as type health.DependencyHealthFunction in field value. How can I do this?
Edit: How DependencyHealthFunction is used?
As its part of Dependency struct, it's simply used as following: d.HealthFunction() where d is a variable of type *Dependency.
This is abstract:
HealthFunction: vault.Client.GetHealth,
If we were to call HealthFunction(), what code do you expect to run? vault.Client.GetHealth is just a promise that such a function exists; it isn't a function itself. Client is just an interface.
You need to create something that conforms to Client and pass its GetHealth. For example, if you had a existing DefaultClient such as:
defaultClient := DefaultClient{}
Then you could pass its function:
HealthFunction: defaultClient.GetHealth,
Now when you later call HealthFunction() it will be the same as calling defaultClient.GetHealth().
https://play.golang.org/p/9Lw7uc0GaE
I believe the issue is related to understanding how interfaces are treated in Go.
An interface simply defines a method or set of methods that a particular type must satisfy to be considered as "implementing" the interface.
For example:
import "fmt"
type Greeter interface {
SayHello() string
}
type EnglishGreeter struct{}
// Satisfaction of SayHello method
func (eg *EnglishGreeter) SayHello() string {
return "Hello"
}
type SpanishGreeter struct{}
func (sg *SpanishGreeter) SayHello() string {
return "Ola"
}
func GreetPerson(g Greeter) {
fmt.Println(g.SayHello())
}
func main() {
eg := &EnglishGreeter{}
sg := &SpanishGreeter{}
// greet person in english
GreetPerson(eg)
// greet person in spanish
GreetPerson(sg)
}
You can add this behavior into a custom struct by simply having a Greeter field inside the struct. ie
type FrontEntrance struct {
EntranceGreeter Greeter
}
fe := &FrontEntrance { EntranceGreeter: &EnglishGreeter{} }
// then call the SayHello() method like this
fe.EntranceGreeter.SayHello()
Interfaces in golang are useful for composing common expected behavior for types based on the methods that they satisfy.
Hope this helps.