Plugin symbol as function return - go

I'm running into a Go behaviour which I don't understand. My idea is to import a plugin which implements an interface that is out of both packages. If a struct is returned it works fine, but to be sure it implements the interface, I want to return an interface which fails.
Interface definition:
package iface
type IPlugin interface{
SayHello(string)
SayGoodby(string)
WhatsYourName() string
}
The main program looks like this:
package main
import (
"plugin"
"plugin_test/iface"
"errors"
"fmt"
)
//go:generate go build -buildmode=plugin -o ./pg/test.so ./pg/test.go
func main(){
path := "pg/test.so"
plug, err := plugin.Open(path)
if err != nil {
panic(err)
}
sym, err := plug.Lookup("Greeter")
if err != nil {
panic(err)
}
var pg iface.IPlugin
pg, ok := sym.(iface.IPlugin)
if !ok {
panic(errors.New("error binding plugin to interface"))
}
fmt.Printf("You're now connected to: %s \n", pg.WhatsYourName())
pg.SayHello("user")
pg.SayGoodby("user")
}
The plugin (stored in pg/test.go)
package main
import (
"fmt"
"plugin_test/iface"
)
type testpl struct {}
func(pl testpl) SayHello(s string){
fmt.Printf("Plugin says hello to %s \n", s)
}
func(pl testpl) SayGoodby(s string){
fmt.Printf("Plugin says goodby to %s \n", s)
}
func(pl testpl) WhatsYourName() string{
return "my name is Test-Plugin"
}
/* This function works */
func getPlugin() testpl{
return testpl{}
}
/* This function doesn't work */
func getPlugin() iface.IPlugin{
return testpl{}
}
/* This function also doesn't work */
func getPlugin() interface{}{
return testpl{}
}
var Greeter = getPlugin()
I tried every getPlugin function on its own.
The function returning testpl prints the expected output:
You're now connected to: my name is Test-Plugin
Plugin says hello to user
Plugin says goodby to user
The other functions end on sym.(iface.IPlugin)
panic: error binding plugin to interface
goroutine 1 [running]:
main.main()
/home/../../../main.go:27 +0x343
exit status 2
Can someone explain why this isn't possible? Wouldn't it be easier to create a plugin if it did't let you build your plugin in such a case?

What you want is possible, but there is something in the background that prevents it from working.
This is namely that you want to lookup a variable named Greeter from the plugin. Plugin.Lookup() will return a pointer to this variable! If it wouldn't, you could only inspect its value, but you couldn't change it.
You can verify this by simply printing the type of the value stored in sym:
fmt.Printf("%T\n", sym)
In your first case func getPlugin() testpl, output will be:
*main.testpl
In your second case func getPlugin() iface.IPlugin, output will be:
*iface.IPlugin
(Yes, it's a pointer to an interface!)
In your third case func getPlugin() interface{}, output will be:
*interface {}
So your first example works because the value stored in sym is of type *main.testpl, which also implements iface.IPlugin (because main.testpl implements it, so does the pointer type).
Back to your 2nd example: func getPlugin() iface.IPlugin
The value stored in sym is of type *iface.IPlugin. A value of pointer type to interface never satisfies any interfaces (except the empty interface), so attempting to type-assert iface.IPlugin from a value of type *iface.IPlugin will never succeed. You have to type assert *iface.IPlugin type, which you can dereference after to obtain a value of type iface.IPlugin. It could look like this:
pgPtr, ok := sym.(*iface.IPlugin)
if !ok {
panic(errors.New("error binding plugin to interface"))
}
pg := *pgPtr
And now everything works as expected!
To avoid such hassle and confusion, you may implement your plugin to expose a function which returns you the Greeter:
func Greeter() iface.IPlugin { return testpl{} }
And then get rid of the Greeter global var of course. If you do this, you may lookup the Greeter symbol which will be of type:
func() iface.IPlugin
The difference is that looking up a function does not require the plugin package to return a pointer to the value, while in case of a variable it does. A simple function type, no pointer-to-interface-kung-fu. Using it to obtain the greeter would be:
Greeter, err := p.Lookup("Greeter")
if err != nil {
panic(err)
}
greeterFunc, ok := GetFilter.(func() iface.IPlugin)
if !ok {
panic(errors.New("not of expected type"))
}
greeter := greeterFunc()
// And using it:
fmt.Printf("You're now connected to: %s \n", greeter.WhatsYourName())
greeter.SayHello("user")
greeter.SayGoodby("user")
See related / similar question: go 1.8 plugin use custom interface

Related

How to type-switch values with generic types [duplicate]

Just started learning generics. I'm making a command processor and I honestly don't know how to word this so I'm just going to show an example problem:
var ErrInvalidCommand = errors.New("invalid command")
type TransactionalFn[T any] func(ctx context.Context, db T) error
func NewTransactionalCommand[T any](fn TransactionalFn[T]) *TransactionalCommand[T] {
return &TransactionalCommand[T]{
fn: fn,
}
}
type TransactionalCommand[T any] struct {
fn TransactionalFn[T]
}
func (cmd *TransactionalCommand[T]) StartTransaction() error {
return nil
}
func (cmd *TransactionalCommand[T]) Commit() error {
return nil
}
func (cmd *TransactionalCommand[T]) Rollback() error {
return nil
}
type CMD interface{}
type CommandManager struct{}
func (m *CommandManager) Handle(ctx context.Context, cmd CMD) error {
switch t := cmd.(type) {
case *TransactionalCommand[any]:
return m.handleTransactionalCommand(ctx, t)
default:
fmt.Printf("%T\n", cmd)
return ErrInvalidCommand
}
}
func (m *CommandManager) handleTransactionalCommand(ctx context.Context, cmd *TransactionalCommand[any]) error {
if err := cmd.StartTransaction(); err != nil {
return err
}
if err := cmd.fn(ctx, nil); err != nil {
if err := cmd.Rollback(); err != nil {
return err
}
}
if err := cmd.Commit(); err != nil {
return err
}
return nil
}
// tests
type db struct{}
func (*db) Do() {
fmt.Println("doing stuff")
}
func TestCMD(t *testing.T) {
ctx := context.Background()
fn := func(ctx context.Context, db *db) error {
fmt.Println("test cmd")
db.Do()
return nil
}
tFn := bus.NewTransactionalCommand(fn)
mng := &bus.CommandManager{}
err := mng.Handle(ctx, tFn)
if err != nil {
t.Fatal(err)
}
}
mng.handle returns ErrInvalidCommand so the test fails because cmd is *TransactionalCommand[*db] and not *TransactionalCommand[any]
Let me give another, more abstract example:
type A[T any] struct{}
func (*A[T]) DoA() { fmt.Println("do A") }
type B[T any] struct{}
func (*B[T]) DoB() { fmt.Println("do B") }
func Handle(s interface{}) {
switch x := s.(type) {
case *A[any]:
x.DoA()
case *B[any]:
x.DoB()
default:
fmt.Printf("%T\n", s)
}
}
func TestFuncSwitch(t *testing.T) {
i := &A[int]{}
Handle(i) // expected to print "do A"
}
Why doesn't this switch statement case *A[any] match *A[int]?
How to make CommandManager.Handle(...) accept generic Commands?
Why does the generic type switch fail to compile?
This is in fact the result of an intentional decision of the Go team. It turned out that allowing type switches on parametrized types can cause confusion
In an earlier version of this design, we permitted using type assertions and type switches on variables whose type was a type parameter, or whose type was based on a type parameter. We removed this facility because it is always possible to convert a value of any type to the empty interface type, and then use a type assertion or type switch on that. Also, it was sometimes confusing that in a constraint with a type set that uses approximation elements, a type assertion or type switch would use the actual type argument, not the underlying type of the type argument (the difference is explained in the section on identifying the matched predeclared type)
From the Type Parameters Proposal
Let me turn the emphasized statement into code. If the type constraint uses type approximation (note the tildes)...
func PrintStringOrInt[T ~string | ~int](v T)
...and if there also was a custom type with int as the underlying type...
type Seconds int
...and if PrintOrString() is called with a Seconds parameter...
PrintStringOrInt(Seconds(42))
...then the switch block would not enter the int case but go right into the default case, because Seconds is not an int. Developers might expect that case int: matches the type Seconds as well.
To allow a case statement to match both Seconds and int would require a new syntax, like, for example,
case ~int:
As of this writing, the discussion is still open, and maybe it will result in an entirely new option for switching on a type parameter (such as, switch type T).
More details, please refer to proposal: spec: generics: type switch on parametric types
Trick: convert the type into 'any'
Luckily, we do not need to wait for this proposal to get implemented in a future release. There is a super simple workaround available right now.
Instead of switching on v.(type), switch on any(v).(type).
switch any(v).(type) {
...
This trick converts v into an empty interface{} (a.k.a. any), for which the switch happily does the type matching.
Source: A tip and a trick when working with generics
*A[any] does not match *A[int] because any is a static type, not a wildcard. Therefore instantiating a generic struct with different types yields different types.
In order to correctly match a generic struct in a type switch, you must instantiate it with a type parameter:
func Handle[T any](s interface{}) {
switch x := s.(type) {
case *A[T]:
x.DoA()
case *B[T]:
x.DoB()
default:
panic("no match")
}
}
Though in absence of other function arguments to infer T, you will have to call Handle with explicit instantiation. T won't be inferred from the struct alone.
func main() {
i := &A[int]{}
Handle[int](i) // expected to print "do A"
}
Playground: https://go.dev/play/p/2e5E9LSWPmk
However when Handle is actually a method, as in your database code, this has the drawback of choosing the type parameter when instantiating the receiver.
In order to improve the code here you can make Handle a top-level function:
func Handle[T any](ctx context.Context, cmd CMD) error {
switch t := cmd.(type) {
case *TransactionalCommand[T]:
return handleTransactionalCommand(ctx, t)
default:
fmt.Printf("%T\n", cmd)
return ErrInvalidCommand
}
}
Then you have the problem of how to supply the argument db T to the command function. For this, you might:
simply pass an additional *db argument to Handle and handleTransactionalCommand, which also helps with type parameter inference. Call as Handle(ctx, &db{}, tFn). Playground: https://go.dev/play/p/6WESb86KN5D
pass an instance of CommandManager (like solution above but *db is wrapped). Much more verbose, as it requires explicit instantiation everywhere. Playground: https://go.dev/play/p/SpXczsUM5aW
use a parametrized interface instead (like below). So you don't even have to type-switch. Playground: https://go.dev/play/p/EgULEIL6AV5
type CMD[T any] interface {
Exec(ctx context.Context, db T) error
}

Golang plugin type assertion

I'm writing a simple app which loads plugin in a predefined format. Example plugin is the following:
package main
import (
"errors"
"fmt"
"strings"
)
var (
ok bool
InvConfig = errors.New("invalid config")
)
type Processor struct {
logEverything bool
}
func (p *Processor) Init(config map[string]interface{}) error {
p.logEverything, ok = config["log_everything"].(bool)
if !ok {
return InvConfig
}
return nil
}
func (p *Processor) Process(buf []byte) []byte {
if p.logEverything {
fmt.Printf("Shouter got data: %v\n", buf)
}
return []byte(strings.ToUpper(string(buf)))
}
func GetProcessor() *Processor {
return &Processor{}
}
I can't quite apprehend how to load such a structure in my main program. So, I declare an interface:
type Processor interface {
Init(map[string]interface{}) error
Process(buf []byte) []byte
}
Then I load "getter" function and try to cast it to a function returning interface to then call it:
p, err := plugin.Open(filepath)
if err != nil {
logrus.Fatalf("Error opening plugin %s: %v", pluginName, err)
}
procGetterInter, err := p.Lookup("GetProcessor")
if err != nil {
logrus.Fatalf("Error loading processor getter for plugin %s: %v", pluginName, err)
}
procGetter, ok := procGetterInter.(func() interface{})
if !ok {
logrus.Fatalf("Error casting processor getter for plugin %s: %T", pluginName, procGetterInter)
}
But the cast fails with an error:
Error casting processor getter for plugin simple_shout: func() *main.Processor
If I return an actual instance (not a pointer) from GetProcessor and try to cast the function to the one returning Processor, I get the same result:
Error casting processor getter for plugin simple_shout: func() main.Processor
How to get a struct instance from plugin (therefore load the function returning it) and type-assert it's an expected interface in my case?
UPD: If I remove everything from Processor interface (that is, it becomes just an empty interface):
type Processor interface {}
And try to cast procGetterInter to a function returning a pointer to Processor interface:
procGetter, ok := procGetterInter.(func() *Processor)
I still get the same error:
plugin.Symbol is func() *main.Processor, not func() *main.Processor (types from different scopes)
Why doesn't it cast even to pointer to an empty interface?
TL;DR:
Check out a full working demo here: https://github.com/jvmatl/go-plugindemo
The long, but (hopefully!) informative answer:
Plugins are tricky in several ways, and #icza's answer is totally correct, but to understand why it's correct and how it applies to your question, you need to understand that the flexible nature of go's interfaces does not apply to complex types.
You have probably already run across this in other contexts:
This is legal in Go:
var a interface{}
var b int
a = b // yep, an int meets the spec for interface{} !
But this is not:
var aa []interface{}
var bb []int
aa = bb // cannot use bb (type []int) as type []interface {} in assignment
Similarly, with functions, this is legal:
type Runner interface {
Run()
}
type UsainBolt struct{}
func (ub *UsainBolt) Run() {
fmt.Println("Catch me if you can!")
}
var a Runner
var b *UsainBolt
a = b // Yep, a (pointer to) Usain Bolt is a runner!
But this is not:
var aa func() Runner
var bb func() *UsainBolt
aa = bb // cannot use bb (type func() *UsainBolt) as type func() Runner in assignment
Now let's look at defined function types. This is where it gets really interesting:
type RunnerGetter func() Runner
var rg RunnerGetter
rg = getUsain // <-- Nope: doesn't compile: "cannot use getUsain (type func() *UsainBolt) as type RunnerGetter in assignment"
rg = getRunner // <-- This *assignment* is allowed: getRunner is assignable to a type RunnerGetter
var i interface{} = getRunner
rg = i.(RunnerGetter) // compiles, but panics at runtime: "interface conversion: interface {} is func() main.Runner, not main.RunnerGetter"
In other words, the language is ok with assigning func getRunner() Runner to a variable of type RunnerGetter, but the type assertion fails, because the type assertion is asking: is this thing actually a variable of type RunnerGetter? And the answer is no, it's a func() Runner which is close, but not quite right, so we panic.
But this works:
var rg RunnerGetter
var i interface{}
i = rg // after this assignment, i *is* a RunnerGetter
rg = i.(RunnerGetter) // so this assertion passes.
Ok, with all that background out of the way, the issue is that the symbol you lookup from your plugin must be exactly the same type as your type assertion says it is, not just close-enough-to-allow-assignment.
As #icza stated, you have a couple of options:
Option 1: Quick and Dirty, gets the job done
In your plugin
func GetGeneric() interface{} {
return &Processor{}
}
In your main: (error-handling skipped for clarity)
p, _ := plugin.Open(pluginFile) // load plugin
newIntf, _ := p.Lookup("Getgeneric") // find symbol
newProc, _ := newIntf.(func() interface{}) // assert symbol to generic constructor
shoutProc, _ := newProc().(processors.Processor) // call generic constructor, type assert the return value
// Now use your new plugin!
shoutProc.Init(map[string]interface{}{"log_everything": true})
output := shoutProc.Process([]byte("whisper"))
Option 2: Cleaner, better if you have many plugins
Declare the interface all you plugins have to meet in another package:
package processors
// Every plugin must be able to give me something that meets this interface
type Processor interface {
Init(map[string]interface{}) error
Process(buf []byte) []byte
}
In your plugin:
type ShoutProcessor struct {
configured bool
logEverything bool
}
func NewProcessor() processors.Processor {
return &ShoutProcessor{}
}
In your main:
p, _ := plugin.Open(pluginFile) // load plugin
newProcIntf, _ := p.Lookup("NewProcessor") // lookup constructor
newProc, _ := newProcIntf.(func() processors.Processor) // assert the type of the func
shoutProc := newProc() // call the constructor, get a new ShoutProcessor
// ready to rock and roll!
shoutProc.Init(map[string]interface{}{"log_everything": true})
output := shoutProc.Process([]byte("whisper"))
The function inside the plugin has a signature:
func GetProcessor() *Processor
You lookup this symbol as an interface{} and you try to type-assert a value of type
func() interface{}
These types do not match because these function types have different return types. Spec: Function types:
A function type denotes the set of all functions with the same parameter and result types.
So you may only type assert the same function type, but the problem is that you can't refer to identifiers declared in the plugin (the function's return type is a custom type defined in the plugin).
So a simple solution is to move the type declaration to another package, a common package that will be used by both the plugin and the main app (that loads the plugin).
Another solution is to declare your function to return an interface{} value so you can type assert this function, and you can call it, and you will obtain a value of type interface{}. Then your main app may define an interface type holding the methods you're interested in, and in the main app you can type assert to this interface type.
See details and examples here: go 1.8 plugin use custom interface
Also see related questions:
Is it possible to share a custom data type between a go plugin and an application?
Plugin symbol as function return

Golang Parameters conversion

Can somebody explain how can this happened?
I put interface as parameter in a function. While invoking this function I pass struct into it, but it didn't give me error. Here's the code
package main
import (
"fmt"
"github.com/myusername/gomodel/domain"
"github.com/myusername/gomodel/model"
)
func main() {
db := model.InitDB()
newFunc(db)
}
func newFunc(db domain.IUser) {
r, err := db.CreateUserTable()
if err != nil {
fmt.Println("error", err)
}
fmt.Println(r)
}
I've implemented the interface somewhere else in the code, because the program just work as the implemented interface expected to be.
IUser is an interface whose member is:
type IUser interface {
CreateUserTable() (sql.Result, error)
}
InitDB is a function to open the database and return struct of database:
type DB struct {
*sql.DB
}
//InitDB initializes the database
func InitDB() *DB {
db, err := sql.Open(dbDriver, dbName)
if err != nil {
log.Fatal("failed to initialize database: ",err)
}
err2 := db.Ping()
if err2 != nil {
log.Fatal(err2)
}
return &DB{db}
}
My question is: how can a function with a parameter type interface be passed a different type of parameter? And how is this working under the hood?
As per Golang Spec
An interface type specifies a method set called its interface. A
variable of interface type can store a value of any type with a method
set that is any superset of the interface. Such a type is said to
implement the interface.
This is because interface can be implemented as a wrapper to every type. Interface actually points to two things mainly one is the underlying type which is a struct here and other one is the value of that type which is a pointer to DB
You see newFunc is actually taking interface{} as an argument, So you can pass anything to it of type T which can be of primitive types too.
func main() {
db := model.InitDB()
newFunc(db)
}
So In case you want to get the underlying value you need to type assert. Interface works like a wrapper to the struct here and save its type and value which can be get using type assertion.

Typecasting one function into another in golang

I have the following code:
package vault
type Client interface {
GetHealth() error
}
func (c DefaultClient) GetHealth () error {
resp := &VaultHealthResponse{}
err := c.get(resp, "/v1/sys/health")
if err != nil {
return err
}
return nil;
}
Now, I want to use this function as part of this struct:
type DependencyHealthFunction func() error
type Dependency struct {
Name string `json:"name"`
Required bool `json:"required"`
Healthy bool `json:"healthy"`
Error error `json:"error,omitempty"`
HealthFunction DependencyHealthFunction
}
Basically, set the value of HealthFunction to GetHealth. Now, when I do the following:
func (config *Config) GetDependencies() *health.Dependency {
vaultDependency := health.Dependency{
Name: "Vault",
Required: true,
Healthy: true,
HealthFunction: vault.Client.GetHealth,
}
temp1 := &vaultDependency
return temp1;
}
This gives me an error and it says cannot use vault.Client.GetHealth (type func(vault.Client) error) as type health.DependencyHealthFunction in field value. How can I do this?
Edit: How DependencyHealthFunction is used?
As its part of Dependency struct, it's simply used as following: d.HealthFunction() where d is a variable of type *Dependency.
This is abstract:
HealthFunction: vault.Client.GetHealth,
If we were to call HealthFunction(), what code do you expect to run? vault.Client.GetHealth is just a promise that such a function exists; it isn't a function itself. Client is just an interface.
You need to create something that conforms to Client and pass its GetHealth. For example, if you had a existing DefaultClient such as:
defaultClient := DefaultClient{}
Then you could pass its function:
HealthFunction: defaultClient.GetHealth,
Now when you later call HealthFunction() it will be the same as calling defaultClient.GetHealth().
https://play.golang.org/p/9Lw7uc0GaE
I believe the issue is related to understanding how interfaces are treated in Go.
An interface simply defines a method or set of methods that a particular type must satisfy to be considered as "implementing" the interface.
For example:
import "fmt"
type Greeter interface {
SayHello() string
}
type EnglishGreeter struct{}
// Satisfaction of SayHello method
func (eg *EnglishGreeter) SayHello() string {
return "Hello"
}
type SpanishGreeter struct{}
func (sg *SpanishGreeter) SayHello() string {
return "Ola"
}
func GreetPerson(g Greeter) {
fmt.Println(g.SayHello())
}
func main() {
eg := &EnglishGreeter{}
sg := &SpanishGreeter{}
// greet person in english
GreetPerson(eg)
// greet person in spanish
GreetPerson(sg)
}
You can add this behavior into a custom struct by simply having a Greeter field inside the struct. ie
type FrontEntrance struct {
EntranceGreeter Greeter
}
fe := &FrontEntrance { EntranceGreeter: &EnglishGreeter{} }
// then call the SayHello() method like this
fe.EntranceGreeter.SayHello()
Interfaces in golang are useful for composing common expected behavior for types based on the methods that they satisfy.
Hope this helps.

Return map like 'ok' in Golang on normal functions

In Go, the following works (note one use of the map has one return, the other has two returns)
package main
import "fmt"
var someMap = map[string]string { "some key": "hello" }
func main() {
if value, ok := someMap["some key"]; ok {
fmt.Println(value)
}
value := someMap["some key"]
fmt.Println(value)
}
However, I have no idea how to do this same thing with my own function. Is it possible to have similar behavior with an optional return like map?
For example:
package main
import "fmt"
func Hello() (string, bool) {
return "hello", true
}
func main() {
if value, ok := Hello(); ok {
fmt.Println(value)
}
value := Hello()
fmt.Println(value)
}
Wont compile (due to the error multiple-value Hello() in single-value context) ... is there a way to make this syntax work for the function Hello()?
map is different because it is a built-in type and not a function. The 2 forms of accessing an element of a map is specified by the Go Language Specification: Index Expressions and backed by the compiler.
With functions you can't do this. If a function has 2 return values, you have to "expect" both of them or none at all.
However you are allowed to assign any of the return values to the Blank identifier:
s, b := Hello() // Storing both of the return values
s2, _ := Hello() // Storing only the first
_, b3 := Hello() // Storing only the second
You can also choose not to store any of the return values:
Hello() // Just executing it, but storing none of the return values
Note: you could also assign both of the return values to the blank identifier, although it has no use (other than validating that it has exactly 2 return values):
_, _ = Hello() // Storing none of the return values; note the = instead of :=
You can also try these on the Go Playground.
Helper function
If you use it many times and you don't want to use the blank identifier, create a helper function which discards the 2nd return value:
func Hello2() string {
s, _ := Hello()
return s
}
And now you can do:
value := Hello2()
fmt.Println(value)
Go 1.18 generics update: Go 1.18 adds generics support, it is now possible to write a generic First() function which discards the second (or any further) return values:
func First[T any](first T, _ ...any) T {
return first
}
This is available in github.com/icza/gog, as gog.First() (disclosure: I'm the author).
Using it:
value := First(Hello())
fmt.Println(value)
In addition to the explanation of #icza:
I don't recommend using a helper function there. Especially if the Hello function is your own function.
However, if you can't control it, then it's fine to use a helper.
If it's your own function, it's better to change the signature of your function. Probably, you made a design mistake somewhere.
You can also do this:
package main
import "fmt"
func Hello() (string, bool) {
return "hello", true
}
func main() {
// Just move it one line above: don't use a short-if
value, ok := Hello()
if ok {
fmt.Println(value)
}
}

Resources