I'm porting an app from Play (Scala) to Go and wondering how to implement dependency injection. In Scala I used the cake pattern, while in Go I implemented a DAO interface along with an implementation for Mongo.
Here below is how I tried to implement a pattern that let me change the DAO implementation as needed (e.g. test, different DB, etc.):
1. entity.go
package models
import (
"time"
"gopkg.in/mgo.v2/bson"
)
type (
Entity struct {
Id bson.ObjectId `json:"id,omitempty" bson:"_id,omitempty"`
CreatedAt time.Time `json:"createdAt,omitempty" bson:"createdAt,omitempty"`
LastUpdate time.Time `json:"lastUpdate,omitempty" bson:"lastUpdate,omitempty"`
}
)
2. user.go
package models
import (
"time"
)
type (
User struct {
Entity `bson:",inline"`
Name string `json:"name,omitempty" bson:"name,omitempty"`
BirthDate time.Time `json:"birthDate,omitempty" bson:"birthDate,omitempty"`
}
)
3. dao.go
package persistence
type (
DAO interface {
Insert(entity interface{}) error
List(result interface{}, sort string) error
Find(id string, result interface{}) error
Update(id string, update interface{}) error
Remove(id string) error
Close()
}
daoFactory func() DAO
)
var (
New daoFactory
)
4. mongoDao.go (DB info and collection name are hard-coded since it's just an example)
package persistence
import (
"fmt"
"time"
"errors"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
"github.com/fatih/structs"
"cmd/server/models"
)
type (
mongoDAO struct{
session *mgo.Session
}
)
func NewMongoDAO() DAO {
dialInfo := &mgo.DialInfo{
Addrs: []string{"localhost:27017"},
Timeout: 60 * time.Second,
Database: "test",
}
session, err := mgo.DialWithInfo(dialInfo)
if err != nil {
panic(err)
}
session.SetMode(mgo.Monotonic, true)
return &mongoDAO{session}
}
func (dao *mongoDAO) Insert(entity interface{}) error {
doc := entity.(*models.User)
doc.Id = bson.NewObjectId()
doc.CreatedAt = time.Now().UTC()
doc.LastUpdate = time.Now().UTC()
return dao.session.DB("test").C("users").Insert(doc)
}
func (dao *mongoDAO) List(result interface{}, sort string) error {
return dao.session.DB("test").C("users").Find(nil).Sort(sort).All(result)
}
func (dao *mongoDAO) Find(id string, result interface{}) error {
if !bson.IsObjectIdHex(id) {
return errors.New(fmt.Sprintf("%s is not a valid hex id", id))
}
oid := bson.ObjectIdHex(id)
return dao.session.DB("test").C("users").FindId(oid).One(result)
}
func (dao *mongoDAO) Update(id string, update interface{}) error {
if !bson.IsObjectIdHex(id) {
return errors.New(fmt.Sprintf("%s is not a valid hex id", id))
}
oid := bson.ObjectIdHex(id)
doc := update.(*models.User)
doc.LastUpdate = time.Now().UTC()
return dao.session.DB("test").C("users").Update(oid, bson.M{"$set": structs.Map(update)})
}
func (dao *mongoDAO) Remove(id string) error {
if !bson.IsObjectIdHex(id) {
return errors.New(fmt.Sprintf("%s is not a valid hex id", id))
}
oid := bson.ObjectIdHex(id)
return dao.session.DB("test").C("users").RemoveId(oid)
}
func (dao *mongoDAO) Close() {
dao.session.Close()
}
func init() {
New = NewMongoDAO
}
Finally, here is how I use the types above:
5. userController.go
package controllers
import (
"net/http"
"github.com/labstack/echo"
"cmd/server/models"
"cmd/server/persistence"
)
type (
UserController struct {
dao persistence.DAO
}
)
func NewUserController(dao persistence.DAO) *UserController {
return &UserController{dao}
}
func (userController *UserController) CreateUser() echo.HandlerFunc {
return func(context echo.Context) error {
user := &models.User{}
if err := context.Bind(user); err != nil {
return err
}
if err := userController.dao.Insert(user); err != nil {
return err
}
return context.JSON(http.StatusCreated, user)
}
}
func (userController *UserController) UpdateUser() echo.HandlerFunc {
return func(context echo.Context) error {
user := &models.User{}
if err := context.Bind(user); err != nil {
return err
}
id := context.Param("id")
if err := userController.dao.Update(id, user); err != nil {
return err
}
return context.JSON(http.StatusOK, user)
}
}
....
The code above is 90% fine... I've just a problem in mongoDao.go with methods Insert and Update where the compiler forces me to cast input entity to a specific type (*models.User), but this prevents me from having a generic DAO component that works for all types. How do I fix this issue?
How about creating an interface that you implement for the Entity struct?
type Entitier interface {
GetEntity() *Entity
}
The implementation would simply return a pointer to itself that you can now use in the Insert and Update methods of your DAO. This would also have the added benefit of letting you be more specific in the declarations of your DAO methods. Instead of simply stating that they take an arbitrary interface{} as argument you could now say that they take an Entitier.
Like so:
func (dao *mongoDAO) Update(id string, update Entitier) error
Here's a minimal complete example of what I mean:
http://play.golang.org/p/lpVs_61mfM
Hope this gives you some ideas! You might want to adjust naming of Entity/Entitier/GetEntity for style and clarity once you've settled on the pattern to use.
This generalization
DAO interface {
Insert(entity interface{}) error
looks over-helming
You both assert to *models.User for mongo
doc := entity.(*models.User)
and do
user := &models.User{}
userController.dao.Insert(user)
when use your generic DAO interface.
Why don't you just define interface more precisely?
DAO interface {
Insert(entity *models.User) error
Related
I try to implement some unit testing in my go code and find the topic of mocking method quite difficult.
I have the following example where I hope you can help me :)
On the first layer I have the following code:
package api
import (
"fmt"
"core"
)
type createUserDTO struct {
Id string
}
func ApiMethod() {
fmt.Println("some incoming api call wit user")
incomingUserData := &createUserDTO{Id: "testId"}
mapedUser := incomingUserData.mapUser()
mapedUser.Create()
}
func (createUserDTO *createUserDTO) mapUser() core.User {
return &core.UserCore{Id: createUserDTO.Id}
}
The second layer has the following code:
package core
import (
"fmt"
)
type CoreUser struct{ Id string }
type User interface {
Create()
}
func (user CoreUser) Create() {
fmt.Println("Do Stuff")
}
My question now is, how do I test every public method in the api package without testing the core package. Especially the method Create().
Based on the comments, I put together a trivial GitHub repository to show how I usually deal with structuring projects in Go. The repository doesn't take into consideration the test part for now but it should be pretty easy to insert them with this project structure.
Let's start with the general folders' layout:
controllers
services
db
dto
models
Now, let's see the relevant files.
models/user.go
package models
import "gorm.io/gorm"
type User struct {
*gorm.Model
Id string `gorm:"primaryKey"`
}
func NewUser(id string) *User {
return &User{Id: id}
}
Simple struct definition here.
dto/user.go
package dto
import "time"
type UserDTO struct {
Id string `json:"id"`
AddedOn time.Time `json:"added_on"`
}
func NewUserDTO(id string) *UserDTO {
return &UserDTO{Id: id}
}
We enrich the model struct with a dummy AddedOn field which needs only for the sake of the demo.
db/user.go
package db
import (
"gorm.io/gorm"
"userapp/models"
)
type UserDb struct {
Conn *gorm.DB
}
type UserDbInterface interface {
SaveUser(user *models.User) error
}
func (u *UserDb) SaveUser(user *models.User) error {
if dbTrn := u.Conn.Create(user); dbTrn.Error != nil {
return dbTrn.Error
}
return nil
}
Here, we define an interface for using the User repository. In our tests, we can provide a mock instead of an instance of the UserDb struct.
services/user.go
package services
import (
"time"
"userapp/db"
"userapp/dto"
"userapp/models"
)
type UserService struct {
DB db.UserDbInterface
}
type UserServiceInterface interface {
AddUser(inputReq *dto.UserDTO) (*dto.UserDTO, error)
}
func NewUserService(db db.UserDbInterface) *UserService {
return &UserService{
DB: db,
}
}
func (u *UserService) AddUser(inputReq *dto.UserDTO) (*dto.UserDTO, error) {
// here you can write complex logic
user := models.NewUser(inputReq.Id)
// invoke db repo
if err := u.DB.SaveUser(user); err != nil {
return nil, err
}
inputReq.AddedOn = time.Now()
return inputReq, nil
}
This is the layer that bridges connections between the presentation layer and the underlying repositories.
controllers/user.go
package controllers
import (
"encoding/json"
"io"
"net/http"
"userapp/dto"
"userapp/services"
)
type UserController struct {
US services.UserServiceInterface
}
func NewUserController(userService services.UserServiceInterface) *UserController {
return &UserController{
US: userService,
}
}
func (u *UserController) Save(w http.ResponseWriter, r *http.Request) {
reqBody, err := io.ReadAll(r.Body)
if err != nil {
panic(err)
}
r.Body.Close()
var userReq dto.UserDTO
json.Unmarshal(reqBody, &userReq)
userRes, err := u.US.AddUser(&userReq)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
json.NewEncoder(w).Encode(err)
return
}
w.WriteHeader(http.StatusCreated)
json.NewEncoder(w).Encode(userRes)
}
Here, we defined the controller that (through Dependency Injection) uses the UserService struct.
You can find everything in my repository on GitHub
Let me know if it clarifies a little bit.
I am using Go 1.19 on a windows machine with 8 cores, operating system is Windows 10 Pro.
I used the mockgen tool to generate the mock. When I debug my test I see the mocked method is recorded when I execute the EXPECT() function.
The mocked function is called, but the test fails with 'missing call' on the mocked function.
I cannot see what I am doing wrong, can anyone please point it out ?
Directory Structure :
cmd
configure.go
configure_test.go
mocks
mock_validator.go
validator
validator.go
user
user.go
go.mod
main.go
* Contents of main.go
package main
import (
"localdev/mockexample/cmd"
)
func main() {
cmd.Configure()
}
* Contents of configure.go
package cmd
import (
"fmt"
"localdev/mockexample/user"
"os"
"localdev/mockexample/validator"
)
var (
name, password string
)
func Configure() {
name := os.Args[1]
password := os.Args[2]
user, err := validate(validator.NewValidator(name, password))
if err != nil {
fmt.Printf("%v\n", err)
return
}
fmt.Printf("Credentials are valid. Welcome: %s %s\n", user.FirstName, user.LastName)
}
func validate(validator validator.Validator) (*user.Data, error) {
user, err := validator.ValidateUser()
if err != nil {
return nil, fmt.Errorf("some thing went wrong. %v", err)
}
return user, nil
}
* Contents of validator.go
package validator
import (
"fmt"
"localdev/mockexample/user"
)
//go:generate mockgen -destination=../mocks/mock_validator.go -package=mocks localdev/mockexample/validator Validator
type Validator interface {
ValidateUser() (*user.Data, error)
}
type ValidationRequest struct {
Command string
Name string
Password string
}
func (vr ValidationRequest) ValidateUser() (*user.Data, error) {
if vr.Name == "bob" && vr.Password == "1234" {
return &user.Data{UserID: "123", UserName: "bsmith", FirstName: "Bob", LastName: "Smith"}, nil
}
return nil, fmt.Errorf("invalid credentials")
}
func NewValidator(name string, password string) Validator {
return &ValidationRequest{Name: name, Password: password}
}
* Contents of user.go
package user
type Data struct {
UserID string `json:"user_id"`
UserName string `json:"user_name"`
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
}
* Contents of configure_test.go
package cmd
import (
"localdev/mockexample/mocks"
"localdev/mockexample/user"
"os"
"testing"
"github.com/golang/mock/gomock"
)
func TestConfigure(t *testing.T) {
t.Run("ConfigureWithMock", func(t *testing.T) {
os.Args[1] = "bob"
os.Args[2] = "1234"
ctrl := gomock.NewController(t)
mockValidator := mocks.NewMockValidator(ctrl)
//mockValidator.EXPECT().ValidateUser().AnyTimes() // zero more calls, so this will also pass.
userData := user.Data{UserID: "testId"}
mockValidator.EXPECT().ValidateUser().Return(&userData, nil).Times(1) //(gomock.Any(), gomock.Any()) //(&userData, nil)
Configure()
})
}
Contents of generated mock
// Code generated by MockGen. DO NOT EDIT.
// Source: localdev/mockexample/validator (interfaces: Validator)
// Package mocks is a generated GoMock package.
package mocks
import (
user "localdev/mockexample/user"
reflect "reflect"
gomock "github.com/golang/mock/gomock"
)
// MockValidator is a mock of Validator interface.
type MockValidator struct {
ctrl *gomock.Controller
recorder *MockValidatorMockRecorder
}
// MockValidatorMockRecorder is the mock recorder for MockValidator.
type MockValidatorMockRecorder struct {
mock *MockValidator
}
// NewMockValidator creates a new mock instance.
func NewMockValidator(ctrl *gomock.Controller) *MockValidator {
mock := &MockValidator{ctrl: ctrl}
mock.recorder = &MockValidatorMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use.
func (m *MockValidator) EXPECT() *MockValidatorMockRecorder {
return m.recorder
}
// ValidateUser mocks base method.
func (m *MockValidator) ValidateUser() (*user.Data, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "ValidateUser")
ret0, _ := ret[0].(*user.Data)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// ValidateUser indicates an expected call of ValidateUser.
func (mr *MockValidatorMockRecorder) ValidateUser() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ValidateUser", reflect.TypeOf((*MockValidator)(nil).ValidateUser))
}
The root problem is that the function Configure never uses the mock structure, so you get a missing call(s) to *mocks.MockValidator.ValidateUser() error.
In the file configure_test.go, mockValidator is simply not used at all. There must be some kind of injection of that mock in order to be called by the Configure function.
You could make the following changes to fix the test, as an example of what I'm referring to injection. Not saying this is the best approach but I'm trying to make the fewer possible changes to your code.
configure_test.go:
func TestConfigure(t *testing.T) {
t.Run("ConfigureWithMock", func(t *testing.T) {
os.Args[1] = "bob"
os.Args[2] = "1234"
ctrl := gomock.NewController(t)
mockValidator := mocks.NewMockValidator(ctrl)
//mockValidator.EXPECT().ValidateUser().AnyTimes() // zero more calls, so this will also pass.
userData := user.Data{UserID: "testId"}
mockValidator.
EXPECT().
ValidateUser("bob", "1234").
Return(&userData, nil).
Times(1) //(gomock.Any(), gomock.Any()) //(&userData, nil)
Configure(mockValidator)
})
}
configure.go
func Configure(v validator.Validator) {
name := os.Args[1]
password := os.Args[2]
user, err := v.ValidateUser(name, password)
if err != nil {
fmt.Printf("some thing went wrong. %v\n", err)
return
}
fmt.Printf("Credentials are valid. Welcome: %s %s\n", user.FirstName, user.LastName)
}
validator.go
type Validator interface {
ValidateUser(name, password string) (*user.Data, error)
}
type ValidationRequest struct {
Command string
// Name string
// Password string
}
func (vr ValidationRequest) ValidateUser(name, password string) (*user.Data, error) {
if name == "bob" && password == "1234" {
return &user.Data{UserID: "123", UserName: "bsmith", FirstName: "Bob", LastName: "Smith"}, nil
}
return nil, fmt.Errorf("invalid credentials")
}
func NewValidator() Validator {
return &ValidationRequest{}
}
Take into account that you need to generate the mock again. Hope this helps you to understand mock testing.
I have this code and I wanna write a unit tests for update function.
how can i mock FindByUsername function ?
I try to overwrite u.FindByUsername but it's doesn't work.
also, I can write some function to give u *UserLogic and userName string as input parameters and execute u.FindByUsername() and mock this function but it's not a clean solution I need a better solution for mocking methods inside UserOperation interface.
package logic
import (
"errors"
"fmt"
)
var (
dataStore = map[string]*User{
"optic": &User{
Username: "bla",
Password: "ola",
},
}
)
//UserOperation interface
type UserOperation interface {
Update(info *User) error
FindByUsername(userName string) (*User, error)
}
//User struct
type User struct {
Username string
Password string
}
//UserLogic struct
type UserLogic struct {
UserOperation
}
//NewUser struct
func NewUser() UserOperation {
return &UserLogic{}
}
//Update method
func (u *UserLogic) Update(info *User) error {
userInfo, err := u.FindByUsername(info.Username)
if err != nil {
return err
}
fmt.Println(userInfo.Username, userInfo.Password)
fmt.Println("do some update logic !!!")
return nil
}
//FindByUsername method
func (u *UserLogic) FindByUsername(userName string) (*User, error) {
userInfo := &User{}
var exist bool
if userInfo, exist = dataStore[userName]; !exist {
return nil, errors.New("user not found")
}
return userInfo, nil
}
Update
I try to mock function with this code
func TestUpdate2(t *testing.T) {
var MockFunc = func(userName string) (*User, error) {
return &User{Username:"foo", Password:"bar"},nil
}
user := NewUser()
user.FindByUsername = MockFunc
user.Update(&User{Username:"optic", Password:"ola"})
}
You're mixing two levels of abstraction in your UserOperation interface: Update depends on FindByUsername. To make Update testable you need to inject the UserFinder functionality into your Update method. You can do this e.g. by defining a field in the UserLogic struct:
type UserOperation interface {
Update(info *User) error
}
type UserFinder func(userName string) (*User, error)
type UserLogic struct {
UserOperation
FindByUsername UserFinder
}
//NewUser struct
func NewUser() *UserLogic { // return structs, accept interfaces!
return &UserLogic{
findByUsername: FindByUsername
}
}
func (u *UserLogic) Update(info *User) error {
userInfo, err := u.findByUsername(info.Username)
if err != nil {
return err
}
fmt.Println(userInfo.Username, userInfo.Password)
fmt.Println("do some update logic !!!")
return nil
}
func FindByUsername(userName string) (*User, error) {
userInfo := &User{}
var exist bool
if userInfo, exist = dataStore[userName]; !exist {
return nil, errors.New("user not found")
}
return userInfo, nil
}
I have the following code in a golang plugin module:
plug.go
package main
import "fmt"
var (
Thing = New("first thing")
ThingFactory = thingFactory{}
)
type thing struct {
i int
s string
}
func New(s string) thing {
return thing{s: s}
}
func (t *thing) Say() string {
t.i++
return fmt.Sprintf("%s - %d", t.s, t.i)
}
type thingFactory struct{}
func (t thingFactory) Make(s string) thing {
return New(s)
}
it is compiled as a .so object and used in another program:
main.go
package main
import (
"fmt"
"plugin"
)
func main() {
p, err := plugin.Open("../plug/plug.so")
if err != nil {
panic(err)
}
symbol, err := p.Lookup("Thing")
if err != nil {
panic(err)
}
thing := symbol.(Sayer)
fmt.Println(thing.Say())
symbol, err = p.Lookup("ThingFactory") // <-problems start here
if err != nil {
panic(err)
}
factory := symbol.(GetSayer)
madeThing := factory.Make("how about me?")
fmt.Println(madeThing.Say())
fmt.Println(madeThing.Say())
}
type Sayer interface {
Say() string
}
type GetSayer interface {
Make(string) Sayer
}
I'm able to lookup the Thing, and call Say() on it, but the second interface conversion panics:
first thing - 1
panic: interface conversion: *main.thingFactory is not main.GetSayer: missing method Make
even though the runtime recognizes the first symbol as a Sayer it doesn't recognize that thingFactory obviously has a Make() method, which should return something that is also a Sayer.
Am I missing something obvious here?
The first problem is that in your plugin thingFactory (more precicely *thingfactory) does not have a method described in your main app's GetSayer interface:
Make(string) Sayer
You have:
Make(string) thing
So (first) you have to change thingFactory.Make() to this:
type Sayer interface {
Say() string
}
func (t thingFactory) Make(s string) Sayer {
th := New(s)
return &th
}
After this it still won't work. And the reason for this is because the plugin's Sayer type is not identical to your main app's Sayer type. But they must be the same in order to implement your main app's GetSayer interface.
One solution is to "outsource" the Sayer interface to its own package, and use this common, shared package both in the plugin and in the main app.
Let's create a new package, call it subplay:
package subplay
type Sayer interface {
Say() string
}
Import this package and use it in the plugin:
package main
import (
"fmt"
"path/to/subplay"
)
var (
Thing = New("first thing")
ThingFactory = thingFactory{}
)
type thing struct {
i int
s string
}
func New(s string) thing {
return thing{s: s}
}
func (t *thing) Say() string {
t.i++
return fmt.Sprintf("%s - %d", t.s, t.i)
}
type thingFactory struct{}
func (t thingFactory) Make(s string) subplay.Sayer {
th := New(s)
return &th
}
And also import and use it in the main app:
package main
import (
"fmt"
"path/to/subplay"
"plugin"
)
func main() {
p, err := plugin.Open("../plug/plug.so")
if err != nil {
panic(err)
}
symbol, err := p.Lookup("Thing")
if err != nil {
panic(err)
}
thing := symbol.(subplay.Sayer)
fmt.Println(thing.Say())
symbol, err = p.Lookup("ThingFactory")
if err != nil {
panic(err)
}
factory := symbol.(GetSayer)
madeThing := factory.Make("how about me?")
fmt.Println(madeThing.Say())
fmt.Println(madeThing.Say())
}
type GetSayer interface {
Make(string) subplay.Sayer
}
Now it will work, and output will be:
first thing - 1
how about me? - 1
how about me? - 2
See related questions:
go 1.8 plugin use custom interface
How do Go plugin dependencies work?
Your plugin Make method should return a Sayer object not thing
type Sayer interface {
Say() string
}
func (t *thingFactory) Make(s string) Sayer {
return New(s)
}
I want to use custom interface based on go plugin, but I found it's not support.
Definition of filter.Filter
package filter
import (
"net/http"
"github.com/valyala/fasthttp"
)
// Context filter context
type Context interface {
SetStartAt(startAt int64)
SetEndAt(endAt int64)
GetStartAt() int64
GetEndAt() int64
GetProxyServerAddr() string
GetProxyOuterRequest() *fasthttp.Request
GetProxyResponse() *fasthttp.Response
NeedMerge() bool
GetOriginRequestCtx() *fasthttp.RequestCtx
GetMaxQPS() int
ValidateProxyOuterRequest() bool
InBlacklist(ip string) bool
InWhitelist(ip string) bool
IsCircuitOpen() bool
IsCircuitHalf() bool
GetOpenToCloseFailureRate() int
GetHalfTrafficRate() int
GetHalfToOpenSucceedRate() int
GetOpenToCloseCollectSeconds() int
ChangeCircuitStatusToClose()
ChangeCircuitStatusToOpen()
RecordMetricsForRequest()
RecordMetricsForResponse()
RecordMetricsForFailure()
RecordMetricsForReject()
GetRecentlyRequestSuccessedCount(sec int) int
GetRecentlyRequestCount(sec int) int
GetRecentlyRequestFailureCount(sec int) int
}
// Filter filter interface
type Filter interface {
Name() string
Pre(c Context) (statusCode int, err error)
Post(c Context) (statusCode int, err error)
PostErr(c Context)
}
// BaseFilter base filter support default implemention
type BaseFilter struct{}
// Pre execute before proxy
func (f BaseFilter) Pre(c Context) (statusCode int, err error) {
return http.StatusOK, nil
}
// Post execute after proxy
func (f BaseFilter) Post(c Context) (statusCode int, err error) {
return http.StatusOK, nil
}
// PostErr execute proxy has errors
func (f BaseFilter) PostErr(c Context) {
}
This pkg is in my go app project.
load plugin file
package proxy
import (
"errors"
"plugin"
"strings"
"github.com/fagongzi/gateway/pkg/conf"
"github.com/fagongzi/gateway/pkg/filter"
)
var (
// ErrKnownFilter known filter error
ErrKnownFilter = errors.New("unknow filter")
)
const (
// FilterHTTPAccess access log filter
FilterHTTPAccess = "HTTP-ACCESS"
// FilterHeader header filter
FilterHeader = "HEAD" // process header fiter
// FilterXForward xforward fiter
FilterXForward = "XFORWARD"
// FilterBlackList blacklist filter
FilterBlackList = "BLACKLIST"
// FilterWhiteList whitelist filter
FilterWhiteList = "WHITELIST"
// FilterAnalysis analysis filter
FilterAnalysis = "ANALYSIS"
// FilterRateLimiting limit filter
FilterRateLimiting = "RATE-LIMITING"
// FilterCircuitBreake circuit breake filter
FilterCircuitBreake = "CIRCUIT-BREAKE"
// FilterValidation validation request filter
FilterValidation = "VALIDATION"
)
func newFilter(filterSpec *conf.FilterSpec) (filter.Filter, error) {
if filterSpec.External {
return newExternalFilter(filterSpec)
}
input := strings.ToUpper(filterSpec.Name)
switch input {
case FilterHTTPAccess:
return newAccessFilter(), nil
case FilterHeader:
return newHeadersFilter(), nil
case FilterXForward:
return newXForwardForFilter(), nil
case FilterAnalysis:
return newAnalysisFilter(), nil
case FilterBlackList:
return newBlackListFilter(), nil
case FilterWhiteList:
return newWhiteListFilter(), nil
case FilterRateLimiting:
return newRateLimitingFilter(), nil
case FilterCircuitBreake:
return newCircuitBreakeFilter(), nil
case FilterValidation:
return newValidationFilter(), nil
default:
return nil, ErrKnownFilter
}
}
func newExternalFilter(filterSpec *conf.FilterSpec) (filter.Filter, error) {
p, err := plugin.Open(filterSpec.ExternalPluginFile)
if err != nil {
return nil, err
}
s, err := p.Lookup("NewExternalFilter")
if err != nil {
return nil, err
}
sf := s.(func() (filter.Filter, error))
return sf()
}
This is the code of load plugin in my go app project
package main
import (
"C"
"strings"
"time"
"github.com/CodisLabs/codis/pkg/utils/log"
"github.com/fagongzi/gateway/pkg/filter"
"github.com/valyala/fasthttp"
)
// AccessFilter record the http access log
// log format: $remoteip "$method $path" $code "$agent" $svr $cost
type AccessFilter struct {
}
// NewExternalFilter create a External filter
func NewExternalFilter() (filter.Filter, error) {
return &AccessFilter{}, nil
}
// Name return name of this filter
func (f *AccessFilter) Name() string {
return "HTTP-ACCESS"
}
// Pre pre process
func (f *AccessFilter) Pre(c filter.Context) (statusCode int, err error) {
return 200, nil
}
// Post execute after proxy
func (f *AccessFilter) Post(c filter.Context) (statusCode int, err error) {
cost := (c.GetStartAt() - c.GetEndAt())
log.Infof("%s %s \"%s\" %d \"%s\" %s %s",
GetRealClientIP(c.GetOriginRequestCtx()),
c.GetOriginRequestCtx().Method(),
c.GetProxyOuterRequest().RequestURI(),
c.GetProxyResponse().StatusCode(),
c.GetOriginRequestCtx().UserAgent(),
c.GetProxyServerAddr(),
time.Duration(cost))
return 200, nil
}
// PostErr post error process
func (f *AccessFilter) PostErr(c filter.Context) {
}
// GetRealClientIP get read client ip
func GetRealClientIP(ctx *fasthttp.RequestCtx) string {
xforward := ctx.Request.Header.Peek("X-Forwarded-For")
if nil == xforward {
return strings.SplitN(ctx.RemoteAddr().String(), ":", 2)[0]
}
return strings.SplitN(string(xforward), ",", 2)[0]
}
This is the definition of plugin, it's in my plugin project. The plugin project and go app project are different projects.
I found errors:
panic: interface conversion: plugin.Symbol is func() (filter.Filter, error), not func() (filter.Filter, error)
You can find code in this project https://github.com/fagongzi/gateway/tree/go18-plugin-support.
filter.Filter is in pkg/filter package.
load plugin file in proxy/factory.go
plugin go file is in another project.
Custom interfaces work just fine.
But one important thing: you can only type assert types from values looked up from plugins that are defined outside of the plugin (you can't refer types defined in plugins). This also applies to each component of "composite types", for example you can only type assert a function type whose parameter and result types are also defined outside of the plugin.
1. With a common package outside of the plugin
One solution is to define the interface in a package outside of the plugin, and both the plugin and your app can import it and refer to it.
Define it in package filter:
package filter
type Filter interface {
Name() string
Age() int
}
The plugin is in package pq and imports package filter:
package main
import (
"fmt"
"filter"
)
type plgFilter struct{}
func (plgFilter) Name() string { return "Bob" }
func (plgFilter) Age() int { return 23 }
func GetFilter() (f filter.Filter, err error) {
f = plgFilter{}
fmt.Printf("[plugin GetFilter] Returning filter: %T %v\n", f, f)
return
}
And the main app that also imports (the same) package filter, loads the plugin, looks up GetFilter(), calls it and also uses the returned Filter:
package main
import (
"fmt"
"filter"
"plugin"
)
func main() {
p, err := plugin.Open("pg/pg.so")
if err != nil {
panic(err)
}
GetFilter, err := p.Lookup("GetFilter")
if err != nil {
panic(err)
}
filter, err := GetFilter.(func() (filter.Filter, error))()
fmt.Printf("GetFilter result: %T %v %v\n", filter, filter, err)
fmt.Println("\tName:", filter.Name())
fmt.Println("\tAge:", filter.Age())
}
Output:
[plugin GetFilter] Returning filter: main.plgFilter {}
GetFilter result: main.plgFilter {} <nil>
Name: Bob
Age: 23
2. With plugin returning interface{}, and interface defined in main app
Another solution is to have the plugin function return a value of type interface{}. Your main app can define the interface it expects, and it can use type assertion on the interface{} value returned by the plugin.
No filter package this time.
The plugin is in package pq:
package main
import (
"fmt"
)
type plgFilter struct{}
func (plgFilter) Name() string { return "Bob" }
func (plgFilter) Age() int { return 23 }
func GetFilterIface() (f interface{}, err error) {
f = plgFilter{}
fmt.Printf("[plugin GetFilterIface] Returning filter: %T %v\n", f, f)
return
}
And the main app:
package main
import (
"fmt"
"plugin"
)
func main() {
p, err := plugin.Open("pg/pg.so")
if err != nil {
panic(err)
}
GetFilterIface, err := p.Lookup("GetFilterIface")
if err != nil {
panic(err)
}
filterIface, err := GetFilterIface.(func() (interface{}, error))()
fmt.Printf("GetFilterIface result: %T %v %v\n", filterIface, filterIface, err)
myfilter := filterIface.(MyFilter)
fmt.Println("\tName:", myfilter.Name())
fmt.Println("\tAge:", myfilter.Age())
}
type MyFilter interface {
Name() string
Age() int
}
Output:
[plugin GetFilterIface] Returning filter: main.plgFilter {}
GetFilterIface result: main.plgFilter {} <nil>
Name: Bob
Age: 23
Also see related question: How do Go plugin dependencies work?