passing channel param to a dynamic loaded func in golang - go

I'm new to golang and golang plugins. I'm having trouble passing a chan object to this functions. If I switch to int it works. Not sure what exactly I am missing. Thanks for any help.
dataunit.go
package main
type DataUnit struct {
i int
s string
}
modcounter.go
package main
import (
"fmt"
// "strconv"
)
type module string
//func (m module) RunMod(i int) {
func (m module) RunMod(in <-chan *DataUnit) {
//fmt.Println("Hello Universe " + strconv.Itoa(i))
fmt.Println("Hello Universe ")
n := <-in
fmt.Printf(n.s)
}
var Module module
modmain.go
package main
import (
"fmt"
"os"
"plugin"
)
type DataUnit struct {
i int
s string
}
type Module interface {
//RunMod(i int)
RunMod(in <-chan *DataUnit)
}
func main() {
out := make(chan *DataUnit, 2000)
plug, err := plugin.Open("./modcounter.so")
if err != nil {
fmt.Printf("FATAL (plugin.Open): " + err.Error())
os.Exit(1)
}
symModule, err := plug.Lookup("Module")
if err != nil {
fmt.Printf(err.Error())
os.Exit(1)
}
var module Module
module, ok:= symModule.(Module)
if !ok {
fmt.Println("unexpected type from module symbol")
os.Exit(1)
}
//module.RunMod(5)
module.RunMod(out)
}
go build -buildmode=plugin -o modcounter.so modcounter.go dataunit.go
go build modmain.go dataunit.go
./modmain
unexpected type from module symbol

If you are still learning golang then plugins are definitely not the place to start. I believe plugin support is still experimental and doesn't work on all OS - in 1.10 I believe Darwin was added to previous Linux.
A quick search reveals a pre-existing issue reported similar to yours -looks like it's in go 1.10:
https://github.com/golang/go/issues/24351
You can maybe track through the repository and get a branch or version where this is fixed, but whilst your learning you aren't going to have any confidence in whether problems with this area of functionality are with your code or the plugin system. I recommend therefore sticking to learning go with core language features for now.

In go is possible to pass channels of objects!!
For example, if you to use <-chan http.Header will works fine. The question is that the params must be shared between modules and application. So if you reallocate DataUnit for another package will work.
My test was structured like:
My interface:
//in modmain.go
type Module interface {
RunMod(in <-chan *mydata.DataUnit)
}
My module:
//in modcounter.go
func (m module) RunMod(in <-chan *mydata.DataUnit) {
fmt.Println("Hello Universe ")
n := <-in
fmt.Printf("%v", n.S)
}
My data:
//in dataunit.go
type DataUnit struct {
I int //export field
S string //export field
}
The result:
P.S.: Docker with golang 1.10 was used for tests.
#in Dockerfile
FROM golang:1.10
COPY . /go/
RUN export GOPATH=$GOPATH:/go/
RUN cd /go/src/mydata && go build dataunit.go
RUN cd /go/src/app && go build modmain.go
RUN cd /go/src/app && go build -buildmode=plugin -o modcounter.so modcounter.go
WORKDIR /go/src/app
RUN ls -l
CMD ["/go/src/app/modmain"]

Related

Grpc Golang using UnimplementedServer with other embedded interfaces

I've recently updated to the latest protoc and Go plugins with an existing proto 3 codebase, and am running into trouble with the new UnimplementedServer functionality. The struct that is used for the Grpc server already embeds another interface describing the methods implemented by this service. After embedding the UnimplementedServer reference in my struct, I get an ambiguous error from the compiler and it tells me I am not implementing my service method anymore. Is there some issue with the way I have structured the code? Code to reproduce follows, with libprotoc 3.17.3, protoc-gen-go v1.27.1, and protoc-gen-go-grpc 1.1.0:
api/proto/core_service.proto:
package testbed;
option go_package = "internal/proto";
service Core {
rpc CreateThing(Thing) returns (ThingResponse) {};
}
message Thing {
string name = 1;
}
message ThingResponse {
int64 result = 1;
}
internal/core.go:
import (
"context"
"testbed.org/demo/internal/proto"
)
type CoreApi interface {
CreateThing(ctx context.Context, t *proto.Thing) (*proto.ThingResponse, error)
}
internal/core_endpoints.go:
import (
"context"
"testbed.org/demo/internal/proto"
)
type CoreEndpoints struct {}
func (ce CoreEndpoints) CreateThing(_ context.Context, _ *proto.Thing) (*proto.ThingResponse, error) {
return nil, nil
}
cmd/service.go:
import (
//"context"
. "testbed.org/demo/internal"
"testbed.org/demo/internal/proto"
"google.golang.org/grpc"
)
type MyServer struct {
CoreApi
proto.UnimplementedCoreServer
}
func main() {
mySvr := &MyServer{CoreApi: &CoreEndpoints{}}
//_, _ = mySvr.CoreApi.CreateThing(context.Background(), &proto.Thing{})
grpcSvr := grpc.NewServer()
proto.RegisterCoreServer(grpcSvr, mySvr)
}
Build:
protoc -I api/proto/ api/proto/*.proto --go_out=. --go-grpc_out=.
go build -o bin/svc cmd/service.go
cmd/service.go:19:26: MyServer.CreateThing is ambiguous
cmd/service.go:19:26: cannot use mySvr (type *MyServer) as type "testbed.org/demo/internal/proto".CoreServer in argument to "testbed.org/demo/internal/proto".RegisterCoreServer:
*MyServer does not implement "testbed.org/demo/internal/proto".CoreServer (missing CreateThing method)
gmake: *** [Makefile:8: service] Error 2
As the log says, 'Myserver' does not implement the coreserver interface.
type MyServer struct {
endpoint CoreEndpoints
proto.UnimplementedCoreServer
}
func (srv MyServer) CreateThing(ctx context.Context, in *proto.Thing)(*proto.ThingResponse, error) {
return srv.endpoint.CreateThing(ctx,in)
}

Golang - Discard as WriteCloser

I needed to create an equivalent of ioutil.Discard that can satisfy a "WriteCloser" interface. With a bit of Googling I came up with
package main
import (
"io"
"io/ioutil"
"strings"
"fmt"
)
type discardCloser struct {
io.Writer
}
func (discardCloser) Close() error {
return nil
}
func main() {
src := strings.NewReader("Hello world")
dst := discardCloser{Writer: ioutil.Discard}
count, err := io.Copy(dst, src)
fmt.Println(count, err)
err = dst.Close()
fmt.Println(err)
}
Go playground link here
Is there a more idiomatic way of doing this?
Background: some standard library methods return a WriteCloser, such as net/smtp.Data. When implementing automated tests, it's nice to be able to exercise functions like this, while sending their output to Discard.
I took bereal's tip and looked at NopCloser. The approach works nicely, and is useful in test functions built around the libraries that require a WriteCloser.
I renamed the type myWriteCloser as it can be used to promote "real" writers such as as &bytes.Buffer, as well as the special system discard writer.
type myWriteCloser struct {
io.Writer
}
func (myWriteCloser) Close() error {
return nil
}

How to interact with Telegraf using Go external plugin?

I have a dummy GO plugin, using that, i want send data to telegraf. But, i'm not able to find any way to send data from the plugin to telegraf.
this external Go plugin looks like below
package main
import (
"fmt"
"time"
)
type greeting string
type n int
func (g greeting) Greet() {
for i := 0; i < 10; i++ {
timer1 := time.NewTimer(2 * time.Second)
<-timer1.C
fmt.Println("value ", i)
sum := 0
sum += i * 100
fmt.Println(sum)
}
}
// exported
var Greeter greeting
And the main file looks like
import (
"fmt"
"os"
"plugin"
)
type Greeter interface {
Greet()
}
func main() {
var mod string
mod = "./eng/eng.so"
// load module
// 1. open the so file to load the symbols
plug, err := plugin.Open(mod)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// 2. look up a symbol (an exported function or variable)
// in this case, variable Greeter
symGreeter, err := plug.Lookup("Greeter")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// 3. Assert that loaded symbol is of a desired type
// in this case interface type Greeter (defined above)
var greeter Greeter
greeter, ok := symGreeter.(Greeter)
if !ok {
fmt.Println("unexpected type from module symbol")
os.Exit(1)
}
// 4. use the module
greeter.Greet()
}
Can anyone help me find a way or a direction on how to make interaction between GO plugin and telegraf work. Any heads up is appreciated.
Currently, Telegraf does not support the use of external plugins.
An issue was made and closed regarding this requested feature.
If you were planning on writing code in Golang I would suggest creating a custom Telegraf plugin. Telegraf plugins do not use go external plugin package instead they are built with Telegraf itself. Your plugin must satisfy the specific interface based on the type of plugin you want to create. Based on your question I will assume that you desire to create an input plugin, Telegraf has examples for creating each type of plugin in their source code.

How to import imported package's vendor package

I am using etcd's wal package (https://godoc.org/github.com/coreos/etcd/wal) to do write-ahead logging. wal has go.uber.org/zap in its vendor packages. In wal's create function func Create(lg *zap.Logger, dirpath string, metadata []byte) (*WAL, error), I need to pass in zap.Logger.
I have tried to import go.uber.org/zap but go compiler complains "type mismatch" when I pass in zap.Logger.
package main
import (
"github.com/coreos/etcd/wal"
)
func check(e error) {
if e != nil {
panic(e)
}
}
func main() {
metadata := []byte{}
w, err := wal.Create(zap.NewExample(), "/tmp/hello", metadata)
// err := w.Save(s, ents)
}
How should I use zap.Logger in my project?
It seems like the package github.com/coreos/etcd/wal is not meant to be used outside of the etcd project. If you really need to use it, please, follow the steps below.
Place the following code in the $GOPATH/src/yourpackage/main.go file.
package main
import (
"fmt"
"go.etcd.io/etcd/wal"
"go.uber.org/zap"
)
func main() {
metadata := []byte{}
w, err := wal.Create(zap.NewExample(), "/tmp/hello", metadata)
fmt.Println(w, err)
}
mkdir $GOPATH/src/yourpackage/vendor
cp -r $GOPATH/src/go.etcd.io $GOPATH/src/yourpackage/vendor/
mv $GOPATH/src/yourpackage/vendor/go.etcd.io/etcd/vendor/go.uber.org $GOPATH/src/yourpackage/vendor/
go build yourpackage

How to use golang viper Watchconfig & onConfigChange

I am trying to read my application configuration using golang viper and would like to read the latest config always. Please find my code below
config.go
package config
import (
"github.com/spf13/viper"
"log"
"github.com/fsnotify/fsnotify"
"time"
)
type Reader interface {
GetAllKeys() []string
Get(key string) interface{}
GetBool(key string) bool
GetString(key string) string
}
type viperConfigReader struct {
viper *viper.Viper
}
var TestConfReader *viperConfigReader
func (v viperConfigReader) GetAllKeys() []string{
return v.viper.AllKeys()
}
func (v viperConfigReader) Get(key string) interface{} {
return v.viper.Get(key)
}
func (v viperConfigReader) GetBool(key string) bool {
return v.viper.GetBool(key)
}
func (v viperConfigReader) GetString(key string) string {
return v.viper.GetString(key)
}
func init() {
v:= viper.New()
v.SetConfigName("test")
v.AddConfigPath("/tmp/")
err := v.ReadInConfig()
if err != nil {
log.Panic("Not able to read configuration", err.Error())
}
TestConfReader = &viperConfigReader{
viper: v,
}
go func() {
for {
time.Sleep(time.Second * 5)
v.WatchConfig()
v.OnConfigChange(func(e fsnotify.Event) {
log.Println("config file changed", e.Name)
})
}
}()
}
main.go
package main
import (
"github.com/xxxx/xxxx/config"
"log"
"time"
)
func main() {
conf := config.TestConfReader
log.Println(conf.GetAllKeys())
log.Println(conf.GetString("test1"))
time.Sleep(20 * time.Second)
log.Println(conf.GetString("test1"))
}
When the main program is running, I tried to update the config and expected to see OnConfigChange log message but it never showed up.
How can I fix this program ?.
Can someone provide an example of using viper watchconfig & onconfigchange methods to read the latest config
The comment by ymonad is on the right track, depending on your OS you may be experiencing problems with viper/fsnotify.
For example, I ran your example code on Mac OS X (Sierra) and noticed the same symptom you described: when the config file is in /tmp the viper WatchConfig call was not causing the viper OnConfigChange function to be called.
However, when I change the AddConfigPath to use the current working directory or my home directory then I do see logging from your OnConfigChange function. For example, try:
v.AddConfigPath("./")
I'd recommend experimenting with different directory locations to see if this is some sort of viper/fsnotify bug or limitation. For some reason, it doesn't appear to detect changes from the /tmp directory on Mac OS X, at least, it doesn't for my setup. I couldn't find any mention of problems with /tmp on OS X, but the fsnotify CONTRIBUTING.md does mention limitations with "shared" directories under Vagrant so perhaps there are some filesystem configurations that do not trigger notifications:
Notice: fsnotify file system events won't trigger in shared folders. The tests get around this limitation by using the /tmp directory.
Also, you don't need to keep calling WatchConfig and OnConfigChange via your goroutine. You can eliminate the goroutine completely and just move the relevant lines into init.

Resources