I am completely new to go and am planning on building a bittorent client as my first project to learn the language. One aspect is econding and decoding of bittorrent files. An example that I've found creates a struct containing a bufio.Reader, and then uses some interfaces to add additional methods. Can someone explain the mechanism for constructing these interfaces in such a way that overloads the struct? I'm not clear on how the functions in this file are being added to the decoder struct (https://github.com/marksamman/bencode/blob/master/decoder.go):
package bencode
import (
"bufio"
"errors"
"io"
"strconv"
)
type decoder struct {
bufio.Reader
}
func (decoder *decoder) readDictionary() (map[string]interface{}, error) {
dict := make(map[string]interface{})
}
func (decoder *decoder) readString() (string, error) {
...
}
func (decoder *decoder) readList() ([]interface{}, error) {
...
}
func Decode(reader io.Reader) (map[string]interface{}, error) {
decoder := decoder{*bufio.NewReader(reader)}
if firstByte, err := decoder.ReadByte(); err != nil {
return make(map[string]interface{}), nil
} else if firstByte != 'd' {
return nil, errors.New("bencode data must begin with a dictionary")
}
return decoder.readDictionary()
}
Related
I have following code that uses generics. I know that one can't use generics with methods, but can with types. Technically, my code complies with both restrictions, but still I get en error
./main.go:12:9: cannot use generic type GenericCacheWrapper[T any] without instantiation
The instantiation is on the first line of main function.
Is there any way to achieve this? May this be considered a Golang bug?
import (
"encoding/json"
"fmt"
)
type GenericCacheWrapper[T any] struct {
Container T
}
func (c GenericCacheWrapper) MarshalBinary() (data []byte, err error) {
return json.Marshal(c.Container)
}
func (c GenericCacheWrapper) UnmarshalBinary(data []byte) error {
return json.Unmarshal(data, &c.Container)
}
func main() {
wrapper := GenericCacheWrapper[int]{Container: 4}
data, err := wrapper.MarshalBinary()
if err != nil {
panic(err)
}
fmt.Println(data)
}
https://go.dev/play/p/9sWxXYmAcUH
You just have to add [T] to the end of GenericCacheWrapper or [_] if you want to make it clear that you are not actually using T in the functions.
package main
import (
"encoding/json"
"fmt"
)
type GenericCacheWrapper[T any] struct {
Container T
}
func (c GenericCacheWrapper[T]) MarshalBinary() (data []byte, err error) {
return json.Marshal(c.Container)
}
func (c GenericCacheWrapper[T]) UnmarshalBinary(data []byte) error {
return json.Unmarshal(data, &c.Container)
}
func main() {
wrapper := GenericCacheWrapper[int]{Container: 4}
data, err := wrapper.MarshalBinary()
if err != nil {
panic(err)
}
fmt.Println(data)
}
This rule is defined in the language spec:
A generic type may also have methods associated with it. In this case, the method receivers must declare the same number of type parameters as present in the generic type definition.
But the reason behind this isn't very clear, perhaps to make implementation of the compiler/type checking easier.
Related: Go error: cannot use generic type without instantiation
Say I have the following code:
package lib
import (
"errors"
"strconv"
)
var ErrSomething = errors.New("foobar")
func SomeFunc(str string) (int, error) {
i, err := strconv.Atoi(str)
if err != nil {
// how to combine ErrSomething with err?
return 0, fmt.Errorf("%w: %w", ErrSomething, err)
}
// do other things, potentially return other errors
return i
}
How do I combine the error returned from strconv.Atoi with my named error ErrSomething. The reason for combining is so that users of my SomeFunc() can check what exactly went wrong using my error "constants" while not losing information about the underlying error.
I have read similar questions but the usual answer is to just do: return 0, fmt.Errorf("foobar: %w", err) but this way my users can't check the error using errors.Is(err, ???)
You can achieve the desired behavior by creating an error type that implements the Is and Unwrap methods as follows:
package lib
import (
"fmt"
"strconv"
)
type FoobarError struct {
msg string
original error
}
func (err *FoobarError) Error() string {
return fmt.Sprintf("%s: %s", err.msg, err.original.Error())
}
func (err *FoobarError) Unwrap() error {
return err.original
}
func (err *FoobarError) Is(target error) bool {
_, ok := target.(*FoobarError)
return ok
}
func SomeFunc() error {
// strconv.ErrSyntax is used as a dummy error here for the error
// that might be returned by strconv.Atoi or any other operation.
err := strconv.ErrSyntax
return &FoobarError{"foobar", err}
}
Usage:
package main
import (
"errors"
"fmt"
"strconv"
"lib"
)
func main() {
if err := lib.SomeFunc(); err != nil {
fmt.Println(err) // foobar: invalid syntax
fmt.Println(errors.Is(err, &lib.FoobarError{})) // true
fmt.Println(errors.Is(err, strconv.ErrSyntax)) // true
}
}
You can read more about this approach here.
Bonus
Similar to Go's os.IsExist, you may be interested in adding a helper function to your library that makes it easier for the user to check the error:
package lib
import (
"errors"
// ...
)
// ...
func IsFoobar(err error) bool {
return errors.Is(err, &FoobarError{})
}
Usage:
package main
// ...
func main() {
err := lib.SomeFunc();
if lib.IsFoobar(err) {
// ...
}
}
I have a project consisting of 4 parts:
A gateway (gateway/gateway.go) it's a package that knows how to talk with an application server and open this connection channel.
A runner (runner/runner.go) it's the main (go build -o runner/runner runner/runner.go) It loads and execute Modules (using reflect I run functions from the module)!
A framework (framework/framework.go) Implements many functionalities calling the gateway.
Modules (aka Plugins in Go) (modules/sample.go) (go build -buildmode plugin -o modules/sample.so ./modules/sample.go) Using the framework, does customer logic! When init I export the reflect.Value of struct, then runner can run methods of this struct.
I want the runner instantiate the gateway and the framework obtain this instance without create a dependency between runner/framework.
Why? To avoid Go error 'plugin was built with a different version of package' when runner loads the module!
If I update the runner (with the framework changed), I will invalidate old modules.
I already do that using 2 ways I don't like:
Using context, but all functions from module and framework need receive a parameter context, then framework extract the gateway.
Just let the framework instantiate gateway, but then the runner cannot use gateway.
There have been a lot of headaches with Go plugins, especially the plugin compiler version must exactly match the program's compiler version. But the example works.
runner/runner.go
package main
import (
"context"
"fmt"
"os"
"plugin"
"reflect"
"../gateway"
"../dep"
)
var a *gateway.Gateway
func main() {
myModule := os.Args[1]
if _, err := plugin.Open(myModule); err != nil {
os.Exit(1)
}
mod, err := dep.NewModule()
if err != nil {
os.Exit(1)
}
a = gateway.NewGW()
ctx := context.WithValue(context.Background(), "gateway", a)
modreflect, err := mod.Init(ctx, dep.Config{})
if err != nil {
os.Exit(1)
}
if !modreflect.IsValid() {
os.Exit(1)
}
modnode, err := mod.Start(ctx)
if err != nil {
os.Exit(1002)
}
for {
if len(modnode) <= 0 {
break
}
modnoderefl := modreflect.MethodByName(modnode)
if !modnoderefl.IsValid() {
break
}
result := modnoderefl.Call([]reflect.Value{reflect.ValueOf(ctx)})
if len(result) != 2 {
break
}
modnode = result[0].String()
}
mod.End(ctx)
}
gateway/gateway.go
package gateway
type Gateway struct {}
fun NewGW() *Gateway {
a := Gateway{}
return &a
}
dep/dep.go
package dep
import (
"errors"
"context"
"reflect"
)
// Config is a configuration provider.
type Config map[string]interface{}
// Module is the interface implementated by types that
// register themselves as modular plug-ins.
type Module interface {
Init(ctx context.Context, config Config) (reflect.Value,error)
Start(ctx context.Context) (string,error)
End(ctx context.Context) error
}
var themod = []func() Module{}
func RegisterModule(ctor func() Module) {
themod = append(themod, ctor)
}
func NewModule() (Module, error) {
if len(themod) == 0 {
return nil, errors.New("Module not registered")
}
return themod[0](), nil
}
framework/framework.go
package framework
import (
"fmt"
"context"
"../gateway"
)
type PlayFileInput struct {
Path string
}
func Play(ctx context.Context, p PlayFileInput) error {
if a := ctx.Value("gateway"); a != nil {
if a.(*gateway.Gateway) != nil {
_, err := a.(*gateway.Gateway).Exec("Playback", p.Path)
return err
}
}
return nil
}
modules/sample.go
package main
import "C"
import (
"context"
"fmt"
"os"
"reflect"
"../dep"
"../framework"
)
type MyModuleImpl struct {}
func init() {
dep.RegisterModule(func() dep.Module {
return &MyModuleImpl{}
})
}
func (m *MyModuleImpl) Init(ctx context.Context, config dep.Config) (reflect.Value, error) {
return reflect.ValueOf(m), nil
}
func (m *MyModuleImpl) Start(ctx context.Context) (string,error) {
return "Menu_1",nil
}
func (n *MyModuleImpl)Menu_1(ctx context.Context) (string, error) {
framework.Play(ctx, framework.PlayFileInput{Path: "welcome.wav"})
return "Menu_2",nil
}
func (n *MyModuleImpl)Menu_2(ctx context.Context) (string, error) {
return "Menu_3", nil
}
// ....
// ....
func (m *MyModuleImpl) End(ctx context.Context) error {
return nil
}
I have the following code in a golang plugin module:
plug.go
package main
import "fmt"
var (
Thing = New("first thing")
ThingFactory = thingFactory{}
)
type thing struct {
i int
s string
}
func New(s string) thing {
return thing{s: s}
}
func (t *thing) Say() string {
t.i++
return fmt.Sprintf("%s - %d", t.s, t.i)
}
type thingFactory struct{}
func (t thingFactory) Make(s string) thing {
return New(s)
}
it is compiled as a .so object and used in another program:
main.go
package main
import (
"fmt"
"plugin"
)
func main() {
p, err := plugin.Open("../plug/plug.so")
if err != nil {
panic(err)
}
symbol, err := p.Lookup("Thing")
if err != nil {
panic(err)
}
thing := symbol.(Sayer)
fmt.Println(thing.Say())
symbol, err = p.Lookup("ThingFactory") // <-problems start here
if err != nil {
panic(err)
}
factory := symbol.(GetSayer)
madeThing := factory.Make("how about me?")
fmt.Println(madeThing.Say())
fmt.Println(madeThing.Say())
}
type Sayer interface {
Say() string
}
type GetSayer interface {
Make(string) Sayer
}
I'm able to lookup the Thing, and call Say() on it, but the second interface conversion panics:
first thing - 1
panic: interface conversion: *main.thingFactory is not main.GetSayer: missing method Make
even though the runtime recognizes the first symbol as a Sayer it doesn't recognize that thingFactory obviously has a Make() method, which should return something that is also a Sayer.
Am I missing something obvious here?
The first problem is that in your plugin thingFactory (more precicely *thingfactory) does not have a method described in your main app's GetSayer interface:
Make(string) Sayer
You have:
Make(string) thing
So (first) you have to change thingFactory.Make() to this:
type Sayer interface {
Say() string
}
func (t thingFactory) Make(s string) Sayer {
th := New(s)
return &th
}
After this it still won't work. And the reason for this is because the plugin's Sayer type is not identical to your main app's Sayer type. But they must be the same in order to implement your main app's GetSayer interface.
One solution is to "outsource" the Sayer interface to its own package, and use this common, shared package both in the plugin and in the main app.
Let's create a new package, call it subplay:
package subplay
type Sayer interface {
Say() string
}
Import this package and use it in the plugin:
package main
import (
"fmt"
"path/to/subplay"
)
var (
Thing = New("first thing")
ThingFactory = thingFactory{}
)
type thing struct {
i int
s string
}
func New(s string) thing {
return thing{s: s}
}
func (t *thing) Say() string {
t.i++
return fmt.Sprintf("%s - %d", t.s, t.i)
}
type thingFactory struct{}
func (t thingFactory) Make(s string) subplay.Sayer {
th := New(s)
return &th
}
And also import and use it in the main app:
package main
import (
"fmt"
"path/to/subplay"
"plugin"
)
func main() {
p, err := plugin.Open("../plug/plug.so")
if err != nil {
panic(err)
}
symbol, err := p.Lookup("Thing")
if err != nil {
panic(err)
}
thing := symbol.(subplay.Sayer)
fmt.Println(thing.Say())
symbol, err = p.Lookup("ThingFactory")
if err != nil {
panic(err)
}
factory := symbol.(GetSayer)
madeThing := factory.Make("how about me?")
fmt.Println(madeThing.Say())
fmt.Println(madeThing.Say())
}
type GetSayer interface {
Make(string) subplay.Sayer
}
Now it will work, and output will be:
first thing - 1
how about me? - 1
how about me? - 2
See related questions:
go 1.8 plugin use custom interface
How do Go plugin dependencies work?
Your plugin Make method should return a Sayer object not thing
type Sayer interface {
Say() string
}
func (t *thingFactory) Make(s string) Sayer {
return New(s)
}
I want to be able to unmarshal yaml files less rigidly. That is, my library has a predefined number of options the yaml file must have. Then, the user should be able to extend this to include any custom options.
Here is what I have
package main
import (
"net/http"
"yamlcms"
"github.com/julienschmidt/httprouter"
)
type Page struct {
*yamlcms.Page
Title string
Date string
}
func getBlogRoutes() {
pages := []*Page{}
yamlcms.ReadDir("html", pages)
}
// This section is a work in progress, I only include it for loose context
func main() {
router := httprouter.New()
//blogRoutes := getBlogRoutes()
//for _, blogRoute := range *blogRoutes {
// router.Handle(blogRoute.Method, blogRoute.Pattern,
// func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {})
//}
http.ListenAndServe(":8080", router)
}
Here is the yamlcms package:
package yamlcms
import (
"io/ioutil"
"os"
"strings"
"gopkg.in/yaml.v2"
)
type Page struct {
Slug string `yaml:"slug"`
File string `yaml:"file"`
}
func (page *Page) ReadFile(file string) (err error) {
fileContents, err := ioutil.ReadFile(file)
if err != nil {
return
}
err = yaml.Unmarshal(fileContents, &page)
return
}
func isYamlFile(fileInfo os.FileInfo) bool {
return !fileInfo.IsDir() && strings.HasSuffix(fileInfo.Name(), ".yaml")
}
func ReadDir(dir string, pages []*Page) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
pages[i].ReadFile(fileInfo.Name())
}
}
return
}
There is a compiler issue here:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.Page in argument to yamlcms.ReadDir
My main intent in this question is to learn the idiomatic way of doing this kind of thing in Go. Other 3rd-party solutions may exist but I am not immediately interested in them because I have problems like this frequently in Go having to do with inheritance, etc. So along the lines of what I've presented, how can I best (idiomatically) accomplish what I am going for?
EDIT:
So I've made some changes as suggested. Now I have this:
type FileReader interface {
ReadFile(file string) error
}
func ReadDir(dir string, pages []*FileReader) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
(*pages[i]).ReadFile(fileInfo.Name())
}
}
return
}
However, I still get a similar compiler error:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.FileReader in argument to yamlcms.ReadDir
Even though main.Page should be a FileReader because it embeds yamlcms.Page.
EDIT: I forgot that slices of interfaces don't work like that. You'd need to allocate a new slice, convert all pages to FileReaders, call the function, and convert them back.
Another possible solution is refactoring yamlcms.ReadDir to return the contents of the files, so that they could be unmarshaled later:
// In yamlcms.
func ReadYAMLFilesInDir(dir string) ([][]byte, error) { ... }
// In client code.
files := yamlcms.ReadYAMLFilesInDir("dir")
for i := range pages {
if err := yaml.Unmarshal(files[i], &pages[i]); err != nil { return err }
}
The original answer:
There are no such things as inheritance or casting in Go. Prefer composition and interfaces in your designs. In your case, you can redefine your yamlcms.ReadDir to accept an interface, FileReader.
type FileReader interface {
ReadFile(file string) error
}
Both yamlcms.Page and main.Page will implement this, as the latter embeds the former.