Using different levels of interfaces - go

I'm trying to organize my code using interfaces in Go.
I have 2 sources of data: FTP and API. In each source, I have several structs that make the logic vary depending on the case.
In this question, I will omit API and stick with FTP.
My problem comes from the impossibility to say: FTPAcq is also an Acquisition
If FetchMeters(), when I do ftp.Decrypt(nil) I would like ftp to be "compatible" with FTPAcq
Here is my code:
package main
import (
"github.com/dutchcoders/goftp"
log "github.com/sirupsen/logrus"
"os"
)
type Acquisition interface {
FetchMeters() ([]Meter, error)
Name() string
}
type FTPAcq interface {
Unzip(file string) string
Decrypt(file string) string
}
//type APIAcq interface {
// FetchMeter(meterID string) (Meter, error)
//}
func main() {
var acqs []Acquisition
ftp, err := NewFTPDriver(os.Getenv("FTP_USER"), os.Getenv("FTP_PASSWD"), os.Getenv("FTP_ADDR"), os.Getenv("FTP_PORT"))
if err != nil {
panic(err)
}
ftp1 := NewFTPDriverSGE(*ftp)
ftp2 := NewFTPDriverTA(*ftp)
acqs = append(acqs, ftp1, ftp2)
for _, acq := range acqs {
tmpMeters, err := acq.FetchMeters()
if err != nil {
log.Warn(acq.Name(), " got error :", err)
}
log.Info(tmpMeters)
}
}
type Meter struct {
ID string
OperationID string
Unit string
}
//FtpSGE is a implementation of acquisition Interface (see driver.go)
type FTP struct {
Username string
Password string
Url string
Port string
client *goftp.FTP
}
type FTPSGE struct {
FTP
}
type FTPTA struct {
FTP
}
func (f FTPSGE) Unzip(path string) []string {
return nil
}
func (f FTPTA) Unzip(path string) []string {
return nil
}
func (f FTPSGE) Decrypt(path string) []string {
return nil
}
func (f FTPTA) Decrypt(path string) []string {
return nil
}
func (ftp FTP) FetchMeters() ([]Meter, error) {
log.Info(ftp.Name(), " is running")
files := ftp.Download(nil)
files = ftp.Decrypt("") // I have several implementation of Decrypt
files = ftp.Unzip("") // I have several implementation of Unzip
log.Info(files)
return nil, nil
}
func (ftp FTP) Name() string {
panic("FTP ")
}
func (ftp FTP) Download(files []string) []string {
panic("implement me")
}
func NewFTPDriver(user, password, url, port string) (*FTP, error) {
var err error
ftp := &FTP{
Username: user,
Password: password,
Url: url,
Port: port,
}
if ftp.client, err = goftp.Connect(url + ":" + port); err != nil {
return nil, err
}
if err = ftp.client.Login(user, password); err != nil {
return nil, err
}
return ftp, nil
}
func NewFTPDriverSGE(f FTP) *FTPSGE {
return &FTPSGE{f}
}
func NewFTPDriverTA(f FTP) *FTPTA {
return &FTPTA{f}
}
In my case, I get:
ftp.Decrypt undefined (type FTP has no field or method Decrypt)
How should I do?

FTP does not implement FTPAcq. It implements only Acquisition. It doesn't even have Decrypt() as a method, interface or not.
FTPSGE and FTPTA implement FTPAcq, but they are not the same type as FTP.
I don't know what you're trying to accomplish, but perhaps something to try is embedding FTP in FTPSGE and FTPTA. This gives those 2 types the fields and methods of the embedded type and still allows you to define additional methods on those types (the methods for FTPAcq in your case).
For example
type FTPSGE {
FTP
}
// OR
type FTPSGE {
*FTP
}
Which you then create as so: x := FTPSGE{ftp1}. Keep in mind that this will create a copy of ftp1 inside x. If ftp1 is type FTP (not a pointer), the entire struct gets copied. If ftp1 is type *FTP (a pointer, which seems to be what you're using), the pointer is copied and x.FTP still points to the same data as ftp1.
This means FTPSGE will implement both Acquisition and FTPAcq.
You'll have to be careful of whether or not interfaces are implemented on the value or pointer: func (a A) Something() vs func (a *A) Somthing().
Here is some reading on methods, interfaces, and embedding.
https://www.ardanlabs.com/blog/2014/05/methods-interfaces-and-embedded-types.html
https://travix.io/type-embedding-in-go-ba40dd4264df
https://golang.org/doc/effective_go.html (and lots of other info)

Related

Semantics - Passing outer interface type value to inner interface variable

Below is the prototype code for 3 layer API(with some design limitations):
// Sample program demonstrating interface composition.
package main
import (
"errors"
"fmt"
"io"
"math/rand"
"time"
)
func init() {
rand.Seed(time.Now().UnixNano())
}
// =============================================================================
// Data is the structure of the data we are copying.
type Data struct {
Line string
}
// =============================================================================
// Puller declares behavior for pulling data.
type Puller interface {
Pull(d *Data) error
}
// Storer declares behavior for storing data.
type Storer interface {
Store(d *Data) error
}
// PullStorer declares behavior for both pulling and storing.
type PullStorer interface {
Puller
Storer
}
// =============================================================================
// Xenia is a system we need to pull data from.
type Xenia struct {
Host string
Timeout time.Duration
}
// Pull knows how to pull data out of Xenia.
func (*Xenia) Pull(d *Data) error {
switch rand.Intn(10) {
case 1, 9:
return io.EOF
case 5:
return errors.New("Error reading data from Xenia")
default:
d.Line = "Data"
fmt.Println("In:", d.Line)
return nil
}
}
// Pillar is a system we need to store data into.
type Pillar struct {
Host string
Timeout time.Duration
}
// Store knows how to store data into Pillar.
func (*Pillar) Store(d *Data) error {
fmt.Println("Out:", d.Line)
return nil
}
// =============================================================================
// System wraps Xenia and Pillar together into a single system.
type System struct {
Xenia
Pillar
}
// =============================================================================
// pull knows how to pull bulks of data from any Puller.
func pull(p Puller, data []Data) (int, error) {
for i := range data {
if err := p.Pull(&data[i]); err != nil {
return i, err
}
}
return len(data), nil
}
// store knows how to store bulks of data from any Storer.
func store(s Storer, data []Data) (int, error) {
for i := range data {
if err := s.Store(&data[i]); err != nil {
return i, err
}
}
return len(data), nil
}
// Copy knows how to pull and store data from any System.
func Copy(ps PullStorer, batch int) error {
data := make([]Data, batch)
for {
i, err := pull(ps, data)
if i > 0 {
if _, err := store(ps, data[:i]); err != nil {
return err
}
}
if err != nil {
return err
}
}
}
// =============================================================================
func main() {
sys := System{
Xenia: Xenia{
Host: "localhost:8000",
Timeout: time.Second,
},
Pillar: Pillar{
Host: "localhost:9000",
Timeout: time.Second,
},
}
if err := Copy(&sys, 3); err != io.EOF {
fmt.Println(err)
}
}
My understanding is,
If you pass u, the interface contains type of u, and a pointer to a copy of u. If you pass &u, the interface contains the type of &u and the address of u.
So, below is my understanding for variables p, s & ps, from above code:
But, I would like to confirm,
1) On pull(ps, data), does first element of p contain type *System or Xenia type, as first element?
2) On store(ps, data[:i]), does first element of s contain type *System or Pillar type?
The type is *System.
You can confirm this by adding simple print statement, e.g. fmt.Printf("%T\n", p)

How to avoid a long switch-case statement in Go

I'm writing a chat bot in Go and wondering how can I avoid a long switch-case statement similar to this one:
switch {
// #bot search me HMAC
case strings.Contains(message, "search me"):
query := strings.Split(message, "search me ")[1]
return webSearch(query), "html"
// #bot thesaurus me challenge
case strings.Contains(message, "thesaurus me"):
query := strings.Split(message, "thesaurus me ")[1]
return synonyms(query), "html"
Should I define those handlers each in a separate package or should I just use structs and interfaces? Which method will allow me to have a good structure, avoid switch-case and let external developers to easier create handlers?
I think packages will be a better choice but I'm not sure how to register the handlers with the main bot. Would appreciate an example.
You could use a map[string]command similar to how the net/http package registers handlers. Something akin to this:
https://play.golang.org/p/9YzHyLodAQ
package main
import (
"fmt"
"errors"
)
type BotFunc func(string) (string, error)
type BotMap map[string]BotFunc
var Bot = BotMap{}
func (b BotMap) RegisterCommand(command string, f BotFunc) error {
if _, exists := b[command]; exists {
return errors.New("command already exists")
}
b[command] = f
return nil
}
func (b BotMap) Execute(statement string) (string, error) {
// parse out command and query however you choose (not this way obviously)
command := statement[:9]
query := statement[10:]
return b.ExecuteQuery(command, query)
}
func (b BotMap) ExecuteQuery(command, query string) (string, error) {
if com, exists := b[command]; exists {
return com(query)
}
return "", errors.New("command doesn't exist")
}
func main() {
err := Bot.RegisterCommand("search me", func(query string) (string, error) {
fmt.Println("search", query)
return "searched", nil
})
if err != nil {
fmt.Println(err)
return
}
err = Bot.RegisterCommand("thesaurus me", func(query string) (string, error) {
fmt.Println("thesaurus", query)
return "thesaurused", nil
})
if err != nil {
fmt.Println(err)
return
}
result, err := Bot.Execute("search me please")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(result)
}
Obviously there's a lot of checks missing here, but this is the basic idea.

Decode JSON into Interface Value

As encoding/json needs a non-nil interface to unmarshal into: how can I reliably make a (full) copy of a user-provided pointer type, store that in my User interface, and then JSON decode into that ad-hoc?
Note: the goal here is to do this 'unattended' - that is, pull the bytes from Redis/BoltDB, decode into the interface type, and then check that the GetID() method the interface defines returns a non-empty string, with request middleware.
Playground: http://play.golang.org/p/rYODiNrfWw
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
"net/http"
"os"
"time"
)
type session struct {
ID string
User User
Expires int64
}
type User interface {
GetID() string
}
type LocalUser struct {
ID string
Name string
Created time.Time
}
func (u *LocalUser) GetID() string {
return u.ID
}
type Auth struct {
key []byte
// We store an instance of userType here so we can unmarshal into it when
// deserializing from JSON (or other non-gob decoders) into *session.User.
// Attempting to unmarshal into a nil session.User would otherwise fail.
// We do this so we can unmarshal from our data-store per-request *without
// the package user having to do so manually* in the HTTP middleware. We can't
// rely on the user passing in an fresh instance of their User satisfying type.
userType User
}
func main() {
// auth is a pointer to avoid copying the struct per-request: although small
// here, it contains a 32-byte key, options fields, etc. outside of this example.
var auth = &Auth{key: []byte("abc")}
local := &LocalUser{"19313", "Matt", time.Now()}
b, _, _, err := auth.SetUser(local)
if err != nil {
log.Fatalf("SetUser: %v", err)
}
user, err := auth.GetUser(b)
if err != nil {
log.Fatalf("GetUser: %#v", err)
}
fmt.Fprintf(os.Stdout, "%v\n", user)
}
func (auth *Auth) SetUser(user User) (buf []byte, id string, expires int64, err error) {
sess := newSession(user)
// Shallow copy the user into our config. struct so we can copy and then unmarshal
// into it in our middleware without requiring the user to provide it themselves
// at the start of every request
auth.userType = user
b := bytes.NewBuffer(make([]byte, 0))
err = json.NewEncoder(b).Encode(sess)
if err != nil {
return nil, id, expires, err
}
return b.Bytes(), sess.ID, sess.Expires, err
}
func (auth *Auth) GetUser(b []byte) (User, error) {
sess := &session{}
// Another shallow copy, which means we're re-using the same auth.userType
// across requests (bad).
// Also note that we need to copy into it session.User so that encoding/json
// can unmarshal into its fields.
sess.User = auth.userType
err := json.NewDecoder(bytes.NewBuffer(b)).Decode(sess)
if err != nil {
return nil, err
}
return sess.User, err
}
func (auth *Auth) RequireAuth(h http.Handler) http.Handler {
fn := func(w http.ResponseWriter, r *http.Request) {
// e.g. user, err := store.Get(r, auth.store, auth.userType)
// This would require us to have a fresh instance of userType to unmarshal into
// on each request.
// Alternative might be to have:
// func (auth *Auth) RequireAuth(userType User) func(h http.Handler) http.Handler
// e.g. called like http.Handle("/monitor", RequireAuth(&LocalUser{})(SomeHandler)
// ... which is clunky and using closures like that is uncommon/non-obvious.
}
return http.HandlerFunc(fn)
}
func newSession(u User) *session {
return &session{
ID: "12345",
User: u,
Expires: time.Now().Unix() + 3600,
}
}
If you need to deep copy an interface, add that method to your interface.
type User interface {
GetID() string
Copy() User
}
type LocalUser struct {
ID string
Name string
Created time.Time
}
// Copy receives a copy of LocalUser and returns a pointer to it.
func (u LocalUser) Copy() User {
return &u
}
Because the application will decode to a User and the argument to the JSON decoder must be a pointer value, we can assume that User values are pointer values. Given this assumption, the following code can be used to create a new zero value for decoding:
uzero := reflect.New(reflect.TypeOf(u).Elem()).Interface().(User)
playground example

Interface/Struct "Does not implement X, Wrong type or Method, not sure why I am getting this error

Hi guys fairly new to Golang, I understand that interfaces are kind of like contracts that guarantee that certain things will operate a certain way, thats cool and all, and if I make a local copy of it I can basically re-write how it operates (From what I understand, please correct me if I'm wrong)
Here is what I have so far
package register
import (
"log"
"net/http"
"github.com/yohcop/openid-go"
)
var nonceStore = &openid.SimpleNonceStore{
Store: make(map[string][]*openid.Nonce)}
var discoveryCache = &SimpleDiscoveryCache{}
type DiscoveredInfo interface {
OpEndpoint() string
OPLocalID() string
ClaimedID() string
}
type SimpleDiscoveredInfo struct {
opEndpoint, opLocalID, claimedID string
}
type SimpleDiscoveryCache map[string]DiscoveredInfo
func (s *SimpleDiscoveryCache) Put(id string, info DiscoveredInfo) {
db := common.ConnectDB()
rows, err := db.Query("INSERT INTO discovery_cache SET id=?, opendpoint=?, oplocalid=?, claimedid=?",
id, info.OpEndpoint(), info.OPLocalID(), info.ClaimedID())
if err != nil {
panic("Error: " + err.Error())
}
log.Println(rows)
}
func (s *SimpleDiscoveryCache) Get(id string) DiscoveredInfo {
db := common.ConnectDB()
rows, err := db.Query("SELECT FROM discovery_cache WHERE id=?", id)
if err != nil {
panic("Error: " + err.Error())
}
log.Println(rows)
var opEndpoint, opLocalID, claimedID string
for rows.Next() {
err := rows.Scan(&opEndpoint, &opLocalID, &claimedID)
if err != nil {
panic("Help!")
}
}
return &SimpleDiscoveredInfo{
opEndpoint, opLocalID, claimedID,
}
}
func DiscoverHandler(w http.ResponseWriter, r *http.Request) {
url, err := openid.RedirectURL("http://steamcommunity.com/openid", "http://localhost:1337/login/return", "http://localhost")
if err != nil {
http.Error(w, "Failed to login", 500)
}
http.Redirect(w, r, url, 303)
}
func CallbackHandler(w http.ResponseWriter, r *http.Request) {
fullUrl := "http://localhost:1337" + r.URL.String()
id, err := openid.Verify(fullUrl, discoveryCache, nonceStore)
if err != nil {
http.Error(w, "Failed", 500)
}
log.Println(id)
}
Basically I am trying to make my own DiscoveryCache so that it uses a database instead of memory for storage (as instructed to do by the Go-OpenID package located here: https://github.com/yohcop/openid-go
The part I'm trying to recreate is located here: https://github.com/yohcop/openid-go/blob/master/discovery_cache.go
Now I have done (what I assume) everything that should need doing to make this work, but I keep getting this error:
controllers/register/register.go:60: cannot use SimpleDiscoveredInfo literal (type *SimpleDiscoveredInfo) as type openid.DiscoveredInfo in return argument:
*SimpleDiscoveredInfo does not implement openid.DiscoveredInfo (missing ClaimedID method)
controllers/register/register.go:78: cannot use discoveryCache (type *SimpleDiscoveryCache) as type openid.DiscoveryCache in argument to openid.Verify:
*SimpleDiscoveryCache does not implement openid.DiscoveryCache (wrong type for Put method)
have Put(string, DiscoveredInfo)
want Put(string, openid.DiscoveredInfo)
If anybody could inform me on what I have done wrong that would be much appreciated. Thanks! If you need any more information please let me know.
SimpleDiscoveredInfo doesn't implement the interface's methods, you need something like this:
func (sdi *SimpleDiscoveredInfo) OpEndpoint() string { return sdi.opEndpoint }
func (sdi *SimpleDiscoveredInfo) OpLocalID() string { return sdi.opLocalID }
func (sdi *SimpleDiscoveredInfo) ClaimedID() string { return sdi.claimedID }
var _ openid.DiscoveredInfo = (*SimpleDiscoveredInfo)(nil)
http://play.golang.org/p/qVTTKfhNHu
For
controllers/register/register.go:78: cannot use discoveryCache (type *SimpleDiscoveryCache) as type openid.DiscoveryCache in argument to openid.Verify:
*SimpleDiscoveryCache does not implement openid.DiscoveryCache (wrong type for Put method)
have Put(string, DiscoveredInfo)
want Put(string, openid.DiscoveredInfo)
Your types need to return openid.DiscoveredInfo not DiscoveredInfo.

Golang downcasting list of structs

I want to be able to unmarshal yaml files less rigidly. That is, my library has a predefined number of options the yaml file must have. Then, the user should be able to extend this to include any custom options.
Here is what I have
package main
import (
"net/http"
"yamlcms"
"github.com/julienschmidt/httprouter"
)
type Page struct {
*yamlcms.Page
Title string
Date string
}
func getBlogRoutes() {
pages := []*Page{}
yamlcms.ReadDir("html", pages)
}
// This section is a work in progress, I only include it for loose context
func main() {
router := httprouter.New()
//blogRoutes := getBlogRoutes()
//for _, blogRoute := range *blogRoutes {
// router.Handle(blogRoute.Method, blogRoute.Pattern,
// func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {})
//}
http.ListenAndServe(":8080", router)
}
Here is the yamlcms package:
package yamlcms
import (
"io/ioutil"
"os"
"strings"
"gopkg.in/yaml.v2"
)
type Page struct {
Slug string `yaml:"slug"`
File string `yaml:"file"`
}
func (page *Page) ReadFile(file string) (err error) {
fileContents, err := ioutil.ReadFile(file)
if err != nil {
return
}
err = yaml.Unmarshal(fileContents, &page)
return
}
func isYamlFile(fileInfo os.FileInfo) bool {
return !fileInfo.IsDir() && strings.HasSuffix(fileInfo.Name(), ".yaml")
}
func ReadDir(dir string, pages []*Page) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
pages[i].ReadFile(fileInfo.Name())
}
}
return
}
There is a compiler issue here:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.Page in argument to yamlcms.ReadDir
My main intent in this question is to learn the idiomatic way of doing this kind of thing in Go. Other 3rd-party solutions may exist but I am not immediately interested in them because I have problems like this frequently in Go having to do with inheritance, etc. So along the lines of what I've presented, how can I best (idiomatically) accomplish what I am going for?
EDIT:
So I've made some changes as suggested. Now I have this:
type FileReader interface {
ReadFile(file string) error
}
func ReadDir(dir string, pages []*FileReader) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
(*pages[i]).ReadFile(fileInfo.Name())
}
}
return
}
However, I still get a similar compiler error:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.FileReader in argument to yamlcms.ReadDir
Even though main.Page should be a FileReader because it embeds yamlcms.Page.
EDIT: I forgot that slices of interfaces don't work like that. You'd need to allocate a new slice, convert all pages to FileReaders, call the function, and convert them back.
Another possible solution is refactoring yamlcms.ReadDir to return the contents of the files, so that they could be unmarshaled later:
// In yamlcms.
func ReadYAMLFilesInDir(dir string) ([][]byte, error) { ... }
// In client code.
files := yamlcms.ReadYAMLFilesInDir("dir")
for i := range pages {
if err := yaml.Unmarshal(files[i], &pages[i]); err != nil { return err }
}
The original answer:
There are no such things as inheritance or casting in Go. Prefer composition and interfaces in your designs. In your case, you can redefine your yamlcms.ReadDir to accept an interface, FileReader.
type FileReader interface {
ReadFile(file string) error
}
Both yamlcms.Page and main.Page will implement this, as the latter embeds the former.

Resources