I have a few structs in a package that make a simple finance API call, one implements the request to the API, one stores the response.
I'm wondering if it's appropriate to just name the structs "Request" and "Response" respectively, or should I prefix them with the subject/package- like "FinanceRequest" and "FinanceResponse"? Or would that make external calls redundant by using finance.FinanceRequest (finance word used twice)?
Looking for thoughts on the matter (golang convention/preference)...
Samples:
package finance
type Request struct {
//...
}
type Response struct {
//...
}
func DoSomething(r *Request) (*Response, error) {
//...
}
or
package finance
type FinanceRequest struct {
//...
}
type FinanceResponse struct {
//...
}
func DoSomething(r *FinanceRequest) (*FinanceResponse, error) {
//...
}
The convention is to use Request and Response. Here's what Effective Go has to say:
The importer of a package will use the name to refer to its contents, so exported names in the package can use that fact to avoid stutter. (Don't use the import . notation, which can simplify tests that must run outside the package they are testing, but should otherwise be avoided.) For instance, the buffered reader type in the bufio package is called Reader, not BufReader, because users see it as bufio.Reader, which is a clear, concise name. Moreover, because imported entities are always addressed with their package name, bufio.Reader does not conflict with io.Reader.
The golint tool will complain about the names FinanceRequest and FinanceResponse.
Related
If I have a requests package that defines an interface TextExtractor with a GetText method that returns a Text type, the implementations must fulfill the TextExtractor contract exactly, and they are forced to import the Text type.
I have two possible implementations of TextExtractor - one that uses AWS Comprehend and one that uses AWS Textract.
aws_comprehend.go
package aws_comprehend
type AwsComprehend struct{}
func (a *AwsComprehend) GetText() *Text {
// do some stuff with aws comprehend...
return &Text{
Text: "text",
Language: "language",
}
}
type Text struct {
Text string
Language string
}
request.go
package requests
import "fmt"
type TextExtractor interface {
GetText() *Text
}
type Text struct {
Text string
Language string
}
func HandleRequest(textExtractor TextExtractor) {
text := textExtractor.GetText()
fmt.Println(text)
}
main.go
package main
import (
"aws_comprehend"
"requests"
)
func main() {
textExtractor := new(aws_comprehend.AwsComprehend)
requests.HandleRequest(textExtractor)
// this does not work:
// cannot use textExtractor (variable of type *aws_comprehend.AwsComprehend) as
// requests.TextExtractor value in argument to requests.HandleRequest:
// *aws_comprehend.AwsComprehend does not implement requests.TextExtractor
// (wrong type for method GetText)
// have GetText() *aws_comprehend.Text
// want GetText() *requests.Text
}
I understand why this doesn't work. It's because GoLang doesn't support Covariant Result Types. But my question is, what is the standard way to code this situation? The fact that GoLang provides implicit interfaces means that isolating packages is very easy: the calling package defines the interfaces that it uses, and it is passed implementations that fulfill those interfaces. This means that packages don't have to reference each other at all. But if a package defines an interface that returns anything more than a primitive value, then you have to deliberately share those value types. The code above would be fine if GetText returned a string. But the fact that it returns a struct or another interface, means the code can't be written this way.
I want the requests package not to know anything about the aws_comprehend package. This is because I have two implementations of the TextExtractor interface: One that uses AWS Comprehend, and one that uses AWS Textract. I also would prefer not to include a "intermediate" package that has interfaces that both the requests package and the aws_comprehend package inherit from. If both packages have to inherit the same interface, then it seems like it's just indirect coupling to me and it ruins the idea of a implicit interfaces.
I understand that GoLang is very opinionated - So what is the standard way to solve this problem?
first, your file layout is not valid. you cant have two files in the same
folder, with different packages. so below is a corrected layout. also I removed
all pointers, because they aren't needed for this example and are just distracting.
Finally, I updated the method to return the correct type, so that the code
actually compiles:
aws_comprehend/aws_comprehend.go:
package aws_comprehend
import "hello/requests"
type AwsComprehend struct{}
func (AwsComprehend) GetText() requests.Text {
return requests.Text{}
}
requests/request.go:
package requests
import "fmt"
type TextExtractor interface {
GetText() Text
}
type Text struct {
Language string
Text string
}
func HandleRequest(textExtractor TextExtractor) {
text := textExtractor.GetText()
fmt.Println(text)
}
main.go:
package main
import (
"hello/aws_comprehend"
"hello/requests"
)
func main() {
var textExtractor aws_comprehend.AwsComprehend
requests.HandleRequest(textExtractor)
}
I am newer to golang, so I have some courses that I bought from udemy to help break me into the language. One of them I found very helpful for a general understanding as I took on a project in the language.
In the class that I took, all of the sql related functions were in the sqlc folder with the structure less broken out:
sqlc
generatedcode
store
One of those files is a querier that is generated by sqlc that contains an interface with all of the methods that were generated. Here is the general idea of what it currently looks like: https://github.com/techschool/simplebank/tree/master/db/sqlc
package db
import (
"context"
"github.com/google/uuid"
)
type Querier interface {
AddAccountBalance(ctx context.Context, arg AddAccountBalanceParams) (Account, error)
CreateAccount(ctx context.Context, arg CreateAccountParams) (Account, error)
...
}
var _ Querier = (*Queries)(nil)
Would it be possible to wrap both what sqlc generates AND any queries that a developer creates (dynamic queries) into a single querier? I'm also trying to have it so that the sqlc generated code is in its own folder. The structure I am aiming for is:
sql
sqlc
generatedcode
store - (wraps it all together)
dynamicsqlfiles
This should clear up what a I mean by store: https://github.com/techschool/simplebank/blob/master/db/sqlc/store.go
package db
import (
"context"
"database/sql"
"fmt"
)
// Store defines all functions to execute db queries and transactions
type Store interface {
Querier
TransferTx(ctx context.Context, arg TransferTxParams) (TransferTxResult, error)
}
// SQLStore provides all functions to execute SQL queries and transactions
type SQLStore struct {
db *sql.DB
*Queries
}
// NewStore creates a new store
func NewStore(db *sql.DB) Store {
return &SQLStore{
db: db,
Queries: New(db),
}
}
I'm trying to run everything through that store (both generated and my functions), so I can make a call similar to the CreateUser function in this file (server.store.): https://github.com/techschool/simplebank/blob/master/api/user.go
arg := db.CreateUserParams{
Username: req.Username,
HashedPassword: hashedPassword,
FullName: req.FullName,
Email: req.Email,
}
user, err := server.store.CreateUser(ctx, arg)
if err != nil {
if pqErr, ok := err.(*pq.Error); ok {
switch pqErr.Code.Name() {
case "unique_violation":
ctx.JSON(http.StatusForbidden, errorResponse(err))
return
}
}
ctx.JSON(http.StatusInternalServerError, errorResponse(err))
return
}
I've tried creating something that houses another querier interface that embeds the generated one, then creating my own db.go that uses the generated DBTX interface but has its own Queries struct, and New function. It always gives me an error that the Queries struct I created aren't implementing the functions I made, despite having it implemented in one of the custom methods I made.
I deleted that branch, and have been clicking through the simplebank project linked above to see if I can find another way this could be done, or if I missed something. If it can't be done, that's okay. I'm just using this as a good opportunity to learn a little more about the language, and keep some code separated if possible.
UPDATE:
There were only a few pieces I had to change, but I modified the store.go to look more like:
// sdb is imported, but points to the generated Querier
// Store provides all functions to execute db queries and transactions
type Store interface {
sdb.Querier
DynamicQuerier
}
// SQLStore provides all functions to execute SQL queries and transactions
type SQLStore struct {
db *sql.DB
*sdb.Queries
*dynamicQueries
}
// NewStore creates a new Store
func NewStore(db *sql.DB) Store {
return &SQLStore{
db: db,
Queries: sdb.New(db),
dynamicQueries: New(db),
}
}
Then just created a new Querier and struct for the methods I would be creating. Gave them their own New function, and tied it together in the above. Before, I was trying to figure out a way to reuse as much of the generated code as possible, which I think was the issue.
Why I wanted the Interface:
I wanted a structure that separated the files I would be working in more from the files that I would never touch (generated). This is the new structure:
I like how the generated code put everything in the Querier interface, then checked that anything implementing it satisfied all of the function requirements. So I wanted to replicate that for the dynamic portion which I would be creating on my own.
It might be complicating it a bit more than it would 'NEED' to be, but it also provides an additional set of error checking that is nice to have. And in this case, even while maybe not necessary, it ended up being doable.
Would it be possible to wrap both what sqlc generates AND any queries that a developer creates (dynamic queries) into a single querier?
If I'm understanding your question correctly I think that you are looking for something like the below (playground):
package main
import (
"context"
"database/sql"
)
// Sample SQL C Code
type DBTX interface {
ExecContext(context.Context, string, ...interface{}) (sql.Result, error)
PrepareContext(context.Context, string) (*sql.Stmt, error)
QueryContext(context.Context, string, ...interface{}) (*sql.Rows, error)
QueryRowContext(context.Context, string, ...interface{}) *sql.Row
}
type Queries struct {
db DBTX
}
func (q *Queries) DeleteAccount(ctx context.Context, id int64) error {
// _, err := q.db.ExecContext(ctx, deleteAccount, id)
// return err
return nil // Pretend that this always works
}
type Querier interface {
DeleteAccount(ctx context.Context, id int64) error
}
//
// Your custom "dynamic" queries
//
type myDynamicQueries struct {
db DBTX
}
func (m *myDynamicQueries) GetDynamicResult(ctx context.Context) error {
// _, err := q.db.ExecContext(ctx, deleteAccount, id)
// return err
return nil // Pretend that this always works
}
type myDynamicQuerier interface {
GetDynamicResult(ctx context.Context) error
}
// Combine things
type allDatabase struct {
*Queries // Note: You could embed this directly into myDynamicQueries instead of having a seperate struct if that is your preference
*myDynamicQueries
}
type DatabaseFunctions interface {
Querier
myDynamicQuerier
}
func main() {
// Basic example
var db DatabaseFunctions
db = getDatabase()
db.DeleteAccount(context.Background(), 0)
db.GetDynamicResult(context.Background())
}
// getDatabase - Perform whatever is needed to connect to database...
func getDatabase() allDatabase {
sqlc := &Queries{db: nil} // In reality you would use New() to do this!
myDyn := &myDynamicQueries{db: nil} // Again it's often cleaner to use a function
return allDatabase{Queries: sqlc, myDynamicQueries: myDyn}
}
The above is all in one file for simplicity but could easily pull from multiple packages e.g.
type allDatabase struct {
*generatedcode.Queries
*store.myDynamicQueries
}
If this does not answer your question then please show one of your failed attempts (so we can see where you are going wrong).
One general comment - do you really need the interface? A common recommendation is "Accept interfaces, return structs". While this may not always apply I suspect you may be introducing interfaces where they are not really necessary and this may add unnecessary complexity.
I thought that the Store, which was housing both Queriers, was tying it all together. Can you explain a little with the example above (in the question post) why it's not necessary? How does SQLStore get access to all of the Querier interface functions?
The struct SQLStore is what is "tying it all together". As per the Go spec:
Given a struct type S and a named type T, promoted methods are included in the method set of the struct as follows:
If S contains an embedded field T, the method sets of S and *S both include promoted methods with receiver T. The method set of *S also includes promoted methods with receiver *T.
If S contains an embedded field *T, the method sets of S and *S both include promoted methods with receiver T or *T.
So an object of type SQLStore:
type SQLStore struct {
db *sql.DB
*sdb.Queries
*dynamicQueries
}
var foo SQLStore // Assume that we are actually providing values for all fields
Will implement all of the methods of sdb.Queries and, also, those in dynamicQueries (you can also access the sql.DB members via foo.db.XXX). This means that you can call foo.AddAccountBalance() and foo.MyGenericQuery() (assuming that is in dynamicQueries!) etc.
The spec says "In its most basic form an interface specifies a (possibly empty) list of methods". So you can think of an interface as a list of functions that must be implemented by whatever implementation (e.g. struct) you assign to the interface (the interface itself does not implement anything directly).
This example might help you understand.
Hopefully that helps a little (as I'm not sure which aspect you don't understand I'm not really sure what to focus on).
I'm writing a piece of code that relies on some implementation.
I want to decouple the implementation from my code, and make the implementation as independent as possible.
I thought of achieving this approach by using interfaces instead of concrete types, like so:
package mypackage
type MyType interface {
Title() string
Price() int
}
type TypeGetter interface {
GetType() MyType
}
func MyHandler(tg TypeGetter) {
t := tg.GetType()
fmt.Printf("Title: %s, Price: %d", t.Title(), t.Price())
}
And an implementation might be something like this:
package external
// CustomType implicitly implements the MyType interface
type CustomType struct {
title string
price int
}
func (t CustomType) Title() string { return t.title }
func (t CustomType) Price() int { return t.price }
// CustomTypeGetter implicitly implements the TypeGetter interface. Or is it???
type CustomTypeGetter struct {
}
func (g CustomTypeGetter) GetType() CustomType {
return CustomType{"Hello", 42}
}
Then, the code would do something like this:
package main
import "mypackage"
import "external"
func main() {
tg := external.CustomTypeGetter{}
mypackage.MyHandler(tg) // <--- the compiler does not like this
}
I hope the example speaks for itself: I have no coupling between "mypackage" and the "external" package, which may be replaced, substituted my mocks for testing, etc.
The problem: the compiler complains that the call to MyHandler has an object that implements:
func GetType() CustomType, instead of:
func GetType() MyType
The only solution I found is to move the interface declarations (MyType and TypeGetter) to a third package, and then both "mypackage" and "external" packages can use it.
But I want to avoid that.
Isn't Go's concept of implicit implementation of interfaces contradict the idea of a third common package?
Is there a way to implement such thing, without binding the two packages together?
Isn't Go's concept of implicit implementation of interfaces contradict the idea of a third common package?
I think it does. Go authors introduced an implicit interface implementation to eliminate unnecessary dependencies between packages. That works well for simple interfaces like io.Reader, but you cannot apply it everywhere.
One of the language creators, Rob Pike, says that the non-declarative satisfaction of interfaces is not the essential part of the idea behind interfaces in Go. It's a nice feature, but not all elements of the language are practical or possible to use every time.
For complex interfaces, you need to import a package where the interface is defined. For example, if you want to implement an SQL driver that works with the sql package from the standard library, you must import the sql/driver package.
I would recommend not introducing interfaces at the beginning of your project. Usually, it leads to situations where you need to solve artificial problems like rewriting the interface each time you updates your understanding of the domain model. It is hard to come up with a good abstraction from the first attempt, and, in many cases, it is unnecessary, in my opinion.
I need to query external source for products. I don't care how the external sources store the data (db, file, network). I just need a "product" type. So it's either I define a Product type, forcing the external implementations to import and use it, or the Go way - define a Product interface and let the implementations implicitly implement this interface. Which apparently doesn't work
I see two loosely related goals here:
Define an interface to swap implementations of the product source.
A package that implements the product source should not import the package that defines the interface.
From my experience, I would recommend doing point 1 only when you have at least one working implementation of the product source service.
Point 2 is not always possible to achieve, and it is fine; please see the example from the standard Go library above.
P.S.
Please, consider not creating Product interface. While it does makes sense to come up with the PorductSource interface eventually, Product is most probably just a set of data; struct is a perfect way to represent such information. Please, see this very relevant code smaple and this article for inspiration.
The problem with your approach is that you want someone to implement an interface that refers to your type (MyType). This obviously cannot be done without the implementation referring to your type. This is the only thing that prevents the above code from working.
If you get rid of the MyType:
type TypeGetter interface {
GetType() interface {
Title() string
Price() int
}
}
And the implementation:
func (g CustomTypeGetter) GetType() interface {
Title() string
Price() int
} {
return CustomType{"Hello", 42}
}
Then this code will work:
func main() {
tg := external.CustomTypeGetter{}
mypackage.MyHandler(tg)
}
Yes, this requires repetition, but only because you don't want an unknown / future implementation to refer to your type (to not depend on it).
In this case you may change MyHandler() to take a value of type MyType (get rid of the "factory"):
func MyHandler(t MyType) {
fmt.Printf("Title: %s, Price: %d", t.Title(), t.Price())
}
And any value that implements MyType may be passed. Add a "factory" to the external package:
func NewCustomType(title string, price int) CustomType {
return CustomType{
title: title,
price: price,
}
}
And use it like this:
func main() {
t := external.NewCustomType("title", 1)
mypackage.MyHandler(t)
}
If you truly require the factory pattern, then yes, creating a 3rd package that will hold MyType is the way to go. Then both your app and the implementations may refer to this 3rd package.
The package valyala/fasthttp implements the following function type:
type RequestHandler func(ctx *RequestCtx)
It is used in buaazp/fasthttprouter like this:
func (r *Router) Handle(method, path string, handle fasthttp.RequestHandler) {
//...
}
I am trying to wrap these like this (open for suggestions on implementation):
//myapp/router
type Request struct {
fasthttp.RequestCtx
}
type RequestHandler func(*Request)
func Handle(method string, path string, handler RequestHandler) {
//I need to access the fasthttp.RequestCtx stuff in here...
}
How can I achieve this? Or, if this is not the way to go at all, how can I achieve my goal as mentioned below for a router package?
BACKGROUND
Goal: My wish is to wrap tooling packages (sessions, database, routing, etc.) in order to make my app agnostic to the implementation of these packages. I wish to do this primarily for the purpose of being able to extend these with domain-specific functionality, and being able to switch one 3rd party lib for another, if I ever would need to do so. It also makes debugging and logging easier.
Method: I create native types and functions, which map to the functionality of the imported packages.
Problem: I am stuck on how to wrap a foreign (i.e. imported) function type properly.
At all your idea looks very good. Some things you could change:
//myapp/router
// Using a composition is idiomatic go code
// this should work. It can't get better.
type Request struct {
fasthttp.RequestCtx
}
// I would make the RequestHandler as a real Handler. In go it would be
// a interface
type RequestHandler interface{
Request(*Request)
}
// If you have a function, which needs to access parameters from `Request`
// you should take this as an input.
func Handle(method string, path string, req *Request) {
//Access Request via req.Request ...
}
Because if you pass a function or an interface into your function, which needs also Request as input the caller needs to create that before he calls your Handle function. Why not change that function just for the input you really need?
I understand that if go code is structured such that it's programmed to interfaces, it's trivial to mock; however, I'm working with a code base that I cannot change (that is not mine) and this is not the case.
This code base is heavily interconnected and nothing is programmed to an interface, only structs, so no dependency injection.
The structs, themselves, only contain other structs, so I can't mock out that way either. I don't believe that I can do anything about methods, and the few functions that exist are not variables, so there's no way that I know of to swap them out. Inheritance isn't a thing in golang, so that's a no go as well.
In scripting languages like python, we can modify the objects at runtime, aka monkey patch. Is there something comparable that I can do in golang? Trying to figure out some way to test/benchmark without touching the underlying code.
When I have run into this situation my approach is to use my own interface as a wrapper which allows mocking in tests. For example.
type MyInterface interface {
DoSomething(i int) error
DoSomethingElse() ([]int, error)
}
type Concrete struct {
client *somepackage.Client
}
func (c *Concrete) DoSomething(i int) error {
return c.client.DoSomething(i)
}
func (c *Concrete) DoSomethingElse() ([]int, error) {
return c.client.DoSomethingElse()
}
Now you can mock the Concrete in the same way you would mock somepackage.Client if it too were an interface.
As pointed out in the comments below by #elithrar, you can embed the type you want to mock so you are only forced to add methods which need mocking. For example:
type Concrete struct {
*somepackage.Client
}
When done like that, additional methods like DoSomethingNotNeedingMocking could be called directly on Concrete without having to add it to the interface / mock it out.
There is an available monkey patching library for Go. It only works for Intel/AMD systems (targeting OSX and Ubuntu in particular).
Depending on the situation, you can apply the "Dependency Inversion Principle" and leverage Go's implicit interfaces.
To do this, you define an interface of your requirements in the package with the usage (as opposed to defining what you provide in the package that implements it; like you might in Java).
Then you can test your code in isolation from the dependency.
Typical object with a struct dependency:
// Object that relies on a struct
type ObjectUnderTestBefore struct {
db *sql.DB
}
func (o *ObjectUnderTestBefore) Delete() error {
o.db.Exec("DELETE FROM sometable")
}
Apply Dependency Inversion Principle (with implicit interface)
// subset of sql.DB which defines our "requirements"
type dbExec interface {
Exec(query string, args ...interface{}) (sql.Result, error)
}
// Same object with it's requirement defined as an local interface
type ObjectUnderTestWithDIP struct {
// *sql.DB will implicitly implement this interface
db dbExec
}
func (o *ObjectUnderTestWithDIP) Delete() error {
o.db.Exec("DELETE FROM sometable")
}