I have two functions as shown below which look similar, but using different functions to query the db. Since Go doesn't encourage overloaading methods, is the redundancy acceptable? Or should I refactor them into one function? All comments are welcomed.
var getCustomers = func() ([]customer, error) {
return nil, nil
}
var getCustomerById = func(int64) (*customer, error) {
return nil, nil
}
func listCustomer(w http.ResponseWriter, r *http.Request) *appError {
cus, err := getCustomers()
if err != nil {
return &appError{err, "No customer found", 404}
}
res, err := json.Marshal(cus)
if err != nil {
return &appError{err, "Can't display record", 500}
}
fmt.Fprint(w, string(res))
return nil
}
func viewCustomer(w http.ResponseWriter, r *http.Request, id int64) *appError {
cus, err := getCustomerByID(id)
if err != nil {
return &appError{err, "No customer found", 404}
}
res, err := json.Marshal(cus)
if err != nil {
return &appError{err, "Can't display record", 500}
}
fmt.Fprint(w, string(res))
return nil
}
Suggestion to use -1 to list all customer, but I'm not sure if this is the best it can be:
func viewCustomer(w http.ResponseWriter, r *http.Request, id int64) *appError {
var c *customer
var clist []customer
var err error
if id < 0 {
clist, err = getCustomers()
res, err := json.Marshal(clist)
if err != nil {
return &appError{err, "Can't display record", 500}
}
fmt.Fprint(w, string(res))
return nil
} else {
c, err = getCustomerById(id)
res, err := json.Marshal(c)
if err != nil {
return &appError{err, "Can't display record", 500}
}
fmt.Fprint(w, string(res))
return nil
}
}
Since Go doesn't encourage overloaading methods, is the redundancy acceptable? Or should I refactor them into one function?
The answer to your question depends heavily on the real code and your task. If listing one customer is very different from listing several customers (that is, you need different information and have different presentation logic), I'd say that duplication here is not that bad, since the difference may grow larger in the future, so a DRY solution could turn into an if and switch mess quickly. (I've had a similar experience on a non-Go project and since then I think that DRY is good but you should not be fanatic about it.)
On the other hand, if you're making, say a JSON API, you could make it more DRY. Define your getCustomers function like this:
func customers(ids ...int64) ([]customer, error) { /* ... */ }
This way you can:
call it like customers() and get all customers;
call customers(42) and get a customer whose ID is 42;
call customers(someIDs...) and get multiple customers by their IDs.
All can be done in one handler in a straightforward way.
All comments are welcomed.
Two nitpicks on your code:
Getter methods in Go are idiomatically named foo and not getFoo, so I used customers in my example. You would most probably call it as DB.Customers(ids...), so it looks a lot like a getter method.
What's up with var foo = func() notation? Unless there is a reason for that (like using these functions as closures), I'd suggest sticking with the func foo() notation, since it's more idiomatic and generally easier to grep. EDIT: as OP pointed out in the comments, another case for var foo = func() notation is of course functions that can be faked in testing, as shown in Andrew Gerrand's Testing Techniques talk.
I hope that helps.
Related
I'm writing a chat bot in Go and wondering how can I avoid a long switch-case statement similar to this one:
switch {
// #bot search me HMAC
case strings.Contains(message, "search me"):
query := strings.Split(message, "search me ")[1]
return webSearch(query), "html"
// #bot thesaurus me challenge
case strings.Contains(message, "thesaurus me"):
query := strings.Split(message, "thesaurus me ")[1]
return synonyms(query), "html"
Should I define those handlers each in a separate package or should I just use structs and interfaces? Which method will allow me to have a good structure, avoid switch-case and let external developers to easier create handlers?
I think packages will be a better choice but I'm not sure how to register the handlers with the main bot. Would appreciate an example.
You could use a map[string]command similar to how the net/http package registers handlers. Something akin to this:
https://play.golang.org/p/9YzHyLodAQ
package main
import (
"fmt"
"errors"
)
type BotFunc func(string) (string, error)
type BotMap map[string]BotFunc
var Bot = BotMap{}
func (b BotMap) RegisterCommand(command string, f BotFunc) error {
if _, exists := b[command]; exists {
return errors.New("command already exists")
}
b[command] = f
return nil
}
func (b BotMap) Execute(statement string) (string, error) {
// parse out command and query however you choose (not this way obviously)
command := statement[:9]
query := statement[10:]
return b.ExecuteQuery(command, query)
}
func (b BotMap) ExecuteQuery(command, query string) (string, error) {
if com, exists := b[command]; exists {
return com(query)
}
return "", errors.New("command doesn't exist")
}
func main() {
err := Bot.RegisterCommand("search me", func(query string) (string, error) {
fmt.Println("search", query)
return "searched", nil
})
if err != nil {
fmt.Println(err)
return
}
err = Bot.RegisterCommand("thesaurus me", func(query string) (string, error) {
fmt.Println("thesaurus", query)
return "thesaurused", nil
})
if err != nil {
fmt.Println(err)
return
}
result, err := Bot.Execute("search me please")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(result)
}
Obviously there's a lot of checks missing here, but this is the basic idea.
I have created an object mapping in Go that is not relational, it is very simple.
I have several structs that looks like this:
type Message struct {
Id int64
Message string
ReplyTo sql.NullInt64 `db:"reply_to"`
FromId int64 `db:"from_id"`
ToId int64 `db:"to_id"`
IsActive bool `db:"is_active"`
SentTime int64 `db:"sent_time"`
IsViewed bool `db:"is_viewed"`
Method string `db:"-"`
AppendTo int64 `db:"-"`
}
To create a new message I just run this function:
func New() *Message {
return &Message{
IsActive: true,
SentTime: time.Now().Unix(),
Method: "new",
}
}
And then I have a message_crud.go file for this struct that looks like this:
To find a message by a unique column (for example by id) I run this function:
func ByUnique(column string, value interface{}) (*Message, error) {
query := fmt.Sprintf(`
SELECT *
FROM message
WHERE %s = ?
LIMIT 1;
`, column)
message := &Message{}
err := sql.DB.QueryRowx(query, value).StructScan(message)
if err != nil {
return nil, err
}
return message, nil
}
And to save a message (insert or update in the database) I run this method:
func (this *Message) save() error {
s := ""
if this.Id == 0 {
s = "INSERT INTO message SET %s;"
} else {
s = "UPDATE message SET %s WHERE id=:id;"
}
query := fmt.Sprintf(s, sql.PlaceholderPairs(this))
nstmt, err := sql.DB.PrepareNamed(query)
if err != nil {
return err
}
res, err := nstmt.Exec(*this)
if err != nil {
return err
}
if this.Id == 0 {
lastId, err := res.LastInsertId()
if err != nil {
return err
}
this.Id = lastId
}
return nil
}
The sql.PlaceholderPairs() function looks like this:
func PlaceholderPairs(i interface{}) string {
s := ""
val := reflect.ValueOf(i).Elem()
count := val.NumField()
for i := 0; i < count; i++ {
typeField := val.Type().Field(i)
tag := typeField.Tag
fname := strings.ToLower(typeField.Name)
if fname == "id" {
continue
}
if t := tag.Get("db"); t == "-" {
continue
} else if t != "" {
s += t + "=:" + t
} else {
s += fname + "=:" + fname
}
s += ", "
}
s = s[:len(s)-2]
return s
}
But every time I create a new struct, for example a User struct I have to copy paste the "crud section" above and create a user_crud.go file and replace the words "Message" with "User", and the words "message" with "user". I repeat alot of code and it is not very dry. Is there something I could do to not repeat this code for things I would reuse? I always have a save() method, and always have a function ByUnique() where I can return a struct and search by a unique column.
In PHP this was easy because PHP is not statically typed.
Is this possible to do in Go?
Your ByUnique is almost generic already. Just pull out the piece that varies (the table and destination):
func ByUnique(table string, column string, value interface{}, dest interface{}) error {
query := fmt.Sprintf(`
SELECT *
FROM %s
WHERE %s = ?
LIMIT 1;
`, table, column)
return sql.DB.QueryRowx(query, value).StructScan(dest)
}
func ByUniqueMessage(column string, value interface{}) (*Message, error) {
message := &Message{}
if err := ByUnique("message", column, value, &message); err != nil {
return nil, err
}
return message, error
}
Your save is very similar. You just need to make a generic save function along the lines of:
func Save(table string, identifier int64, source interface{}) { ... }
Then inside of (*Message)save, you'd just call the general Save() function. Looks pretty straightforward.
Side notes: do not use this as the name of the object inside a method. See the link from #OneOfOne for more on that. And do not get obsessed about DRY. It is not a goal in itself. Go focuses on code being simple, clear, and reliable. Do not create something complicated and fragile just to avoid typing a simple line of error handling. This doesn't mean that you shouldn't extract duplicated code. It just means that in Go it is usually better to repeat simple code a little bit rather than create complicated code to avoid it.
EDIT: If you want to implement Save using an interface, that's no problem. Just create an Identifier interface.
type Ider interface {
Id() int64
SetId(newId int64)
}
func (msg *Message) Id() int64 {
return msg.Id
}
func (msg *Message) SetId(newId int64) {
msg.Id = newId
}
func Save(table string, source Ider) error {
s := ""
if source.Id() == 0 {
s = fmt.Sprintf("INSERT INTO %s SET %%s;", table)
} else {
s = fmt.Sprintf("UPDATE %s SET %%s WHERE id=:id;", table)
}
query := fmt.Sprintf(s, sql.PlaceholderPairs(source))
nstmt, err := sql.DB.PrepareNamed(query)
if err != nil {
return err
}
res, err := nstmt.Exec(source)
if err != nil {
return err
}
if source.Id() == 0 {
lastId, err := res.LastInsertId()
if err != nil {
return err
}
source.SetId(lastId)
}
return nil
}
func (msg *Message) save() error {
return Save("message", msg)
}
The one piece that might blow up with this is the call to Exec. I don't know what package you're using, and it's possible that Exec won't work correctly if you pass it an interface rather than the actual struct, but it probably will work. That said, I'd probably just pass the identifier rather than adding this overhead.
You probably want to use an ORM.
They eliminate a lot of the boilerplate code you're describing.
See this question for "What is an ORM?"
Here is a list of ORMs for go: https://github.com/avelino/awesome-go#orm
I have never used one myself, so I can't recommend any. The main reason is that an ORM takes a lot of control from the developer and introduces a non-negligible performance overhead. You need to see for yourself if they fit your use-case and/or if you are comfortable with the "magic" that's going on in those libraries.
I don't recommend doing this, i personally would prefer being explicit about scanning into structs and creating queries.
But if you really want to stick to reflection you could do:
func ByUnique(obj interface{}, column string, value interface{}) error {
// ...
return sql.DB.QueryRowx(query, value).StructScan(obj)
}
// Call with
message := &Message{}
ByUnique(message, ...)
And for your save:
type Identifiable interface {
Id() int64
}
// Implement Identifiable for message, etc.
func Save(obj Identifiable) error {
// ...
}
// Call with
Save(message)
The approach i use and would recommend to you:
type Redirect struct {
ID string
URL string
CreatedAt time.Time
}
func FindByID(db *sql.DB, id string) (*Redirect, error) {
var redirect Redirect
err := db.QueryRow(
`SELECT "id", "url", "created_at" FROM "redirect" WHERE "id" = $1`, id).
Scan(&redirect.ID, &redirect.URL, &redirect.CreatedAt)
switch {
case err == sql.ErrNoRows:
return nil, nil
case err != nil:
return nil, err
}
return &redirect, nil
}
func Save(db *sql.DB, redirect *Redirect) error {
redirect.CreatedAt = time.Now()
_, err := db.Exec(
`INSERT INTO "redirect" ("id", "url", "created_at") VALUES ($1, $2, $3)`,
redirect.ID, redirect.URL, redirect.CreatedAt)
return err
}
This has the advantage of using the type system and mapping only things it should actually map.
I try to implement repository pattern in Go app (simple web service) and try to find better way to escape code duplication.
Here is a code
Interfaces are:
type IRoleRepository interface {
GetAll() ([]Role, error)
}
type ISaleChannelRepository interface {
GetAll() ([]SaleChannel, error)
}
And implementation:
func (r *RoleRepository) GetAll() ([]Role, error) {
var result []Role
var err error
var rows *sql.Rows
if err != nil {
return result, err
}
connection := r.provider.GetConnection()
defer connection.Close()
rows, err = connection.Query("SELECT Id,Name FROM Position")
defer rows.Close()
if err != nil {
return result, err
}
for rows.Next() {
entity := new(Role)
err = sqlstruct.Scan(entity, rows)
if err != nil {
return result, err
}
result = append(result, *entity)
}
err = rows.Err()
if err != nil {
return result, err
}
return result, err
}
func (r *SaleChannelRepository) GetAll() ([]SaleChannel, error) {
var result []SaleChannel
var err error
var rows *sql.Rows
if err != nil {
return result, err
}
connection := r.provider.GetConnection()
defer connection.Close()
rows, err = connection.Query("SELECT DISTINCT SaleChannel 'Name' FROM Employee")
defer rows.Close()
if err != nil {
return result, err
}
for rows.Next() {
entity := new(SaleChannel)
err = sqlstruct.Scan(entity, rows)
if err != nil {
return result, err
}
result = append(result, *entity)
}
err = rows.Err()
if err != nil {
return result, err
}
return result, err
}
As you can see differences are in a few words. I try to find something like Generics from C#, but didnt find.
Can anyone help me?
No, Go does not have generics and won't have them in the forseeable future¹.
You have three options:
Refactor your code so that you have a single function which accepts an SQL statement and another function, and:
Queries the DB with the provided statement.
Iterates over the result's rows.
For each row, calls the provided function whose task is to
scan the row.
In this case, you'll have a single generic "querying" function,
and the differences will be solely in "scanning" functions.
Several variations on this are possible but I suspect you have the idea.
Use the sqlx package which basically is to SQL-driven databases what encoding/json is to JSON data streams: it uses reflection on your types to create and execute SQL to populate them.
This way you'll get reusability on another level: you simply won't write boilerplate code.
Use code generation which is the Go-native way of having "code templates" (that's what generics are about).
This way, you (usually) write a Go program which takes some input (in whatever format you wish), reads it and writes out one or more files which contain Go code, which is then compiled.
In your, very simple, case, you can start with a template of your Go function and some sort of a table which maps SQL statement to the types to create from the data selected.
I'd note that your code indeed looks woefully unidiomatic.
No one in their right mind implements "repository patterns" in Go, but that's sort of okay so long it keeps you happy—we all are indoctrinated to a certain degree with the languages/environments we're accustomed to,—but your connection := r.provider.GetConnection() looks alarming: the Go's database/sql is drastically different from "popular" environments and frameworks so I'd highly recommend to start with this and this.
¹ (Update as of 2021-05-31) Go will have generics as the proposal to implement them has been accepted and the work implementing them is in progress.
Forgive me if I'm misunderstanding, but a better pattern might be something like the following:
type RepositoryItem interface {
Name() string // for example
}
type Repository interface {
GetAll() ([]RepositoryItem, error)
}
At the moment, you essentially have multiple interfaces for each type of repository, so unless you're going to implement multiple types of RoleRepository, you might as well not have the interface.
Having generic Repository and RepositoryItem interfaces might make your code more extensible (not to mention easier to test) in the long run.
A contrived example might be (if we assume a Repository vaguely correlates to a backend) implementations such as MySQLRepository and MongoDBRepository. By abstracting the functionality of the repository, you're protecting against future mutations.
I would very much advise seeing #kostix's answer also, though.
interface{} is the "generic type" in Go. I can imagine doing something like this:
package main
import "fmt"
type IRole struct {
RoleId uint
}
type ISaleChannel struct {
Profitable bool
}
type GenericRepo interface{
GetAll([]interface{})
}
// conceptual repo to store all Roles and SaleChannels
type Repo struct {
IRoles []IRole
ISaleChannels []ISaleChannel
}
func (r *Repo) GetAll(ifs []interface{}) {
// database implementation here before type switch
for _, v := range ifs {
switch v := v.(type) {
default:
fmt.Printf("unexpected type %T\n", v)
case IRole:
fmt.Printf("Role %t\n", v)
r.IRoles = append(r.IRoles, v)
case ISaleChannel:
fmt.Printf("SaleChannel %d\n", v)
r.ISaleChannels = append(r.ISaleChannels, v)
}
}
}
func main() {
getter := new(Repo)
// mock slice
data := []interface{}{
IRole{1},
IRole{2},
IRole{3},
ISaleChannel{true},
ISaleChannel{false},
IRole{4},
}
getter.GetAll(data)
fmt.Println("IRoles: ", getter.IRoles)
fmt.Println("ISaleChannels: ", getter.ISales)
}
This way you don't have to end up with two structs and/or interfaces for IRole and ISale
What's the best signature for a function that returns an optional value and a possible error?
For example:
func findColor(name string) (RGB, error) {
...
}
(The empty RGB value is black, a valid color, so you can't use it to infer that no value was found. Assume the error might come from something like a database connection.)
The two options that seem best are a boolean return value:
func findColor(name string) (RGB, bool, error) {
...
}
c, ok, err := findColor(myname)
if !ok {
...
} else if err != nil {
...
}
...
Or a special error value:
var ColorNotFound = errors.New(...)
func findColor(name string) (RGB, error) {
...
}
c, err := findColor(...)
if err == ColorNotFound {
...
} else if err != nil {
...
}
...
(Making special errors seems like a pain.)
What's the most idiomatic approach?
The convention in Go is to return (value, error) and if error != nil then value is (or may be) invalid.
If you have special errors you need to do something with (like io.EOF) then making a specific error is normal practice. So I would say your 3rd example is the most idiomatic, if you want to do something different for ColorNotFound.
var ColorNotFound = errors.New(...)
func findColor(name string) (RGB, error) {
// ...
}
c, err := findColor(...)
if err == ColorNotFound {
// Do something special if ColorNotFound...
} else if err != nil {
// Some other kind of error...
}
You could make findColor return *RGB and then compare it to nil:
c, err := findColor(name)
if err != nil { /* Handle error. */ }
if c == nil { /* Handle no color. */ }
This is unsafe though, since if you try to call methods on a nil pointer, they can cause a panic.
I'd recommend sticking with a special ErrColorNotFound approach.
Is there a more built-in wrapper to make a function that returns (X, error) successfully execute or abort, like regexp.MustCompile?
I'm talking about something like this, but more "built-in".
There is not. The best you'll get is something like this:
func Must(fn func() (interface{}, error)) interface{} {
v, err := fn()
if err != nil {
log.Fatalln(err)
}
return v
}
Then to use it:
Must(func() (interface{}, error) {
return template.ParseGlob(pattern)
}).(*template.Template)
Assuming that template.ParseGlob(pattern) is the call you wanted to wrap.
Go does not have parametric polymorphism, so this kind of code will end up requiring type assertions to restore the original type and so (in my opinion) is more effort than it's worth. The tidiest, idiomatic error handling you'll get for long chains of potential failure is simply to check for an error, and return it. Defer your cleanup handlers:
func MyFunc() (err error) {
a, err := blah1()
if err != nil {
return
}
defer a.Close()
b, err := blah2(a)
if err != nil {
return
}
defer b.Close()
// ad nauseam
}
Long and tedious, but at least it's explicit and easy to follow. Here are two modules I wrote that are crying out for parametric polymorphism that might give you some ideas for dealing without it:
bitbucket.org/anacrolix/dms/futures
bitbucket.org/anacrolix/dms/cache
Since Go 1.18 we can define typed Must instead of interface{}:
func Must[T any](obj T, err error) T {
if err != nil {
panic(err)
}
return obj
}
How to use: https://go.dev/play/p/ajQAjfro0HG
func success() (int, error) {
return 0, nil
}
func fail() (int, error) {
return -1, fmt.Errorf("Failed")
}
func main() {
n1 := Must(success())
fmt.Println(n1)
var n2 int = Must(fail())
fmt.Println(n2)
}
Must fails inside main, when fail() returns non-nil error
You can even define Mustn for more than 1 return parameter, e.g.
func Must2[T1 any, T2 any](obj1 T1, obj2 T2, err error) (T1, T2) {
if err != nil {
panic(err)
}
return obj1, obj2
}
I don't think a built-in mechanism would make sense since you could very well handle a non-nil error in various ways, as does the examples in the template package itself: see "text/template/examplefiles_test.go", illustrating 2 different usage of 'err':
// Here starts the example proper.
// T0.tmpl is the first name matched, so it becomes the starting template,
// the value returned by ParseGlob.
tmpl := template.Must(template.ParseGlob(pattern))
err := tmpl.Execute(os.Stdout, nil)
if err != nil {
log.Fatalf("template execution: %s", err)
}
// Output:
// T0 invokes T1: (T1 invokes T2: (This is T2))
In the particular case of the helper function (*Template) Must(), transforming an error into an exception (panic) isn't always the right course for all go programs (as debated in this thread), and to cover all the possible way to handle an error would mean to create a lot of "built-in" mechanisms.
I have encountered the same problem myself and decided to develop the following solution: https://github.com/boramalper/must
Example:
database := must.MV(sql.Open("sqlite3", "...")).(*sql.DB)
defer must.M(database.Close())
must.M(database.Ping())
// Use MustValVoid (or MVV shortly) if you don't care about
// the return value.
must.MVV(database.Exec(`
PRAGMA foreign_key_check;
...
`))
I am not sure why all the answers here are using the log package, when the
source itself uses panic:
func MustCompile(str string) *Regexp {
regexp, err := Compile(str)
if err != nil {
panic(`regexp: Compile(` + quote(str) + `): ` + err.Error())
}
return regexp
}
My recommendation would be instead of a generic Must wrapper, just implement
Must variants as needed in your code.
https://github.com/golang/go/blob/go1.16.5/src/regexp/regexp.go#L305-L314