Is there a more built-in wrapper to make a function that returns (X, error) successfully execute or abort, like regexp.MustCompile?
I'm talking about something like this, but more "built-in".
There is not. The best you'll get is something like this:
func Must(fn func() (interface{}, error)) interface{} {
v, err := fn()
if err != nil {
log.Fatalln(err)
}
return v
}
Then to use it:
Must(func() (interface{}, error) {
return template.ParseGlob(pattern)
}).(*template.Template)
Assuming that template.ParseGlob(pattern) is the call you wanted to wrap.
Go does not have parametric polymorphism, so this kind of code will end up requiring type assertions to restore the original type and so (in my opinion) is more effort than it's worth. The tidiest, idiomatic error handling you'll get for long chains of potential failure is simply to check for an error, and return it. Defer your cleanup handlers:
func MyFunc() (err error) {
a, err := blah1()
if err != nil {
return
}
defer a.Close()
b, err := blah2(a)
if err != nil {
return
}
defer b.Close()
// ad nauseam
}
Long and tedious, but at least it's explicit and easy to follow. Here are two modules I wrote that are crying out for parametric polymorphism that might give you some ideas for dealing without it:
bitbucket.org/anacrolix/dms/futures
bitbucket.org/anacrolix/dms/cache
Since Go 1.18 we can define typed Must instead of interface{}:
func Must[T any](obj T, err error) T {
if err != nil {
panic(err)
}
return obj
}
How to use: https://go.dev/play/p/ajQAjfro0HG
func success() (int, error) {
return 0, nil
}
func fail() (int, error) {
return -1, fmt.Errorf("Failed")
}
func main() {
n1 := Must(success())
fmt.Println(n1)
var n2 int = Must(fail())
fmt.Println(n2)
}
Must fails inside main, when fail() returns non-nil error
You can even define Mustn for more than 1 return parameter, e.g.
func Must2[T1 any, T2 any](obj1 T1, obj2 T2, err error) (T1, T2) {
if err != nil {
panic(err)
}
return obj1, obj2
}
I don't think a built-in mechanism would make sense since you could very well handle a non-nil error in various ways, as does the examples in the template package itself: see "text/template/examplefiles_test.go", illustrating 2 different usage of 'err':
// Here starts the example proper.
// T0.tmpl is the first name matched, so it becomes the starting template,
// the value returned by ParseGlob.
tmpl := template.Must(template.ParseGlob(pattern))
err := tmpl.Execute(os.Stdout, nil)
if err != nil {
log.Fatalf("template execution: %s", err)
}
// Output:
// T0 invokes T1: (T1 invokes T2: (This is T2))
In the particular case of the helper function (*Template) Must(), transforming an error into an exception (panic) isn't always the right course for all go programs (as debated in this thread), and to cover all the possible way to handle an error would mean to create a lot of "built-in" mechanisms.
I have encountered the same problem myself and decided to develop the following solution: https://github.com/boramalper/must
Example:
database := must.MV(sql.Open("sqlite3", "...")).(*sql.DB)
defer must.M(database.Close())
must.M(database.Ping())
// Use MustValVoid (or MVV shortly) if you don't care about
// the return value.
must.MVV(database.Exec(`
PRAGMA foreign_key_check;
...
`))
I am not sure why all the answers here are using the log package, when the
source itself uses panic:
func MustCompile(str string) *Regexp {
regexp, err := Compile(str)
if err != nil {
panic(`regexp: Compile(` + quote(str) + `): ` + err.Error())
}
return regexp
}
My recommendation would be instead of a generic Must wrapper, just implement
Must variants as needed in your code.
https://github.com/golang/go/blob/go1.16.5/src/regexp/regexp.go#L305-L314
Related
I am working on a ETL type application and a large majority of the error handling I do is just retrying API requests until they succeed (they randomly fail on occasion due to connection etc.).
Therefore I have noticed a lot of code duplication that looks a lot like
for err != nil {
a,b,err = myfunc(c, d, e)
}
return a, b
So basically just keep doing the function until the error goes away (I put a sleep and other error checking as necessary as well to avoid rate limiting).
What I would like to do is simplify this to just one function that takes an arbitrary function, finds the error type in its output (if it has one) and runs it recursively until err!=nil. My problem is that although go seems to let you use any (interface{}) as an input it is not variadic on function definitions e.g.
(type func(a int) (int, int, error)) as the type func(...any) []any
I am wondering if this is impossible to do in go and if so any suggestions to get around it/get similar functionality more idiomatically.
Trying to test with something like this but the compiler does not like it.
func main() {
Deal(SometimesFail, 10)
}
func Deal(f func(...any) []any, inputs ...any) []any {
outputs := f(inputs)
for _, val := range outputs {
err, ok := val.(error)
if ok {
for err != nil {
outputs = Deal(f, inputs...)
}
return outputs
}
continue
}
return outputs
}
func SometimesFail(a int) (int, int, error) {
random := rand.Intn(a)
if random%2 == 0 {
return random, random, nil
} else {
return random, random, errors.New("error")
}
}
I guess what I could do to get around this is create a type for each function out/input scheme and allow the generic function to take any of these. This would keep the code duplication at a minimum while still achieving the goal.
The following: func(any), func(any, any), func(...any) are all different types and you can't assign one type to another. There is no single function type that would include all of them.
One way to work around this is to decouple function invocation (which must know the exact type of the function) from the retrial logic:
type result struct {
vals []any
err error
}
func main() {
vals := repeatUntilSuccess(func(inputs ...any) result {
val, err := failingRandomly(10)
return result{[]any{val}, err}
})
fmt.Println(vals)
}
func repeatUntilSuccess(fn func(...any) result) []any {
res := fn()
for res.err != nil {
res = fn()
}
return res.vals
}
func failingRandomly(i int) (int, error) {
random := rand.Intn(i)
if random%2 == 0 {
return random, nil
} else {
return random, errors.New("bad luck")
}
}
I try to implement repository pattern in Go app (simple web service) and try to find better way to escape code duplication.
Here is a code
Interfaces are:
type IRoleRepository interface {
GetAll() ([]Role, error)
}
type ISaleChannelRepository interface {
GetAll() ([]SaleChannel, error)
}
And implementation:
func (r *RoleRepository) GetAll() ([]Role, error) {
var result []Role
var err error
var rows *sql.Rows
if err != nil {
return result, err
}
connection := r.provider.GetConnection()
defer connection.Close()
rows, err = connection.Query("SELECT Id,Name FROM Position")
defer rows.Close()
if err != nil {
return result, err
}
for rows.Next() {
entity := new(Role)
err = sqlstruct.Scan(entity, rows)
if err != nil {
return result, err
}
result = append(result, *entity)
}
err = rows.Err()
if err != nil {
return result, err
}
return result, err
}
func (r *SaleChannelRepository) GetAll() ([]SaleChannel, error) {
var result []SaleChannel
var err error
var rows *sql.Rows
if err != nil {
return result, err
}
connection := r.provider.GetConnection()
defer connection.Close()
rows, err = connection.Query("SELECT DISTINCT SaleChannel 'Name' FROM Employee")
defer rows.Close()
if err != nil {
return result, err
}
for rows.Next() {
entity := new(SaleChannel)
err = sqlstruct.Scan(entity, rows)
if err != nil {
return result, err
}
result = append(result, *entity)
}
err = rows.Err()
if err != nil {
return result, err
}
return result, err
}
As you can see differences are in a few words. I try to find something like Generics from C#, but didnt find.
Can anyone help me?
No, Go does not have generics and won't have them in the forseeable future¹.
You have three options:
Refactor your code so that you have a single function which accepts an SQL statement and another function, and:
Queries the DB with the provided statement.
Iterates over the result's rows.
For each row, calls the provided function whose task is to
scan the row.
In this case, you'll have a single generic "querying" function,
and the differences will be solely in "scanning" functions.
Several variations on this are possible but I suspect you have the idea.
Use the sqlx package which basically is to SQL-driven databases what encoding/json is to JSON data streams: it uses reflection on your types to create and execute SQL to populate them.
This way you'll get reusability on another level: you simply won't write boilerplate code.
Use code generation which is the Go-native way of having "code templates" (that's what generics are about).
This way, you (usually) write a Go program which takes some input (in whatever format you wish), reads it and writes out one or more files which contain Go code, which is then compiled.
In your, very simple, case, you can start with a template of your Go function and some sort of a table which maps SQL statement to the types to create from the data selected.
I'd note that your code indeed looks woefully unidiomatic.
No one in their right mind implements "repository patterns" in Go, but that's sort of okay so long it keeps you happy—we all are indoctrinated to a certain degree with the languages/environments we're accustomed to,—but your connection := r.provider.GetConnection() looks alarming: the Go's database/sql is drastically different from "popular" environments and frameworks so I'd highly recommend to start with this and this.
¹ (Update as of 2021-05-31) Go will have generics as the proposal to implement them has been accepted and the work implementing them is in progress.
Forgive me if I'm misunderstanding, but a better pattern might be something like the following:
type RepositoryItem interface {
Name() string // for example
}
type Repository interface {
GetAll() ([]RepositoryItem, error)
}
At the moment, you essentially have multiple interfaces for each type of repository, so unless you're going to implement multiple types of RoleRepository, you might as well not have the interface.
Having generic Repository and RepositoryItem interfaces might make your code more extensible (not to mention easier to test) in the long run.
A contrived example might be (if we assume a Repository vaguely correlates to a backend) implementations such as MySQLRepository and MongoDBRepository. By abstracting the functionality of the repository, you're protecting against future mutations.
I would very much advise seeing #kostix's answer also, though.
interface{} is the "generic type" in Go. I can imagine doing something like this:
package main
import "fmt"
type IRole struct {
RoleId uint
}
type ISaleChannel struct {
Profitable bool
}
type GenericRepo interface{
GetAll([]interface{})
}
// conceptual repo to store all Roles and SaleChannels
type Repo struct {
IRoles []IRole
ISaleChannels []ISaleChannel
}
func (r *Repo) GetAll(ifs []interface{}) {
// database implementation here before type switch
for _, v := range ifs {
switch v := v.(type) {
default:
fmt.Printf("unexpected type %T\n", v)
case IRole:
fmt.Printf("Role %t\n", v)
r.IRoles = append(r.IRoles, v)
case ISaleChannel:
fmt.Printf("SaleChannel %d\n", v)
r.ISaleChannels = append(r.ISaleChannels, v)
}
}
}
func main() {
getter := new(Repo)
// mock slice
data := []interface{}{
IRole{1},
IRole{2},
IRole{3},
ISaleChannel{true},
ISaleChannel{false},
IRole{4},
}
getter.GetAll(data)
fmt.Println("IRoles: ", getter.IRoles)
fmt.Println("ISaleChannels: ", getter.ISales)
}
This way you don't have to end up with two structs and/or interfaces for IRole and ISale
What's the best signature for a function that returns an optional value and a possible error?
For example:
func findColor(name string) (RGB, error) {
...
}
(The empty RGB value is black, a valid color, so you can't use it to infer that no value was found. Assume the error might come from something like a database connection.)
The two options that seem best are a boolean return value:
func findColor(name string) (RGB, bool, error) {
...
}
c, ok, err := findColor(myname)
if !ok {
...
} else if err != nil {
...
}
...
Or a special error value:
var ColorNotFound = errors.New(...)
func findColor(name string) (RGB, error) {
...
}
c, err := findColor(...)
if err == ColorNotFound {
...
} else if err != nil {
...
}
...
(Making special errors seems like a pain.)
What's the most idiomatic approach?
The convention in Go is to return (value, error) and if error != nil then value is (or may be) invalid.
If you have special errors you need to do something with (like io.EOF) then making a specific error is normal practice. So I would say your 3rd example is the most idiomatic, if you want to do something different for ColorNotFound.
var ColorNotFound = errors.New(...)
func findColor(name string) (RGB, error) {
// ...
}
c, err := findColor(...)
if err == ColorNotFound {
// Do something special if ColorNotFound...
} else if err != nil {
// Some other kind of error...
}
You could make findColor return *RGB and then compare it to nil:
c, err := findColor(name)
if err != nil { /* Handle error. */ }
if c == nil { /* Handle no color. */ }
This is unsafe though, since if you try to call methods on a nil pointer, they can cause a panic.
I'd recommend sticking with a special ErrColorNotFound approach.
I have two functions as shown below which look similar, but using different functions to query the db. Since Go doesn't encourage overloaading methods, is the redundancy acceptable? Or should I refactor them into one function? All comments are welcomed.
var getCustomers = func() ([]customer, error) {
return nil, nil
}
var getCustomerById = func(int64) (*customer, error) {
return nil, nil
}
func listCustomer(w http.ResponseWriter, r *http.Request) *appError {
cus, err := getCustomers()
if err != nil {
return &appError{err, "No customer found", 404}
}
res, err := json.Marshal(cus)
if err != nil {
return &appError{err, "Can't display record", 500}
}
fmt.Fprint(w, string(res))
return nil
}
func viewCustomer(w http.ResponseWriter, r *http.Request, id int64) *appError {
cus, err := getCustomerByID(id)
if err != nil {
return &appError{err, "No customer found", 404}
}
res, err := json.Marshal(cus)
if err != nil {
return &appError{err, "Can't display record", 500}
}
fmt.Fprint(w, string(res))
return nil
}
Suggestion to use -1 to list all customer, but I'm not sure if this is the best it can be:
func viewCustomer(w http.ResponseWriter, r *http.Request, id int64) *appError {
var c *customer
var clist []customer
var err error
if id < 0 {
clist, err = getCustomers()
res, err := json.Marshal(clist)
if err != nil {
return &appError{err, "Can't display record", 500}
}
fmt.Fprint(w, string(res))
return nil
} else {
c, err = getCustomerById(id)
res, err := json.Marshal(c)
if err != nil {
return &appError{err, "Can't display record", 500}
}
fmt.Fprint(w, string(res))
return nil
}
}
Since Go doesn't encourage overloaading methods, is the redundancy acceptable? Or should I refactor them into one function?
The answer to your question depends heavily on the real code and your task. If listing one customer is very different from listing several customers (that is, you need different information and have different presentation logic), I'd say that duplication here is not that bad, since the difference may grow larger in the future, so a DRY solution could turn into an if and switch mess quickly. (I've had a similar experience on a non-Go project and since then I think that DRY is good but you should not be fanatic about it.)
On the other hand, if you're making, say a JSON API, you could make it more DRY. Define your getCustomers function like this:
func customers(ids ...int64) ([]customer, error) { /* ... */ }
This way you can:
call it like customers() and get all customers;
call customers(42) and get a customer whose ID is 42;
call customers(someIDs...) and get multiple customers by their IDs.
All can be done in one handler in a straightforward way.
All comments are welcomed.
Two nitpicks on your code:
Getter methods in Go are idiomatically named foo and not getFoo, so I used customers in my example. You would most probably call it as DB.Customers(ids...), so it looks a lot like a getter method.
What's up with var foo = func() notation? Unless there is a reason for that (like using these functions as closures), I'd suggest sticking with the func foo() notation, since it's more idiomatic and generally easier to grep. EDIT: as OP pointed out in the comments, another case for var foo = func() notation is of course functions that can be faked in testing, as shown in Andrew Gerrand's Testing Techniques talk.
I hope that helps.
In order to determine whether a given type implements an interface using the reflect package, you need to pass a reflect.Type to reflect.Type.Implements(). How do you get one of those types?
As an example, trying to get the type of an uninitialized error (interface) type does not work (it panics when you to call Kind() on it)
var err error
fmt.Printf("%#v\n", reflect.TypeOf(err).Kind())
Do it like this:
var err error
t := reflect.TypeOf(&err).Elem()
Or in one line:
t := reflect.TypeOf((*error)(nil)).Elem()
Even Shaws response is correct, but brief. Some more details from the reflect.TypeOf method documentation:
// As interface types are only used for static typing, a common idiom to find
// the reflection Type for an interface type Foo is to use a *Foo value.
writerType := reflect.TypeOf((*io.Writer)(nil)).Elem()
fileType := reflect.TypeOf((*os.File)(nil)).Elem()
fmt.Println(fileType.Implements(writerType))
For googlers out there I just ran into the dreaded scannable dest type interface {} with >1 columns (XX) in result error.
Evan Shaw's answer did not work for me. Here is how I solved it. I am also using the lann/squirrel library, but you could easily take that out.
The solution really isn't that complicated, just knowing the magic combination of reflect calls to make.
The me.GetSqlx() function just returns an instance to *sqlx.DB
func (me *CommonRepo) Get(query sq.SelectBuilder, dest interface{}) error {
sqlst, args, err := query.ToSql()
if err != nil {
return err
}
// Do some reflection magic so that Sqlx doesn't hork on interface{}
v := reflect.ValueOf(dest)
return me.GetSqlx().Get(v.Interface(), sqlst, args...)
}
func (me *CommonRepo) Select(query sq.SelectBuilder, dest interface{}) error {
sqlst, args, err := query.ToSql()
if err != nil {
return err
}
// Do some reflection magic so that Sqlx doesn't hork on interface{}
v := reflect.ValueOf(dest)
return me.GetSqlx().Select(v.Interface(), sqlst, args...)
}
Then to invoke it you can do:
func (me *myCustomerRepo) Get(query sq.SelectBuilder) (rec Customer, err error) {
err = me.CommonRepo.Get(query, &rec)
return
}
func (me *myCustomerRepo) Select(query sq.SelectBuilder) (recs []Customer, err error) {
err = me.CommonRepo.Select(query, &recs)
return
}
This allows you to have strong types all over but have all the common logic in one place (CommonRepo in this example).