How can I safely terminate a request outside of a dispatcher (handlefunc) without an explicit return in handlefunc? In the code below, I end up seeing both "Bad request" and "Not found" in my response, but I'd like f() to abort the request -- so I wouldn't have to clutter dispatcher with error checks and returns.
package main
import (
"net/http"
)
func f(w http.ResponseWriter, r *http.Request, k string) string{
if v, ok := r.Form[k]; !ok{
http.Error(w, "Bad request", http.StatusBadRequest)
return ""
}else{
return v[0]
}
}
func main(){
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request){
foo := f(w, r, "foo") //make sure foo is passed, if not abort with "bad request" without requiring an explicit return here
bar := f(w, r, "bar") //make sure bar is passed, if not abort with "bad request" without requiring an explicit return here
//lookup in db...
//abort with a 404 if not found in db
http.NotFound(w, r)
})
http.ListenAndServe(":6000", nil)
}
The above was an attempt to refactor the following:
func main(){
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request){
if foo,ok:=r.Form["foo"];!ok{
http.Error(w, "Bad request", http.StatusBadRequest)
return
}
if bar,ok:=r.Form["bar"];!ok{
http.Error(w, "Bad request", http.StatusBadRequest)
return
}
//lookup in db...
//abort with a 404 if not found in db
http.NotFound(w, r)
})
http.ListenAndServe(":6000", nil)
}
I don't think trying to abort from within a function is what you want to do here. Instead, make the validation less repetitive. You could write something like package validate with func Require(form url.Values, required ...string) error, and, in your handler, if err := validate.Require(r.Form, "foo", "bar"); err != nil { ... }.
Or you could do a more general validator: maybe take a map of field name to validation type (number, string, etc.) and return another map that's nil if no errors and {"fieldName": error} otherwise.
Redditors talked some about this and one wrote a library that uses struct tags to validate. There's another struct-based validation implementation in a new form rendering/validation toolkit. (I've tried neither.) The Redditors raised the tricky question of how much you can abstract validation before getting a "framework" that gets in the way when your app gets more complicated, and I don't have a simple answer.
There are cases where something really unexpected happens and the server can't do anything better than give the user an opaque "500 Internal Server Error" response, but the problem manifests deep below your HandlerFunc. The canonical example is a programming error like a nil pointer dereference. The advice from the Go team is to use panic when "the error is unrecoverable." (A tool like negroni.Recovery can help log panics, etc.)
Unnecessary panics are lame because:
panics make it easier to forget necessary error handling entirely, because it's implied that things can panic but explicit they can err
panic/recover error handling code is ugly and hard to maintain
panics introduce inconsistency with the standard err returns
panics use stack unwinding, which is much slower than normal control flow
I did a quick grep and essentially all calls to panic in the standard library and other official repositories are there to catch programming/internal errors, not crash on surprise external conditions like bad input or I/O errors. Things like file/object not found or invalid user input can usually be handled more gracefully than by crashing the request (it's at least possible to return a specific error), and they aren't really unexpected (it's the Internet; you'll get invalid input and requests for things that don't exist). So panic is not for normal request aborts--it's for crashing when what "can't happen" has happened and the request can't continue.
Again, here, I'd handle invalid input with a standard if block in the handler function, but rearrange your validation so that it doesn't require a lot of code per required argument.
Related
Everywhere I look, the "way" to "wrap" errors in Go is to use fmt.Errof with the %w verb
https://go.dev/blog/go1.13-errors
However, fmt.Errorf does not recursively wrap errors. There is no way to use it to wrap three previously defined errors (Err1, Err2, and Err3) and then check the result by using Is() and get true for each those three errors.
FINAL EDIT:
Thanks to #mkopriva's answer and comments below it, I now have a straightforward way to implement this (although, I am still curious if there is some standard type which does this). In the absence of an example, my attempts at creating one failed. The piece I was missing was adding an Is and As method to my type. Because the custom type needs to contain an error and a pointer to the next error, the custom Is and As methods allows us to compare the error contained in the custom type, rather than the custom type itself.
Here is a working example: https://go.dev/play/p/6BYGgIb728k
Highlights from the above link
type errorChain struct {
err error
next *errorChain
}
//These two functions were the missing ingredient
//Defined this way allows for full functionality even if
//The wrapped errors are also chains or other custom types
func (c errorChain) Is(err error) bool { return errors.Is(c.err, err) }
func (c errorChain) As(target any) bool { return errors.As(c.err, target) }
//Omitting Error and Unwrap methods for brevity
func Wrap(errs ...error) error {
out := errorChain{err: errs[0]}
n := &out
for _, err := range errs[1:] {
n.next = &errorChain{err: err}
n = n.next
}
return out
}
var Err0 = errors.New("error 0")
var Err1 = errors.New("error 1")
var Err2 = errors.New("error 2")
var Err3 = errors.New("error 3")
func main() {
//Check basic Is functionality
errs := Wrap(Err1, Err2, Err3)
fmt.Println(errs) //error 1: error 2: error 3
fmt.Println(errors.Is(errs, Err0)) //false
fmt.Println(errors.Is(errs, Err2)) //true
}
While the Go source specifically mentions the ability to define an Is method, the example does not implement it in a way that can solve my issue and the discussion do not make it immediately clear that it would be needed to utilize the recursive nature of errors.Is.
AND NOW BACK TO THE ORIGINAL POST:
Is there something built into Go where this does work?
I played around with making one of my own (several attempts), but ran into undesirable issues. These issues stem from the fact that errors in Go appear to be compared by address. i.e. if Err1 and Err2 point to the same thing, they are the same.
This causes me issues. I can naively get errors.Is and errors.As to work recursively with a custom error type. It is straightforward.
Make a type that implements the error interface (has an Error() string method)
The type must have a member that represents the wrapped error which is a pointer to its own type.
Implement an Unwrap() error method that returns the wrapped error.
Implement some method which wraps one error with another
It seems good. But there is trouble.
Since errors are pointers, if I make something like myWrappedError = Wrap(Err1, Err2) (in this case assume Err1 is being wrapped by Err2). Not only will errors.Is(myWrappedError, Err1) and errors.Is(myWrappedError, Err2) return true, but so will errors.Is(Err2, Err1)
Should the need arise to make myOtherWrappedError = Wrap(Err3, Err2) and later call errors.Is(myWrappedError, Err1) it will now return false! Making myOtherWrappedError changes myWrappedError.
I tried several approaches, but always ran into related issues.
Is this possible? Is there a Go library which does this?
NOTE: I am more interested in the presumably already existing right way to do this rather than the specific thing that is wrong with my basic attempt
Edit 3: As suggested by one of the answers, the issue in my first code is obviously that I modify global errors. I am aware, but failed to adequately communicate. Below, I will include other broken code which uses no pointers and modifies no globals.
Edit 4: slight modification to make it work more, but it is still broken
See https://go.dev/play/p/bSytCysbujX
type errorGroup struct {
err error
wrappedErr error
}
//...implemention Unwrap and Error excluded for brevity
func Wrap(inside error, outside error) error {
return &errorGroup{outside, inside}
}
var Err1 = errorGroup{errors.New("error 1"), nil}
var Err2 = errorGroup{errors.New("error 2"), nil}
var Err3 = errorGroup{errors.New("error 3"), nil}
func main() {
errs := Wrap(Err1, Err2)
errs = Wrap(errs, Err3)
fmt.Println(errs)//error 3: error 2: error 1
fmt.Println(errors.Is(errs, Err1)) //true
fmt.Println(errors.Is(errs, Err2)) //false <--- a bigger problem
fmt.Println(errors.Is(errs, Err3)) //false <--- a bigger problem
}
Edit 2: playground version shortened
See https://go.dev/play/p/swFPajbMcXA for an example of this.
EDIT 1: A trimmed version of my code focusing on the important parts:
type errorGroup struct {
err error
wrappedErr *errorGroup
}
//...implemention Unwrap and Error excluded for brevity
func Wrap(errs ...*errorGroup) (r *errorGroup) {
r = &errorGroup{}
for _, err := range errs {
err.wrappedErr = r
r = err
}
return
}
var Err0 = &errorGroup{errors.New("error 0"), nil}
var Err1 = &errorGroup{errors.New("error 1"), nil}
var Err2 = &errorGroup{errors.New("error 2"), nil}
var Err3 = &errorGroup{errors.New("error 3"), nil}
func main() {
errs := Wrap(Err1, Err2, Err3)//error 3: error 2: error 1
fmt.Println(errors.Is(errs, Err1)) //true
//Creating another wrapped error using the Err1, Err2, or Err3 breaks the previous wrap, errs.
_ = Wrap(Err0, Err2, Err3)
fmt.Println(errors.Is(errs, Err1)) //false <--- the problem
}
You can use something like this:
type errorChain struct {
err error
next *errorChain
}
func Wrap(errs ...error) error {
out := errorChain{err: errs[0]}
n := &out
for _, err := range errs[1:] {
n.next = &errorChain{err: err}
n = n.next
}
return out
}
func (c errorChain) Is(err error) bool {
return c.err == err
}
func (c errorChain) Unwrap() error {
if c.next != nil {
return c.next
}
return nil
}
https://go.dev/play/p/6oUGefSxhvF
Your code modifies package-global error values, so it is inherently broken. This defect has nothing to do with Go's error handling mechanics.
Per the documentation you linked, there are two error-handling helpers: Is, and As. Is lets you recursively unwrap an error, looking for a specific error value, which is necessarily a package global for this to be useful. As, on the other hand, lets you recursively unwrap an error looking for any wrapped error value of a given type.
How does wrapping work? You wrap error A in a new error value B. A Wrap() helper would necessarily return a new value, as fmt.Errorf does in the examples in the linked documentation. A Wrap helper should never modify the value of the error being wrapped. That value should be considered immutable. In fact, in any normal implementation, the value would be of type error, so that you can wrap any error, rather than just wrapping concentric values of your custom error type in each other; and, in that case, you have no access to the fields of the wrapped error to modify them anyway. Essentially, Wrap should be roughly:
func Wrap(err error) error {
return &errGroup{err}
}
And that's it. That's not very useful, because your implementation of errGroup doesn't really do anything - it provides no details about the error that occurred, it's just a container for other errors. For it to have value, it should have a string error message, or methods like some other error types' IsNotFound, or something that makes it more useful than just using error and fmt.Errorf.
Based on the usage in your example code, it also looks like you're presuming the use case is to say "I want to wrap A in B in C", which I've never seen in the wild and I cannot think of any scenario where that would be needed. The purpose of wrapping is to say "I've recieved error A, I'm going to wrap it in error B to add context, and return it". The caller might wrap that error in error C, and so on, which is what makes recursive wrapping valuable.
For example: https://go.dev/play/p/XeoONx19dgX
Instead of chaining/wrapping, you will "soon" (Go 1.20, as seen in Go 1.20-rc1 in Dec. 2022) be able to return a slice/tree of errors.
(In the meantime, mdobak/go-xerrors is a good alternative)
The release note explains:
Wrapping multiple errors
Go 1.20 expands support for error wrapping to permit an error to wrap
multiple other errors.
An error e can wrap more than one error by providing an Unwrap method
that returns a []error.
The errors.Is and errors.As functions have been updated to inspect
multiply wrapped errors.
The fmt.Errorf function now supports multiple occurrences of the %w
format verb, which will cause it to return an error that wraps all of
those error operands.
The new function errors.Join returns an error wrapping a list of
errors.
That comes from:
proposal: errors: add support for wrapping multiple errors
Background
Since Go 1.13, an error may wrap another by providing an Unwrap method returning the wrapped error.
The errors.Is and errors.As functions operate on chains of wrapped errors.
A common request is for a way to combine a list of errors into a single error.
Proposal
An error wraps multiple errors if its type has the method
Unwrap() []error
Reusing the name Unwrap avoids ambiguity with the existing singular Unwrap method.
Returning a 0-length list from Unwrap means the error doesn't wrap anything.
Callers must not modify the list returned by Unwrap.
The list returned by Unwrap must not contain any nil errors.
We replace the term "error chain" with "error tree".
The errors.Is and errors.As functions are updated to unwrap multiple errors.
Is reports a match if any error in the tree matches.
As finds the first matching error in a inorder preorder traversal of the tree.
The errors.Join function provides a simple implementation of a multierr.
It does not flatten errors.
// Join returns an error that wraps the given errors.
// Any nil error values are discarded.
// The error formats as the text of the given errors, separated by newlines.
// Join returns nil if errs contains no non-nil values.
func Join(errs ...error) error
The fmt.Errorf function permits multiple instances of the %w formatting verb.
The errors.Unwrap function is unaffected: It returns nil when called on an error with an Unwrap() []error method.
Why should this be in the standard library?
This proposal adds something which cannot be provided outside the standard library: Direct support for error trees in errors.Is and errors.As.
Existing combining errors operate by providing Is and As methods which inspect the contained errors, requiring each implementation to duplicate this logic, possibly in incompatible ways.
This is best handled in errors.Is and errors.As, for the same reason those functions handle singular unwrapping.
In addition, this proposal provides a common method for the ecosystem to use to represent combined errors, permitting interoperation between third-party implementations.
So far (Sept. 2022) this proposal seems a likely accept has been accepted!
CL 432575 starts the implementation.
There arr several approaches but there is one thing that you should keep in mind: if you have multiple errors, you may need to handle it as a slice of errors
For instance, imagine you need to check if all errors are the same, or there is at least one error of certain type you can use the snippet below.
You can extend this concept or use some existing library to handle multierrors
type Errors []error
func (errs Errors) String() string {
…
}
func (errs Errors) Any(target error) bool{
for _, err := range errs {
if errors.Is(err,target) {
return true
}
}
return false
}
func (errs Errors) All(target error) bool{
if len(errs) == 0 { return false }
for _, err := range errs {
if !errors.Is(err,target) {
return false
}
}
return true
}
Currently my web app is just a router and handlers.
What are some important things I am missing to make this production worthy?
I believe I have to set the # of procs to ensure this uses maximum goroutines?
Should I be using output buffering?
Anything else you see missing that is best-practise?
var (
templates = template.Must(template.ParseFiles("templates/home.html")
)
func main() {
r := mux.NewRouter()
r.HandleFunc("/", WelcomeHandler)
http.ListenAndServe(":9000", r)
}
func WelcomeHandler(w http.ResponseWriter, r *http.Request) {
homePage, err := api.LoadHomePage()
if err != nil {
}
tmpl := "home"
renderTemplate(w, tmpl, homePage)
}
func renderTemplate(w http.ResponseWriter, tmpl string, hp *HomePage) {
err := templates.ExecuteTemplate(w, tmpl+".html", hp)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
You don't need to set/change runtime.GOMAXPROCS() as since Go 1.5 it defaults to the number of available CPU cores.
Buffering output? From the performance point of view, you don't need to. But there may be other considerations for which you may.
For example, your renderTemplate() function may potentially panic. If executing the template starts writing to the output, it involves setting the HTTP response code and other headers prior to writing data. And if a template execution error occurs after that, it will return an error, and so your code attempts to send back an error response. At this point HTTP headers are already written, and this http.Error() function will try to set headers again => panic.
One way to avoid this is to first render the template into a buffer (e.g. bytes.Buffer), and if no error is returned by the template execution, then you can write the content of the buffer to the response writer. If error occurs, then of course you won't write the content of the buffer, but send back an error response just like you did.
To sum it up, your code is production ready performance-wise (excluding the way you handle template execution errors).
WelcomeHandler should return when err != nil is true.
Log the error when one is hit to help investigation.
Place templates = template.Must(template.ParseFiles("templates/home.html") in the init. Split it into separate lines. If template.ParseFiles returns an then error make a Fatal log. And if you have multiple templates to initialize then initialize them in goroutines with a common WaitGroup to speed up the startup.
Since you are using mux, HTTP Server is too clean with its URLs might also be good to know.
You might also want to reconsider the decision of letting the user's know why they got the http.StatusInternalServerError response.
Setting the GOMAXPROCS > 1 if you have more the one core would definitely be a good idea but I would keep it less than number of cores available.
I'm currently using gorm and gin framework. I wonder what would happen if an error happened and my app didn't handle it?
Example:
if err := db.Where("name = ?", "jinzhu").First(&user).Error; err != nil {
// error handling...
}
In the above example, the error is being handled.
if db.Model(&user).Related(&credit_card).RecordNotFound() {
// no credit card found error handling
}
In the next example above, only the RecordNotFound() error is being handled, but what if it throws a different error? what will happen?
Will my app automatically respond with a 500 server internal error and will the app keep on running properly?
In the next example above, only the RecordNotFound() error is being handled, but what if it throws a different error?
If you won't catch the error it will continue on the code. Error is not a special type it's a simple struct
err := FunctionThatReturnsError()
if err == myError.RecordNotFound() { // if err is not RecordNotFound then it won't enter the if simple as that.
// Do something.
}
// continue code.
Will my app automatically respond with a 500 server internal error and will the app keep on running properly?
There will be no response if the go routine doesn't panic or you return a response. If you want to handle it you can do:
err := FunctionThatReturnsError()
if err == myError.RecordNotFound() {
panic("RecordNotFound")
}
or
err := FunctionThatReturnsError()
if err == myError.RecordNotFound() {
c.JSON(500, "Record not found"}
}
I don't recommend the panic method. If you're curious google why.
Go doesn't have exceptions. Instead of catching exceptions, you get errors via return values from functions. So there's no throwing or anything special going on behind the scenes, just a function that returns an error value, and like any other return value - you can discard it.
I wouldn't recommend discarding errors though. If you feel lazy or lost about what to do with an error - just log it:
log.Error(err)
You never know if an error you discarded is causing this mysterious bug you can swear is coming from anywhere but your own code.
I wonder what would happen if an error happened and my app didn't handle it?
Then the state of the app is undefined. If you don't check error values your app will be using values which are undefined (probably nil for pointers and "zeros" for values) or assuming that side effect occurred but it might not.
Let's say you have a function with signature func CreateStruct() (T, err)
and call it like that t, _ := CreateStruct() (not checking for error) you should not expect t variable to have a proper value set.
If you have function like func Update() err and you call it without error checking then you can't know whether update was performed or not.
Of course everything depends on API and implementation. But you get the idea.
but what if it throws a different error?
It's impossible. There is not throwing error mechanism in Go. Error can only be returned as a normal value.
You should never be lazy with handling errors. It's very important part of programming and Go makes it easier to realize.
I have started learning Go today.
One thing that makes me crazy, it's the err returned parameter.
Let's assume I need to nest few functions. like this:
return string(json.Marshal(MyData))
or more complex example:
return func1(func2(func3(MyData)))
Is it really necessary to write:
tmp1 , _ = func3(MyData)
tmp2 , _ = func2(tmp1)
tmp3 , _ = func1(tmp2)
return tmp3
That's annoying!
Is there any way to make the code looks cleaner?
It is possible to define a function to ignore errors, but Go's lack of generics make it so you'd have to use interface{} and typecasts all over the place, losing a lot of static guarantees from the typechecker in the process. It is extremely ugly. Don't do this.
func ignoreError(val interface {}, err error) interface {} {
return val
}
At every call to ignoreError() you would have to make a type cast to the expected return type.
Playground example
One possible abstraction pattern you will often see is to use a generic error handler.
This doesn't prevent you from having to deal with error values, but it does abstract the handling of errors away from the rest of your code.
Note that abstractions like these are considered "non-idiomatic" Go, the "pure" way is to explicitly handle errors in-place. This panic-driven alternative can still be very useful though, especially for quickly prototyping a script where you just want to dump all the errors in a console or logfile.
For reusable packages, I would stick to the verbose explicit way though, because others will expect error-producing functions to actually return error values, rather than using a panic-recover mechanism.
package main
import (
utils
)
func main() {
defer func() {
utils.Handle(func(err error) {
// Handle errors in a generic way,
// for example using println, or writing to http
})
}()
var result, err := someFragileFunction()
Check(err)
}
package utils
func Check(err error) {
if err != nil {
panic(err)
}
}
func Handle(handler func(err error)) {
if r := recover(); r != nil {
if err, ok := r.(error); ok {
handler(err)
} else {
panic(r)
}
}
}
The real answer is: Don't.
Never just ignore the errors.
Seriously. The errors are there for a reason. If a function returns an error,
it almost always means that it's possible, during the operation of your program,
even if it's 100% bug-free, for the function to fail. And if it does,
you don't usually want to just keep going as if nothing happened.
If you're absolutely sure that you're using a function in a way that ensures that it will never return a non-nil error (unless there's a bug in your program, and there always is), you might want to write a Must-style function like in the template package which panics with the returned error value.
Error handling is not noise. It's not clutter. It's not something you want
to get rid of. If it looks like 50% of your program is error
handling, that's because 50% of your program is, and should be, error handling.
I searched around and as far as I can tell, POST form requests are already limited to 10MB (http://golang.org/src/net/http/request.go#L721).
If I were to go about reducing this in my ServeHTTP method, I'm not sure how to properly do it. I would try something like this:
r.Body = http.MaxBytesReader(w, r.Body, MaxFileSize)
err := r.ParseForm()
if err != nil {
//redirect to some error page
return
}
But would returning upon error close the connection as well? How would I prevent having to read everything? I found this: https://stackoverflow.com/a/26393261/2202497, but what if content length is not set and in the middle of reading I realize that the file is too big.
I'm using this as a security measure to prevent someone from hogging my server's resources.
The correct way to limit the size of the request body is to do as you suggested:
r.Body = http.MaxBytesReader(w, r.Body, MaxFileSize)
err := r.ParseForm()
if err != nil {
// redirect or set error status code.
return
}
MaxBytesReader sets a flag on the response when the limit is reached. When this flag is set, the server does not read the remainder of the request body and the server closes the connection on return from the handler.
If you are concerned about malicious clients, then you should also set Server.ReadTimeout, Server.WriteTimeout and possibly Server.MaxHeaderBytes.
If you want to set the request body limit for all of your handlers, then wrap root handler with a handler that sets the limit before delegating to the root handler:
type maxBytesHandler struct {
h http.Handler
n int64
}
func (h *maxBytesHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(w, r.Body, h.n)
h.h.ServeHTTP(w, r)
}
Wrap the root handler when calling ListenAndServe:
log.Fatal(http.ListenAndServe(":8080", &maxBytesHandler{h:mux, n:4096))
or when configuring a server:
s := http.Server{
Addr: ":8080",
Handler: &maxBytesReader{h:mux, n:4096},
}
log.Fatal(s.ListenAndServe())
There's no need for a patch as suggested in another answer. MaxBytesReader is the official way to limit the size of the request body.
Edit: As others cited MaxByteReader is the supported way. It is interesting that the default reader is instead, limitreader after type asserting for max byte reader.
Submit a patch to the Go source code and make it configurable! You are working with an open source project after all. Adding a setter to http.Request and some unit tests for it is probably only 20 minutes worth of work. Having a hardcoded value here is a bit clunky, give back and fix it :).
You can of course implement your own ParseForm(r *http.Request) method if you really need to override this. Go is essentially BSD, so you can copy paste the library ParseForm and change the limit, but thats a bit ugly no?