I have an auth middleware that handle app authentication, with several cases to checks, each check have the same logic in case of an error:
res, err := doSomeCheck()
if err != nil {
log.Println("Authentication failed: %v", err)
json.NewEncoder(w).Encode(struct {Error string}{Error: "something is broke!"})
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusForbidden)
return
}
I want to write this logic once (the only difference between each case is the error and the client message) with some function like this:
func authError(w http.ResponseWriter, err error, clientMsg string) {
log.Println("Authentication failed: %v", err)
json.NewEncoder(w).Encode(struct {
Error string
}{Error: clientMsg})
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusForbidden)
return
}
But w is not a pointer (I don't get it as a pointer to the middleware handler) so I can't change it from the function, authError() doesn't change the actual response.
How can I make this work elegantly?
w is not a pointer, but it's of an interface type, and it wraps a pointer under the hood. So you may pass it as-is, and when you call its methods, it will be reflected at the caller.
Just don't forget that if there's anything written to the response prior, you can't write the header (again). Same, if your authError() writes something to the output, the caller can't take that back. If authError() generates the response, the caller should return in that case.
Also note that you must first set headers, then call ResponseWriter.WriteHeader(), and only then can you write the response body.
If you call ResponseWriter.Write(), that will write the response status if it hasn't been (assuming HTTP 200 OK).
Quoting from ResponseWriter.Write():
// If WriteHeader has not yet been called, Write calls
// WriteHeader(http.StatusOK) before writing the data. If the Header
// does not contain a Content-Type line, Write adds a Content-Type set
// to the result of passing the initial 512 bytes of written data to
// DetectContentType. Additionally, if the total size of all written
// data is under a few KB and there are no Flush calls, the
// Content-Length header is added automatically.
Write([]byte) (int, error)
So your authError() should be something like this:
func authError(w http.ResponseWriter, err error, clientMsg string) {
log.Println("Authentication failed: %v", err)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusForbidden)
err = json.NewEncoder(w).Encode(struct {
Error string
}{Error: clientMsg})
if err != nil {
log.Println("Failed to write response: %v", err)
}
return
}
Related
i receive a response body "bad request" with "httptest.Client().Postform"
type testServer struct {
*httptest.Server
}
func newTestServer(t *testing.T, h http.Handler) *testServer {
ts := httptest.NewTLSServer(h)
jar, err := cookiejar.New(nil)
if err != nil {
t.Fatal(err)
}
ts.Client().Jar = jar
ts.Client().CheckRedirect = func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}
return &testServer{ts}
}
func (ts *testServer) postForm(t *testing.T, urlPath string, form url.Values) (int, http.Header, string) {
rs, err := ts.Client().PostForm(ts.URL+urlPath, form)
if err != nil {
t.Fatal(err)
}
defer rs.Body.Close()
body, err := io.ReadAll(rs.Body)
if err != nil {
t.Fatal(err)
}
bytes.TrimSpace(body)
return rs.StatusCode, rs.Header, string(body)
}
I don't know where is the problem, i have also verified the url it's correct.
Always badrequest with POST but with GET request it's works fine.
this is the handler object :
func (app *application) routes() http.Handler {
router := httprouter.New()
router.NotFound = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
app.notFound(w)
})
dynamic := alice.New(app.sessionManager.LoadAndSave, noSurf, app.Authenticated)
router.Handler(http.MethodGet, "/", dynamic.ThenFunc(app.home))
router.Handler(http.MethodGet, "/user/signup", dynamic.ThenFunc(app.userSignup))
router.Handler(http.MethodPost, "/user/signup", dynamic.ThenFunc(app.userSignupPost))
standart := alice.New(app.recoverPanic, app.logRequest, securityHeaders)
return standart.Then(router)
}
the test function :https://go.dev/play/p/k45-JYTYCOS
the app.userSignupPost:
func (app *application) userSignupPost(w http.ResponseWriter, r *http.Request) {
var form userSignupForm
err := app.decodPostForm(r, &form)
if err != nil {
app.clientError(w, http.StatusBadRequest)
return
}
form.CheckField(validator.NotBlank(form.Name), "name", "this field must not be blank")
form.CheckField(validator.NotBlank(form.Email), "email", "this field must not be blank")
form.CheckField(validator.Matches(form.Email, validator.EmailRX), "email", "this field must be a valid email address")
form.CheckField(validator.NotBlank(form.Password), "password", "this field must not be blank")
form.CheckField(validator.MinChars(form.Password, 8), "password", "password must bee at least 8 caracter long")
if !form.Valid() {
data := app.newTemplateData(r)
data.Form = form
app.render(w, http.StatusUnprocessableEntity, "signup.tmpl.html", data)
return
}
err = app.users.Insert(form.Name, form.Email, form.Password)
if err != nil {
if errors.Is(err, models.ErrDuplicateEmail) {
form.AddFieldError("email", "Email already exist")
data := app.newTemplateData(r)
data.Form = form
app.render(w, http.StatusUnprocessableEntity, "signup.tmpl.html", data)
} else {
fmt.Println("error user postform")
app.serverError(w, err)
}
return
}
app.sessionManager.Put(r.Context(), "flash", "Signup Successful. Please log in")
http.Redirect(w, r, "/user/login", http.StatusSeeOther)
}
It appears that you're using https://github.com/justinas/alice to register handlers - you don't want to do this. That package is for middleware chaining - e.g. "before all requests to this URL, first authenticate the request" - you'd put the authentication into a middleware function and then add it to the chain.
So every POST /user/signup request is getting passed first to app.userSignup() (what you are using to handle GET requests). This is because calling alice.ThenFunc() appends the passed handler to the chain and then returns the entire chain as a handler - you need to read this part of the Alice docs carefully before using it.
Replace this line:
router.Handler(http.MethodPost, "/user/signup", dynamic.ThenFunc(app.userSignupPost))
with
router.Handler(http.MethodPost, "/user/signup", http.HandlerFunc(app.userSignupPost))
You may not need the additional decoration of http.HandlerFunc() - try it with and without to see what works. I cannot say for sure without knowing what the body of app.userSignupPost() looks like (same for the other handler functions as well).
You'll then need to do the same for the other handler registration lines - you shouldn't be using middleware chaining for your end handlers. An http.Handler is used for saying, "send any request to path /PP/ppp with method XXXX to this function." Middleware chaining is for preprocessing (authentication, authorization, etc.) - a whole host of things can be done there, but end request handling shouldn't be one of them.
I'm still curious if your use of PostForm() is going to cause you issues for the reason I cited in my comment on your question - try a raw Post() and see if the behavior differs, but after refactoring to take out the alice goop (at least temporarily). When testing a handler, I'd start off with a much more minimal approach - test that the handler itself works before muddying the waters with both alice and what looks like this package.
I think i found the problem , the session cookie are not the same for get and post request. i don't know why it has changed.They use the same http.Client()
I am using go-kit to create an RPC endpoint. I am creating an endpoint like this
httptransport.NewServer(
endPoint.MakeGetBlogEndPoint(blogService),
transport.DecodeGetBlogRequest,
transport.EncodeGetBlogResponse
Below is my DecodeGetBlogRequest function
func DecodeGetBlogRequest(c context.Context, r *http.Request) (interface{}, error) {
vars := mux.Vars(r)
id, err := strconv.Atoi(vars["id"])
if err != nil {
return nil, err
}
req := endPoint.GetBlogRequest{
ID: id,
}
return req, nil
}
What I want to do is validate the HTTP request in this function and if found invalid, send a response with a valid error code from here only, without passing it to the service layer. i.e. If ID is not a valid number, return 400 Bad Request response from here.
But as I don't have a ResponseWriter reference in this function, I am not sure how to do it.
I am following this example from go-kit docs
https://gokit.io/examples/stringsvc.html
Is it a valid assumption that request/payload should be validated in the transport layer only and the service layer should only be called if the request/payload is valid? If yes, how to do so in this example?
You could use ServerErrorEncoder which returns Server options (can be found in github.com/go-kit/kit/transport/server.go).
Basically in your transport layer, apart from the Decode and Encode functions, you can define an YourErrorEncoderFunc() function which could look like the following. This will catch any error thrown in the transport layer.
YourErrorEncoderFunc(_ context.Context, err error, w http.ResponseWriter).
You will need to attach this function as an option in your endpoint registration like:
ABCOpts := []httptransport.ServerOption{
httptransport.ServerErrorEncoder(YourErrorEncoderFunc),
}
r.Methods("GET").Path("/api/v1/abc/def").Handler(httptransport.NewServer(
endpoints.GetDataEndpoint,
DecodeGetRequest,
EncodeGetResponse,
ABCOpts...,
))
This will stop at transport layer if your request validation is invalid and throw and error in the http response based of whatever format you've written in YourErrorEncoderFunc().
Not 100% sure if this applies to go-kit grpc as well:
You have an error return variable. Use that to indicate there was a problem. In the go grpc module there is a status package to return errors with status codes. If you return an error with a status code, the grpc layer will take the code from the error and send it back.
For example:
func DecodeGetBlogRequest(c context.Context, r *http.Request) (interface{}, error) {
vars := mux.Vars(r)
id, err := strconv.Atoi(vars["id"])
if err != nil {
return nil, status.Error(codes.InvalidArgument, err.Error())
}
req := endPoint.GetBlogRequest{
ID: id,
}
return req, nil
}
Note also that grpc uses different status codes. In Go they are located in the codes package.
Can you return json when http.Error is called?
myObj := MyObj{
MyVar: myVar}
data, err := json.Marshal(myObj)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Write(data)
w.Header().Set("Content-Type", "application/json")
http.Error(w, "some error happened", http.StatusInternalServerError)
I see that it returns 200 with no json but the json is embed in text
I've discovered that it's really easy to read the Go source. If you click on the function in the docs, you will be taken to the source for the Error function: https://golang.org/src/net/http/server.go?s=61907:61959#L2006
// Error replies to the request with the specified error message and HTTP code.
// It does not otherwise end the request; the caller should ensure no further
// writes are done to w.
// The error message should be plain text.
func Error(w ResponseWriter, error string, code int) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(code)
fmt.Fprintln(w, error)
}
So if you want to return JSON, it's easy enough to write your own Error function.
func JSONError(w http.ResponseWriter, err interface{}, code int) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(code)
json.NewEncoder(w).Encode(err)
}
It should be plain text only.
From docs
func Error(w ResponseWriter, error string, code int)
Error replies to the request with the specified error message and HTTP
code. It does not otherwise end the request; the caller should ensure
no further writes are done to w. The error message should be plain
text.
Also I think your usage of http.Error is not correct. When you call w.Write(data), the response is sent and response body will be closed. That is why you are getting 200 status instead of 500 from http.Error.
Instead of using http.Error, you can send your own error response with json just like how you would send any other response by setting the status code to an error code.
Like #ShashankV said, you are writing the response in a wrong way.
As an example, the following is what I did during learning about writing RESTful API serving in Golang:
type Response struct {
StatusCode int
Msg string
}
func respond(w http.ResponseWriter, r Response) {
// if r.StatusCode == http.StatusUnauthorized {
// w.Header().Add("WWW-Authenticate", `Basic realm="Authorization Required"`)
// }
data, err := json.Marshal(r)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, err.Error())
return
}
w.WriteHeader(r.StatusCode)
fmt.Fprintf(w, r.Msg)
}
func Hello(w http.ResponseWriter, r *http.Request) {
resp := Response{http.StatusOK, welcome}
respond(w, resp)
}
Ref: https://github.com/shudipta/Book-Server/blob/master/book_server/book_server.go
Hope, this will help.
My answer is a bit late and there are some good answers already. Here are my 2 cents.
If you want to return JSON in case of error there are multiple ways to do so. I can list two:
Write your own Error handler method
Use the go-boom library
1. Writing your own error handler method
One way is what #craigmj has suggested, i.e. create your own method, for eg.:
func JSONError(w http.ResponseWriter, err interface{}, code int) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(code)
json.NewEncoder(w).Encode(err)
}
2. Use the go-boom library
Another approach is using the go-boom library. For eg., in case the err relates to resource not found, you can do:
err := errors.New("User doesn't exist")
...
boom.NotFound(w, err)
And the response will be:
{
"error": "Not Found",
"message": ",
"statusCode": 404
}
For more check the documentation of the go-boom.
Hope that helps.
My API server has middle ware which is getting token from request header.
If it access is correct, its go next function.
But request went to middle ware and went to next function, c.Request.Body become 0.
middle ware
func getUserIdFromBody(c *gin.Context) (int) {
var jsonBody User
length, _ := strconv.Atoi(c.Request.Header.Get("Content-Length"))
body := make([]byte, length)
length, _ = c.Request.Body.Read(body)
json.Unmarshal(body[:length], &jsonBody)
return jsonBody.Id
}
func CheckToken() (gin.HandlerFunc) {
return func(c *gin.Context) {
var userId int
config := model.NewConfig()
reqToken := c.Request.Header.Get("token")
_, resBool := c.GetQuery("user_id")
if resBool == false {
userId = getUserIdFromBody(c)
} else {
userIdStr := c.Query("user_id")
userId, _ = strconv.Atoi(userIdStr)
}
...
if ok {
c.Nex()
return
}
}
next func
func bindOneDay(c *gin.Context) (model.Oneday, error) {
var oneday model.Oneday
if err := c.BindJSON(&oneday); err != nil {
return oneday, err
}
return oneday, nil
}
bindOneDay return error with EOF. because maybe c.Request.Body is 0.
I want to get user_id from request body in middle ware.
How to do it without problem that c.Request.Body become 0
You can only read the Body from the client once. The data is streaming from the user, and they're not going to send it again. If you want to read it more than once, you're going to have to buffer the whole thing in memory, like so:
bodyCopy := new(bytes.Buffer)
// Read the whole body
_, err := io.Copy(bodyCopy, req.Body)
if err != nil {
return err
}
bodyData := bodyCopy.Bytes()
// Replace the body with a reader that reads from the buffer
req.Body = ioutil.NopCloser(bytes.NewReader(bodyData))
// Now you can do something with the contents of bodyData,
// like passing it to json.Unmarshal
Note that buffering the entire request into memory means that a user can cause you to allocate unlimited memory -- you should probably either block this at a frontend proxy or use an io.LimitedReader to limit the amount of data you'll buffer.
You also have to read the entire body before Unmarshal can start its work -- this is probably no big deal, but you can do better using io.TeeReader and json.NewDecoder if you're so inclined.
Better, of course, would be to figure out a way to restructure your code so that buffering the body and decoding it twice aren't necessary.
Gin provides a native solution to allow you to get data multiple times from c.Request.Body. The solution is to use c.ShouldBindBodyWith. Per the gin documentation
ShouldBindBodyWith ... stores the
request body into the context, and reuse when it is called again.
For your particular example, this would be implemented in your middleware like so,
func getUserIdFromBody(c *gin.Context) (int) {
var jsonBody User
if err := c.ShouldBindBodyWith(&jsonBody, binding.JSON); err != nil {
//return error
}
return jsonBody.Id
}
After the middleware, if you want to bind to the body again, just use ctx.ShouldBindBodyWith again. For your particular example, this would be implemented like so
func bindOneDay(c *gin.Context) (model.Oneday, error) {
var oneday model.Oneday
if err := c.ShouldBindBodyWith(&oneday); err != nil {
return error
}
return oneday, nil
}
The issue we're fighting against is that gin has setup c.Request.Body as an io.ReadCloser object -- meaning that it is intended to be read from only once. So, if you access c.Request.Body in your code at all, the bytes will be read (consumed) and c.Request.Body will be empty thereafter. By using ShouldBindBodyWith to access the bytes, gin saves the bytes into another storage mechanism within the context, so that it can be reused over and over again.
As a side note, if you've consumed the c.Request.Body and later want to access c.Request.Body, you can do so by tapping into gin's storage mechanism via ctx.Get(gin.BodyBytesKey). Here's an example of how you can obtain the gin-stored Request Body as []byte and then convert it to a string,
var body string
if cb, ok := ctx.Get(gin.BodyBytesKey); ok {
if cbb, ok := cb.([]byte); ok {
body = string(cbb)
}
}
In Gorilla, using RecoveryHandler we could suppress the panics. However is there a handler or a library method to respond with a specific Http status code and message for given error type.
For example, in case of a Panic for Mandatory field missing error, one would want to respond with Http 400 and a meaningful message of what exactly is wrong with the payload.
What is the recommended approach to do this?
UPDATE
In code: 2 approaches are listed
Handle errors returned at each method call and build the response.
Instead of returning errors, panic with custom error types and defer the error recovery to a func to build the response. This makes the code easy to read and less repetitive.
func fooHandler(w http.ResponseWriter, r *http.Request) {
//decode the request body into a struct instance
if err := decode(r, myInstance); err != nil {
sendErrorResponse(w,err,http.StatusBadRequest)
return
}
//validate the struct instance for all mandatory keys presence
if err := file.validate(); err != nil {
sendErrorResponse(w,err,http.StatusBadRequest)
return
}
//call DB and validate the response and handle the error
//do some computation and again handle error.
//finally construct response
}
func barHandler(w http.ResponseWriter, r *http.Request) {
//similar to above handler many funcs are called before the response is contruscted
}
func tomHandler(w http.ResponseWriter, r *http.Request) {
//similar to above handler many funcs are called before the response is contruscted
}
func differentHandler(w http.ResponseWriter, r *http.Request) {
defer recoverForErrors(w,r)
// call as many funcs as you need.
// validation, decoding etc will panic instead of returning errors.
// This will avoid the repetitive boiler plate code of handling error and converting to meaningful error response
// instead all this logic is pushed to recoverForErrors func. Which retrieves the error from panic and checks for
// specific error type to construct the error http response
}
It is idiomatic to lean on the interfaces provided by the standard library as much as possible. In this case, the http.Handler interface from the net/http package.
In your case, you can create a new type that allows your handlers to return an error type, and handle all of those error cases centrally.
// StatusError wraps an existing error with a HTTP status code.
type StatusError struct {
Status int
// Allows you to wrap another error
Err error
}
func (e *StatusError) Error() string {
return e.Error()
}
type AppHandler func(w http.ResponseWriter, r *http.Request) error
// Satisfies the http.Handler interface
func (ah AppHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Centralises your error handling
err := ah(w, r)
if err != nil {
switch e := a.(type) {
case *StatusError:
switch e.Status {
case 400:
http.Error(w, e.Err.Error(), 400)
return
case 404:
http.NotFound(w, r)
return
default:
http.Error(w, http.StatusText(500), 500)
return
}
default:
http.Error(w, http.StatusText(500), 500)
return
}
}
// Your handlers will look like this
func SomeHandler(w http.ResponseWriter, r *http.Request) error {
err := decode(r, myInstance)
if err != nil {
return &StatusError{400, err}
}
err := file.validate()
if err != nil {
return &StatusError{400, err}
}
// Continue on...
return nil
}
The benefits you get here include:
No panicking for errors that can be handled
You can centralise your error handling in your ServeHTTP method - i.e. for 400 errors, you might write the error reason to the response. For 500 errors, you might return a generic message since a HTTP 500 isn't something the user can be expected to solve.
Your handler functions return errors explicitly, and you no longer need to remember to use naked return statements to avoid continued execution.
Your StatusError type wraps the error with a status code, but still allows you to inspect/log/write out the wrapped error easily.
Further reading:
http://blog.golang.org/error-handling-and-go
http://elithrar.github.io/article/http-handler-error-handling-revisited/
http://dave.cheney.net/2014/12/24/inspecting-errors