I'm using gin for creating web services.
gin has Bind(&request) which validates the request based on the tags provided in struct.
If the validation fails, it returns an error.
But I want the invalid request to be logged. How do I get the request json?
type SignupRequest struct {
FirstName string `json:"first_name" binding:"required"`
AssociatedBank string `json:"associated_bank" binding:"required"`
}
...
srj := NewSignupRequest()
err = c.Bind(srj)
If the required fails, I won't have the request data to log.
Two options come to mind.
To do this all within Gin, use c.ShouldBindWithBody(...) which has a side effect of storing the request body in the context key/value map under the key gin.Context.BodyBytesKey. Here is the implementation of ShouldBindWithBody for reference:
https://github.com/gin-gonic/gin/blob/master/context.go#L711
Beyond that, you can access the request itself from the gin.Context as c.Request. This is a standard net/http.Request with all the available functionality listed here:
https://golang.org/pkg/net/http/#Request (reading body, headers, etc.).
For option 2, I haven't tested this myself, but I would expect that since c.Bind(...) is reading from the request body behind the scenes, you might find that reading from c.Request.Body after c.Bind() fails doesn't give you the full request body (it's already been at least partially, if not fully, read in c.Bind()). In that case, you may have to tweak your code to read from the request yourself into a byte buffer and then do the binding (marshalling) from the byte buffer to SignupRequest yourself. In that way, you'll have the raw bytes to log or do whatever else you want with if the binding fails.
Related
I recently started rewriting some of my Python services in Go to speed them up and came across this section of the gin documentation:
https://github.com/gin-gonic/gin#goroutines-inside-a-middleware
So I understand the instructions, but I'm trying to understand why? What is the significance of making a copy, and what problem is introduced if I do not make a copy of context for goroutines within handlers?
Gin-gonic is async itself which makes it great.
If you are using concurrency within your handlers it's very likely you will face situation when Context, which is struct goes
outdated since it's holding the data from each request (parameters, keys protected by mutex)
empty, which will cause a fallback to default Context
here's how it looks like:
type Context struct {
writermem responseWriter
Request *http.Request
Writer ResponseWriter
Params Params
handlers HandlersChain
index int8
fullPath string
engine *Engine
params *Params
skippedNodes *[]skippedNode
// This mutex protects Keys map.
mu sync.RWMutex
// Keys is a key/value pair exclusively for the context of each request.
Keys map[string]any
// Errors is a list of errors attached to all the handlers/middlewares who used this context.
Errors errorMsgs
// Accepted defines a list of manually accepted formats for content negotiation.
Accepted []string
// queryCache caches the query result from c.Request.URL.Query().
queryCache url.Values
// formCache caches c.Request.PostForm, which contains the parsed form data from POST, PATCH,
// or PUT body parameters.
formCache url.Values
// SameSite allows a server to define a cookie attribute making it impossible for
// the browser to send this cookie along with cross-site requests.
sameSite http.SameSite
}
In order to foresee such issues and race conditions - you must use Copy whenever your handlers are using go concurrency
here's a quote from gin-gonic repository:
// Copy returns a copy of the current context that can be safely used outside the request's scope.
// This has to be used when the context has to be passed to a goroutine.
I would like to know if we can update tag on a struct instance before unmarshaling data into it.
type Response struct {
Name string `json:"name"`
Payload json.RawMessage `json:"default"`
}
var data Response
<update tag on data.Payload to say `json:"id"`>
json.Unmarshal(server_response, &data)
The motivation is to load common keys in Response struct and delegate API specific response to API handler by passing the raw Payload.
The Payload fields, are complex structs, hence being parsed in their own handlers makes it cleaner.
Need to apply the tag, to let json.Unmarshal know which key from server_response to map to Payload.
The metadata fields (like Name) need some validations. So, if I pass the entire response to handler, each handler has to extract each field and return this metadata, which is well, not a very clean approach.
Decoding response to map[string]interface{} also leads to same issue. I want all the fields of metadata in one struct, populated automatically and handler to parse payload. Decoding to generic map, means copying keys.
Example:
type Response struct {
Version int
Name string
Hash string
Payload json.RawMessage
}
Would like the main function to load the server response in this object, to be able to do all kinds of validations and pass on Payload to handler to let it deal with it.
Using generic map means writing code like:
decodedData.Version = genericMap["version"]
which does not scale to lots of keys.
If I understand your question properly, why don't you simply pass the entire response to the handler? The handler will then know if it needs to read from the stats or the id or whatever field... https://play.golang.org/p/pQXa3Gm_WS0 shows roughly the idea.
An alternative would be to decode your response into a map[string]interface{} and to use https://github.com/mitchellh/mapstructure afterwards to decode parts of the response into structs.
We often have use cases where we only want to update a subset fields on a resource. So if we have a resource Person:
type Person struct {
Age int
Name string
Otherfield string
}
Say the calling client only wants to update the Age field. How would an endpoint be normally set up to handle this?
I believe this should be done with a PATCH request, with only the fields being set as part of the payload, ie:
{
Age: 21
}
However, this won't work with proto3 because as far as I know there are no null fields, only default values. This won't work in many cases where the default value is valid.
Looking at Google own protobuf files (e.g. here), they use FieldMask for partial update.
FieldMask object is passed along with the request, and has the form (in JSON):
{
mask: "Person.Age"
}
This allows the client to tell the server which fields they wish to update, without counting on the partial message itself to figure this out.
I think this adds unnecessary complexity on (each!) client, but we couldn't find any other way to achieve partial updates with proto3.
You can see full documentation of FieldMask here.
Note that it can also be used to filter out responses if the client doesn't need the entire object.
I've been attempting to inject a custom header for a error response status (and failing).
I have a very simple lambda being used
exports.handler = (event, context, callback) => {
// TODO implement
//callback(null, 'Hello from Lambda');
var error = {
name:"error",
message:"I am a failure",
statusCode: 400
};
error["x-test"] = 'foo';
callback(JSON.stringify(error), null);
};
In the api gateway, I've done the following:
set up CORS to include x-test
responsetemplate = "$input.path('$.errorMessage')"
responseparameter to include:
method.response.header.x-test = integration.response.body.x-test
Also, I have a statusCode mapped using '.*statusCode.*?400.*'
This has turned out empty.
so I decided to take a step back and see what happens if I do:
method.response.header.x-test = integration.response.body
I found that I get the stringified response of errorMessage.
{"x-test":"{\"errorMessage\":\"{\\\"name\\\":\\\"error\\\",\\\"message\\\":\\\"I am a failure\\\",\\\"statusCode\\\":400,\\\"x-test\\\":\\\"foo\\\"}\"}"}
So I decided to change the responsetemplate to force it to json by doing the following:
responsetemplate = "$util.parseJson($input.path('$.errorMessage'))"
and I still get the stringified response:
{"x-test":"{\"errorMessage\":\"{\\\"name\\\":\\\"error\\\",\\\"message\\\":\\\"I am a failure\\\",\\\"statusCode\\\":400,\\\"x-test\\\":\\\"foo\\\"}\"}"}
My guess is that it doesn't transform as expected, but only for the final output.
So how would you take a value and shove it into a header?
Thanks,
Kelly
I think this is more of a design choice regarding the limitation imposed by both Lambda and APIGateway. I will try my best to walk through my thoughts.
First of all, in Lambda, callback(error, result) function can either take an error string as first argument, or an object as result response. If you want to pass along a simple error message, you could definitely just do that. However, in your case, as you tried to pass along an entire error object, choosing the second option is clearly a better solution (in contrast to stringifying an object and parse it into object again). As a result, the final line of your Lambda function should be:
callback(null, error);
Yes, in this case, if you test your function in Lambda, the output result will no longer be red and flag it as an error, but this won't matter as you can format your headers and response in APIGateway.
Now you need to set things up in APIGateway, in which you need to make use of the object passed by Lambda.
It's actually rather easy to use method execution interface to configure headers.
In Method Response, you need to add the headers you want to include in the response for a specific status code, which in your case is x-test. (If you want the API to return different status codes, you can also configure that in this panel.)
Then go to Integration Response, under the same status code, you will see the added header available. According to this documentation from AWS, you can use integration.response.body.JSONPath_EXPRESSION to assign the header value (this is another reason that you should return object rather than string in Lambda, as there is no formal API to parse object from string at this stage). This time, as your Lambda is passing an object, the value of x-test header is:
integration.response.body['x-test']
Now your API should have the proper header included.
NOTE: In order to set up different status code in APIGateway, you should leave some distinguishable data fields (your statusCode: 400 should work perfectly) in you response body, so you can use RegEx to match those fields to a specific status code.
So... above doesn't work with success message. I found this blog though talking about error handling design pattern. Apparently what they suggest is only mapping status code when there is an error, in which case no body should be passed (only the errorMessage), as browser won't care about response body for a status code other than 200 anyway.
I guess after all, it is impossible to customize both status code and header at the same time with Lambda passing an object to APIGateway?
This is due to the fact that you are stringifying the error object coming from your Lambda function. API Gateway attempts to resolve the JSON-Path expression and can't evaluate "x-test" in a string. If you return an object instead of a string, this should work.
You may want to consider using proxy integrations which allow you to control the headers and status directly from your Lambda function.
Update: I've written a blog post on this topic with sample code # https://rpgreen.wordpress.com/2017/01/25/how-to-send-response-headers-for-aws-lambda-function-exceptions-in-api-gateway/
I'm calling a REST API somebody else created. It supports JSONP to facilitate cross domain access.
The response I get back from the service looks like:
mycallback('{"token": "123456789"}');
Notice the single quotes wrapping the JSON data; Passing it as a string rather than a raw object. JQuery can handle this, but other libraries seem to expect a raw object instead.
mycallback({"token": "123456789"});
The raw object parameter makes more sense to me since it avoids the need to parse the JSON data, but I want to know for sure before asking the maintainer of the API to make the adjustment:
Which is most correct?
Passing a javascript literal (second) as shown here is more correct as it avoids deserializing the string back to a javascript object.
Passing a string is obviously a bad thing - you have two choices (#1 is preferred):
Ask the developer of the JSONP service to send proper JSONp instead of a string
Make your callback function smart so it uses something like payload = JSON.parse(payload); in case payload is a string.