I have a struct that is populated with Unmarshal from a web response. Then I want to take a particular nested field from this JSON structure and feed it into a different web request. The issue is that the field name gets removed when using Marshal. As a workaround, right now I'm doing the equivalent of:
type Message struct {
Dogs json.RawMessage `json:"doggies"`
}
...
type Doggies struct {
Dogs json.RawMessage `json:"doggies"`
}
var msg Message
// msg gets populated from a request
postBody, err := json.Marshal(Doggies{Dogs: msg.Doggies})
Which is messy. Is there a way that I can get Marshall to include the field name? So instead I could have
postBody, err := json.Marshal(msg.Doggies)
Thank you
Related
My API-gateway starts a tracer and a span for validate email. Then its passed to user-service for validation.
I want to pass this span details to user-service as a json object and start another span as a
tracer.start_span('Validate Email', child_of=API_gateway_span)
To do it, I have used following struct:
type checkEmail struct {
GatewayTracerSpan opentracing.SpanContext `json: gatewayTracerSpan`
Email string `json: email`
Uuid string `json: uuid`
}
In function()
validateEmailSpan := apitracer.tracer.StartSpan("Validate Email")
emailJson := checkEmail{
GatewayTracerSpan: validateEmailSpan.Context(),
Email: email,
Uuid: uuid,
}
But always GatewayTracerSpan is empty value.
I have just started distributed-tracing. Here I selected to use json over native http-headers as its easy for upgrade any protocol change.
Is this possible? If so, am I doing it right? Or what mistakes did I make?
One way to link spans from different services is to use uber-trace-id from the parent span. If you have LogSpans set to true in your ReporterConfig, uber-trace-id is what gets printed out ("Reporting span xxx-xxx-xxx").
Here is how it might look like in the code:
//API Gateway
carrier := opentracing.TextMapCarrier{} //you can use any type of carrier or even create your own
ctx, _ := opentracing.GlobalTracer().Extract(opentracing.TextMap, carrier)
span := apitracer.tracer.StartSpan(name, ext.RPCServerOption(ctx))
_ := span.Tracer().Inject(span.Context(), opentracing.TextMap, carrier)
uberTraceID := carrier["uber-trace-id"]
You can now pass uberTraceID instead of validateEmailSpan.Context() to your other services.
You can use this function in your other services:
//Email service
func NewChildSpanThatFollows(name, uberTraceID string) opentracing.Span {
carrier := opentracing.TextMapCarrier{}
carrier.Set("uber-trace-id", uberTraceID)
ctx, _ := opentracing.GlobalTracer().Extract(opentracing.TextMap, carrier)
span := opentracing.StartSpan(name, opentracing.FollowsFrom(ctx))
_ := span.Tracer().Inject(span.Context(), opentracing.TextMap, carrier)
return span
}
This works for me if I need to see spans between services linked together in a parent-child manner. If other information needs to be passed as well, I would suggest passing it as regular data in the JSON object, then either create my own Carrier or use tags if needed to do a search with that passed data.
span.SetTag("request_id", requestID)
EDIT:
Here you can find a great tutorial on using opentracing. It uses HTTPHeadersCarrier, it has a step by step walkthrough, but it's basically the same process as above.
To my shame, I have not been able to figure out how to get data from the frontend in Gin framework. In Django I get data So:
user=request.data.get('user')
print(user)
Everything is simple and understandable as day.
How should I do it in gin?
user := c.Query("user")
user := c.Param("user")
user := c.Params.ByName("user")
user := c.PostForm("user")
println(user)//emptiness....
Well, I'd say you should fetch some book/HOWTO on how HTTP work and spend some time with it because it appears you're trying to bash the problem at hand without actually understanding what happens between your browser and your backend service.
The real problem here is that there are more moving parts that you appear to be aware of, and the way to go depends on what your frontent does.
You did not tell us exactly how you're doing your request,
but from a solicted comment it appears, you're using that "axios" tingy.
If I managed to google that project correctly,
its README states:
By default, axios serializes JavaScript objects to JSON. To send data in the application/x-www-form-urlencoded format instead, you can use one of the following options.
This means two things:
Unless you somehow tweaked the axios' settings, when you did
axios.post, is supposedly performed an HTTP POST request
with its Content-Type field set to application/json
and its payload (or "body" if you prefer) being
a JSON serialization of that {user:this.user} JavaScript object.
It's therefore futile to attempt to parse the query string.
And it's futile to attempt to parse the request as an HTTP form — which it isn't.
Instead, you supposedly want to interpret the incoming request's body as being JSON-formatted. I have no idea as to how to do that in "go-gin", but in plain Go that would be something like
func (h *Handler) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
defer req.Body.Close()
var user User
dec := json.NewDecoder(req.Body)
err := dec.Decode(&user)
if err != nil {
rw.Header().Set("Content-Type", "text/plain; charset=UTF-8")
rw.WriteHeader(http.StatusBadRequest)
fmt.Fprintln(rw, "Error parsing request body: ", err)
return
}
}
And ideally you'd first check that the content type of the incoming request was indeed application/json and reject it right away with http.StatusBadRequest if it isn't.
An example of a working code to do that is
// VerifyContentTypeIsJSON makes sure the HTTP header of a server
// http.Request contains the Content-Type field and it indicates
// the request payload is JSON.
// The implementation is based on RFC 7231 (section 3.1.1.5) and RFC 8259.
func VerifyContentTypeIsJSON(header http.Header) error {
var s string
if values := header["Content-Type"]; len(values) > 0 {
s = values[0]
} else {
return errors.New("missing Content-Type")
}
if s == "" {
return errors.New("empty Content-Type")
}
if i := strings.IndexByte(s, ';'); i != -1 {
s = strings.TrimSpace(s[:i])
}
if strings.ToLower(s) != "application/json" {
return fmt.Errorf("unknown Content-Type: %v, must be application/json", s)
}
return nil
}
Having this function, you'd have something like this
after defer req.Body.Close() and actually parsing it:
if err := VerifyContentTypeIsJSON(req.Header); err != nil {
rw.Header().Set("Content-Type", "text/plain; charset=utf-8")
rw.WriteHeader(http.StatusBadRequest)
fmt.Fprintln(rw, err)
return
}
(Note that "go-gin" might have something akin to this already built-in, so please check this.)
The User type should be some struct type matching the shape of the JSON object you intend to unmarshal from the request. Something like this:
type User struct {
User string `json:"user"`
}
None that in the two places my example returned an
error to the user it used content type of plain text
(in UTF-8 encoding). This may be OK but may be not.
Say, your clients might expect a JSON-formatted document
of some agreed-upon shape.
Or you may use content negotiation, but I'd recommend to get simple things straight first.
Literature to check:
HTTP POST request explained at MDN.
URL's query string.
XHR explained at MDN — see also links there.
"Writing Web Applications in Go",
and this in general.
And to maybe answer that part of your question regarding
why it "just worked" in Django.
I can only guess, but I think it merely implements tons of magic which looks at the incoming request and tries to guess how to extract data from it.
The problem is that guessing may indeed work well for
one-off throwaway scripts, but when you're about implementing something like web API (what many not quite correctly call "REST", but let's not digress) it's best
to be very explicit about what your endpoint accept
precisely and how precisely they react to requests — both legitimate and non-well-formed.
Regarding magic in Go, you may read this.
you need to call c.Request.ParseForm() before using it from request.Form
says here:
For all requests, ParseForm parses the raw query from the URL and updates r.Form
For other HTTP methods, or when the Content-Type is not application/x-www-form-urlencoded, the request Body is not read, and r.PostForm is initialized to a non-nil, empty value.
If you're expecting a JSON body in the request, you can do it this way with gin. Create a struct for the data you want to grab out of the body. Use json tags for the JSON key names unless you're going to exactly match your Go field names to them. Then call the BindJSON method on the gin context.
For example:
type User struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
Login string `json:"login"`
}
func (h *Handler) HandleUser(gctx *gin.Context) {
user := new(User)
err := gctx.BindJSON(user)
if err != nil {
text := fmt.Sprintf("Failed to read user data from request: %s", err)
log.Error(text)
gctx.JSON(http.StatusBadRequest, gin.H{"error": text})
return
}
// do something with user
}
Server GIN can't handle routine default application/json requests from axios!!! What???
Requests should be sent as application/x-www-form-urlencoded.
My decision in Vue project:
Use vue-resource instead axios (axios.post=>this.$http.post) with option
Vue.http.options.emulateJSON = true; in main.js
I am learning golang and want to write generic response from microservices response.
My General Response is like:
type GeneralResponse struct {
Success string
Message string
Data string
Error string
}
In the Data section I want to return any json, say list of Person, Instruments or any type of objects.
But it should be another json.
I tried assigning other json objects but it did not work.
It is fine if I dump json array as string into it but it should unmarshal from receiver end.
How should I go about it?
I am trying over here. https://play.golang.org/p/dc0uKtS76aA
You should use RawMessage in the type definition
type GeneralResponse struct {
Success string
Message string
Data json.RawMessage
Error string
}
and subsequently push a Marshalled json into that attribute.
You can do that by encoding other types in []bytes and setting them to the Data attribute.
Like in https://play.golang.org/p/CyoN5pe_aNV
If you put marshalled JSON into a string, it will be marshalled as a string (because it's a string) and the receiving end will have to unmarshal it twice (because it's been marshalled twice). What you want is probably more along the lines of:
type GeneralResponse struct {
Success string
Message string
Data interface{} // interface{} can be anything
Error string
}
This way you can put any data into Data and it will be marshalled directly into the response.
You can use json.RawMessage for that.
I have implemented the encoding part, you find more here to decode - https://golang.org/pkg/encoding/json/#RawMessage
json.RawMessage comes to rescue in case you wants to capture whole json without knowing its format.
type GeneralResponse struct {
Success string
Message string
Data json.RawMessage
Error string
}
Checkout this code. I have modified your code to dump data into response
So I've been trying to modify the request structure in golang using middleware, i tried creating a custom structure and embedding the request object and some more data but i can't type assert it to *http.Request, can anybody please help, thanks in advance.
Edit: so here is what my structure looks like
type CustomRequest struct {
*http.Request
*profile.User // This is the data i want to embed into the request
}
// then my middlware will be something like
func Middleware(next http.HandlerFunc) http.HandlerFunc {
return http.HandleFunc(func (w http.ResponseWriter, r *http.Request)) {
user := &User{
// User Details Are Here
}
customRequest := &CustomRequest{
r,
&user,
}
req := customRequest.(*http.Request)
next.ServeHttp(w, req)
}
That isn't how type assertion works.
For an expression x of interface type and a type T, the primary
expression
x.(T) asserts that x is not nil and that the value stored in x is of
type T. The notation x.(T) is called a type assertion.
You type assert interfaces to their underlying type.
You can't type assert one type to another, that would be type conversion, but in this case you can't convert between the two. You can only convert two types that are convertible according to the description in the spec above.
If you want to modify the *http.Request just do it directly, the fields are exported. If you want the request to hold extra data just write it in the request body as JSON or in the url.
EDIT: For passing data around you can also use a context, but I am not sure of what you're doing. There is also github.com/gorilla/context
I have several different structures.
Here show two:
type AdsResponse struct {
Body struct {
Docs []struct {
ID int `json:"ID"`
// others
} `json:"docs"`
} `json:"response"`
Header `json:"responseHeader"`
}
type OtherResponse struct {
Body struct {
Docs []struct {
ID int `json:"ID"`
// others
} `json:"docs"`
} `json:"response"`
Header `json:"responseHeader"`
}
but i don't know how i can do for this method accepts and return both.
func Get(url string, response Response) (Response, bool) {
res, err := goreq.Request{
Uri: url,
}.Do()
// several validations
res.Body.FromJsonTo(&response)
return response, true
}
And use like this:
var struct1 AdsResponse
var struct2 OtherResponse
Get("someURL", struct1)
Get("someURL", struct2)
There are any form?
Your code example is somewhat confusing since both structs appear to be identical. I'll assume that they differ somewhere in "others".
First, I generally recommend creating a wrapper around these kinds of JSON deserializations. Working directly on the JSON structure is fragile. Most of your program should not be aware of the fact that the data comes down in JSON. So for instance, you can wrap this in an Ads struct that contains an AdsResponse, or just copies the pieces it cares about out of it. Doing that will also make some of the below slightly easier to implement and less fragile.
The most common solution is probably to create an interface:
type Response interface {
ID() int
}
You make both Ads and Others conform to Response. Then you can return Response. If necessary, you can type-switch later to figure out which one you have and unload other data.
switch response := response.(type) {
case Ads:
...
case Other:
...
}
I don't quite get why you have the reponse as a parameter and as a return. I think you dont need to return it. You should pass a pointer to the reponse and fill it with the data. Also, I'd return an Error instead of a boolean, but that is another topic.
Anyway, the solution is to use interface{} (empty interface).
You are lucky because the function you are using (FromJsonTo) accepts an empty interface as a parameter, so you can safely change your parameter type to interface{} and just pass it to FromJsonTo. Like this:
func Get(url string, response interface{}) bool {
res, err := goreq.Request{
Uri: url,
}.Do()
// several validations
res.Body.FromJsonTo(response)
return true
}
Warning: I did not compile the code.
Then you would call this function with the url and a pointer to one of the reponse structs like this:
var struct1 AdsResponse
var struct2 OtherResponse
Get("someURL", &struct1)
Get("someURL", &struct2)
The way to achieve this is through Go's interfaces.
Two options:
empty interface
Get(url string, response interface{}) (Response, bool)
This option allows any value to be given to this function.
custom interface
Creating a custom interface will allow you to narrow down the types that can be provided as arguments to your function.
In this case you'll have to create an interface that all your Response structs will need to abide by. Any struct really that abides by that interface will be able to be used as an argument of your function.
Something like this:
type MyResponse interface {
SomeFunction()
}
Then your function signature could look like
Get(url string, response MyResponse) (MyResponse, bool)
As long as AdsResponse and OtherResponse abide by the MyResponse interface, they will be allowed to be used as arguments to the function.
Follow the solution working at Go Playground
Go has no polymorphic or any other OO like behaviour, so, when you try to pass a AdsResponse or OtherResponse struct as an Response (or any interface{}), these values becomes an Response (or other param type specified), and is not possible to Go to infer the real type that originate these interface{} and correctly decode your json to these struct types as expected.
This kind of thing should works perfectly in OO languages, like Java, C# etc. There is no hierarchy generalization/specialization on structs/interfaces in Go.
You would need to do a type assertion in your Rest executor, or a switch case, but it seems that you need a generic REST executor, like a generic lib some thing like that. Would not reasonable create a switch case for each struct in your program. Maybe you have dozens or hundreds of structs soon.
I think that a reasonable solution is the rest client pass a lambda function to do the last step for your, that is just create a correct struct destination type and call json decode.
As i say above, the return type of executeRest() in my example will became an interface{}, but the rest client can securely do the type assertion of returned value after executeRest() call.