My API-gateway starts a tracer and a span for validate email. Then its passed to user-service for validation.
I want to pass this span details to user-service as a json object and start another span as a
tracer.start_span('Validate Email', child_of=API_gateway_span)
To do it, I have used following struct:
type checkEmail struct {
GatewayTracerSpan opentracing.SpanContext `json: gatewayTracerSpan`
Email string `json: email`
Uuid string `json: uuid`
}
In function()
validateEmailSpan := apitracer.tracer.StartSpan("Validate Email")
emailJson := checkEmail{
GatewayTracerSpan: validateEmailSpan.Context(),
Email: email,
Uuid: uuid,
}
But always GatewayTracerSpan is empty value.
I have just started distributed-tracing. Here I selected to use json over native http-headers as its easy for upgrade any protocol change.
Is this possible? If so, am I doing it right? Or what mistakes did I make?
One way to link spans from different services is to use uber-trace-id from the parent span. If you have LogSpans set to true in your ReporterConfig, uber-trace-id is what gets printed out ("Reporting span xxx-xxx-xxx").
Here is how it might look like in the code:
//API Gateway
carrier := opentracing.TextMapCarrier{} //you can use any type of carrier or even create your own
ctx, _ := opentracing.GlobalTracer().Extract(opentracing.TextMap, carrier)
span := apitracer.tracer.StartSpan(name, ext.RPCServerOption(ctx))
_ := span.Tracer().Inject(span.Context(), opentracing.TextMap, carrier)
uberTraceID := carrier["uber-trace-id"]
You can now pass uberTraceID instead of validateEmailSpan.Context() to your other services.
You can use this function in your other services:
//Email service
func NewChildSpanThatFollows(name, uberTraceID string) opentracing.Span {
carrier := opentracing.TextMapCarrier{}
carrier.Set("uber-trace-id", uberTraceID)
ctx, _ := opentracing.GlobalTracer().Extract(opentracing.TextMap, carrier)
span := opentracing.StartSpan(name, opentracing.FollowsFrom(ctx))
_ := span.Tracer().Inject(span.Context(), opentracing.TextMap, carrier)
return span
}
This works for me if I need to see spans between services linked together in a parent-child manner. If other information needs to be passed as well, I would suggest passing it as regular data in the JSON object, then either create my own Carrier or use tags if needed to do a search with that passed data.
span.SetTag("request_id", requestID)
EDIT:
Here you can find a great tutorial on using opentracing. It uses HTTPHeadersCarrier, it has a step by step walkthrough, but it's basically the same process as above.
Related
Below is the signature of the WithValue func.
func WithValue(parent Context, key, val interface{}) Context
I took a look at Go official documentation but they didn't mention how a value would get marshalled when the context is passed over to a remote service.
For example, I can pass in anything as val such as a complex struct containing exported and un-exported fields. I wonder if my un-exported fields would ever reach the remote service. If they do, how does it work behind the scene? The whole context object gets marshalled into []byte regardless of fields?
One concrete sample of a complex object is the internal implementation of a Span in opentracing.
// Implements the `Span` interface. Created via tracerImpl (see
// `basictracer.New()`).
type spanImpl struct {
tracer *tracerImpl
event func(SpanEvent)
sync.Mutex // protects the fields below
raw RawSpan
// The number of logs dropped because of MaxLogsPerSpan.
numDroppedLogs int
}
The above Span can be passed around from service to service by creating a derived context using the func below.
// ContextWithSpan returns a new `context.Context` that holds a reference to
// the span. If span is nil, a new context without an active span is returned.
func ContextWithSpan(ctx context.Context, span Span) context.Context {
if span != nil {
if tracerWithHook, ok := span.Tracer().(TracerContextWithSpanExtension); ok {
ctx = tracerWithHook.ContextWithSpanHook(ctx, span)
}
}
return context.WithValue(ctx, activeSpanKey, span)
}
What is the high-level protocols/rules that Go uses to pass around complex values stored in Go context? Would the func(SpanEvent) and the mutex really travel over the wire to the next service?
I'd be very grateful if you could explain the expected behavior or point me in the direction of some articles I've not found.
I have a struct that is populated with Unmarshal from a web response. Then I want to take a particular nested field from this JSON structure and feed it into a different web request. The issue is that the field name gets removed when using Marshal. As a workaround, right now I'm doing the equivalent of:
type Message struct {
Dogs json.RawMessage `json:"doggies"`
}
...
type Doggies struct {
Dogs json.RawMessage `json:"doggies"`
}
var msg Message
// msg gets populated from a request
postBody, err := json.Marshal(Doggies{Dogs: msg.Doggies})
Which is messy. Is there a way that I can get Marshall to include the field name? So instead I could have
postBody, err := json.Marshal(msg.Doggies)
Thank you
To my shame, I have not been able to figure out how to get data from the frontend in Gin framework. In Django I get data So:
user=request.data.get('user')
print(user)
Everything is simple and understandable as day.
How should I do it in gin?
user := c.Query("user")
user := c.Param("user")
user := c.Params.ByName("user")
user := c.PostForm("user")
println(user)//emptiness....
Well, I'd say you should fetch some book/HOWTO on how HTTP work and spend some time with it because it appears you're trying to bash the problem at hand without actually understanding what happens between your browser and your backend service.
The real problem here is that there are more moving parts that you appear to be aware of, and the way to go depends on what your frontent does.
You did not tell us exactly how you're doing your request,
but from a solicted comment it appears, you're using that "axios" tingy.
If I managed to google that project correctly,
its README states:
By default, axios serializes JavaScript objects to JSON. To send data in the application/x-www-form-urlencoded format instead, you can use one of the following options.
This means two things:
Unless you somehow tweaked the axios' settings, when you did
axios.post, is supposedly performed an HTTP POST request
with its Content-Type field set to application/json
and its payload (or "body" if you prefer) being
a JSON serialization of that {user:this.user} JavaScript object.
It's therefore futile to attempt to parse the query string.
And it's futile to attempt to parse the request as an HTTP form — which it isn't.
Instead, you supposedly want to interpret the incoming request's body as being JSON-formatted. I have no idea as to how to do that in "go-gin", but in plain Go that would be something like
func (h *Handler) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
defer req.Body.Close()
var user User
dec := json.NewDecoder(req.Body)
err := dec.Decode(&user)
if err != nil {
rw.Header().Set("Content-Type", "text/plain; charset=UTF-8")
rw.WriteHeader(http.StatusBadRequest)
fmt.Fprintln(rw, "Error parsing request body: ", err)
return
}
}
And ideally you'd first check that the content type of the incoming request was indeed application/json and reject it right away with http.StatusBadRequest if it isn't.
An example of a working code to do that is
// VerifyContentTypeIsJSON makes sure the HTTP header of a server
// http.Request contains the Content-Type field and it indicates
// the request payload is JSON.
// The implementation is based on RFC 7231 (section 3.1.1.5) and RFC 8259.
func VerifyContentTypeIsJSON(header http.Header) error {
var s string
if values := header["Content-Type"]; len(values) > 0 {
s = values[0]
} else {
return errors.New("missing Content-Type")
}
if s == "" {
return errors.New("empty Content-Type")
}
if i := strings.IndexByte(s, ';'); i != -1 {
s = strings.TrimSpace(s[:i])
}
if strings.ToLower(s) != "application/json" {
return fmt.Errorf("unknown Content-Type: %v, must be application/json", s)
}
return nil
}
Having this function, you'd have something like this
after defer req.Body.Close() and actually parsing it:
if err := VerifyContentTypeIsJSON(req.Header); err != nil {
rw.Header().Set("Content-Type", "text/plain; charset=utf-8")
rw.WriteHeader(http.StatusBadRequest)
fmt.Fprintln(rw, err)
return
}
(Note that "go-gin" might have something akin to this already built-in, so please check this.)
The User type should be some struct type matching the shape of the JSON object you intend to unmarshal from the request. Something like this:
type User struct {
User string `json:"user"`
}
None that in the two places my example returned an
error to the user it used content type of plain text
(in UTF-8 encoding). This may be OK but may be not.
Say, your clients might expect a JSON-formatted document
of some agreed-upon shape.
Or you may use content negotiation, but I'd recommend to get simple things straight first.
Literature to check:
HTTP POST request explained at MDN.
URL's query string.
XHR explained at MDN — see also links there.
"Writing Web Applications in Go",
and this in general.
And to maybe answer that part of your question regarding
why it "just worked" in Django.
I can only guess, but I think it merely implements tons of magic which looks at the incoming request and tries to guess how to extract data from it.
The problem is that guessing may indeed work well for
one-off throwaway scripts, but when you're about implementing something like web API (what many not quite correctly call "REST", but let's not digress) it's best
to be very explicit about what your endpoint accept
precisely and how precisely they react to requests — both legitimate and non-well-formed.
Regarding magic in Go, you may read this.
you need to call c.Request.ParseForm() before using it from request.Form
says here:
For all requests, ParseForm parses the raw query from the URL and updates r.Form
For other HTTP methods, or when the Content-Type is not application/x-www-form-urlencoded, the request Body is not read, and r.PostForm is initialized to a non-nil, empty value.
If you're expecting a JSON body in the request, you can do it this way with gin. Create a struct for the data you want to grab out of the body. Use json tags for the JSON key names unless you're going to exactly match your Go field names to them. Then call the BindJSON method on the gin context.
For example:
type User struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
Login string `json:"login"`
}
func (h *Handler) HandleUser(gctx *gin.Context) {
user := new(User)
err := gctx.BindJSON(user)
if err != nil {
text := fmt.Sprintf("Failed to read user data from request: %s", err)
log.Error(text)
gctx.JSON(http.StatusBadRequest, gin.H{"error": text})
return
}
// do something with user
}
Server GIN can't handle routine default application/json requests from axios!!! What???
Requests should be sent as application/x-www-form-urlencoded.
My decision in Vue project:
Use vue-resource instead axios (axios.post=>this.$http.post) with option
Vue.http.options.emulateJSON = true; in main.js
Our application provides functionality which enables a customer to create dynamic forms and business rules. We recently decided to explore Google infrastructures so we don't have to spend time tweaking and adjusting our infrastructure.
Thus far, we have managed well using a NOSQL database such as arangodb to store random data sets through their JSON HTTP REST APIs, that stores any sort of data structure, so long as it is a valid JSON. However, Google data store go client library and Datastore doesn;t work with JSON and also imposes rules like no silce []type, no map map[type]type failing with errors such as datastore: invalid Value type e.t.c
I explored option of implementing PropertyLoadSaver interface load/save functions with modifications to create PropertyList and Property to crate a []Property. Case below Collection is type Collection map[string]interface{} which holds an sort of data set
func (m Collection) Save() ([]datastore.Property, error) {
data := []datastore.Property{}
for key, value := range m {
if util.IsSlice(value) {
props := datastore.PropertyList{}
for _, item := range value.([]string) {
props = append(props, datastore.Property{Name: key, Value: item})
}
data = append(data, datastore.Property{Name: key, Value: props})
} else {
data = append(data, datastore.Property{Name: key, Value: value, NoIndex: true})
}
}
json.NewEncoder(os.Stdout).Encode(data)
return data, nil
}
Yes, we can create a struct which we can populate based on map data and save that to Datastore. We were however wondering if there possibly is an easier way to just receive a map and save it to Datastore with no added complexity.
Alternative
type Person struct{
Name string
Surname string
Addresses []Address
...
}
type Address struct{
Type string
Detail string
}
This map[string]interface{}{Name:"Kwasi", Surname:"Gyasi-Agyei", Addresses:...} can than be marshaled into above struct to be saved by Datastore go client lib.
Am however more interested in taking advantage of PropertList, []Property, unless that route is unnecessarily complex. What am basically asking is, which is the most appropriate route that offer the same type of flexibility as a schemaless database.
I'm using golang with couchbase integration component called go-couchbase. It's enable to connect with couchbase and retrieve data. However I have a problem to send start key and skip value and limit value with this API. Because there is no functionality found by myself.
url : - github.com/couchbaselabs/go-couchbase
Please let me know any method to send these values to couchbase and retrieve data?
That start key is only mentioned once, as a parameter to a couhbase view:
// View executes a view.
//
// The ddoc parameter is just the bare name of your design doc without
// the "_design/" prefix.
//
// Parameters are string keys with values that correspond to couchbase
// view parameters. Primitive should work fairly naturally (booleans,
// ints, strings, etc...) and other values will attempt to be JSON
// marshaled (useful for array indexing on on view keys, for example).
//
// Example:
//
// res, err := couchbase.View("myddoc", "myview", map[string]interface{}{
// "group_level": 2,
// "start_key": []interface{}{"thing"},
// "end_key": []interface{}{"thing", map[string]string{}},
// "stale": false,
// })
func (b *Bucket) View(ddoc, name string, params map[string]interface{}) (ViewResult, error) {
I suppose the skip one (mentioned in "Pagination with Couchbase") is just another parameter to add to the params map[string]interface{}.