I'm trying to implement data contract logic.
I've 2 services switching messages with each other.
Service A send messages in some format + the file descriptor proto of the .proto file used to generate it. Service B gets them both, the message and the file descriptor proto and has to ensure the message doesn't break the schema defenition.
What I've did until now is to create a dynamic message instance using the descriptor proto and tried to unmarshal the message into the dynamic message instance, and in case no error occurred during the unmarshal process it counts as a success (message doesn't break the schema).
Is it ok to rely on the unmarshal function in order to decide whether the message is ok?
I noticed that even in case I'm sending messages with totally different schemas the unmarshal succeed. the only way I found to cause the unmarshal to fail is by sending proto2 messages with missing required fields.
So is it by design that every message can be unmarshled using a totally different schema definition?
I'm using the official Go protobuf library
Yes, it is ok to use unmarshal function to check if your proto is valid or not.
No, it's not design to unmarshal arbitrary message but it is designed to unmarshal only that fields which are present in your protobuf message. For example, you have proto message with structure as follow:
message MyMessage {
uint64 id = 1;
string name = 2;
string surname = 3;
}
If your server receives message that contains just id and name and your server is trying to unmarshal this message, id and name fields would be unmarshaled while surname field in your structure would be empty.
This approach appliable for JSONs too.
Related
Consider the following protobuf:
message SendRequest {
/** The raw text of the message that the caller wishes to send. */
optional string message = 1;
}
Note that the string message is being used as a field name. The protobuf compilers seems to be fine with this, even though message is the protobuf keyword.
Is this usage kosher in the sense that keywords are defined by some standard to be ignored in field names, or could this break my application in the future?
Yes, it is possible because in runtime a proto field is identified by their id (in your case 1) and not with their name.
I am using protobuf for my project. If its possible to create a message that have something like this.
message GenericType{
T value = 1;
}
You can use google.protobuf.Any type for serialization, but you have to be aware that each application who works with your data should to know how to pack/unpack these messages.
I'm building a microservice system with multiple disconnected components, and I'm currently trying to find out how to implement knowing which fields on an object should be updated based on the protobuf data provided.
The flow is this:
The client sends a JSON-request to an API.
The API translates the JSON-data into a protobuf struct, which is then sent along to the microservice responsible for handling it.
The microservice receives the data from the API and performs any action on it, in this case, I'm trying to change a single value in a MySQL table, such as a client's email address.
Now, the problem I have is that since protobuf (understandably) doesn't allow pointers, the protobuf object will contain zero-values for everything not provided. This means that if a customer wants to update their email address, I can't know if they also set IncludeInMailLists to false - or if it was simply not provided (having its zero-value) and shouldn't change.
The question is: how will I - from the protobuf object - know if a value is expressively set to 0, or just not provided?
My current solution is pretty much having a special UpdateCustomer-object which also has an array of Fields specifying which fields the microservice should care about, but it feels like bad solution.
Someone must have solved this better already. How should I implement it?
Protobufs field masks are one way.
https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.FieldMask
https://github.com/golang/protobuf/issues/225
But if you are using grpc then there's a (sort of) built in way.
Grpc wrappers
Since proto3 (protobufs v3) there's been no distinction between a primitive that is not set, and a primitive that's been set to the "zero value" (false, 0, "", etc).
Instead you can use objects or in protobufs language a "message", as objects can be nil / null. You've not mentioned what language you are working in but hopefully these examples make sense.
Given an RPC service such as:
import "google/protobuf/wrappers.proto";
service Users {
rpc UpdateUser(UpdateUserRequest) returns (UpdateUserResponse)
}
message UpdateUserRequest {
int32 user_id = 1;
google.protobuf.StringValue email = 2;
}
message UpdateUserResponse {}
Note the import "google/protobuf/wrappers.proto"; is important.
It given you access to the google protobufs wrappers source code here. These are not objects that have methods that allow you to test for presence.
Grpc generated code in java gives you methods such as .hasEmail() which returns true if the value is present. The getter on an unset value will still return you the zero value. I think the golang version uses pointers that you can test for nil instead of an explicit hasX() method.
More info / discussion in this github issue
I am trying to log an error to Stackdriver Error Reporting in Go. On the first page of the Error Reporting, there is stated "Reporting errors from your application can be achieved by logging application errors to Google Stackdriver Logging or..." (https://cloud.google.com/error-reporting/docs/). How do I do that with the Go client libraries?
The Entry provided by the logging library is constructed like this:
github.com/GoogleCloudPlatform/.../logging.go#L412
type Entry struct {
Timestamp time.Time
Severity Severity
Payload interface{}
Labels map[string]string
InsertID string
HTTPRequest *HTTPRequest
Operation *logpb.LogEntryOperation
LogName string
Resource *mrpb.MonitoredResource
}
Do I need to marshal this JSON structure into the Payload? Or can I insert the stacktrace as string?
There is a dedicated Go package that should help you achieve this: import "cloud.google.com/go/errorreporting"
You can configure it to report errors via Stackdriver Logging, and it will take care of the sending the correct log structure.
From the docs:
// Payload must be either a string or something that
// marshals via the encoding/json package to a JSON object
// (and not any other type of JSON value).
Looks like inserting stacktrace as a string is the way to go.
How can I pass a meaningful error message back to client? I can implement the ExceptionMapper to produce meaningful error message.
But how can I have writer to pass the message back to client?
I haven't used jersey personally, but my general approach here would be to simply include the result-state (including error message) in the returned message, i.e.
message GetCustomerResult {
optional string errorMessage = 1;
optional Customer customer = 2;
}
or similar, such that all your messages have a consistent way to report failure. If your error state is more than just a string, you could declare a message for that, and include it on all results, so you can just pass that to any shared error-handling code.
You can throw WebApplicationException which takes Response in a constructor and gets mapped by Jersey automatically to that response. You can put the meaningful message into the response body.