Suppose I have a proto message like this:
message WorkflowParameters {
oneof parameters {
WorkflowAParams a = 1;
WorkflowBParams b = 2;
}
}
And I want to have another message where the type of workflow can be specified. Something like this:
message ListWorkflowsRequest {
// The type of workflows to fetch
WorkflowParameters.parameters workflow_type = 1;
}
The above doesn't work (it throws "WorkflowParameters.parameters" is not a type.) What's the recommended way of doing this?
It's not possible. oneof is only a thin syntatic sugar/behavior change, and has no effect on the actual schema. It affects the generated code's behavior, but not the serialized format. In the following example, these two messages are interchangeable and wire-compatible:
message WorkflowParameters {
oneof parameters {
WorkflowAParams a = 1;
WorkflowBParams b = 2;
}
}
message WorkflowParameters2 {
WorkflowAParams a = 1;
WorkflowBParams b = 2;
}
Now, if you just want to specify which part of a oneof will be set, you could theoretically use the generated code constants, and a simple int field:
message ListWorkflowsRequest {
// The field number of WorkflowParameters that should be filled.
int32 workflow_type = 1;
}
All language generators should have convenient enough constants created, like WorkflowParameters::A_FIELD_NUMBER for C++.
Related
I recently started working with protobuf in my golang project. I created below simple protobuf file. I have three different endpoints.
GetLink takes CustomerRequest as an input parameter and returns back CustomerResponse
GetBulkLinks takes BulkCustomerRequest as an input parameter and returns back BulkCustomerResponse
StreaLinks takes StreamRequest as an input parameter and returns back CustomerResponse
I am wondering if there is any way we can improve below proto file because CustomerRequest and BulkCustomerRequest has mostly everything in common except resources field so there is a duplication. And same goes with StreamRequest input parameter as it only takes clientId as the input parameter. Is there anything in protocol buffer which can reuse stuff from another message type?
Is there any better or efficient way to organize below proto file which reuses message accordingly?
syntax = "proto3";
option go_package = "github.com/david/customerclient/gen/go/data/v1";
package data.v1;
service CustomerService {
rpc GetLink(CustomerRequest) returns (CustomerResponse) {};
rpc GetBulkLinks(BulkCustomerRequest) returns (BulkCustomerResponse) {};
rpc StreaLinks(StreamRequest) returns (CustomerResponse) {};
}
message CustomerRequest {
int32 clientId = 1;
string resources = 2;
bool isProcess = 3;
}
message BulkCustomerRequest {
int32 clientId = 1;
repeated string resources = 2;
bool isProcess = 3;
}
message StreamRequest {
int32 clientId = 1;
}
message CustomerResponse {
string value = 1;
string info = 2;
string baseInfo = 3;
string link = 4;
}
message BulkCustomerResponse {
map<string, CustomerResponse> customerResponse = 1;
}
Is there anything in protocol buffer which can reuse stuff from another message type?
Composition.
However keep in mind that request and response payloads may change over time. Even if it looks like they have something in common today, they may diverge tomorrow. After all they are used in different RPCs. Then excessive coupling achieves the opposite effect and becomes technical debt.
Since your schema has literally no more than three fields for each message, I would leave everything as is. Anyway, if you really must, you may consider the following:
Extract the GetLink and GetBulkLinks common fields in a separate message and compose with that:
message CustomerRequestParams {
int32 clientId = 1;
bool isProcess = 2;
}
message CustomerRequest {
CustomerRequestParams params = 1;
string resources = 2;
}
message BulkCustomerRequest {
CustomerRequestParams params = 1;
repeated string resources = 2;
}
StreamRequest looks just fine with repeating clientId. Arguably a stream is conceptually different from a unary RPC, so just keep them separated. And it's just one field.
I have the following proto3 message structure:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
google.protobuf.Timestamp createTime = 4;
}
I am using the "same" structure for some requests and responses on my app. What I want to achieve is to mark somehow (if possible) the createTime field as not required, in case we are talking about a request object, and required in case we are taking about a response object.
Is it possible to do this without creating a separate message ?
Thanks
To my knowledge, it's not possible and I'd discourage pursuing solutions other than defining distinct message types: one which includes the optional field and one which does not.
One way to solve this is to define a message that includes the mandatory fields and another than extends it:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
}
message SomeRequest {
BaseBuildContent content = 1;
}
message SomeResponse {
BaseBuildContent content = 1;
google.protobuf.Timestamp createTime = 2;
}
NOTE Protobuf style guide recommends message names be PascalCased and field names be snake_cased.
I have a hash of key values pairs.
I do not have the keys, nor the values.
I can assume most will be there, but sometimes keys are removed and others are added. Is there any way for me to have a message with variable keys?
{
"knownkey1": "value",
"knownkey2": {
"unknown-key1": "value",
"unknown-key2": "value"
}
}
or is the best ways to just serialize it using json stringify in the message? i would think that would defeat the whole purpose of using grpc.
message Rate {
string ticker = 1;
string value = 2;
}
message GetAllResponse {
string lastUpdated = 1;
repeated Rate payload = 2;
}
Looks like you can just use the maps type as outlined here:
https://developers.google.com/protocol-buffers/docs/proto3#maps
Given a message that looks like this,
message Event {
required int32 event_id = 1;
oneof EventType {
FooEvent foo_event = 2;
BarEvent bar_event = 3;
BazEvent baz_event = 4;
}
}
I want to define another map which uses the EventType oneof as a type. Precisely, I want to define something like this
message Sample {
map<string, Event.EventTypeCase> someMap = 1;
}
But, this is not working. I get the error that
PROTOC FAILED: "Event.EventTypeCase" is not defined.
I want to define another map which uses the EventType oneof as a type.
It isn't a type in the DSL, so: you can't. It is a conceptual grouping of fields in a specific message. The existence of Event.EventTypeCase is an implementation detail, not something that is not even mentioned in the DSL specification as far as I know (although protoc may or may not detect conflicts if you define your own enum or message with the same name)
Right now I have an application where my iPhone app sends a request that is handled in .NET/C#, serialized to XML, and parsed on the app in objective-c. The current response class structure has one base class (BaseResponse) and many (over 25) subclasses for each type of request that corresponds to different things that need to be returned. Right now I'm looking to see if protobuf would be faster and easier than XML. From what I understand, a .proto file for this class structure is:
Message BaseResponse {
Required Int field1 = 1;
Optional SubResponse1 sub1= 2;
Optional SubResponse2 sub2 = 3;
Etc....
}
Message SubResponse1 {
....
}
Message SubResponse2 {
....
}
Etc for each sub response.
My question is: if I have over 25 of these optional elements (of which only 1 will be non null), does that completely wipe away the size and performance benefit of using protobuf? Does protobuf make sense for this application?
No, it does not impact the performance benefit - you'll just need to check which one is non-null in the objective-C code. Since protobuf only serializes the non-null values it will still be very efficient over the wire. The protobuf specification itself doesn't actually include inheritance, so you are right to say that you need to spoof it via encapsulation - but since you mention C#, note that what you have described (including how the data appears on the wire, i.e. it will be 100% comptible) can be done directly via inheritance if you use protobuf-net as the C# implementation - which should be possible with your existing model. For example:
[ProtoContract]
[ProtoInclude(2, typeof(SubResponse1))]
[ProtoInclude(3, typeof(SubResponse2))]
public class BaseResponse
{
// note Name and IsRequired here are optional - only
// included to match your example
[ProtoMember(1, IsRequired = true, Name="field1")]
public int Field1 { get; set; }
/*...*/
}
[ProtoContract]
public class SubResponse1 : BaseResponse
{/*...*/}
[ProtoContract]
public class SubResponse2 : BaseResponse
{/*...*/}
You can get the .proto via:
var proto = Serializer.GetProto<BaseResponse>();
Which gives:
message BaseResponse {
required int32 field1 = 1 [default = 0];
// the following represent sub-types; at most 1 should have a value
optional SubResponse1 SubResponse1 = 2;
optional SubResponse2 SubResponse2 = 3;
}
message SubResponse1 {
}
message SubResponse2 {
}