Given a message that looks like this,
message Event {
required int32 event_id = 1;
oneof EventType {
FooEvent foo_event = 2;
BarEvent bar_event = 3;
BazEvent baz_event = 4;
}
}
I want to define another map which uses the EventType oneof as a type. Precisely, I want to define something like this
message Sample {
map<string, Event.EventTypeCase> someMap = 1;
}
But, this is not working. I get the error that
PROTOC FAILED: "Event.EventTypeCase" is not defined.
I want to define another map which uses the EventType oneof as a type.
It isn't a type in the DSL, so: you can't. It is a conceptual grouping of fields in a specific message. The existence of Event.EventTypeCase is an implementation detail, not something that is not even mentioned in the DSL specification as far as I know (although protoc may or may not detect conflicts if you define your own enum or message with the same name)
Related
I recently started working with protobuf in my golang project. I created below simple protobuf file. I have three different endpoints.
GetLink takes CustomerRequest as an input parameter and returns back CustomerResponse
GetBulkLinks takes BulkCustomerRequest as an input parameter and returns back BulkCustomerResponse
StreaLinks takes StreamRequest as an input parameter and returns back CustomerResponse
I am wondering if there is any way we can improve below proto file because CustomerRequest and BulkCustomerRequest has mostly everything in common except resources field so there is a duplication. And same goes with StreamRequest input parameter as it only takes clientId as the input parameter. Is there anything in protocol buffer which can reuse stuff from another message type?
Is there any better or efficient way to organize below proto file which reuses message accordingly?
syntax = "proto3";
option go_package = "github.com/david/customerclient/gen/go/data/v1";
package data.v1;
service CustomerService {
rpc GetLink(CustomerRequest) returns (CustomerResponse) {};
rpc GetBulkLinks(BulkCustomerRequest) returns (BulkCustomerResponse) {};
rpc StreaLinks(StreamRequest) returns (CustomerResponse) {};
}
message CustomerRequest {
int32 clientId = 1;
string resources = 2;
bool isProcess = 3;
}
message BulkCustomerRequest {
int32 clientId = 1;
repeated string resources = 2;
bool isProcess = 3;
}
message StreamRequest {
int32 clientId = 1;
}
message CustomerResponse {
string value = 1;
string info = 2;
string baseInfo = 3;
string link = 4;
}
message BulkCustomerResponse {
map<string, CustomerResponse> customerResponse = 1;
}
Is there anything in protocol buffer which can reuse stuff from another message type?
Composition.
However keep in mind that request and response payloads may change over time. Even if it looks like they have something in common today, they may diverge tomorrow. After all they are used in different RPCs. Then excessive coupling achieves the opposite effect and becomes technical debt.
Since your schema has literally no more than three fields for each message, I would leave everything as is. Anyway, if you really must, you may consider the following:
Extract the GetLink and GetBulkLinks common fields in a separate message and compose with that:
message CustomerRequestParams {
int32 clientId = 1;
bool isProcess = 2;
}
message CustomerRequest {
CustomerRequestParams params = 1;
string resources = 2;
}
message BulkCustomerRequest {
CustomerRequestParams params = 1;
repeated string resources = 2;
}
StreamRequest looks just fine with repeating clientId. Arguably a stream is conceptually different from a unary RPC, so just keep them separated. And it's just one field.
Suppose I have a proto message like this:
message WorkflowParameters {
oneof parameters {
WorkflowAParams a = 1;
WorkflowBParams b = 2;
}
}
And I want to have another message where the type of workflow can be specified. Something like this:
message ListWorkflowsRequest {
// The type of workflows to fetch
WorkflowParameters.parameters workflow_type = 1;
}
The above doesn't work (it throws "WorkflowParameters.parameters" is not a type.) What's the recommended way of doing this?
It's not possible. oneof is only a thin syntatic sugar/behavior change, and has no effect on the actual schema. It affects the generated code's behavior, but not the serialized format. In the following example, these two messages are interchangeable and wire-compatible:
message WorkflowParameters {
oneof parameters {
WorkflowAParams a = 1;
WorkflowBParams b = 2;
}
}
message WorkflowParameters2 {
WorkflowAParams a = 1;
WorkflowBParams b = 2;
}
Now, if you just want to specify which part of a oneof will be set, you could theoretically use the generated code constants, and a simple int field:
message ListWorkflowsRequest {
// The field number of WorkflowParameters that should be filled.
int32 workflow_type = 1;
}
All language generators should have convenient enough constants created, like WorkflowParameters::A_FIELD_NUMBER for C++.
I have the following proto3 message structure:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
google.protobuf.Timestamp createTime = 4;
}
I am using the "same" structure for some requests and responses on my app. What I want to achieve is to mark somehow (if possible) the createTime field as not required, in case we are talking about a request object, and required in case we are taking about a response object.
Is it possible to do this without creating a separate message ?
Thanks
To my knowledge, it's not possible and I'd discourage pursuing solutions other than defining distinct message types: one which includes the optional field and one which does not.
One way to solve this is to define a message that includes the mandatory fields and another than extends it:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
}
message SomeRequest {
BaseBuildContent content = 1;
}
message SomeResponse {
BaseBuildContent content = 1;
google.protobuf.Timestamp createTime = 2;
}
NOTE Protobuf style guide recommends message names be PascalCased and field names be snake_cased.
How to send message type as object in ProtoBuf - Proto3 Syntax?
I want to transfer object instead of string or number.
Example
{
name: 'One',
date: 'date',
some: 'some',
...
...
}
syntax = "proto3";
package db;
service Proxy
{
rpc myFunction(Request) returns (Response);
}
message Request
{
string name = 1;
}
message Response
{
object keyvalue = 1;
}
Here, I am getting error
throw Error("no such Type or Enum '" + path + "' in " + this);
^
Error: no such Type or Enum 'object' in Type
--
Workaround
I can convert it to string in server side, and then I can JSON.parse() at client.
But I want to know, if there any better way to do it.
protocol-buffer does not support the object data type!
But you can emulate your data as hierarchically by using protocol buffer message type itself.
syntax = "proto3";
package db;
service Proxy
{
rpc myFunction(Request) returns (Response);
}
message Request
{
string name = 1;
}
message Response
{
message obj {
string key1 = 1;
string key2 = 2
}
obj keyvalue = 1; // Here you have created your own type obj.
}
In the above example, you can see that the Response message now has "keyvalue" field of type obj(which is a custom type you have just built).
Now you will pass Object in a callback from the server instead of primitive type.
callback(null, { keyvalue: { key1: "value1", key2: "value2" } });
Let's say if keys are unknown to you but key/value pair data type is same and known to you then, in this case, you can use
map<type, type>
message Response
{
map<string, string> keyvalue = 1;
}
callback(null, { keyvalue: { "key1": "value1", "key5": "value2" } });
References:-
https://developers.google.com/protocol-buffers/docs/proto3#other
https://developers.google.com/protocol-buffers/docs/proto#scalar
https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Struct
Depending on what you're trying to accomplish, you can likely use one of three types.
The bytes type represents an arbitrary string of bytes. You can choose to encode your types in anyway you feel appropriate.
The any type represents an arbitrary protobuf message, allowing the client or server to encode non-predefined messages. This is a string of bytes, with a unique identifier per message type and language implementations will have ways to unpack the message.
You mention using Javascript. You could likely define a Javascript object message using a combination of oneof for the basic types and map for object types.
If all of the keys and values you want to send have the same type, you can use a map type. Your example shows string keys and string values, so that could use the type map<string, string>.
If you want to send arbitrary object data with similar structure to JSON objects, you can use the google.protobuf.Struct message type.
Right now I have an application where my iPhone app sends a request that is handled in .NET/C#, serialized to XML, and parsed on the app in objective-c. The current response class structure has one base class (BaseResponse) and many (over 25) subclasses for each type of request that corresponds to different things that need to be returned. Right now I'm looking to see if protobuf would be faster and easier than XML. From what I understand, a .proto file for this class structure is:
Message BaseResponse {
Required Int field1 = 1;
Optional SubResponse1 sub1= 2;
Optional SubResponse2 sub2 = 3;
Etc....
}
Message SubResponse1 {
....
}
Message SubResponse2 {
....
}
Etc for each sub response.
My question is: if I have over 25 of these optional elements (of which only 1 will be non null), does that completely wipe away the size and performance benefit of using protobuf? Does protobuf make sense for this application?
No, it does not impact the performance benefit - you'll just need to check which one is non-null in the objective-C code. Since protobuf only serializes the non-null values it will still be very efficient over the wire. The protobuf specification itself doesn't actually include inheritance, so you are right to say that you need to spoof it via encapsulation - but since you mention C#, note that what you have described (including how the data appears on the wire, i.e. it will be 100% comptible) can be done directly via inheritance if you use protobuf-net as the C# implementation - which should be possible with your existing model. For example:
[ProtoContract]
[ProtoInclude(2, typeof(SubResponse1))]
[ProtoInclude(3, typeof(SubResponse2))]
public class BaseResponse
{
// note Name and IsRequired here are optional - only
// included to match your example
[ProtoMember(1, IsRequired = true, Name="field1")]
public int Field1 { get; set; }
/*...*/
}
[ProtoContract]
public class SubResponse1 : BaseResponse
{/*...*/}
[ProtoContract]
public class SubResponse2 : BaseResponse
{/*...*/}
You can get the .proto via:
var proto = Serializer.GetProto<BaseResponse>();
Which gives:
message BaseResponse {
required int32 field1 = 1 [default = 0];
// the following represent sub-types; at most 1 should have a value
optional SubResponse1 SubResponse1 = 2;
optional SubResponse2 SubResponse2 = 3;
}
message SubResponse1 {
}
message SubResponse2 {
}