I want to serialize the below message partially i.e. I want to serialize first three properties id, name, companyName and don't want to serialize the age.
I am working Protocol Buffer Java API.
message Person{
required int32 id = 1;
required string name = 2;
optional string companyName = 3;
`optional int32 age = 3;`
}
can somebody help me on that?
Reetesh
You'll need to make a copy of the message, then remove age from the copy, then serialize:
person.toBuilder().clearAge().build().toByteArray()
Related
I recently started working with protobuf in my golang project. I created below simple protobuf file. I have three different endpoints.
GetLink takes CustomerRequest as an input parameter and returns back CustomerResponse
GetBulkLinks takes BulkCustomerRequest as an input parameter and returns back BulkCustomerResponse
StreaLinks takes StreamRequest as an input parameter and returns back CustomerResponse
I am wondering if there is any way we can improve below proto file because CustomerRequest and BulkCustomerRequest has mostly everything in common except resources field so there is a duplication. And same goes with StreamRequest input parameter as it only takes clientId as the input parameter. Is there anything in protocol buffer which can reuse stuff from another message type?
Is there any better or efficient way to organize below proto file which reuses message accordingly?
syntax = "proto3";
option go_package = "github.com/david/customerclient/gen/go/data/v1";
package data.v1;
service CustomerService {
rpc GetLink(CustomerRequest) returns (CustomerResponse) {};
rpc GetBulkLinks(BulkCustomerRequest) returns (BulkCustomerResponse) {};
rpc StreaLinks(StreamRequest) returns (CustomerResponse) {};
}
message CustomerRequest {
int32 clientId = 1;
string resources = 2;
bool isProcess = 3;
}
message BulkCustomerRequest {
int32 clientId = 1;
repeated string resources = 2;
bool isProcess = 3;
}
message StreamRequest {
int32 clientId = 1;
}
message CustomerResponse {
string value = 1;
string info = 2;
string baseInfo = 3;
string link = 4;
}
message BulkCustomerResponse {
map<string, CustomerResponse> customerResponse = 1;
}
Is there anything in protocol buffer which can reuse stuff from another message type?
Composition.
However keep in mind that request and response payloads may change over time. Even if it looks like they have something in common today, they may diverge tomorrow. After all they are used in different RPCs. Then excessive coupling achieves the opposite effect and becomes technical debt.
Since your schema has literally no more than three fields for each message, I would leave everything as is. Anyway, if you really must, you may consider the following:
Extract the GetLink and GetBulkLinks common fields in a separate message and compose with that:
message CustomerRequestParams {
int32 clientId = 1;
bool isProcess = 2;
}
message CustomerRequest {
CustomerRequestParams params = 1;
string resources = 2;
}
message BulkCustomerRequest {
CustomerRequestParams params = 1;
repeated string resources = 2;
}
StreamRequest looks just fine with repeating clientId. Arguably a stream is conceptually different from a unary RPC, so just keep them separated. And it's just one field.
I have the following proto3 message structure:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
google.protobuf.Timestamp createTime = 4;
}
I am using the "same" structure for some requests and responses on my app. What I want to achieve is to mark somehow (if possible) the createTime field as not required, in case we are talking about a request object, and required in case we are taking about a response object.
Is it possible to do this without creating a separate message ?
Thanks
To my knowledge, it's not possible and I'd discourage pursuing solutions other than defining distinct message types: one which includes the optional field and one which does not.
One way to solve this is to define a message that includes the mandatory fields and another than extends it:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
}
message SomeRequest {
BaseBuildContent content = 1;
}
message SomeResponse {
BaseBuildContent content = 1;
google.protobuf.Timestamp createTime = 2;
}
NOTE Protobuf style guide recommends message names be PascalCased and field names be snake_cased.
I am trying to figure out a decent approach toward dealing with multiple uses for a struct. Let me explain the scenario.
I have a struct that represents the Model in gorm. In the current implementation, I have validation bound to this struct so when a request hits the endpoint I would validate against the model's struct. This works fine for most cases. But then there are some instances where I want to have more control over the request and the response.
This is possible by introducing a few additional internal structs that will parse the request and response. And I can decouple the validation from the model into the request specific struct. I am trying to figure out what the best practice is around these patterns. Pretty sure a lot of peeps would have faced a similar situation.
// Transaction holds the transaction details.
type Transaction struct {
Program Program
ProgramID uuid.UUID
Type string
Value float64
Reference string
}
// TransactionRequest for the endpoint.
type TransactionRequest struct {
ProgramKey string `json:"program_key" validator:"required"`
Type string `json:"type" validator:"required,oneof=credit debit"`
Value float64 `json:"value" validator:"required,numeric"`
Reference string `json:"reference" validator:"required"`
}
Update:
I managed to find a balance by introducing additional tags for update requests, I wrote about how I achieved it here
I faced similar problem and for solving that, defined a method for validating and didn't use tags. I had to, because I follow DDD and we should validate in service layer not API layer.
here is a sample of my apporach:
type Station struct {
types.GormCol
CompanyID types.RowID `gorm:"not null;unique" json:"company_id,omitempty"`
CompanyName string `gorm:"not null;unique" json:"company_name,omitempty"`
NodeCode uint64 `json:"node_code,omitempty"`
NodeName string `json:"node_name,omitempty"`
Key string `gorm:"type:text" json:"key,omitempty"`
MachineID string `json:"machine_id,omitempty"`
Detail string `json:"detail,omitempty"`
Error error `sql:"-" json:"user_error,omitempty"`
Extra map[string]interface{} `sql:"-" json:"extra_station,omitempty"`
}
// Validate check the type of
func (p *Station) Validate(act action.Action) error {
fieldError := core.NewFieldError(term.Error_in_companys_form)
switch act {
case action.Save:
if p.CompanyName == "" {
fieldError.Add(term.V_is_required, "Company Name", "company_name")
}
if p.CompanyID == 0 {
fieldError.Add(term.V_is_required, "Company ID", "company_id")
}
}
if fieldError.HasError() {
return fieldError
}
return nil
}
file's address: https://github.com/syronz/sigma-mono/blob/master/model/station.model.go
We are planning to move grpc micro architecture. So we need a general adapter for converting protocol buffer Request object to existing POJO object vice versa.
In our current scenario, We have :-
message RequestIdentity {
Tenant tenant = 1;
string userToken = 2;
string ipAddress = 3;
}
message StudentRequest {
RequestIdentity requestIdentity = 1;
string id = 2;
string name = 3;
}
As above scenario, we have to convert into our generic ServiceRequest<Student> POJO object . Protocol buffer object has two part:
RequestIdentity - is responsible for creating ServiceRequest.
id and name - are payload of Student.
like wise convert different request using general adapter ? any idea
Right now I have an application where my iPhone app sends a request that is handled in .NET/C#, serialized to XML, and parsed on the app in objective-c. The current response class structure has one base class (BaseResponse) and many (over 25) subclasses for each type of request that corresponds to different things that need to be returned. Right now I'm looking to see if protobuf would be faster and easier than XML. From what I understand, a .proto file for this class structure is:
Message BaseResponse {
Required Int field1 = 1;
Optional SubResponse1 sub1= 2;
Optional SubResponse2 sub2 = 3;
Etc....
}
Message SubResponse1 {
....
}
Message SubResponse2 {
....
}
Etc for each sub response.
My question is: if I have over 25 of these optional elements (of which only 1 will be non null), does that completely wipe away the size and performance benefit of using protobuf? Does protobuf make sense for this application?
No, it does not impact the performance benefit - you'll just need to check which one is non-null in the objective-C code. Since protobuf only serializes the non-null values it will still be very efficient over the wire. The protobuf specification itself doesn't actually include inheritance, so you are right to say that you need to spoof it via encapsulation - but since you mention C#, note that what you have described (including how the data appears on the wire, i.e. it will be 100% comptible) can be done directly via inheritance if you use protobuf-net as the C# implementation - which should be possible with your existing model. For example:
[ProtoContract]
[ProtoInclude(2, typeof(SubResponse1))]
[ProtoInclude(3, typeof(SubResponse2))]
public class BaseResponse
{
// note Name and IsRequired here are optional - only
// included to match your example
[ProtoMember(1, IsRequired = true, Name="field1")]
public int Field1 { get; set; }
/*...*/
}
[ProtoContract]
public class SubResponse1 : BaseResponse
{/*...*/}
[ProtoContract]
public class SubResponse2 : BaseResponse
{/*...*/}
You can get the .proto via:
var proto = Serializer.GetProto<BaseResponse>();
Which gives:
message BaseResponse {
required int32 field1 = 1 [default = 0];
// the following represent sub-types; at most 1 should have a value
optional SubResponse1 SubResponse1 = 2;
optional SubResponse2 SubResponse2 = 3;
}
message SubResponse1 {
}
message SubResponse2 {
}