I get an error when I try to compile a proto file to convert to .java.
Could you point out what I'm missing ?
protoc --proto_path=src\main\resources\proto --java_out=src\main\java src\main\resources\proto\PayloadProtocol.proto
PayloadProtocol.proto:32:14: "DataContainer" is not defined.
PayloadProtocol.proto: warning: Import BackendCommunicationService.proto but not used.
Payload.proto
import "BackendCommunicationService.proto";
package com.fleetboard.tp.payload.protocol.protobuf;
option java_package = "com.fleetboard.tp.proto.protocol";
message TPMessage {
required int32 serviceId = 1; // telematic service (TS) id, who owns this message
required int32 functionId = 2; // function id refers to the Java class for the payload
optional uint64 requestId = 3; // Identifier to associate the request to a response
optional TPPayload payload = 4; // serialized representation of a TP message
optional uint64 durability = 5; // life time of message - used from backend
optional DataContainer dataPayload = 6;**
}
BackendCommunicationService.proto
package com.fleetboard.tp.backend.protobuf;
option java_package = "com.fleetboard.tp.proto.backend";
message DataContainer {
required DeviceApplication application = 1; // The container's recipient (MT) or sender (MO)
required string fileName = 2; // File name (no path), length up to 255
required uint64 fileTime = 3; // File time as ms since 1970-01-01 00:00 UTC
}
Fully qualify the name in the importing file:
com.fleetboard.tp.backend.protobuf.DataContainer
or
.com.fleetboard.tp.backend.protobuf.DataContainer
(the . ensures it starts at the root)
You could also try using just the disjoint part, but I don't know if it will work:
backend.protobuf.DataContainer
(since both have the com.fleetboard.tp. prefix)
Related
I am facing with the problem that protobuf I defined enum but its value is int32
Now I want someway or somehow to change all the protobuf defined to string
Or any code-hack for doing it in gateway without changing the protobuf.
Enum defined
enum TimeUnit {
seconds = 0;
minutes = 1;
hours = 2;
days = 3;
months = 4;
}
message CacheDuration {
uint32 Value = 1;
TimeUnit Units = 2;
}
What i got from generated code now is
And it is the return value for front end to use. So they would see the value of Units = int32 like this:
The services communicate by generated struct protobuf.
I want to make it change to
"Units":"days"
Thanks
You can use String method in your go code:
generatedTimeUnitEnum.String() // output: days
I need to use Protocol Buffers to serialize JSON messages received from the Google Drive drives:list method and write them to the BigQuery Storage Write API (GRPC). This is working for all field types except timestamp. I cannot for the life of me generate go classes that include timestamps. To begin, I'm following this document, although have also tried everything I can find online including here on stackoverflow to no avail.
On MacOS 12.6, protoc is installed from this zip to /usr/local/bin and the contents of include from the zip are installed to /usr/local/include.
This is the drives.proto file I need to create a class for:
syntax = "proto3";
option go_package = "./driveBuffers";
import "google/protobuf/timestamp.proto";
message Drive {
string id =1;
string name =2;
string colorRgb = 3;
string backgroundImageLink =4;
bool hidden = 5;
string orgUnitId = 6;
timestamp createdTime = 7;
message restrictions {
bool adminManagedRestrictions = 1;
bool domainUsersOnly = 2;
bool copyRequiresWriterPermission = 3;
bool driveMembersOnly = 4;
}
}
If I remove the field with type timestamp, the tool creates a file named ./driveBuffers/drives.pb.go. With the timestamp type, this error is thrown:
% protoc --go_out=. -I ./ -I /usr/local/include/ drives.proto
drives.proto:11:3: "timestamp" is not defined.
Thank you.
You should refer the type as google.protobuf.Timestamp. As example:
string orgUnitId = 6;
google.protobuf.Timestamp createdTime = 7;
In the following example:
try (ParquetWriter<Example> writer =
new ProtoParquetWriter<>(
new Path("file:/tmp/foo.parquet"),
Example.class,
SNAPPY,
DEFAULT_BLOCK_SIZE,
DEFAULT_PAGE_SIZE)) {
writer.write(
Example.newBuilder()
.setTs(System.currentTimeMillis())
.setTenantId("tenant")
.setSomeFlag(false)
.setSomeInt(1)
.setOtherInt(0)
.build());
}
}
And example .proto file:
syntax = "proto3";
package com.example;
message Example {
uint64 ts = 1;
string tenantId = 2;
bool someFlag = 3;
int32 someInt = 4;
int32 otherInt = 2;
}
The resulting parquet file won't have the fields someFlag and otherInt because they are false and 0 respectively.
Is there a way to make it write it anyway or should I handle this on the reader side?
In proto3, presence tracking was not enabled historically, and the only presence rule was around zero defaults. Fortunately this changed recently in new versions of protoc. The optional keyword can now be used in from of fields in proto3 to enable this. So: add optional, and any compliant implementation should do what you want. The defaults are still zero/false/etc, but if they are explicitly set: they are serialized.
syntax = "proto3";
package com.example;
message Example {
optional uint64 ts = 1;
optional string tenantId = 2;
optional bool someFlag = 3;
optional int32 someInt = 4;
optional int32 otherInt = 2; // [sic]
}
Also, the second 2 should be a 5
The problem:
I have 2 files in the root directory.
I use Makefile to generate Go code from .proto files.
But the language field in the Video struct is a value not a pointer to the value.
And the subtitles field in the Video struct is an array of values not an array of pointers to the value.
The question is:
How can I make protoc generate a pointer to the value?
video.pb.go
type Video struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
Title string `protobuf:"bytes,2,opt,name=title,proto3" json:"title,omitempty"`
// I need *language.ISO639_1 below
Languages language.ISO639_1 `protobuf:"varint,3,opt,name=languages,proto3,enum=language.ISO639_1" json:"languages,omitempty"`
// I need []*language.ISO639_1 below
Subtitles []language.ISO639_1 `protobuf:"varint,4,rep,packed,name=subtitles,proto3,enum=language.ISO639_1" json:"subtitles,omitempty"`
}
Makefile
gen:
# Video
protoc -I. --go_out=plugins=grpc,paths=source_relative:video video.proto
# Language
protoc -I. --go_out=plugins=grpc,paths=source_relative:language language.proto
language.proto
syntax = "proto3";
package language;
option go_package = "example.com/group/repo/language;language";
enum ISO639_1 {
UNKNOWN = 0;
zh = 1;
}
video.proto
syntax = "proto3";
package video;
import "language.proto";
option go_package = "example.com/group/repo/video;video";
message Video {
string id = 1;
string title = 2;
language.ISO639_1 language = 3;
repeated language.ISO639_1 subtitles = 4;
}
protoc version: libprotoc 3.11.4
Starting in proto3 version 3.12, the field presence feature is supported experimentally, meaning you can again use the optional keyword, similarly to proto2.
You can achieve this by passing a flag --experimental_allow_proto3_optional to protoc when generating the pb.go files (ensure you're running new enough version of protoc and protoc-gen-go to support this experimental feature).
So given this pseudo-.proto file:
enum ISO639_1 {
UNKNOWN = 0;
zh = 1;
}
message Video {
string id = 1;
string title = 2;
optional ISO639_1 language = 3;
repeated ISO639_1 subtitles = 4;
}
You should get generated struct with:
type Video struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Language *ISO639_1 `protobuf:"varint,3,opt,name=language,proto3,enum=grpctrace.ISO639_1,oneof" json:"language,omitempty"`
Subtitles []ISO639_1 `protobuf:"varint,4,rep,packed,name=subtitles,proto3,enum=grpctrace.ISO639_1" json:"subtitles,omitempty"`
}
While this solves your issue with the language, I'm not sure you'll be able to work around the repeated field, to get a pointer value. Maybe this thread might be helpful - https://stackoverflow.com/a/25637833/13183366
File my_protocol.proto:
Message MyProtocol {
required int32 protocolId = 1;
required **unknownType** protocolBody = 2;
}
I want to parse protocolBody according to protocolId.
For example,
protocolId = 10001 represents a login event, then the protocolBody should be a login message and can be parsed by login.proto.
I don't know which type should be set on the unknownType.
Protocol buffers parses the whole message in the one go.
You could have a message
Message MyProtocolGetId {
required int32 protocolId = 1;
}
Parse that then reparse with the correct message id. Alternatively the ProtocolBody could be stored as bytes and passed seperately
You can fake it, but it will be awkward. You have to use optional fields, and indicate which one if present using a flag. Make sure to double check that the field is set in your receiver code:
Message OneOfManyTypes {
required int32 kind = 1;
optional MessageType1 ty1 = 2;
optional MessageType2 ty2 = 3;
...
}