In my proto file, I want to define a map as a custom option, tried a few things but none is working.
my metadata proto file:
syntax = "proto2";
import "google/protobuf/descriptor.proto";
package com.util;
option java_package = "com.util";
message MyMeta {
optional bool needValidation = 1;
map<string, string> fileMap = 2;
}
extend google.protobuf.FieldOptions {
optional MyMeta meta = 80412;
}
my proto file
syntax = "proto3";
package com.test;
import "util/meta.proto";
import "google/protobuf/timestamp.proto";
message MyMeta {
int32 id = 1 [(com.util.meta).needValidation = false, /*looking for a way to set a map (com.meta).tableOptions = {"t,raw_orders"}]*/;
}
Is this possible?
This works:
message MyMeta {
int32 id = 1 [
(com.util.meta) = {
needValidation: false,
fileMap: [
{
key: "t",
value: "raw_orders"
}
];
}];
}
The protobuf map is syntactic sugar for a repeated field of map entry messages with key and value fields. 1
In your case, your meta message
message MyMeta {
optional bool needValidation = 1;
map<string, string> fileMap = 2;
}
is equivalent to
message MyMeta {
optional bool needValidation = 1;
message FileMapEntry {
string key = 1;
string value = 2;
}
repeated FileMapEntry fileMap = 2;
}
Of course, it would be much nicer if there was a more obvious way to specify a map value in an option.
Seems it is not possible. To make my logic, I created a string like "StoreOrders:raw_orders_test,OrderItems:raw_order_items_test
message MyMeta {
int32 id = 1 [(com.util.meta).needValidation = false, (com.meta).tableOptions = "TableA:valueA,TableB:valueB;
}
and in my jav code and splitting the string to create a hash map.
Related
I am trying to create a protocol buffer message with fields that are either a message or one of a choice of some constant (like an enum). Something that is logically equivalent to:
message Error {
oneof error_type {
EMPTY_METHOD_NAME = 0
ExecutionError execution_error = 1;
}
message ExecutionError {
string value = 1;
}
}
Essentially, I would like a field that can represent either an error type that is just a name with no fields, or an error type that has fields. How would I do this with protobuf3?
See Enumerations in the Language Guide (proto3)
message Error {
oneof error_type {
Foo foo = 1;
ExecutionError execution_error = 2;
}
enum Foo {
X = 0;
Y = 1;
Z = 2;
}
message ExecutionError {
string value = 1;
}
}
I use nanopb and a .proto file that looks like this (simplified):
syntax = "proto3";
message Type1
{
bool a = 1;
}
message Type2
{
int32 b = 1;
}
message Message
{
int32 id = 1;
oneof oneMessage {
Type1 messageType1 = 2;
Type2 messageType2 = 3;
}
}
I now need to split the oneMessage to a different .proto file and import it. I can do something like:
file a.proto:
syntax = "proto3";
import "b.proto"
message Message
{
int32 id = 1;
oneofTypeMessage aMessage = 2;
}
file b.proto
syntax = "proto3";
message Type1
{
bool a = 1;
}
message Type2
{
int32 b = 1;
}
message oneofTypeMessage
{
oneof oneMessage {
Type1 messageType1 = 2;
Type2 messageType2 = 3;
}
}
But this means I can't get to my message types with message.messageType1, triggering lots of changes in the existing source code. How can I split the oneof part to another file without changing the way I access to it?
I've tried declaring Message in both files but it is not permitted.
No. oneMessage is not a message type, and to import something from another file, it must be a message type. Adding a new oneofTypeMessage would add an additional node in the graph, and would not be data-compatible with the original data.
You can move Type1 and Type2, but the oneof must stay as-was; so: this is valid:
file a.proto:
syntax = "proto3";
import "b.proto"
message Message
{
int32 id = 1;
oneof oneMessage {
Type1 messageType1 = 2;
Type2 messageType2 = 3;
}
}
file b.proto:
syntax = "proto3";
message Type1
{
bool a = 1;
}
message Type2
{
int32 b = 1;
}
I have a method that I would like defined called FindAll which requires no parameters. ProtoC is complaining.
Expected type name.
This is for line:
rpc findAll () returns (BenchmarksList);
syntax = "proto3";
package helloWorldGRPC;
service HelloWorldGRPCService {
rpc findById (BenchmarksById) returns (Benchmarks);
rpc findAll () returns (BenchmarksList);
}
message BenchmarksById {
string id = 1;
}
message BenchmarksList {
repeated Benchmarks benchmarks = 1;
}
message Benchmarks {
string trans_id = 1;
string protocol = 2;
string database = 3;
string updated_at = 4;
string created_at = 5;
repeated Action actions = 6;
}
message Action {
string trans_id = 1;
int32 payload_length = 2;
string payload = 3;
string status = 4;
string updated_at = 5;
string created_at = 6;
}
The preferred way is to pass Empty - as tooling may recognize and optimize for that scenario. But in reality, there is nothing "special" about that type and any message - empty or otherwise - will suffice.
There's a defined .proto file. syntax: proto3.
syntax = "proto3";
package proto;
import "google/protobuf/descriptor.proto";
extend google.protobuf.ServiceOptions {
string api_prefix = 50001;
}
extend google.protobuf.MethodOptions {
string api_url = 50002;
string api_method = 50003;
}
option go_package = "proto;proto";
service Foo {
option (api_prefix) = "/math";
rpc Add(MathRequest) returns (MathResponse) {
option (api_url) = "/:number1/:number2";
option (api_method) = "PUT";
}
}
message MathRequest {
string number1 = 1;
string number2 = 2;
}
message MathResponse {
string number = 1;
}
And there's a .pb.go file generated by protoc --go_out=. *.proto
How can I get api_url option value in services, and api_url option value in RPC method?
I got a solution myself.
var fd protoreflect.FileDescriptor
fd = pb.File_math_proto
s := fd.Services().Get(0)
m := s.Methods().Get(0).Options()
n := proto.GetExtension(m, pb.E_Http)
e := n.(*pb.HttpRule)
I would like to know how to transform a Protobuf Any Type to the original Protobuf message type and vice versa. In Java from Message to Any is easy:
Any.Builder anyBuilder = Any.newBuilder().mergeFrom(protoMess.build());
But how can I parse that Any back to the originial message (e.g. to the type of "protoMess")? I could probably parse everything on a stream just to read it back in, but that's not what I want. I want to have some transformation like this:
ProtoMess.MessData.Builder protoMessBuilder = (ProtoMess.MessData.Builder) transformToMessageBuilder(anyBuilder)
How can I achieve that? Is it already implemented for Java? The Protobuf Language Guide says there were pack and unpack methods, but there are none in Java.
Thank you in Advance :)
The answer might be a bit late but maybe this still helps someone.
In the current version of Protocol Buffers 3 pack and unpack are available in Java.
In your example packing can be done like:
Any anyMessage = Any.pack(protoMess.build()));
And unpacking like:
ProtoMess protoMess = anyMessage.unpack(ProtoMess.class);
Here is also a full example for handling Protocol Buffers messages with nested Any messages:
ProtocolBuffers Files
A simple Protocol Buffers file with a nested Any message could look like:
syntax = "proto3";
import "google/protobuf/any.proto";
message ParentMessage {
string text = 1;
google.protobuf.Any childMessage = 2;
}
A possible nested message could then be:
syntax = "proto3";
message ChildMessage {
string text = 1;
}
Packing
To build the full message the following function can be used:
public ParentMessage createMessage() {
// Create child message
ChildMessage.Builder childMessageBuilder = ChildMessage.newBuilder();
childMessageBuilder.setText("Child Text");
// Create parent message
ParentMessage.Builder parentMessageBuilder = ParentMessage.newBuilder();
parentMessageBuilder.setText("Parent Text");
parentMessageBuilder.setChildMessage(Any.pack(childMessageBuilder.build()));
// Return message
return parentMessageBuilder.build();
}
Unpacking
To read the child message from the parent message the following function can be used:
public ChildMessage readChildMessage(ParentMessage parentMessage) {
try {
return parentMessage.getChildMessage().unpack(ChildMessage.class);
} catch (InvalidProtocolBufferException e) {
e.printStackTrace();
return null;
}
}
EDIT:
If your packed messages can have different Types, you can read out the typeUrl and use reflection to unpack the message. Assuming you have the child messages ChildMessage1 and ChildMessage2 you can do the following:
#SuppressWarnings("unchecked")
public Message readChildMessage(ParentMessage parentMessage) {
try {
Any childMessage = parentMessage.getChildMessage();
String clazzName = childMessage.getTypeUrl().split("/")[1];
String clazzPackage = String.format("package.%s", clazzName);
Class<Message> clazz = (Class<Message>) Class.forName(clazzPackage);
return childMessage.unpack(clazz);
} catch (ClassNotFoundException | InvalidProtocolBufferException e) {
e.printStackTrace();
return null;
}
}
For further processing, you could determine the type of the message with instanceof, which is not very efficient. If you want to get a message of a certain type, you should compare the typeUrl directly:
public ChildMessage1 readChildMessage(ParentMessage parentMessage) {
try {
Any childMessage = parentMessage.getChildMessage();
String clazzName = childMessage.getTypeUrl().split("/")[1];
if (clazzName.equals("ChildMessage1")) {
return childMessage.unpack("ChildMessage1.class");
}
return null
} catch (InvalidProtocolBufferException e) {
e.printStackTrace();
return null;
}
}
Just to add information in case someone has the same problem ... Currently to unpack you have to do (c# .netcore 3.1 Google.Protobuf 3.11.4)
Foo myobject = anyMessage.Unpack<Foo>();
I know this question is very old, but it still came up when I was looking for the answer. Using #sundance answer I had to answer do this a little differently. The problem being that the actual message was a subclass of the actual class. So it required a $.
for(Any x : in.getDetailsList()){
try{
String clazzName = x.getTypeUrl().split("/")[1];
String[] split_name = clazzName.split("\\.");
String nameClass = String.join(".", Arrays.copyOfRange(split_name, 0, split_name.length - 1)) + "$" + split_name[split_name.length-1];
Class<Message> clazz = (Class<Message>) Class.forName(nameClass);
System.out.println(x.unpack(clazz));
} catch (Exception e){
e.printStackTrace();
}
}
With this being the definition of my proto messages
syntax = "proto3";
package cb_grpc.msg.Main;
service QueryService {
rpc anyService (AnyID) returns (QueryResponse) {}
}
enum Buckets {
main = 0;
txn = 1;
hxn = 2;
}
message QueryResponse{
string content = 1;
string code = 2;
}
message AnyID {
Buckets bucket = 1;
string docID = 2;
repeated google.protobuf.Any details = 3;
}
and
syntax = "proto3";
package org.querc.cb_grpc.msg.database;
option java_package = "org.querc.cb_grpc.msg";
option java_outer_classname = "database";
message TxnLog {
string doc_id = 1;
repeated string changes = 2;
}