I am using protobuf and grpc as interface between a client and server.
The server is written in C and the client uses python to communicate to the server.
I have a message created in protobuf like below.
message value_obj {
uint32 code = 1;
uint32 value = 2;
}
message list_of_maps {
map<uint32, value_obj> mapObj1 = 1;
map<uint32, value_obj> mapObj2 = 2;
}
I tried creating objects in Python like below:
obj = list_of_maps()
mapObjToStore = value_obj()
mapObjToStore.code = 10
obj.mapObj1[1].CopyFrom(mapObjToStore)
When I try to receive the message in server, I get wrong values (huge numbers!).
Any help on this will be greatly appreciated.
You can try using python dictionary for that:
map1 = {}
obj1 = value_obj()
map1[1] = obj1
map2 = {}
listOfMaps = list_of_maps(mapObj1=map1, mapObj2=map2)
Related
I am facing with the problem that protobuf I defined enum but its value is int32
Now I want someway or somehow to change all the protobuf defined to string
Or any code-hack for doing it in gateway without changing the protobuf.
Enum defined
enum TimeUnit {
seconds = 0;
minutes = 1;
hours = 2;
days = 3;
months = 4;
}
message CacheDuration {
uint32 Value = 1;
TimeUnit Units = 2;
}
What i got from generated code now is
And it is the return value for front end to use. So they would see the value of Units = int32 like this:
The services communicate by generated struct protobuf.
I want to make it change to
"Units":"days"
Thanks
You can use String method in your go code:
generatedTimeUnitEnum.String() // output: days
I have a proto message of the following form defined:
message A {
message B {
message C {
optional string details = 1;
optional string time = 2;
}
repeated C c = 1;
}
repeated B b = 1;
}
and I want to write a java code to clear the field details from the object.
A a;
a.clearField(b.c.details);
NOTE that here b and c both are repeated fields.
Is there any way in which I can achieve this for Java protocol buffers?
Proto buffers are immutable, thus you can't edit A directly.
However you can achieve your goal with Proto builder. The code would look more-less like:
a = a.toBuilder()
.setB(
indexOfB,
a.getB(indexOfB).toBuilder()
.setC(
indexOfC,
a.getB(indexOfB).getC(indexOfC).toBuilder()
.clearDetails()
.build())
.build())
.build();
For protofile:
syntax = "proto3";
package messagepb;
import "github.com/gogo/protobuf/gogoproto/gogo.proto";
option (gogoproto.marshaler_all) = true;
option (gogoproto.sizer_all) = true;
option (gogoproto.unmarshaler_all) = true;
option (gogoproto.goproto_getters_all) = false;
service KV {
// Put puts the given key into the store.
// A put request increases the revision of the store,
// and generates one event in the event history.
rpc Put(PutRequest) returns (PutResponse) {}
}
message PutRequest {
bytes key = 1;
bytes value = 2;
}
message ResponseHeader {
repeated PutRequest l = 3;
}
I get the following proto struct:
type ResponseHeader struct {
L []*PutRequest `protobuf:"bytes,3,rep,name=l" json:"l,omitempty"`
}
But how do I get following protostruct:
type ResponseHeader struct {
L []PutRequest `protobuf:"bytes,3,rep,name=l" json:"l,omitempty"`
}
That is I want having data locality (and thus slices of contig data, not pointers to spread out stuff)
I needed to use: [(gogoproto.nullable) = false]
as in:
repeated PutRequest l = 3 [(gogoproto.nullable) = false];
And got:
L []PutRequest `protobuf:"bytes,3,rep,name=l" json:"l"`
because there is no extend in proto3, so I combine the base message with google.protobuf.Any type message, but it's binary length is too long
.proto file
message TradeMessage {
google.protobuf.Any message = 1;
string code = 2;
}
message Connect {
int32 seq = 1;
string appid = 2;
string clientid = 3;
string ver = 4;
}
...
.java file
TradeProtocol.Connect inner = TradeProtocol.Connect.newBuilder()
.setSeq(1)
.setAppid("test")
.build();
TradeProtocol.TradeMessage packet = TradeProtocol.TradeMessage.newBuilder()
.setMessage(Any.pack(inner))
.setCode(2)
.build();
service send packet to client, client can decode all message to base TradeMessage, the problem is the inner's length is 8 bytes, while packet's length is 56 bytes. the same function implement use proto2's extend just ten more bytes, so is there any way to implement extend function in proto3 or reduce the packet's length ? thanks
One alternative is to use oneof:
message Connect {
int32 seq = 1;
string appid = 2;
string clientid = 3;
string ver = 4;
}
message TradeMessage {
string code = 1;
oneof inner {
Connect inner_connect = 2;
SomeOtherMessage inner_other = 3;
...
}
}
The encoded size will still be larger than with extend, but only by 1-2 bytes.
Hi I am utilizing Protobuf for my personal project about neural networks.
Here is my Protobuf definitions:
syntax = "proto3";
package NGNET;
message InputLayer {
string name = 1;
uint32 size = 2;
}
message ComputeLayer {
string name = 1;
uint32 size = 2;
repeated LayerLink inputs = 3;
}
message LayerLink {
InputLayer il_input = 1;
ComputeLayer cl_input = 2;
uint32 output_size = 3;
repeated float weights = 4;
}
message NNET {
string name = 1;
repeated ComputeLayer outputs = 3;
}
The network is created like this:
ComputeLayer output1 = ComputeLayer(10, "output1");
ComputeLayer output2 = ComputeLayer(10, "output2");
ComputeLayer hidden = ComputeLayer(100, "hidden");
InputLayer input1 = InputLayer(784, "input1");
InputLayer input2 = InputLayer(784, "input2");
output1.link(&hidden);
output2.link(&hidden);
hidden.link(&input1);
hidden.link(&input2);
hidden.link(&extra);
The link functions are defined as:
void ComputeLayer::link(ComputeLayer* to_link) {
NGNET::LayerLink* link = new NGNET::LayerLink();
link->set_output_size(internal->size());
link->set_allocated_cl_input(to_link->getInternal());
internal->mutable_inputs()->AddAllocated(link);
}
void ComputeLayer::link(InputLayer* to_link) {
NGNET::LayerLink* link = new NGNET::LayerLink();
link->set_output_size(internal->size());
link->set_allocated_il_input(to_link->getInternal());
internal->mutable_inputs()->AddAllocated(link);
}
Note: The getInternal() function returns a NGNET::ComputeLayer or NGNET::InputLayer
Then the outputs are liked to a NNET with:
nnet->mutable_outputs()->AddAllocated(output1->getInternal());
nnet->mutable_outputs()->AddAllocated(output2->getInternal());
When nnet is deleted the program crashes with a segment fault.
I believe this is due to the hidden layer gets deleted twice. Is there any way I can safely free the memory that was allocated ?
Thanks.
The add_allocated_*() and set_allocated_*() methods take ownership of the pointer they are given. This means that you have to make sure that no other code will delete those pointers later, because the Protobuf implementation will delete them when the message is destroyed.
If you don't want Protobuf to take ownership of these objects, you should make copies instead:
link->mutable_il_input()->CopyFrom(*to_link->getInternal());
nnet->mutable_outputs()->Add()->CopyFrom(*output2->getInternal());
Generally, unless you are doing intense memory allocation optimizations, you probably never want to call the "allocated" protobuf accessors.