i am planning to call following message using grpcurl.
How to send a .google.protobuf.BytesValue via grpcurl?
message TestRequest {
.google.protobuf.BytesValue TestJson = 1;
string Temp = 2;
}
BytesValues is a message (wrapper) for bytes.
Try Base64 encoding the bytes field and passing it that way.
Protobuf's JSON encoding of bytes is a Base64 string.
Related
I am trying to use the protobuf-c library to send messages in between sockets in C.
Steps to reproduce
Create a .proto file with the messages:
syntax = "proto3";
message MyMsg {}
Compile it with protoc c_out=. rpc.proto I get my rpc.pb-c.h/c files ready to use.
Include the header, try to pack an empty message.
int main () {
MyMsg msg = MY_MSG__INIT;
size_t buflen = my_msg__get_packed_size(&msg);
printf("buflen: %zu\n", buflen);
}
prints 0.
Further details
Even if I ignore this and I allocate a larger buffer for packing the msg, the my_msg__pack(buf), function will not write any bytes into the buffer.
If I define a non-empty message, I do get some bytes into the buffer. Still, it seems to me the generated code only packs the body of the message, not the header, indicating the type of the message.
Question
Is there some other API I need to use in order to pack the whole message?
Since your message is defined as an empty one, e.g. no fields, the length of the serialized message is, of course, 0. This is true, even if you define a message with some fields, but NOT set any field of the message.
message MyMsg {
int32 i = 1; // if you don't call MyMsg::set_i(int), the length of the serialized message is still 0.
}
not the header, indicating the type of the message.
Protobuf does NOT encode meta data, e.g. message type, into the serialized value. Instead, the protobuf definition file is the meta data. That's why you must have a protobuf file, if you want to parse the serialized data.
Also if you have 2 messages with exactly the same definition, you can even parse the serialized data of one message to build another message.
message A {
}
message B {
// have the same fields definition with A
}
A a;
serialized_data = serialize(a);
B b;
parse(b, serialized_data);
I have a message and I would like to package it into an any repeated google proto type::
Is there a way to encode an repeated any message type?
Can I even use repeated tag with google.protobuf.any?
message Onesensor{
string name=1
string type=2
int32_t reading=3
}
/** Any Message **/
message RepeatedAny{
repeated google.protobuf.any sensors = 1;
}
I am looking for an example, currently using nanopb to encode.
Sure, it is just a regular message.
https://github.com/nanopb/nanopb/tree/master/tests/any_type shows how to encode a single Any message, encoding many is like encoding any array. You'll have a choice between allocating statically, allocating dynamically or using callbacks. Or you can just encode a single subfield at a time into output stream, because concatenating encoded concatenates arrays in protobuf format.
I think I found my issue, I cannot use(repeated tag on the google.protobuf.any, as I would like to append the RepeatedAny messages in the final binary):
message Onesensor{
string name=1
string type=2
int32_t reading=3
}
message RepeatedAny{
repeated google.protobuf.any sensors = 1;
}
Instead I should use something like this:
message Onesensor{
string name=1
string type=2
int32_t reading=3
}
message SensorAny{
google.protobuf.any sensor = 1;
}
message RepeatedAny{
repeated SensorAny sensors = 1;
}
I should not use the repeated tag on the google.protobuf.any, I should be using it on a message that contains the google.protobuf.any instead, so that the protobinary can contain the format (sensors1), (sensors2).....(sensorsN), one or more SensorAny messages.
Below is the sample code, if someone finds this question in the future for nanopb:
/* First encode the SensorAny message by setting the value of the first field,
The first field of this message is of type google.protobuf.any, so it should have
1. sensor.type_url
2. sensor.value
*/
void* pBufAny = calloc(1, sBufSize);
pb_ostream_t ostream_any = pb_ostream_from_buffer(pBufAny, sBufSize);
SensorAny SensorAnyProto = SensorAny_init_default;
SensorAnyProto.has_message = true;
SensorAnyProto.sensor.type_url.arg = "type.googleapis.com/SensorAny.proto";
SensorAnyProto.sensor.type_url.funcs.encode = Proto_encode_string;
ProtoEncodeBufferInfo_t BufInfo = {
.Buffer = pBuf, /* I have already filled and encoded Onesensor message previously as pBuf */
.BufferSize = ostream.bytes_written,
};
SensorAnyProto.sensor.value.funcs.encode = Proto_encode_buffer;
SensorAnyProto.sensor.value.arg = &BufInfo;
pb_encode(&ostream_any, SensorAny_fields, &SensorAnyProto);
free(pBuf);
// Now Use the above encoded Any message buffer pBufAny to set the first repeated field in RepeatedAny
RepeatedAny SensorAnyRepeated = RepeatedAny_init_default;
ProtoEncodeBufferInfo_t AnyBufInfo = {
.Buffer = pBufAny,
.BufferSize = ostream_any.bytes_written,
};
AnyRepeated.sensors.arg=&AnyBufInfo;
AnyRepeated.sensors.funcs.encode = Proto_encode_buffer;
void* pBufAnyRepeated = calloc(1, sBufSize);
pb_ostream_t ostream_repeated = pb_ostream_from_buffer(pBufAnyRepeated, sBufSize);
!pb_encode(&ostream_repeated, RepeatedAny_fields, &AnyRepeated);
free(pBufAny);
I have a code snippet that use protocol buffer but in the writeTo method string values is not encoded
After writing a output stream i can read the string values
business::Company company;
company.set_name("Example Ltd.");
company.set_url("http://www.example.com");
// 1st employee
{
business::Employee *employee = company.add_employee();
employee->set_first_name("John");
employee->set_last_name("Doe");
employee->set_email("john.doe#example.com");
}
fstream output("company.bin", ios::out | ios::trunc | ios::binary);
company.SerializeToOstream(&output);
File my_protocol.proto:
Message MyProtocol {
required int32 protocolId = 1;
required **unknownType** protocolBody = 2;
}
I want to parse protocolBody according to protocolId.
For example,
protocolId = 10001 represents a login event, then the protocolBody should be a login message and can be parsed by login.proto.
I don't know which type should be set on the unknownType.
Protocol buffers parses the whole message in the one go.
You could have a message
Message MyProtocolGetId {
required int32 protocolId = 1;
}
Parse that then reparse with the correct message id. Alternatively the ProtocolBody could be stored as bytes and passed seperately
You can fake it, but it will be awkward. You have to use optional fields, and indicate which one if present using a flag. Make sure to double check that the field is set in your receiver code:
Message OneOfManyTypes {
required int32 kind = 1;
optional MessageType1 ty1 = 2;
optional MessageType2 ty2 = 3;
...
}
I've been modifying the 'inspect' WFP example (bundled with the WinDDK) with the aim of being able to parse the payload of all incoming TCP packets (from a specified IP address) for certain strings. (I've already modified 'inspect' such that only TCP packets are caught by the filter)
So far my modifications have been on the 'TLInspectTransportClassify' classifyFn, as shown below. My aim is to have access to the payload of each TCP packet that is caught.
FWPS_STREAM_CALLOUT_IO_PACKET* ioPacket = (FWPS_STREAM_CALLOUT_IO_PACKET*)layerData;
FWPS_STREAM_DATA* streamData;
SIZE_T streamLength;
BYTE* stream = NULL;
SIZE_T bytesCopied = 0;
[...]
if(ioPacket == NULL) {
DbgPrint("ioPacket == NULL\n");
return STATUS_INSUFFICIENT_RESOURCES;
}
streamData = ioPacket->streamData;
if(!streamData) { // why is this always NULL? shouldn't our payload be here?
DbgPrint("streamData == NULL: no data\n");
classifyOut->actionType = FWP_ACTION_PERMIT;
classifyOut->rights &= ~FWPS_RIGHT_ACTION_WRITE;
goto Exit;
}
DbgPrint("tcp packet has some data\n");
streamLength = streamData->dataLength;
stream = ExAllocatePoolWithTag(NonPagedPool,
streamLength,
'yftN');
if (!stream)
return STATUS_INSUFFICIENT_RESOURCES;
RtlZeroMemory(stream,streamLength);
FwpsCopyStreamDataToBuffer0(
streamData,
stream,
streamLength,
&bytesCopied);
// should now have our tcp payload in 'stream' buffer(?)
DbgPrint("reached parsing code\n");
[...]
From my understanding, after declaring ioPacket as above, ioPacket->streamData should contain the packet's payload. However, ioPacket->streamData is ALWAYS NULL for me. How do I get the packet's payload? Am I doing something wrong.
Thanks in advance.
'TLInspectTransportClassify' is on TRANSPORT_LAYER where layerData should be casted into NET_BUFFER_LIST.
FWPS_STREAM_CALLOUT_IO_PACKET is for FWPM_LAYER_STREAM_V4/FWPM_LAYER_STREAM_V6
See MSDN classifyFn0.
http://msdn.microsoft.com/en-us/library/ff544890(VS.85).aspx
Management Filtering Layer Identifiers
http://msdn.microsoft.com/en-us/library/ff557101(VS.85).aspx