I have been playing around lately with GRPC and Protocol Buffers in order to get familiar with both frameworks in C++.
I wanted to experiment with the reflection functionality, so I have set up a very simple service where the (reflection-enabled) server exposes the following interface file:
syntax = "proto3";
package helloworld;
service Server {
rpc Add (AddRequest) returns (AddReply) {}
}
message AddRequest {
int32 arg1 = 1;
int32 arg2 = 2;
}
message AddReply {
int32 sum = 1;
}
On the client side I have visibility of the previous method thanks to the grpc::ProtoReflectionDescriptorDatabase. Therefore, I am able to create a message by means of a DynamicMessageFactory. However, I haven't been able to actually send the message to the server, nor, find any specific details in the documentation. Maybe it's too obvious and I'm completely lost...
Any hints will be deeply appreciated!
using namespace google::protobuf;
void demo()
{
std::shared_ptr<grpc::Channel> channel = grpc::CreateChannel("localhost:50051", grpc::InsecureChannelCredentials());
// Inspect exposed method
grpc::ProtoReflectionDescriptorDatabase reflection_database(channel);
std::vector<std::string> output;
reflection_database.GetServices(&output);
DescriptorPool reflection_database_pool(&reflection_database);
const ServiceDescriptor* service = reflection_database_pool.FindServiceByName(output[0]);
const MethodDescriptor* method = service->method(0);
// Create request message
const Descriptor* input_descriptor = method->input_type();
FileDescriptorProto input_proto;
input_descriptor->file()->CopyTo(&input_proto);
DescriptorPool pool;
const FileDescriptor* input_file_descriptor = pool.BuildFile(input_proto);
const Descriptor* input_message_descriptor = input_file_descriptor->FindMessageTypeByName(input_descriptor->name());
DynamicMessageFactory factory;
Message* request = factory.GetPrototype(input_message_descriptor)->New();
// Fill request message (sum 1 plus 2)
const Reflection* reflection = request->GetReflection();
const FieldDescriptor* field1 = input_descriptor->field(0);
reflection->SetInt32(request, field1, 1);
const FieldDescriptor* field2 = input_descriptor->field(1);
reflection->SetInt32(request, field2, 2);
// Create response message
const Descriptor* output_descriptor = method->output_type();
FileDescriptorProto output_proto;
output_descriptor->file()->CopyTo(&output_proto);
const FileDescriptor* output_file_descriptor = pool.BuildFile(output_proto);
const Descriptor* output_message_descriptor = output_file_descriptor->FindMessageTypeByName(output_descriptor->name());
Message* response = factory.GetPrototype(output_message_descriptor)->New();
// How to create a call...?
// ...is grpc::BlockingUnaryCall the way to proceed?
}
It's been a few years, but since you didn't get an answer, I'll make an attempt. You also didn't tag your question with a specific language, but it looks like you are using CPP. I can't provide a solution for CPP, but I can for JVM languages.
First of all, the following is taken from an open-source library I'm developing called okgrpc. It's the first of its kind attempt to create a dynamic gRPC client/CLI in Java.
Here are the general steps to make a call using DynamicMessage:
Get all the DescriptorProtos.FileDescriptorProto for the service you want to call using gRPC reflection.
Create indices for all types and methods in that service.
Find the Descriptors.MethodDescriptor corresponding to the method you want to call.
Convert your input to DynamicMessage. How to do this will depend on the input, of course. If JSON string, you can use the JsonFormat class.
Build an io.grpc.MethodDescriptor with method name, type (unary etc), request and response marshallers. You'll need to write your own DynamicMessage marshaller.
Use ClientCalls API to execute the RPC.
Obviously, the devil is in the details. If using Java, you can use my library, and let it be my problem. If using another language, good luck.
Related
Given the following code how can I convert the v8::Local<v8::Value> into a uint32_t. Or other types based on the Is* method?
v8::Local<v8::Value> value;
v8::Local<v8::Context> context = v8::Context::New(v8::Isolate::GetCurrent());
if(value->IsUint32()) {
v8::MaybeLocal<Int32> maybeLocal = value->Uint32Value(context);
uint32_t i = maybeLocal;
}
Your posted code doesn't work because value->Uint32Value(context) doesn't return a v8::MaybeLocal<Int32>. C++ types are your friend (just like TypeScript)!
You have two possibilities:
(1) You can use Value::Uint32Value(...) which returns a Maybe<uint32_t>. Since you already checked that value->IsUint32(), this conversion cannot fail, so you can extract the uint32_t wrapped in the Maybe using Maybe::ToChecked().
(2) You can use Value::ToUint32(...) which returns a MaybeLocal<Uint32>. Again, since you already checked that value->IsUint32(), that cannot fail, so you can get a Local<Uint32> via MaybeLocal::ToLocalChecked(), and then simply use -> syntax to call the wrapped Uint32's Value() method, which gives a uint32_t.
If you're only interested in the final uint32_t (and not in the intermediate Local<Uint32>, which you could pass back to JavaScript), then option (1) will be slightly more efficient.
Note that IsUint32() will say false for objects like {valueOf: () => 42; }. If you want to handle such objects, then attempt the conversion, and handle failures, e.g.:
Maybe<uint32_t> maybe_uint = value->Uint32Value(context);
if (maybe_uint.IsJust()) {
uint32_t i = maybe_uint.FromJust();
} else {
// Conversion failed. Maybe it threw an exception (use a `v8::TryCatch` to catch it), or maybe the object wasn't convertible to a uint32.
// Handle that somehow.
}
Also, note that most of these concepts are illustrated in V8's samples and API tests. Reading comments and implementations in the API headers themselves also provides a lot of insight.
Final note: you'll probably want to track the current context you're using, rather than creating a fresh context every time you need one.
Google docs propose the following model (https://cloud.google.com/apis/design/errors#error_model) for sending rich errors in gRPC but it seems that the error string is sent to the user every time. What I want to do is to send a code and then map it to a string when it reaches the client.
What I want to know is whatever the proto3 language supports writing data so that I would use it client-side, without defining a custom structure for the purposes of mapping error codes to error messages.
In your proto definition, define a simple enum with any extra error codes:
enum extraStatusCode {
UNKNOWN = 0; // not set/used
TOO_MANY_FOOS = 1;
NOT_ENOUGH_BARS = 2;
}
And include it as a top-level field in any returned message:
message User {
string uid = 1;
string email = 2;
// ...
extraStatusCode = 15;
}
if a message is sent with a non-zero extraStatusCode - then an edge case was encountered.
I would like to model messages for bidirectional streaming. In both directions I can expect different types of messages and I am unsure as to what the better practice would be. The two ideas as of now:
message MyMessage {
MessageType type = 1;
string payload = 2;
}
In this case I would have an enum that defines which type of message that is and a JSON payload that will be serialized and deserialized into models both client and sever side. The second approach is:
message MyMessage {
oneof type {
A typeA = 1;
B typeB = 2;
C typeC = 3;
}
}
In the second example a oneof is defined such that only one of the message types can be set. Both sides a switch must be made on each of the cases (A, B, C or None).
If you know all of the possible types ahead of time, using oneof would be the way to go here as you have described.
The major reason for using protocol buffers is the schema definition. With the schema definition, you get types, code generation, safe schema evolution, efficient encoding, etc. With the oneof approach, you will get these benefits for your nested payload.
I would not recommend using string payload since using a string for the actual payload removes many of the benefits of using protocol buffers. Also, even though you don't need a switch statement to deserialize the payload, you'll likely need a switch statement at some point to make use of the data (unless you're just forwarding the data on to some other system).
Alternative option
If you don't know the possible types ahead of time, you can use the Any type to embed an arbitrary protocol buffer message into your message.
import "google/protobuf/any.proto";
message MyMessage {
google.protobuf.Any payload = 1;
}
The Any message is basically your string payload approach, but using bytes instead of string.
message Any {
string type_url = 1;
bytes value = 2;
}
Using Any has the following advantages over string payload:
Encourages the use of protocol buffer to define the dynamic payload contents
Tooling in the protocol buffer library for each language for packing and unpacking protocol buffer messages into the Any type
For more information on Any, see:
https://developers.google.com/protocol-buffers/docs/proto3#any
https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto
I has a Message define like this:
message Command{
oneof type{
Point point = 1;
Rotate rotate = 2;
Move move = 3;
... //about 100 messages
}
}
Then protoc generate the SerializeWithCachedSizes function:
void Command::SerializeWithCachedSizes(
::google::protobuf::io::CodedOutputStream* output) const {
// ##protoc_insertion_point(serialize_start:coopshare.proto.Command)
::google::protobuf::uint32 cached_has_bits = 0;
(void) cached_has_bits;
// .coopshare.proto.Point point = 1;
if (has_point()) {
::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray(
1, *type_.point_, output);
}
// .coopshare.proto.Rotate rotate = 2;
if (has_rotate()) {
::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray(
2, *type_.rotate_, output);
}
// .coopshare.proto.Move move = 3;
if (has_move()) {
::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray(
3, *type_.move_, output);
}
The "oneof" message saves the specific type in _oneof_case_. I think using switch-case is more efficient.
But why protobuf still generate code like this?
Oneofs are internally handled similar to optional fields. In fact, descriptor.proto represents them as a set of optional fields that just have an extra oneof_index to indicate that they belong together. This is a reasonable choice, because it allowed oneofs to be used immediately with many libraries before any special support was added.
I assume that the C++ code generator uses the same structure for both optional fields and oneofs.
It is possible that switch-case could generate more efficient code, and in that case it would be useful to propose that as an improvement to the protobuf project. However, like Jorge Bellón pointed out in the comments, it is entirely possible that the compiler will be able to optimize this structure automatically. One would have to test and benchmark to be sure.
I have a very simple music player, and I'd like to make it into a music server.
I plan in using gRPC to communicate between the clients and the server.
However, I'm not sure how I should design the protocol messages to handle the playback.
I envision two types of design :
A message for each type of query. This method defines clearly all possible actions, but seems to create a lot of redundant code.
message Play{
bool flag = 1; // False means Pause
}
message Stop{
bool flag = 1;
}
A unique message, with a key containing the action. This approach seems more flexible, but also more prone to errors. I could use an enum object to limits the possible actions though.
message Playback{
enum Action {
PLAY = 0;
STOP = 1;
}
Action action = 1;
}
Basically, I guess that what's I'm asking here is whether I should define an action by the type of the message or by its content.
Is there a rule of thumb or a design pattern to apply here ?
I would recommend to use the oneof construct here:
syntax = "proto3";
message Play {
}
message Stop {
}
message Command {
oneof command {
Play play = 1;
Stop stop = 2;
...
}
}
Empty messages are fine when there are no parameters that you need to pass, and this also leaves open an easy way to extend the messages in the future, for example changing Play to:
message Play {
string filename = 1;
}
would allow including an optional filename with the request, while retaining compatibility with the old version.