I'm looking to create a gRPC response that returns a list of maps. Here is what I imagined the structure being:
message GetSettingsResponse {
repeated map<string, string> settings = 1;
}
Repeating maps is not supported, however, and I had to nest the map in a separate message to make it work:
message GetSettingsResponse {
repeated Setting settings = 1;
}
message Setting {
map<string, string> setting = 1;
}
This works, but it forces us to write some confusing code on both the client and server. Is there any way to avoid this solution and get closer to my desired structure?
No, basically. What you have is the closest you can do in protobuf.
Related
I am fairly new to API building, so this may be a broader question than I originally posed.
I am creating an API in Golang (using protobuf 3 and gRPC) that has two similar endpoints:
GET /project/genres
GET /project/{id}
The problem is that when I run curl localhost:8080/project/genres, the pattern matching results in the /project/{id} endpoint getting called with genres as the id. Is there some simple way around this, or do I have to build something into the server code to call the proper function based on the type?
I also tried flipping the ordering of these definitions, just in case the pattern matching had some order of operations that I didn't know about, but this didn't make a difference.
Here are the definitions in my proto file:
message EmptyRequest { }
message ProjectRequest {
string id = 1;
}
message GenreResponse {
int32 id = 1;
string name = 2;
}
message ProjectResponse {
int32 id = 1;
string name = 2;
}
service ProjectService {
rpc GetProject(ProjectRequest) returns (ProjectResponse) {
option (google.api.http) = {
get: "/v1/project/{id}"
};
}
rpc GetGenres(EmptyRequest) returns (GenreResponse) {
option (google.api.http) = {
get: "/v1/project/genres"
};
}
}
I was able to specify a check in the url path template to get around this issue:
rpc GetGenres(EmptyRequest) returns (GenreResponse) {
option (google.api.http) = {
get: "/v1/project/{id=genres}"
};
}
This seems to have fixed the problem. I don't know if there are other solutions, or if this is the right way to do this, but I'm happy to accept other answers if something better comes in.
I would like to set the value of a field which is of the type map using the reflection api for a protobuf message.
I have tried using the set field method as shown below. This does not seem to work. I can make it work by using the suggestion in Filling up a map field in Proto using reflection API. But this not seem to be a nice solution!
message OuterMessage {
map<string, InnerMessage> mapInner = 1;
}
Map<String, InnerMessage> map = new HashMap<>();
map.put("R1", im1);
map.put("R2", im2);
OuterMessage.newBuilder().setField(fieldDescriptor, map);
I am getting the error,
java.lang.ClassCastException: java.base/java.util.HashMap cannot be cast to java.base/java.util.List
Which is understandable when i look at the implementation. But how else would i set a map?
I want to preserve my application from future issues with backward compatibility. Now I have this version of test.proto:
syntax = "proto3";
service TestApi {
rpc DeleteFoo(DeleteFooIn) returns (BoolResult) {}
rpc DeleteBar(DeleteBarIn) returns (BoolResult) {}
}
message DeleteFooIn {
int32 id = 1;
}
message DeleteBarIn {
int32 id = 1;
}
message BoolResult {
bool result = 1;
}
I'm interested in a case when I will want to change result message of DeleteBar() to a message like "DeleteBarOut":
syntax = "proto3";
service TestApi {
rpc DeleteFoo(DeleteFooIn) returns (BoolResult) {}
rpc DeleteBar(DeleteBarIn) returns (DeleteBarOut) {}
}
message DeleteFooIn {
int32 id = 1;
}
message DeleteBarIn {
int32 id = 1;
}
message DeleteBarOut {
reserved 1;
string time = 2;
}
message BoolResult {
bool result = 1;
}
The question is about backward compatibility on-wire with the old .proto. Can I change the name of the result message from "BoolResult" to "DeleteBarOut"?
Or I should save the old name of the message and edit fields list of "BoolResult"? But then how can I save DeleteFoo() from any changes in this solution?
When making a breaking change to an API like this, it is a common practice to support both versions while transitioning. In order to do this, you would need to add a version field to the request message and then in your request handler, route the message to different backends based on which version is specified. Once there is no more traffic going to the v1 backend you can make a hard cutover to v2 and stop supporting v1.
Unfortunately, if you just change the RPC definition without versioning, it is impossible to avoid a version incompatibility between the server and the client. The other option of course is to add a new RPC endpoint rather than modifying an existing one.
In general if you are making breaking API changes you're going to have an unpleasant time.
I am writing protoc plugin in Go which should generate documentation for our GRPC services and currently struggle in attempt to know right order of options.
First, how the protobuf looks like
syntax = "proto3";
option go_package = "sample";
package sample
import "common/extensions.proto"
message SimpleMessage {
// Id represents the message identifier.
string id = 1;
int64 num = 2;
}
message Response {
int32 code = 1;
}
enum ErrorCodes {
RESERVED = 0;
OK = 200
ERROR = 6000
PANIC = 6001
}
service EchoService {
rpc Echo (SimpleMessage) returns (Response) {
// common.grpc_status is an extension defined somewhere
// these are list of possible return statuses
option (common.grpc_status) = {
status: "OK"
status: "ERROR"
status: "PANIC" // Every status string will must be one of ErrorCodes items
};
option (common.middlewares) = {
middleware: "csrf"
middleware: "session"
}
}
};
As you see, there're two options here. The problem is protoc doesn't bind position directly to tokens. It leaves this information in a special sections where it can be restored via using so called "paths". And these paths are rely on order, while options are hidden and can only be retrieved using proto.GetExtension function which doesn't report option index either.
I need this token location information to report errors. Is there any way to get option index or something equivalent?
I am thinking about using standalone parser just to retrieve the right order, but this feels somewhat awkward. Hope there's a better way.
// given a set of Item objects, group them by the managers of creator and owners
Map<String, List<Item>> managersItems =
itemSet.parallelStream().flatMap(item -> {
// get the list of the creator and owners
List<String> users = new ArrayList();
users.add(item.getCreator());
users.addAll(item.getOwners());
return Stream.of(users.toArray(new String[] {})).map(user -> {
LdapUserInfo ldapUser = LdapUserInfoFactory.create(user);
String manager = ldapUser.getManager();
return new AbstractMap.SimpleImmutableEntry<String, Item(manager, item);
});
}).collect(
Collectors.groupingBy(Map.Entry::getKey, Collectors.mapping(Map.Entry::getValue, Collectors.toList())));
This code compiles fine in Eclipse Mars, but gets the following eror in Eclipse Luna:
Type mismatch: cannot convert from Map<Object,List<Object>> to Map<String,List<WeblabInfo>>
If I do not assign the returned to a Map with Map<String, List<Item>> managersItem = in Eclipse Luna, the error is at Map.Entry::getKey and Map.Entry::getValue statement with message:
The type Map.Entry does not define getKey(Object) that is applicable here".
What did I do wrong?
You didn't do anything wrong. Eclipse compiler has problems with type inference that causes these issues. If Luna compatibility is important, you will have to add explicit types to lambda expressions. Try, for example, Map.Entry::<String,Item>getKey
On another note, it's not necessary to convert a List to array to stream it. You can directly call users.stream(). But even creating the List isn't necessary. You can use Stream.concat(Stream.of(item.getCreator()), item.getOwners().stream()) instead (granted, it's a bit unwieldy).
Finally (and most importantly), avoid using parallelStream with blocking code, such as looking up data in an external system. Parallel streams are designed to handle CPU-bound tasks.
I was able to come up with this solution from Misha's answer. This is working with Eclipse Luna Java compiler
Map<String, List<Item>> managersItems = itemSet
.stream()
.<Map.Entry<String, Item>> flatMap(item -> {
return Stream.concat(Stream.of(item.getCreatorLogin()), item.getOwners().stream()).map(
user -> {
LdapUserInfo ldapUser = LdapUserInfoFactory.create(user);
String manager = ldapUser.getManagerLoginName();
return new AbstractMap.SimpleEntry<String, Item>(manager, info);
});
})
.collect(Collectors.groupingBy(Map.Entry<String, Item>::getKey,
Collectors.mapping(Map.Entry<String, Item>::getValue,
Collectors.toList())));