I have an object that is something like this:
{
"colorRestrictions": {
"availableColors":[ ... ],
"excludedColors": [ ... ]
}
}
How do I translate this into a proto file considering that colorRestriction is an object that can have either of the arrays?
The property oneof doesn't allow for repeated within the declaration.
I guess it's something common but I couldn't find a straight answer, any suggestions?
Thanks
Fields are all optional in Protobuf 3. So just define ordinary fields.
message ColorRestriction {
repeated Color available_colors = 1;
repeated Color excluded_colors = 2;
}
oneof is for mutually-exclusive fields. It doesn't require that a field is present; a oneof can be unset.
Even in Protobuf 2 repeated fields couldn't be marked as required. repeated fields don't have normal field presence because "empty list" is encoded identically to "unset list." When you need presence information in a repeated field (e.g., to include it in a oneof or distinguish "empty list" from "unset list") you can wrap it in a message.
message ColorRestriction {
ColorList available_colors = 1;
ColorList excluded_colors = 2;
}
message ColorList {
repeated Color colors = 1;
}
Related
I want to understand if the messages bellow are compatible from the perspective of protobuf and serialization/deserialization.
message HelloReply {
string message = 1;
string personalized_message = 2;
}
message HelloReply {
string personalized_message = 2;
string message = 1;
}
Does the order matter for compatibility in any situation?
The textual order is largely irrelevant, although it may impact some code generation tooling - but most languages don't care about declaration order, so even that: won't matter. The fields are still defined semantically equivalent - the numbers match the existing meaning (name) and type. It is the number that is the determining feature in identifying a field.
At the protocol level:
parsers must allow fields in any order
serializers should (but not must) write fields in ascending numerical field order
I've got a situation where the other end of the grpc communication is not in sync with their releases. My higher ups, would like me to therefore add a field that is going to work if the other side does or doesn't fill it out, for a short time period (like two weeks)
I believe I can do this by adding it to the end of the proto message such that the indices for the other fields do not change. From, what I've Googled, the optional field is not avail prior to version 3.15, so I have to use a work around.
The workaround that was described to me was to use oneof. However, I am not 100% sure what that looks like. All examples show the oneof field by itself. Are the indices that belong to the oneof values indendent of the indices that belong to the rest of the message?
message TestMessage {
string somefield = 1;
int someotherfield = 2;
oneof mynewoptionalfield
{
string mynewfield = ???? Does this have to be 3 or is it 1?
int ifihadanother = ???? Does this need to be 4 or 2?
}
}
Questions:
What are the indices I use where the ??? marks are
Is this the proper work around to use when the other side isn't going to recompile and deploy with the changes to the protofile?
How do I then check if the field was filled in my C++ code?
Your use-case is exactly what protobufs were designed to handle. All you need to do is: add a new field to the message. In the easiest case, the client application code simply doesn't look at the new field until the server roll-out is complete and so doesn't notice sometimes it is present and other times missing.
You are correct that you should not change the indices (field ids) of the pre-existing fields. Although I'll note that you can add your new field anywhere within the message; the order the fields are written in does not matter for protobuf.
So you'd just add another field like:
message TestMessage {
string somefield = 1;
int someotherfield = 2;
string mynewfield = 3;
}
You don't have to use 3 as the id. You could use 4, or 10, or 10000. But small numbers are more efficient for protobuf and it is typical to just choose the "next" id. On-the-wire protobuf uses the id to identify the field, so it is important you don't change the id later.
In protobuf 3, all fields are "optional" in the protobuf 2 sense; there are no "required" fields. However, protobuf 2 also provided "field presence" for all fields. Protobuf 3 only provided field presence for oneofs and messages... until the recent re-introduction of the "optional" keyword.
In protobuf 3 if you call textMessage.getMynewfield() it will always return a non-null string. If the string was not sent, it will use the empty string (""). For integers 0 is returned and for messages the "default message" (all defaults) is returned. This is plenty for many use-cases, and may be enough for you.
But let's say you need to distinguish between "" and <notsent>. That's what field presence provides. Messages in protobuf 3 have "has" methods that return true if a value is present. But primitives don't have that presence information. One option is to "box" the primitive with standard wrappers that make the primitive a message. Another option available in newer versions of protobuf is the optional keyword. Both options will provide a method like textMessage.hasMynewfield().
message TestMessage {
string somefield = 1;
int someotherfield = 2;
google.protobuf.StringValue mynewfield = 3;
// -or-
optional string mynewfield = 3;
}
Am I able to use
message Foo {
map<string, string> foo = 1;
}
in place of
message Foo {
repeated KeyValuePair foo = 1;
}
message KeyValuePair {
string key = 1;
string value = 2;
}
?
The first source is in proto3 and the second is in proto2.
As long as you don't have duplicate keys, they will be very similar. If you have duplicate keys, using a map will have different behaviour, as duplicatws will either cause overwrites or an exception (I can't recall which, sorry). Also, "repeated" is usually implemented as a list/array/etc, so: order is retained. "map" is usually implemented with some kind of map/dictionary structure, where order is not usually guaranteed.
So: if order doesn't matter and you always have unique keys: you're fine.
I am using protocol buffers defined like this:
message Index {
message albums {
repeated string name = 1;
}
map<string, albums> artists_albums= 1;
map<int32, albums> year_albums = 2;
}
It generates go code like this:
type Index struct {
ArtistsAlbums map[string]*IndexAlbums
YearAlbums map[int32]*IndexAlbums
}
How can I make it generate map values of type IndexAlbums instead of *IndexAlbums?
If you use gogoprotobuf then there is an extension that will allow that
map<string, albums> artists_albums = 1 [(gogoproto.nullable) = false];
With regular goprotobuf I don't believe there is a way.
nullable, if false, a field is generated without a pointer (see warning below).
Warning about nullable: According to the Protocol
Buffer specification, you should be able to tell whether a field is
set or unset. With the option nullable=false this feature is lost,
since your non-nullable fields will always be set. It can be seen as a
layer on top of Protocol Buffers, where before and after marshalling
all non-nullable fields are set and they cannot be unset.
I'm wondering if it is possible to use Google Protocol Buffers' enum constants as a field number of other messages, like
enum Code {
FOO = 100;
BAR = 101;
}
message Message {
required string foo = FOO;
}
This code doesn't work because FOO's type is enum Code and only a number can be used as a field number.
I am trying to build polymorphic message definitions like this animal example, that defines Cat = 1; in enum Type and required Cat animal = 100; as a unique extension number.
I thought it'd be nice to do
message Message {
required string foo = FOO.value;
}
, so that I can ensure the uniqueness of the extension field number without introducing another magic number.
So the question: is it possible to refer an enum's integer value in the protocol buffer language?
No, there is no way to do this. Sorry.
BTW, two enumerants of the same enum type can actually have the same numeric value, so defining these values in an enum does not actually ensure uniqueness.