I want to be able to supply a pre-instantiated message by name rather than populating it from scratch. For example, given the following schema:
message Animal {
required int32 num_legs = 1;
required int32 num_eyes = 2;
}
message Zoo {
repeated Animal animals;
}
I want to be able quickly to define a Zoo in my config file by choosing from a set of known animals:
// config.json
zoo: {
animals: [snake, bird]
}
where snake and bird are already defined:
// animals.json
bird: {
num_legs: 2
num_eyes: 2
}
snake: {
num_legs: 0
num_eyes: 2
}
What's the most elegant way to do this?
The protobuf API has methods to convert protobuf⬌JSON. For C++ you can use util::JsonStringToMessage, other languages have their API versions too (you didn't specify a language). Wrap this in a simple helper function and use your language's multi-line string constant syntax to embed the message in JSON format directly into your source.
To get your named predefined messages, use a language that has string interpolation. (Not native to C++, unfortunately, but here is a SO answer talking about ways you might do it.)
Related
Here is how I have defined a message:
message PhoneNumberAndTypes {
repeated Type types = 1;
Phone number = 2;
}
And when I added the types using .addTypes() and printed the object in console, it came out like below:
phone_numbers {
types: HOME
types: MOBILE
phone_number {
number: "9912300000"
}
}
But I hoped for the types to be like [HOME,MOBILE].
What change should I make?
Actually its a List inside while I debugged it. The "printing" on console did not necessarily reflect the actual object properties.
[HOME, MOBILE]
I'm working on several web server projects in Go, and there is a common problem that I'm always facing. I know we can achieve something like polymorphism in Go with interfaces and methods, but many times I had a scenario that I needed polymorphism on some data-holder structs that (maybe) just had some common fields, and no methods at all.
For example consider a story writing platform, where each user can write short stories and novels:
type ShortStory struct {
Name string
ID int
Body string
}
type LongStory struct {
Name string
ID int
Chapters []string
}
Now I simply want to have a data layer function, say GetStories(), which fetches all stories written by a user from database.
func GetStories(id int) []SOME_TYPE {
...
}
There are really no methods that I want to have on my ShortStory and LongStory structs. I know I can add a dummy method and let them satisfy some Storier interface, then use that interface as return type. But since there is no method I would want on a data container model, adding a dummy method just for the language to enable a feature, seems like a poor design choice to me.
I can also make the function return []interface{}, but that's against the whole idea of "typed language" I believe.
Another way is to have two separate GetShortStories() and GetLongStories() methods, which return a slice of their own type. But at some point I would finally want to merge those two slices into one and there I would again need a []interface{}. Yes, I can return a JSON like:
{
"short_stories" : [...],
"long_stories" : [...]
}
But I want my json to be like:
[{...}, {...}, {...}]
And I wouldn't change my APIs because of a language's limits!
I'm not a pro in Go, so am I missing something here? Is there a Go-ish approach to this, or is it really bad language design on Golang's side?
If you cannot express what you want to do using the features of a language, you should first try to change the way you structure your program before blaming the language itself. There are concepts that cannot be expressed in Go but can be expressed well in other languages, and there are concepts you cannot express well in other languages but you can in Go. Change the way you solve the problem to effectively use the language.
One way you can address your problem is using a different type of struct:
type Story struct {
Name string
ID int
ShortBody string
Chapters []string
}
If the Chapters is empty, then it is a short story.
Another way:
type Story struct {
Name string
ID int
Content StoryContent
}
type StoryContent interface {
Type() string
}
type ShortStory interface {
StoryContent
Body() string
}
type LongStory interface {
StoryContent
Chapters() []string
}
etc.
Is it (practically) possible to change the type name of a protobuf message type (or enum) without breaking communications?
Obviously the using code would need to be adpated to re-compile. The question is if old clients that use the same structure, but the old names, would continue to work?
Example, base on the real file:
test.proto:
syntax = "proto3";
package test;
// ...
message TestMsgA {
message TestMsgB { // should be called TestMsgZZZ going forward
// ...
enum TestMsgBEnum { // should be called TestMsgZZZEnum going forward
// ...
}
TestMsgBEnum foo = 1;
// ...
}
repeated TestMsgB bar = 1;
// ...
}
Does the on-the-wire format of the protobuf payload change in any way if type or enum names are changed?
If you're talking about the binary format, then no: names don't matter and will not impact your ability to load data; For enums, only the integer value is stored in the payload. For fields, only the field-number is stored.
Obviously if you swap two names, confusion could happen, but: it should load as long as the structure matches.
If you're talking about the JSON format, then it may matter.
I'm wondering if it is possible to use Google Protocol Buffers' enum constants as a field number of other messages, like
enum Code {
FOO = 100;
BAR = 101;
}
message Message {
required string foo = FOO;
}
This code doesn't work because FOO's type is enum Code and only a number can be used as a field number.
I am trying to build polymorphic message definitions like this animal example, that defines Cat = 1; in enum Type and required Cat animal = 100; as a unique extension number.
I thought it'd be nice to do
message Message {
required string foo = FOO.value;
}
, so that I can ensure the uniqueness of the extension field number without introducing another magic number.
So the question: is it possible to refer an enum's integer value in the protocol buffer language?
No, there is no way to do this. Sorry.
BTW, two enumerants of the same enum type can actually have the same numeric value, so defining these values in an enum does not actually ensure uniqueness.
in short, is there a way to define a protobuf Message that contains another Message of arbitrary type? Something like:
message OuterMsg {
required int32 type = 1;
required Message nestedMsg = 2; //Any sort of message can go here
}
I suspect that there's a way to do this because in the various protobuf-implementations, compiled messages extend from a common Message base class.
Otherwise I guess I have to create a common base Message for all sorts of messages like this:
message BaseNestedMessage {
extensions 1 to max;
}
and then do
message OuterMessage {
required int32 type = 1;
required BaseNestedMessage nestedMsg = 2;
}
Is this the only way to achieve this?
The most popular way to do is to make optional fields for each message type:
message UnionMessage
{
optional MsgType1 msg1 = 1;
optional MsgType2 msg2 = 2;
optional MsgType3 msg3 = 3;
}
This technique is also described in the official Google documentation, and is well-supported across implementations:
https://developers.google.com/protocol-buffers/docs/techniques#union
Not directly, basically; protocol buffers very much wants to know the structure in advance, and the type of the message is not included on the wire. The common Message base-class is an implementation detail for providing common plumbing code - the protocol buffers specification does not include inheritance.
There are, therefore, limited options:
use different field-numbers per message-type
serialize the message separately, and include it as a bytes type, and convey the "what is this?" information separately (presumably a discriminator / enumeration)
I should also note that some implementations may provide more support for this; protobuf-net (C# / .NET) supports (separately) both inheritance and dynamic message-types (i.e. what you have above), but that is primarily intended for use only from that one library to that one library. Because this is all in addition to the specification (remaining 100% valid in terms of the wire format), it may be unnecessarily messy to interpret such data from other implementations.
Alternatively to multiple optional fields, the oneof keyword can be used since v2.6 of Protocol Buffers.
message UnionMessage {
oneof data {
string a = 1;
bytes b = 2;
int32 c = 3;
}
}