The proto2 version of Protocol Buffers allows to specify default values for message elements:
optional double scaling_factor = 3 [default = 1.0];
Why is this no longer possible in proto3? I consider this a neat feature to save additional bytes on the wire without the need of writing any wrapper code.
My understanding is that proto3 no longer allows you to detect field presence and no longer supports non-zero default values because this makes it easier to implement protobufs in terms of "plain old structs" in various languages, without the need to generate accessor methods. This is perceived as making Protobuf easier to use in those languages.
(I personally think that languages which lack accessors and properties aren't very good languages and protobuf should not design down to them, but it's not my project anymore.)
This is a work around instead of a direct answer to your question, but I've found myself using wrappers.proto optional values and then setting the default value myself programatically when I absolutely must know if this was a default value or a value that was explicitly set.
Not optimal that your code has to enforce the value instead of the generated code itself, but if you own both sides, at least it's a viable alternative versus having no idea if the value was the default or explicity set as such, especially when looking at a bool set to false.
I am unclear how this affects bytes on the wire. For the instances where I've used it, message length was not a design constraint.
Proto File
import "google/protobuf/wrappers.proto";
google.protobuf.BoolValue optional_bool = 1;
Java code
//load or receive message here
if( !message.hasOptionalBool() )
message.setOptionalBool( BoolValue.newBuilder().setValue( true ) );
In my autogenerated file .pb.cc I see few places like this:
if (this->myint() != 0) {
and few like this:
myint_ = 0;
So, why not to enable default value and generate
static ::google::protobuf::int32 myint_defaultvalue = 5;
...
if (this->myint() != myint_defaultvalue) {
...
...
myint_ = myint_defaultvalue;
...
instead?
Related
I would like to use freemarker to generate java code that instantiates a BigDecimal.
The value of the BigDecimal is present at generation time.
the BigDecimal API would work like this:
BigDecimal copy = new BigDecimal(original.toString());
Alas, the freemarker magic uses numeric conversion on my value of original, so this does not work (in a freemarkter template):
BigDecimal copy = new BigDecimal("${original?c}");
None of the numeric conversions (percent, number, computer, ...) works - c/computer most interestingly, because it outputs 0 if the value becomes too big.
With considerable pain, I might be able to wrap the BigDecimal in another Object that just gives me a toString and is not a number, so freemarker might leave its value untouched and my generated BigDecimal is correct, but that's only a last resort.
Maybe there is a way to invoke the toString() method and print the result into the target document?
ERRATUM:
because it outputs 0 if the value becomes too big should read because it outputs 0 if the value becomes too small (see #ddkany's answer)
Update: FreeMarker 2.3.32 now supports lossless formatting with ?c, which is not based on DecimalFormat, but mostly on toString. To use that, you have to set the c_format configuration setting to any other value than its backward compatible default, "legacy". Like setting it to "JavaScript or JSON" is fine for most projects (that's also the default if you just set the incompatible_improvements configuration setting to 2.3.32). See the fine details of how numers are formatted in the documentation of the c built-in: https://freemarker.apache.org/docs/ref_builtins_number.html#ref_builtin_c
Old answer, for 2.3.31 and before:
What FreeMarke does by default is whatever Java's DecimalFormat does (for the localized medium format by default, again defined by the Java platform).
?c uses new DecimalFormat("0.################") with fixed US locale (and some more symbol adjustments for INF and NaN). So I don't know how that gives 0 for a huge BigDecimal number. Are you sure about that? I guess it was actually a very small number, so it was rounded down. Well, switching to "scientific" format would make more sense then, though.
To have whatever formatting logic you need, you can register your own number formatter like, in this case, configuration.setCustomNumberFormats(Map.of("toString", new MyToStringTemplateNumberFormatFactory()), and then you can use ${original?string.#toString}. Or rather, you can set the number_format to #toString, and then just use ${original}.
We have a communication channel on which we send protobufs. In order to be able to send more than one type of protobuf, we double-serialise:
message Coprolite {
enum Bezoar {
KBezoarUndef = 0;
KBezoarFriedEggs = 1;
KBezoarHam = 2;
}
Bezoar payload_type = 1;
bytes payload = 2;
}
If I have a FriedEggs protobuf, I serialise it, assign it to the payload of a Coprolite, set the payload_type to KBezoarFriedEggs, serialise the Coprolite, and send it on its way.
On receipt, I deserialise, check what I've got, and deserialise that.
This works on all of our platforms. I've not, however, found examples of others doing this this way (nor any other way, really). So this suggests I should ask for advice. Is there a better strategy or a reason that I should be wary of this?
If you want to prevent having to set payload_type you can use an oneof. Oneof's implicitly serialize the payload type by adding the tag number in front. Just like any other field serialized. So you have to write less administrative code.
There is however one advantage in your approach over using oneof's. Deepening on your programming language and how oneof's and byte arrays are implemented on your platform and in your protobuf library. An oneof implemented as an union might allocate memory the size of the largest nested message. So depending on your situation dynamically allocating the bytes array might use less memory when you send a lot of small message and only sometimes a big one.
There are two common approaches here; the simplest (especially when the types are predictable in advance) is usually oneof:
message Coprolite {
oneof payload_type {
FriedEggs eggs = 1;
Ham ham = 2;
}
}
This acts like a discriminated union, where you can check the embedded type.
In some niche scenarios, you may prefer to use Any
I have a piece of code in which I call a function with argument of a string value. The string value is taken from a string array using the index.
if (validateRejectionCategory(rejectionData[0])) {
......
}
In this case is 0 still considered a magic number? Should I abstract 0 to a variable like REJECTION_CATEOGRY_POSITION? Does the function name not make it clear enough that the value would be a rejectionCategory?
Let me know your thoughts please.
In general extracting every numerical value to a constant is a bad habit. You have to maintain balance as with all things in life. Here it's clear that you want to validate the first element of rejectionData. Most uses of 0, 1, 2 are not magic, but are used in an algorithm.
You usually want to extract numeric values that aren't easily explainable in the context of code.
E.g. if your code looks like this in some main-like file:
app.modalTimeooutSeconds = 3.0
it's not really more readable to rewrite it as
let timeoutSeconds = 3.0
app.modalTimeoutSeconds = timeoutSeconds
unless you want to have a config file that are constants are stored, but you may not have enough constants to support having such a file.
Everything is context dependent and there are no good answers.
I've recently encountered all sorts of wrappers in Google's protobuf package. I'm struggling to imagine the use case. Can anyone shed the light: what problem were these intended to solve?
Here's one of the documentation links: https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/string-value (it says nothing about what can this be used for).
One thing that will be different in behavior between this, and simple string type is that this field will be written less efficiently (a couple extra bytes, plus a redundant memory allocation). For other wrappers, the story is even worse, since the repeated variants of those fields will be written inefficiently (official Google's Protobuf serializer doesn't support packed encoding for non-numeric types).
Neither seems to be desirable. So, what's this all about?
There's a few reasons, mostly to do with where these are used - see struct.proto.
StringValue can be null, string often can't be in a language interfacing with protobufs. e.g. in Go strings are always set; the "zero value" for a string is "", the empty string, so it's impossible to distinguish between "this value is intentionally set to empty string" and "there was no value present". StringValue can be null and so solves this problem. It's especially important when they're used in a StructValue, which may represent arbitrary JSON: to do so it needs to distinguish between a JSON key which was set to empty string (StringValue with an empty string) or a JSON key which wasn't set at all (null StringValue).
Also if you look at struct.proto, you'll see that these aren't fully fledged message types in the proto - they're all generated from message Value, which has a oneof kind { number_value, string_value, bool_value... etc. By using a oneof struct.proto can represent a variety of different values in one field. Again this makes sense considering what struct.proto is designed to handle - arbitrary JSON - you don't know what type of value a given JSON key has ahead of time.
In addition to George's answer, you can't use a Protobuf primitive as the parameter or return value of a gRPC procedure.
Now that we've reached Swift 2.0, I've decided to convert my, as yet unfinished, OS X app to Swift. Making progress but I've run into some issues with using termios and could use some clarification and advice.
The termios struct is treated as a struct in Swift, no surprise there, but what is surprising is that the array of control characters in the struct is now a tuple. I was expecting it to just be an array. As you might imagine it took me a while to figure this out. Working in a Playground if I do:
var settings:termios = termios()
print(settings)
then I get the correct details printed for the struct.
In Obj-C to set the control characters you would use, say,
cfmakeraw(&settings);
settings.c_cc[VMIN] = 1;
where VMIN is a #define equal to 16 in termios.h. In Swift I have to do
cfmakeraw(&settings)
settings.c_cc.16 = 1
which works, but is a bit more opaque. I would prefer to use something along the lines of
settings.c_cc.vim = 1
instead, but can't seem to find any documentation describing the Swift "version" of termios. Does anyone know if the tuple has pre-assigned names for it's elements, or if not, is there a way to assign names after the fact? Should I just create my own tuple with named elements and then assign it to settings.c_cc?
Interestingly, despite the fact that pre-processor directives are not supposed to work in Swift, if I do
print(VMIN)
print(VTIME)
then the correct values are printed and no compiler errors are produced. I'd be interested in any clarification or comments on that. Is it a bug?
The remaining issues have to do with further configuration of the termios.
The definition of cfsetspeed is given as
func cfsetspeed(_: UnsafeMutablePointer<termios>, _: speed_t) -> Int32
and speed_t is typedef'ed as an unsigned long. In Obj-C we'd do
cfsetspeed(&settings, B38400);
but since B38400 is a #define in termios.h we can no longer do that. Has Apple set up replacement global constants for things like this in Swift, and if so, can anyone tell me where they are documented. The alternative seems to be to just plug in the raw values and lose readability, or to create my own versions of the constants previously defined in termios.h. I'm happy to go that route if there isn't a better choice.
Let's start with your second problem, which is easier to solve.
B38400 is available in Swift, it just has the wrong type.
So you have to convert it explicitly:
var settings = termios()
cfsetspeed(&settings, speed_t(B38400))
Your first problem has no "nice" solution that I know of.
Fixed sized arrays are imported to Swift as tuples, and – as far as I know – you cannot address a tuple element with a variable.
However,Swift preserves the memory layout of structures imported from C, as
confirmed by Apple engineer Joe Groff:. Therefore you can take the address of the tuple and “rebind” it to a pointer to the element type:
var settings = termios()
withUnsafeMutablePointer(to: &settings.c_cc) { (tuplePtr) -> Void in
tuplePtr.withMemoryRebound(to: cc_t.self, capacity: MemoryLayout.size(ofValue: settings.c_cc)) {
$0[Int(VMIN)] = 1
}
}
(Code updated for Swift 4+.)