See https://github.com/grpc/grpc-node/issues/1202.
Usually in CRUD operations, the value not provided means do not change that field, and the empty array [] means to clear all items inside that field.
But if you tries to implement CRUD operations and provide them as services via grpc, then the above scenario is hard to implement.
service CRUD {
rpc updateTable(updateRequest) returns updateResponse {}
}
message updateRequest {
repeated string a = 1;
string b = 2;
}
message updateResponse {
boolean success = 1;
}
If you load the package with default options, then the client can't delete items of a by
client.CRUD.updateTable({a: []})
because the argument {a: []} becomes {} when it arrives the server side.
If you load the package with options {arrays: true}, then the field a will be cleared unintentionally while client side only tries to update other fields:
client.CRUD.updateTable({b: 'updated value'})
because the argument {b: 'updated value'} becomes {a: [], b: 'updated value'} when it arrives the server side.
Can anyone share some better ideas regards how to handle these 2 scenarios with grpc-node and proto3?
The protobuf encoding doesn't distinguish between these two cases. Since protobuf is language-agnostic, it doesn't understand the conceptual nuance of "undefined" versus "[]" of Javascript.
You would need to pass additional information inside the proto message in order to distinguish between the two cases.
I would highly suggest reading the design documentations here: https://developers.google.com/protocol-buffers
Related
I'm implementing a policy enforcement point between a client and a server that receives messages from the server, and, if the client doesn't have adequate authorization to see some parts of the message deletes those parts before sending them to the client.
message {
string not_sensitive = 1;
optional string sensitive = 2;
}
pseudo code
from_server >> my_msg;
if (!authorized) {
my_msg.delete("sensitive");
}
to_client << my_msg;
Yes.
As I understand current v3 pb schema language, all fields are optional. But regardless of that, a field marked optional in v2 is something that need not be there. So expanding your pseudo code to, say, C++ (see here), one can see that the generated class would end up with a has_sensitive() method and a clear_sensitive() method. Calling the latter and then serialising the object would result in wire format data that omitted the sensitive field.
I've got a situation where the other end of the grpc communication is not in sync with their releases. My higher ups, would like me to therefore add a field that is going to work if the other side does or doesn't fill it out, for a short time period (like two weeks)
I believe I can do this by adding it to the end of the proto message such that the indices for the other fields do not change. From, what I've Googled, the optional field is not avail prior to version 3.15, so I have to use a work around.
The workaround that was described to me was to use oneof. However, I am not 100% sure what that looks like. All examples show the oneof field by itself. Are the indices that belong to the oneof values indendent of the indices that belong to the rest of the message?
message TestMessage {
string somefield = 1;
int someotherfield = 2;
oneof mynewoptionalfield
{
string mynewfield = ???? Does this have to be 3 or is it 1?
int ifihadanother = ???? Does this need to be 4 or 2?
}
}
Questions:
What are the indices I use where the ??? marks are
Is this the proper work around to use when the other side isn't going to recompile and deploy with the changes to the protofile?
How do I then check if the field was filled in my C++ code?
Your use-case is exactly what protobufs were designed to handle. All you need to do is: add a new field to the message. In the easiest case, the client application code simply doesn't look at the new field until the server roll-out is complete and so doesn't notice sometimes it is present and other times missing.
You are correct that you should not change the indices (field ids) of the pre-existing fields. Although I'll note that you can add your new field anywhere within the message; the order the fields are written in does not matter for protobuf.
So you'd just add another field like:
message TestMessage {
string somefield = 1;
int someotherfield = 2;
string mynewfield = 3;
}
You don't have to use 3 as the id. You could use 4, or 10, or 10000. But small numbers are more efficient for protobuf and it is typical to just choose the "next" id. On-the-wire protobuf uses the id to identify the field, so it is important you don't change the id later.
In protobuf 3, all fields are "optional" in the protobuf 2 sense; there are no "required" fields. However, protobuf 2 also provided "field presence" for all fields. Protobuf 3 only provided field presence for oneofs and messages... until the recent re-introduction of the "optional" keyword.
In protobuf 3 if you call textMessage.getMynewfield() it will always return a non-null string. If the string was not sent, it will use the empty string (""). For integers 0 is returned and for messages the "default message" (all defaults) is returned. This is plenty for many use-cases, and may be enough for you.
But let's say you need to distinguish between "" and <notsent>. That's what field presence provides. Messages in protobuf 3 have "has" methods that return true if a value is present. But primitives don't have that presence information. One option is to "box" the primitive with standard wrappers that make the primitive a message. Another option available in newer versions of protobuf is the optional keyword. Both options will provide a method like textMessage.hasMynewfield().
message TestMessage {
string somefield = 1;
int someotherfield = 2;
google.protobuf.StringValue mynewfield = 3;
// -or-
optional string mynewfield = 3;
}
Just started to learn reason react and struggle with a graphql setup trying to read an ENUM value.
setup
reason react
apollo graphql
graphql_ppx
github graphql endpoint
i am fetching the latest pull request data over the github api and reading the status property which is an enum and defined in the gql docs as:
OPEN
CLOSED
MERGED
checking the network tab, i see the states are received as strings. within the application when I log the field i get a bunch of integers reflecting the values. can smb explain me, how i can "print" the data as string to my view and why they are translated to integers? is there somewhere a type generated which i could use for a variant switch?
let stateEnum = data->map(node => node##state);
Js.log(stateEnum) // possible values: 880069578, 982149804 or -1059826260
// somehow switch these values here?! :)
// current type of `stateEnum` is option('a)
thanks a lot in advance and have a nice day!
GraphQL Enums are represented as Reason polymorphic variants. Under the hood, in runtime, they are just integers. If you want to display them to the user you have two options:
1. Map them to string by hand using a switch
let status =
switch(node#status) {
| `OPEN => “Open”
// other cases
}
You can use BuckleScript functionality do generate jsConverters:
[#bs.deriving jsConverter]
type status = [ |`OPEN | `CLOSED /* other cases */]
this will generate two functions for you: statusToJs and statusFromJs. They help you convert variant to and from string.
Here is BuckleScript documentation about it: https://bucklescript.github.io/docs/en/generate-converters-accessors#convert-between-js-string-enum-and-bs-polymorphic-variant
As #Herku mentioned in his comment, the key was just to do this:
// asume that your enum is on a gqp property called `state`
// and we use the built in lib `Belt.Option` and the fn `getWithDefault`
// this way we are sure, that `stateEnum` is defined with one of the valid enum values
let stateEnum = data->map(node => node##state)->getWithDefault(`OPEN);
// next we switch the polymorphic variant
let state = switch(stateEnum) {
| `OPEN => "open"
| `CLOSED => "close"
| `MERGED` => "merged"
}
// str = let str = ReasonReact.string;
str(state);
I have a proto schema defined as below,
message User {
int64 id = 1;
bool email_subscribed = 2;
bool sms_subscribed = 3;
}
Now as per official proto3 documentation, default values are not serialized to save space during wire transmission. But in my case I want to receive whether the client has explicitly set true/false for fields email_subscribed/sms_subscribed (because the values were true before but now the user wants to unsubscribe). Hence, when the client sends false for any of these fields, the generator code serializer just omits these fields.
How do I achieve this and avoid the omission of these fields for the above scenario?
PS: I am using Javascript as my GRPC client and Python and GRPC Server.
Update: this has changed recently with the re-introduction of presence tracking info proto3 via a new meaning of the optional keyword:
message User {
optional int64 id = 1;
optional bool email_subscribed = 2;
optional bool sms_subscribed = 3;
}
With this change (now available in protoc etc), explicit assignment is transmitted even if it is the implicit default value.
You cannot under proto3. Your best bet is probably to define a tri-bool enum with not-specified as the first item with value zero, and some true / false values after that.
This will require the same space as a protobuf bool, but will not be binary compatible - so you cannot simply change the declared member type on existing messages. Well, I guess if you make true === 1, then at least that still works - and for the transition you'd have to anticipate false / not specified being ambiguous until you've flushed any old data.
The other option is to add a bool fooSpecified member for every bool foo, but that takes more space and is error-prone due to being manual.
Another option will be to use wrappers with proto3. They basically wrap your value in a message so on the parent message it can be left null.
This way you can differentiate null / false / true on your bool field with a some extra work.
I am trying to understand the difference between map and pluck transformational operators in RxJS.
Can anyone help me with this?
The docs say
Pluck : Like map, but meant only for picking one of the nested
properties of every emitted object.
Therefore, let's say you have
[{ name: 'Joe', age: 30, job: { title: 'Developer', language: 'JavaScript' },
{ name: 'Sarah', age: 35 }]
and you want a list of all job titles.
Using map would be kind of a pain (because of the nullability of job), but with 'pluck' you can write pluck('job', 'title') and it will traverse the tree looking for job.title - and won't fail if job is null.
Example taken from : https://www.learnrxjs.io/operators/transformation/pluck.html
https://jsfiddle.net/btroncone/n592m597/
As #mgm87 said, you can perform an operation with map.
On the opposite, pluck is just taking a value.
For example, with map you could do something like that:
this.http.get('...some api url to get a user...')
.map(response => response.json())
.map(user => user.age > 18 ? 'major': 'minor')
.do(isMajorOrMinor => console.log(isMajorOrMinor))
So you can manipulate your data down the chain even conditionally.
BUT, for me one of the big differences is that map is typed.
Which means if you have some data let say:
interface IUser {
name: string;
age: number;
dogs: IDog[];
}
And you receive at some point a user, from which you want to get his dogs:
user$
.map(user => user.dogs)
.do(dogs => ...) // here, you do NOT need to precise (dogs: IDog[]) because Typescript will make a type inference
And that's why I'm always using map even to just "pluck" some data.
Stop using pluck!
Pluck is now planned to be removed in RxJS v8.
Do you know what is the reason?
Because after the addition of the optional chaining operator in JS, it's essentially, just a weak version of Map.
So what's the difference between the two?
Both are used to "transform" the data that is going to be emitted.
Map can be used to... map an observable emission (like we do in JS with Array.prototype.map), while Pluck is used to select/pick a property to emit (without having to emit properties that we don't care for, hence improving the performance).
But even before the optional chaining operator, you could just map the properties instead of plucking them. The result & performance were/are about the same.
pluck('prop')
is just a shorthand for:
map(x => x.prop)
Well, then what was the reason behind the implementation of Pluck?
It was basically implemented to achieve path traversal safety, meaning you could try to pluck a nested property without getting the error (that Map would throw) if the property is not defined:
pluck('foo', 'bar', 'baz'); // no error if foo is not defined
map(x => x.foo.bar.baz) // error if foo is not defined
With optional chaining, this advantage doesn't exists anymore, since we can just do this:
map(x => x?.foo?.bar?.baz)
This is the main reason why the Pluck operator is going to be deprecated and removed in the future.
Another important reason is TS typing for pluck is quite complex and not as robust as the map operator.
Got that info in GitHub:
The commit of Pluck deprecation is almost one year old but I still don't see any warning of future deprecation in the docs, hence I am posting here since I think it's a good thing to know. I already stopped plucking for this reason.
Map can perform an operation on each emitted value.
http://reactivex.io/rxjs/class/es6/Observable.js~Observable.html#instance-method-map
https://www.learnrxjs.io/operators/transformation/map.html
// value from observable = 10
map(x => 10*x)
// value from map = 100
Pluck simply picks one of the nested properties of each emitted value.
http://reactivex.io/rxjs/class/es6/Observable.js~Observable.html#instance-method-pluck
https://www.learnrxjs.io/operators/transformation/pluck.html
// value from observable = {p = 10, w = 100}
pluck('p')
// value from pluck = 10
They are very similar, but as I understand it, map works with an array whereas pluck takes the values from an object.
This is the place to go for specifics.