We often have use cases where we only want to update a subset fields on a resource. So if we have a resource Person:
type Person struct {
Age int
Name string
Otherfield string
}
Say the calling client only wants to update the Age field. How would an endpoint be normally set up to handle this?
I believe this should be done with a PATCH request, with only the fields being set as part of the payload, ie:
{
Age: 21
}
However, this won't work with proto3 because as far as I know there are no null fields, only default values. This won't work in many cases where the default value is valid.
Looking at Google own protobuf files (e.g. here), they use FieldMask for partial update.
FieldMask object is passed along with the request, and has the form (in JSON):
{
mask: "Person.Age"
}
This allows the client to tell the server which fields they wish to update, without counting on the partial message itself to figure this out.
I think this adds unnecessary complexity on (each!) client, but we couldn't find any other way to achieve partial updates with proto3.
You can see full documentation of FieldMask here.
Note that it can also be used to filter out responses if the client doesn't need the entire object.
Related
Say I have a query that returns an array of Customer objects, each one having an id, name, and email, all of which are non-nullable. My resolver loads the data from some source, maybe a database or maybe a downstream system. Most of the data that comes back is fine, but maybe for some reason we're missing the email address for a single customer.
Currently if my resolver just returns the array of all known customers, including the dodgy one, then the entire operation will fail with something like:
{
"data": {
"customers": null
},
"errors": [{
"message": "Cannot return null for non-nullable field Customer.email"
// More error stuff here
}]
}
I get that the validation is supposed to be strict, but now on my UI it fails to load the entire list of customers just because one of them is invalid. It makes it too easy for a single record to poison the entire database.
What I would like is a way to still return the error, but to also have the data field be populated with all of the other, valid Customer records. Is this possible to do with apollo server? Or do I need to manually validate all of the data before trying to return it from my resolver? That would be quite arduous as my real data structure is much more complex.
From this discussion, it seems there's no way to do exactly what I was asking (filter out the invalid records on the server side).
The right approach to this is to either:
Make the email field nullable, so the server can return every Customer, some of which may have incomplete data; or
Make the elements of the top-level Customer array nullable, so the server can return e.g. [customer1, null, customer3]
Option 1 means that the client gets access to more data on an error, so it could still show the rest of the customer's details even if the email is missing. However it makes the data structure less reliable as now anywhere that uses the email might need to do a null check.
Option 2 is a more aggressive failure, nulling out the entire customer because of a single bad field. But it makes it simpler for the client to filter out bad customer records at the top level, allowing the rest of the frontend code to be confident that every customer has an email address.
On that basis I think I prefer option 2.
You might be interested in this article: https://blog.logrocket.com/handling-graphql-errors-like-a-champ-with-unions-and-interfaces/
It talks about using union types to report result status rather than traditional GQL error reporting. Here's an example of how it might look in your case:
type Customer {
id: ID!
name: String!
email: String!
}
interface CustomerErrorInterface {
message: String!
}
type MissingCustomerEmailError implements CustomerErrorInterface {
id: ID!
}
union CustomerResult = Customer | CustomerErrorInterface
type Query {
getCustomers: [CustomerResult!]!
}
Then, on the client side, your query would look something like this:
query getCustomers() {
__typename
... on Customer {
id
name
email
}
... on CustomerErrorInterface {
message
}
... on MissingCustomerEmailError {
id
}
}
This allows you to maintain type safety and avoid nulling things that shouldn't be nullable, while still reporting meaningful results (both successes and error types) that are associated with the data you do have. Using an error interface rather than a concrete error type in the union allows you to leave the option for reporting new types of errors in the future in a backward-compatible way.
The difference between this and the "default" GQL way of error handling (with the top-level errors output) is that reporting the error in an interface allows still giving meaningful reference data (like the id field) in easy locations rather than needing to parse error messages, and different types of errors can provide different reference data as appropriate. Error interface type handling also better fits with regular GraphQL output type patterns (in my opinion) rather than needing to switch on an enum to know how to handle different types of errors.
Allowing null in the output list may accomplish something similar in this particular case, but it makes it impossible to know which piece of data has an error, especially if the data in question does not correspond directly to a user-supplied input that can be found through the path provided by traditional error handling.
Given the following GQL
query getMembers {
repository(owner: "nasa", name: "cumulus") {
mentionableUsers(first: 100) {
nodes {
login
organization(login: "nasa") {
login
}
}
}
}
}
(Query against GitHub v4 GraphQL)
the value for login under organization is either "nasa" or null
I am trying to figure out if it's possible to use #skip against the login/organization so that only contributors to the repo, who are members of the nasa org are shown. I believe for this particular query you can do it another way, but this is just an example.
How would you use #skip/#include with a non boolean. There is minimal documentation on this. While I could filter the response JSON in my client side app, it would be more efficient to receive less data sent over the network and then to parse in my app.
Playing in GraphQLi I received errors trying this various ways - maybe its only possible if the field returns a boolean itself?
e.g., I couldn't do login #skip(if login==null). I also tried setting a value to null in the variables section and the referencing it in the query, but none of the variations I tried work.
What I would really like to do is not include the parent if the child field is some value. e.g., if login=null then don't include that mentionable user. There is no search field option on mentionableUser. From my reading, I am guessing that the only way to do this would be if the API was modified to put a search or filter field on the mentionalbeUsers, otherwise I would need to do this with my client?
Couple of points.
Both the #skip and #include directives provide the same functionality -- allowing the client to arbitrarily chose whether a field should be included in the request.
Let's say we have a query like:
query ($skipBar: Boolean!) {
foo
bar #skip(if: $skipBar)
}
If I set skipBar to true, I am effectively just sending this query:
query {
foo
}
If I set it to false, I am effectively just sending this query:
query {
foo
bar
}
As a client, my logic has to determine the value to assign to skipBar, but I could just as easily use that same logic to decide between sending one of those two queries. In other words, like variables and fragments, #skip and #include are simply a convenient way to keep things DRY on the client-side. They cannot be used to filter the result returned by the server.
GraphQL syntax does not support expressions (or for that matter, any sort of references to parts of the response). Additionally, #skip and #include only take a single argument (if) and that argument must be passed a Boolean -- either as a variable or as a literal value. Even if you could somehow pass an expression to the if argument, though, the directives determine whether the field is included in the request, period. So if the skipped field is part of a returned List (like a List of nodes), it will be absent from every node when it's skipped.
So, is there a workaround?
Not really :( As you've already guessed, if the GitHub API doesn't provide a way to filter a field, there's not much you can do as a client -- you'll have to apply the filtering logic client-side.
I'm building a microservice system with multiple disconnected components, and I'm currently trying to find out how to implement knowing which fields on an object should be updated based on the protobuf data provided.
The flow is this:
The client sends a JSON-request to an API.
The API translates the JSON-data into a protobuf struct, which is then sent along to the microservice responsible for handling it.
The microservice receives the data from the API and performs any action on it, in this case, I'm trying to change a single value in a MySQL table, such as a client's email address.
Now, the problem I have is that since protobuf (understandably) doesn't allow pointers, the protobuf object will contain zero-values for everything not provided. This means that if a customer wants to update their email address, I can't know if they also set IncludeInMailLists to false - or if it was simply not provided (having its zero-value) and shouldn't change.
The question is: how will I - from the protobuf object - know if a value is expressively set to 0, or just not provided?
My current solution is pretty much having a special UpdateCustomer-object which also has an array of Fields specifying which fields the microservice should care about, but it feels like bad solution.
Someone must have solved this better already. How should I implement it?
Protobufs field masks are one way.
https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.FieldMask
https://github.com/golang/protobuf/issues/225
But if you are using grpc then there's a (sort of) built in way.
Grpc wrappers
Since proto3 (protobufs v3) there's been no distinction between a primitive that is not set, and a primitive that's been set to the "zero value" (false, 0, "", etc).
Instead you can use objects or in protobufs language a "message", as objects can be nil / null. You've not mentioned what language you are working in but hopefully these examples make sense.
Given an RPC service such as:
import "google/protobuf/wrappers.proto";
service Users {
rpc UpdateUser(UpdateUserRequest) returns (UpdateUserResponse)
}
message UpdateUserRequest {
int32 user_id = 1;
google.protobuf.StringValue email = 2;
}
message UpdateUserResponse {}
Note the import "google/protobuf/wrappers.proto"; is important.
It given you access to the google protobufs wrappers source code here. These are not objects that have methods that allow you to test for presence.
Grpc generated code in java gives you methods such as .hasEmail() which returns true if the value is present. The getter on an unset value will still return you the zero value. I think the golang version uses pointers that you can test for nil instead of an explicit hasX() method.
More info / discussion in this github issue
Is it possible to have the Help Page sample generator ignore certain properties of a particular type?
For example, we use the same DTO for object Request and Response messages, for both POST and PUT requests. When user is POSTing a model (creating a new record) they don't need to provide the ID field.
But once its created and we serialize the new record into the response body, the ID field is included and returned to the client.
So in the POST request sample, I don't want the ID field to be displayed because for post request it doesn't make sense.
But the POST response sample, I do want the ID field displayed...
I am aware that there is the ApiExplorerSettings attribute which can be applied to a Class or Method...but is there anything similar for a Property?
Something like this would be great:
public class MyDTO
{
[ApiExplorerSettings(IgnoreForRequestApi = true, IgnoreForResponseApi = false)]
public int Id { get; set; }
// Other properties omitted for brevity...
}
Using the following annotation I've successfully hidden a property from the generation!
[ApiExplorerSettings(IgnoreApi = true)]
No, there isn't a similar option for a property. HelpPage uses formatter instances configured on the application to serialize the samples and as you can imagine the formatters must not have this knowledge within themselves.
Regarding workarounds:
a. You could explicitly set the raw sample for a particular action's requestsample via the SetSampleRequest extension of HttpRequestMessage. You should be able to see some examples about this in the file at *Areas\HelpPage\App_Start\HelpPageConfig.cs*.
b. In the file Areas\HelpPage\SampleGeneration\HelpPageSampleGenerator.cs, there is a method called WriteSampleObjectUsingFormatter which uses the application's formatter instances to write the samples. Here you would need to create new formatter instances having similar settings as your normal application has(so that they reflect the exact serialization/deserialization semantics that your application would normally react to when actual requests are made) and then try to hide the properties which you want to. We want to create new instances because we do not want to disturb the normal functioning of the application.
Example: In case of Json, you could create a new Json formatter instance and provide a ContractResolver which can hide the properties. Check this link: http://james.newtonking.com/projects/json/help/html/ConditionalProperties.htm
In case of Xml, I am not sure how we can hide properties without using the IgnoreDataMember attribute and also being non-intrusive.
Currently I would prefer option 'a' as its comparatively a simple workaround than 'b'.
ASP.NET WEB API uses Json.NET for JSON and DataContarctSerailizer for XML formatting so if you add [JsonIgnore] annotations over properties that you do not want included in your serialization should work just fine.
I have a RESTful URL that requires either the offset or the prefix request parameter (but not both).
GET /users?offset=0&count=20
GET /users?prefix=J&count=20
What's the best way to enforce this rule? Spring has the #RequestParam annotation with the 'required' property for optional parameters, but I want to enforce an "either-or" rule on these two parameters. I know I could do it in the code, but is there another way to do it?
Also, what's the proper way to handle "impractical" requests? Say I have 100 million users; the following request, although properly RESTful, is not something I want to support:
GET /users <-- Gets all 100 million users, crashes server and browser!
What should I send back?
You can create two methods and choose one of them with #RequestMapping's params attribute:
#RequestMapping(..., params = {"prefix", "!offset"})
public String usersWithPrefix(#RequestParam("prefix") ...) { ... }
#RequestMapping(..., params = {"offset", "!prefix"})
public String usersWithOffset(#RequestParam("offset") ...) { ... }
what's the proper way to handle "impractical" requests?
The lesser-practiced principles of REST include the requirement that resources be "discoverable". If you are asked for a complete list of 800 million users and you don't want to provide it, you might instead consider serving a page that describes in some way how to filter the collection: for example, an XForms document or HTML containing a FORM element with fields for offset/prefix/count, or a URI template with the appropriate parameters
Or you could just send a "413 Entity too large" error - edit: no you can't. Sorry, I misread the description of whath this code is for
If you decide to go down the route of just sending the first page, I think I would send it as an HTTP redirect to /users?offset=0&count=20 so that the client has a better idea they've not got the full collection (and if your response contains a link to access subsequent pages, even better)