I'm trying to use gRPC to build a simple CRUD service, but I keep finding myself creating messages with big overlaps.
This is best described by an example:
message Todo {
// id is only available for a persisted entity in database.
string id = 1;
string content = 2;
// this is only available for users with admin role.
string secret_content = 3;
}
service Todos {
rpc CreateTodo(CreateRequest) returns (CreateResponse) {}
rpc ReadTodo(ReadRequest) returns (ReadResponse) {}
}
message CreateRequest {
// this todo is not supposed to have id,
// should I create another version of Todo without an id field?
Todo todo
}
message CreateResponse {
// this todo will always have an id.
Todo todo = 1;
}
message ReadRequest {
string id = 1;
}
message ReadResponse {
// this todo should only have the secret_content field if the
// user is authenticated as an admin, if not, the field should not
// fallback to the zero value, the whole field must be missing.
Todo todo = 1;
}
Is this a good approach to build a CRUD like resource with gRPC? That is, having a single message (Todo) representing the resource, and wrapping this message in response/request types per action.
Should the Todo type message have all fields covered by all requests/responses, and not set the ones which are not in use by each?
Should the Todo type message have all fields covered by all requests/responses, and not set the ones which are not in use by each?
Yes, this seems like a reasonable design. In protobuf v2, you would have marked such fields optional to make it easier to understand. But in v3, all fields are optional by default anyway.
Related
Protobuf provides service keyword that defines rpc-interface of one application.
I also want to use concept of entity which means that is part of service (one service contains multiple entities). Each entity type has own unique identifier that gives possibility to address different entities in service.
I would like to use proto like this
message UserReq {
string username = 1;
string password = 2;
}
message RegReq {
uint8 result_code = 1;
}
message RemoteEntityInterface
{
MyEntity entity = 1;
}
message GiveItemResult
{
uint8 result_code = 1;
}
service MyService {
rpc RegisterUser (UserReq) returns (RegReq) {}
rpc Login(UserReq) returns (RemoteEntityInterface) {}
}
entity MyEntity
{
rpc GiveItem (GiveItemReq) returns (GiveItemResult) {}
}
As you can see in example, I used unknown for protobuf keyword entity, this keyword means that MyService can return the interface to some remote object (MyEntity) by using Login remote method.
What are the ways to do this? (maybe write plugin or known way to modify source code of protobuf). Or maybe there are more flexible solutions than protobuf?
I also would like to use multiple parameters per one rpc; adding java-like attributes to rpc; service and entity; and data-model for entity (variables/fields) to add real-time replication support from entity to another service.
I think it is very flexible for services in game-development.
The only official way to extend .proto syntax is to define custom options.
For example, you could have something like:
extend google.protobuf.ServiceOptions {
optional bool is_entity = 123456;
}
service MyEntity
{
option (is_entity) = true;
rpc GiveItem (GiveItemReq) returns (GiveItemResult) {}
}
The default code generator will not do anything special with this option, but you can access it from your own code and from a protoc plugin if you write one.
Given that i have multiple models, each needed to have their own create/get/get list API.
Do i need to add two different types of messages (single and list) for every model?
For example :
If i have a student type -
message Student{
string name = 1;
}
and a rpc:
rpc CreateStudent(Student) returns (google.protobuf.Empty){
..............
}
If i'd like to add a rpc to create a list of students, or get a list of students
rpc CreateStudends(??????) returns (google.protobuf.Empty){
..............
}
rpc GetAllStudents() returns (??????){
..............
}
Do i need to also define
message StudentList{
repeated Student students = 1;
}
Or is there a way to use a list type directly in the message input/output?
Yes, basically - you would want a different message type per element type, or maybe a single root type with a oneof style content. Raw protobuf does not include a concept of generics or templates.
Some libraries do, but: that's outside of the specification.
You can simply add the stream keyword to your RPCs. No need to define a message field as repeated, stream will send or receive multiple independent messages.
message Student {
string name = 1;
}
with RPCs:
rpc CreateStudent(Student) returns (google.protobuf.Empty) {
..............
}
rpc CreateStudents(stream Student) returns (google.protobuf.Empty) {
..............
}
rpc GetAllStudents() returns (stream Student) {
..............
}
It's good practice to send/stream a response object rather than empty. Otherwise, you only have the gRPC response code to indicate a problem and will need to reference the logs to debug.
I've configured a FileNet workflow subscription on Add, Update and Delete events. The workflow calls a Java component to send a notification message (to a third party).
We would like to see "before" and "after" property values in the notification message for "Update" events.
The "Event" object that triggers the subscription has a "Modified Properties" member, so I was hoping I could just create a corresponding "ModifiedProperties" string array in the workflow, and have the subscription map "Update.ModifiedProperties = ModifiedProperties". Unfortunately, the Event's "ModifiedProperties" only gives the NEW value, not the "before" value.
<= So I don't see any way to get "before/after" values directly from the subscription...
It looks like the "UpdateEvent" object also has an "OriginalObject" member ... and I might be able to use the Java API to get the "before" value from the OriginalObject.
Q: Does this sound plausible method for getting the before/after document property values?
Q: Any ideas how to pass the "OriginalObject" object from the subscription to the workflow, so the Java component can use it?
The target platform is P8 5.2.1; I'm developing on P8 5.5.
You are right, the only way to the original values is through the OriginalObject object. And the quickest way to get data to a workflow is using a subscribable object.
Therefore, a solution to your problem is to define a custom object containing the properties describing the new and the old property values. You create this custom object in a custom event handler triggered on an update event from the document. Here you can populate the properties of the custom object using the original object:
Document document = (Document) event.get_OriginalObject();;
Iterator<?> iterator = event.get_ModifiedProperties().iterator();
while (iterator.hasNext()) {
String modifiedProperty = (String) iterator.next();
// TODO: Fetch the values from the original object
// and set them on the custom object. The details depend
// on the data structure you choose.
}
Next you create a Workflow subscription triggered on the creation of the custom object. You can map the properties of your custom object to the data fields of your workflow. In the workflow that is started you can define an attachment and specify that the custom object is the initiating attachment. Using the CE_Operation queue methods you can now and delete the custom object when your processing is finished.
if(objEvent instanceof UpdateEvent) { try { String strModifiedProperties = ""; UpdateEvent updateEvent = (UpdateEvent) objEvent; StringList propertyNames = updateEvent.get_ModifiedProperties(); Iterator iterModifiedProps = propertyNames.iterator(); while(iterModifiedProps.hasNext()) { String modifiedProperty = (String) iterModifiedProps.next(); strModifiedProperties = strModifiedProperties+modifiedProperty+","; } strModifiedProperties = strModifiedProperties.substring(0, strModifiedProperties.lastIndexOf(",")); } catch (Exception e) { System.out.println("onEvent : Exception while executing UpdateEvent: "+e.getMessage()); } }
PUT, DELETE, POST can be operated as shown below.
By the way, I do not know how to do GET.
Please help me.
// PUT & DELETE (mapped to WRITE, DELETE of MD-SAL)
public void onDataTreeChanged(Collection<DataTreeModification<GreetingRegistry>> changes) {
for(DataTreeModification<GreetingRegistry> change: changes) {
DataObjectModification<GreetingRegistry> rootNode = change.getRootNode();
if(rootNode.getModificationType() == WRITE) {
...
}
else if(rootNode.getModificationType() == DELETE) {
...
}
}
// POST (mapped to RPC of MD-SAL)
public Future<RpcResult<HelloWorldOutput>> helloWorld(HelloWorldInput input)
{
HelloWorldOutputBuilder helloBuilder = new HelloWorldOutputBuilder();
helloBuilder.setGreeting("Hello " + input.getName());
return RpcResultBuilder.success(helloBuilder.build()).buildFuture();
}
// GET (???)
How should I implement it?
You don't actually have to implement anything for GET in your code, when you want to read from YANG modeled MD-SAL data, the GET method is be available by default, and returns whatever data you ask for in the URL. It is important to point to the correct URL.
If you want to do some processing on data before returning it to the user, you can use RPCs with POST, and do the processing in the RPC based methods. In your example above, you could pot the search keys into HelloWorldInput, do the processing in helloWorld(), and return results in HelloWorldOutput.
I've only began with DDD and currently trying to grasp the ways to do different things with it. I'm trying to design it using asynchronous events (no event-sourcing yet) with CQRS. Currently I'm stuck with validation of commands. I've read this question: Validation in a Domain Driven Design , however, none of the answers seem to cover complex validation across different aggregate roots.
Let's say I have these aggregate roots:
Client - contains list of enabled services, each service can have a value-object list of discounts and their validity.
DiscountOrder - an order to enable more discounts on some of the services of given client, contains order items with discount configuration.
BillCycle - each period when bills are generated is described by own billcycle.
Here's the usecase:
Discount order can be submitted. Each new discount period in discount order should not overlap with any of BillCycles. No two discounts of same type can be active at the same time on one service.
Basically, using Hibernate in CRUD style, this would look something similar to (java code, but question is language-agnostic):
public class DiscountProcessor {
...
#Transactional
public void processOrder(long orderId) {
DiscOrder order = orderDao.get(orderId);
BillCycle[] cycles = billCycleDao.getAll();
for (OrderItem item : order.getItems()) {
//Validate billcycle overlapping
for (BillCycle cycle : cycles) {
if (periodsOverlap(cycle.getPeriod(), item.getPeriod())) {
throw new PeriodsOverlapWithBillCycle(...);
}
}
//Validate discount overlapping
for (Discount d : item.getForService().getDiscounts()) {
if (d.getType() == item.getType() && periodsOverlap(d.getPeriod(), item.getPeriod())) {
throw new PeriodsOverlapWithOtherItems(...);
}
}
//Maybe some other validations in future or stuff
...
}
createDiscountsForOrder(order);
}
}
Now here are my thoughts on implementation:
Basically, the order can be in three states: "DRAFT", "VALIDATED" and "INVALID". "DRAFT" state can contain any kind of invalid data, "VALIDATED" state should only contain valid data, "INVALID" should contain invalid data.
Therefore, there should be a method which tries to switch the state of the order, let's call it order.validate(...). The method will perform validations required for shift of state (DRAFT -> VALIDATED or DRAFT -> INVALID) and if successful - change the state and transmit a OrderValidated or OrderInvalidated events.
Now, what I'm struggling with, is the signature of said order.validate(...) method. To validate the order, it requires several other aggregates, namely BillCycle and Client. I can see these solutions:
Put those aggregates directly into the validate method, like
order.validateWith(client, cycles) or order.validate(new
OrderValidationData(client, cycles)). However, this seems a bit
hackish.
Extract the required information from client and cycle
into some kind of intermediate validation data object. Something like
order.validate(new OrderValidationData(client.getDiscountInfos(),
getListOfPeriods(cycles)).
Do validation in a separate service
method which can do whatever it wants with whatever aggregates it
wants (basically similar to CRUD example above). However, this seems
far from DDD, as method order.validate() will become a dummy state
setter, and calling this method will make it possible to bring an
order unintuitively into an corrupted state (status = "valid" but
contains invalid data because nobody bothered to call validation
service).
What is the proper way to do it, and could it be that my whole thought process is wrong?
Thanks in advance.
What about introducing a delegate object to manipulate Order, Client, BillCycle?
class OrderingService {
#Injected private ClientRepository clientRepository;
#Injected private BillingRepository billRepository;
Specification<Order> validSpec() {
return new ValidOrderSpec(clientRepository, billRepository);
}
}
class ValidOrderSpec implements Specification<Order> {
#Override public boolean isSatisfied(Order order) {
Client client = clientRepository.findBy(order.getClientId());
BillCycle[] billCycles = billRepository.findAll();
// validate here
}
}
class Order {
void validate(ValidOrderSpecification<Order> spec) {
if (spec.isSatisfiedBy(this) {
validated();
} else {
invalidated();
}
}
}
The pros and cons of your three solutions, from my perspective:
order.validateWith(client, cycles)
It is easy to test the validation with order.
#file: OrderUnitTest
#Test public void should_change_to_valid_when_xxxx() {
Client client = new ClientFixture()...build()
BillCycle[] cycles = new BillCycleFixture()...build()
Order order = new OrderFixture()...build();
subject.validateWith(client, cycles);
assertThat(order.getStatus(), is(VALID));
}
so far so good, but there seems to be some duplicate test code for DiscountOrderProcess.
#file: DiscountProcessor
#Test public void should_change_to_valid_when_xxxx() {
Client client = new ClientFixture()...build()
BillCycle[] cycles = new BillCycleFixture()...build()
Order order = new OrderFixture()...build()
DiscountProcessor subject = ...
given(clientRepository).findBy(client.getId()).thenReturn(client);
given(cycleRepository).findAll().thenReturn(cycles);
given(orderRepository).findBy(order.getId()).thenReturn(order);
subject.processOrder(order.getId());
assertThat(order.getStatus(), is(VALID));
}
#or in mock style
#Test public void should_change_to_valid_when_xxxx() {
Client client = mock(Client.class)
BillCycle[] cycles = array(mock(BillCycle.class))
Order order = mock(Order.class)
DiscountProcessor subject = ...
given(clientRepository).findBy(client.getId()).thenReturn(client);
given(cycleRepository).findAll().thenReturn(cycles);
given(orderRepository).findBy(order.getId()).thenReturn(order);
given(client).....
given(cycle1)....
subject.processOrder(order.getId());
verify(order).validated();
}
order.validate(new OrderValidationData(client.getDiscountInfos(),
getListOfPeriods(cycles))
Same as the above one, you still need to prepare data for both OrderUnitTest and discountOrderProcessUnitTest. But I think this one is better as order is not tightly coupled with Client and BillCycle.
order.validate()
Similar to my idea if you keep validation in the domain layer. Sometimes it is just not any entity's responsibility, consider domain service or specification object.
#file: OrderUnitTest
#Test public void should_change_to_valid_when_xxxx() {
Client client = new ClientFixture()...build()
BillCycle[] cycles = new BillCycleFixture()...build()
Order order = new OrderFixture()...build();
Specification<Order> spec = new ValidOrderSpec(clientRepository, cycleRepository);
given(clientRepository).findBy(client.getId()).thenReturn(client);
given(cycleRepository).findAll().thenReturn(cycles);
subject.validate(spec);
assertThat(order.getStatus(), is(VALID));
}
#file: DiscountProcessor
#Test public void should_change_to_valid_when_xxxx() {
Order order = new OrderFixture()...build()
Specification<Order> spec = mock(ValidOrderSpec.class);
DiscountProcessor subject = ...
given(orderingService).validSpec().thenReturn(spec);
given(spec).isSatisfiedBy(order).thenReturn(true);
given(orderRepository).findBy(order.getId()).thenReturn(order);
subject.processOrder(order.getId());
assertThat(order.getStatus(), is(VALID));
}
Do the 3 possible states reflect your domain or is that just extrapolation ? I'm asking because your sample code doesn't seem to change Order state but throw an exception when it's invalid.
If it's acceptable for the order to stay DRAFT for a short period of time after being submitted, you could have DiscountOrder emit a DiscountOrderSubmitted domain event. A handler catches the event and (delegates to a Domain service that) examines if the submit is legit or not. It would then issue a ChangeOrderState command to make the order either VALIDATED or INVALID.
You could even suppose that the change is legit by default and have processOrder() directly take it to VALIDATED, until proven otherwise by a subsequent INVALID counter-order given by the validation service.
This is not much different from your third solution or Hippoom's one though, except every step of the process is made explicit with its own domain event. I guess that with your current aggregate design you're doomed to have a third party orchestrator (as un-DDD and transaction script-esque as it may sound) that controls the process, since the DiscountOrder aggregate doesn't have native access to all information to tell if a given transformation is valid or not.