I have been reading a lot about Event Driven Architecture and it makes a lot of sense to me, but the issue of giving users immediate feedback is confusing me.
Say there is a service ('EmployeeService') that holds a list of all employees. The business logic for creating an employee is located in this service.
The UI from another system uses this service. The requirement is (whether you like it or not) that it has a grid of employees, and an 'add employee button' which brings up a form, and when you submit the form, it returns you to the gird with the new employee in it. The grid show derived fields which are calculated by the service (this is important to note!).
Traditionally, on submit, I would have displayed a loading screen, synchronously sent a WCF request to register the employee, and when that had completed, and forwarded to the grid (which would definately by now have the new employee).
With EDA, on submit, I would 'fire and forget' a command to register the user - but then what? I could forward to the grid but there is a chance the new employee might not be in there yet? I could manually add to the grid assuming all will be OK, but how do I display the derived data calculated by the service? Or maybe I could have a 'new employee pending graphic' displayed on the grid if it has not been created yet, then make the page check back every few secons until it has?
This is a common scenario so what is the common solution for this?
You can register a callback when sending the command and block until the command is complete.
If you've downloaded the NServiceBus package, just reference the AsyncPagesMvc3 sample solution. It has an example of exactly what you're looking for.
If your EmployeeService is a SOA service then the Add Employee Button also belongs to the EmployeeService. The user interface is simple a composite of multiple services. You can deploy part of the EmployeeService locally on the client which handles the employee creation and the calculation (if the calculation is not a complex one).
For example:
public class AddEmployeeView
{
public IBus Bus { get; set; }
public void AddNewClicked() {
// async calculate
// store directly in the employee service database
// or dispatch command internally
// refresh employee list as the service is the only owner of that data
Bus.Publish<NewEmployeeAdded>(m => { });
}
}
So the above AddEmployeeView belongs to the EmployeeService. The EmployeeService alone has knowledge how to calculate and store new employees (even in its own database) and is the only logical publisher of the NewEmployeeAdded event. And here goes away your complexity.
Related
I use event sourcing to store my object.
Changes are captured via domain events, holding only the minimal information required, e.g.
GroupRenamedDomainEvent
{
string Name;
}
GroupMemberAddedDomainEvent
{
int MemberId;
string Name;
}
However elsewhere in my application I want to be notified if a Group is updated in general. I don’t want to have to accumulate or respond to a bunch of more granular and less helpful domain events.
My ideal event to subscribe to is:
GroupUpdatedIntegrationEvent
{
int Id;
string Name;
List<Member> Members;
}
So what I have done is the following:
Update group aggregate.
Save down generated domain events.
Use these generated domain events to to see whether to trigger my integration event.
For the example above, this might look like:
var groupAggregate = _groupAggregateRepo.Load(id);
groupAggregate.Rename(“Test”);
groupAggregate.AddMember(1, “John”);
_groupAggregateRepo.Save(groupAggregate);
var domainEvents = groupAggregate.GetEvents();
if (domainEvents.Any())
{
_integrationEventPublisher.Publish(
new GroupUpdatedIntegrationEvent
{
Id = groupAggregateId,
Name = groupAggregate.Name,
Members = groupAggregate.Members
});
}
This means my integration events used throughout the application are not coupled to what data is used in my event sourcing domain events.
Is this a sensible idea? Has anyone got a better alternative? Am I misunderstanding these terms?
Of course you're free to create and publish as many events as you want, but I don't see (m)any benefits there.
You still have coupling: You just shifted the coupling from one Event to another. Here it really depends how many Event Consumers you've got. And if everything is running in-memory or stored in a DB. And if your Consumers need some kind of Replay mechanism.
Your integration Events can grow over time and use much bandwidth: If your Group contains 1000 Members and you add 5 new Members, you'll trigger 5 integration events that always contain all members, instead of just the small delta. It'll use much more network bandwidth and hard drive space (if persisted).
You're coupling your Integration Event to your Domain Model. I think this is not good at all. You won't be able to simply change the Member class in the future, because all Event Consumers depend on it. A solution could be to instead use a separate MemberDTO class for the Integration Event and write a MemberToMemberDTO converter.
Your Event Consumers can't decide which changes they want to handle, because they always just receive the full blown current state. The information what actually changed is lost.
The only real benefit I see is that you don't have to again write code to apply your Domain Events.
In general it looks a bit like Read Model in CQRS. Maybe that's what you're looking for?
But of course it depends. If your solution fits your application's needs, then it'll be fine to do it that way. Rules are made to show you the right direction, but they're also meant to be broken when they get in your way (and you know what you're doing).
Say my AggregateRoot is the Order-Model. There is a collection with OrderItems (Entities).
I have only one Repository for the AggregateRoot (Order) but not for OrderItems.
What should I do when the client wants to update only a small change like the Remarks-field on one OrderItem?
My current understanding ist that the client sends the update by a DTO. Then the middleware loads the whole Order, then update the single detail, and commit the whole Order to repository.
If I understood it correctly is that a good practice in real life or do you handle it differnently? It sounds not performant and not maintenance friendly to me.
As everything in DDD the answer lays in Domain rules. Everything has to gravitated arround rules not arround data structures.
Warning: Too simplistic example below!
You have to chage the Remarks field of one order item so make you a question: What restrictions and invariats has the Remarks field changing operation? Has OrderItem all info it needs for this? If yes then in this case OrderItem is your aggregate root.
Are some of Remarks not allowed into a OrderItem because it belongs to some type of Order but other Order types allows this Remarks. Then the Order is your aggregate root.
This gives you a clue about how you have to approach it BUT as your comment loading a Order with all OrderItems just to change one OrderItem Remark is absolutely not performant.
“I’m sorry that I coined the term ‘objects,’ because it gets many
people to focus on the lesser idea. The big idea is ‘messaging’” ~
Alan Kay
Remember that I said that DDD has to gravitated arround rules and not arround data structures?
So, do not think about data structures. Model everything arround the commands and the events (the messages) and the rules. Make your persistence repositories bring the appropiate Aggregate Root for that command, use the AR to apply the command, return a domain event whith the changes produced and use that event to persist the new system state and notify other services about the change.
Code example from Aggregate root invariant enforcement with application quotas
class ApplicationService{
public void registerUser(RegisterUserCommand registerUserCommand){
var user = new UserEntity(registerUserCommand.userData); //avoid wrong entity state; ctor. fails if some data is incorrect
RegistrationAggregate agg = aggregatesRepository.Handle(registerUserCommand); //handle is overloaded for every command we need. Use registerUserCommand.tenantId to bring total_active_users and quota from persistence, create RegistrarionAggregate fed with TenantData
var userRegisteredEvent = agg.registerUser(user); //return domain changes expressed as a event
persistence.Handle(userRegisteredEvent); //handle is overloaded for every event we need; open transaction, persist userRegisteredEvent.fromTenant.total_active_users where tenantId, optimistic concurrency could fail if total_active_users has changed since we read it (rollback transaction), persist userRegisteredEvent.user in relationship with tenantId, commit transaction
eventBus.publish(userRegisteredEvent); //notify external sources for eventual consistency
}
This allows you to bring a OrderItemRemarkManagerAggregate into memory from persistence that has just the info you need to chage the Remarks (i.e. OrderItem ID, current Remarks, OrderItem status, OrderType belonged, etc); just use it to apply the operation and apply the changes into persistence.
Later you could worry about reusing an aggregate for several operations (allways in the same Bounded Context of course) or even refactor as you need.
Scenario:
I have 2 Microservices (which both use CQRS + Event Sourcing internally)
Microservice 1 manages Contacts (= Aggregate Root)
Microservice 2 manages Invoices (= Aggregate Root)
The recipient of an invoice must be a valid contact.
CreateInvoiceCommand:
{
"content": "my invoice content",
"recipient": "42"
}
I now read lot's of times, that the write side (= the command handler) shouldn't call the read side.
Taking this into account, the Invoices Microservice must listen to all ContactCreated and ContactDeleted events in order to know if the given recipient id is valid.
Then I'd have thousands of Contacts within the Invoices Microservice, even if I know that only a few of them will ever receive an Invoice.
Is there any best practice to handle those scenarios?
The recipient of an invoice must be a valid contact.
So the first thing you need to be aware of - if two entities are part of different aggregates, you can't really implement "apply a change to this entity only if that entity satisfies a specification", because that entity could change between the moment you evaluate the specification and the moment you perform the write.
In other words - you can only get eventual consistency across an aggregate boundary.
The aggregate is the authority for its own state, but everything else (for example, the contents of the command message), it pretty much has to accept that some external authority has checked the data.
There are a couple approaches you can take here
1) You can blindly accept that the recipient specified in the command is valid.
2) You can try to verify the validity of the recipient from some external authority (aka: a read model of some other aggregate) between receiving it from the untrusted source and submitting it to the domain model.
3) You can blindly accept the command as described, but treat the invoice as provisional until the validity of the recipient is confirmed. That means there is a second command to run on the invoice that certifies the recipient.
Note - from the point of view of the model, these different commands are equivalent, but at the application layer they don't need to be -- you can restrict access to the command to trusted sources (don't make it part of the public api, require authorization that is only available to trusted sources, etc).
Approach #3 is the most microservicy, as the two commands can be separated in time -- you can accept the CreateInvoice command as soon as it arrives, and certify the recipient asynchronously.
Where would you put approach 4), where the Invoices Microservice has it's own Contacts Store which gets updated whenever there's a ContactCreated or ContactDeleted event? Then both entities are part of the same service and boundary. Now it should be possible to make things consistent, right?
No. You've made the two entities part of the same service, but the problem was never that they were in different services, but that they are in separate aggregates -- meaning we can be changing the entity states concurrently, which means that we can't ensure that they are immediately synchronized.
If you wanted immediate consistency, you need a model that draws your boundaries differently.
For instance, if the invoice entities were modeled as part of the Contacts aggregate, then the aggregate can ensure the invariant that new invoices require a valid recipient -- the domain model uses the copy of the state in memory to confirm that the recipient was valid when we loaded, and the write into the book of record verifies that the book of record hadn't changed since the load happened.
The write of the aggregate state is a compare-and-swap in the book of record; if some concurrent process had invalidated the recipient, the CAS operation would fail.
The trade off, of course, is that any change to the Contact aggregate would also cause the invoice to fail; concurrent editing of different invoices with the same recipient goes out the window.
Aggregates are all or nothing; they aren't separable.
Now, one out might be that your Invoice aggregate has a part that must be immediately consistent with the recipient, and another part where eventually consistent, or even inconsistent, is acceptable. In which case your goal is to refactor the model.
The recipient of an invoice must be a valid contact.
This is a business rule. The question should be asked, what does this business rule mean for my application? Who should take responsibility for implementing this rule, or can the responsibility be shared?
One possibility is that, yes, the business rule is about invoices so it should be the responsibility of the Invoices Service to implement it.
However, the business rule is really about the creation of invoices. And the owner of invoice creation in your architecture is, strangely, not the Invoices Service. The reason for this is that the name of the command is CreateInvoiceCommand.
Let's think about this - the Invoices Service will never just create an invoice on its own. It just provides the capability. In this architecture, the actual owner of invoice creation is the sender of the command.
Using this line of reasoning, if the business rule is saying that invoice creation cannot happen against an invalid recipient, then it becomes the responsibility of the command sender to ensure this business rule is implemented.
This would be a very different scenario if, rather than receiving a command, the Invoices Service subscribed to events. As an example, an event called WidgetSold. In this scenario, the owner of invoice creation clearly would be the Invoicing service, and so the business rule would be implemented there instead.
If the user clicks the create invoice for contact 42 button, it's the
user's responsibility to take care that contact 42 exists
Yes, that is correct. The user's intention is to create an invoice. The business rules around invoice creation should, therefore, be enforced at this point. How this happens (or whether this happens at all) is a different question.
But what if the user doesn't care? Then it would create an invoice
with an invalid recipient id.
Also correct. As you say, there are side-effects to this approach, one of which is that you can end up with inconsistent data across your system. That is one of the realities of SOA.
Isn't this somehow similar to this: The Invoice has a currencyCode
property, it's a String.
I don't know if I agree or not. Is asking is this a valid ISO currency? different to asking is entity 42 valid according to another system?. I would think so.
Isn't it kinda the same as given recipient is not null and is valid
according to my Contacts Database?
I agree that in reality, you could implement this validation in the service. I am just saying that I don't think it's the right place for it. If you wanted to do this, you would have to either call out the another service or store all contacts locally, as you framed your question originally. I think it's simpler to just do it outside of the service.
I think that the answer depends on how resilient you want the system to be, that is, how to handle the situation in wich the Contacts Microservice is down (not responding or very slow).
1. You want to be very resilient
If the Contacts Microservice is down, you want to be able to emit invoices for some (maybe most) of the contacts. In this case you listen to the ContactCreated and ContactDeleted and maintain a (eventually consistent) local list of valid contacts; they should be named accordingly to the Ubiquitous language in this bounded context, like Payers (or something like that). Then, in the Application layer, when building the CreateInvoiceCommand you check that Payer is valid and create the command.
2. You don't need to be resilient
If the Contacts Microservice is down, you refuse to generate invoices. In this case, when building the command you make a request to the Invoices Microservice API endpoint and verify that the Payer is valid.
In any case, you check for contact's validity before the command is dispatched.
Say I have a TodoStore. The TodoStore is responsible for keeping my TODO items. Todo items are stored in a database.
I want to know what is the recommended way for loading all todo items into the store and how the views should interact with the store to load the TODO items on startup.
The first alternative is to create a loadTodos action that will retrieve the Todos from the database and emit a TODOS_LOADED event. Views will then call the loadTodos action and then listen to the TODOS_LOADED event and then update themselves by calling TodoStore.getTodos().
Another alternative is to not have a loadTodos action, and have a TodoStore.getTodos() that will return a promise with the existing TODO items. If the TodoStore has already loaded the TODO items, it just returns them; if not, then it will query from the database and return the retrieved items. In this case, even though the store now has loaded the TODO items, it will not emit a TODOS_LOADED event, since getTodos isn't an action.
function getTodos() {
if (loaded)
return Promise.resolve($todoItems);
else
return fetchTodoItemsFromDatabase().then(todoItems) {
loaded = true;
$todoItems = todoItems;
return $todoItems;
});
}
I'm sure many will say that that breaks the Flux architecture because the getTodos function is changing the store state, and store state should only be changed though actions sent in from the dispatcher.
However, if you consider that state for the TodoStore is the existing TODO items in the database, then getTodos isn't really changing any state. The TODO items are exactly the same, hence no view need to be updated or notified. The only thing is that now the store has already retrieved the data, so it is now cached in the store. From the View's perspective, it shouldn't really care about how the Store is implemented. It shouldn't really care if the store still needs to retrieve data from the database or not. All views care about is that they can use the Store to get the TODO items and that the Store will notify them when new TODO items are created, deleted, or changed.
Hence, in this scenario, views should just call TodoStore.getTodos() to render themselves on load, and register an event handler on TODO_CHANGE to be notified when they need to update themselves due to a addition, deletion, or change.
What do you think about these two solutions. Are they any other solutions?
The views do not have to be the entities that call loadTodos(). This can happen in a bootstrap file.
You're correct that you should try your best to restrict the data flow to actions inside the dispatch payload. Sometimes you need to derive data based on the state of other stores, and this is what Dispatcher.waitFor() is for.
What is Flux-like about your fetchTodoItemsFromDatabase() solution is that no other entity is setting data on the store. The store is updating itself. This is good.
My only serious criticism of this solution is that it could result in a delay in rendering if you are actually getting the initial data from the server. Ideally, you would send down some data with the HTML. You would also want to make sure to call for the stores' data within your controller-views' getInitialState() method.
Here is my opinion about that, very close to yours.
I maintain the state of my application in Store via Immutable.Record and Immutable.OrderedMap from Immutable.js
I have a top controller-view component that get its state from the Store.
Something such as the following :
function getInitialState() {
return {
todos: TodoStore.getAll()
}
}
TodoStore.getAll methods will retrieve the data from the server via a APIUtils.getTodos() request if it's internal _todos map is empty. I advocate for read data triggered in Store and write data triggered in ActionCreators.
By the time the request is processing, my component will render a simple loading spinner or something like that
When the request resolves, APIUtils trigger an action such as TODO_LIST_RECEIVE_SUCCESS or TODO_LIVE_RECEIVE_FAIL depending on the status of the response
My TodoStore will responds to these action by updating its internal state (populating it's internal Immutable.OrderedMap with Immutable.Record created from action payloads.
If you want to see an example through a basic implementation, take a look to this answer about React/Flux and xhr/routing/caching .
I know it's been a couple of years since this was asked, but it perfectly summed up the questions I am struggling with this week. So to help any others that may come across this question, I found this blog post that really helped me out by Nick Klepinger: "ngrx and Tour of Heroes".
It is specifically using Angular 2 and #ngrx/store, but answers your question very well.
I am having a register form that user fill in, once the form fill in, the registration is postponed till the user recieve an email to check that his email is ok, then when he reply the email, the registration is fully proceed.
I would like also the process to be automatic, without the need for the user to entered any number ect, just the need for him to click on the email to confirm, and automatic on my side too.
I want the process to be available to some others task than the registration process as well if possible.
Do I need to do all way check/123456 with the 123456 generated or is there a better way to do, a component, nuget that can do the job ? you experiences return would be helpful
thanks
You appear to be on the right track.
I do a similar thing quite often. It isn't generic since I do not need it but you can change that quite easily.
Once a user registers I send an activation e-mail that contains a link that when clicked navigates to something such as http://domain/member/activate/id where the id is a GUID. That is all I need. I look up the member id and activate it. Chances of someone guessing a new member's id are rather slim and even if they do it is very far from a train smash.
From what I can tell you need something more generic and a bit more security. You could always create some key (like a new GUID) that you store along with the user and then the activation calls: http://domain/member/activate/id?key={the guid}. You could even encrypt the key. But that is up to you.
For something that can be re-used and is more generic you could create an activation component that contains a list of activation ids. But to then activate a specific type of entity (such as user registration or form validation) you would need to have a way to perform that on the back-end. For example, when you call http://domain/activation/activate/id internally is looks up the id and gets the type. The activation request is, of course, created when, e.g., your user registers. You would probably want to store some expiry/timeout since a user may never activate.
For each type you could have an activation mechanism on the relevant controller: member/activate/key or formvalidation/activate/key. So when you activation controller receives a request to activate it has a list of routes conigured per type and the navigates to the relevant route to perform the 'actual' activation.
Another option is to have an in-process component perform activation e.g.: an IActivator interface implemented by a MemberActivator or FormValdationActivator and then have an IActivatorProvider implementation that gives you the relevant provider based on a type (.NET Type or a string type you provider --- that's up to you).
You could even pass an ActivationRequestCommand along to a service bus endpoint if you are so inclined.
So you have many options. HTH
If you are a developer I suggest you at least try to start with the code... then ask again if you get stuck providing a sample of what you've done.
What you are looking to do is basically use a temporary url that posts to a page to complete the registration...