In an architecture implementing the "event-sourcing" pattern,
there might be historical events with invalid schema within the "event store".
The resulting problem:
upon event-replay those invalid historical events will NOT be understood by the consumers.
Also the depending events will NOT be understood by their consumers.
What is the usual approach to fix the situation? I can think about:
fix events in the "event store", keeping the same event timestamp
remove events with invalid schema from "event store", issue compensating events with current timestamp
What to do with the ecosystem, which should react on changes in the history?
I can think about:
notify consumers of corrected events, let them react in the present
notify consumers of corrected events, let them fix the history in "event store"
Related
I'm quite new to DDD and Event Driven architectures. And after searching this issue I did not manage to get a clear answer by myself, so I'll appreciate if someone can clarify this.
Suppose I currently have an app with 2 bounded contexts. ContextA emits some InterestingEvent that propagates through an integration event to ContextB. This allows ContextB to have its own representation about some eventual state each time this event is emitted and picked up.
So far so good.
But this happy path assumes that ContextA and ContextB have always existed or ContextB has always been subscribed to the InterestingEvent.
Suppose now ContextA existed a while before ContextB... or they both existed, but ContextB was not subscribed initially to InterestingEvent. As a result from this, a number of InterestingEvents have already happened when ContextB goes online or subscribes to the event.
What are the strategies for ContextB to deal with this "first sync" or "past events" situation?
Some crazy thoughts:
Is ContextA suppose to replay all events ContextB requires to be up to date? That could be problematic in the sense that some of this events may have side effects in other contexts as well.
Do ContextB may ask for an update batch from ContextA at bootstrap? Kind of a fat-full-state Godzilla event that carries all the state ContextB needs to be on sync?
Thanks in advance.
What are the strategies for ContextB to deal with this "first sync" or "past events" situation?
The most common strategy I've seen in expert discussion is that new subscribers pull event histories, rather than expecting them to be pushed.
In other words, the basic idea is that the thing-being-subscribed-to is an append-only ordered sequence of events; as the subscriber, I track some token that identifies the most recent event that I have seen, and when I'm ready for new events I ask the provider "Have there been any events since [token]?"
For new subscribers, the basic idea is the same, except that we use some special token/message that means "I haven't seen any events yet".
Of course, event histories are not the only way you can copy information across a boundary; in some domains, it will make sense to ask for a snapshot of information aka import aka bulk transfer. That representation of information can, of course, include in it some metadata that the representation is valid as at [token], and the subscriber, after processing the import, and then ask if there have been any updates since token.
Recommended viewing: Polyglot Data (Greg Young, 2014)
How to deal with correlated events in an Event Driven Architecture? Concretely, what if multiple events must be triggered in order for some action to be performed. For example, I have a microservice that listens to two events foo and bar and only performs an action when both of the events arrive and have the same correlation id.
One way would be to keep an internal data structure inside the microservice that does the book keeping and when everything is satisfied an appropriate action is triggered. However, the problem with this approach is that the microservice is not immutable anymore.
Is there a better approach?
A classic example is where an order comes in at sales and an event is published. Both Finance and Shipping are subscribed to the event, but shipping is also subscribed to the event coming from finance.
The funny thing is that you have no idea on the order in which the messages arrive. The event from sales might cause a technical error, because the database is offline. It might get queued again or end up in an error queue for operations to retry it. In the meantime the event from finance might arrive. So theoretically
the event from sales should arrive first and then the finance event, but in practice it can be the other way around.
There are a number of solutions here, but I've never liked the graphical ones. As a .NET developer I've used K2 and Windows Workflow Foundation in the past, but the solutions most flexible are created in code, not via a graphical interface.
I currently would use NServiceBus or MassTransit for this. On a sidenote, I currently work at Particular Software and we make NServiceBus. NServiceBus has Sagas for this kind of work (documentation) and you can also read on my weblog about a presentation, incl. code on GitHub.
The term saga is kind of loaded, but it basically handles long running (business) processes. Gregor Hohpe calls it a Process Manager (link).
To summarize what sagas do : they are instantiated by incoming messages and have state. Incoming messages are bound/dispatched to a specific saga instance based on a correlationid, for example a customer id or order id. Once the message (event) is processed, state is stored until a new message arrives, or until the code marks the saga as completed and the state is removed from storage.
As said, in the .NET world MassTransit and NServiceBus support this, but there are most likely alternatives in other environments.
If i understand correctly, it looks like you need a CEP ( complex event processor), like ws02 cep or other , which does exactly that.
cep's can aggregate events and perform actions when certain conditions
have been met.
Using the Pub/Sub model with NSB, the following two scenarios seemingly cause the dead-letter queue to fill up, eventually resulting in a "Insufficient resources" error.
1) Publishing an event type that has no subscribers
2) Subscriber is offline
For our purposes we are not interested in historical events when the subscriber starts up, so the incoming queue is purged on startup. Events published while the subscriber is offline fill up the dead-letter queue, however.
Have i misunderstood the command vs. event? This is the behaviour i was expecting from Commands, but expected events to disappear if not subscribed to.
When using NServiceBus, events are considered just as important as commands, and thus are subject to the same guarantees regarding durability, delivery, etc.
So, if your subscriber does not care about events when it is offline, it could unsubscribe before shutting down - this way, it's an explicit decision made by your subscriber that it does not care about what happens when it's not around to hear it... just make sure that it doesn't get confused or chokes somehow if there's a few (old) events lying in its input queue when it comes back online later on, because stuff might get published in the time between the unsubscribe message is sent and it gets to the publisher.
Another option is to supply the [TimeToBeReceived(...)] attribute on your event messages, but that should only be used if it can be safely determined that the event contents lose their relevance after a fixed time for all subscribers.
I've heard that the '$signup' event in Mixpanel is somehow special. What exactly does it do?
All I've been able to find are very vague statements like "there will be a slight improvement in performance for your report" and "$signup is particularly useful for retention analysis."
There is no need to send the "$signup" event since ~2012. The special "$signup" event was hardcoded as the first event in Retention reports, but the query API and web app allow users to specify any event.
This should be easy to follow, but after some reading I still can find an answer.
So, say that the user needs to change his mobile number, to accomplished that, we might have a command as: ChangedUserMobileNumber
holding the new number. The domain responsible for handling the command will perform the change in the aggregate and publish an event: UserMobilePhoneChanged
There is a subscriber for that event in another domain, which also holds the user mobile number in its aggregate but according to our software architect, events can not old any data so what we end up is rather stupid to say the least:
The Domain 1, receives the command to update the mobile number, the number is updated and one event is published, also, because the event cannot hold data, the command handler in the Domain 1 issues yet another command which is sent to Domain 2. The subscriber of that event lives in Domain 2 too, we then have a Saga to handle both the event and the command.
In terms of implementation we are using NServiceBus, so we have this saga to handle these message and in it we have this line of code, where the entity.IsMobilePhoneUpdated field stored in a saga entity is changed when the event is handeled.
bool isReady = (entity.IsMobilePhoneUpdated && entity.MobilePhoneNumber != null);
Effectively the Saga is started by both the command and the event raised in the Domain 1, and until this condition is met, the saga is kept alive.
If it was up to me, I would be sending the mobile number in the event itself, I just want to get a few other opinions on this.
Thanks
I'm not sure how a UserMobilePhoneChanged event could be useful in any way unless it contained the new phone number. User asks to change a number, the event shoots out that it has. Should be very simple indeed. Why does your architect say that events shouldn't contain any information?
In the first event based system i've designed events also had no data. I also did enforce that rule. At the time that sounded like a clever decision. After a while i realised that it was dumb, and i was making a lot of workarounds because of it. Also this caused a lot of querying form the event subscribers, even for trivial data. I had no problem changing this "rule" after i realised i'm doing it wrong.
Events should have all the data required to make them meaningful. Also they should only have the data that makes sense for that event. ( No point in having the user address in a ChangePhoneNumber message )
If your architect imposes such a restriction, it's not going to be easy to develop a CQRS system. How are the read models updated? Since the events have no data then you either query something to get the data ( the write side ? ) of find some way of sending a command to the read model ( then what's the point of publishing events? ). To fix your problem you should try to have a professional discussion with this architect, preferably including other tech heads and without offending anybody try to get him to relax this constraint.
On argument you could use is Event Sourcing. Event Sourcing is complementary to CQRS and would not make sense without events that have data. Even more when using event sourcing, the only data you have is the data stored in the events. Even if you don't actually implement event sourcing you can use it's existence as a reason for events to have data.
There is little point in finding a technical solution to a people problem.