I am using Spring Boot and not much experience on Transactions stuff...
#Service
#Transactional
class FundTransferService {
public void doSomeFunds(){
if(realPaymentGateway()){
//then do db call, to update User Transaction details, WHAT IF SERVER GOES DOWN HERE OR ANY EXCEPTION??
}
}
public boolean realPaymentGateway(){
//Using Braintree to transfer Funds
}
}
Above there are 2 things happening, paymentGateway(which is some rest call) and if success, then only updating DB with User Transaction details.
I want above 2 THINGS should happen Atomic, I mean either both (Rest and DB) shud Success, or rollback everything..
My problem is:
Q1) While Updating DB details, due to some exception or server goes down, then might rollback happen only for DB stuff, but not RESTCALL..
Q2) Should I first update DB and then go for PaymentGateway for transfering funds or reverse should be correct? Please suggest me..
Hope you understand my query, so whats the solution for above problem?
You cannot atomically do what you want to do, like span a distributed transaction, because Braintree is a REST service outside of your control.
You can however, call Braintree, and afterwards update your database accordingly. This way, there's a minimal time-window right between your brain-tree call and your transaction commit, that your server gets unexpectedly "killed", but there's no easy way around that in the scope of this answer.
You could also have some sort of write-ahead-log, where you try and remember what REST service you are going to call and then reconcile with the actual REST calls that happened, but this is a bit more elaborate and quite possibly overkill.
You can apply transactions only for transactional resources. As I understood you have 1 non-transactional input (further, TX) (call with payment request) and 2 outputs (non-TX remote REST call and TX SQL database). If you want to keep real-time consistency, I can propose the next architectural approach:
- put payment request into message queue. MQ Broker must support XA protocol for 2PC commit (for instance, ActiveMQ);
- your service reads message from queue using XA-connection, sends REST request to remote server and saves data into DB (which also supports XA protocol) using XA-connection;
- if REST call will failed or something else, you will rollback changes and start from processing payment request from the queue again. BTW, your remote REST resource must be idempotent;
If you're using non-transactional resoureces no way to keep it in consistent .
Also, you can lookup information, for instance, on Atomikos site for using 2PC.
Related
You have a command/operation which means you both need to save something in database end send an event/message to another system. For example you have an OrderService and when a new order is created you want to publish an "OrderCreated"-event for another system/systems to react on (either direct message or using a message broker) and do something.
The easiest (and naive) implementation is to save in db and if successful then send message. But of course this is not bullet proof because the other service/message broker is down or your service crash before sending message.
One (and common?) solution is to implement "outbox pattern", i.e. instead of publish messages directly you save the message to an outbox table in your local database as part of your database transaction (in this example save to outbox table as well as order table) and have a different process (polling db or using change data capture) reading the outbox table and publish messages.
What is your solution to this dilemma, i.e. "update database and send message or do neither"? Note: I am not talking about using SAGAs (could be part of a SAGA though but this is next level).
I have in the past used different approaches:
"Do nothing", i.e just try to send the message and hope it will be sent. Which might be fine in some cases especially with a stable message broker running on same machine.
Using DTC (in my case MSDTC). Beside all the problem with DTC it might not work with your current solution.
Outbox pattern
Using an orchestrator which will retry process if you have not got a "completed" event.
In my current project it is not handled well IMO and I want to change it to be more resilient and self correcting. Sometimes when a service is calling another service and it fails the user might retry and it might work ok. But some operations might require out support to fix it (if it is even discovered).
ATM it is not a Microservice solution but rather two large (legacy) monoliths communicating and is running on same server but moving to a Microservice architecture in the near future and might run on multiple machines.
I have REST API gateway which calls one of the microservices with MassTransit request client. This request is not durable and is meant to live for a short time - essentially it's just replacement of "traditional" synchronous (via HTTP/GRPC/etc) gateway-microservice communication.
On microservice side I have consumer which under the hood uses DbContext and Transaction (EFC) to perform some work in database. After the work is done it should publish "WorkDoneEvent" (to be consumed later by other microservices) and return result of the work to api gateway. Event must be published atomically along with transaction used to perform the work. It does not matter if ApiGateway will receive response / will retry request - as soon as transaction is commited both work result and sending "WorkDoneEvent" must be guaranteed.
Normally this is done with transactional outbox which first saves published event to database within same transaction as the work is done. (And then some process constantly "polls" outbox and tries send message to the broker, when done it removes message from outbox). As far as I know.
MassTransit seems to have transactional outbox built in: https://masstransit-project.com/advanced/middleware/transactions.html#transactional-bus.
However in docs it clearly states:
Never use the TransactionalBus or TransactionalEnlistmentBus when writing consumers. These tools are very specific and should be used only in the scenarios described.
And this is exactly what I want to do...
Why I should not do it?
I'd suggest using the InMemoryOutbox, which is part of MassTransit. It's significantly lighter weight, is designed to work in a consumer, and will not publish your events until after the consumer has completed (but prior to acknowledging the message at the broker). The only consideration is that your consumer should be idempotent (which needs to be the case in your approach as well) and if the operation was already performed on a retry, it should republish the events.
There are videos, articles, and a sample to go along with it.
Assume some spring bean has transactional method and it does two things:
asynchronously send JMS messages
updates DB
The problem is that when the message is consumed, the DB transaction sometimes not have committed, and handler failed to find in DB expected value.
The evident solution - is to send in transaction, but I use async send available on ActiveMQ to increase throughput. If I make send transactional, I forfeit asynchrocity and also risk getting OuOfMemory.
How would you solve this problem?
You need to use an XA-enabled transaction manager, and send the JMS message and do the update to the DB in the same, distributed transaction.
This doesn't change anything to asynchronicity: the send call will return before the receiver receives the message. But when it receives it, the transaction will be committed, and the receiver will find the updates in the database.
I don't see what OutOfMemoryErrors have to do with this.
I'm pretty new to MSMQ 4.0. I got stuck with below scenario;
Service A takes User Details and Returns an User ID.
Then Service B takes Billing detials with User ID.
Now I have to Queue these steps. I'm planning to use Transaction Queue.
Could some one please help me with
1)Get the ID from first message and include it in the second message.
2)If at least one step failed I have to rollback(transaction Queue does it for me) retry or 5 times and if it still failed then move it to VerifyAdminQueue for verification by Admin.I dont like using DeadLetter Queue etc.,
Thanks in advance.
Services built with MSMQ queues are truly one-way. This means that there is no built in concept of a response. There are many ways you can implement a request-response communication pattern using MSMQ but with all of them you will need to construct and send the response back to the caller yourself.
With one way actions, rollback is very simple, and indeed MSMQ will rollback any failed steps in the transmission of a message. More complex operations such as request-response however lack any concept of a transaction in MSMQ and so any rollback across more than one message transmission steps will require you to write compensatory code.
I have an application that must log all events that the user does in a remote database, so,I´ve choice to use the webservice format (the application call the webservice with the event parameters).
So, i did a remote EJB to perform that, but it is running with a bad performance, because the application needs to wait for the webservice´s response to proceed the request.
Is JMS an alternative?
What you suggest?
Thanks.
JMS will be much lighter & can asynchronously process the events. They can be used to capture application events or audit logs for the activities occurring in the system. Can send a message to a queue with proper details & those can be fetched at receiving end to process further.
If you are using EJB-3.1, then can annotate your method with #Asynchronous which returns AsyncResult implementation of Future which can be used to retrieve result, but can also be used with methods returning void.