I have a application (APP1) where I'm working with hibernate and JMS. I have a method annotated with #Transactional. This method inserts data in a database and later sends data to a MQ.
I'm encountering a problem where the consumer application (APP2) gets the message from the MQ and calls APP1, but as APP1 is not finished commiting data the database, it returns an error saying that the data could not be found.
Is there a way where I can commit the database inserts before sending the JMS message?
#Transactional
public void create(Data data){
doSomeValidation();
anotherClass.insertInDB(data.name, data.email);
thirdClass.sendDataToMQ(data.id);
}
#Transactional
public void insertInDB(String name, String email) {
Entity entity = new Entity();
entity.setName(name);
entity.setEmail(email);
repository.saveAndFlush(entity)
}
#Transactional
public void sendDataToMQ(int id) {
Entity entity = repository.findById(id);
jmsTemplate.setSessionTransacted(true);
jmsTemplate.convertAndSend(someDestination, new Message(entity)message -> {
return message;
});
doSomeUpdateInDB(int id);
}
Related
I have two transaction manager for two database. I need to persist same data into both databases. If one transaction failed, other one need rollback. I have done like below
public interface DataService {
void saveData();
}
#Service
public class DataServiceImpl implements DataService {
#Autowired
private DataRepository dataRepository;
#Autowired
private OrDataRepository orDataRepository;
#Autowired
#Qualifier("orService")
private OrService orDataServiceImpl;
#Override
#Transactional(transactionManager = "transactionManager", rollbackFor = {RuntimeException.class})
public void saveData() {
Data data = new Data();
data.setCompKey(UUID.randomUUID().toString().substring(1,5));
data.setName("data");
dataRepository.save(data);
orDataServiceImpl.save();
//throw new RuntimeException("");
}
}
public interface OrService {
void save();
}
#Service("orService")
public class OrDataServiceImpl implements OrService {
#Autowired
private OrDataRepository orDataRepository;
#Override
#Transactional(rollbackFor = {RuntimeException.class})
public void save() {
OrData data = new OrData();
data.setCompKey(UUID.randomUUID().toString().substring(1,5));
data.setName("ordata");
orDataRepository.save(data);
}
}
I have two transaction manager (entityManager & orEntityManager) for two different DB.
If any exception in OrDataServiceImpl save method, data is not getting persisted in both DB. But if any exception in DataServiceImpl saveData method, data is getting persisted into OrData table.
I want to rollback the data from both DB if any exception.
chainedTransactionManager is deprecated. So can't use. atomikos and bitronix also can't use due to some restrictions. Kindly suggest better way to achieve distributed transation
The code need to be refactored, edit the DataServiceImpl.save() method.
Comment the orDataServiceImpl.save() line
public void saveData() {
Data data = new Data();
data.setCompKey(UUID.randomUUID().toString().substring(1,5));
data.setName("data");
dataRepository.save(data);
//orDataServiceImpl.save();
//throw new RuntimeException("");
}
Refactor/Edit the OrDataService Interface
public interface OrDataService {
void save(String uuid);
void delete(String uuid);
//will be use for compensating transaction
}
Update the OrDataServiceImpl class to implement above interface
Write new orchestration Method and use compensating transaction to rollback
pseudo code
call OrDataServiceImpl.save()
if step#1 was success
-> DataServiceImpl.saveData()
if Exception at step#3,
->OrDataServiceImpl.delete() [//to rollback]
else if, Exception at step#1
//do nothing
Background of the code:
In order to replicate a production scenario, I have created a dummy app that will basically save something in DB in a transaction, and in the same method, it publishEvent and publishEvent send a message to rabbitMQ.
Classes and usages
Transaction Starts from this method.:
#Override
#Transactional
public EmpDTO createEmployeeInTrans(EmpDTO empDto) {
return createEmployee(empDto);
}
This method saves the record in DB and also triggers publishEvent
#Override
public EmpDTO createEmployee(EmpDTO empDTO) {
EmpEntity empEntity = new EmpEntity();
BeanUtils.copyProperties(empDTO, empEntity);
System.out.println("<< In Transaction : "+TransactionSynchronizationManager.getCurrentTransactionName()+" >> Saving data for employee " + empDTO.getEmpCode());
// Record data into a database
empEntity = empRepository.save(empEntity);
// Sending event , this will send the message.
eventPublisher.publishEvent(new ActivityEvent(empDTO));
return createResponse(empDTO, empEntity);
}
This is ActivityEvent
import org.springframework.context.ApplicationEvent;
import com.kuldeep.rabbitMQProducer.dto.EmpDTO;
public class ActivityEvent extends ApplicationEvent {
public ActivityEvent(EmpDTO source) {
super(source);
}
}
And this is TransactionalEventListener for the above Event.
//#Transactional(propagation = Propagation.REQUIRES_NEW)
#TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)
public void onActivitySave(ActivityEvent activityEvent) {
System.out.println("Activity got event ... Sending message .. ");
kRabbitTemplate.convertAndSend(exchange, routingkey, empDTO);
}
This is kRabbitTemplate is a bean config like this :
#Bean
public RabbitTemplate kRabbitTemplate(ConnectionFactory connectionFactory) {
final RabbitTemplate kRabbitTemplate = new RabbitTemplate(connectionFactory);
kRabbitTemplate.setChannelTransacted(true);
kRabbitTemplate.setMessageConverter(kJsonMessageConverter());
return kRabbitTemplate;
}
Problem Definition
When I am saving a record and sending a message on rabbitMQ using the above code flow, My messages are not delivered on the server means they lost.
What I understand about the transaction in AMQP is :
If the template is transacted, but convertAndSend is not called from Spring/JPA Transaction then messages are committed within the template's convertAndSend method.
// this is a snippet from org.springframework.amqp.rabbit.core.RabbitTemplate.doSend()
if (isChannelLocallyTransacted(channel)) {
// Transacted channel created by this template -> commit.
RabbitUtils.commitIfNecessary(channel);
}
But if the template is transacted and convertAndSend is called from Spring/JPA Transaction then this isChannelLocallyTransacted in doSend method will evaluate false and commit will be done in the method which initiated Spring/JPA Transaction.
What I found after investigating the reason for message loss in my above code.
Spring transaction was active when I called convertAndSend method, so it was supposed to commit the message in Spring transaction.
For that, RabbitTemplate binds the resources and registers the Synchronizations before sending the message in bindResourceToTransaction of org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.
public static RabbitResourceHolder bindResourceToTransaction(RabbitResourceHolder resourceHolder,
ConnectionFactory connectionFactory, boolean synched) {
if (TransactionSynchronizationManager.hasResource(connectionFactory)
|| !TransactionSynchronizationManager.isActualTransactionActive() || !synched) {
return (RabbitResourceHolder) TransactionSynchronizationManager.getResource(connectionFactory); // NOSONAR never null
}
TransactionSynchronizationManager.bindResource(connectionFactory, resourceHolder);
resourceHolder.setSynchronizedWithTransaction(true);
if (TransactionSynchronizationManager.isSynchronizationActive()) {
TransactionSynchronizationManager.registerSynchronization(new RabbitResourceSynchronization(resourceHolder,
connectionFactory));
}
return resourceHolder;
}
In my code, after resource bind, it is not able to registerSynchronization because TransactionSynchronizationManager.isSynchronizationActive()==false. and since it fails to registerSynchronization, spring commit did not happen for the rabbitMQ message as AbstractPlatformTransactionManager.triggerAfterCompletion calls RabbitMQ's commit for each synchronization.
What problem I faced because of the above issue.
Message was not committed in the spring transaction, so the message lost.
As resource was added in bindResourceToTransaction, this resource remained bind and did not let add the resource for any other message to send in the same thread.
Possible Root Cause of TransactionSynchronizationManager.isSynchronizationActive()==false
I found the method which starts the transaction removed the synchronization in triggerAfterCompletion of org.springframework.transaction.support.AbstractPlatformTransactionManager class. because status.isNewSynchronization() evaluated true after DB opertation (this usually not happens if I call convertAndSend without ApplicationEvent).
private void triggerAfterCompletion(DefaultTransactionStatus status, int completionStatus) {
if (status.isNewSynchronization()) {
List<TransactionSynchronization> synchronizations = TransactionSynchronizationManager.getSynchronizations();
TransactionSynchronizationManager.clearSynchronization();
if (!status.hasTransaction() || status.isNewTransaction()) {
if (status.isDebug()) {
logger.trace("Triggering afterCompletion synchronization");
}
// No transaction or new transaction for the current scope ->
// invoke the afterCompletion callbacks immediately
invokeAfterCompletion(synchronizations, completionStatus);
}
else if (!synchronizations.isEmpty()) {
// Existing transaction that we participate in, controlled outside
// of the scope of this Spring transaction manager -> try to register
// an afterCompletion callback with the existing (JTA) transaction.
registerAfterCompletionWithExistingTransaction(status.getTransaction(), synchronizations);
}
}
}
What I Did to overcome on this issue
I simply added #Transactional(propagation = Propagation.REQUIRES_NEW) along with on #TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT) in onActivitySave method and it worked as a new transaction was started.
What I need to know
Why this status.isNewSynchronization in triggerAfterCompletion method when using ApplicationEvent?
If the transaction was supposed to terminate in the parent method, why I got TransactionSynchronizationManager.isActualTransactionActive()==true in Listner class?
If Actual Transaction Active, was it supposed to remove the synchronization?
In bindResourceToTransaction, do spring AMQP assumed an active transaction without synchronization? if the answer is yes, why not to synchronization. init if it is not activated?
If I am propagating a new transaction then I am losing the parent transaction, is there any better way to do it?
Please help me on this, it is a hot production issue, and I am not very sure about the fix I have done.
This is a bug; the RabbitMQ transaction code pre-dated the #TransactionalEventListener code, by many years.
The problem is, with this configuration, we are in a quasi-transactional state, while there is indeed a transaction in process, the synchronizations are already cleared because the transaction has already committed.
Using #TransactionalEventListener(phase = TransactionPhase.BEFORE_COMMIT) works.
I see you already raised an issue:
https://github.com/spring-projects/spring-amqp/issues/1309
In future, it's best to ask questions here, or raise an issue if you feel there is a bug. Don't do both.
I had written rest services in spring that is running perfectly fine.
Now, I need to add perform some db transactions before returning response to user.
This db transaction is independent to response retrieved.
For example,
#PostMapping("login")
public TransactionResponse loginAuthentication(#Valid #RequestBody LoginRequestBody loginRequest) {
TransactionResponse transactionResponse = new TransactionResponse();
try {
transactionResponse = loginService.validateUser(loginRequest);
//independent transaction needs to be executed in a separate thread
loginSerice.addLoginLog(transactionResponse);
//return below response without waiting to compelete above log transaction
return transactionResponse;
}
catch (Exception e) {
return CommonUtils.setErrorResponse(transactionResponse, e);
}
}
I read upon async controller in spring mvc link. Although controller
executes respective functionality in a separate thread but I don't want to wait for db transaction to be completed. After getting response from service layer, it should be forwarded to user without any delay.
Any Suggestions !!
Spring version is 4.3
I posted this answer to help the fellow developers with same kind of requirement (to execute a void function in a separate thread).
Since I am not experienced in multithreading/asynchronous environment, I want to keep it simple by using spring asynchronous methods.
So, First I created the Thread Pool
#Configuration
#EnableAsync
public class ThreadConfig {
#Bean
public TaskExecutor threadPoolTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(4);
executor.setMaxPoolSize(4);
executor.setThreadNamePrefix("WEBAPP");
executor.initialize();
return executor;
}
}
Then I created a service that will execute my code in a separate thread.
#Async
#Service
#Transactional
public class LoggingService {
public void logintransaction() throws Exception{
System.out.println("start login loggin");
Thread.sleep(5000);
System.out.println("exit");
}
}
Lastly, I called the above service on my controller. As I can see Total Time Taken is printed first, then "start login loggin" is printed. This means my new method is executed in a new thread.
#Autowired
private LoggingService loggingService;
#PostMapping("login")
public TransactionResponse loginAuthentication(#Valid #RequestBody LoginRequestBody loginRequest) {
long startTime = System.currentTimeMillis();
TransactionResponse transactionResponse = new TransactionResponse();
try {
transactionResponse = loginService.validateUser(loginRequest);
//independent transaction needs to be executed in a separate thread
//loginSerice.addLoginLog(transactionResponse);
loggingService.logintransaction();
//return below response without waiting to compelete above log transaction
System.err.println("Total Time Taken=>"+(System.currentTimeMillis() - startTime));
return transactionResponse;
}
catch (Exception e) {
return CommonUtils.setErrorResponse(transactionResponse, e);
}
}
Thanks
Brief:
Is there a way to ensure that a connection to the database is returned to the pool?
Not-brief:
Data flow:
I have some long running tasks that could be sent to the server in large volume bursts.
Each of the requests is recorded in the DB that the submission was started. Then send that request off for processing.
If failure or success the request is recorded after the task is completed.
The issue is that after the submission is recorded all the way through the long running task, the connection pool uses an "active" connection. This could potential use up any size pool I have if the burst was large enough.
I am using spring boot with the following structure:
Controller - responds at "/" and has the "service" autowired.
Service - Contains all the JPA repositories and #Transactional methods to interact with the database.
When every the first service method call is made from the controller it opens an active connection and doesn't release it until the controller method returns.
So, Is there a way to return the connection to the pool after each service method?
Here is the service class in total:
#Service
#Slf4j
class SubmissionService {
#Autowired
CompanyRepository companyRepository;
#Autowired
SubmissionRepository submissionRepository;
#Autowired
FailureRepository failureRepository;
#Autowired
DataSource dataSource
#Transactional(readOnly = true)
public Long getCompany(String apiToken){
if(!apiToken){
return null
}
return companyRepository.findByApiToken(apiToken)?.id
}
#Transactional
public void successSubmission(Long id) {
log.debug("updating submission ${id} to success")
def submissionInstance = submissionRepository.findOne(id)
submissionInstance.message = "successfully analyzed."
submissionInstance.success = true
submissionRepository.save(submissionInstance)
}
#Transactional
public long createSubmission(Map properties) {
log.debug("creating submission ${properties}")
dataSource.pool.logPoolState()
def submissionInstance = new Submission()
for (key in properties.keySet()) {
if(submissionInstance.hasProperty(key)){
submissionInstance."${key}" = properties.get(key)
}
}
submissionInstance.company = companyRepository.findOne(properties.companyId)
submissionRepository.save(submissionInstance)
return submissionInstance.id
}
#Transactional
public Long failureSubmission(Exception e, Object analysis, Long submissionId){
//Track the failures
log.debug("updating submission ${submissionId} to failure")
def submissionInstance
if (submissionId) {
submissionInstance = submissionRepository.findOne(submissionId)
submissionRepository.save(submissionInstance)
}
def failureInstance = new Failure(submission: submissionInstance, submittedJson: JsonOutput.toJson(analysis), errorMessage: e.message)
failureRepository.save(failureInstance)
return failureInstance.id
}
}
It turns out that #M.Deinum was onto the right track. Spring Boot JPA automatically turns on the "OpenEntityManagerInViewFilter" if the application property spring.jpa.open_in_view is set to true, which it is by default. I found this in the JPA Configuration Source.
After setting this to false, the database session wasn't held onto, and my problems went away.
I'm relatively new to XA transactions. I've been struggling a few days to make a simple XA transaction work to no avail.
First, I tried to use two different databases. I set up 2 XA datasources and had succeeded in rolling back the first database operation when the second fails. So far, so good. But then I tried to replace second datasource with JMS connectionFactory and cannot reproduce the same behavior.
Here's the relevant code:
Database logic:
#Stateless
public class FirstDB implements FirstDBLocal {
#PersistenceContext(unitName = "xaunit")
private EntityManager em;
public void doSomething() {
SomeEntity someEntity = em.find(SomeEntity.class, 1234L);
someEntity.setSomeFlag(false);
}
}
JMS code:
#Stateless
public class SecondJMS implements SecondJMSLocal {
#Resource(mappedName = "java:/JmsXA")
private ConnectionFactory connFactory;
#Resource(mappedName = "queue/Some.Queue")
private Queue q;
#Override
#TransactionAttribute(TransactionAttributeType.MANDATORY)
public void sendMsg() {
Session session = null;
Connection conn = null;
MessageProducer producer = null;
try {
conn = connFactory.createConnection("guest", "guest");
session = conn.createSession(false, Session.AUTO_ACKNOWLEDGE);
producer = session.createProducer(q);
// Not sure if I need this, but I found it in the sample code
conn.start();
TextMessage tm = session.createTextMessage(new Date().toString());
producer.send(tm);
throw new RuntimeException("Fake exception");
} catch (JMSException e) {
e.printStackTrace();
} catch (RuntimeException e) {
e.printStackTrace();
} finally {
// close all resources
}
}
}
The glue code:
#Stateless
public class TestDBandJMS implements TestDBandJMSLocal {
#EJB
private FirstDBLocal firstDBLocal;
#EJB
private SecondJMSLocal secondJMSLocal;
public void doStuff() {
firstDBLocal.doSomething();
secondJMSLocal.sendMsg();
}
}
XA Connection Factory configuration (everything is JBoss default, except for commented out security settings):
<tx-connection-factory>
<jndi-name>JmsXA</jndi-name>
<xa-transaction/>
<rar-name>jms-ra.rar</rar-name>
<connection-definition>org.jboss.resource.adapter.jms.JmsConnectionFactory</connection-definition>
<config-property name="SessionDefaultType" type="java.lang.String">javax.jms.Topic</config-property>
<config-property name="JmsProviderAdapterJNDI" type="java.lang.String">java:/DefaultJMSProvider</config-property>
<max-pool-size>20</max-pool-size>
<!-- <security-domain-and-application>JmsXARealm</security-domain-and-application> -->
<depends>jboss.messaging:service=ServerPeer</depends>
</tx-connection-factory>
I also have very simple MDB which just prints out received message to console (not going to post the code, since it's trivial).
The problem is, when the exception is thrown in JMS code, the message is still received by MDB and SomeEntity is successfully updated in the database code (whereas I expect it to rollback).
Here is the JMS log. One fishy thing that I see there is this:
received ONE_PHASE_COMMIT request
Like I said, I'm not too familiar with XA yet, but I expect to see here TWO_PHASE_COMMIT, because there should be 2 resources which participate in the active transaction.
Any help would be much appreciated.
UPDATE
It worked eventually, after I tried #djmorton's suggestion.
One other important thing to keep in mind when working with JBoss 5.1 is that the lookup name for XA JMS ConnectionFactory is "java:/JmsXA". I tried the same with
#Resource(mappedName = "XAConnectionFactory")
private ConnectionFactory connFactory;
and it didn't work.
You are catching your RuntimeException after throwing it in your sendMsg() method. The Exception will not trigger a transaction rollback unless it is thrown up the stack. When using Container managed transactions, the container adds interceptors to the method calls to setup the transactions and handle rollbacks when unchecked exceptions are thrown. If the exception isn't thrown out of the method the interceptor doesn't know it needs to role the transaction back.
Edit 1:
Note that only a RuntimeException or a subclass of RuntimeException being thrown will cause the transaction to rollback. A checked exception (One that extends Exception rather than RuntimeException) will not cause a rollback unless it is annotated with #ApplicationException(rollback=true).
The other alternative is to inject an EJBContext object, and call .setRollbackOnly() to force the transaction to rollback when the method goes out of scope:
#Stateless
public class SomeEjb {
#Resource
private EJBContext context;
#TransactionAttribute(TransactionAttributeType.MANDATORY)
public void rollMeBack() {
context.setRollbackOnly();
}
}