I have some code within a spring transaction with the isolation level set to SERIALIZABLE. This code does a few things firstly it deletes all records from a table that have a flag set, next it performs a select to ensure invalid records can not be written and finally the new records are written.
The problem is that the select continues to return the records that were deleted if the code is run with the transaction annotation. My understanding is that because we are performing these operations within the same spring transaction that the previous delete operation will be considered when performing the select.
We are using Spring Boot 2.1 and Hibernate 5.2
A summary of the code is shown below:
#HystrixCommand
public void deleteRecord(EntityObj entityObj) {
fooRepository.deleteById(entityObj.getId());
//Below line added as part of debugging but I don't think I should really need it?
fooRepository.flush();
}
public List<EntityObj> findRecordByProperty(final String property) {
return fooRepository.findEntityObjByProperty(property);
}
#Transactional(isolation = Isolation.SERIALIZABLE)
public void debugReadWrite() {
EntitiyObject entitiyObject = new EntityObject();
entitiyObject.setId(1);
deleteRecord(entitiyObject);
List<EntityObj> results = findRecordByProperty("bar");
if (!results.isEmpty()) {
throw new RuntimeException("Should be no results!")
}
}
The transaction has not committed yet, you need to complete the transaction and then find the record.
decorating the deleteRecord with propagation = Propagation.REQUIRES_NEW) should solve the issue
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void deleteRecord(EntityObj entityObj) {
fooRepository.deleteById(entityObj.getId());
// flush not needed fooRepository.flush();
}
A flush is not needed because when deleteRecord completes the translation will be committed.
under the hood
//start transaction
public void deleteRecord(EntityObj entityObj) {
fooRepository.deleteById(entityObj.getId());
}
//commit transaction
Turns out the issue was due to our use of Hystrix. The transaction is started outside of Hystirx and then at a later point goes through a Hystrix command. The Hystrix command is using a threadpool and so the transaction is lost while executing on the new thread from the Hystrix threadpool. See this github issue for more info:
https://github.com/spring-cloud/spring-cloud-netflix/issues/1381
Related
I have a case where I need to initiate calls to four different services in the same method. According to our business logic all four steps must succeed, rollback otherwise. The problem is that since all the steps occur in the same transaction (via #Transactional annotation) the second call in the method cant process the result of the first one. I have tried Propagation.REQUIRES_NEW on the first call but since it starts a new transaction the changes it makes are committed and not rollbacked whether the other calls succeed or not. At some point it seemed like Propagation.NESTED could do the work but I get a NestedTransactionNotSupportedException which warns me of setting the nestedTransactionAllowed property to true. But I could not found any document about how to do that. Any ideas? And also would appreciate any other suggestion for the problem. Thanks.
Root method:
#Override
#Transactional
public GuestUserDto createGuestUser(GuestUserRegisterRequest existingGuestUser) {
guestUserDeactivationDelegateService.deactivateGuestUser(existingGuestUser);
UserDto userDto = guestUserRegistrationDelegateService.registerGuestUser(
guestUserRegisterDtoConverter.convert(existingGuestUser));
createGuestUserAddress(existingGuestUser);
guestUserRegisterHistoryService.createRegisterHistory(
guestUserRegistrationHistoryDtoBuilder.build(userDto.getMemberId(), existingGuestUser));
return guestUserDtoConverter.convert(userDto);
}
Deactivation method:
#Transactional
public void deactivateGuestUser(GuestUserRegisterRequest existingUser) {
deactivateSmsVerification(existingUser.getMemberId());
String emailPlaceholder = existingUser.getMemberId() + existingUser.getEmail();
userGraphClientService.updateUserStatus(createUserRequest(existingUser.getMemberId()));
updateGuestUserStatus(existingUser.getMemberId(), emailPlaceholder);
}
Seems that the main cause was Hibernate's flush operations order, as it was giving priority to insert over update. Flushing manually after updating in DAO layer solved the problem.
NestedTransactionAllowed can be set in HibernateTransactionManager as follows.
HibernateTransactionManager manager = new HibernateTransactionManager(sessionFactory);
manager.setNestedTransactionAllowed(true);
return manager;
Please refer to https://www.demo2s.com/java/spring-hibernatetransactionmanager-setnestedtransactionallowed-boolean.html
I'm using jpa , spring data and entity listeners to audit my entities precisely on postUpdate , postPersist , PostRemove
This is a pseudo code of my entity listener class
public class EntityListener extends AuditingEntityListener {
#PostUpdate
public void postPersist(Object auditedEntity) {
writer.saveEntity(auditedEntity,"UPDATE");
}
This the pseudo code of the Writer class
public class Writer {
#Async
public void saveEntity(Object auditedEntity, String action) {
try {
//some code to prepare the history entity
historyDAO.save(entity);
} catch (Exception e) {
}
}
when an exception is thrown in Writer class , the auditedEntity is updated or inserted however the historyEntity where i store the audit action doesnt
The problem is i need to invoke the saveEntity method in another thread for performance issue (#Async) but in that case a new transaction is open instead of the previously one which opened
how can i solve the rollack issue for both transactions
so when an exception is throwen both historyEntity and auditedEntity not persisted
I understand that you want to rollback both the child and the parent transaction when an exception is thrown from within Writer.saveEntity.
The problem is that the thread with the original transaction would still need to wait for all these complicated operations to finish before it could mark the transaction as committed. You can't easily span a transaction across multiple threads, either.
The only thing you could probably do to speed things up is you could run the logic of generating the history entities in parallel, and then save them all just before the transaction commits.
One way of doing that that I can think of is using a Hibernate interceptor:
public class AuditInterceptor extends EmptyInterceptor {
private List<Callable<BaseEntity>> historyEntries;
private ExecutorService executor;
...
public void beforeTransactionCompletion(Transaction tx) {
List<Future<BaseEntity>> futures = executor.invokeAll(historyEntries);
if (executor.awaitTermination(/* some timeout here */)) {
futures.stream().map(Future::get).forEach(entity -> session.save(object));
} else {
/* rollback */
}
}
}
Your listener code then becomes:
#PostUpdate
public void postPersist(Object auditedEntity) {
interceptor.getHistoryEntries().add(new Callable<BaseEntity> {
/* history entry generation logic goes here */
});
}
(note that the above code is greatly simplified, you could use any other asynchronous execution API, the basic idea is that you need to block in AuditInterceptor.beforeTransactionCompletion, waiting for all the history entries to be generated)
However, I would strongly advise against using the above technique, as it is rather complicated and error prone.
If you look here: https://docs.jboss.org/hibernate/orm/5.1/userguide/html_single/chapters/events/Events.html, you'll find that Hibernate interceptors have more interesting methods that could help you gather auditing info, and that perhaps your implementation could make use of them, possibly avoiding the need for complicated logic altogether (Hibernate already does track changes to fields of individual entities, so you get that information for free).
Why reinvent the wheel, though? If you dig even deeper, you'll find the Hibernate Envers module (http://hibernate.org/orm/envers/, works for both JPA and pure Hibernate) which gives you business auditing out of the box. Envers already digs into the above mechanism, so hopefully the performance issue would go away.
Final note: have you measured how long history entry generation takes? I would guess that executing for loops and if statements might be cheaper than database access operations. If I were you, I wouldn't do any of the above unless I was absolutely sure that's where the performance bottleneck was.
I currently have a setup where data is inserted into a database, as well as indexed into Solr. These two steps are wrapped in a spring-managed transaction via the #Transaction annotation. What I've noticed is that spring-data-solr issues an update with the following parameters whenever the transaction is closed : params{commit=true&softCommit=false&waitSearcher=true}
#Transactional
public void save(Object toSave){
dbRepository.save(toSave);
solrRepository.save(toSave);
}
The rate of commits into solr is fairly high, so ideally I'd like send data to the solr index, and have solr auto commit at regular intervals. I have the autoCommit (and autoSoftCommit) set in my solrconfig.xml, but since spring-data-solr is sending those commit parameters, it does a hard commit every time.
I'm aware that I can drop down to the SolrTemplate API and issue commits manually, I would like to keep the solr repository.save call within a spring-managed transaction if possible. Is there a way to modify the parameters that are sent to solr on commit?
After putting in an IDE debug breakpoint in org.springframework.data.solr.repository.support.SimpleSolrRepository here:
private void commitIfTransactionSynchronisationIsInactive() {
if (!TransactionSynchronizationManager.isSynchronizationActive()) {
this.solrOperations.commit(solrCollectionName);
}
}
I discovered that wrapping my code as #Transactional (and other details to actually enable the framework to begin/end code as a transaction) doesn't achieve what we expect with "Spring Data for Apache Solr". The stacktrace shows the Proxy and Transaction Interceptor classes for our code's Transactional scope but then it also shows the framework starting its own nested transaction with another Proxy and Transaction Interceptor of its own. When the framework exits its CrudRepository.save() method my code calls, the action to commit to Solr is done by the framework's nested transaction. It happens before our outer transaction is exited. So, the attempt to batch-process many saves with one commit at the end instead of one commit for every save is futile. It seems, for this area in my code, I'll have to make use of SolrJ to save (update) my entities to Solr and then have "my" transaction's exit be followed with a commit.
If using Spring Solr, I found using the SolrTemplate bean allows you to 'batch' updates when adding data to the Solr index. By using the bean for SolrTemplate, you can use "addBeans" method, which will add a collection to the index and not commit until the end of the transaction. In my case, I started out using solrClient.add() and taking up to 4 hours for my collection to get saved to the index by iterating over it, as it commits after every single save. By using solrTemplate.addBeans(Collect<?>), it finishes in just over 1 second, as the commit is on the entire collection. Here is a code snippet:
#Resource
SolrTemplate solrTemplate;
public void doReindexing(List<Image> images) {
if (images != null) {
/* CMSSolrImage is a class with #SolrDocument mappings.
* the List<Image> images is a collection pulled from my database
* I want indexed in Solr.
*/
List<CMSSolrImage> sImages = new ArrayList<CMSSolrImage>();
for (Image image : images) {
CMSSolrImage sImage = new CMSSolrImage(image);
sImages.add(sImage);
}
solrTemplate.saveBeans(sImages);
}
}
The way I've done something similar is to create a custom repository implementation of the save methods.
Interface for the repository:
public interface FooRepository extends SolrCrudRepository<Foo, String>, FooRepositoryCustom {
}
Interface for the custom overrides:
public interface FooRepositoryCustom {
public Foo save(Foo entity);
public Iterable<Foo> save(Iterable<Foo> entities);
}
Implementation of the custom overrides:
public class FooRepositoryImpl {
private SolrOperations solrOperations;
public SolrSampleRepositoryImpl(SolrOperations fooSolrOperations) {
this.solrOperations = fooSolrOperations;
}
#Override
public Foo save(Foo entity) {
Assert.notNull(entity, "Cannot save 'null' entity.");
registerTransactionSynchronisationIfSynchronisationActive();
this.solrOperations.saveBean(entity, 1000);
commitIfTransactionSynchronisationIsInactive();
return entity;
}
#Override
public Iterable<Foo> save(Iterable<Foo> entities) {
Assert.notNull(entities, "Cannot insert 'null' as a List.");
if (!(entities instanceof Collection<?>)) {
throw new InvalidDataAccessApiUsageException("Entities have to be inside a collection");
}
registerTransactionSynchronisationIfSynchronisationActive();
this.solrOperations.saveBeans((Collection<? extends T>) entities, 1000);
commitIfTransactionSynchronisationIsInactive();
return entities;
}
private void registerTransactionSynchronisationIfSynchronisationActive() {
if (TransactionSynchronizationManager.isSynchronizationActive()) {
registerTransactionSynchronisationAdapter();
}
}
private void registerTransactionSynchronisationAdapter() {
TransactionSynchronizationManager.registerSynchronization(SolrTransactionSynchronizationAdapterBuilder
.forOperations(this.solrOperations).withDefaultBehaviour());
}
private void commitIfTransactionSynchronisationIsInactive() {
if (!TransactionSynchronizationManager.isSynchronizationActive()) {
this.solrOperations.commit();
}
}
}
and you also need to provide a SolrOperations bean for the right solr core:
#Configuration
public class FooSolrConfig {
#Bean
public SolrOperations getFooSolrOperations(SolrClient solrClient) {
return new SolrTemplate(solrClient, "foo");
}
}
Footnote: auto commit is (to my mind) conceptually incompatible with a transaction. An auto commit is a promise from solr that it will try to start to write it to disk within a certain time limit. Many things might stop that from actually happening however - a timely power or hardware failure, errors between the document and the schema, etc. But the client won't know that solr failed to keep its promise, and the transaction will see a success when it actually failed.
In the service layer, I have some method who have a transactional annotation.
#Transactional
public void process() throws ProcessPaymentException{
try{
.... do some operation
catch (ProcessPaymentException ppe) {
save db problem issue.
}
}
It seem like if there are a issue, there are roll back... and nothing is saved in the db...
ProcessPaymentException extend Exception
Is there a way to rollback the process in the try but do the save in the catch?
Edit
Nested transaction could be a solution if this link is ok
https://www.credera.com/blog/technology-insights/java/common-oversights-utilizing-nested-transactions-spring/
Existing answer of using ControllerAdvise should help in normal setup that incoming requests are coming through Spring MVC (i.e. through a Controller).
For cases that is not, or you do not want to tie your exception handling logic to Spring MVC, here are some alternatives I can think of
(Here I assume you want to rely on declarative transaction control instead of programmatically controlling transactions yourself)
Separate service/component to save error in different transaction.
In short, you can have a separate service, which create its own transaction by propagation REQUIRES_NEW. e.g.
#Service
public class FooService
#Inject
private ErrorAuditService errorAuditService;
#Transactional
public void process() throws ProcessPaymentException{
try{
.... do some operation
catch (ProcessPaymentException ppe) {
errorAuditService.saveErrorAudit(ppe.getErrorText());
throw ppe; // I guess you want to re-throw the exception
}
}
}
#Service
public class ErrorAuditService
#Transactional(propagation=REQUIRES_NEW)
public void saveErrorAudit() {
// save to DB
}
}
One step further, if the error handling it the same for different services, you may create an advise, which will be called when service method throws exception. In that advise, you can save the error in db (using ErrorAuditService), and rethrow the exception.
Because processes of try-catch are wrapped by the same transaction.
The transaction manager do rollback whenever an exception is thrown. So, not thing would be saved.
Is there a way to rollback the process in the try but do the save in the catch?
Yes. Create Exception Handler to save db problem issue after rollback.
this is the idea
#ControllerAdvice
public class HandlerName {
#ExceptionHandler(ProcessPaymentException.class)
public void saveDbIssue(ProcessPaymentException ex) {
// save db problem issue.
}
But it only works if u want to save static data.
The below code is not working for rollback when any exception occurs while insertion of records in database.I am using Spring 4 framework and annotation .
*/I am using below code for transaction management and it will not roll back for any exception./
#Transactional(rollbackFor = RuntimeException.class)
public boolean insertBatch(List<String> query) throws SQLException {
boolean flag= false;
try
{
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
String[] Sql= query.toArray(new String[query.size()]);
jdbcTemplate.batchUpdate(Sql);
flag=true;
}catch(DataAccessException e )
{
flag=false;
MessageResource.setMessages("Constraints Violation ! CSV data value not matched with database constraints ");
LOGGER.info("CSV file Data not expected as database table structure defination like constraint violation/Data Type lenght/NUll etc for same data value" );
LOGGER.error( "Cause for error: "+ e.getRootCause().getMessage());
LOGGER.debug( "Details explain : "+ e.toString());
throw new RuntimeException("Roll back operation");
//transactionManager.rollback(status);
}
return flag;
}**
Actullay answaer provided by Sir, M.Deinum is below:
Spring uses proxies to apply AOP this will only work for methods called from the outside. Internal method calls don't pass through the proxy hence no transactions and depending on your queries you get one large or multiple smaller commits. Make sure that the outer method (the one called to initiate everything) is transactional. – M. Deinum 14 hours ago
#Transactional(rollbackFor = RuntimeException.class)
This will rollback only if a RuntimeException or a subclass is thrown from the annotated method. If you want to rollback for any Exception (such as SQLException, which is NOT a RuntimeException), you should do:
#Transactional(rollbackFor = Exception.class)
And if you want to try a rollback for whatever error that might happen
#Transactional(rollbackFor = Throwable.class)
Altough in this last case the runtime might be so broken that not even the rollback can complete.
Use Prepared statement from connection object and the do a execute batch object. On the connection object use conn.setAutoCommit(false). Prepeared statement has 4 times better performance than JdbcTemplate for batch insertion of 1000 records.
Reference : JdbcTemplate.batchUpdate() returns 0, on insert error for one item but inserts the remaining item into sql server db despite using #Transactional