I am attempting to use a RepositoryItemWriter to delete Products by their Ids using the RepositoryItemWriter as recommended. I have configured a RepositoryItemWriter as follows:
#Repository
public interface ProductRepository extends CrudRepository<Product,Long> {
}
Next, I specify a RepositoryItemWriter bean as follows:
class ProductRepositoryItemWriter extends RepositoryItemWriter<Product>{
private CrudRepository productRepository;
ProductRepositoryItemWriter(CrudRepository productRepository) {
super.setRepository(this.productRepository =
productRepository
}
#Transactional
#Override
protected void doWrite(List<? extends Product> products) {
this.productRepository.deleteAll(products);
}
}
My step looks like this:
public Step processStep(#Qualifier("jpaTransactionManager") final PlatformTransactionManager jpaTransactionManager) {
return stepBuilderFactory.get("processStep")
.transactionManager(jpaTransactionManager)
.chunk(120)
.reader(productJpaItemReader)
.writer((ItemWriter)productRepositoryWriter)
.build();
}
I see the deletes occurring but the products are not deleted so that the next step fails to insert the products. That step follows the delete step like this on("COMPLETED").to("uploadStep).end()
#Bean("repopulateFlow")
Flow repopulateFlow() {
FlowBuilder<Flow> flowBuilder = new FlowBuilder<>.
("repopulateFlow);
flowBuilder.start(deleteStep).on("COMPLETED")
.to(upLoadStep)
.end();
return flowBuilder.build();
}
The deleteStep uses my ProductRepositoryItemWriter to delete the rows and then the next step in the flow tries to re-insert the data but that step finds data in the table that the deleteStep should have deleted from the table. How can I achieve what I am trying to do?
I ran delete step alone in a job and it does not delete the rows in the table after it completes. I use a HikariCP pool using properties that populate a DataSourceProperties object to create the datasource. I wonder if Spring is not setting it to auto-commit true or I need to create a Hikari Pool and then set the auto-commit property to. true.
The uploadStep fails because rows are still there so I get a ConstraintsViolationException. I removed the #Transactional but still see the problem?
You need to remove #Transactional on your doWrite method. The writer is already executed in a transaction driven by Spring Batch.
Related
I'm using SpringBoot 2.x with SpringData-JPA accessing the database via a CrudRepository.
Basically, I would like to call the CrudRepository's methods to update or persist the data. In one use case, I would like to delete older entries from the database (for the brevity of this example assume: delete all entries from the table) before I insert a new element.
In case persisting the new element fails for any reason, the delete operation shall be rolled back.
However, the main problem seems to be that new transactions are opened for every method called from the CrudRepository. Even though, a transaction was opened by the method from the calling service. I couldn't get the repository methods to use the existing transaction.
Getting transaction for [org.example.jpatrans.ChairUpdaterService.updateChairs]
Getting transaction for [org.springframework.data.jpa.repository.support.SimpleJpaRepository.deleteWithinGivenTransaction]
Completing transaction for [org.springframework.data.jpa.repository.support.SimpleJpaRepository.deleteWithinGivenTransaction]
I've tried using different Propagation. (REQUIRED, SUPPORTED, MANDATORY) on different methods (service/repository) to no avail.
Changing the methods #Transactional annoation to #Transactional(propagation = Propagation.NESTED) sounded that this would just do that, but didn't help.
JpaDialect does not support savepoints - check your JPA provider's capabilities
Can I achieve the expected behaviour, not using an EntityManager directly?
I also would like to avoid to having to be using native queries as well.
Is there anything I have overlooked?
For demonstration purposes, I've created a very condensed example.
The complete example can be found at https://gitlab.com/cyc1ingsir/stackoverlow_jpa_transactions
Here are the main (even more simplified) details:
First I've got a very simple entity defined:
#Entity
#Table(name = "chair")
#Data
#AllArgsConstructor
#NoArgsConstructor
public class Chair {
// Not auto generating the id is on purpose
// for later testing with non unique keys
#Id
private int id;
#Column(name = "legs", nullable = false)
private Integer legs;
}
The connection to the database is made via the CrudRepository:
#Repository
public interface ChairRepository extends CrudRepository<Chair, Integer> {
}
This is being called from another bean (main methods here are updateChairs and doUpdate):
#Slf4j
#Service
#AllArgsConstructor
#Transactional
public class ChairUpdater {
ChairRepository repository;
/*
* Initialize the data store with some
* sample data
*/
public void initializeChairs() {
repository.deleteAll();
Chair chair4 = new Chair(1, 4);
Chair chair3 = new Chair(2, 3);
repository.save(chair4);
repository.save(chair3);
}
public void addChair(int id, Integer legCount) {
repository.save(new Chair(id, legCount));
}
/*
* Expected behaviour:
* when saving a given chair fails ->
* deleting all other is rolled back
*/
#Transactional
public void updateChairs(int id, Integer legCount) {
Chair chair = new Chair(id, legCount);
repository.deleteAll();
repository.save(chair);
}
}
The goal, I want to achieve is demonstrated by these two test cases:
#Slf4j
#RunWith(SpringRunner.class)
#DataJpaTest
#Import(ChairUpdater.class)
public class ChairUpdaterTest {
private static final int COUNT_AFTER_ROLLBACK = 3;
#Autowired
private ChairUpdater updater;
#Autowired
private ChairRepository repository;
#Before
public void setup() {
updater.initializeChairs();
}
#Test
public void positiveTest() throws UpdatingException {
updater.updateChairs(3, 10);
}
#Test
public void testRollingBack() {
// Trying to update with an invalid element
// to force rollback
try {
updater.updateChairs(3, null);
} catch (Exception e) {
LOGGER.info("Rolled back?", e);
}
// Adding a valid element after the rollback
// should succeed
updater.addChair(4, 10);
assertEquals(COUNT_AFTER_ROLLBACK, repository.findAll().spliterator().getExactSizeIfKnown());
}
}
Update:
It seems to work, if the repository is not extended from either CrudRepository or JpaRepository but from a plain Repository, definening all needed methods explicitly. For me, that seems to be a workaround rather than beeing a propper solution.
The question it boils down to seems to be: Is it possible to prevent SimpleJpaRepository from opening new transactions for every (predefined) method used from the repository interface? Or, if that is not possible, how to "force" the transaction manager to reuse the transaction, opened in the service to make a complete rollback possible?
Hi I found this documentation that looks will help you:
https://www.logicbig.com/tutorials/spring-framework/spring-data/transactions.html
Next an example take from the previous web site:
#Configuration
**#ComponentScan
#EnableTransactionManagement**
public class AppConfig {
....
}
Then we can use transactions like this:
#Service
public class MyExampleBean{
**#Transactional**
public void saveChanges() {
**repo.save(..);
repo.deleteById(..);**
.....
}
}
Yes this is possible. First alter the #Transactional annotation so that it includes rollBackFor = Exception.class.
/*
* Expected behaviour:
* when saving a given chair fails ->
* deleting all other is rolled back
*/
#Transactional(rollbackFor = Exception.class)
public void updateChairs(int id, Integer legCount) {
Chair chair = new Chair(id, legCount);
repository.deleteAll();
repository.save(chair);
}
This will cause the transaction to roll back for any exception and not just RuntimeException or Error.
Next you must add enableDefaultTransactions = false to #EnableJpaRepositories and put the annotation on one of your configuration classes if you hadn't already done so.
#Configuration
#EnableJpaRepositories(enableDefaultTransactions = false)
public class MyConfig{
}
This will cause all inherited jpa methods to stop creating a transaction by default whenever they're called. If you want custom jpa methods that you've defined yourself to also use the transaction of the calling service method, then you must make sure that you didn't annotate any of these custom methods with #Transactional. Because that would prompt them to start their own transactions as well.
Once you've done this all of the repository methods should be executed using the service method transaction only. You can test this by creating and using a custom update method that is annotated with #Modifying. For more on testing please see my answer in this SO thread. Spring opens a new transaction for each JpaRepository method that is called within an #Transactional annotated method
I am using Spring boot application, on that i am trying to achieve Transactional management. But Spring doesn't rollback the data which saved in same method.
Code base: https://github.com/vinothr/spring-boot-transactional-example
Can any one help me?
This is my repository class for 'Test' entity.
#Repository
public interface TestRepository extends CrudRepository<com.example.demo.Test, Long> {
}
I have created one end-point which used to save the data to 'Test' entity. After save happen, I thrown RunTimeException, but it is not rollbacking the saved value
#GetMapping("/test")
#Transactional
public void create() {
System.out.println("test");
final Test p = createTest();
testRepository.save(p);
final Test p1 = createTest();
testRepository.save(p1);
throw new RuntimeException();
}
It works fine after I changed into 'InnoDB' engine because I was using 'MyISAM' engine which doesn't support transaction.
ALTER TABLE my_table ENGINE = InnoDB;
Try indicate #Transactional(rollbackFor = RuntimeException.class)
I use a hibernate as JPA provider
#RestController
public class RestController {
private final TestService testService;
#PostMapping(value = "/file/{entityId}", produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public void test(#PathVariable #NotNull UUID entityId) {
testService.delete(entityId);
}
}
class TestService {
#AutoWired
EntityRepository repo; // <- Crud repository from Spring Data
public void delete(UUID id2){
//if row not exists with id == id2
throw NoFoundException
// else
//remove from database using repo.
}
}
And how to resolve the following case:
"if row not exists with id == id2 " evaluated to false, because object exists in fact.
Other thread deleted that row.
"remove from database using repo" <- error, there is no such row because it was removed by other thread in the step 2.
You can use #Transactional on your Service methods to ensure your database operations run in a transaction. By default, you can roll back the transaction if you throw a unchecked exception inside the annotated method. You can also specify on which exceptions to rollback using #Transactional's rollbackFor Parameter
Not sure why you've got a delete method that is basically doing exactly the same as the SimpleJpaRepository delete method, so for starters I'd change your code to
repo.delete(entityId)
and get rid of test service.
If you are worried about getting a EmptyResultDataAccessException when there is no data to delete, either catch the exception and ignore it or use pessimistic locking on whatever else is doing deletes, as explained
here
You can use the annotation #Transaction for your service method delete(UUID id2).Default propagation of #Transaction is Propagation.REQUIRED which means that if you get an existing transaction it continues that and if you do not have existing transaction it will create one for you.
Using Spring 4.3.12, Spring Data JPA 1.11.8 and Hibernate 5.2.12.
We use the OpenEntityManagerInViewFilter to ensure our entity relationships do not throw LazyInitializationException after an entity has been loaded. Often in our controllers we use a #ModelAttribute annotated method to load an entity by id and make that loaded entity available to a controller's request mapping handler method.
In some cases like auditing we have entity modifications that we want to commit even when some other transaction may error and rollback. Therefore we annotate our audit work with #Transactional(propagation = Propagation.REQUIRES_NEW) to ensure this transaction will commit successfully regardless of any other (if any) transactions which may or may not complete successfully.
What I've seen in practice using the OpenEntityManagerInviewFilter, is that when Propagation.REQUIRES_NEW transactions attempt to commit changes which occurred outside the scope of the new transaction causing work which should always result in successful commits to the database to instead rollback.
Example
Given this Spring Data JPA powered repository (the EmployeeRepository is similarly defined):
import org.springframework.data.jpa.repository.JpaRepository;
public interface MethodAuditRepository extends JpaRepository<MethodAudit,Long> {
}
This service:
#Service
public class MethodAuditorImpl implements MethodAuditor {
private final MethodAuditRepository methodAuditRepository;
public MethodAuditorImpl(MethodAuditRepository methodAuditRepository) {
this.methodAuditRepository = methodAuditRepository;
}
#Override #Transactional(propagation = Propagation.REQUIRES_NEW)
public void auditMethod(String methodName) {
MethodAudit audit = new MethodAudit();
audit.setMethodName(methodName);
audit.setInvocationTime(LocalDateTime.now());
methodAuditRepository.save(audit);
}
}
And this controller:
#Controller
public class StackOverflowQuestionController {
private final EmployeeRepository employeeRepository;
private final MethodAuditor methodAuditor;
public StackOverflowQuestionController(EmployeeRepository employeeRepository, MethodAuditor methodAuditor) {
this.employeeRepository = employeeRepository;
this.methodAuditor = methodAuditor;
}
#ModelAttribute
public Employee loadEmployee(#RequestParam Long id) {
return employeeRepository.findOne(id);
}
#GetMapping("/updateEmployee")
// #Transactional // <-- When uncommented, transactions work as expected (using OpenEntityManagerInViewFilter or not)
public String updateEmployee(#ModelAttribute Employee employee, RedirectAttributes ra) {
// method auditor performs work in new transaction
methodAuditor.auditMethod("updateEmployee"); // <-- at close of this method, employee update occurrs trigging rollback
// No code after this point executes
System.out.println(employee.getPin());
employeeRepository.save(employee);
return "redirect:/";
}
}
When the updateEmployee method is exercised with an invalid pin number updateEmployee?id=1&pin=12345 (pin number is limited in the database to 4 characters), then no audit is inserted into the database.
Why is this? Shouldn't the current transaction be suspended when the MethodAuditor is invoked? Why is the modified employee flushing when this Propagation.REQUIRES_NEW transaction commits?
If I wrap the updateEmployee method in a transaction by annotating it as #Transactional, however, audits will persist as desired. And this will work as expected whether or not the OpenEntityManagerInViewFilter is used.
While your application (server) tries to make two separate transactions you are still using a single EntityManager and single Datasource so at any given time JPA and the database see just one transaction. So if you want those things to be separated you need to setup two Datasources and two EntityManagers
I currently have a setup where data is inserted into a database, as well as indexed into Solr. These two steps are wrapped in a spring-managed transaction via the #Transaction annotation. What I've noticed is that spring-data-solr issues an update with the following parameters whenever the transaction is closed : params{commit=true&softCommit=false&waitSearcher=true}
#Transactional
public void save(Object toSave){
dbRepository.save(toSave);
solrRepository.save(toSave);
}
The rate of commits into solr is fairly high, so ideally I'd like send data to the solr index, and have solr auto commit at regular intervals. I have the autoCommit (and autoSoftCommit) set in my solrconfig.xml, but since spring-data-solr is sending those commit parameters, it does a hard commit every time.
I'm aware that I can drop down to the SolrTemplate API and issue commits manually, I would like to keep the solr repository.save call within a spring-managed transaction if possible. Is there a way to modify the parameters that are sent to solr on commit?
After putting in an IDE debug breakpoint in org.springframework.data.solr.repository.support.SimpleSolrRepository here:
private void commitIfTransactionSynchronisationIsInactive() {
if (!TransactionSynchronizationManager.isSynchronizationActive()) {
this.solrOperations.commit(solrCollectionName);
}
}
I discovered that wrapping my code as #Transactional (and other details to actually enable the framework to begin/end code as a transaction) doesn't achieve what we expect with "Spring Data for Apache Solr". The stacktrace shows the Proxy and Transaction Interceptor classes for our code's Transactional scope but then it also shows the framework starting its own nested transaction with another Proxy and Transaction Interceptor of its own. When the framework exits its CrudRepository.save() method my code calls, the action to commit to Solr is done by the framework's nested transaction. It happens before our outer transaction is exited. So, the attempt to batch-process many saves with one commit at the end instead of one commit for every save is futile. It seems, for this area in my code, I'll have to make use of SolrJ to save (update) my entities to Solr and then have "my" transaction's exit be followed with a commit.
If using Spring Solr, I found using the SolrTemplate bean allows you to 'batch' updates when adding data to the Solr index. By using the bean for SolrTemplate, you can use "addBeans" method, which will add a collection to the index and not commit until the end of the transaction. In my case, I started out using solrClient.add() and taking up to 4 hours for my collection to get saved to the index by iterating over it, as it commits after every single save. By using solrTemplate.addBeans(Collect<?>), it finishes in just over 1 second, as the commit is on the entire collection. Here is a code snippet:
#Resource
SolrTemplate solrTemplate;
public void doReindexing(List<Image> images) {
if (images != null) {
/* CMSSolrImage is a class with #SolrDocument mappings.
* the List<Image> images is a collection pulled from my database
* I want indexed in Solr.
*/
List<CMSSolrImage> sImages = new ArrayList<CMSSolrImage>();
for (Image image : images) {
CMSSolrImage sImage = new CMSSolrImage(image);
sImages.add(sImage);
}
solrTemplate.saveBeans(sImages);
}
}
The way I've done something similar is to create a custom repository implementation of the save methods.
Interface for the repository:
public interface FooRepository extends SolrCrudRepository<Foo, String>, FooRepositoryCustom {
}
Interface for the custom overrides:
public interface FooRepositoryCustom {
public Foo save(Foo entity);
public Iterable<Foo> save(Iterable<Foo> entities);
}
Implementation of the custom overrides:
public class FooRepositoryImpl {
private SolrOperations solrOperations;
public SolrSampleRepositoryImpl(SolrOperations fooSolrOperations) {
this.solrOperations = fooSolrOperations;
}
#Override
public Foo save(Foo entity) {
Assert.notNull(entity, "Cannot save 'null' entity.");
registerTransactionSynchronisationIfSynchronisationActive();
this.solrOperations.saveBean(entity, 1000);
commitIfTransactionSynchronisationIsInactive();
return entity;
}
#Override
public Iterable<Foo> save(Iterable<Foo> entities) {
Assert.notNull(entities, "Cannot insert 'null' as a List.");
if (!(entities instanceof Collection<?>)) {
throw new InvalidDataAccessApiUsageException("Entities have to be inside a collection");
}
registerTransactionSynchronisationIfSynchronisationActive();
this.solrOperations.saveBeans((Collection<? extends T>) entities, 1000);
commitIfTransactionSynchronisationIsInactive();
return entities;
}
private void registerTransactionSynchronisationIfSynchronisationActive() {
if (TransactionSynchronizationManager.isSynchronizationActive()) {
registerTransactionSynchronisationAdapter();
}
}
private void registerTransactionSynchronisationAdapter() {
TransactionSynchronizationManager.registerSynchronization(SolrTransactionSynchronizationAdapterBuilder
.forOperations(this.solrOperations).withDefaultBehaviour());
}
private void commitIfTransactionSynchronisationIsInactive() {
if (!TransactionSynchronizationManager.isSynchronizationActive()) {
this.solrOperations.commit();
}
}
}
and you also need to provide a SolrOperations bean for the right solr core:
#Configuration
public class FooSolrConfig {
#Bean
public SolrOperations getFooSolrOperations(SolrClient solrClient) {
return new SolrTemplate(solrClient, "foo");
}
}
Footnote: auto commit is (to my mind) conceptually incompatible with a transaction. An auto commit is a promise from solr that it will try to start to write it to disk within a certain time limit. Many things might stop that from actually happening however - a timely power or hardware failure, errors between the document and the schema, etc. But the client won't know that solr failed to keep its promise, and the transaction will see a success when it actually failed.