Spring boot change connection schema dynamically inside transaction - spring-boot

In my Spring boot application i need to read data from a specific schema and write on another one, to do so i follow this guide (https://github.com/spring-projects/spring-data-examples/tree/main/jpa/multitenant/schema) and i used this answer (https://stackoverflow.com/a/47776205/10857151) to be able to change at runtime the schema used.
But if this works fine inside a service without any transaction scope, this doesn't works on a more complex architecture (exception: session/EntityManager is closed) where there are couple of service that share transaction to ensure rollback.
THE BELLOW IS A SIMPLE EXAMPLE OF THE ARCHITECTURE
//simple jpa repository
private FirstRepository repository;
private SecondRepository secondRepository;
private Mapper mapper;
private SchematUpdater schemaUpdater;
#Transactional
public void entrypoint(String idSource,String idTarget) {
//copy first object
firstCopyService(idSource, idTarget);
//copy second object
secondCopyService(idSource, idTarget);
}
#Transactional
public void firstCopyService(String idSource,String idTarget) {
//change schema to the source default
schemaUpdater.changeToSurceSchema();
Object obj=repository.get(idSource);
//convert obj before persist - set new id reference and other things
obj=mapper.prepareObjToPersist(obj,idTarget);
//change schema to the target default
schemaUpdater.changeToTargetSchema();
repository.saveAndFlush(obj);
}
#Transactional
public void secondCopyService(String idSource,String idTarget) {
schemaUpdater.changeToSurceSchema();
Object obj=secondRepository.get(idSource);
//convert obj before persist
obj=mapper.prepareObjToPersist(obj);
//change schema to the target default
schemaUpdater.changeToTargetSchema();
secondRepository.saveAndFlush(obj);
}
I need to know what could be the best solution to ensure this dynamical switch and maintain the transaction scope on each service, without causing problems connected to restore and clean entity manager session.
Thanks

Related

Combining multi-tenant Spring application with distributed JTA transactions

I have a multi-tenant (database per tenant) Spring application. I have configured multiple data source beans for each tenant but only one entity manager factory bean because the tenants have the same tables with the same structure, i.e. the same entities. Unfortunately the uniqueness of the entity manager factory in context of how SharedEntityManagerCreator works produces difficulties for me when using distributed JTA transactions across these tenants. The SharedEntityManagerCreator before creating a new entity manager uses the entity manager factory bean instance as key to search if an entity manager object already exists in the resources of the current transaction:
public static EntityManager doGetTransactionalEntityManager(EntityManagerFactory emf, #Nullable Map<?, ?> properties, boolean synchronizedWithTransaction) throws PersistenceException {
EntityManagerHolder emHolder = (EntityManagerHolder)TransactionSynchronizationManager.getResource(emf);
/* ... */
}
and if such exits it is reused. Therefore changing the tenant in the current transaction has no effect, because the entity manager is not recreated but is reused, which means that the entity manager object contains the reference to the previous data source and the operations are executed on the previous tenant but not on the new tenant.
I found a quick solution. Since the entity manager object is wrapped in a EntityManagerHolder object inside transaction resources I created another class that extends from EntityManagerHolder and does not wrap one entity manager object but a map of entity managers with tenant as keys:
public class MultiTenantEntityManagerHolder extends EntityManagerHolder {
private Map<String, EntityManager> entityManagers = new HashedMap<>();
private EntityManagerFactory entityManagerFactory;
#Override
public EntityManager getEntityManager() {
String tenantId = <get current tenant>;
if(!entityManagers.containsKey(tenantId)) {
entityManagers.put(tenantId, entityManagerFactory.createEntityManager());
}
return entityManagers.get(tenantId);
}
}
Then an object of type MultiTenantEntityManagerHolder is created at the beginning of transaction an placed inside resources:
TransactionSynchronizationManager.bindResource(entityManagerFactory, new MultiTenantEntityManagerHolder(entityManagerFactory));
But I'm looking at this solution as a hack, that may not work in the next version of Spring. Therefore I have two questions: is my current solution really a hack, i.e. a weak solution that should be abandoned? What are possible other approaches for this problem?

Spring repository caching

I am writing an application with Spring 5 and Hibernate. There's a service that receives an entity in a different state than currently persisted. It performs some processing and saves the entity to database (using Spring CrudRepository).
public void saveEntity(Entity entity) {
ProcessingStatus processingStatus = doSomeProcessing(entity);
if (processingStatus == ProcessingStatus.SUCCESS) {
entity.setProcessingStatus(ProcessingStatus.SUCCESS);
repository.save(entity);
} else {
Entity originalEntity = repository.findById(entity.getId());
originalEntity.setProcessingStatus(ProcessingStatus.FAILURE);
repository.save(originalEntity);
}
}
So if processing was successful we're just marking entity with success status and save it. Otherwise any changes that comes with new version of entity shouldn't be applied. So original entity from repository is retrieved, its status is changed and then it is saved.
The problem is that line
Entity originalEntity = repository.findById(entity.getId());
actually retrieves already modified object (probably cached by Hibernate?), not the original one from database. So originalEntity has the same set of attributes as entity (received as method argument). What would be the best way to retrieve original object in such case?
I suppose this saveEntity method is annotated with #Transactional.
For this reason, also if not explicitly stored to with repository.save(entity), you have your update object because you are in the same hibernate session.
You can detach your modified entity and then you will read the clean one (WARNING! if your entity has not been already persisted you will get a null)
To detach and Entity you have to inject PersistenceContext in your repository
#Repository
public class EntityRepository {
#PersistenceContext
private EntityManager entityManager;
public void detachOrderItem(Object object) {
entityManager.detach(object);
}
}
And then you have to call detach before findById
} else {
entityRepository.detach(entity);
Entity originalEntity = repository.findById(entity.getId());
originalEntity.setProcessingStatus(ProcessingStatus.FAILURE);
repository.save(originalEntity);
}
Another simpler solution could be to run public void saveEntity(Entity entity) out of a Transaction (only nested method could live a a transaction)

Spring Data Solr #Transaction Commits

I currently have a setup where data is inserted into a database, as well as indexed into Solr. These two steps are wrapped in a spring-managed transaction via the #Transaction annotation. What I've noticed is that spring-data-solr issues an update with the following parameters whenever the transaction is closed : params{commit=true&softCommit=false&waitSearcher=true}
#Transactional
public void save(Object toSave){
dbRepository.save(toSave);
solrRepository.save(toSave);
}
The rate of commits into solr is fairly high, so ideally I'd like send data to the solr index, and have solr auto commit at regular intervals. I have the autoCommit (and autoSoftCommit) set in my solrconfig.xml, but since spring-data-solr is sending those commit parameters, it does a hard commit every time.
I'm aware that I can drop down to the SolrTemplate API and issue commits manually, I would like to keep the solr repository.save call within a spring-managed transaction if possible. Is there a way to modify the parameters that are sent to solr on commit?
After putting in an IDE debug breakpoint in org.springframework.data.solr.repository.support.SimpleSolrRepository here:
private void commitIfTransactionSynchronisationIsInactive() {
if (!TransactionSynchronizationManager.isSynchronizationActive()) {
this.solrOperations.commit(solrCollectionName);
}
}
I discovered that wrapping my code as #Transactional (and other details to actually enable the framework to begin/end code as a transaction) doesn't achieve what we expect with "Spring Data for Apache Solr". The stacktrace shows the Proxy and Transaction Interceptor classes for our code's Transactional scope but then it also shows the framework starting its own nested transaction with another Proxy and Transaction Interceptor of its own. When the framework exits its CrudRepository.save() method my code calls, the action to commit to Solr is done by the framework's nested transaction. It happens before our outer transaction is exited. So, the attempt to batch-process many saves with one commit at the end instead of one commit for every save is futile. It seems, for this area in my code, I'll have to make use of SolrJ to save (update) my entities to Solr and then have "my" transaction's exit be followed with a commit.
If using Spring Solr, I found using the SolrTemplate bean allows you to 'batch' updates when adding data to the Solr index. By using the bean for SolrTemplate, you can use "addBeans" method, which will add a collection to the index and not commit until the end of the transaction. In my case, I started out using solrClient.add() and taking up to 4 hours for my collection to get saved to the index by iterating over it, as it commits after every single save. By using solrTemplate.addBeans(Collect<?>), it finishes in just over 1 second, as the commit is on the entire collection. Here is a code snippet:
#Resource
SolrTemplate solrTemplate;
public void doReindexing(List<Image> images) {
if (images != null) {
/* CMSSolrImage is a class with #SolrDocument mappings.
* the List<Image> images is a collection pulled from my database
* I want indexed in Solr.
*/
List<CMSSolrImage> sImages = new ArrayList<CMSSolrImage>();
for (Image image : images) {
CMSSolrImage sImage = new CMSSolrImage(image);
sImages.add(sImage);
}
solrTemplate.saveBeans(sImages);
}
}
The way I've done something similar is to create a custom repository implementation of the save methods.
Interface for the repository:
public interface FooRepository extends SolrCrudRepository<Foo, String>, FooRepositoryCustom {
}
Interface for the custom overrides:
public interface FooRepositoryCustom {
public Foo save(Foo entity);
public Iterable<Foo> save(Iterable<Foo> entities);
}
Implementation of the custom overrides:
public class FooRepositoryImpl {
private SolrOperations solrOperations;
public SolrSampleRepositoryImpl(SolrOperations fooSolrOperations) {
this.solrOperations = fooSolrOperations;
}
#Override
public Foo save(Foo entity) {
Assert.notNull(entity, "Cannot save 'null' entity.");
registerTransactionSynchronisationIfSynchronisationActive();
this.solrOperations.saveBean(entity, 1000);
commitIfTransactionSynchronisationIsInactive();
return entity;
}
#Override
public Iterable<Foo> save(Iterable<Foo> entities) {
Assert.notNull(entities, "Cannot insert 'null' as a List.");
if (!(entities instanceof Collection<?>)) {
throw new InvalidDataAccessApiUsageException("Entities have to be inside a collection");
}
registerTransactionSynchronisationIfSynchronisationActive();
this.solrOperations.saveBeans((Collection<? extends T>) entities, 1000);
commitIfTransactionSynchronisationIsInactive();
return entities;
}
private void registerTransactionSynchronisationIfSynchronisationActive() {
if (TransactionSynchronizationManager.isSynchronizationActive()) {
registerTransactionSynchronisationAdapter();
}
}
private void registerTransactionSynchronisationAdapter() {
TransactionSynchronizationManager.registerSynchronization(SolrTransactionSynchronizationAdapterBuilder
.forOperations(this.solrOperations).withDefaultBehaviour());
}
private void commitIfTransactionSynchronisationIsInactive() {
if (!TransactionSynchronizationManager.isSynchronizationActive()) {
this.solrOperations.commit();
}
}
}
and you also need to provide a SolrOperations bean for the right solr core:
#Configuration
public class FooSolrConfig {
#Bean
public SolrOperations getFooSolrOperations(SolrClient solrClient) {
return new SolrTemplate(solrClient, "foo");
}
}
Footnote: auto commit is (to my mind) conceptually incompatible with a transaction. An auto commit is a promise from solr that it will try to start to write it to disk within a certain time limit. Many things might stop that from actually happening however - a timely power or hardware failure, errors between the document and the schema, etc. But the client won't know that solr failed to keep its promise, and the transaction will see a success when it actually failed.

How to link JPA persistence context with single database transaction

Latest Spring Boot with JPA and Hibernate: I'm struggling to understand the relationship between transactions, the persistence context and the hibernate session and I can't easily avoid the dreaded no session lazy initialization problem.
I update a set of objects in one transaction and then I want to loop through those objects processing them each in a separate transaction - seems straightforward.
public void control() {
List<> entities = getEntitiesToProcess();
for (Entity entity : entities) {
processEntity(entity.getId());
}
}
#Transactional(value=TxType.REQUIRES_NEW)
public List<Entity> getEntitiesToProcess() {
List<Entity> entities = entityRepository.findAll();
for (Entity entity : entities) {
// Update a few properties
}
return entities;
}
#Transactional(value=TxType.REQUIRES_NEW)
public void processEntity(String id) {
Entity entity = entityRepository.getOne(id);
entity.getLazyInitialisedListOfObjects(); // throws LazyInitializationException: could not initialize proxy - no Session
}
However, I get a problem because (I think) the same hibernate session is being used for both transactions. When I call entityRepository.getOne(id) in the 2nd transaction, I can see in the debugger that I am returned exactly the same object that was returned by findAll() in the 1st transaction without a DB access. If I understand this correctly, it's the hibernate cache doing this? If I then call a method on my object that requires a lazy evaluation, I get a "no session" error. I thought the cache and the session were linked so that's my first confusion.
If I drop all the #Transactional annotations or if I put a #Transactional on the control method it all runs fine, but the database commit isn't done until the control method completes which is obviously not what I want.
So, I have a few questions:
How can I make the hibernate session align with my transaction scope?
What is a good pattern for doing the separation transactions in a loop with JPA and declarative transaction management?
I want to retain the declarative style (i.e. no xml), and don't want to do anything Hibernate specific.
Any help appreciated!
Thanks
Marcus
Spring creates a proxy around your service class, which means #Transactional annotations are only applied when annotated methods are called through the proxy (where you have injected this service).
You are calling getEntitiesToProcess() and processEntity() from within control(), which means those calls are not going through proxy but instead have the transactional scope of the control() method (if you aren't also calling control() from another method in the same class).
In order for #Transactional to apply, you need to do something like this
#Autowired
private ApplicationContext applicationContext;
public void control() {
MyService myService = applicationContext.getBean(MyService.class);
List<> entities = myService.getEntitiesToProcess();
for (Entity entity : entities) {
myService.processEntity(entity.getId());
}
}

Adding #Transactional causes "collection with cascade="all-delete-orphan" was no longer referenced"

I am upgrading a working project from Spring2+Hibernate3 to Spring3+Hibernate4. Since HibernateTemplate and HibernateDAOSupport have been retired, I did the following
Before (simplified)
public List<Object> loadTable(final Class<?> cls)
{
Session s = getSession(); // was calling the old Spring getSession
Criteria c = s.createCriteria(cls);
List<Object> objects = c.list();
if (objects == null)
{
objects = new ArrayList<Object>();
}
closeSession(s);
return objects;
}
Now (simplified)
#Transactional(propagation=Propagation.REQUIRED)
public List<Object> loadTable(final Class<?> cls)
{
Session s = sessionFactory.getCurrentSession();
Criteria c = s.createCriteria(cls);
List<Object> objects = c.list();
if (objects == null)
{
objects = new ArrayList<Object>();
}
return objects;
}
I also added the transaction annotation declaration to Spring XML and removed this from Hibernate properties
"hibernate.current_session_context_class", "org.hibernate.context.ThreadLocalSessionContext"
The #Transactional annotation seems to have worked as I see this in the stacktrace
at com.database.spring.DatabaseDAOImpl$$EnhancerByCGLIB$$7d20ef95.loadTable(<generated>)
During initialization, the changes outlined above seem to work for a few calls to the loadTable function but when it gets around to loading an entity with a parent, I get the "collection with cascade="all-delete-orphan" was no longer referenced" error. Since I have not touched any other code that sets/gets parents or children and am only trying to fix the DAO method, and the query is only doing a sql SELECT, can anyone see why the code got broken?
The problem seems similar to Spring transaction management breaks hibernate cascade
This is unlikely problem of Spring, but rather issue with your entity handling / definition. When you are using deleteOrphans on a relation, the underlying PersistentSet MUST NOT be removed from the entity itself. You are allowed only to modify the set instance itself. So if you are trying to do anything clever within your entity setters, that is the cause.
Also as far as I remember there are some issues when you have deleteOrphans on both sides of the relation and/or load/manipulate both sides within one session.
Btw. I don't think "hibernate.current_session_context_class", "org.hibernate.context.ThreadLocalSessionContext" is necessary. In our project, this is the only configuration we have:
#Bean
public LocalSessionFactoryBuilder sessionFactoryBuilder() {
return ((LocalSessionFactoryBuilder) new LocalSessionFactoryBuilder(
dataSourceConfig.dataSource()).scanPackages(ENTITY_PACKAGES).
setProperty("hibernate.id.new_generator_mappings", "true").
setProperty("hibernate.dialect", dataSourceConfig.dialect()).
setProperty("javax.persistence.validation.mode", "none"));
}
#Bean
public SessionFactory sessionFactory() {
return sessionFactoryBuilder().buildSessionFactory();
}
The issue was with Session Management. The same block of transactional code was being called by other modules that were doing their own session handling. To add to our woes, some of the calling modules were Spring beans while others were written in direct Hibernate API style. This disorganization was sufficient work to keep us away from moving up to Hibernate 4 immediately.
Moral of the lesson (how do you like that English?): Use a consistent DAO implementation across the entire project and stick to a clearly defined session and transaction management strategy.

Resources