we are building one application where we need to log Entity updates in to History table. I am trying to achieve this by hibernate interceptor, and we could able to mange to get all the changes but having difficulties in inserting them into audit table.
My JPA configuration
public class JPAConfiguration {
----
#Bean(name = "entityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactoryBean() throws SQLException {
LocalContainerEntityManagerFactoryBean factoryBean = new LocalContainerEntityManagerFactoryBean();
factoryBean.setDataSource(dataSource());
factoryBean.setPackagesToScan(new String[] {"com.yyy.persist"});
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
vendorAdapter.setShowSql(true);
// thsi is required in order to enable Query DSL
vendorAdapter.setDatabasePlatform("org.hibernate.dialect.Oracle10gDialect");
factoryBean.setJpaVendorAdapter(vendorAdapter);
// factoryBean.setMappingResources(mappingResources);
// adding hibernate interceptor
Properties jpaProperties = new Properties();
jpaProperties.setProperty("hibernate.ejb.interceptor", "com.yyy.admin.service.AuditInterceptor");
factoryBean.setJpaProperties(jpaProperties);
return factoryBean;
}
My Interceptor
public class AuditInterceptor extends EmptyInterceptor {
public boolean onFlushDirty(Object entity, Serializable id, Object[] currentState, Object[] previousState,
String[] propertyNames, Type[] types) {
if ( entity instanceof Auditable ) {
// updates++;
for (int i = 0; i < propertyNames.length; i++) {
if ((currentState[i] == null && previousState[i] != null)
|| (currentState[i] != null && previousState[i] == null) || (currentState[i] != null
&& previousState[i] != null && !currentState[i].equals(previousState[i]))) {
AuditLog audit = new AuditLog();
audit.setAction("UPDATE");
audit.setFieldChanged(propertyNames[i]);
audit.setOldvalue(previousState[i] != null ? previousState[i].toString() : "");
audit.setNewvalue(currentState[i] != null ? currentState[i].toString() : "");
audit.setTimeStamp(new Date());
audit.setUsername(userName);
entities.add(audit);
}
}
// iterate elements on the report build a entity
}
return false;
}
public void afterTransactionCompletion(Transaction tx) {
if (tx.wasCommitted()) {
if (entities != null) {
for (AuditLog e : entities) {
System.out.println(e);
//.save(e);
}
entities = new ArrayList<AuditLog>();
}
}
}
}
in method afterTransactionCompletion I need to write all audit entities into DB, Autowire not working as this is not spring managed bean, is there any way to get DB session in this method so that I can perform inserts .?
The typical solution to inject Spring Beans into non-spring managed class is thru static resource holder. For example you have a class called StaticServiceHolder and annotate is with #Component then create static fields for the spring bean you want to inject thru setter. Like:
#Component
public class StaticServiceHolder
{
public static AuditService auditService;
#Autowired
public void setAuditService(AuditService auditService)
{
StaticServiceHolder.auditService = auditService;
}
}
Or Even easier if you have a lot of these stuff need to be injected, then you can Autowire the ApplicationContext. This way you can get whatever bean you need.
#Component
public class ApplicationContextHolder implements ApplicationContextAware {
public static ApplicationContext applicationContext;
#Override
public void setApplicationContext(ApplicationContext ctx) {
ApplicationContextHolder.applicationContext = ctx;
}
}
....
//in your hibernate interceptor
YourAuditService auditService = ApplicationContextHolder.applicationContext.getBean(YourAuditService.class);
auditService.saveAuditLog();
Either way, you should be able to persist your stuff in DB as long as the service you are using is Transactional. Hope this work for you.
For Transaction manager setup:
#Bean
public PlatformTransactionManager transactionManager(EntityManagerFactory emf)
{
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(emf);
return transactionManager;
}
I know this is a little late, but maybe helpful for others.
By using the spring bean instead of class name as value for "hibernate.ejb.interceptor", hibernate takes the spring bean instead of instantiating a new class.
More to find here:
Autowired to hibernate Interceptor
Related
I am using #converter (Hibernate )to convert pojo in encrypted format which is from hibernate but key are placed in property file which would not be resolve by #propertySource (Spring annotation)
is there any way to manage bean creation seq in above case.
Please find the below code snippet for Converter, I had created another bean from encryption/decryption, but you can create config bean for properties and read properties from there.
#Component
#Converter
#Configurable
public class HashMapConverter implements AttributeConverter<Map<String, Object>, String> {
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
/*
* Define your application properties bean here and read the properties from
* there
*/
private static ConfigEncryptionKeyConverter configEncryptionKeyConverter;
#Autowired
public void initEncryptionKeyConverter(ConfigEncryptionKeyConverter configEncryptionKeyConverter) {
// Set your beans here.
HashMapConverter.configEncryptionKeyConverter = configEncryptionKeyConverter;
}
#Override
public String convertToDatabaseColumn(Map<String, Object> attribute) {
try {
return configEncryptionKeyConverter.convertToDatabaseColumn(OBJECT_MAPPER.writeValueAsString(attribute));
} catch (final JsonProcessingException e) {
throw new ApplicationErrorException(e.getLocalizedMessage());
}
}
#SuppressWarnings("unchecked")
#Override
public Map<String, Object> convertToEntityAttribute(String dbData) {
Map<String, Object> attribute = null;
if (dbData != null) {
try {
attribute = OBJECT_MAPPER.readValue(configEncryptionKeyConverter.convertToEntityAttribute(dbData),
Map.class);
} catch (final IOException e) {
throw new ApplicationErrorException(e.getLocalizedMessage());
}
}
return attribute;
}
}
Hope this will help.
I want to perform transaction across database using JTA (using atomikos) configuration.
I have below code which I wanted to perform in one transaction. However when I run the application, it saves entityObject1 and update eventObject2 and doesnt rollback when an exception is thrown when i run l.intValue() statement. below is all code that I am using with configuration for JTA.
Am i missing anything? Could anyone please help.
public void testJTATRansaction() {
service1.saveEvent1(eventObject1);
service2.updateEvent2(eventObject2);
}
saveEvent1 method in service1:
#Transactional(propagation=Propagation.REQUIRED, rollbackFor = Exception.class)
public int saveEvent1(Object eventObject1) {
return repository1.save(eventObject1);
}
updateEvent2 method in service2:
#Transactional(propagation=Propagation.REQUIRED, rollbackFor = Exception.class)
public int updateEvent2(Object eventObject2) {
int i = l.intValue(); //l is null object, to throw error
return repository2.updateEvent2(eventObject2);
}
I am using default save method from repository1 (JPARepository save method).
updateEvent2 method in repository2 class:
#Modifying
#Transactional(propagation=Propagation.REQUIRED, rollbackFor = Exception.class)
#Query(UPDATE_EVENTS)
public int updateEvent2(
#Param(value = "eventObject2") Object eventObject2);
I am using spring boot application class to initialise my application:
#SpringBootApplication
#ComponentScan("com.cbc.event")
#EnableTransactionManagement
public class RatingDaemonApplication {
public static void main(String[] args) throws Exception {
SpringApplication.run(RatingDaemonApplication.class);
}
}
I have below JTA configuration:
#Configuration
#ComponentScan
#EnableTransactionManagement
public class JTATransactionConfig {
#Bean
public JpaVendorAdapter jpaVendorAdapter() {
HibernateJpaVendorAdapter hibernateJpaVendorAdapter = new HibernateJpaVendorAdapter();
hibernateJpaVendorAdapter.setShowSql(true);
hibernateJpaVendorAdapter.setGenerateDdl(true);
hibernateJpaVendorAdapter.setDatabase(Database.MYSQL);
return hibernateJpaVendorAdapter;
}
#Bean(name = "userTransaction")
public UserTransaction userTransaction() throws Throwable {
UserTransactionImp userTransactionImp = new UserTransactionImp();
userTransactionImp.setTransactionTimeout(10000);
return userTransactionImp;
}
#Bean(name = "atomikosTransactionManager", initMethod = "init", destroyMethod = "close")
public TransactionManager atomikosTransactionManager() throws Throwable {
UserTransactionManager userTransactionManager = new UserTransactionManager();
userTransactionManager.setForceShutdown(false);
AtomikosJtaPlatform.transactionManager = userTransactionManager;
return userTransactionManager;
}
#Bean(name = "transactionManager")
#DependsOn({ "userTransaction", "atomikosTransactionManager" })
public PlatformTransactionManager transactionManager() throws Throwable {
UserTransaction userTransaction = userTransaction();
AtomikosJtaPlatform.transaction = userTransaction;
TransactionManager atomikosTransactionManager = atomikosTransactionManager();
return new JtaTransactionManager(userTransaction, atomikosTransactionManager);
}
}
and datasource configuration is:
#Configuration
#DependsOn("transactionManager")
#PropertySource({"classpath:application.properties"})
#EnableJpaRepositories(basePackages = {"com.cbc.repository"},
transactionManagerRef="transactionManager", entityManagerFactoryRef = "entityMF")
public class dataSourceConfiguration {
#Autowired
Environment env;
#Autowired
JpaVendorAdapter jpaVendorAdapter;
public DataSource eventsDS() {
AtomikosDataSourceBean xaDS = new AtomikosDataSourceBean();
xaDS.setXaDataSourceClassName(env.getProperty(DRIVER_CLASS_NAME));
xaDS.setXaDataSource(getMysqlXADataSource());
xaDS.setUniqueResourceName("DS");
xaDS.setMaxPoolSize(3);
return xaDS;
}
private MysqlXADataSource getMysqlXADataSource() {
MysqlXADataSource ds = new MysqlXADataSource();
ds.setPinGlobalTxToPhysicalConnection(true);
ds.setURL(env.getProperty(URL));
ds.setUser(env.getProperty(USER));
ds.setPassword(env.getProperty(PASSWORD));
return ds;
}
#Bean(name="entityMF")
public LocalContainerEntityManagerFactoryBean importedEventsEntityMF() {
Map<String, Object> properties = new HashMap<>();
properties.put("hibernate.transaction.jta.platform", AtomikosJtaPlatform.class.getName());
properties.put("javax.persistence.transactionType", "JTA");
LocalContainerEntityManagerFactoryBean entityManager = new LocalContainerEntityManagerFactoryBean();
entityManager.setJtaDataSource(eventsDS());
entityManager.setJpaVendorAdapter(jpaVendorAdapter);
entityManager.setPackagesToScan("com.cbc.events");
entityManager.setPersistenceUnitName("persistenceUnit");
entityManager.setJpaPropertyMap(properties);
return entityManager;
}
}
I have below AtomikosJtaPlatform class
public class AtomikosJtaPlatform extends AbstractJtaPlatform {
private static final long serialVersionUID = 1L;
static TransactionManager transactionManager;
static UserTransaction transaction;
#Override
protected TransactionManager locateTransactionManager() {
return transactionManager;
}
#Override
protected UserTransaction locateUserTransaction() {
return transaction;
}
}
This is from spring documentation
When the propagation setting is PROPAGATION_REQUIRED, a logical transaction scope is created for each method upon which the setting is applied. Each such logical transaction scope can determine rollback-only status individually, with an outer transaction scope being logically independent from the inner transaction scope. Of course, in case of standard PROPAGATION_REQUIRED behavior, all these scopes will be mapped to the same physical transaction. So a rollback-only marker set in the inner transaction scope does affect the outer transaction's chance to actually commit (as you would expect it to).
However, in the case where an inner transaction scope sets the rollback-only marker, the outer transaction has not decided on the rollback itself, and so the rollback (silently triggered by the inner transaction scope) is unexpected. A corresponding UnexpectedRollbackException is thrown at that point. This is expected behavior so that the caller of a transaction can never be misled to assume that a commit was performed when it really was not. So if an inner transaction (of which the outer caller is not aware) silently marks a transaction as rollback-only, the outer caller still calls commit. The outer caller needs to receive an UnexpectedRollbackException to indicate clearly that a rollback was performed instead.
Transaction propagation
Try changing the method declarations as below and give it a go
public int saveEvent1(Object eventObject1) throws UnexpectedRollbackException
public int updateEvent2(Object eventObject2) throws UnexpectedRollbackException
To avoid such things Its a good idea to have a separate method in one of those service classes or a completely different service class , and call both repository operations in one go , with transaction annotation
Also when you have the service methods annotated with transaction annotation then you dont need to annotate you repository methods , the more annotations you have related to transactions more complex it is to resolve issue.
Using h2 datasource,the distributed transaction is success.
But use mysql datasource,it is tested fail.
(1) First doubt the atomikos do not support MysqlXADataSource good.
(2) second think the JPA and hibernate is not support JTA so good.
Then I tink use jdbc.
#Configuration
public class ArticleConfigure {
#ConfigurationProperties("second.datasource")
#Bean(name="articleDataSourceProperties")
public DataSourceProperties secondDataSourceProperties() {
return new DataSourceProperties();
}
//#Bean(name = "articleDataSource")
#Bean(name = "articleDataSource")
public DataSource articleDataSource() {
MysqlXADataSource mdatasource = new MysqlXADataSource();
mdatasource.setUrl(secondDataSourceProperties().getUrl());
mdatasource.setUser(secondDataSourceProperties().getUsername());
mdatasource.setPassword(secondDataSourceProperties().getPassword());
/*JdbcDataSource h2XaDataSource = new JdbcDataSource();
h2XaDataSource.setURL(secondDataSourceProperties().getUrl());*/
//atomikos datasource configure
com.atomikos.jdbc.AtomikosDataSourceBean xaDataSource = new AtomikosDataSourceBean();
xaDataSource.setXaDataSource(mdatasource);
xaDataSource.setMaxPoolSize(30);
xaDataSource.setUniqueResourceName("axds1");
return xaDataSource;
}
#Bean(name = "twojdbcTemplate")
public JdbcTemplate twojdbcTemplate() {
return new JdbcTemplate(articleDataSource());
}
}
TransactionConfig.
#Configuration
#EnableTransactionManagement
#ComponentScan(basePackages="cn.crazychain")
public class TransactionConfig {
#Bean(name = "userTransaction")
public UserTransaction userTransaction() throws Throwable {
UserTransactionImp userTransactionImp = new UserTransactionImp();
userTransactionImp.setTransactionTimeout(10000);
//return new BitronixTransactionManager();
return userTransactionImp;
}
#Bean(name = "atomikosTransactionManager", initMethod = "init", destroyMethod = "close")
//#Bean(name = "atomikosTransactionManager")
public TransactionManager atomikosTransactionManager() throws Throwable {
UserTransactionManager userTransactionManager = new UserTransactionManager();
userTransactionManager.setForceShutdown(false);
//return TransactionManagerServices.getTransactionManager();
return userTransactionManager;
}
#Bean(name = "customerJtaTransactionManager")
#DependsOn({ "userTransaction", "atomikosTransactionManager" })
public PlatformTransactionManager transactionManager() throws Throwable {
UserTransaction userTransaction = userTransaction();
TransactionManager atomikosTransactionManager = atomikosTransactionManager();
return new JtaTransactionManager(userTransaction, atomikosTransactionManager);
}
}
Whether there is bug in hibernate jpa whith ax.
The conclusion is that JTA works fine with jdbc and AtomikosDataSourceBean.
The origin reference open source project is https://github.com/fabiomaffioletti/mul-at.git
The whole source code of mine is
https://github.com/lxiaodao/crazychain.
JTA transaction manager will only work if you use JNDI. JTA tx manager listens to Datasource and bring under a transaction only if the datasource bean is in Java/Web container and not in app. container.
Either you need to use JNDI for JTA to work or start using JPA transaction manager.
JTA transaction manager is mainly used in Distributed Transaction and is prone to transaction rollback failures.
We need to track database metrics so we are using datasource-proxy to track this to integrate the same in spring boot project we have created custom datasource
as below
#Component
#Slf4j
#ConfigurationProperties(prefix = "spring.datasource")
public class DataSourceBeanConfig
{
public DataSource actualDataSource()
{
EmbeddedDatabaseBuilder databaseBuilder = new EmbeddedDatabaseBuilder();
return databaseBuilder.setType(EmbeddedDatabaseType.H2).build();
}
#Bean
#Primary
public DataSource dataSource() {
// use pretty formatted query with multiline enabled
PrettyQueryEntryCreator creator = new PrettyQueryEntryCreator();
creator.setMultiline(true);
log.info("Inside Proxy Creation");
SystemOutQueryLoggingListener listener = new SystemOutQueryLoggingListener();
listener.setQueryLogEntryCreator(creator);
return ProxyDataSourceBuilder
.create(actualDataSource())
.countQuery()
.name("MyDS")
.listener(listener)
.build();
}
}
When we run main application datasource-proxy is picked up but when we use #DataJpaTest it is not picking up. How to enable datasource-proxy in JUNIT test cases?
Edit::
Using Spring BeanPostProcessor to configure Proxy DataSource
#Slf4j
#Configuration
public class DataSourceBeanConfig implements BeanPostProcessor
{
public Object postProcessAfterInitialization(Object bean, String beanName)
throws BeansException
{
if (bean instanceof DataSource)
{
System.out.println("AfterInitialization : " + beanName);
// use pretty formatted query with multiline enabled
PrettyQueryEntryCreator creator = new PrettyQueryEntryCreator();
creator.setMultiline(true);
log.info("Inside Proxy Creation");
SystemOutQueryLoggingListener listener = new SystemOutQueryLoggingListener();
listener.setQueryLogEntryCreator(creator);
return ProxyDataSourceBuilder.create((DataSource) bean).countQuery()
.name("MyDS").listener(listener).build();
}
return bean; // you can return any other object as well
}
}
Here is the solution we need to create TestConfiguration to use in #DataJpaTest
#RunWith(SpringRunner.class)
#DataJpaTest
public class DataTestJPA
{
#TestConfiguration
static class ProxyDataSourceConfig implements BeanPostProcessor
{
public Object postProcessAfterInitialization(Object bean, String beanName)
throws BeansException
{
if (bean instanceof DataSource)
{
return ProxyDataSourceBuilder
.create((DataSource) bean)
.countQuery()
.name("MyDS")
.build();
// #formatter:on
}
return bean; // you can return any other object as well
}
}
}
Eclipse Link Multitenancy is not working properly.
Example Entity (the schema is being created by liquibase):
#Entity
#Table(name = "ENTITIES")
#Multitenant(MultitenantType.SINGLE_TABLE)
#TenantDiscriminatorColumn(name = "TENANT_ID", contextProperty = "eclipselink.tenant-id")
public class EntityClass
To set the multitenancy property on entity managers I use an aspect, like following:
#Around("execution(* javax.persistence.EntityManagerFactory.*(..))")
public Object invocate(ProceedingJoinPoint joinPoint) throws Throwable {
final Object result = joinPoint.proceed();
if (result instanceof EntityManager) {
EntityManager em = (EntityManager) result;
final String tenantId = TenantContext.getCurrentTenantId();
LOG.debug("Set EntityManager property for tenant {}.", tenantId);
em.setProperty(EntityManagerProperties.MULTITENANT_PROPERTY_DEFAULT,
tenantId);
return em;
}
return result;
}
When I start the Spring Boot application this works perfectly. To have tenant information available during integration tests, I defined an annotation:
#Target({ElementType.TYPE, ElementType.METHOD})
#Retention(RetentionPolicy.RUNTIME)
public #interface AsTenant {
String value();
}
To bind this value, I use a TestExecutionListener:
#Override
public void beforeTestMethod(TestContext testContext) throws Exception {
final Method testMethod = testContext.getTestMethod();
final AsTenant asTenantAnnotation = testMethod
.getAnnotation(AsTenant.class);
if (asTenantAnnotation != null) {
TenantContext.setCurrentTenantId(asTenantAnnotation.value());
}
}
By debugging I can clearly say that the TestExectionListener is called before any EM is created and that the property is properly set for the EMs. When persisting anything to the database, Eclipse Link does not set a value for the column.
Maybe anybody can help me out with this, I have no Idea why EclipseLink Multitenancy is not working.
Ok, I got it working. If anybody ever faces a similar problem, here is my solution to it.
If using transactions, the context property for the tenant discrimination has to be set after the transaction is started (http://www.eclipse.org/eclipselink/documentation/2.5/solutions/multitenancy002.htm).
EntityManager em = createEntityManager(MULTI_TENANT_PU);
em.getTransaction().begin();
em.setProperty(EntityManagerProperties.MULTITENANT_PROPERTY_DEFAULT, "my_id");
To realise this in Spring Boot/Data environment, I customized Spring's JpaTransactionManager. This stays in addition to the Aspect in the Question, since there is not Transaction for SELECT queries.
public class MultitenantJpaTransactionManager extends JpaTransactionManager {
/* (non-Javadoc)
* #see org.springframework.orm.jpa.JpaTransactionManager#doBegin(java.lang.Object, org.springframework.transaction.TransactionDefinition)
*/
#Override
protected void doBegin(Object transaction, TransactionDefinition definition) {
super.doBegin(transaction, definition);
final EntityManagerHolder emHolder = (EntityManagerHolder) TransactionSynchronizationManager.getResource(getEntityManagerFactory());
final EntityManager em = emHolder.getEntityManager();
final String tenantId = TenantContext.getCurrentTenantId();
if (tenantId != null) {
em.setProperty(EntityManagerProperties.MULTITENANT_PROPERTY_DEFAULT, tenantId);
}
}
}
This is easily wired via JpaConfiguration:
/**
* Configures Eclipse Link as JPA Provider.
*/
#Configuration
#EnableTransactionManagement
#AutoConfigureAfter({ DataSourceAutoConfiguration.class })
public class JpaConfiguration extends JpaBaseConfiguration {
#Bean
#Override
public PlatformTransactionManager transactionManager() {
return new MultitenantJpaTransactionManager();
}
#Override
protected AbstractJpaVendorAdapter createJpaVendorAdapter() {
EclipseLinkJpaVendorAdapter adapter = new EclipseLinkJpaVendorAdapter();
return adapter;
}
#Override
protected Map<String, Object> getVendorProperties() {
HashMap<String, Object> properties = new HashMap<String, Object>();
properties.put(PersistenceUnitProperties.WEAVING, detectWeavingMode());
return properties;
}
private String detectWeavingMode() {
return InstrumentationLoadTimeWeaver.isInstrumentationAvailable()
? "true" : "static";
}
}
Disclaimer: This does not answer the above query but provides an alternative.
Using bytecode instrumentation, I have created a java example on Multi-Tenancy (Table per Tenant) with Eclipse Link and Spring Data. This idea is chosen to utilize the complete power of Spring Data.
One can execute MultiTenantTest to see it working.
The idea is open-sourced and is available at Maven Central
Steps:
1.Include dependency
<dependency>
<groupId>org.bitbucket.swattu</groupId>
<artifactId>jpa-agent</artifactId>
<version>2.0.2</version>
</dependency>
2.Create a class as shown below. Package, Class and method has to be exactly same.
package org.swat.jpa.base;
import javax.persistence.EntityManager;
public class EntityManagerFactoryListener {
/**
* This method is called by JPA Agent.
*
* #param entityManager the entity manager
*/
public static void afterCreateEntityManager(EntityManager entityManager) {
//Business logic to set appropriate values in entityManager
final String tenantId = TenantContext.getCurrentTenantId();
if (tenantId != null) {
em.setProperty(EntityManagerProperties.MULTITENANT_PROPERTY_DEFAULT, tenantId);
}
}
}
3.Add javaagent when starting java
-javaagent:{path-to-jpa-agent-jar}
Context
I'm trying to develop a batch service with Spring Boot, using JPA Repository. Using two different datasources, I want the batch-related tables created in a in-memory database, so that it does not pollute my business database.
Following multiple topics on the web, I came up with this configuration of my two datasources :
#Configuration
public class DataSourceConfiguration {
#Bean(name = "mainDataSource")
#Primary
#ConfigurationProperties(prefix="spring.datasource")
public DataSource mainDataSource(){
return DataSourceBuilder.create().build();
}
#Bean(name = "batchDataSource")
public DataSource batchDataSource( #Value("${batch.datasource.url}") String url ){
return DataSourceBuilder.create().url( url ).build();
}
}
The first one, mainDataSource, uses the default Spring database configuration. The batchDataSource defines an embedded HSQL database, in which I want the batch and step tables to be created.
# DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url=jdbc:mariadb://localhost:3306/batch_poc
spring.datasource.username=root
spring.datasource.password=
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.datasource.max-age=10000
spring.datasource.initialize=false
# JPA (JpaBaseConfiguration, HibernateJpaAutoConfiguration)
spring.jpa.generate-ddl=false
spring.jpa.show-sql=true
spring.jpa.database=MYSQL
# SPRING BATCH (BatchDatabaseInitializer)
spring.batch.initializer.enabled=false
# ----------------------------------------
# PROJECT SPECIFIC PROPERTIES
# ----------------------------------------
# BATCH DATASOURCE
batch.datasource.url=jdbc:hsqldb:file:C:/tmp/hsqldb/batchdb
Here is my batch config :
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
private static final Logger LOG = Logger.getLogger( BatchConfiguration.class );
#Bean
public BatchConfigurer configurer(){
return new CustomBatchConfigurer();
}
#Bean
public Job importElementsJob( JobBuilderFactory jobs, Step step1 ){
return jobs.get("importElementsJob")
.incrementer( new RunIdIncrementer() )
.flow( step1 )
.end()
.build();
}
#Bean
public Step step1( StepBuilderFactory stepBuilderFactory, ItemReader<InputElement> reader,
ItemWriter<List<Entity>> writer, ItemProcessor<InputElement, List<Entity>> processor ){
return stepBuilderFactory.get("step1")
.<InputElement, List<Entity>> chunk(100)
.reader( reader )
.processor( processor )
.writer( writer )
.build();
}
#Bean
public ItemReader<InputElement> reader() throws IOException {
return new CustomItemReader();
}
#Bean
public ItemProcessor<InputElement, List<Entity>> processor(){
return new CutsomItemProcessor();
}
#Bean
public ItemWriter<List<Entity>> writer(){
return new CustomItemWriter();
}
}
The BatchConfigurer, using the in-memory database :
public class CustomBatchConfigurer extends DefaultBatchConfigurer {
#Override
#Autowired
public void setDataSource( #Qualifier("batchDataSource") DataSource dataSource) {
super.setDataSource(dataSource);
}
}
And, finally, my writer :
public class CustomItemWriter implements ItemWriter<List<Entity>> {
private static final Logger LOG = Logger.getLogger( EntityWriter.class );
#Autowired
private EntityRepository entityRepository;
#Override
public void write(List<? extends List<Entity>> items)
throws Exception {
if( items != null && !items.isEmpty() ){
for( List<Entity> entities : items ){
for( Entity entity : entities ){
Entity fromDb = entityRepository.findById( entity.getId() );
// Insert
if( fromDb == null ){
entityRepository.save( entity );
}
// Update
else {
// TODO : entityManager.merge()
}
}
}
}
}
}
The EntityRepository interface extends JpaRepository.
Problem
When I separate the datasources this way, nothing happens when I call the save method of the repository. I see the select queries from the call of findById() in the logs. But nothing for the save. And my output database is empty at the end.
When I come back to a unique datasource configuration (removing the configurer bean and letting Spring Boot manage the datasource alone), the insert queries work fine.
Maybe the main datasource configuration is not good enough for JPA to perform the inserts correctly. But what is missing ?
I finally solved the problem implementing my own BatchConfigurer, on the base of the Spring class BasicBatchConfigurer, and forcing the use of Map based jobRepository and jobExplorer. No more custom datasource configuration, only one datasource which I let Spring Boot manage : it's easier that way.
My custom BatchConfigurer :
public class CustomBatchConfigurer implements BatchConfigurer {
private static final Logger LOG = Logger.getLogger( CustomBatchConfigurer.class );
private final EntityManagerFactory entityManagerFactory;
private PlatformTransactionManager transactionManager;
private JobRepository jobRepository;
private JobLauncher jobLauncher;
private JobExplorer jobExplorer;
/**
* Create a new {#link CustomBatchConfigurer} instance.
* #param entityManagerFactory the entity manager factory
*/
public CustomBatchConfigurer( EntityManagerFactory entityManagerFactory ) {
this.entityManagerFactory = entityManagerFactory;
}
#Override
public JobRepository getJobRepository() {
return this.jobRepository;
}
#Override
public PlatformTransactionManager getTransactionManager() {
return this.transactionManager;
}
#Override
public JobLauncher getJobLauncher() {
return this.jobLauncher;
}
#Override
public JobExplorer getJobExplorer() throws Exception {
return this.jobExplorer;
}
#PostConstruct
public void initialize() {
try {
// transactionManager:
LOG.info("Forcing the use of a JPA transactionManager");
if( this.entityManagerFactory == null ){
throw new Exception("Unable to initialize batch configurer : entityManagerFactory must not be null");
}
this.transactionManager = new JpaTransactionManager( this.entityManagerFactory );
// jobRepository:
LOG.info("Forcing the use of a Map based JobRepository");
MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean( this.transactionManager );
jobRepositoryFactory.afterPropertiesSet();
this.jobRepository = jobRepositoryFactory.getObject();
// jobLauncher:
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(getJobRepository());
jobLauncher.afterPropertiesSet();
this.jobLauncher = jobLauncher;
// jobExplorer:
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
jobExplorerFactory.afterPropertiesSet();
this.jobExplorer = jobExplorerFactory.getObject();
}
catch (Exception ex) {
throw new IllegalStateException("Unable to initialize Spring Batch", ex);
}
}
}
My configuration class looks like this now :
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Bean
public BatchConfigurer configurer( EntityManagerFactory entityManagerFactory ){
return new CustomBatchConfigurer( entityManagerFactory );
}
[...]
}
And my properties files :
# DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url=jdbc:mariadb://localhost:3306/inotr_poc
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.datasource.max-age=10000
spring.datasource.initialize=true
# JPA (JpaBaseConfiguration, HibernateJpaAutoConfiguration)
spring.jpa.generate-ddl=false
spring.jpa.show-sql=true
spring.jpa.database=MYSQL
# SPRING BATCH (BatchDatabaseInitializer)
spring.batch.initializer.enabled=false
Thanks for the above posts! I have been struggling for the past couple of days to get my Spring Boot with Batch application working with an in-memory Map based job repository and a resourceless transaction manager. ( I cannot let Spring Batch to use my application datasource for the Batch meta data tables as I don't have the DDL access to create the BATCH_ tables there )
Finally arrived at the below configuration after looking at the above posts and it worked perfectly!!
public class CustomBatchConfigurer implements BatchConfigurer {
private static final Logger LOG = LoggerFactory.getLogger(CustomBatchConfigurer.class);
// private final EntityManagerFactory entityManagerFactory;
private PlatformTransactionManager transactionManager;
private JobRepository jobRepository;
private JobLauncher jobLauncher;
private JobExplorer jobExplorer;
/**
* Create a new {#link CustomBatchConfigurer} instance.
* #param entityManagerFactory the entity manager factory
public CustomBatchConfigurer( EntityManagerFactory entityManagerFactory ) {
this.entityManagerFactory = entityManagerFactory;
}
*/
#Override
public JobRepository getJobRepository() {
return this.jobRepository;
}
#Override
public PlatformTransactionManager getTransactionManager() {
return this.transactionManager;
}
#Override
public JobLauncher getJobLauncher() {
return this.jobLauncher;
}
#Override
public JobExplorer getJobExplorer() throws Exception {
return this.jobExplorer;
}
#PostConstruct
public void initialize() {
try {
// transactionManager:
LOG.info("Forcing the use of a Resourceless transactionManager");
this.transactionManager = new ResourcelessTransactionManager();
// jobRepository:
LOG.info("Forcing the use of a Map based JobRepository");
MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean( this.transactionManager );
jobRepositoryFactory.afterPropertiesSet();
this.jobRepository = jobRepositoryFactory.getObject();
// jobLauncher:
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(getJobRepository());
jobLauncher.afterPropertiesSet();
this.jobLauncher = jobLauncher;
// jobExplorer:
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
jobExplorerFactory.afterPropertiesSet();
this.jobExplorer = jobExplorerFactory.getObject();
}
catch (Exception ex) {
throw new IllegalStateException("Unable to initialize Spring Batch", ex);
}
}
}
And below is the bean i added in my job configuration class
#Bean
public BatchConfigurer configurer(){
return new CustomBatchConfigurer();
}
Eria's answer worked! However, I had modified it to use:
org.springframework.batch.support.transaction.ResourcelessTransactionManager
From CustomBatchConfigurer:
#PostConstruct
public void initialize() {
try {
// transactionManager:
LOGGER.info("Forcing the use of ResourcelessTransactionManager for batch db");
this.transactionManager = new ResourcelessTransactionManager();
//the rest of the code follows...
}