Spring boot quartz 2.2.1 compilation error - spring

This is the code i have used for saving new cron job to the postgresql database.But it is showing some compile time exceptions.I have originally migrated quartz 1.8.6 to 2.2.1 with spring boot web app.see the attachment.
private static final String JOBNAME = "sampleJobRunner";
private static final String JOB_GROUPNAME = "sampleGroup";
public void saveScheduledTime(String name) throws SchedulerException, ParseException {
Scheduler scheduler;
CronTrigger trigger;
String cronExp = null;
scheduler = this.quartzScheduler.getObject();
trigger = (CronTrigger) scheduler.getTrigger(name, JOB_GROUPNAME);
if (scheduler != null) {
cronExp = "0 0 12 ? 1 MON#1 *";
if (trigger != null) {
rescheduleJob(scheduler, trigger.getName(), cronExp);
} else {
createNewCronTrigger(scheduler, cronExp, name);
}
}
}
private void rescheduleJob(Scheduler scheduler, String triggerName, String cronExp)
throws SchedulerException, ParseException {
CronTrigger trigger = (CronTrigger) scheduler.getTrigger(triggerName, JOB_GROUPNAME);
trigger.setCronExpression(cronExp);
scheduler.rescheduleJob(triggerName, JOB_GROUPNAME, trigger);
}
private void createNewCronTrigger(Scheduler scheduler, String cronExp, String sentType)
throws SchedulerException, ParseException {
CronTrigger trigger = new CronTrigger(sentType, JOB_GROUPNAME, JOBNAME, JOB_GROUPNAME, cronExp);
scheduler.scheduleJob(trigger);
scheduler.start();
}
Spring boot quartz configuration
#Configuration
#ConditionalOnProperty(name = "quartz.enabled")
public class SchedulerConfig {
#Bean
public JobFactory jobFactory(ApplicationContext applicationContext) {
AutowiringSpringBeanJobFactory jobFactory = new AutowiringSpringBeanJobFactory();
jobFactory.setApplicationContext(applicationContext);
return jobFactory;
}
#Bean
public SchedulerFactoryBean schedulerFactoryBean(DataSource dataSource, JobFactory jobFactory,
#Qualifier("sampleJobTrigger") Trigger sampleJobTrigger) throws IOException {
SchedulerFactoryBean factory = new SchedulerFactoryBean();
// this allows to update triggers in DB when updating settings in config
// file:
factory.setOverwriteExistingJobs(true);
factory.setDataSource(dataSource);
factory.setJobFactory(jobFactory);
factory.setQuartzProperties(quartzProperties());
// factory.setTriggers(sampleJobTrigger);
return factory;
}
#Bean
public Properties quartzProperties() throws IOException {
PropertiesFactoryBean propertiesFactoryBean = new PropertiesFactoryBean();
propertiesFactoryBean.setLocation(new ClassPathResource("/quartz.properties"));
propertiesFactoryBean.afterPropertiesSet();
return propertiesFactoryBean.getObject();
}
#Bean
public JobDetailFactoryBean sampleJobDetail() {
return createJobDetail(SampleJob.class);
}
#Bean(name = "sampleJobTrigger")
public SimpleTriggerFactoryBean sampleJobTrigger(#Qualifier("sampleJobDetail") JobDetail jobDetail,
#Value("${samplejob.frequency}") long frequency) {
return createTrigger(jobDetail, frequency);
}
private static JobDetailFactoryBean createJobDetail(Class jobClass) {
JobDetailFactoryBean factoryBean = new JobDetailFactoryBean();
factoryBean.setJobClass(jobClass);
// job has to be durable to be stored in DB:
factoryBean.setDurability(true);
return factoryBean;
}
private static SimpleTriggerFactoryBean createTrigger(JobDetail jobDetail, long pollFrequencyMs) {
SimpleTriggerFactoryBean factoryBean = new SimpleTriggerFactoryBean();
factoryBean.setJobDetail(jobDetail);
factoryBean.setStartDelay(0L);
factoryBean.setRepeatInterval(pollFrequencyMs);
factoryBean.setRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY);
// in case of misfire, ignore all missed triggers and continue :
factoryBean.setMisfireInstruction(SimpleTrigger.MISFIRE_INSTRUCTION_RESCHEDULE_NEXT_WITH_REMAINING_COUNT);
return factoryBean;
}
}

To fix the complilation errors in your code:
use scheduler.getTrigger(new TriggerKey(name, JOB_GROUPNAME));
use scheduler.rescheduleJob(new TriggerKey(name, JOB_GROUPNAME), trigger);
cast CronTrigger to CronTriggerImpl which has the setCronExpression() method
See my sample project for Spring Boot + Quartz on github: https://github.com/davidkiss/spring-boot-quartz-demo. I updated the SchedulerConfig class with a new createCronTrigger helper method that allows to create cron triggers in Spring Boot:
private static CronTriggerFactoryBean createCronTrigger(JobDetail jobDetail, String cronExpression) {
CronTriggerFactoryBean factoryBean = new CronTriggerFactoryBean();
factoryBean.setJobDetail(jobDetail);
factoryBean.setCronExpression(cronExpression);
factoryBean.setMisfireInstruction(SimpleTrigger.MISFIRE_INSTRUCTION_FIRE_NOW);
return factoryBean;
}
If you use Spring Boot in your project, I believe using the SchedulerConfig class is a cleaner way of configuring quartz jobs in Spring Boot then calling the saveScheduledTime() method.
In order to create any cron triggers, I'd add additional methods in the SchedulerConfig class to create the appropriate trigger beans similar to sampleJobTrigger.

This is purely a result of the migration from Quartz 1.8.6 to 2.2.1. Quartz did some code changes with 2.x version. You need to make changes in your application to respect that. Have a look at this: http://www.quartz-scheduler.org/documentation/quartz-2.x/migration-guide.html

Related

Spring batch with quartz configuration not working - Looking for spring metadata table instead of quartz table

I have requirement to develop a spring boot batch using quartz as scheduler.
I have created spring batch application and also configured quartz to trigger job and persist job/trigger data in quartz related RDBMS tables.
Spring boot application starts successfully but when it triggers the job as per cron config. Its throws below error.
2020-04-27 20:42:09.967 INFO 1460 --- [ main] c.b.g.batch.IemsBatchApplication : Started IemsBatchApplication in 12.112 seconds (JVM running for 13.22)
org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? and JOB_KEY = ?]; nested exception is java.sql.SQLSyntaxErrorException: ORA-00942: table or view does not exist
I am not able to find the root cause that why application is trying to persist data in spring meta data table rather than quartz tables.
Quartz related tables are already created in my db.
Please help me.
Please find below the code for the same.
#Configuration
public class BatchSchedulerConfig {
#Autowired
#Qualifier("batchDataSource")
private DataSource batchDataSource;
#Autowired
JobLauncher jobLauncher;
#Autowired
JobLocator jobLocator;
#Bean
public JobDetailFactoryBean jobDetailFactoryBean() {
JobDetailFactoryBean jobDetailFactoryBean = new JobDetailFactoryBean();
jobDetailFactoryBean.setJobClass(BatchJobLauncherConfig.class);
Map<String, Object> map = new HashMap<String, Object>();
map.put("jobName", IEMSBatchConstant.MonthlyElectricityMeterReadingJob);
jobDetailFactoryBean.setJobDataAsMap(map);
jobDetailFactoryBean.setGroup("etl_group");
jobDetailFactoryBean.setName("etl_job");
return jobDetailFactoryBean;
}
#Bean
public CronTriggerFactoryBean cronTriggerFactoryBean(JobDetailFactoryBean jobDetailFactoryBean) {
CronTriggerFactoryBean cronTriggerFactoryBean = new CronTriggerFactoryBean();
cronTriggerFactoryBean.setJobDetail(jobDetailFactoryBean.getObject());
cronTriggerFactoryBean.setStartDelay(3000);
cronTriggerFactoryBean.setName("cron_trigger");
cronTriggerFactoryBean.setGroup("cron_group");
cronTriggerFactoryBean.setCronExpression("0 0/2 * 1/1 * ? *");
return cronTriggerFactoryBean;
}
#Bean
public SchedulerFactoryBean schedulerFactoryBean(CronTriggerFactoryBean cronTriggerFactoryBean) throws IOException {
SchedulerFactoryBean scheduler = new SchedulerFactoryBean();
Map<String, Object> map = new HashMap<String, Object>();
map.put("joblocator", jobLocator);
map.put("JobLauncher", jobLauncher);
scheduler.setSchedulerContextAsMap(map);
scheduler.setTriggers(cronTriggerFactoryBean.getObject());
scheduler.setDataSource(batchDataSource);
scheduler.setQuartzProperties(quartzProperties());
// scheduler.setSchedulerName(schedulerName);
return scheduler;
}
public Properties quartzProperties() throws IOException {
PropertiesFactoryBean propertiesFactoryBean = new PropertiesFactoryBean();
propertiesFactoryBean.setLocation(new ClassPathResource("/quartz.properties"));
propertiesFactoryBean.afterPropertiesSet();
return propertiesFactoryBean.getObject();
}
}
#Configuration
#EnableConfigurationProperties
public class BatchBaseConfig {
private static final Logger log = LoggerFactory.getLogger(BatchBaseConfig.class);
#Bean
public JobLauncher jobLauncher(JobRepository jobRepository) throws Exception {
SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
simpleJobLauncher.setJobRepository(jobRepository);
return simpleJobLauncher;
}
#Bean
public JobRepository jobRepository(DataSource batchDataSource) throws Exception {
DataSourceTransactionManager batchTransactionManager = new DataSourceTransactionManager();
JobRepositoryFactoryBean jobRepositoryFactoryBean = new JobRepositoryFactoryBean();
batchTransactionManager.setDataSource(batchDataSource);
jobRepositoryFactoryBean.setTransactionManager(batchTransactionManager);
jobRepositoryFactoryBean.setDatabaseType(DatabaseType.ORACLE.getProductName());
//jobRepositoryFactoryBean.setIsolationLevelForCreate("ISOLATION_DEFAULT");
jobRepositoryFactoryBean.setDataSource(batchDataSource);
jobRepositoryFactoryBean.setTablePrefix("QRTZ_");
jobRepositoryFactoryBean.afterPropertiesSet();
return jobRepositoryFactoryBean.getObject();
}
#Bean(name = "batchDataSource")
#Primary
#ConfigurationProperties(prefix = "quartz")
public DataSource batchDataSource() {
DataSource batchDbSrc = DataSourceBuilder.create().build();
return batchDbSrc;
}
#Bean(name = "appDataSource")
#ConfigurationProperties(prefix = "app")
public DataSource appDataSource() {
DataSource appDbSrc = DataSourceBuilder.create().build();
return appDbSrc;
}
#Bean
public JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor(JobRegistry jobRegistry) {
JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor = new JobRegistryBeanPostProcessor();
jobRegistryBeanPostProcessor.setJobRegistry(jobRegistry);
return jobRegistryBeanPostProcessor;
}
}
#Getter
#Setter
public class BatchJobLauncherConfig extends QuartzJobBean {
private static final Logger log = LoggerFactory.getLogger(BatchJobLauncherConfig.class);
private JobLauncher jobLauncher;
private JobLocator joblocator;
private String jobName;
#Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
try {
JobExecution jobExecution = jobLauncher.run(joblocator.getJob(jobName), new
JobParameters());
log.info("{}_{} was completed successfully", jobExecution.getJobConfigurationName(),
jobExecution.getId());
} catch (JobExecutionAlreadyRunningException | JobRestartException |
JobInstanceAlreadyCompleteException
| JobParametersInvalidException | NoSuchJobException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
#Configuration
public class JobConfig {
private static final Logger log = LoggerFactory.getLogger(JobConfig.class);
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private ReadingFromDB readingFromDB;
#Autowired
private ReadingFileWriter readingFileWriter;
#Qualifier(value = "meterReadingJob")
#Bean
public Job meterReadingJob() throws Exception {
log.info("Enter into meterReadingJob method ");
return this.jobBuilderFactory.get("meterReadingJob")
.start(createReadingFile()).build();
}
#Bean
public Step createReadingFile() throws Exception {
return this.stepBuilderFactory.get("createReadingFile").chunk(1)
.reader(readingFromDB).writer(readingFileWriter).build();
}
}
Application.properties
spring.batch.job.enabled=false
spring.main.allow-bean-definition-overriding=true
#Spring's default configuration to generate datasource
app.jdbcUrl=jdbc:oracle:thin:#//<Removed>
app.username=<Removed>
app.password=<Removed>
app.driverClassName=oracle.jdbc.OracleDriver
spring.app.jpa.database-platform=org.hibernate.dialect.Oracle10gDialect
#Quartz dataSource
quartz.jdbcUrl=jdbc:oracle:thin:#//<Removed>
quartz.username=<Removed>
quartz.password=<Removed>
quartz.driverClassName=oracle.jdbc.OracleDriver
Quartz.properties
#scheduler name will be "MyScheduler"
org.quartz.scheduler.instanceName=MyNewScheduler
org.quartz.scheduler.instanceId=AUTO
#maximum of 3 jobs can be run simultaneously
org.quartz.threadPool.class=org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount=50
#Quartz persistent jobStore config
org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.tablePrefix=QRTZ_
#org.quartz.jobStore.dataSource=batchDataSource
org.quartz.jobStore.useProperties=false
org.quartz.jobStore.isClustered=false
Both datasources are getting created and entries are also going for triggers and jobs in datasource names batchDataSource.
Issue comes when job gets triggered and batch tries to insert job details in DB.
Thanks in advance.

Spring Quartz: Disable Quartz scheduler

We have an service with a quartz scheduler.
This service can be scaled accordingly some needs.
Our Quartz scheduler is not in a cluster mode.
So, we need to able or disable scheduler according to an environment variable.
Service can't be splitted in order to have two independent services.
This is our related Quartz configuration class:
#Configuration
public class QuartzSchedulerConfiguration {
private static final String HOURLY_CRON_EXPRESSION = "0 0 * * * ?";
private static final String MIDNIGHT_CRON_EXPRESSION = "0 0 0 * * ?";
#Bean
public JobFactory jobFactory(ApplicationContext applicationContext) {
AutowiringSpringBeanJobFactory jobFactory = new AutowiringSpringBeanJobFactory();
jobFactory.setApplicationContext(applicationContext);
return jobFactory;
}
#Bean
public SchedulerFactoryBean schedulerFactoryBean(JobFactory jobFactory, Trigger[] fitxersJobTrigger)
throws IOException {
SchedulerFactoryBean factory = new SchedulerFactoryBean();
factory.setOverwriteExistingJobs(true);
factory.setAutoStartup(true);
factory.setJobFactory(jobFactory);
factory.setQuartzProperties(quartzProperties());
factory.setTriggers(fitxersJobTrigger);
return factory;
}
#Bean
public JobDetailFactoryBean loadPendingDocumentsJobDetail() {
return createJobDetailFactoryBean(LoadPendingDocumentsJob.class);
}
#Bean
public SimpleTriggerFactoryBean loadPendingDocumentsJobTrigger(
#Qualifier("loadPendingDocumentsJobDetail") JobDetail jobDetail) {
long interval = jobsConfiguration.get().getParameters().stream()
.filter(param -> "loadPendingDocumentsJobInterval".equals(param.getName()))
.findAny()
.map(param -> (Integer)param.getValue())
.orElse(600000); // every 10 minutes
LOG.debug("loadPendingDocumentsJobInterval = " + interval);
return createIntervalTriggerFactoryBean(jobDetail, interval);
}
private CronTriggerFactoryBean createCronTriggerFactoryBean(JobDetail jobDetail, String expression) {
CronTriggerFactoryBean factoryBean = new CronTriggerFactoryBean();
factoryBean.setJobDetail(jobDetail);
factoryBean.setCronExpression(expression);
factoryBean.setMisfireInstruction(SimpleTrigger.MISFIRE_INSTRUCTION_FIRE_NOW);
return factoryBean;
}
private JobDetailFactoryBean createJobDetailFactoryBean(Class<? extends Job> jobClass) {
JobDetailFactoryBean factoryBean = new JobDetailFactoryBean();
factoryBean.setJobClass(jobClass);
factoryBean.setDurability(true);
return factoryBean;
}
private SimpleTriggerFactoryBean createIntervalTriggerFactoryBean(JobDetail jobDetail, long interval) {
SimpleTriggerFactoryBean factoryBean = new SimpleTriggerFactoryBean();
factoryBean.setJobDetail(jobDetail);
factoryBean.setStartDelay(0L);
factoryBean.setRepeatInterval(interval);
factoryBean.setRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY);
factoryBean.setMisfireInstruction(SimpleTrigger.MISFIRE_INSTRUCTION_FIRE_NOW);
return factoryBean;
}
}
Any ideas?
You can try using the #ConditionalOnProperty Annotation on your configuration class.

Job doesn't reschedule immediately when triggers update in quartz

I'm trying to develop an spring application using Quartz which reads trigger time for Job from database and run it. I have managed to implement this scenario successfully.However, during job execution when i update the trigger time in database, it doesn't run the trigger the job according to the new time but it always run old time.
Code:
#Configuration
#ComponentScan
public class QuartzConfiguration {
#Bean
public MethodInvokingJobDetailFactoryBean methodInvokingJobDetailFactoryBean() {
MethodInvokingJobDetailFactoryBean obj = new MethodInvokingJobDetailFactoryBean();
obj.setTargetBeanName("jobone");
obj.setTargetMethod("myTask");
return obj;
}
#Autowired
public TestModelRepository testModelRepository;
#Bean
public JobDetailFactoryBean jobDetailFactoryBean(){
JobDetailFactoryBean factory = new JobDetailFactoryBean();
factory.setJobClass(MyJobTwo.class);
TestModel result = testModelRepository.findOne((long) 1);
factory.setGroup("mygroup");
factory.setName("myjob");
return factory;
}
public TestModelRepository getTestModelRepository() {
return testModelRepository;
}
public void setTestModelRepository(TestModelRepository testModelRepository) {
this.testModelRepository = testModelRepository;
}
#Bean
public CronTriggerFactoryBean cronTriggerFactoryBean(){
CronTriggerFactoryBean stFactory = new CronTriggerFactoryBean();
stFactory.setJobDetail(jobDetailFactoryBean().getObject());
//stFactory.setStartDelay(3000;
stFactory.setName("mytrigger");
stFactory.setGroup("mygroup");
TestModel result = testModelRepository.findOne((long) 1);
stFactory.setCronExpression(result.getCronTime());
return stFactory;
}
#Bean
public SchedulerFactoryBean schedulerFactoryBean() {
SchedulerFactoryBean scheduler = new SchedulerFactoryBean();
scheduler.setTriggers(cronTriggerFactoryBean().getObject());
return scheduler;
}
}
Job:
#PersistJobDataAfterExecution
#DisallowConcurrentExecution
public class MyJobTwo extends QuartzJobBean {
public static final String COUNT = "count";
private static URI jiraServerUri = URI.create("");
JiraRestClient restClient = null;
private String name;
protected void executeInternal(JobExecutionContext ctx) throws JobExecutionException {
final AsynchronousJiraRestClientFactory factory = new AsynchronousJiraRestClientFactory();
restClient = factory.createWithBasicHttpAuthentication(jiraServerUri,"", "");
}
public void setName(String name) {
this.name = name;
}
Any idea?
Just in case if someone needs solution.
SchedulerFactoryBean scheduler = new SchedulerFactoryBean();
scheduler.setOverwriteExistingJobs(true);

Quartz + Spring Batch in Spring boot

I'm trying to develop an spring application which have integration of Quartz and Spring Batch. For some reasons, i'm not able to run it properly and getting some compilation errors.
Code:
QuartzConfiguration
#Configuration
#ComponentScan("com.concretepage")
public class QuartzConfiguration {
#Bean
public MethodInvokingJobDetailFactoryBean methodInvokingJobDetailFactoryBean() {
MethodInvokingJobDetailFactoryBean obj = new MethodInvokingJobDetailFactoryBean();
obj.setTargetBeanName("jobone");
obj.setTargetMethod("myTask");
return obj;
}
#Bean
public CronTriggerFactoryBean cronTriggerFactoryBean(){
CronTriggerFactoryBean stFactory = new CronTriggerFactoryBean();
SpringBatchJobs batch = new SpringBatchJobs();
stFactory.setJobDetail(batch.job()); // here i'm getting some compilcation error like "The method job() from the type SpringBatchJobs refers to the missing type Job"
stFactory.setStartDelay(3000);
stFactory.setName("mytrigger");
stFactory.setGroup("mygroup");
stFactory.setCronExpression("0/1 * * * * ?");
return stFactory;
}
#Bean
public SchedulerFactoryBean schedulerFactoryBean() {
SchedulerFactoryBean scheduler = new SchedulerFactoryBean();
scheduler.setTriggers(cronTriggerFactoryBean().getObject());
return scheduler;
}
}
Spring Batch:
#EnableBatchProcessing
public class SpringBatchJobs {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
#Bean
protected Tasklet tasklet() {
return new Tasklet() {
#Override
public RepeatStatus execute(StepContribution contribution,
ChunkContext context) {
return RepeatStatus.FINISHED;
}
};
}
#Bean
public Job job() throws Exception {
return this.jobs.get("job").start(step1()).build();
}
#Bean
protected Step step1() throws Exception {
return this.steps.get("step1").tasklet(tasklet()).build();
}
}
The problem come on the following line
stFactory.setJobDetail(batch.job()); // here i'm getting some compilcation error like "The method job() from the type SpringBatchJobs refers to the missing type Job"
How can i run Spring Batch jobs using Quartz2 in spring boot? Any idea
Spring batch Job need to be launched by using spring batch job launcher.
Please refer below link for exact details on how to do it.
https://examples.javacodegeeks.com/enterprise-java/spring/batch/quartz-spring-batch-example/

Batch with Spring Boot & JPA - use in-memory datasource for batch-related tables

Context
I'm trying to develop a batch service with Spring Boot, using JPA Repository. Using two different datasources, I want the batch-related tables created in a in-memory database, so that it does not pollute my business database.
Following multiple topics on the web, I came up with this configuration of my two datasources :
#Configuration
public class DataSourceConfiguration {
#Bean(name = "mainDataSource")
#Primary
#ConfigurationProperties(prefix="spring.datasource")
public DataSource mainDataSource(){
return DataSourceBuilder.create().build();
}
#Bean(name = "batchDataSource")
public DataSource batchDataSource( #Value("${batch.datasource.url}") String url ){
return DataSourceBuilder.create().url( url ).build();
}
}
The first one, mainDataSource, uses the default Spring database configuration. The batchDataSource defines an embedded HSQL database, in which I want the batch and step tables to be created.
# DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url=jdbc:mariadb://localhost:3306/batch_poc
spring.datasource.username=root
spring.datasource.password=
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.datasource.max-age=10000
spring.datasource.initialize=false
# JPA (JpaBaseConfiguration, HibernateJpaAutoConfiguration)
spring.jpa.generate-ddl=false
spring.jpa.show-sql=true
spring.jpa.database=MYSQL
# SPRING BATCH (BatchDatabaseInitializer)
spring.batch.initializer.enabled=false
# ----------------------------------------
# PROJECT SPECIFIC PROPERTIES
# ----------------------------------------
# BATCH DATASOURCE
batch.datasource.url=jdbc:hsqldb:file:C:/tmp/hsqldb/batchdb
Here is my batch config :
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
private static final Logger LOG = Logger.getLogger( BatchConfiguration.class );
#Bean
public BatchConfigurer configurer(){
return new CustomBatchConfigurer();
}
#Bean
public Job importElementsJob( JobBuilderFactory jobs, Step step1 ){
return jobs.get("importElementsJob")
.incrementer( new RunIdIncrementer() )
.flow( step1 )
.end()
.build();
}
#Bean
public Step step1( StepBuilderFactory stepBuilderFactory, ItemReader<InputElement> reader,
ItemWriter<List<Entity>> writer, ItemProcessor<InputElement, List<Entity>> processor ){
return stepBuilderFactory.get("step1")
.<InputElement, List<Entity>> chunk(100)
.reader( reader )
.processor( processor )
.writer( writer )
.build();
}
#Bean
public ItemReader<InputElement> reader() throws IOException {
return new CustomItemReader();
}
#Bean
public ItemProcessor<InputElement, List<Entity>> processor(){
return new CutsomItemProcessor();
}
#Bean
public ItemWriter<List<Entity>> writer(){
return new CustomItemWriter();
}
}
The BatchConfigurer, using the in-memory database :
public class CustomBatchConfigurer extends DefaultBatchConfigurer {
#Override
#Autowired
public void setDataSource( #Qualifier("batchDataSource") DataSource dataSource) {
super.setDataSource(dataSource);
}
}
And, finally, my writer :
public class CustomItemWriter implements ItemWriter<List<Entity>> {
private static final Logger LOG = Logger.getLogger( EntityWriter.class );
#Autowired
private EntityRepository entityRepository;
#Override
public void write(List<? extends List<Entity>> items)
throws Exception {
if( items != null && !items.isEmpty() ){
for( List<Entity> entities : items ){
for( Entity entity : entities ){
Entity fromDb = entityRepository.findById( entity.getId() );
// Insert
if( fromDb == null ){
entityRepository.save( entity );
}
// Update
else {
// TODO : entityManager.merge()
}
}
}
}
}
}
The EntityRepository interface extends JpaRepository.
Problem
When I separate the datasources this way, nothing happens when I call the save method of the repository. I see the select queries from the call of findById() in the logs. But nothing for the save. And my output database is empty at the end.
When I come back to a unique datasource configuration (removing the configurer bean and letting Spring Boot manage the datasource alone), the insert queries work fine.
Maybe the main datasource configuration is not good enough for JPA to perform the inserts correctly. But what is missing ?
I finally solved the problem implementing my own BatchConfigurer, on the base of the Spring class BasicBatchConfigurer, and forcing the use of Map based jobRepository and jobExplorer. No more custom datasource configuration, only one datasource which I let Spring Boot manage : it's easier that way.
My custom BatchConfigurer :
public class CustomBatchConfigurer implements BatchConfigurer {
private static final Logger LOG = Logger.getLogger( CustomBatchConfigurer.class );
private final EntityManagerFactory entityManagerFactory;
private PlatformTransactionManager transactionManager;
private JobRepository jobRepository;
private JobLauncher jobLauncher;
private JobExplorer jobExplorer;
/**
* Create a new {#link CustomBatchConfigurer} instance.
* #param entityManagerFactory the entity manager factory
*/
public CustomBatchConfigurer( EntityManagerFactory entityManagerFactory ) {
this.entityManagerFactory = entityManagerFactory;
}
#Override
public JobRepository getJobRepository() {
return this.jobRepository;
}
#Override
public PlatformTransactionManager getTransactionManager() {
return this.transactionManager;
}
#Override
public JobLauncher getJobLauncher() {
return this.jobLauncher;
}
#Override
public JobExplorer getJobExplorer() throws Exception {
return this.jobExplorer;
}
#PostConstruct
public void initialize() {
try {
// transactionManager:
LOG.info("Forcing the use of a JPA transactionManager");
if( this.entityManagerFactory == null ){
throw new Exception("Unable to initialize batch configurer : entityManagerFactory must not be null");
}
this.transactionManager = new JpaTransactionManager( this.entityManagerFactory );
// jobRepository:
LOG.info("Forcing the use of a Map based JobRepository");
MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean( this.transactionManager );
jobRepositoryFactory.afterPropertiesSet();
this.jobRepository = jobRepositoryFactory.getObject();
// jobLauncher:
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(getJobRepository());
jobLauncher.afterPropertiesSet();
this.jobLauncher = jobLauncher;
// jobExplorer:
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
jobExplorerFactory.afterPropertiesSet();
this.jobExplorer = jobExplorerFactory.getObject();
}
catch (Exception ex) {
throw new IllegalStateException("Unable to initialize Spring Batch", ex);
}
}
}
My configuration class looks like this now :
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Bean
public BatchConfigurer configurer( EntityManagerFactory entityManagerFactory ){
return new CustomBatchConfigurer( entityManagerFactory );
}
[...]
}
And my properties files :
# DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url=jdbc:mariadb://localhost:3306/inotr_poc
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.datasource.max-age=10000
spring.datasource.initialize=true
# JPA (JpaBaseConfiguration, HibernateJpaAutoConfiguration)
spring.jpa.generate-ddl=false
spring.jpa.show-sql=true
spring.jpa.database=MYSQL
# SPRING BATCH (BatchDatabaseInitializer)
spring.batch.initializer.enabled=false
Thanks for the above posts! I have been struggling for the past couple of days to get my Spring Boot with Batch application working with an in-memory Map based job repository and a resourceless transaction manager. ( I cannot let Spring Batch to use my application datasource for the Batch meta data tables as I don't have the DDL access to create the BATCH_ tables there )
Finally arrived at the below configuration after looking at the above posts and it worked perfectly!!
public class CustomBatchConfigurer implements BatchConfigurer {
private static final Logger LOG = LoggerFactory.getLogger(CustomBatchConfigurer.class);
// private final EntityManagerFactory entityManagerFactory;
private PlatformTransactionManager transactionManager;
private JobRepository jobRepository;
private JobLauncher jobLauncher;
private JobExplorer jobExplorer;
/**
* Create a new {#link CustomBatchConfigurer} instance.
* #param entityManagerFactory the entity manager factory
public CustomBatchConfigurer( EntityManagerFactory entityManagerFactory ) {
this.entityManagerFactory = entityManagerFactory;
}
*/
#Override
public JobRepository getJobRepository() {
return this.jobRepository;
}
#Override
public PlatformTransactionManager getTransactionManager() {
return this.transactionManager;
}
#Override
public JobLauncher getJobLauncher() {
return this.jobLauncher;
}
#Override
public JobExplorer getJobExplorer() throws Exception {
return this.jobExplorer;
}
#PostConstruct
public void initialize() {
try {
// transactionManager:
LOG.info("Forcing the use of a Resourceless transactionManager");
this.transactionManager = new ResourcelessTransactionManager();
// jobRepository:
LOG.info("Forcing the use of a Map based JobRepository");
MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean( this.transactionManager );
jobRepositoryFactory.afterPropertiesSet();
this.jobRepository = jobRepositoryFactory.getObject();
// jobLauncher:
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(getJobRepository());
jobLauncher.afterPropertiesSet();
this.jobLauncher = jobLauncher;
// jobExplorer:
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
jobExplorerFactory.afterPropertiesSet();
this.jobExplorer = jobExplorerFactory.getObject();
}
catch (Exception ex) {
throw new IllegalStateException("Unable to initialize Spring Batch", ex);
}
}
}
And below is the bean i added in my job configuration class
#Bean
public BatchConfigurer configurer(){
return new CustomBatchConfigurer();
}
Eria's answer worked! However, I had modified it to use:
org.springframework.batch.support.transaction.ResourcelessTransactionManager
From CustomBatchConfigurer:
#PostConstruct
public void initialize() {
try {
// transactionManager:
LOGGER.info("Forcing the use of ResourcelessTransactionManager for batch db");
this.transactionManager = new ResourcelessTransactionManager();
//the rest of the code follows...
}

Resources