Separate Configuration File for Same Job executed under separate scenarios - spring-boot

I am using Spring Batch with Spring boot.
I have two jobs which I execute using the below:
private static final Logger LOGGER = LoggerFactory.getLogger(TimeWorkedJobLauncher.class);
private final Job job;
//private final Job jobDS;
private final JobLauncher jobLauncher;
#Autowired
private ResourceLoader resourceLoader;
#Autowired
TimeWorkedJobLauncher(#Qualifier("timeworkedJob") Job job, JobLauncher jobLauncher) {
System.out.println("Inside TimeWorkedJobLauncher");
this.job = job;
this.jobLauncher = jobLauncher;
//this.jobDS = jobDS;
}
#Async
#Scheduled(cron = "${reports.timeWorked.job.cron}")
public void execute() throws JobParametersInvalidException, JobExecutionAlreadyRunningException,
JobRestartException, JobInstanceAlreadyCompleteException {
LOGGER.info("Starting TimeWorked job");
JobParametersBuilder builder = new JobParametersBuilder();
builder.addDate("date", new Date());
builder.addLong("time",System.currentTimeMillis());
builder.addString(LoaderConstants.JOB_PARAMETER, resourceLoader.getTimeWorkedFileLoc());
jobLauncher.run(job, builder.toJobParameters());
LOGGER.info("Stopping TimeWorked job");
}
#Async
#Scheduled(cron = "${reports.timeWorked.job.cronds}")
public void executeDSJob() throws JobParametersInvalidException, JobExecutionAlreadyRunningException,
JobRestartException, JobInstanceAlreadyCompleteException {
JobParametersBuilder builder = new JobParametersBuilder();
builder.addDate("date", new Date());
builder.addLong("time",System.currentTimeMillis());
builder.addString(LoaderConstants.JOB_PARAMETER, resourceLoader.getTimeWorkedFileLocForDS());
jobLauncher.run(job, builder.toJobParameters());
LOGGER.info("Stopping TimeWorked job for DS");
}
The only difference is that when I run the executeDSJob() I need the job to execute different beforeStep() and afterStep() methods as the ones executed when I run the job using execute().
I have two configuration classes configured, the only difference being that one class returns
#Bean
public StepExecutionListener timeworkedCountProcessor() {
return new TimeWorkedProcessorDS();
}
and the other returns
#Bean
public StepExecutionListener timeworkedCountProcessor() {
return new TimeWorkedProcessor();
}
But when I run the execute() method the TimeWorkedProcessorDS() is used and the afterStep() and beforeStep() of this class are being used.
How can I do this?

Related

Spring Batch - How to Automatically Restart a Scheduled Job with Multi-Threaded Steps if Failed?

I'm new to Spring Batch. I have a scheduled job which needs to run every 2 hours. This job has several multi-threaded steps which should run independent from each other. The job is currently launched using a JobLauncher, as mentioned below.
#Component
#EnableScheduling
public class JobScheduler {
private static final Logger logger = LoggerFactory.getLogger(JobScheduler.class);
#Autowired
private JobLauncher jobLauncher;
#Autowired
private Job job;
#Scheduled(cron = "0 0 */2 * * ?")
#Retryable(maxAttempts = 3, backoff = #Backoff(delay = 60000),
include = {SQLException.class, RuntimeException.class})
public void automatedTask() {
JobParameters jobParameters = new JobParametersBuilder().addLong("time", System.currentTimeMillis()).toJobParameters();
try {
JobExecution jobExecution = jobLauncher.run(job, jobParameters);
} catch (JobInstanceAlreadyCompleteException | JobRestartException | JobParametersInvalidException |
JobExecutionAlreadyRunningException ex) {
logger.error("Error occurred when executing job scheduler", ex);
}
}
}
Mentioned below is my BatchConfig class.
#Configuration
#EnableBatchProcessing
#EnableRetry
public class BatchConfig {
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private DataSource dataSource;
#Bean
#StepScope
public JdbcPagingItemReader<Model> reader1() {
StringBuffer selectClause = new StringBuffer();
selectClause.append("SELECT ");
selectClause.append("* ");
StringBuffer fromClause = new StringBuffer();
fromClause.append("FROM ");
fromClause.append("TABLENAME");
OraclePagingQueryProvider oraclePagingQueryProvider = new OraclePagingQueryProvider();
oraclePagingQueryProvider.setSelectClause(selectClause.toString());
oraclePagingQueryProvider.setFromClause(fromClause.toString());
Map<String, Order> orderByKeys = new HashMap<>();
orderByKeys.put("id", Order.ASCENDING);
oraclePagingQueryProvider.setSortKeys(orderByKeys);
JdbcPagingItemReader<Model> jdbcPagingItemReader = new JdbcPagingItemReader<>();
jdbcPagingItemReader.setSaveState(false);
jdbcPagingItemReader.setDataSource(dataSource);
jdbcPagingItemReader.setQueryProvider(oraclePagingQueryProvider);
jdbcPagingItemReader.setRowMapper(BeanPropertyRowMapper.newInstance(Model.class));
return jdbcPagingItemReader;
}
#Bean
#StepScope
public JdbcPagingItemReader<Model> reader2() {
}
#Bean
#StepScope
public JdbcPagingItemReader<Model> reader3() {
}
#Bean
#StepScope
public ItemWriter<Model> writer1() {
return new CustomItemWriter1();
}
#Bean
#StepScope
public ItemWriter<Model> writer2() {
return new CustomItemWriter2();
}
#Bean
#StepScope
public ItemWriter<Model> writer3() {
return new CustomItemWriter3();
}
#Bean
public Step step1() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(4);
taskExecutor.setMaxPoolSize(4);
taskExecutor.afterPropertiesSet();
return stepBuilderFactory.get("step1")
.<Model, Model>chunk(1000)
.reader(reader1())
.writer(writer1())
.faultTolerant()
.skipPolicy(new AlwaysSkipItemSkipPolicy())
.skip(Exception.class)
.listener(new CustomSkipListener())
.taskExecutor(taskExecutor)
.build();
}
#Bean
public Step step2() {
}
#Bean
public Step step3() {
}
#Bean
public Job myJob() {
return jobBuilderFactory.get("myJob").incrementer(new RunIdIncrementer())
// .listener(new CustomJobExecutionListener())
.start(step1()).on("*").to(step2())
.from(step1()).on(ExitStatus.FAILED.getExitCode()).to(step2())
.from(step2()).on("*").to(step3())
.from(step2()).on(ExitStatus.FAILED.getExitCode()).to(step3())
.end().build();
}
}
I've added conditional flow to the job so that every next step should work regardless of a failure in the previous step. Everything works fine in the initial steps but if an exception is thrown in the last step, the Exit Status of the whole job becomes FAILED. To solve this AND to solve any other failures in the job, I tried to implement restart functionality. Please note that I'm not saving the state in the readers due to multi-threading and I'm not sure whether this could affect the restarting.
I have referred the accepted solution in the below question,
https://stackoverflow.com/questions/38846457/how-can-you-restart-a-failed-spring-batch-job-and-let-it-pick-up-where-it-left-o
but I don't quite understand how or where to call the jobOperator.restart method at.
I've tried it like below, expecting the job to restart after launching, if failed. But it didn't work at all. Also, this implementation would stop the functionality of #Retryable annotation due to the try-catch block with Exception class caught.
#Component
#EnableScheduling
public class JobScheduler {
private static final Logger logger = LoggerFactory.getLogger(JobScheduler.class);
#Autowired
private JobLauncher jobLauncher;
#Autowired
private Job job;
#Autowired
private JobRepository jobRepository;
#Autowired
private JobRegistry jobRegistry;
#Autowired
private DataSource dataSource;
#Scheduled(cron = "0 0 */2 * * ?")
#Retryable(maxAttempts = 3, backoff = #Backoff(delay = 60000),
include = {SQLException.class, RuntimeException.class})
public void automatedTask() {
JobParameters jobParameters = new JobParametersBuilder().addLong("time", System.currentTimeMillis()).toJobParameters();
try {
JobExecution jobExecution = jobLauncher.run(job, jobParameters);
JobExplorer jobExplorer = this.getJobExplorer(dataSource);
JobOperator jobOperator = this.getJobOperator(jobLauncher, jobRepository, jobRegistry, jobExplorer);
List<JobInstance> jobInstances = jobExplorer.getJobInstances("myJob",0,1);
if(!jobInstances.isEmpty()){
JobInstance jobInstance = jobInstances.get(0);
List<JobExecution> jobExecutions = jobExplorer.getJobExecutions(jobInstance);
if(!jobExecutions.isEmpty()){
for(JobExecution execution: jobExecutions){
if(execution.getStatus().equals(BatchStatus.FAILED)){
jobOperator.restart(execution.getId());
}
}
}
}
} catch (Exception ex) {
logger.error("Error occurred when executing job scheduler", ex);
}
}
#Bean
public JobOperator getJobOperator(final JobLauncher jobLauncher, final JobRepository jobRepository,
final JobRegistry jobRegistry, final JobExplorer jobExplorer) {
final SimpleJobOperator jobOperator = new SimpleJobOperator();
jobOperator.setJobLauncher(jobLauncher);
jobOperator.setJobRepository(jobRepository);
jobOperator.setJobRegistry(jobRegistry);
jobOperator.setJobExplorer(jobExplorer);
return jobOperator;
}
#Bean
public JobExplorer getJobExplorer(final DataSource dataSource) throws Exception {
final JobExplorerFactoryBean bean = new JobExplorerFactoryBean();
bean.setDataSource(dataSource);
bean.setTablePrefix("BATCH_");
bean.setJdbcOperations(new JdbcTemplate(dataSource));
bean.afterPropertiesSet();
return bean.getObject();
}
}
I then tried adding a custom JobExecutionListener like below, expecting it to restart it after running the job, if failed. But it just fails as all the Autowired beans are becoming NULL.
public class CustomJobExecutionListener {
private static final Logger logger = LoggerFactory.getLogger(CustomJobExecutionListener.class);
#Autowired
private JobLauncher jobLauncher;
#Autowired
private JobRepository jobRepository;
#Autowired
private JobRegistry jobRegistry;
#Autowired
private DataSource dataSource;
#BeforeJob
public void beforeJob(JobExecution jobExecution) {
}
#AfterJob
public void afterJob(JobExecution jobExecution) {
try {
JobExplorer jobExplorer = this.getJobExplorer(dataSource);
JobOperator jobOperator = this.getJobOperator(jobLauncher, jobRepository, jobRegistry, jobExplorer);
if(jobExecution.getStatus().equals(BatchStatus.FAILED)){
jobOperator.restart(jobExecution.getId());
}
} catch (Exception ex) {
logger.error("Unknown error occurred when executing after job execution listener", ex);
}
}
#Bean
public JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor(JobRegistry jobRegistry) {
final JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor = new JobRegistryBeanPostProcessor();
jobRegistryBeanPostProcessor.setJobRegistry(jobRegistry);
return jobRegistryBeanPostProcessor;
}
#Bean
public JobOperator getJobOperator(final JobLauncher jobLauncher, final JobRepository jobRepository,
final JobRegistry jobRegistry, final JobExplorer jobExplorer) {
final SimpleJobOperator jobOperator = new SimpleJobOperator();
jobOperator.setJobLauncher(jobLauncher);
jobOperator.setJobRepository(jobRepository);
jobOperator.setJobRegistry(jobRegistry);
jobOperator.setJobExplorer(jobExplorer);
return jobOperator;
}
#Bean
public JobExplorer getJobExplorer(final DataSource dataSource) throws Exception {
final JobExplorerFactoryBean bean = new JobExplorerFactoryBean();
bean.setDataSource(dataSource);
bean.setTablePrefix("BATCH_");
bean.setJdbcOperations(new JdbcTemplate(dataSource));
bean.afterPropertiesSet();
return bean.getObject();
}
}
What am I doing wrong? How should the restart functionality be implemented for this job?
Appreciate your kind help!
Please note that I'm not saving the state in the readers due to multi-threading and I'm not sure whether this could affect the restarting.
It certainly affects restartability. Multi-threading in steps is incompatible with restartability. From the javadoc of the JdbcPagingItemReader that you are using, you can read the following:
The implementation is thread-safe in between calls to open(ExecutionContext),
but remember to use saveState=false if used in a multi-threaded client
(no restart available).
Without restart data, Spring Batch cannot restart the step from where it left-off. This is a trade-off that you have accepted by using a multi-threaded step.
but I don't quite understand how or where to call the jobOperator.restart method at.
Now with regard to restarting the failed job, a few notes:
Trying to restart a job in a JobExecutionListener is incorrect. This listener is called in the scope of the current job execution, while a restart will have its own, distinct job execution
JobOperator#restart should not be called inside the scheduled method, otherwise it will be called for every scheduled run. You can find an example here: https://stackoverflow.com/a/55137314/5019386

JobParameters cannot be found in spring batch

I need your help plizz!!
i have a spring batch app and i need to pass job parameters to my quartz job.
Here is my JobLauncher in which i want to pass each line of list as job parameters :
#Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
try {
JdbcTemplate Jd = new JdbcTemplate(ds);
List<QuartzParameters> list=null;
list=Jd.query("select * from FLUX_INFO",new QuartzParametersMapper());
for (QuartzParameters i : list) {
Job job = jobLocator.getJob(jobName);
JobParametersBuilder JP = new JobParametersBuilder();
JP.addString("PAYS", i.getPAYS());
JP.addString("CRON", i.getCRON());
JP.addString("CLASSAPP",i.getCLASSAPP());
JobParameters paramJobParameters=JP.toJobParameters();
JobExecution jobExecution = jobLauncher.run(job, paramJobParameters);
log.info("{}_{} was completed successfully", job.getName(), jobExecution.getId());
} } catch (Exception e) {
log.error("Encountered job execution exception!");
}}}
And here is my batch config class :
#Value("#{jobParameters[CLASSAPP]}")
private String ClassApp;
#Scope("step")
#Bean
public JdbcCursorItemReader<FichierEclate> readerDB(){
JdbcCursorItemReader<FichierEclate> reader = new JdbcCursorItemReader<FichierEclate>();
reader.setDataSource(ds);
reader.setSql(query(ClassApp));
reader.setRowMapper(new FichierEclateRowMapper());
return reader;
}
i am also using another parameter in Quartz config :
#Value("#{jobParameters[CRON]}")
private String CRON_EXPRESSION;
#StepScope
#Bean
public JobDetailFactoryBean jobDetailFactoryBean() {
JobDetailFactoryBean factory = new JobDetailFactoryBean();
factory.setJobClass(QuartzJobLauncher.class);
Map map = new HashMap();
map.put("jobName", "JobFinal");
map.put("jobLauncher", jobLauncher);
map.put("jobLocator", jobLocator);
map.put("CRON_EXPRESSION", CRON_EXPRESSION );
factory.setJobDataAsMap(map);
factory.setGroup("etl_group");
factory.setName("etl_job");
return factory;}
I am also using the job parameter in the processor class :
#Value("#{jobParameters[CLASSAPP]}")
private String ClassApp;
#Scope("step")
#Override
public TdfFile process(FichierEclate item) throws Exception {
//some code and use the **CLASSAPP** variable }
i tried to use #StepScope But same problem !! i get this :
Error creating bean with name 'batchConfig': Unsatisfied dependency expressed through field 'ClassApp'; nested exception is org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.expression.spel.SpelEvaluationException: EL1008E: Property or field 'jobParameters' cannot be found on object of type 'org.springframework.beans.factory.config.BeanExpressionContext' - maybe not public or not valid?
Here is QuartzConfiguration class :
#Configuration
public class QuartzConfiguration {
#Value("#{jobParameters[CRON]}")
private String CRON_EXPRESSION;
#Autowired
private JobLauncher jobLauncher;
#Autowired
private JobLocator jobLocator;
public QuartzConfiguration() {
super();
}
#Bean
public JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor(JobRegistry jobRegistry) {
JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor = new JobRegistryBeanPostProcessor();
jobRegistryBeanPostProcessor.setJobRegistry(jobRegistry);
return jobRegistryBeanPostProcessor;
}
#Bean
#Scope(value="step", proxyMode=ScopedProxyMode.TARGET_CLASS)
public JobDetailFactoryBean jobDetailFactoryBean() {
JobDetailFactoryBean factory = new JobDetailFactoryBean();
factory.setJobClass(QuartzJobLauncher.class);
Map map = new HashMap();
map.put("jobName", "JobFinal");
map.put("jobLauncher", jobLauncher);
map.put("jobLocator", jobLocator);
map.put("CRON_EXPRESSION", CRON_EXPRESSION );
factory.setJobDataAsMap(map);
factory.setGroup("etl_group");
factory.setName("etl_job");
return factory;
}
// Job is scheduled after every 3 minutes
#Bean
public CronTriggerFactoryBean cronTriggerFactoryBean() {
CronTriggerFactoryBean stFactory = new CronTriggerFactoryBean();
stFactory.setJobDetail(jobDetailFactoryBean().getObject());
stFactory.setStartDelay(3000);
stFactory.setName("cron_trigger");
stFactory.setGroup("cron_group");
stFactory.setCronExpression(CRON_EXPRESSION);
// stFactory.getJobDataMap().get(app);
return stFactory;
}
#Bean
public SchedulerFactoryBean schedulerFactoryBean() {
SchedulerFactoryBean scheduler = new SchedulerFactoryBean();
scheduler.setTriggers(cronTriggerFactoryBean().getObject());
return scheduler;
}}
And here is the jobLauncher class :
public class QuartzJobLauncher extends QuartzJobBean {
private static final Logger log = LoggerFactory.getLogger(QuartzJobLauncher.class);
#Autowired
private DataSource ds;
private String jobName;
private JobLauncher jobLauncher;
private JobLocator jobLocator;
public String getJobName() {
return jobName;
}
public void setJobName(String jobName) {
this.jobName = jobName;
}
public JobLauncher getJobLauncher() {
return jobLauncher;
}
public void setJobLauncher(JobLauncher jobLauncher) {
this.jobLauncher = jobLauncher;
}
public JobLocator getJobLocator() {
return jobLocator;
}
public void setJobLocator(JobLocator jobLocator) {
this.jobLocator = jobLocator;
}
#Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
try {
JdbcTemplate Jd = new JdbcTemplate(ds);
List<QuartzParameters> list=null;
list=Jd.query("select * from FLUX_INFO",new QuartzParametersMapper());
for (QuartzParameters i : list) {
Job job = jobLocator.getJob(jobName);
JobParametersBuilder JP = new JobParametersBuilder();
JP.addString("PAYS", i.getPAYS());
JP.addString("CRON", i.getCRON());
JP.addString("CLASSAPP",i.getCLASSAPP());
JobParameters paramJobParameters=JP.toJobParameters();
final JobExecution jobExecution = jobLauncher.run(job, paramJobParameters);
log.info("{}_{} was completed successfully", job.getName(), jobExecution.getId());
}} catch (Exception e) {
log.error("Encountered job execution exception!");
}}}
I tried to add #Scope(value="step", proxyMode=ScopedProxyMode.TARGET_CLASS) on classes directly or on Beans !! Both ways did not work !
Did anyone try a solution to solve this pliz ?!
Since you are injecting the job parameter in a field of your class, then #Scope("step") should be placed on the class itself, something like:
#Component
#Scope("step")
public class MyProcessor implements ItemProcessor<FichierEclate, TdfFile> {
#Value("#{jobParameters[CLASSAPP]}")
private String ClassApp;
#Override
public TdfFile process(FichierEclate item) throws Exception {
//some code and use the **CLASSAPP** variable
}
}
The same thing should be done for other classes where you want to inject jobParameters[CRON] and jobParameters[CLASSAPP].

How to restart the Spring Batch Job automatically when abnormal exceptions are arises?

Before raising the question I went through many links like : How can you restart a failed spring batch job and let it pick up where it left off? and Spring Batch restart uncompleted jobs from the same execution and step and https://learning.oreilly.com/library/view/the-definitive-guide/9781484237243/html/215885_2_En_6_Chapter.xhtml, but this doesn't solved my query yet.
I am using Spring Boot Batch application, in my project I've 3 Jobs which runs sequentially on scheduled basis daily 2 PM in the night wrapped up in a single method and each jobs has 5 steps which performs chunk-based processing does not use tasklet.
I often see an issues like network fluctuations, database is down and abnormal issues spring batch is sopping a while job and getting a lot of issues of data loss since there is no way to automatically restart from where if failed.
I want to developed ability to automatically restart the batch jobs when any type of abnormal exceptions arises. Is there any way if we can do that ?
I've configured batch jobs like below.
MyApplication.java
#SpringBootApplication
#EnableBatchProcessing
#EnableScheduling
public class MyApplication {
public static void main(String[] args) {
SpringApplication.run(MyApplication.class, args);
}
}
MyJob.java
#Configuration
public class MyJob {
#Value("${skip.limit}")
private Integer skipLimit;
#Value("${chunk.size}")
private Integer chunkSize;
#Bean(name="myJobCache")
public CacheManager myJobCache() {
return new ConcurrentMapCacheManager();
}
#Bean("customerJob")
public Job customerJob(JobBuilderFactory jobBuilderFactory,
StepBuilderFactory stepBuilderFactory,
JdbcCursorItemReader<Customer> customerReader,
JdbcCursorItemReader<Department> departmentReader,
JdbcCursorItemReader<Stock> stockItemReader,
JdbcCursorItemReader<Advisory> advisoryItemReder) throws Exception {
return jobBuilderFactory.get("customerJob")
.incrementer(new RunIdIncrementer())
.start(customerStep(stepBuilderFactory, customerReader))
.next(departmentStep(stepBuilderFactory, departmentReader))
.next(stackStep(stepBuilderFactory))
.......
.......
.......
.listener(customerListener())
.build();
}
#Bean
public Step customerStep(StepBuilderFactory stepBuilderFactory,
JdbcCursorItemReader<Customer> customerReader) {
return stepBuilderFactory.get("customerStep").
<Customer, NewCustomer>chunk(chunkSize)
.reader(customerReader)
.processor(customerProcessor())
.writer(customerWriter())
.faultTolerant()
.skip(Exception.class)
.skipLimit(skipLimit)
.listener(customerSkipListener())
.listener(customerStepListener())
.build();
}
#Bean
public CustomerProcessor customerProcessor() {
return new CustomerProcessor(myJobCache());
}
#Bean
public CustomerWriter customerWriter() {
return new CustomerWriter();
}
// Like this all other jobs are configured
}
MyScheduler.java
public class MyScheduler {
#Autowired
private JobLauncher customerJobLauncher;
#Autowired
private JobLauncher abcJobLauncher;
#Autowired
private JobLauncher xyzJobLauncher;
#Autowired
#Qualifier(value = "customerJob")
private Job customerJob;
#Autowired
#Qualifier(value = "abcJob")
private Job abcJob;
#Autowired
#Qualifier(value = "xyzJob")
private Job xyzJob;
#Scheduled(cron = "0 0 */1 * * *") // run at every hour for testing
public void handle() {
JobParameters params = new JobParametersBuilder()
.addString("cust.job.id", String.valueOf(System.currentTimeMillis()))
.addDate("cust.job.date", new Date()).toJobParameters();
long diff = 0;
try {
JobExecution jobExecution = customerJobLauncher.run(customerJob, params);
Date start = jobExecution.getCreateTime();
JobParameters job2Params = new JobParametersBuilder()
.addString("abc.job.id", String.valueOf(System.currentTimeMillis()))
.addDate("abc.job.date", new Date()).toJobParameters();
JobExecution job2Execution = abcJobLauncher.run(abcJob, job2Params);
JobParameters job3Params = new JobParametersBuilder()
.addString("xyz.job.id", String.valueOf(System.currentTimeMillis()))
.addDate("xyx.job.date", new Date()).toJobParameters();
JobExecution job3Execution = xyzJobLauncher.run(xyzJob, job3Params);
Date end = job3Execution.getEndTime();
diff = end.getTime() - start.getTime();
log.info(JobExecutionTimeCalculate.getJobExecutionTime(diff));
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException
| JobParametersInvalidException e) {
log.error("Job Failed : " + e.getMessage());
}
}
}

bugs accuring during spring batch scheduling

hey guys i am working on spring batch with scheduling (using cron trigger) it is working but with bugs which are the following:
let's assumme that the cron value launches the batch each 10 seconds, when i launch the first and after, for example 3 seconds I launch another one, spring will not be aware of the gap of the 3 seconds and it will launch them both like i have triggered them in the same time
here is my code
this is the class of the job i'll launch
#Component
public class JobThread implements Runnable {
#Autowired
private JobLauncher jobLauncher;
#Autowired
#Lazy
private Job job;
public JobParameters jobParameters;
private Logger log = Logger.getLogger(JobThread.class);
public synchronized void runBatch() {
jobParameters = new JobParametersBuilder().addLong("LaunchTime", System.currentTimeMillis())
.addString("TenantID", BatchController.getCurrentTenant().get()).toJobParameters();
try {
JobExecution jobExecution = jobLauncher.run(job, jobParameters);
log.info("Job's Status:::" + jobExecution.getStatus());
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException
| JobParametersInvalidException e) {
e.printStackTrace();
}
}
#Override
public void run() {
this.runBatch();
}
}
the controller which will invoke the job
#RestController
#RequestMapping("tenant/batch")
public class BatchController {
#Autowired
private ThreadPoolTaskScheduler taskScheduler;
#Autowired
#Qualifier("threadPoolTaskExecutor")
private ThreadPoolTaskExecutor taskExecutor;
#Autowired
private JobThread jobThread;
private static ThreadLocal<String> currentTenant;
#PostMapping("/schedule")
public void setBatch(#RequestBody BatchBean cron) {
currentTenant = new ThreadLocal<String>() {
#Override
protected String initialValue() {
new TenantContext();
return TenantContext.getCurrentTenant();
}
};
//cron = "*/10 * * * * *";
taskScheduler.schedule(taskExecutor.createThread(jobThread), new CronTrigger(cron.getCron()));
}
I hope that i have been clear enough
Thanks in advance
The problem is your code it isn't thread-safe and thus potentially dangerous. Also your ThreadLocal isn't going to work as the job will execute in a different thread, and it won't have access to the ThreadLocal.
Don't recreate your ThreadLocal in your controller. Define it once and leave it like that.
Your JobThread is a singleton which keeps state (the parameters) so only the last one will remain.
Program to interfaces TaskScheduler instead of concrete implementatations
Don't create a thread as your JobThread is Runnable already.
Instead of your JobThread to be a singleton, construct a new one as needed and pass in the required parameters.
Your JobThread should look something like this.
public class JobThread implements Runnable {
private final Logger log = Logger.getLogger(JobThread.class);
private final JobLauncher jobLauncher;
private final Job job;
private final String tenant;
public JobThread(JobLauncher launcher, Job job, String tenant) {
this.jobLauncher=launcher;
this.job=job;
this.tenant=tenant;
}
#Override
public void run() {
JobParameters jobParameters = new JobParametersBuilder()
.addLong("LaunchTime", System.currentTimeMillis())
.addString("TenantID", tenant);
try {
JobExecution jobExecution = jobLauncher.run(job, jobParameters);
log.info("Job's Status:::" + jobExecution.getStatus());
} catch (JobExecutionException e) {
log.error(e.getMessage(), e);
}
}
}
Then in your controller inject the needed JobLauncer and Job. When needed construct a new JobThread and pass in the needed information.
#RestController
#RequestMapping("tenant/batch")
public class BatchController {
#Autowired
private TaskScheduler taskScheduler;
#Autowired
private JobLauncher jobLauncher;
#Autowired
#Lazy
private Job job;
#PostMapping("/schedule")
public void setBatch(#RequestBody BatchBean cron) {
//cron = "*/10 * * * * *";
String tenant = TenantContext.getCurrentTenant();
JobThread task = new JobThread(this.jobLauncher, this.job, tenant);
taskScheduler.schedule(task, new CronTrigger(cron.getCron()));
}
On a final note, the precision of System.currentTimeMillis might differ on your OS/System/Architecture. See the javadoc of said method.

spring batch stop a job after 10 seconds

I am trying to run a job in the background, allowing me to stop it by some condition or after a timeout has occurred.
I have these two chunks of code:
One:
#ContextConfiguration(classes={EmbeddedISpringBatchConfiguration.class, MonitoredJobConfig.class})
#RunWith(SpringJUnit4ClassRunner.class)
public class RunMonitoredJob2 {
#Autowired
private JobLauncher jobLauncher;
#Autowired
private JobRepository jobRepository;
#Autowired
private Job job;
#Test
public void testLaunchJob() throws Exception {
JobExecution execution = jobLauncher.run(job, new JobParameters());
Thread.sleep(10000);
execution.stop();
}
Two:
#ContextConfiguration(classes={EmbeddedISpringBatchConfiguration.class, MonitoredJobConfig.class})
#RunWith(SpringJUnit4ClassRunner.class)
public class RunMonitoredJob {
#Autowired
private JobRepository jobRepository;
#Autowired
private Job job;
#Test
public void testLaunchJob() throws Exception {
JobExecution execution = jobLauncher().run(job, new JobParameters());
Thread.sleep(10000);
execution.stop();
}
public JobLauncher jobLauncher() {
final SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
final SimpleAsyncTaskExecutor simpleAsyncTaskExecutor = new SimpleAsyncTaskExecutor();
jobLauncher.setTaskExecutor(simpleAsyncTaskExecutor);
return jobLauncher;
}
In option one, The job first finishes and only then goes to the "sleep"
In option tow, the job does not run the steps in my job.
Please advise.
Try using scheduler, which will invoke after specific time interval, instead of Thread.sleep
#Scheduled(fixedDelayString = "${cron.batch-timeout-check-delay}", initialDelayString = "${cron.batch-timeout-check-initial-delay}")
public void execute() throws Exception{
//get jobExceutionId of job to stop
JobExceution jobExecution = jobExplorer.getJobExceution(jobExceutionId);
boolean isTimeout = (new Date().getTime() - jobExecution.getCreateTime().getTime()) > 50000 //Here timout
if(isTimeout){
jobexceution.stop();
}
}

Resources