how to get Spring Batch job instance id from execute method in TASKLET - spring

I am using a layout using Spring Batch 3.0 version.
Create a Job and execute the placement by executing the JobLauncher run method of the TASKLET.
I want to know more accurately whether the Job is executed or not through insert logic in the query in TASKLET with the corresponding JobId and other tables other than the metatables.
public class SampleScheduler {
protected final Logger log = LoggerFactory.getLogger(this.getClass());
#Autowired
private JobLauncher jobLauncher;
#Autowired
private Job sampleJob;
public void run() {
try {
String dateParam = new Date().toString();
JobParameters param = new JobParametersBuilder().addString("date",dateParam).toJobParameters();
JobExecution execution = jobLauncher.run(sampleJob, param);
log.debug("###################################################################");
log.debug("Exit Status : " + execution.getStatus());
log.debug("###################################################################");
} catch (Exception e) {
// e.printStackTrace();
log.error(e.toString());
}
}
}
Code for calling tasklet -
public class SampleTasklet implements Tasklet{
#Autowired
private SampleService sampleService;
#Override
public RepeatStatus execute(StepContribution contribution,
ChunkContext chunkContext) throws Exception {
sampleService.query();
return RepeatStatus.FINISHED;
}
}
This is my tasklet code.
StepContext stepContext = chunkContext.getStepContext();
StepExecution stepExecution = stepContext.getStepExecution();
JobExecution jobExecution = stepExecution.getJobExecution();
long jobInstanceId = jobExecution.getJobId();
Is it right to try this in the TASKLET code above?

how to get Spring Batch job instance id from execute method in TASKLET
The org.springframework.batch.core.step.tasklet.Tasklet#execute method gives you access to the ChunkContext which in turn allows you to get the parent StepExecution and JobExecution. You can then get the job instance id from the job execution.
Is it right to try this in the TASKLET code above?
Yes, that's the way to go.

Related

How to restart the Spring Batch Job automatically when abnormal exceptions are arises?

Before raising the question I went through many links like : How can you restart a failed spring batch job and let it pick up where it left off? and Spring Batch restart uncompleted jobs from the same execution and step and https://learning.oreilly.com/library/view/the-definitive-guide/9781484237243/html/215885_2_En_6_Chapter.xhtml, but this doesn't solved my query yet.
I am using Spring Boot Batch application, in my project I've 3 Jobs which runs sequentially on scheduled basis daily 2 PM in the night wrapped up in a single method and each jobs has 5 steps which performs chunk-based processing does not use tasklet.
I often see an issues like network fluctuations, database is down and abnormal issues spring batch is sopping a while job and getting a lot of issues of data loss since there is no way to automatically restart from where if failed.
I want to developed ability to automatically restart the batch jobs when any type of abnormal exceptions arises. Is there any way if we can do that ?
I've configured batch jobs like below.
MyApplication.java
#SpringBootApplication
#EnableBatchProcessing
#EnableScheduling
public class MyApplication {
public static void main(String[] args) {
SpringApplication.run(MyApplication.class, args);
}
}
MyJob.java
#Configuration
public class MyJob {
#Value("${skip.limit}")
private Integer skipLimit;
#Value("${chunk.size}")
private Integer chunkSize;
#Bean(name="myJobCache")
public CacheManager myJobCache() {
return new ConcurrentMapCacheManager();
}
#Bean("customerJob")
public Job customerJob(JobBuilderFactory jobBuilderFactory,
StepBuilderFactory stepBuilderFactory,
JdbcCursorItemReader<Customer> customerReader,
JdbcCursorItemReader<Department> departmentReader,
JdbcCursorItemReader<Stock> stockItemReader,
JdbcCursorItemReader<Advisory> advisoryItemReder) throws Exception {
return jobBuilderFactory.get("customerJob")
.incrementer(new RunIdIncrementer())
.start(customerStep(stepBuilderFactory, customerReader))
.next(departmentStep(stepBuilderFactory, departmentReader))
.next(stackStep(stepBuilderFactory))
.......
.......
.......
.listener(customerListener())
.build();
}
#Bean
public Step customerStep(StepBuilderFactory stepBuilderFactory,
JdbcCursorItemReader<Customer> customerReader) {
return stepBuilderFactory.get("customerStep").
<Customer, NewCustomer>chunk(chunkSize)
.reader(customerReader)
.processor(customerProcessor())
.writer(customerWriter())
.faultTolerant()
.skip(Exception.class)
.skipLimit(skipLimit)
.listener(customerSkipListener())
.listener(customerStepListener())
.build();
}
#Bean
public CustomerProcessor customerProcessor() {
return new CustomerProcessor(myJobCache());
}
#Bean
public CustomerWriter customerWriter() {
return new CustomerWriter();
}
// Like this all other jobs are configured
}
MyScheduler.java
public class MyScheduler {
#Autowired
private JobLauncher customerJobLauncher;
#Autowired
private JobLauncher abcJobLauncher;
#Autowired
private JobLauncher xyzJobLauncher;
#Autowired
#Qualifier(value = "customerJob")
private Job customerJob;
#Autowired
#Qualifier(value = "abcJob")
private Job abcJob;
#Autowired
#Qualifier(value = "xyzJob")
private Job xyzJob;
#Scheduled(cron = "0 0 */1 * * *") // run at every hour for testing
public void handle() {
JobParameters params = new JobParametersBuilder()
.addString("cust.job.id", String.valueOf(System.currentTimeMillis()))
.addDate("cust.job.date", new Date()).toJobParameters();
long diff = 0;
try {
JobExecution jobExecution = customerJobLauncher.run(customerJob, params);
Date start = jobExecution.getCreateTime();
JobParameters job2Params = new JobParametersBuilder()
.addString("abc.job.id", String.valueOf(System.currentTimeMillis()))
.addDate("abc.job.date", new Date()).toJobParameters();
JobExecution job2Execution = abcJobLauncher.run(abcJob, job2Params);
JobParameters job3Params = new JobParametersBuilder()
.addString("xyz.job.id", String.valueOf(System.currentTimeMillis()))
.addDate("xyx.job.date", new Date()).toJobParameters();
JobExecution job3Execution = xyzJobLauncher.run(xyzJob, job3Params);
Date end = job3Execution.getEndTime();
diff = end.getTime() - start.getTime();
log.info(JobExecutionTimeCalculate.getJobExecutionTime(diff));
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException
| JobParametersInvalidException e) {
log.error("Job Failed : " + e.getMessage());
}
}
}

bugs accuring during spring batch scheduling

hey guys i am working on spring batch with scheduling (using cron trigger) it is working but with bugs which are the following:
let's assumme that the cron value launches the batch each 10 seconds, when i launch the first and after, for example 3 seconds I launch another one, spring will not be aware of the gap of the 3 seconds and it will launch them both like i have triggered them in the same time
here is my code
this is the class of the job i'll launch
#Component
public class JobThread implements Runnable {
#Autowired
private JobLauncher jobLauncher;
#Autowired
#Lazy
private Job job;
public JobParameters jobParameters;
private Logger log = Logger.getLogger(JobThread.class);
public synchronized void runBatch() {
jobParameters = new JobParametersBuilder().addLong("LaunchTime", System.currentTimeMillis())
.addString("TenantID", BatchController.getCurrentTenant().get()).toJobParameters();
try {
JobExecution jobExecution = jobLauncher.run(job, jobParameters);
log.info("Job's Status:::" + jobExecution.getStatus());
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException
| JobParametersInvalidException e) {
e.printStackTrace();
}
}
#Override
public void run() {
this.runBatch();
}
}
the controller which will invoke the job
#RestController
#RequestMapping("tenant/batch")
public class BatchController {
#Autowired
private ThreadPoolTaskScheduler taskScheduler;
#Autowired
#Qualifier("threadPoolTaskExecutor")
private ThreadPoolTaskExecutor taskExecutor;
#Autowired
private JobThread jobThread;
private static ThreadLocal<String> currentTenant;
#PostMapping("/schedule")
public void setBatch(#RequestBody BatchBean cron) {
currentTenant = new ThreadLocal<String>() {
#Override
protected String initialValue() {
new TenantContext();
return TenantContext.getCurrentTenant();
}
};
//cron = "*/10 * * * * *";
taskScheduler.schedule(taskExecutor.createThread(jobThread), new CronTrigger(cron.getCron()));
}
I hope that i have been clear enough
Thanks in advance
The problem is your code it isn't thread-safe and thus potentially dangerous. Also your ThreadLocal isn't going to work as the job will execute in a different thread, and it won't have access to the ThreadLocal.
Don't recreate your ThreadLocal in your controller. Define it once and leave it like that.
Your JobThread is a singleton which keeps state (the parameters) so only the last one will remain.
Program to interfaces TaskScheduler instead of concrete implementatations
Don't create a thread as your JobThread is Runnable already.
Instead of your JobThread to be a singleton, construct a new one as needed and pass in the required parameters.
Your JobThread should look something like this.
public class JobThread implements Runnable {
private final Logger log = Logger.getLogger(JobThread.class);
private final JobLauncher jobLauncher;
private final Job job;
private final String tenant;
public JobThread(JobLauncher launcher, Job job, String tenant) {
this.jobLauncher=launcher;
this.job=job;
this.tenant=tenant;
}
#Override
public void run() {
JobParameters jobParameters = new JobParametersBuilder()
.addLong("LaunchTime", System.currentTimeMillis())
.addString("TenantID", tenant);
try {
JobExecution jobExecution = jobLauncher.run(job, jobParameters);
log.info("Job's Status:::" + jobExecution.getStatus());
} catch (JobExecutionException e) {
log.error(e.getMessage(), e);
}
}
}
Then in your controller inject the needed JobLauncer and Job. When needed construct a new JobThread and pass in the needed information.
#RestController
#RequestMapping("tenant/batch")
public class BatchController {
#Autowired
private TaskScheduler taskScheduler;
#Autowired
private JobLauncher jobLauncher;
#Autowired
#Lazy
private Job job;
#PostMapping("/schedule")
public void setBatch(#RequestBody BatchBean cron) {
//cron = "*/10 * * * * *";
String tenant = TenantContext.getCurrentTenant();
JobThread task = new JobThread(this.jobLauncher, this.job, tenant);
taskScheduler.schedule(task, new CronTrigger(cron.getCron()));
}
On a final note, the precision of System.currentTimeMillis might differ on your OS/System/Architecture. See the javadoc of said method.

Spring Batch Job taking previous execution parameters

I am using spring cloud dataflow and have created a spring cloud task which contains a job. This job has a parameter called last_modified_date, which is optional. In the code, I have specified which date to take in case last_modified_date is null, that is, it has not been passed as a parameter. The issue is that if for one instance of the job I pass the last_modified_date but for the next one I don't, it picks up the one in the last execution rather than passing it as null and getting it from the code.
#Component
#StepScope
public class SalesforceAdvertiserLoadTasklet implements Tasklet {
#Value("#{jobParameters['last_modified_date']}")
protected Date lastModifiedDate;
private static final Logger logger =
LoggerFactory.getLogger(SalesforceAdvertiserLoadTasklet.class);
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext)
throws Exception {
if(lastModifiedDate == null) {
lastModifiedDate =
Date.from(LocalDate.now().minusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant());
}
logger.info("In Method: runSalesforceAdvertiserLoadJob launch started on last_modified_date {}",
lastModifiedDate);
logger.info("Getting advertisers from SalesForce");
try {
getAdvertisersFromSalesforceAndAddtoDb();
} catch (JsonSyntaxException | IOException | ParseException e) {
logger.error("ERROR--> {}", e.getMessage());
}
return RepeatStatus.FINISHED;
}
#Bean
public JobParametersIncrementer runIdIncrementor() {
return new RunIdIncrementer();
}
#Bean
public Job salesforceAdvertiserLoadJob() {
return jobBuilderFactory.get(SalesforceJobName.salesforceAdvertiserLoadJob.name())
.incrementer(runIdIncrementor())
.listener(batchJobsExecutionListener)
.start(stepsConfiguration.salesforceAdvertiserLoadStep()).build();
}
Is there a way I can stop the new job instance from taking parameters from the previous job instance?
I think that you didn't provide JobParametersIncrementer to your JobBuilder. Example:
Job job = jobBuilderFactory.get(jobName)
.incrementer(new RunIdIncrementer())
.start(step)
.end()
.build();

Separate Configuration File for Same Job executed under separate scenarios

I am using Spring Batch with Spring boot.
I have two jobs which I execute using the below:
private static final Logger LOGGER = LoggerFactory.getLogger(TimeWorkedJobLauncher.class);
private final Job job;
//private final Job jobDS;
private final JobLauncher jobLauncher;
#Autowired
private ResourceLoader resourceLoader;
#Autowired
TimeWorkedJobLauncher(#Qualifier("timeworkedJob") Job job, JobLauncher jobLauncher) {
System.out.println("Inside TimeWorkedJobLauncher");
this.job = job;
this.jobLauncher = jobLauncher;
//this.jobDS = jobDS;
}
#Async
#Scheduled(cron = "${reports.timeWorked.job.cron}")
public void execute() throws JobParametersInvalidException, JobExecutionAlreadyRunningException,
JobRestartException, JobInstanceAlreadyCompleteException {
LOGGER.info("Starting TimeWorked job");
JobParametersBuilder builder = new JobParametersBuilder();
builder.addDate("date", new Date());
builder.addLong("time",System.currentTimeMillis());
builder.addString(LoaderConstants.JOB_PARAMETER, resourceLoader.getTimeWorkedFileLoc());
jobLauncher.run(job, builder.toJobParameters());
LOGGER.info("Stopping TimeWorked job");
}
#Async
#Scheduled(cron = "${reports.timeWorked.job.cronds}")
public void executeDSJob() throws JobParametersInvalidException, JobExecutionAlreadyRunningException,
JobRestartException, JobInstanceAlreadyCompleteException {
JobParametersBuilder builder = new JobParametersBuilder();
builder.addDate("date", new Date());
builder.addLong("time",System.currentTimeMillis());
builder.addString(LoaderConstants.JOB_PARAMETER, resourceLoader.getTimeWorkedFileLocForDS());
jobLauncher.run(job, builder.toJobParameters());
LOGGER.info("Stopping TimeWorked job for DS");
}
The only difference is that when I run the executeDSJob() I need the job to execute different beforeStep() and afterStep() methods as the ones executed when I run the job using execute().
I have two configuration classes configured, the only difference being that one class returns
#Bean
public StepExecutionListener timeworkedCountProcessor() {
return new TimeWorkedProcessorDS();
}
and the other returns
#Bean
public StepExecutionListener timeworkedCountProcessor() {
return new TimeWorkedProcessor();
}
But when I run the execute() method the TimeWorkedProcessorDS() is used and the afterStep() and beforeStep() of this class are being used.
How can I do this?

spring batch stop a job after 10 seconds

I am trying to run a job in the background, allowing me to stop it by some condition or after a timeout has occurred.
I have these two chunks of code:
One:
#ContextConfiguration(classes={EmbeddedISpringBatchConfiguration.class, MonitoredJobConfig.class})
#RunWith(SpringJUnit4ClassRunner.class)
public class RunMonitoredJob2 {
#Autowired
private JobLauncher jobLauncher;
#Autowired
private JobRepository jobRepository;
#Autowired
private Job job;
#Test
public void testLaunchJob() throws Exception {
JobExecution execution = jobLauncher.run(job, new JobParameters());
Thread.sleep(10000);
execution.stop();
}
Two:
#ContextConfiguration(classes={EmbeddedISpringBatchConfiguration.class, MonitoredJobConfig.class})
#RunWith(SpringJUnit4ClassRunner.class)
public class RunMonitoredJob {
#Autowired
private JobRepository jobRepository;
#Autowired
private Job job;
#Test
public void testLaunchJob() throws Exception {
JobExecution execution = jobLauncher().run(job, new JobParameters());
Thread.sleep(10000);
execution.stop();
}
public JobLauncher jobLauncher() {
final SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
final SimpleAsyncTaskExecutor simpleAsyncTaskExecutor = new SimpleAsyncTaskExecutor();
jobLauncher.setTaskExecutor(simpleAsyncTaskExecutor);
return jobLauncher;
}
In option one, The job first finishes and only then goes to the "sleep"
In option tow, the job does not run the steps in my job.
Please advise.
Try using scheduler, which will invoke after specific time interval, instead of Thread.sleep
#Scheduled(fixedDelayString = "${cron.batch-timeout-check-delay}", initialDelayString = "${cron.batch-timeout-check-initial-delay}")
public void execute() throws Exception{
//get jobExceutionId of job to stop
JobExceution jobExecution = jobExplorer.getJobExceution(jobExceutionId);
boolean isTimeout = (new Date().getTime() - jobExecution.getCreateTime().getTime()) > 50000 //Here timout
if(isTimeout){
jobexceution.stop();
}
}

Resources