I have 2 jobs where i have to launch the job2 based on the the status of the job1.
is it right to make the following call:
JobExecution jobExecution1 = jobLauncher.run(job1, jobParameters1);
if(jobExecution1.getStatus()== BatchStatus.COMPLETED){
JobExecution jobExecution2 = jobLauncher.run(job2, jobParameters2);
}
i have a doubt that the initial jobexecution1 status might not be final when the condition is checked.
if anyone could explain more about yhis process it would be great
thanks in advance .
It depends on the TaskExecutor implementation that you configure in your JobLauncher:
If the task executor is synchronous, then the first job will be run until completion (either success or failure) before launching the second job. In this case, your code is correct.
If the task executor is asynchronous, then the call JobExecution jobExecution1 = jobLauncher.run(job1, jobParameters1); will return immediately and the status of jobExecution1 would be unknown at that point. This means it is incorrect to check the status of job1 right after launching it as a condition to launch job2 (job1 will be run in a separate thread and could still be running at that point).
By default, Spring Batch configures a synchronous task executor in the JobLauncher.
There are multiple ways. We are using below option by chaining jobs by decider. Below is the code snippet . Please let me know if you face any issues.
public class MyDecider implements JobExecutionDecider {
#Override
public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) {
String exitCode = new Random().nextFloat() < .70f ? "CORRECT":"INCORRECT";
return new FlowExecutionStatus(exitCode);
}
}
#Bean
public JobExecutionDecider myDecider() {
return new MyDecider();
}
#Bean
public Job myTestJob() {
return this.jobBuilderFactory.get("myTestJob")
.start(step1())
.next(step2())
.on("FAILED").to(step3())
.from(step4())
.on("*").to(myDecider())
.on("PRESENT").to(myNextStepOrJob)
.next(myDecider()).on("CORRECT").to(myNextStepOrJob())
.from(myDecider()).on("INCORRECT").to(myNextStepOrJob())
.from(decider())
.on("NOT_PRESENT").to(myNextStepOrJob())
.end()
.build();
}
Related
I developed two different batches, let's say they're named as batch1 and batch2.
In an integration test, I'm trying to prove that the execution of twice of the same batch (batch1 then batch1 for instance) isn't duplicating data saved in database in the writer.
The issue I face is that the first batch is running successfully, but the second isn't doing anything (reader, processor, writer are not called). However, the batch status is marked as COMPLETED.
Here's the code:
#Test
void testBatchWorksIfJobIsRanTwice()
throws JobInstanceAlreadyCompleteException, JobExecutionAlreadyRunningException, JobParametersInvalidException, JobRestartException {
// given
createData();
// when
JobExecution firstJobExecution = runJobAndGetExecution("batch1_id1", jobRepository, job);
assertEquals(BatchStatus.COMPLETED, firstJobExecution.getStatus());
createMoreData();
JobExecution secondJobExecution = runJobAndGetExecution("batch1_id2", jobRepository, job);
assertEquals(BatchStatus.COMPLETED, secondJobExecution.getStatus());
// then
// assertions on data
}
public static JobExecution runJobAndGetExecution(String jobId, JobRepository jobRepository, Job job)
throws JobInstanceAlreadyCompleteException, JobExecutionAlreadyRunningException, JobParametersInvalidException, JobRestartException {
JobParameters param = new JobParametersBuilder()
.addString("JobID", jobId)
.addLong("currentTime", System.currentTimeMillis())
.toJobParameters();
SimpleJobLauncher launcher = new SimpleJobLauncher();
launcher.setJobRepository(jobRepository);
launcher.setTaskExecutor(new SyncTaskExecutor());
return launcher.run(job, param);
}
Please note that I pass a timestamp param in the job parameters to ensure that params are different.
Please also note that running the two different batches one after the other (batch1 and then batch2 is working fine, but batch1 then batch1 or batch2 then batch2 is not)
Any idea why it seems to run ? (I put some breakpoints and everything seems to happen correctly, just the reader, processer and writer are not called).
I resolved it. It was a misconception of the reader I implemented that was never reinitializing data, because I missunderstood the way reader works. The reader is called only once even if we're running the job 10 times.
The comment from Mahmoud Ben Hassine on the post is complety right
I've developed spring batch job which gets the data from JDBC. The problem I'm facing, it's executing on project startup regardless of enabled_property. The value of the property is FALSE. I've tried to create a conditional bean on property but it didn't also worked and job is being executed on project startup.
Following the my code snippet.
#Bean
#ConditionalOnProperty(value = "wallet-manager.djeezyConfig.enableJob" , havingValue = "false")
public Job createJob() {
return jobBuilderFactory.get("DJeezy wallet cleaner job")
.incrementer(new RunIdIncrementer())
.flow(Step1())
.end()
.build();
}
#Bean
#ConditionalOnProperty(value = "wallet-manager.djeezyConfig.enableJob" , havingValue = "false")
public Step Step1() {
return stepBuilderFactory.get("DJeezy wallet cleaner job - step1")
.<ResellerWallet,ResellerWallet> chunk(wConfig.getDjeezyConfig().getChunkSize())
.reader(resellerWalletItemReader)
//.processor(resellerWalletProcessor)
.writer(resellerWalletItemWriter)
.faultTolerant()
.skip(EmptyResultDataAccessException.class)
.build();
}
I've also tried to commented the #Scheduled annotation but it stills executing the job and steps.
//#Scheduled(fixedDelay = 15000)
public void scheduleByFixedRate() throws Exception {
if(config.getDjeezyConfig().isEnableJob()) {
System.out.println("Batch job starting");
JobParameters jobParameters = new JobParametersBuilder()
.addString("time", format.format(Calendar.getInstance().getTime())).toJobParameters();
jobLauncher.run(job, jobParameters);
System.out.println("Batch job executed successfully\n");
}
}
Can someone please guide me what I'm missing here? and how can I prevent my job and step being executed on startup.
spring.batch.job.enabled=false
hope you are using this property in your properties file
this should work
I'm currently trying to launch a Job programatically in my Spring application.
Here is my actual code:
public void launchJob(String[] args)
throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException,
JobInstanceAlreadyCompleteException {
String jobName = args[0];
JobLauncher jobLauncher = context.getBean(JobLauncher.class);
Job job = context.getBean(jobName, Job.class);
JobParameters jobParameters = getJobParameters(args);
JobExecution jobExecution = jobLauncher.run(job, jobParameters);
BatchStatus batchStatus = jobExecution.getStatus();
}
And how I launch it :
String[] args = {"transformXML2CSV", "myFile.xml", "myFile.csv"};
springBatchJobRunner.launchJob(args);
But I have some troubles during the launch, the first is to retrieve the app context, my first try was to annotate my class with a #Service and use an #Autowired like this:
#Autowired
private ApplicationContext context;
But with this way my context is always null.
My second try was to get the context by this way:
ApplicationContext context = new AnnotationConfigApplicationContext(SpringBatchNoDatabaseConfiguration.class);
The SpringBatchNoDatabaseConfiguration is my #Configuration and the Job is inside it.
Using this my context is not null but I have a strange behavior and I can't understand why and how to prevent this:
I run the launchJob function from my processor class, then when I get the context by AnnotationConfigApplicationContext my processor class is rebuild and I have a NullPointerException in it stopping all the process.
I really don't understand the last part, why it's relaunching my processor class when I get the context ?
As indicated in comments above, you run a parent batch (with spring-batch) which at a moment needs your job to process an xml file.
I suggest you keep the same spring-batch context and run the process xml file job as a nested job of the parent job. You can do that using JobStep class and spring batch controlling step flow feature. As an example, here is what your parent job would like to :
public Job parentJob(){
JobParameters processXmlFileJobParameters = getJobParameters(String[]{"transformXML2CSV", "myFile.xml", "myFile.csv"});
return this.jobBuilderFactory.get("parentJob")
.start(firstStepOfParentJob())
.on("PROCESS_XML_FILE").to(processXmlFileStep(processXmlFileJobParameters)
.from(firstStepOfParentJob()).on("*").to(lastStepOfParentJob())
.from(processXmlFileStep(processXmlFileJobParameters))
.next(lastStepOfParentJob())
.end().build();
}
public Step firstStepOfParentJob(){
return stepBuilderFactory.get("firstStepOfParentJob")
// ... depends on your parent job's business
.build();
}
public Step lastStepOfParentJob(){
return stepBuilderFactory.get("lastStepOfParentJob")
// ... depends on your parent job's business
.build();
}
public Step processXmlFileStep(JobParameters processXmlFileJobParameters){
return stepBuilderFactory.get("processXmlFileStep")
.job(processXmlFileJob())
.parametersExtractor((job,exec)->processXmlFileJobParameters)
.build();
}
public Job processXmlFileJob(){
return jobBuilderFactory.get("processXmlFileJob")
// ... describe here your xml process job
.build();
}
It seems a second Spring context is initialized with the instruction new AnnotationConfigApplicationContext(SpringBatchNoDatabaseConfiguration.class) so Spring initialiezs your beans a second time.
I recommend you to use Spring boot to automatically launch your jobs at sartup
If you don't want to use Spring boot and launch your job manually, your main method should look like this :
public static void main(String[] args){
String[] localArgs = {"transformXML2CSV", "myFile.xml", "myFile.csv"};
ApplicationContext context = new AnnotationConfigApplicationContext(SpringBatchNoDatabaseConfiguration.class);
Job job = context.getBean(Job.class);
JobLauncher jobLauncher = context.getBean(JobLauncher.class);
JobParameters jobParameters = getJobParameters(localArgs);
jobExecution = jobLauncher.run(job, jobParameters);
// Ohter code to do stuff after the job ends...
}
Notice that this code should be completed with your own needs. The class SpringBatchNoDatabaseConfigurationhas to be anotated with #Configuration and #EnableBatchProcessingand should define the beans of your job like this
You can also use the Spring batch class CommandLineJobRunner with your java config as explain here : how to launch Spring Batch Job using CommandLineJobRunner having Java configuration
It saves writing the code above
How can I stop a job in spring batch ? I tried to use this method using the code below:
public class jobListener implements JobExecutionListener{
#Override
public void beforeJob(JobExecution jobExecution) {
jobExecution.setExitStatus(ExitStatus.STOPPED);
}
#Override
public void afterJob(JobExecution jobExecution) {
// TODO Auto-generated method stub
}
}
I tried also COMPLETED,FAILED but this method doesn't work and the job continues to execute. Any solution?
You can use JobOperator along with JobExplorer to stop a job from outside the job (see https://docs.spring.io/spring-batch/reference/html/configureJob.html#JobOperator). The method is stop(long executionId) You would have to use JobExplorer to find the correct executionId to stop.
Also from within a job flow config you can configure a job to stop after a steps execution based on exit status (see https://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html#stopElement).
I assume you want to stop a job by a given name.
Here is the code.
String jobName = jobExecution.getJobInstance().getJobName(); // in most cases
DataSource dataSource = ... //#Autowire it in your code
JobOperator jobOperator = ... //#Autowire it in your code
JobExplorerFactoryBean factory = new JobExplorerFactoryBean();
factory.setDataSource(dataSource);
factory.setJdbcOperations(new JdbcTemplate(dataSource));
JobExplorer jobExplorer = factory.getObject();
Set<JobExecution> jobExecutions = jobExplorer.findRunningJobExecutions(jobName);
jobExecutions.forEach(jobExecution -> jobOperator.stop(jobExecution.getId()));
I am working on a project where we are using Spring Boot, Spring Batch and Camel.
The batch process is started by a call to a rest endpoint. The rest controller starts a camel route that starts the spring batch job flow (via spring batch camel component).
I have no control over the external application that calls my application. My application is part of a bigger nightly work flow.
The batch job can take a long time to complete and therefore the external application periodically polls my batch job via another rest endpoint asking if the job is complete. It does this by polling a status rest endpoint with the id of the jobExecution it wants a status on.
To accomplish this flow I have implemented a rest controller that starts the camel route via a ProducerTemplate. My problem is returning the job execution id immediately after starting the camel route. I don't want the rest call to wait until the job is complete to return.
startJobViaRestCall ------> createBatchJob ----> runBatchJobUntilDone
|
|
Return jobExecutionData |
<----------------------------------
I have tried using async calls and futures, but with no luck. I have also tried to use Camels wiretap to no avail. The problem is that there is only "onComplete" events. I need an hook that returns as soon as the job has been created, but not run.
For example, the following code waits until the batch job is done before returning the JobExecution data I want to send back (as json). It makes sense as extractFutureBody will wait until the response is ready.
#RestController
#Slf4j
public class BatchJobController {
#Autowired
ProducerTemplate producerTemplate;
#RequestMapping(value = "/batch/job/start", method = RequestMethod.GET)
#ResponseBody
public String startBatchJob() {
log.info("BatchJob start called...");
String jobExecution = producerTemplate.extractFutureBody(producerTemplate.asyncRequestBody(BatchRoute.ENDPOINT_JOB_START, ""), String.class);
return jobExecution;
}
}
The camel route is a simple call to the spring-batch-component
public class BatchRoute<I, O> extends BaseRoute {
private static final String ROUTE_START_BATCH = "spring-batch:springBatchJob";
#Override
public void configure() {
super.configure();
from(ENDPOINT_JOB_START).to(ROUTE_START_BATCH);
}
}
Any ideas as to how I can return the JobExecution data as soon as it is available?
Not sure How you could do it in Camel, but here is sample Job execution using spring-rest.
#RestController
public class KpRest {
private static final Logger LOG = LoggerFactory.getLogger(KpRest.class);
private static String RUN_ID_KEY = "run.id";
#Autowired
private JobLauncher launcher;
private final AtomicLong incrementer = new AtomicLong();
#Autowired
private Job job;
#RequestMapping("/hello")
public String sayHello(){
try {
JobParameters parameters = new JobParametersBuilder().addLong(RUN_ID_KEY, incrementer.incrementAndGet()).toJobParameters();
JobExecution execution = launcher.run(job, parameters);
LOG.info("JobId {}, JobStatus {}", execution.getJobId(), execution.getStatus().getBatchStatus());
return String.valueOf(execution.getJobId());
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException
| JobParametersInvalidException e) {
LOG.info("Job execution failed, {}", e);
}
return "Some Error";
}
}
You can make the Job async by modifying JobLauncher.
#Bean
public JobLauncher simpleJobLauncher(JobRepository jobRepository){
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
return jobLauncher;
}
Refer the documentation for more info