Spring boot batch - cannot stop the job. Scheduler called only twice - spring

I am trying to stop and restart a spring batch job in every 10 seconds. I am trying to do this with job_execution_id but it throws an error saying "No job configuration with the name [loadUserJob] was registered".
This is a test application for a real process where I want to start and stop on specific time of the day. Here is the scheduler method.
#Scheduled(cron = "*/10 * * * * *")
public void batchScheduler() {
LOG.info("---Beginning of batchScheduler()---");
if(isRunning) {
try {
LOG.info("....stopping the job!");
LOG.info("-----------> 2");
jobExecutionId = jobRepository.getLastJobExecution("dataFilterJob", jobParameter).getId();
LOG.info("##### ExecutionID-1: " + jobExecutionId);
this.isRunning = false;
this.jobOperator.stop(jobExecutionId);
} catch (NoSuchJobExecutionException | JobExecutionNotRunningException e) {
this.isRunning = true;
LOG.info("Error in Stopping job!!");
e.printStackTrace();
}
}
else {
try {
LOG.info("Restarting the job....");
LOG.info("-----------> 3");
LOG.info("##### ExecutionID-2: " + jobExecutionId);
this.isRunning = true;
this.jobOperator.restart(jobExecutionId);
} catch (JobInstanceAlreadyCompleteException | NoSuchJobExecutionException | NoSuchJobException
| JobRestartException | JobParametersInvalidException e) {
this.isRunning = false;
LOG.info("Error in Restarting the job!!");
e.printStackTrace();
}
}
LOG.info("---End of batchScheduler()---");
}
jobRepository and jobOperator are autowired in the same class. Not sure why it can't find the job with stopped execution id.
Here is the whole class. I am using CommandLineRunner
#SpringBootApplication
#EnableScheduling
public class DatafilterBatchApplication implements CommandLineRunner{
public static final Logger LOG = LogManager.getLogger(DatafilterBatchApplication.class);
#Autowired
private JobLauncher jobLauncher;
#Autowired
private ApplicationContext context;
#Autowired
private JobRepository jobRepository;
#Autowired
private JobOperator jobOperator;
#Autowired
private JobExplorer jobExplorer;
#Autowired
private JobRegistry jobRegistry;
long jobExecutionId;
boolean isRunning = false;
boolean isNotRunning = false;
private String batchName;
private String filePath;
private long currentMillis = System.currentTimeMillis();
private JobParameters jobParameter;
public static void main(String[] args) {
SpringApplication.run(DatafilterBatchApplication.class, args);
}
#Override
public void run(String... args) throws Exception {
LOG.info("---Beginning of run()---");
try {
if(args.length == 2) {
batchName = args[0];
filePath = args[1];
jobParameter = new JobParametersBuilder()
.addLong("time", currentMillis)
.addString("inputFile", filePath)
.toJobParameters();
ExitStatus exitStatus = jobController(jobParameter);
LOG.info("Job completed with status-" + exitStatus);
}else {
LOG.info("Invalid Job Parameters!!");
}
} catch (Exception e) {
e.printStackTrace();
}
LOG.info("---End of run()---");
}
public ExitStatus jobController(JobParameters jobParameters) {
LOG.info("---Beginning of jobController()---");
Job job = this.context.getBean("dataFilterJob", Job.class);
ExitStatus exitStatus = ExitStatus.UNKNOWN;
try {
isRunning = true;
exitStatus = jobLauncher.run(job, jobParameters).getExitStatus();
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException
| JobParametersInvalidException e) {
LOG.info("Error in launching job!!");
e.printStackTrace();
}
LOG.info("---End of jobController()---");
return exitStatus;
}
#Scheduled(cron = "*/10 * * * * *")
public void batchScheduler() {
LOG.info("---Beginning of batchScheduler()---");
if(isRunning) {
try {
LOG.info("....stopping the job!");
LOG.info("-----------> 2");
jobExecutionId = jobRepository.getLastJobExecution("dataFilterJob", jobParameter).getId();
LOG.info("##### ExecutionID-1: " + jobExecutionId);
this.isRunning = false;
this.jobOperator.stop(jobExecutionId);
} catch (NoSuchJobExecutionException | JobExecutionNotRunningException e) {
this.isRunning = true;
LOG.info("Error in Stopping job!!");
e.printStackTrace();
}
}
else {
try {
LOG.info("Restarting the job....");
LOG.info("-----------> 3");
LOG.info("##### ExecutionID-2: " + jobExecutionId);
this.isRunning = true;
this.jobOperator.restart(jobExecutionId);
} catch (JobInstanceAlreadyCompleteException | NoSuchJobExecutionException | NoSuchJobException
| JobRestartException | JobParametersInvalidException e) {
this.isRunning = false;
LOG.info("Error in Restarting the job!!");
e.printStackTrace();
}
}
LOG.info("---End of batchScheduler()---");
}
}
Here is the exception screenshot:
UPDATE
I tried JobRegistryBeanPostProcessor to set JobRegistry and it seems working
#Autowired
private JobRegistry jobRegistry;
#Bean
public JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor() {
JobRegistryBeanPostProcessor postProcessor = new JobRegistryBeanPostProcessor();
postProcessor.setJobRegistry(jobRegistry);
return postProcessor;
}
But now it turned out that the scheduler is called only twice and after the second call it is never been called so the batch never ends. Is that a Scheduler issue or JobRegistryBeanPostProcessor is causing this ? because if I remove it then the Scheduler is called frequently at every 10 seconds but throwing same error: "org.springframework.batch.core.launch.NoSuchJobException: No job configuration with the name [dataFilterJob] was registered."
Here is the whole updated class:
#SpringBootApplication
#EnableScheduling
public class DatafilterBatchApplication implements CommandLineRunner{
public static final Logger LOG = LogManager.getLogger(DatafilterBatchApplication.class);
#Autowired
private JobLauncher jobLauncher;
#Autowired
private ApplicationContext context;
#Autowired
private JobRepository jobRepository;
#Autowired
private JobRegistry jobRegistry;
#Autowired
private JobOperator jobOperator;
#Autowired
private JobExplorer jobExplorer;
private String jobName;
private JobParameters jobParameters;
private String completionStatus;
boolean isRunning = false;
private String filePath;
#Bean
public JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor() {
JobRegistryBeanPostProcessor postProcessor = new JobRegistryBeanPostProcessor();
postProcessor.setJobRegistry(jobRegistry);
return postProcessor;
}
public static void main(String[] args) {
SpringApplication.run(DatafilterBatchApplication.class, args);
}
#Override
public void run(String... args) throws Exception {
LOG.info("---Beginning of run()---");
try {
if(args.length == 2) {
jobName = args[0];
filePath = args[1];
ExitStatus exitStatus = jobController(jobName, filePath);
LOG.info("Job completed with status-" + exitStatus);
}else {
LOG.info("Invalid Job Parameters!!");
}
} catch (Exception e) {
e.printStackTrace();
}
LOG.info("---End of run()---");
}
public ExitStatus jobController(String jobName, String fileName) {
LOG.info("---Beginning of jobController()---");
Job job = this.context.getBean(jobName, Job.class);
ExitStatus exitStatus = ExitStatus.UNKNOWN;
jobParameters = new JobParametersBuilder()
.addLong("time", System.currentTimeMillis())
.addString("inputFile", fileName)
.toJobParameters();
try {
isRunning = true;
exitStatus = jobLauncher.run(job, jobParameters).getExitStatus();
if(exitStatus.getExitCode().equals(ApplicationConstants.JOB_EXITSTATUS_STOPPED)) {
isRunning = false;
}
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException
| JobParametersInvalidException e) {
LOG.info("Error in launching job!!");
isRunning = false;
e.printStackTrace();
}
LOG.info("---End of jobController()---");
return exitStatus;
}
#Scheduled(cron = "*/10 * * * * *")
public void batchScheduler() {
LOG.info("---Beginning of batchScheduler()---");
if(isRunning) {
try {
LOG.info("....stopping the job!");
this.isRunning = false;
jobOperator.stop(jobRepository.getLastJobExecution(jobName, this.jobParameters).getId());
} catch (NoSuchJobExecutionException | JobExecutionNotRunningException e) {
this.isRunning = true;
LOG.info("Error in Stopping job!!");
e.printStackTrace();
}
}else {
try {
LOG.info("Restarting the job....");
this.isRunning = true;
jobOperator.restart(jobRepository.getLastJobExecution(jobName, this.jobParameters).getId());
} catch (JobInstanceAlreadyCompleteException | NoSuchJobExecutionException | NoSuchJobException
| JobRestartException | JobParametersInvalidException e) {
this.isRunning = false;
LOG.info("Error in Restarting the job!!");
e.printStackTrace();
}
}
LOG.info("---End of batchScheduler()---");
}
}
UPDATE
I think the issue was with JobOperator and as I updated before JobRegistryBeanPostProcessor would be the solution. But as I mentioned my scheduler is called only twice. So once the application starts, it calls after 10 seconds and stop the running job. It again call one more time after 10 seconds and restart the job and that's all. The scheduler never gets called. So the job just continue to run. I have cron for mat as - (cron = "*/10 * * * * *") so it should be calling in every 10 seconds. Can anyone help me to find out what is going on here. Is that something to do with my scheduler ?
UPDATE
I think I should take back

Add the #Scheduled annotation to any method that you wish to run automatically and include #EnableScheduling in a configuration file.
and if you want to schedule it in a chosen time add #Scheduled(cron = "0 45 15 * * ?") to the method like here I schedule my method to run every day at 15h45 for more information you can find here:https://dzone.com/articles/running-on-time-with-springs-scheduled-tasks

Related

If exception thrown, spring batch should not go to last method

#Configuration
public class BookStorePoliciesJobConfiguration {
private static final String CRON_EXPRESSION_FOR_JOB = "0 0/10 * 1/1 * ? *";//change
private static final String JOB_NAME = "bookStorePolicyJob";
private static final String JOB_STEP_NAME = JOB_NAME + "Step";
#Autowired
private DataSource nonXAcrmDataSource;
#Autowired
private JtaTransactionManager transactionManager;
#Bean
public CanerScheduledJobFactoryBean bookStorePoliciesScheduledJob() {
MafScheduledJobFactoryBean bean = new MafScheduledJobFactoryBean();
bean.setBatchJobName(JOB_NAME);
bean.setCronExp(CRON_EXPRESSION_FOR_JOB);
bean.setNonXAcrmDataSource(this.nonXAcrmDataSource);
return bean;
}
#Bean
public Job bookStorePolicyJob(#Autowired BookStorePoliciesJobTasklet tasklet,
#Autowired JobRepository batchJobRepository) {
SimpleJob job = new SimpleJob(JOB_NAME);
job.setSteps(Collections.singletonList(bookStorePolicyJobStep(tasklet, batchJobRepository)));
job.setJobRepository(batchJobRepository);
return job;
}
public Step bookStorePolicyJobStep(BookStoreJobTasklet tasklet, JobRepository batchJobRepository) {
TaskletStep step = new TaskletStep(JOB_STEP_NAME);
step.setTasklet(tasklet);
step.setJobRepository(batchJobRepository);
transactionManager.setAllowCustomIsolationLevels(true);
step.setTransactionManager(transactionManager);
return step;
}
}
This is the job:
#Component
public class BookStoreJobTasklet extends CanerTasklet {
#Override
public RepeatStatus doExecute(StepContribution contribution, ChunkContext chunkContext) {
///
//
sendFileBySftp(fileName, file);//if this throws, should not go down line, should exit but it goes
updateParameters();//if it is successfull only!
return RepeatStatus.FINISHED;
}
And the method:
private void sendFileBySftp(String fileName, File file) {
//
//
try{
//
} catch (JSchException | SftpException | IOException e) {
logger.error("error in sendFileFTP {}", e.getMessage());//it comes here and continues. it should exit?
}
logger.info("Session connection closed....");
}
What should i do to stop batch?
i look at here
Make a spring-batch job exit with non-zero code if an exception is thrown
Should i return RepeatStatus for each method so i can check if failed?
Maybe a boolean value and for each catch block, putting it to false?

Spring batch run multiple jobs in parallel

I am new to Spring batch and couldn't figure out how to do this..
Basically I have a spring batch files and both are have to run parallel i.e when I request execute_job1 then BatchConfig1 have to execute and when I request execute_job2 then BatchConfig2 have to execute. How can I do this?
Controller
#RestController
public class JobExecutionController {
#Autowired
JobLauncher jobLauncher;
#Autowired
Job job;
/**
*
* #return
*/
#RequestMapping("/execute_job1")
#ResponseBody
public void executeBatchJob1() {
}
/**
*
* #return
*/
#RequestMapping("/execute_job2")
#ResponseBody
public void executeBatchJob2() {
}
}
BatchConfig1
#Configuration
#EnableBatchProcessing
public class BatchConfig {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
#Bean
public Step stepOne(){
return steps.get("stepOne")
.tasklet(new MyTaskOne())
.build();
}
#Bean
public Step stepTwo(){
return steps.get("stepTwo")
.tasklet(new MyTaskTwo())
.build();
}
#Bean
public Job demoJob(){
return jobs.get("exportUserJob1")
.incrementer(new RunIdIncrementer())
.start(stepOne())
.next(stepTwo())
.build();
}
}
BatchConfig2:
#Configuration
#EnableBatchProcessing
public class BatchConfig2 {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Autowired
public DataSource dataSource;
#Bean
public JdbcCursorItemReader<User> reader() {
JdbcCursorItemReader<User> reader = new JdbcCursorItemReader<User>();
reader.setDataSource(dataSource);
reader.setSql("SELECT id,name FROM user");
reader.setRowMapper(new UserRowMapper());
return reader;
}
public class UserRowMapper implements RowMapper<User> {
#Override
public User mapRow(ResultSet rs, int rowNum) throws SQLException {
User user = new User();
user.setId(rs.getInt("id"));
user.setName(rs.getString("name"));
return user;
}
}
#Bean
public UserItemProcessor processor() {
return new UserItemProcessor();
}
#Bean
public FlatFileItemWriter<User> writer() {
FlatFileItemWriter<User> writer = new FlatFileItemWriter<User>();
writer.setResource(new ClassPathResource("users.csv"));
writer.setLineAggregator(new DelimitedLineAggregator<User>() {
{
setDelimiter(",");
setFieldExtractor(new BeanWrapperFieldExtractor<User>() {
{
setNames(new String[] { "id", "name" });
}
});
}
});
return writer;
}
#Bean
public Step step1() {
return stepBuilderFactory.get("step1").<User, User>chunk(10).reader(reader()).processor(processor())
.writer(writer()).build();
}
#Bean
public Job exportUserJob() {
return jobBuilderFactory.get("exportUserJob2").incrementer(new RunIdIncrementer()).flow(step1()).end().build();
}
}
You can run a job with JobLauncher. The code for the controller:
#RestController
public class JobExecutionController {
public JobExecutionController(JobLauncher jobLauncher,
#Qualifier("demoJob") Job demoJob,
#Qualifier("exportUserJob") Job exportUserJob) {
this.jobLauncher = jobLauncher;
this.demoJob = demoJob;
this.exportUserJob = exportUserJob;
}
JobLauncher jobLauncher;
Job demoJob;
Job exportUserJob;
#RequestMapping("/execute_job1")
#ResponseBody
public void executeBatchJob1() {
try {
JobExecution jobExecution = jobLauncher.run(demoJob, new JobParameters(generateJobParameter()));
log.info("Job started in thread :" + jobExecution.getJobParameters().getString("JobThread"));
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobParametersInvalidException | JobInstanceAlreadyCompleteException e) {
log.error("Something sent wrong during job execution", e);
}
}
#RequestMapping("/execute_job2")
#ResponseBody
public void executeBatchJob2() {
try {
JobExecution jobExecution = jobLauncher.run(exportUserJob, new JobParameters(generateJobParameter()));
log.info("Job started in thread :" + jobExecution.getJobParameters().getString("JobThread"));
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobParametersInvalidException | JobInstanceAlreadyCompleteException e) {
log.error("Something sent wrong during job execution", e);
}
}
private JobParameters generateJobParameter() {
Map<String, JobParameter> parameters = new HashMap<>();
parameters.put("Job start time", new JobParameter(Instant.now().toEpochMilli()));
parameters.put("JobThread", new JobParameter(Thread.currentThread().getId()));
return new JobParameters(parameters);
}
}
To prevent starting your jobs at the application start add to the application.properties next spring batch configuration: spring.batch.job.enabled=false

#Retryable is not working when calling from a method

Below is my application class. The flow is like the DEToken class from here and from DEToken I call RestConnection where I have the #retryable method.
#SpringBootApplication
#EnableRetry
public class SpringBootTrfficApplication implements CommandLineRunner {
Enter code here
#Autowired
DEToken deToken;
#Autowired
SyncService syncService;
public static void main(String[] args) {
SpringApplication.run(SpringBootTrfficApplication.class, args);
}
#Override
public void run(String... args) throws Exception {
deToken.getToken();
}
}
DEToken class: from getToken I am calling RestConnect where I have the #Retrable method:
#Service
public class DEToken {
private Logger logger = LogManager.getLogger(getClass());
#Autowired
RestConnection restConnection;
#Autowired
private Environment env;
public String accessToken;
public void getToken() {
System.out.println("hello from get token");
//String getJsonPayload = "{\"Query\":{\"RegisterExtensionWithDE\":{\"pid\": \"\",\"providerInsName\":" +
//env.getProperty("provider.ins") + "}}}";
//String str = restConnection.restPost(
// env.getProperty("rest.de.url"), getJsonPayload);
try {
String getJsonPayload =
"{\"Query\":{\"RegisterExtensionWithDE\":{\"pid\": \"\",\"providerInsName\":" +
env.getProperty("provider.ins") + "}}}";
StringBuffer tokenResult =
restConnection.restPost(env.getProperty("rest.de.url"),
getJsonPayload);
System.out.println(tokenResult);
JSONObject xmlJSONObj = XML.toJSONObject(tokenResult.toString());
JSONObject registration = new JSONObject();
if (xmlJSONObj.has("Registration")) {
registration = xmlJSONObj.getJSONObject("Registration");
if (registration.has("accessToken")) {
accessToken = registration.get("accessToken").toString();
}
else
logger.info("no accessToken from DE");
}
else
logger.info("no Registration object from DE");
}
catch (Exception e) {
logger.error("Exception while fetching accesstoken from DE ");
logger.error(e.getMessage());
}
}
}
My REST connection class where I have retryable method:
#Service
public class RestConnection {
private Logger logger = LogManager.getLogger(getClass());
#Autowired
private Environment env;
public void setBaseUrl(String value, String ip) {
//baseUrl = value;
HttpsURLConnection.setDefaultHostnameVerifier(
(hostname, session) -> hostname.equals(ip));
}
/*
* REST post call
*/
#Retryable(value = {IOException.class, ConnectException.class},
maxAttempts = 4,
backoff = #Backoff(5000))
public StringBuffer restPost(String restUrl, String payload) {
StringBuffer sb = new StringBuffer();
HttpURLConnection conn = null;
try {
URL url = new URL(restUrl);
String protocol = url.getProtocol();
if (protocol.toLowerCase().equals("http")) {
conn = (HttpURLConnection)url.openConnection();
}
else if (protocol.toLowerCase().equals("https")) {
//setTrustedCert();
conn = (HttpsURLConnection)url.openConnection();
}
else {
logger.info("Protocol is neither HTTP nor HTTPS");
}
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestProperty("Accept", "application/json");
conn.setRequestProperty("version", env.getProperty("de.version"));
conn.setRequestProperty("accessToken", env.getProperty("access.token"));
conn.setRequestProperty("requestHost", env.getProperty("server.de.host"));
conn.setRequestProperty("requestPort", env.getProperty("server.port"));
conn.setRequestProperty("requestProtocol",
env.getProperty("server.de.protocol"));
PrintWriter pout =
new PrintWriter(
new OutputStreamWriter(
conn.getOutputStream(), "UTF-8"),
true);
pout.print(payload);
pout.flush();
pout.close();
InputStream isi = conn.getInputStream();
InputStreamReader isr = new InputStreamReader(isi);
int numCharsRead1;
char[] charArray1 = new char[1024];
while ((numCharsRead1 = isr.read(charArray1)) > 0) {
sb.append(charArray1, 0, numCharsRead1);
}
isr.close();
isi.close();
}
catch (MalformedURLException e) {
logger.error("MalformedURLException in restAccessTokenPOST..." +
e.getMessage());
//e.printStackTrace();
}
catch (IOException e) {
logger.error("IOException in restAccessTokenPOST..." +
e.getMessage());
e.printStackTrace();
}
catch (Exception e) {
logger.error("Exception in restAccessTokenPOST..." +
e.getMessage());
e.printStackTrace();
}
finally {
if (null != conn)
conn.disconnect();
}
return sb;
}
#Recover
public String helpHere(ConnectException cause) {
System.out.println("Recovery place! ConnectException");
return "Hello";
}
#Recover
public String helpHere(IOException cause) {
System.out.println("Recovery place! ArithmeticException");
return "Hello";
}
#Recover
public String helpHere(Exception cause) {
System.out.println("Recovery place! Exception");
return "Hello";
}
#Recover
public String helpHere() {
System.out.println("Recovery place! Exception");
return "Hello";
}
#Recover
public String helpHere(Throwable cause) {
System.out.println("Recovery place! Throwable");
return "Hello";
}
}
Considering you see your function restPost() implementation,
#Retryable(value = {IOException.class, ConnectException.class},
maxAttempts = 4,
backoff = #Backoff(5000))
public StringBuffer restPost(String restUrl, String payload) {
try {
// Your code
}
catch(IOException ex){ // These catch block handles the exception
// and nothing to throw to retryable.
}
catch(MalformedURLException ex){ // More catch blocks that you
// define to handle exception.
}
}
Here you handle all of the exceptions that can be a cause to revoke the retry and recover methods.
Note: Recoverable methods only execute when a exception is thrown, not handled by any try-catch block.
Whatever exception is raised by method restPost() is handled by the method try-catch block itself and there are no exceptions that had been rethrow by a catch block.
Now, Spring-Retry is unable to get any exception (because it is handled by the method try-catch block). So, no recovery method will be executed.
Solution: you should remove those catch blocks from the method definition on which you want to perform retry or recover.
Please do the needful and it will work like a charm... :)

Spring Batch Not reading from DB nor writing to file

I want to use multiple datasources, one for the Spring Batch Metadata and the other for the business data. My batch job just runs and does not even try to connect to the secondaryDataSource. can someone point out what is wrong with my configuration?
#Configuration
#EnableBatchProcessing
public class BatchConfiguration extends DefaultBatchConfigurer {
#Override
#Autowired
public void setDataSource(
#Qualifier("batchDataSource") DataSource batchDataSource) {
super.setDataSource(batchDataSource);
}
}
public class SpringBatchConfig {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
private static final String QUERY_FIND_STUDENTS = "select * from ...";
#Bean
ItemReader<DotDetailsDTO> reader(
#Qualifier("secondaryDataSource") DataSource dataSource)
throws SQLException {
JdbcCursorItemReader<DotDetailsDTO> databaseReader = new JdbcCursorItemReader<>();
databaseReader.setDataSource(dataSource);
databaseReader.setSql(QUERY_FIND_STUDENTS);
databaseReader.setRowMapper(new DOTRowMapper());
return databaseReader;
}
#Bean
public ItemProcessor<DotDetailsDTO, DotDetailsDTO> itemProcessor() {
return new CustomItemProcessor();
}
#Bean
public ItemWriter<DotDetailsDTO> writer() throws Exception {
FlatFileItemWriter<DotDetailsDTO> writer = new FlatFileItemWriter<DotDetailsDTO>();
writer.setResource(new ClassPathResource("file:test.csv"));
DelimitedLineAggregator<DotDetailsDTO> delLineAgg = new DelimitedLineAggregator<DotDetailsDTO>();
delLineAgg.setDelimiter(",");
BeanWrapperFieldExtractor<DotDetailsDTO> fieldExtractor = new BeanWrapperFieldExtractor<DotDetailsDTO>();
fieldExtractor.setNames(new String[] { "airwayBillNumber",
"outboundDate", "orig", "dest", "lotNumber",
"lotFlightNumber", "lotOrig", "lotDest", "lotPcs", "lotWt",
"lotFlightDepartDate", "iataCode" });
delLineAgg.setFieldExtractor(fieldExtractor);
writer.setLineAggregator(delLineAgg);
writer.afterPropertiesSet();
return writer;
}
#Bean
protected Step step1(ItemReader<DotDetailsDTO> reader,
ItemProcessor<DotDetailsDTO, DotDetailsDTO> processor,
ItemWriter<DotDetailsDTO> writer) throws SQLException {
return steps.get("step1").<DotDetailsDTO, DotDetailsDTO> chunk(10)
.reader(reader).processor(processor).writer(writer).build();
}
#Bean(name = "firstBatchJob")
public Job job(#Qualifier("step1") Step step1) {
return jobs.get("firstBatchJob").start(step1).build();
}
}
public class DataSourceConfiguration {
#Bean(name="batchDataSource")
public DataSource dataSource() throws SQLException {
BasicDataSource dataSource = new BasicDataSource();
...
return dataSource;
}
#Bean
public JdbcTemplate jdbcTemplate(
#Qualifier("batchDataSource") final DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
#Primary
#Bean(name="secondaryDataSource")
public DataSource secondaryDataSource() throws SQLException {
OracleDataSource secondaryDataSource = new OracleDataSource();
...
return secondaryDataSource;
}
#Bean
public JdbcTemplate secondaryJdbcTemplate(
#Qualifier("secondaryDataSource") final DataSource secondaryDataSource) {
return new JdbcTemplate(secondaryDataSource);
}
}
public static void main(String[] args) {
// Spring Java config
AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
context.register(DataSourceConfiguration.class);
context.register(BatchConfiguration.class);
context.register(SpringBatchConfig.class);
context.refresh();
JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
Job job = (Job) context.getBean("firstBatchJob");
System.out.println("Starting the batch job");
try {
JobExecution execution = jobLauncher.run(job, new JobParameters());
System.out.println("Job Status : " + execution.getStatus());
System.out.println("Job completed");
} catch (Exception e) {
e.printStackTrace();
System.out.println("Job failed");
}
}
Wow after 2 days I figured out what the issue was. I was not providing new JobParameters because of which I was running the same old broken job over and over.
Below is the fix in the main method.
public static void main(String[] args) {
AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
context.register(DataSourceConfiguration.class);
context.register(BatchConfiguration.class);
context.register(SpringBatchConfig.class);
context.refresh();
JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
Job job = (Job) context.getBean("firstBatchJob");
System.out.println("Starting the batch job");
try {
DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
Date date = new Date();
JobParameters jobParam =
new JobParametersBuilder().addString("jobDate",dateFormat.format(date)).toJobParameters();
JobExecution execution = jobLauncher.run(job, jobParam);
System.out.println("Job Status : " + execution.getStatus());
System.out.println("Job completed : " + execution.getJobId());
} catch (Exception e) {
e.printStackTrace();
System.out.println("Job failed");
}
}

Is there a bug in Spring Batch Step flow function?

In the below piece of code, when StepA fails only StepB and StepC should execute but what actually happens is that all the 3 steps are getting executed! I want to split a spring batch job depending upon whether a step passes or not. I know that there are other ways of doing this by using JobDecider, setting some job parameter, etc but I wanted to know I was doing wrongly here?
#Configuration
#EnableBatchProcessing
public class JobConfig {
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Bean
public PlatformTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
#Bean
public JobRepository jobRepository() {
try {
return new MapJobRepositoryFactoryBean(transactionManager())
.getJobRepository();
} catch (Exception e) {
return null;
}
}
#Bean
public JobLauncher jobLauncher() {
final SimpleJobLauncher launcher = new SimpleJobLauncher();
launcher.setJobRepository(jobRepository());
return launcher;
}
#Bean
public Job job() {
return jobBuilderFactory.get("job").
flow(stepA()).on("FAILED").to(stepC()).next(stepD()).
from(stepA()).on("*").to(stepB()).next(stepC()).end().build();
}
#Bean
public Step stepA() {
return stepBuilderFactory.get("stepA")
.tasklet(new RandomFailTasket("stepA")).build();
}
#Bean
public Step stepB() {
return stepBuilderFactory.get("stepB")
.tasklet(new PrintTextTasklet("stepB")).build();
}
#Bean
public Step stepC() {
return stepBuilderFactory.get("stepC")
.tasklet(new PrintTextTasklet("stepC")).build();
}
#Bean
public Step stepD() {
return stepBuilderFactory.get("stepD")
.tasklet(new PrintTextTasklet("stepD")).build();
}
#SuppressWarnings("resource")
public static void main(String[] args) {
// create spring application context
final ApplicationContext appContext = new AnnotationConfigApplicationContext(
JobConfig.class);
// get the job config bean (i.e this bean)
final JobConfig jobConfig = appContext.getBean(JobConfig.class);
// get the job launcher
JobLauncher launcher = jobConfig.jobLauncher();
try {
// launch the job
JobExecution execution = launcher.run(jobConfig.job(), new JobParameters());
System.out.println(execution.getJobInstance().toString());
} catch (JobExecutionAlreadyRunningException e) {
e.printStackTrace();
} catch (JobRestartException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JobInstanceAlreadyCompleteException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JobParametersInvalidException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
StepA: is a dummy job which fails i.e it throws some exception
public class RandomFailTasket extends PrintTextTasklet {
public RandomFailTasket(String text) {
super(text);
}
public RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
throws Exception {
if (Math.random() < 0.5){
throw new Exception("fail");
}
return RepeatStatus.FINISHED;
}
}
StepB, StepC, StepD are also dummy tasklets:
public class PrintTextTasklet implements Tasklet {
private final String text;
public PrintTextTasklet(String text){
this.text = text;
}
public RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
throws Exception {
System.out.println(text);
return RepeatStatus.FINISHED;
}
}
need to have a look at the xml structure that you are using.
Try using Step listener - and then in the after step method you can check the Step status and then you can implement your logic to call the next step or not

Resources