I am trying to pass a hashmap from one step to another step and use the map to create query and execute in next step. I am getting datasource must not be null while doing same.
Below is my code where I am trying to retrieve value and run query. I would have not retrieved and dynamically passed it yet. But I will be replacing this query dynamically.
#Autowired
DataSource dataSource;
#Override
public void afterPropertiesSet() throws Exception{
JobExecution jobExecution = stepExecution.getJobExecution();
ExecutionContext jobContext = jobExecution.getExecutionContext();
#SuppressWarnings("unchecked")
List<HashMap<String,String>> mapList = (List<HashMap<String, String>>) jobContext.get("mapList");
System.out.println("size of map received:::::::"+ mapList.size());
setSql("select count(*) as countValue from table where id=578");
setRowMapper(new dbMapper());
setDataSource(dataSource);
super.afterPropertiesSet();
}
#BeforeStep
public void saveStepExecution(final StepExecution stepExecution) {
this.stepExecution = stepExecution;
}
Where am I going wrong?
This should probably be a comment, but I don't have enough reputation to add one yet. Does the class that this sample is from already have a setter for dataSource? If so, you need to change setDataSource(dataSource); to super.setDataSource(dataSource);.
Related
Below is my simple step builder....I want the jobexecution context in all phases to retrieve job parameters such as file name, file path, other parameters...
Basically i want to register CSV errors in csv file along with incoming csv record data. Also, want to append the error records in the same CSV in all phases for the failed records....So if some error got occurred, i want to continue the processing and register all the errors to the csv file...some error would happen in reading the record, some error would happen in inserting the records into database.
SimpleStepBuilder simpleStepBuilder = steps.get("employeeJob")
.allowStartIfComplete(true)
.<Employee, Employee>chunk(10)
.reader(employeeFlatFileItemReader)
.faultTolerant()
.skipPolicy(new EmployeeSkipPolicy())
.listener(new EmployeeSkipListener())
.listener(itemProcessListenerSupport())
.processor(employeeProcessor);
#Slf4j
#JobScope
public class EmployeeSkipPolicy implements SkipPolicy {
#Value("#{jobExecution.executionContext}")
private ExecutionContext executionContext;
private JobExecution jobExecution;
#BeforeStep
public void initWriter(StepExecution stepExecution) {
jobExecution = stepExecution.getJobExecution();
}
#Override
public boolean shouldSkip(Throwable inThrowable, int inSkipCount) {
executionContext is null
jobExecution is null
}
Please help me
Thanks
I have to set the line of a record in a csv file within the record object when processing it before passing it to the writer .
What i want to achieve is to declare a global variable lets say line = 1 and each time an itemProcessor runs it should assign its value to the current item , and increment its value by 1 line++.
How can i share a variable over multiple itemProcessor runs ?
You can leverage the ExecutionContext to save the line variable of either the StepExecution if you only need the ExecutionContext in the single Step or JobExecution if you need the ExecutionContext in other Steps of the Job.
Inject either execution in your ItemProcessor:
#Value("#{stepExecution.jobExecution}") JobExecution jobExecution
or
#Value("#{stepExecution}") StepExecution stepExecution
Taking JobExecution as an example:
#StepScope
#Bean
public MyItemProcessor myItemProcessor(#Value("#{stepExecution.jobExecution}") JobExecution jobExecution) {
return new MyItemProcessor(jobExecution);
}
#Bean
public Step myStep() {
return stepBuilderFactory.get("myStep")
...
.processor(myItemProcessor(null))
...
.build()
}
public class MyItemProcessor implements ItemProcessor<MyItem, MyItem> {
private ExecutionContext executionContext;
public MyItemProcessor() {
this.executionContext = jobExecution.getExecutionContext();
}
#Override
public MyItem process(MyItem item) throws Exception {
// get the line from previous item processors, if exists, otherwise start with 0
int line = executionContext.getInt("myLineKey", 0);
item.setLine(line++);
// save the line for other item processors
item.put("myLineKey", line);
return item;
}
}
I created a simple Boot/Spring Batch 3.0.8.RELEASE job. I created a simple class that implements JobParametersIncrementer to go to the database, look up how many days the query should look for and puts those into the JobParameters object.
I need that value in my JdbcCursorItemReader, as it is selecting data based upon one of the looked up JobParameters, but I cannot figure this out via Java annotations. XML examples plenty, not so much for Java.
Below is my BatchConfiguration class that runs job.
`
#Autowired
SendJobParms jobParms; // this guy queries DB and puts data into JobParameters
#Bean
public Job job(#Qualifier("step1") Step step1, #Qualifier("step2") Step step2) {
return jobs.get("DW_Send").incrementer(jobParms).start(step1).next(step2).build();
}
#Bean
protected Step step2(ItemReader<McsendRequest> reader,
ItemWriter<McsendRequest> writer) {
return steps.get("step2")
.<McsendRequest, McsendRequest> chunk(5000)
.reader(reader)
.writer(writer)
.build();
}
#Bean
public JdbcCursorItemReader reader() {
JdbcCursorItemReader<McsendRequest> itemReader = new JdbcCursorItemReader<McsendRequest>();
itemReader.setDataSource(dataSource);
// want to get access to JobParameter here so I can pull values out for my sql query.
itemReader.setSql("select xxxx where rownum <= JobParameter.getCount()");
itemReader.setRowMapper(new McsendRequestMapper());
return itemReader;
}
`
Change reader definition as follows (example for parameter of type Long and name paramCount):
#Bean
#StepScope
public JdbcCursorItemReader reader(#Value("#{jobParameters[paramCount]}") Long paramCount) {
JdbcCursorItemReader<McsendRequest> itemReader = new JdbcCursorItemReader<McsendRequest>();
itemReader.setDataSource(dataSource);
itemReader.setSql("select xxxx where rownum <= ?");
ListPreparedStatementSetter listPreparedStatementSetter = new ListPreparedStatementSetter();
listPreparedStatementSetter.setParameters(Arrays.asList(paramCount));
itemReader.setPreparedStatementSetter(listPreparedStatementSetter);
itemReader.setRowMapper(new McsendRequestMapper());
return itemReader;
}
How can I stop a job in spring batch ? I tried to use this method using the code below:
public class jobListener implements JobExecutionListener{
#Override
public void beforeJob(JobExecution jobExecution) {
jobExecution.setExitStatus(ExitStatus.STOPPED);
}
#Override
public void afterJob(JobExecution jobExecution) {
// TODO Auto-generated method stub
}
}
I tried also COMPLETED,FAILED but this method doesn't work and the job continues to execute. Any solution?
You can use JobOperator along with JobExplorer to stop a job from outside the job (see https://docs.spring.io/spring-batch/reference/html/configureJob.html#JobOperator). The method is stop(long executionId) You would have to use JobExplorer to find the correct executionId to stop.
Also from within a job flow config you can configure a job to stop after a steps execution based on exit status (see https://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html#stopElement).
I assume you want to stop a job by a given name.
Here is the code.
String jobName = jobExecution.getJobInstance().getJobName(); // in most cases
DataSource dataSource = ... //#Autowire it in your code
JobOperator jobOperator = ... //#Autowire it in your code
JobExplorerFactoryBean factory = new JobExplorerFactoryBean();
factory.setDataSource(dataSource);
factory.setJdbcOperations(new JdbcTemplate(dataSource));
JobExplorer jobExplorer = factory.getObject();
Set<JobExecution> jobExecutions = jobExplorer.findRunningJobExecutions(jobName);
jobExecutions.forEach(jobExecution -> jobOperator.stop(jobExecution.getId()));
I have a spring batch JdbcCursorItemReader. It is defined as #JobScope. See method signature below.
#Bean
#JobScope
public JdbcCursorItemReader<MasterList> queryStagingDbReader(
#Value("#{jobParameters['" + JobParamConstants.PARAM_FROM_DATE + "']}") Date jobFromDate,
#Value("#{jobParameters['" + JobParamConstants.PARAM_TO_DATE + "']}") Date jobToDate) {
This JdbcCursorItemReader is part of step 2 in my job.
In step 1 of my job I have only a tasklet. Inside this tasklet I am building a list of dates which I would like my JdbcCursorItemReader in step 2 to know about.
My initial thought was to add my list of dates to the stepExecutionContext in my tasklet like so.
#Bean
#JobScope
public Tasklet createJobDatesTasklet(
#Value("#{jobParameters['" + JobParamConstants.PARAM_FROM_DATE + "']}") Date jobFromDate,
#Value("#{jobParameters['" + JobParamConstants.PARAM_TO_DATE + "']}") Date jobToDate) {
return new Tasklet() {
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
LocalDate start = jobFromDate.toInstant().atZone(ZoneId.systemDefault()).toLocalDate();
LocalDate end = jobToDate.toInstant().atZone(ZoneId.systemDefault()).toLocalDate();
List<LocalDate> jobDates = new ArrayList<>();
while (!start.isAfter(end)) {
jobDates.add(start);
start = start.plusDays(1);
}
//ADDING TO CONTEXT HERE
chunkContext.getStepContext().getStepExecution().getExecutionContext().put("jobDates", jobDates);
return RepeatStatus.FINISHED;
}
};
}
And then to grab the list of dates from my JdbcCursorItemReader. But when I try to grab the step execution context inside my JdbcCursorItemReader it tells me that it cannot wire it in. And I think that is because my bean is #JobScope.
What can I do to grab my list of dates from the StepExecutionContext or alternatively can I do anything else to make this work for me?
thanks in advance
You cannot write it to the step context, rather write it to the Job Context, so that all the steps of the Job have the access to the data as below.
StepContext stepContext = chunkContext.getStepContext();
StepExecution stepExecution = stepContext.getStepExecution();
JobExecution jobExecution = stepExecution.getJobExecution();
ExecutionContext jobContext = jobExecution.getExecutionContext();
jobContext.put("jobDates", jobDates);
The below blog speaks about it
http://techie-mixture.blogspot.com/2016/07/passing-values-between-spring-batch.html