Spring 4.3 with Spring Batch 3.0.8.
I want to have a reference to the job execution id in the processor, so I can put it inside the output object and write it out along with the data to db. Here is my setup below.
I have added the blueReportJobExecutionListener, which gives me the JobExecution ID that I need.... but how do I send that over to my blueReportItemProcessor ?! That's the object that needs that value.
<bean id="blueReportJobExecutionListener" class="com.cloud.cost.listener.BlueReportJobExecutionListener" scope="prototype" />
<bean id="blueReportJobListener" class="com.cloud.cost.listener.BlueReportJobListener" scope="prototype" />
<bean id="blueReportStepListener" class="com.cloud.cost.listener.BlueReportStepListener" scope="prototype" />
<batch:job id="blueReportJob">
<batch:step id="blueReportStep">
<batch:tasklet>
<batch:chunk reader="blueReportCSVFileItemReader" processor="blueReportItemProcessor" writer="mysqlItemWriter"
commit-interval="2">
</batch:chunk>
</batch:tasklet>
<batch:listeners>
<batch:listener ref="blueReportStepListener"/>
</batch:listeners>
</batch:step>
<batch:listeners>
<batch:listener ref="blueReportJobListener"/>
<batch:listener ref="**blueReportJobExecutionListener**"/>
</batch:listeners>
</batch:job>
You can the get the value from Job Execution by simply using #Value annotation.
#Value("#{jobExecutionContext['JOB_ID']}")
Where JOB_ID is the key you have used in the listener to add the job id.
Make sure your processor scope is defined as step otherwise this value will not be autowired.
Related
I am defining my MultiResourceItemReader on this way:
<bean id="multiDataItemReader" class="org.springframework.batch.item.file.MultiResourceItemReader" scope="step">
<property name="resources" value="#{jobExecutionContext['filesResource']}"/>
<property name="delegate" ref="dataItemReader"/>
</bean>
How you can see I want read from the jobExecutionContext the "filesResource" value.
Note: I changed some names to keep the "code privacy". This is executing, Is somebody wants more info please tell me.
I am saving this value in my first step and I am using the reader in the second step, Should I have access to it?
I am saving it in the final lines from my step1 tasklet:
ExecutionContext jobContext = context.getStepContext().getStepExecution().getJobExecution().getExecutionContext();
jobContext.put("filesResource", resourceString);
<batch:job id="myJob">
<batch:step id="step1" next="step2">
<batch:tasklet ref="moveFilesFromTasklet" />
</batch:step>
<batch:step id="step2">
<tasklet>
<chunk commit-interval="500"
reader="multiDataItemReader"
processor="dataItemProcessor"
writer="dataItemWriter" />
</tasklet>
</batch:step>
</batch:job>
I am not really sure what I am forgetting to get the value. The error that I am getting is:
20190714 19:49:08.120 WARN org.springframework.batch.item.file.MultiResourceItemReader [[ # ]] - No resources to read. Set strict=true if this should be an error condition.
I see nothing wrong with your config. The value of resourceString should be an array of org.springframework.core.io.Resource as this is the parameter type of the resources attribute of MultiResourceItemReader.
You can pass an array or a list of String with the absolute path to each resource and it should work. Here is a quick example:
class MyTasklet implements Tasklet {
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) {
List<String> resources = Arrays.asList(
"/full/path/to/resource1",
"/full/path/to/resource2");
chunkContext.getStepContext().getStepExecution().getJobExecution().getExecutionContext()
.put("filesResource", resources);
return RepeatStatus.FINISHED;
}
}
My batch Job takes input of 15 different files and the first step in my job is a Validate Step which will validate the header and total no of records, next step is the chunk based tasklet.
Now in validation, I have different if-else cases in my tasklet to handle different validations for the files and the cases are based on the file name and extension which are been passed as Job parameters. In this case I need to know whether the tasklet reference for my validation step can be decided upon runtime. i.e I'll have bean references for validators for 15 files and based upon the file extension in the job parameters my tasklet ref should be taken at runtime.
<batch:job id="downloadFile">
<batch:step id="validator" >
<batch:tasklet ref="InteropValidator"/>
<batch:next on="FAILED" to="AckStep"/>
<batch:next on="COMPLETED" to="loadFiles"/>
</batch:step>
<batch:step id="loadFiles" next="AckStep">
<partition step="slave" partitioner="partitionerFactory">
<handler grid-size="1" task-executor="taskExecutor" />
</partition>
</batch:step>
<batch:step id="AckStep" >
<batch:tasklet ref="Acksteptasklet"/>
<batch:fail on="FAILED"/>
</batch:step>
</batch:job>
In InteropValidator java, I have implemented the tasklet interface and written code snippet as below:
if("ICTX".equals(FilenameUtils.getExtension(filename)))
{
if(fileValidated && detailValid && agencyValid )
{
cc.getStepContext().getStepExecution().getJobExecution().getExecutionContext().put("STATUSCODE","01");
sc.setExitStatus(ExitStatus.COMPLETED);
}
}
if("SML".equals(FilenameUtils.getExtension(filename)))
{
//Validations for SML File
}
firstly, thanks for attention,in my spring batch project defined many jobs , for example:
<batch:job id="helloWorldJob1" job-repository="jobRepository">
<batch:step id="step1" >
<batch:tasklet>
<batch:chunk reader="itemReader1" writer="itemWriter1"
processor="itemProcessor1">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<batch:job id="helloWorldJob2" job-repository="jobRepository">
<batch:step id="step1" >
<batch:tasklet>
<batch:chunk reader="itemReader2" writer="itemWriter2"
processor="itemProcessor2">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<batch:job id="helloWorldJob3" job-repository="jobRepository">
<batch:step id="step1" >
<batch:tasklet>
<batch:chunk reader="itemReader3" writer="itemWriter3"
processor="itemProcessor3">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
.
.
.
.
how to use pure annotation ? is it the right approach?
Basically good start is Spring batch official documentation. Only thing here to note is that example has one job which runs when you do
mvn spring-boot:run. BatchConfiguration is example how pure java configuration can look like.
On our project we created main configuration like this:
#Configuration
#EnableBatchProcessing(modular = true)
public class JobContextConfig {
#Autowired
private JobRepository jobRepository;
#Bean
public ApplicationContextFactory helloWorldJob1Job() {
return new GenericApplicationContextFactory(HelloWorldJob1JobConfig.class);
}
#Bean
public ApplicationContextFactory helloWorldJob2Job() {
return new GenericApplicationContextFactory(HelloWorldJob2JobConfig.class);
}
#Bean
public ApplicationContextFactory helloWorldJob3Job() {
return new GenericApplicationContextFactory(HelloWorldJob3JobConfig.class);
}
}
And we have separate configuration in separate context for each of jobs. HelloWorldJob1JobConfig would hold everything that that job needs and class would look like BatchConfiguration from spring example. This will create everything you need except triggering so we created launchers for each job (we are launching some jobs over http, some with messaging and some manually so we needed actually to start jobs defined this way with JobLauncher).
Another good resource for integrating spring-batch with spring-boot with pure java configuration is spring batch boot web starter.
I am trying to configure a spring batch step without an item writer using below configuraion. However i get error saying that writer
element has neither a 'writer' attribute nor a element.
I went through the link spring batch : Tasklet without ItemWriter. But could not resolve issue.
Could any one tell me the specific changes to be made in the code snippet I mentioned
<batch:job id="helloWorldJob">
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="cvsFileItemReader"
commit-interval="10">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="cvsFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="resource" value="classpath:cvs/input/report.csv" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names" value="id,sales,qty,staffName,date" />
</bean>
</property>
<property name="fieldSetMapper">
<bean class="com.mkyong.ReportFieldSetMapper" />
<!-- if no data type conversion, use BeanWrapperFieldSetMapper to map by name
<bean
class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<property name="prototypeBeanName" value="report" />
</bean>
-->
</property>
</bean>
</property>
</bean>
For chunk-based step reader and writer are mandatory.
If you don't want a writer use a No-operation ItemWriter that does nothing.
EDIT:
A no-op implementation is an empty implementation of interface tha does...nothing!
Just let your class implements desiderable inteface(s) with empty methods.
No-op ItemWriter:
public class NoOpItemWriter implements ItemWriter {
void write(java.util.List<? extends T> items) throws java.lang.Exception {
// no-op
}
}
I hope you got answer but I want to explain it for other readers, When we use chunk then usually we declare reader, processor and writer. In chunk reader and writer are mandatory and processor is optional. In your case if you don't need writer then u need to make a class which implements ItemWriter. Override write method and keep it blank. Now create a bean of writer class and pass it as reference of writer.
<batch:step id="recordProcessingStep" >
<batch:tasklet>
<batch:chunk reader="fileReader" processor="recordProcessor"
writer="rocordWriter" commit-interval="1" />
</batch:tasklet>
</batch:step>
Your writer class will look like .
public class RecordWriter<T> implements ItemWriter<T> {
#Override
public void write(List<? extends T> items) throws Exception {
// TODO Auto-generated method stub
}
}
In maven repo you can find the framework "spring-batch-samples".
In this framework you will find this Writer :
org.springframework.batch.sample.support.DummyItemWriter
When i have Spring Batch Job Defined as the following:
<batch:job id="FailTask">
<batch:description>My Job Description</batch:description>
<batch:step id="FailTask-step0">
<batch:tasklet ref="sampleFailTask" />
<batch:listeners>
<batch:listener ref="sampleFailStepListener" />
</batch:listeners>
</batch:step>
</batch:job>
How could i get the Job Description in my java code ?
ok i found it:
AutowireCapableBeanFactory beanFactory = applicationContext.getAutowireCapableBeanFactory()
BeanDefinition beanDefinition = ((BeanDefinitionRegistry) beanFactory).getBeanDefinition(taskName);
String taskDescription = beanDefinition.getDescription();
now the question is how to get other properties than Description?
In JobParser there are lines:
Element description = DomUtils.getChildElementByTagName(element, "description");
if (description != null) {
builder.getBeanDefinition().setDescription(description.getTextContent());
}
So, description is set to bean defenition. You may get it from BeanDefinitionRegistry.