I am using Spring Batch 3.0.3. I have a job named JBABC, it inlcudes 3 steps. Is it possible, can i run user defined steps while invoking JBABC. Some roles might need to start/restart job from ABC1, whereas some other roles need a start/restart from ABC2 and other group might need start/restart from ABC3. So basically i am looking for to run a job based on custom/user defined step parameter.
<job id="JBABC" xmlns="http://www.springframework.org/schema/batch">
<step id="ABC1" next="ABC2" >
<tasklet ref="abc1Tasklet"></tasklet>
</step>
<step id="ABC2" next ="ABC3">
<tasklet ref="abc2Tasklet"></tasklet>
</step>
<step id="ABC3">
<tasklet ref="abc3Tasklet"></tasklet>
</step>
</job>
Have a look at http://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html Chapter 5.3.2.
You can implement a first step, whicht returns a specific state depending on a parameter. With the ""-tag, you will be able to define which step should be executed as next.
Depending on how many options you have, you could also define different jobs and use a specific launcher to start the one you have to.
Related
We are migrating XML-style Spring Batch jobs to JavaConfig, and found out that it seems not possible to use a decider() as the first step in the job flow logic, right after start(). We need to put a dummy step in order to invoke a decider.
However, in XML, this configuration works perfectly fine:
<batch:decision decider="exitsDecider" id="exitsInstance">
<batch:next on="CONTINUE" to="jobStartStep" />
<batch:end on="COMPLETED" exit-code="COMPLETED" />
<batch:fail on="FAILED" exit-code="FAILED" />
</batch:decision>
<batch:step id="jobStartStep" next="validateStep">
<batch:tasklet ref="jobStartTasklet" />
</batch:step>
Don't know whether this is an undocumented feature, or just some side of corner case. What would be the equivalent in Java Config?
I have chained a set of Spring batch jobs in an order.
<batch:job id="rootJob">
<batch:step id="rootJob.step1">
<batch:job ref="externalJob1">
<batch:next on="COMPLETE" to="rootJob.step2">
</batch:step>
<batch:split id="rootJob.step2">
<batch:flow>
<batch:step id="splitStep1">
<batch:job ref="externalJob2">
</batch:step>
</batch:flow>
<batch:flow>
<batch:step id="splitStep2">
<batch:job ref="externalJob3">
</batch:step>
</batch:flow>
<batch:next on="COMPLETE" to="rootJob.step3">
</batch:split>
<batch:step id="rootJob.step3">
<batch:job ref="externalJob4">
</batch:step>
</batch:job>
The expectation of job flow execution.
1. On Completion of rootJob.step1 execute rootJob.step2.
2. Execute splitJob1 and splitJob2 in parallel.
3. On Completion of rootJob.step2 execute rootJob.step3
But when deployed and triggered in Jboss. The flow is not executing as expected. The steps are getting triggered in single stretch. The execution is not waiting for previous step to complete and getting launched instantly.
I suspect the TaskExecutor. In standalone we do not specify any task executor (defaults to SyncTaskExecutor) and the job flow works fine. But when deployed in Jboss we use SimpleAsyncTaskExecutor, as using SyncTaskExecutor doesnt even trigger job in Jboss.
What am i missing here or Am i doing something wrong here.? Please suggest.
Resolved the issue.
I had provided the job-launcher="jobLauncher" property like below. So separate threads were launched and the jobs were triggering in parallel.
<batch:job ref="externalJob1" job-launcher="jobLauncher">
Now i have removed the joblauncher reference from all the jobs and the jobs are triggering as designed.
I have a simple batch process with a skip limit set. When the skip limit is surpassed the job fails and it never gets to step two. I would like the process to go to step 3 if the skip limit has passed.
<job id="jobA" incrementer="runIdIncrementer" >
<step id="step1" next="step2">
<tasklet>
<chunk commit-interval="10" reader="dReader" writer="dWriter" skip-limit="100">
<skippable-exception-classes>
<include class="java.lang.Exception"/>
</skippable-exception-classes>
</chunk>
<listeners>
<listener ref="skipListener"/>
</listeners>
</tasklet>
</step>
<step id="step2" next="step3">
<tasklet>
<chunk commit-interval="10" reader="sReader" writer="sWriter"/>
</tasklet>
</step>
<step id="step3">
<tasklet ref="cleanUpStep"/>
</step>
</job>
Is there a way to do this? I have tried setting "next" but an error is thrown stating cant have next attribute and a transition element.
Any help would be great.
You could add a StepExecutionListener to your step. The afterStep(StepExecution stepExecution) will be executed even if the step failed. In this method, you can get the exit status of the step and change it: stepExecution.setExitStatus(ExitStatus.COMPLETED).
You might want to check if the error comes from the skip-limit being exceeded, maybe stepExecution.getFailureExceptions() and search for SkipLimitExceededException (or something like that). You could also get the number of skiped item and compare it with your max (However, if it's an Error on the 100's skip, maybe you should do something else ...)
Note: Skipping a step after having too much exceptions doesn't sound like good design, but as long as you're aware of what you're doing ...
I have managed to fix it using "next". I was including next attribute while also having
next="step2" set in the step declaration. The fix was to remove this from here and then you can add in the next attribute.
<step id="step1">
<tasklet>
<chunk commit-interval="10" reader="dReader" writer="dWriter" skip-limit="100">
<skippable-exception-classes>
<include class="java.lang.Exception"/>
</skippable-exception-classes>
</chunk>
<listeners>
<listener ref="skipListener"/>
</listeners>
</tasklet>
<next on="*" to="step2" />
</step>
The above code will continue to step2 even if the skip limit has been reached. <step id="step1" next="step2"> would cause the job to fail if the limit was reached.
We have small cluster.In which spring xd distributed runtime architecture is component for ETL.
We have scheduled batch job in it using cron.But when job fails or interrupted,We are not getting notified over named channel or Email and is that possible to trigger batch jobs by sending messages to named channels?
Currently,Running on Following Environment:
Spring XD Distributed Runtime - 1.2.1
Hadoop Distribution - PHD3.0
Any help on it would be much appreciated.
You may need to write your own step that handles the notification for you. So in your batch flow you will configure some steps that only get executed when your other steps fail.
Something like...
<job id="job">
<step id="stepA">
<next on="*" to="stepB" />
<next on="FAILED" to="NotifyErrorEmail" />
</step>
<step id="stepB".. />
<step id="NotifyErrorEmail" />
</job>
You can read more in Spring Batch Configuring Steps
I'm using Spring Batch for the first time. I tried some examples and read through documentation. But I have still questions:
Can I skip one phase in chunk oriented processing? For example: I fetch data from database, process it and determine, that I need more, can I skip write phase and execute next step's read phase? Should I use Tasklet instead?
How to implement a conditional flow?
Thank you very much,
Florian
Skip chunks simply by throwing an exception that has been declared as "skippable exception". You can do it as follows:
<step id="step1">
<tasklet>
<chunk reader="reader" writer="writer"
commit-interval="10" skip-limit="10">
<skippable-exception-classes>
<include class="com.myapp.batch.MyException"/>
</skippable-exception-classes>
</chunk>
</tasklet>
</step>
Conditional flow can easily be implemented deciding on the ExitStatus of a step-execution:
<job id="job">
<step id="step1" parent="s1">
<next on="*" to="stepB" />
<next on="FAILED" to="stepC" />
</step>
<step id="stepB" parent="s2" next="stepC" />
<step id="stepC" parent="s3" />
</job>
Read the documentation to gain deeper knowledge on these topics: http://docs.spring.io/spring-batch/reference/html/configureStep.html