We have small cluster.In which spring xd distributed runtime architecture is component for ETL.
We have scheduled batch job in it using cron.But when job fails or interrupted,We are not getting notified over named channel or Email and is that possible to trigger batch jobs by sending messages to named channels?
Currently,Running on Following Environment:
Spring XD Distributed Runtime - 1.2.1
Hadoop Distribution - PHD3.0
Any help on it would be much appreciated.
You may need to write your own step that handles the notification for you. So in your batch flow you will configure some steps that only get executed when your other steps fail.
Something like...
<job id="job">
<step id="stepA">
<next on="*" to="stepB" />
<next on="FAILED" to="NotifyErrorEmail" />
</step>
<step id="stepB".. />
<step id="NotifyErrorEmail" />
</job>
You can read more in Spring Batch Configuring Steps
Related
We are migrating XML-style Spring Batch jobs to JavaConfig, and found out that it seems not possible to use a decider() as the first step in the job flow logic, right after start(). We need to put a dummy step in order to invoke a decider.
However, in XML, this configuration works perfectly fine:
<batch:decision decider="exitsDecider" id="exitsInstance">
<batch:next on="CONTINUE" to="jobStartStep" />
<batch:end on="COMPLETED" exit-code="COMPLETED" />
<batch:fail on="FAILED" exit-code="FAILED" />
</batch:decision>
<batch:step id="jobStartStep" next="validateStep">
<batch:tasklet ref="jobStartTasklet" />
</batch:step>
Don't know whether this is an undocumented feature, or just some side of corner case. What would be the equivalent in Java Config?
I have chained a set of Spring batch jobs in an order.
<batch:job id="rootJob">
<batch:step id="rootJob.step1">
<batch:job ref="externalJob1">
<batch:next on="COMPLETE" to="rootJob.step2">
</batch:step>
<batch:split id="rootJob.step2">
<batch:flow>
<batch:step id="splitStep1">
<batch:job ref="externalJob2">
</batch:step>
</batch:flow>
<batch:flow>
<batch:step id="splitStep2">
<batch:job ref="externalJob3">
</batch:step>
</batch:flow>
<batch:next on="COMPLETE" to="rootJob.step3">
</batch:split>
<batch:step id="rootJob.step3">
<batch:job ref="externalJob4">
</batch:step>
</batch:job>
The expectation of job flow execution.
1. On Completion of rootJob.step1 execute rootJob.step2.
2. Execute splitJob1 and splitJob2 in parallel.
3. On Completion of rootJob.step2 execute rootJob.step3
But when deployed and triggered in Jboss. The flow is not executing as expected. The steps are getting triggered in single stretch. The execution is not waiting for previous step to complete and getting launched instantly.
I suspect the TaskExecutor. In standalone we do not specify any task executor (defaults to SyncTaskExecutor) and the job flow works fine. But when deployed in Jboss we use SimpleAsyncTaskExecutor, as using SyncTaskExecutor doesnt even trigger job in Jboss.
What am i missing here or Am i doing something wrong here.? Please suggest.
Resolved the issue.
I had provided the job-launcher="jobLauncher" property like below. So separate threads were launched and the jobs were triggering in parallel.
<batch:job ref="externalJob1" job-launcher="jobLauncher">
Now i have removed the joblauncher reference from all the jobs and the jobs are triggering as designed.
I have some legacy programs in C that work based on the input file and the result go to the output file. Both files are specified in the program arguments. So the call looks like the following:
prj.exe a.dat a.out
Based on Artem Bilan suggestion I created the project with the following Spring Configuration file. It works in terms of invoking executable! However, I still have the problem with the outbound channel. First, it contains nothing and I am getting the error "unsupported Message payload type". Second, what is more important, I need to process the output file a.out by a Java program. What is the best way to organize this workflow? Is it possible to substitute the useless in this case inbound-channel-adapter to something useful?
<int-file:inbound-channel-adapter id="producer-file-adapter"
channel="inboundChannel" directory="file:/Users/anarinsky/springint/chem"
prevent-duplicates="true">
<int:poller fixed-rate="5000" />
</int-file:inbound-channel-adapter>
<int:channel id="inboundChannel" />
<int:channel id="outboundChannel" />
<int:service-activator input-channel="inboundChannel" output-channel="outboundChannel"
expression="new ProcessBuilder('/Users/anarinsky/springint/chem/prj', '/Users/anarinsky/springint/chem/a.dat', '/Users/anarinsky/springint/chem/a.out').start()">
</int:service-activator>
<int-file:outbound-channel-adapter
channel="outboundChannel" id="consumer-file-adapter"
directory="file:/Users/anarinsky/springint/chem"/>
Something like this:
<int:service-activator expression="new ProcessBuilder('prj.exe', 'a.dat', 'a.out').start()"/>
?
I am using Spring Batch 3.0.3. I have a job named JBABC, it inlcudes 3 steps. Is it possible, can i run user defined steps while invoking JBABC. Some roles might need to start/restart job from ABC1, whereas some other roles need a start/restart from ABC2 and other group might need start/restart from ABC3. So basically i am looking for to run a job based on custom/user defined step parameter.
<job id="JBABC" xmlns="http://www.springframework.org/schema/batch">
<step id="ABC1" next="ABC2" >
<tasklet ref="abc1Tasklet"></tasklet>
</step>
<step id="ABC2" next ="ABC3">
<tasklet ref="abc2Tasklet"></tasklet>
</step>
<step id="ABC3">
<tasklet ref="abc3Tasklet"></tasklet>
</step>
</job>
Have a look at http://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html Chapter 5.3.2.
You can implement a first step, whicht returns a specific state depending on a parameter. With the ""-tag, you will be able to define which step should be executed as next.
Depending on how many options you have, you could also define different jobs and use a specific launcher to start the one you have to.
I'm using Spring Batch for the first time. I tried some examples and read through documentation. But I have still questions:
Can I skip one phase in chunk oriented processing? For example: I fetch data from database, process it and determine, that I need more, can I skip write phase and execute next step's read phase? Should I use Tasklet instead?
How to implement a conditional flow?
Thank you very much,
Florian
Skip chunks simply by throwing an exception that has been declared as "skippable exception". You can do it as follows:
<step id="step1">
<tasklet>
<chunk reader="reader" writer="writer"
commit-interval="10" skip-limit="10">
<skippable-exception-classes>
<include class="com.myapp.batch.MyException"/>
</skippable-exception-classes>
</chunk>
</tasklet>
</step>
Conditional flow can easily be implemented deciding on the ExitStatus of a step-execution:
<job id="job">
<step id="step1" parent="s1">
<next on="*" to="stepB" />
<next on="FAILED" to="stepC" />
</step>
<step id="stepB" parent="s2" next="stepC" />
<step id="stepC" parent="s3" />
</job>
Read the documentation to gain deeper knowledge on these topics: http://docs.spring.io/spring-batch/reference/html/configureStep.html