I have spring batch job configured in my JAVA application, the application runs in a cluster. Hence the same job gets executed twice, which I don't want.
So I want to configure a step within the job which will check if CREATE_DATE is present for that day in BATCH_JOB_EXECUTION table and will continue or fail over.
How can this be configured within a spring batch step ?
Use a JobExecutionDecider.
From Javadoc:
Interface allowing for programmatic access to the decision on what the
status of a flow should be. For example, if some condition that's
stored in the database indicates that the job should stop for a manual
check, a decider implementation could check that value to determine
the status of the flow
Related
In my spring boot project, I need to upload data as .csv.
Later when the job executes, if there is some data mismatch the job will fail.
When the job executes it will update some of the tables. So with a "dry run" without making any data change I want to check if the uploaded data makes any issue.
To avoid that job failure, how can I ensure the data is OK and the job won't fail just after uploading the file?
I am running Spring boot application with 2 instances.Here i am going to use scheduler to run my application.For avoiding Scheduler not to run in two instances at same time using schlock .but schlock i have to mention for atleastfor or atmostfor .My problem is i dont want to release the lock based on time because since using batch application with rest call dont know when my scheduler get complete process.Kindly provide any suggestion running my scheduler with one instance at time without time constraint.
ShedLock requires lockAtMostFor for cases when the node dies. You can try KeepAliveLockProvider to automatically extend the lock.
I have one #Scheduler job which runs on multiple servers in a clustered environment. However I want to restrict the job to run in only in one server and other servers should not run the same job once any other server has started it .
I have explored Spring Batch has lock mechanism using some Database table , but looking for any a solution only in spring task:scheduler.
I had the same problem and the solution what I implemented was a the Lock mechanism with Hazelcast and to made it easy to use I also added a proper annotation and a bit of spring AOP for that. So with this trick I was able to enforce a single schedule over the cluster done with a single annotation.
Spring Batch has this nice functionality that it would not run the job with same job arguments twice.
You can use this feature so that when a spring batch job kicks start in another server it does not run.
Usually people pass a timestamp as argument so it will by pass this logic, which you can change it.
Background:
I created a Stop Job which finds running jobs with the specified name as this:
jobExplorer.findRunningJobExecutions("job_A")
and then, for each execution of job_A it calls:
jobOperator.stop(execution.getId());
Issue
when i call the above stop() method; even though it eventually accomplishes what i want, but it still throws an exception:
WARN o.s.b.c.l.s.SimpleJobOperator [main] Cannot find Job object
org.springframework.batch.core.launch.NoSuchJobException: No job configuration with the name [job_A] was registered
at org.springframework.batch.core.configuration.support.MapJobRegistry.getJob(MapJobRegistry.java:66) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobOperator.stop(SimpleJobOperator.java:403) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
The Cause
This happens when the stop() method tries to located job_A in the JobRegistry.
So even though job_A was found in the "JobRepository" because the repository looks in the database, it was not found in the "JobRegistry" which is a local cache of job beans created within its runtime environment, since job_A is running withing a different runtime instance it was not registered and threw and error.
Concern
Even though job 'A' stops i am still concerned what i have missed because of the exception.
I have searched this issue and found only general answers on how to stop a job, however i did not find anyone explaining how to stop a running job of another runtime.
Any answers would be greatly appreciated.
The JobOperator isn't intended to orchestrate distributed batch environments like you're attempting to do. You really have two options:
Use the JobRepository directly - The part that causes the job to stop successfully in the remote JVM is that the JobRepository is updated and the running job in the other JVM knows to check that periodically. Instead of using the JobOperator to accomplish this, just use the JobRepository directly to make the update.
Use a more robust orchestration tool like Spring Cloud Data Flow - This kind of orchestration (deploying, starting, stopping, etc) for jobs (via Spring Cloud Task) is what Spring Cloud Data Flow is for.
You can read more about Spring Cloud Data Flow at the website: https://cloud.spring.io/spring-cloud-dataflow/
In addition to what Michael mentioned you can solve this by adding some interface to the application allowing you to pass commands to start or stop your job. Something like a webservice. Exposing an end-point to stop it. Now the catch in this case is handling in clustered system may be a bit tricky.
Part of my job is to poll DB for a specific result status and only after that I can continue with next job's steps.
Is it recommended to stop the job's process while doing polling in one of the steps(tasklet I guess) ?
Polling the DB for a specific result sounds like a situation where you need a scheduler.
Spring-batch assumes the scheduling of the job is done from outside it's scope.
You can use #Scheduled spring annotation if you want to keep all inside spring configuration or use an external tool like this.
If you have more complex situations, have a look at Spring Batch Integration.