Is there a way to limit the lifetime of a running spring-batch job to e.g. 23 hours?
We start a batch job daily by a cron job and he job takes about 9 hours. It happened under some circumstances that the DB connection was so slow that the job took over 60 hours to complete. The problem is that the next job instance gets started by the cronjob the next day - and then anotherone the day after - and anotherone...
If this job is not finished within e.g. 23 hours, I want to terminate it and return an error. Is there a way to do that out-of-the-box with spring-batch?
Using a StepListener you can stop a job by calling StepExecution#setTerminateOnly.
stepBuilderFactory
...
.writer(writer)
.listener(timeoutListener)
...
.build()
And the TimeoutListener could look like this
#Component
public class TimeoutListener implements StepListener {
private StepExecution stepExecution;
#BeforeStep
public void beforeStep(StepExecution stepExecution) {
this.stepExecution = stepExecution;
}
#BeforeRead
public void beforeRead() {
if (jobShouldStop()) {
stepExecution.setTerminateOnly();
}
}
private boolean jobShouldStop() {
// ...
}
}
This will gracefully stop the job, without forcefully terminate any running steps.
Spring Batch specifically avoids the issue of job orchestration which this falls into. That being said, you could add a listener to your job that checks for other instances running and calls stop on them before beginning that one. Not knowing what each job does, I'm not sure how effective that would be, but it should be a start.
If you write your own job class to launch the process you can make your class implement StatefulJob interface, which prevents concurrent launches of the same job. Apart from that you can write your own monitoring and stop the job programatically after some period, but it will require some custom coding, I dont know if there is anything build-in for such use case.
Related
I'm using Quarkus. My Quartz jobs are scheduled to run every 10 seconds:
return TriggerBuilder.newTrigger()
.withIdentity("my-job")
.startNow()
.withSchedule(
SimpleScheduleBuilder.simpleSchedule()
.withIntervalInSeconds(10)
.repeatForever()
).build();
This works fine but jobs keep triggering every 10 seconds irrespective of whether or not the last one finishes. I need the next job to start only if there are no jobs currently running job. How do I accomplish this?
Add #DisallowConcurrentExecution on you Job class.
as example :
#DisallowConcurrentExecution
public class MyScheduledJob implements Job {
//execution method
}
There is an issue in running Multiple Cron in Spring. I have created multi module maven project in which there are two separate cron running but at some point of time they coincide and it becomes very tedious to debug from the log.
Is there any way of having a separate log or some way when one cron is running then another should not start, I mean at one point of time only one cron should run.
#Scheduled(cron="0 0 */2 * * *")
public void startAnalysis() {
logger.info("Inside of Analysis scheduler");
// Doing Something
}
#Scheduled(cron="0 0 */6 * * *")
public void startAnalysis() {
logger.info("Inside of Analysis1 scheduler");
// Doing Something
}
Above are the two crons that I am running. Currently, I am using sl4j for logging purpose.
You can create two instances of the logger, each with a different name and configure them to log to different files in your logging framework.
Spring by default uses a single threaded Executor. so no two
#Scheduled tasks will be execute simultaneously. even if there are two
#Scheduled methods in completely unrelated classes will not overlap
and that is because there is only a single thread to execute tasks.
the only possiblity for task to be executed at the same time is to have multi thread executors which defined as:
#EnableScheduling
#Configuration
public class MyConfiguration implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar
scheduledTaskRegistrar) {
scheduledTaskRegistrar.setScheduler(taskExecutor());
}
#Bean(destroyMethod="shutdown")
public Executor taskExecutor() {
return Executors.newScheduledThreadPool(5);
}
}
If you have such custom configuration that defines a thread pool for schedule tasks, you should remove it.
Please use Log4J programmatic config, you can separate different log file.
https://www.codejava.net/coding/how-to-configure-log4j-as-logging-mechanism-in-java?showall=&start=2#ProgrammaticConfiguration
I've a method scheduled to run periodically with Spring Scheduler, it's been working fine and stopped working today with no error. What could be the potential cause ? Is there any alternative way to schedule task periodically using Spring Scheduler that ensures that the method will be executed no matter what?
#Scheduled(cron="0 0/1 * * * ?")
public void executePollingFlows(){
if(applicationConfig.isScheduleEnabled()) {
for (long flowId : applicationConfig.getPollingFlowIds()) {
flowService.executeFlow(flowId);
}
logger.info("Finished executing all polling flows at {}", new Date());
}
}
You may have got Out of Memory exception if the job could not finish its tasks but you try to run it again and again. If it is a Out of Memory exception you may try to create a ThreadPool and check it in every run. If there is no enough space in the ThreadPool you can skip the task for this turn.
There is alternative way to use #Scheduled periodically. You may change your #Scheduled annotation with this:
#Scheduled(fixedRate=1000)
It will still be running in every second and if necessary you can add initialDelay to it:
#Scheduled(initialDelay=1000, fixedRate=1000)
You can find more details about fixedRate, initialDelay and fixedDelay here:
https://docs.spring.io/spring/docs/current/spring-framework-reference/html/scheduling.html
This is the first time I will work with Spring batch, and I got a task to do where I have to run a planned task every night at 00:00, which will truncate a table in the database, and re-fill it.
I don't know what is the best way to do this, I read a tutorial about Scheduling Tasks where I created a scheduled function and in this function I'll write the code which will execute my SQL scripts, as following :
#Component
public class ScheduledTasks {
#Scheduled(fixedRate = 5000)
public void reportCurrentTime(){
//My code goes here
}
}
But I don't know is this the best way to do it, since I found in some tutorials that they are working with JobLauncher and something called job-report.xml.
Any advices how to get this done ?
Use
#Scheduled(cron="0 0 0 * * *")
Look at https://docs.spring.io/spring/docs/current/spring-framework-reference/html/scheduling.html
Don't forget to add #EnableScheduling to your configuration.
I'm looking for an open source/free task scheduler for Windows 7 (development machine) that will allow me to schedule tasks (HTTP requests to a web service) to run every x seconds.
I've tried a couple of Cron clones and windows own Task Scheduler but neither seem to allow tasks to run at intervals less than 60 seconds. Am I missing something? I don't want to have to go and write any custom scripts either if possible.
It is possible to create multiple triggers for one scheduled task. If you create 59 identical triggers with an offset of 1 second to each other, and schedule the task itself to run every minute, you end up the scheduled task to run every second.
You could create those 59 triggers manually using the GUI. However, a much quicker way to create so many triggers is to create a task with one or two triggers, export it to a text file, duplicate the according lines, change the start offsets accordingly, and then re-import the file.
I was actually able to achieve this.
Update: Seems I over complicated it.
In the trigger, where it says "Repeat task every:" you can actually TYPE into the drop-down "1 minute" (It wont let you type the time in seconds)
I did this on a Windows 7 machine.
Also, I clearly did not read the question well enough, as the asker seems to already have been able to get the time down to 1 minute. However, I'll leave this answer here, as it will explain for future readers exactly how to get the time down to one minute.
It does seem as though you cannot get it to run at an interval of less than one minute.
I set up a task with a trigger set to Daily to recur every 1 day.
I check the "Repeat task every:" box. Setting it to 5 Minutes for a duration of 1 day
This makes the task go forever, every 5 minutes.
I then exported the task. It exports to a .xml file.
Under Task > Triggers > CalendarTrigger > Repeition there is the following tag: <Interval>PT5M</Interval> I changed it from PT5M to PT1M. I re-imported the task.
The task now runs every 1 minute.
I have not fully tested this, and I have not tried with less than one minute, but it might be possible by putting PT30S or something for 30 seconds. I'll try it out and report back. Update: You cannot do this, you get an error when importing the task. It's not possible to set this time to less than 1 minute.
The whole trigger looks like this for me:
<Triggers>
<CalendarTrigger>
<Repetition>
<Interval>PT1M</Interval>
<Duration>P1D</Duration>
<StopAtDurationEnd>false</StopAtDurationEnd>
</Repetition>
<StartBoundary>2013-11-07T17:04:51.6062297</StartBoundary>
<Enabled>true</Enabled>
<ScheduleByDay>
<DaysInterval>1</DaysInterval>
</ScheduleByDay>
</CalendarTrigger>
</Triggers>
I've googled this to death, so as far as I can see the answer is, there are none. There are plenty of commercial solutions, but no open source/free programs.
I ended up writing a very simple periodic HTTP GET scheduler in java using quartz scheduler. It may be useful to other so posting a link to the source on guthub https://github.com/bjordan/simple_java_periodic_HTTP_scheduler
Short explanation:
Main program starts a service process that will stay active in memory and will periodically activate a job – do something.
Create a class that extends System.ServiceProcess.ServiceBase class
Implement at least methods OnStart and OnStop
Start and use Quartz.NET scheduler in OnStart to run tasks periodically
Here is my template C# solution for a Windows service and a Linux demon in .NET/Mono https://github.com/mchudinov/ServiceDemon
And a short blogpost about it
class Program
{
public static void Main(string[] args)
{
ServiceBase[] ServicesToRun;
ServicesToRun = new ServiceBase[] { new ServiceDemon.Service() };
ServiceBase.Run(ServicesToRun);
}
}
public class Service : ServiceBase
{
static IScheduler Scheduler { get; set; }
protected override void OnStart(string[] args)
{
StartScheduler();
StartMyJob();
}
protected override void OnStop()
{
Scheduler.Shutdown();
}
void StartScheduler()
{
ISchedulerFactory schedFact = new StdSchedulerFactory();
Scheduler = schedFact.GetScheduler();
Scheduler.Start();
}
void StartMyJob()
{
var seconds = Int16.Parse(ConfigurationManager.AppSettings["MyJobSeconds"]);
IJobDetail job = JobBuilder.Create<Jobs.MyJob>()
.WithIdentity("MyJob", "group1")
.UsingJobData("Param1", "Hello MyJob!")
.Build();
ITrigger trigger = TriggerBuilder.Create()
.WithIdentity("MyJobTrigger", "group1")
.StartNow()
.WithSimpleSchedule(x => x.WithIntervalInSeconds(seconds).RepeatForever())
.Build();
Scheduler.ScheduleJob(job, trigger);
}
}
public class MyJob : IJob
{
public void Execute(IJobExecutionContext context)
{
JobDataMap dataMap = context.JobDetail.JobDataMap;
log.Info(dataMap["Param1"]);
}
}