Call a method on a specific dates using ThreadPoolTaskExecutor - spring

I have a method that I wish to run once using Spring and it needs to run on a given java.util.Date (or LocalDateTime alternatively). I am planning to persist all of the dates that the method should execute to a data source. It should run asynchronously.
One way is to check the DB every day for a date and execute the method if the date has passed and hasn't been executed. Is there a better way?
I know that Spring offers a ThreadPoolTaskScheduler and a ThreadPoolTaskExecutor. I am looking at ScheduledFuture schedule(Runnable task, Date startTime) from the TaskScheduler interface. Would I need to create a Runnable Spring managed bean just to call my method? Or is there a simpler annotation that would do this? An example would really help.
(Looked here too.)

By externalizing the scheduled date (to a database), the typical scheduling practices (i.e. cron based, or fixed scheduling) no longer apply. Given a target Date, you can schedule the task accurately as follows:
Date now = new Date();
Date next = ... get next date from external source ...
long delay = next.getTime() - now.getTime();
scheduler.schedule(Runnable task, delay, TimeUnit.MILLISECONDS);
What remains is to create an efficient approach to dispatching each new task.
The following has a TaskDispatcher thread, which schedules each Task based on the next java.util.Date (which you read from a database). There is no need to check daily; this approach is flexible enough to work with any scheduling scenario stored in the database.
To follow is working code to illustrate the approach.
The example Task used; in this case just sleeps for a fixed time. When the task is complete, the TaskDispatcher is signaled through a CountDownLatch.
public class Task implements Runnable {
private final CountDownLatch completion;
public Task(CountDownLatch completion) {
this.completion = completion;
}
#Override
public void run() {
System.out.println("Doing task");
try {
Thread.sleep(60*1000); // Simulate the job taking 60 seconds
} catch (InterruptedException e) {
e.printStackTrace();
}
completion.countDown(); // Signal that the job is complete
}
}
The dispatcher is responsible for reading the database for the next scheduled Date, launching a ScheduledFuture runnable, and waiting for the task to complete.
public class TaskDispatcher implements Runnable {
private static final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
private boolean isInterrupted = false;
#Override
public void run() {
while (!isInterrupted) {
Date now = new Date();
System.out.println("Reading database for next date");
Date next = ... read next data from database ...
//Date next = new Date(); // Used as test
//next.setTime(now.getTime()+10*1000); // Used as test
long delay = next.getTime() - now.getTime();
System.out.println("Scheduling next task with delay="+delay);
CountDownLatch latch = new CountDownLatch(1);
ScheduledFuture<?> countdown = scheduler.schedule(new Task(latch), delay, TimeUnit.MILLISECONDS);
try {
System.out.println("Blocking until the current job has completed");
latch.await();
} catch (InterruptedException e) {
System.out.println("Thread has been requested to stop");
isInterrupted = true;
}
if (!isInterrupted)
System.out.println("Job has completed normally");
}
scheduler.shutdown();
}
}
The TaskDispatcher was started as follows (using Spring Boot) - start the thread as you normally do with Spring:
#Bean
public TaskExecutor taskExecutor() {
return new SimpleAsyncTaskExecutor(); // Or use another one of your liking
}
#Bean
public CommandLineRunner schedulingRunner(TaskExecutor executor) {
return new CommandLineRunner() {
public void run(String... args) throws Exception {
executor.execute(new TaskDispatcher());
}
};
}
Let me know if this approach will work for your use case.

Take a look at the #Scheduled annotation. It may accomplish what you're looking for.
#Scheduled(cron="*/5 * * * * MON-FRI")
public void scheduledDateWork() {
Date date = new Date(); //or use DAO call to look up date in database
executeLogic(date);
}
Cron Expression Examples from another answer:
"0 0 * * * *" = the top of every hour of every day.
"*/10 * * * * *" = every ten seconds.
"0 0 8-10 * * *" = 8, 9 and 10 o'clock of every day.
"0 0/30 8-10 * * *" = 8:00, 8:30, 9:00, 9:30 and 10 o'clock every day.
"0 0 9-17 * * MON-FRI" = on the hour nine-to-five weekdays
"0 0 0 25 12 ?" = every Christmas Day at midnight

Related

JobRunr Spring Boot: how to get notified if a recurring job - including retries - has failed

I'm using jobrunr 5.1.4 in my spring boot application. I have a simple service declaring a recurring job which allows for some retries. A single failing job run is not that relevant for me. Instead, I'm interested in getting notified after all jobs, i.e. the initial job including all the retries, have failed.
I thought JobRunr's JobServerFilter would be a good idea. But the onProcessed() method never gets triggered in case of an exception only in case of a successful job run. And the ApplyStateFilter gets triggered on every state change. Far too often for my requirement. Leaving me clueless, if a change to a FAILED state was the last in a series of jobs belonging together (initial job + allowed retried jobs).
A simple example would look like this:
#Service
public class JobScheduler {
#Job(name = "My Recurring Job", retries = 2, jobFilters = ExceptionFilter.class)
#Recurring(id = "my-recurring-job", cron = "*/10 * * * *")
public void recurringJob() {
throw new RuntimeException("foo");
}
}
A basic implementation of my JobFilter looks like this:
#Component
public class ExceptionFilter implements JobServerFilter, ApplyStateFilter {
#Override
public void onProcessing(Job job) {
log.info("onProcessing: {}", job.getJobName());
log.info(job.getJobState().getName().name());
}
#Override
public void onProcessed(Job job) {
log.info("onProcessed: {}", job.getJobName());
log.info(job.getJobState().getName().name());
}
#Override
public void onStateApplied(Job job, JobState jobState1, JobState jobState2) {
log.info("onStateApplied: {}", job.getJobName());
log.info("jobState1: {}", jobState1.getName().name());
log.info("jobState2: {}", jobState2.getName().name());
}
}
Is this use case even possible with JobRunr? Or does anyone have an idea how to solve this issue in a different way?
Thank you very much in advance for you support.
I think you're on the right track with onStateApplied from ApplyStateFilter.
You can use the following approach:
#Override
public void onStateApplied(Job job, JobState oldState, JobState newState) {
if (isFailed(newState) && maxAmountOfRetriesReached(job)) {
// your logic here
}
}
OnProcessed is not triggered as your job was not processed (due to the failure).

Spring Boot Scheduler fixedDelay and cron

I'm running a spring boot scheduled process that takes 5-10 seconds to complete. After it completes, 60 seconds elapse before the process begins again (Note that I'm not using fixedRate):
#Scheduled(fixedDelay=60_000)
Now, I want to limit it to run every minute Mon-Fri 9am to 5pm. I can accomplish this with
#Scheduled(cron="0 * 9-16 ? * MON-FRI")
Problem here is that this acts similar to fixedRate - the process triggers EVERY 60 seconds regardless of the amount of time it took to complete the previous run...
Any way to to combine the two techniques?
it worked for me like this
I created a bean that returns a specific task executor and allowed only 1 thread.
#Configuration
#EnableAsync
public class AsyncConfig implements AsyncConfigurer {
#Bean(name = "movProcTPTE")
public TaskExecutor movProcessualThreadPoolTaskExecutor() {
ThreadPoolTaskExecutor exec = new ThreadPoolTaskExecutor();
exec.setMaxPoolSize(1);
exec.initialize();
return exec;
}
}
In my service, I injected my task executor and wrapped my logic with it, so even though my schedule runs every minute, my logic will only run when the task executor is free.
#Service
#EnableScheduling
public class ScheduledService {
#Autowired
private ReportDataService reportDataService;
#Autowired
private AsyncService async;
#Autowired
#Qualifier("movProcTPTE")
private TaskExecutor movProcTaskExecutor;
#Scheduled(cron = "0 * * 1-7 * SAT,SUN")
public void agendamentoImportacaoMovProcessual(){
movProcTaskExecutor.execute(
() -> {
reportDataService.importDataFromSaj();
}
);
}
}
try this:
#Schedules({
#Scheduled(fixedRate = 1000),
#Scheduled(cron = "* * * * * *")
})
You can try this one:
#Scheduled(cron="1 9-16 * * MON-FRI")
Also you can try write correct on this site https://crontab.guru/
You can pass fixed delay (and any other number of optional parameters) to the annotation, like so:
#Scheduled(cron="0 * 9-16 ? * MON-FRI", fixedDelay=60_000)
From the documentation: https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/annotation/Scheduled.html

Chronicle Roll Files Daily

I am trying to implement Chronicle Queue into our system and had a question regarding rolling of files daily but at a specific time as per the local time zone of the process. I read few write-ups regarding how to specify roll cycle but as per documentation the epoch time works as per midnight UTC. What would I need to do to configure a roll cycle let's say every day at 5PM local time zone of the process running? Any suggestions?
public class TestRollCycle {
public class TestClass implements TestEvent {
private int counter = 1;
#Override
public void setOrGetEvent(String event) {
System.out.println("Counter Read Value: " + counter);
counter++;
}
}
public interface TestEvent {
void setOrGetEvent(String event);
}
#Test
public void testRollProducer() {
int insertCount = 1;
String pathOfFile = "rollPath";
// Epoch is 5:15PM EDT
SingleChronicleQueue producerQueue = SingleChronicleQueueBuilder.binary(pathOfFile).epoch(32940000).build();
ExcerptAppender myAppender = producerQueue.acquireAppender();
TestEvent eventWriter = myAppender.methodWriter(TestEvent.class);
while (true) {
String testString = "Insert String";
eventWriter.setOrGetEvent(testString);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Counter Write Value: " + insertCount);
insertCount++;
}
}
#Test
public void testRollConsumer() throws InterruptedException {
String pathOfFile = "rollPath";
// Epoch is 5:15PM EDT
SingleChronicleQueue producerQueue = SingleChronicleQueueBuilder.binary(pathOfFile).epoch(32940000).build();
TestClass myClass = new TestClass();
ExcerptTailer trailer = producerQueue.createTailer();
MethodReader methodReader = trailer.methodReader(myClass);
while (true) {
if (!methodReader.readOne()) {
Thread.sleep(1000);
} else {
//System.out.println(trailer.index());
}
}
}
}
This is a feature we added to Chronicle Queue Enterprise. I suggest you contact sales#chronicle.software if you are will to pay for it.
I think there's a problem in your test - the epoch of 32940000 supplied to the queue builder is 9hr 15m from midnight, so 9:15AM UTC or 5:15AM EDT. It should be another 12 hours later for the roll-time to be 5:15PM.
I've added a test that documents the current behaviour for your use-case, and it passes as expected. Can you double-check that you're supplying the correct epoch offset, and perhaps implement a StoreFileListener in order to capture/log any roll events.
The roll will not actually occur until an event is written to the queue that is after the roll-time boundary. So an idle queue that is not being written-to will not roll without input events.
The test is on github:
https://github.com/OpenHFT/Chronicle-Queue/blob/master/src/test/java/net/openhft/chronicle/queue/impl/single/QueueEpochTest.java

Scheduled tasks not running on time in Spring

I use #Schedule to have the system run tasks at different time. But recently, I found that some of the tasks are being postponed. For example, there is one task which should run on 5:00 every morning, now runs sometimes in the noon or even later in the afternoon. Any ideas? Thanks.
Scheduler service:
#Service
public class DailyReminderTasks extends AbstractTask {
private final static Logger logger = LoggerFactory
.getLogger(DailyReminderTasks.class);
private SendAuditReminderNotificationSerivce sendAuditReminderNotificationSerivce;
#Scheduled(cron = "0 0 5 * * ?")
// run at 5:00 am every day
public void sendAuditReponseReminderLetter() {
try {
sendAuditReminderNotificationSerivce.sendAuditReponseReminder();
} catch (Exception ex) {
logger.error("failed to send reminder: ", ex);
}
}
public SendAuditReminderNotificationSerivce getSendAuditReminderNotificationSerivce() {
return sendAuditReminderNotificationSerivce;
}
#Autowired(required = true)
public void setSendAuditReminderNotificationSerivce(
SendAuditReminderNotificationSerivce sendAuditReminderNotificationSerivce) {
this.sendAuditReminderNotificationSerivce = sendAuditReminderNotificationSerivce;
}
With the information available here, the most likely explanation is that the background thread pool handling task scheduling does not keep up and thus between the time the task is put on the queue and the time it is executed, many hours can pass.

Spring Batch skip job execution if job with same parameters (but different date/time) in progress

Some of jobs run longer then our scheduler interval. It is pretty safe to skip such jobs. How can I detect if job with same name and parameters (except running time) is currently running?
Quartz run Batch jobs as:
import org.springframework.batch.core.JobParametersBuilder;
#Override
protected void executeInternal(org.quartz.JobExecutionContext context) {
Map<String, Object> jobDataMap = context.getMergedJobDataMap();
String jobName = (String) jobDataMap.get("job.name");
JobParametersBuilder builder = new JobParametersBuilder();
builder.addString("company", jobDataMap.get("company"));
builder.addDate("run.date", new Date());
try {
jobLauncher.run(jobLocator.getJob(jobName), jobParametersBuilder.toJobParameters());
}
catch (JobExecutionException e) {
log.error("Could not execute job.", e);
}
}
I need to find if any Executions for specific "job.name"/"company" (regardless "run.date") is running.
I can do that with plain SQL running against BATCH_ tables.
Check supposed to run in Tasklet so JobRepository can be autowired. Is it possible to find out only with JobRepository or other Spring Batch beans?
You can exclude the "run.date" from identifying.
Use public JobParameter(Date parameter, boolean identifying)
From docs of JobParameter.
The identifying flag is used to indicate if the parameter is to be
used as part of the identification of a job instance.
So just add the parameter without builder.addDate("run.date", new Date()); and run the job.
My solution based on JobExplorer.findRunningJobExecutions() (to find latest unfinished job):
/**
* If there are jobs with same parameters, that started later then some number we can run new job again.
*
* {#link JobExplorer#findRunningJobExecutions(String)} selects jobs with BATCH_JOB_EXECUTION.END_TIME is null.
*
* #return null if there is no jobs with same parameters, otherwise the latest running.
*/
public JobExecution getLatestRunningJob(JobExplorer jobExplorer, String jobName, String agency, String branch) {
long now = new Date().getTime();
long minDiff = Long.MAX_VALUE;
JobExecution latestExecution = null;
Set<JobExecution> runningJobs = jobExplorer.findRunningJobExecutions(jobName);
for (JobExecution execution : Optional.ofNullable(runningJobs).orElse(Collections.emptySet())) {
JobParameters params = execution.getJobParameters();
// log.warn("agencyName {}", params.getString("agencyName"));
// log.warn("branchName {}", params.getString("branchName"));
if ( ! agency.equals(params.getString("agencyName")))
continue;
if ( ! branch.equals(params.getString("branchName")))
continue;
// log.warn("create {}", execution.getCreateTime());
// log.warn("start {}", execution.getStartTime());
long diff = now - execution.getCreateTime().getTime();
if (diff < 0) {
log.warn("Impossible, new job executed before old! Old JobExecution id: {}", execution.getJobId());
continue;
}
if (diff < minDiff) {
minDiff = diff;
latestExecution = execution;
}
}
return latestExecution;
}
and decision maker:
/**
* #param execution not null
* #param limitMinutes how long job should be runnning to ignore
*/
public boolean canIgnoreLateJob(JobExecution execution, long limitMinutes) {
long now = new Date().getTime();
long diffMinutes = TimeUnit.MINUTES.convert(now - execution.getCreateTime().getTime(), TimeUnit.MILLISECONDS);
if (diffMinutes < limitMinutes) {
log.warn("Recent JobExecution {} is still not finished, can't run new job, diff (minute): {}",
execution.getJobId(), diffMinutes);
return false;
} else {
log.warn("Very old JobExecution {} is still not finished, can run new job, diff (minute): {}",
execution.getJobId(), diffMinutes);
return true;
}
}
Drawback of this solution is that JobExplorer resolve every field in JobExecution (including all Steps). So performance is not perfect. But solution is reliable.

Resources