I use #Schedule to have the system run tasks at different time. But recently, I found that some of the tasks are being postponed. For example, there is one task which should run on 5:00 every morning, now runs sometimes in the noon or even later in the afternoon. Any ideas? Thanks.
Scheduler service:
#Service
public class DailyReminderTasks extends AbstractTask {
private final static Logger logger = LoggerFactory
.getLogger(DailyReminderTasks.class);
private SendAuditReminderNotificationSerivce sendAuditReminderNotificationSerivce;
#Scheduled(cron = "0 0 5 * * ?")
// run at 5:00 am every day
public void sendAuditReponseReminderLetter() {
try {
sendAuditReminderNotificationSerivce.sendAuditReponseReminder();
} catch (Exception ex) {
logger.error("failed to send reminder: ", ex);
}
}
public SendAuditReminderNotificationSerivce getSendAuditReminderNotificationSerivce() {
return sendAuditReminderNotificationSerivce;
}
#Autowired(required = true)
public void setSendAuditReminderNotificationSerivce(
SendAuditReminderNotificationSerivce sendAuditReminderNotificationSerivce) {
this.sendAuditReminderNotificationSerivce = sendAuditReminderNotificationSerivce;
}
With the information available here, the most likely explanation is that the background thread pool handling task scheduling does not keep up and thus between the time the task is put on the queue and the time it is executed, many hours can pass.
Related
I'm using jobrunr 5.1.4 in my spring boot application. I have a simple service declaring a recurring job which allows for some retries. A single failing job run is not that relevant for me. Instead, I'm interested in getting notified after all jobs, i.e. the initial job including all the retries, have failed.
I thought JobRunr's JobServerFilter would be a good idea. But the onProcessed() method never gets triggered in case of an exception only in case of a successful job run. And the ApplyStateFilter gets triggered on every state change. Far too often for my requirement. Leaving me clueless, if a change to a FAILED state was the last in a series of jobs belonging together (initial job + allowed retried jobs).
A simple example would look like this:
#Service
public class JobScheduler {
#Job(name = "My Recurring Job", retries = 2, jobFilters = ExceptionFilter.class)
#Recurring(id = "my-recurring-job", cron = "*/10 * * * *")
public void recurringJob() {
throw new RuntimeException("foo");
}
}
A basic implementation of my JobFilter looks like this:
#Component
public class ExceptionFilter implements JobServerFilter, ApplyStateFilter {
#Override
public void onProcessing(Job job) {
log.info("onProcessing: {}", job.getJobName());
log.info(job.getJobState().getName().name());
}
#Override
public void onProcessed(Job job) {
log.info("onProcessed: {}", job.getJobName());
log.info(job.getJobState().getName().name());
}
#Override
public void onStateApplied(Job job, JobState jobState1, JobState jobState2) {
log.info("onStateApplied: {}", job.getJobName());
log.info("jobState1: {}", jobState1.getName().name());
log.info("jobState2: {}", jobState2.getName().name());
}
}
Is this use case even possible with JobRunr? Or does anyone have an idea how to solve this issue in a different way?
Thank you very much in advance for you support.
I think you're on the right track with onStateApplied from ApplyStateFilter.
You can use the following approach:
#Override
public void onStateApplied(Job job, JobState oldState, JobState newState) {
if (isFailed(newState) && maxAmountOfRetriesReached(job)) {
// your logic here
}
}
OnProcessed is not triggered as your job was not processed (due to the failure).
I have a Java-Configuration with this annotaions:
#EnableAsync
#EnableCaching
#Configuration
#EnableScheduling
#ComponentScan("XXX")
public class AppConfig implements AsyncConfigurer {
...
#Override
public Executor getAsyncExecutor() {
SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor("Rm-");
executor.setConcurrencyLimit(50);
executor.setThreadPriority(Thread.MAX_PRIORITY);
return executor;
}
}
Then I have 3 Schedulers
/**
* Schedul on application start.
*/
#Transactional
#Scheduled(fixedDelay = Long.MAX_VALUE, initialDelay = 200)
public void initialReplicate() {
replicateFromTime(0l);
}
/**
* Every minute check for synchronization.
*/
#Transactional
#Scheduled(cron = "0 * * * * * ")
public void replicate() {
long minutesSinceTimeRecord = System.currentTimeMillis() / 1000 / 60;
replicateFromTime(minutesSinceTimeRecord);
}
#Transactional
#Scheduled(fixedDelay = 1000)
public void build() {
LOG.info("Check if somethings to build.");
}
What the replicate-from-time does:
It calculates if the day is over, if so, it replicate once per day (and once at appstart) the schema of a database.
For every table in the database one connection is created.
#Async
#Transactional
public Future<Void> readTable(TableModel table, String jdbcurl, Driver driver, Properties props, String schema,
Set<Throwable> errors, String catalogName) {
Connection connect = connectionService.connect(jdbcurl, errors, driver, props);
final DatabaseMetaData metadata;
try {
metadata = connect.getMetaData();
} catch (SQLException e) {
errors.add(e);
try {
connect.close();
} catch (SQLException e2) {
errors.add(e2);
}
return new AsyncResult<Void>(null);
}
...
There are 804 tables of any kind in the database (views, tables, temporary-tables, ...). For every table one connection is created. Makes 804 threads.
The database is on the machin as the app. I test from localhost.
Problem:
On the remote-server, on appstart it replicates successfully. But no other scheduler are executed anymore except the public void build()-Scheduler. But only 48 times (see the limit of 50 threads in the configuration).
From local-server, on appstart it replicates successfully. All 3 schedulers are working as expected.
The tomcat on the server should run all schedulers, but it runs only one. The tomcat on my local machine run all schedulers.
Confusion
I expected to have more problems in a local environment, connected to a remote server than inside the remote-server itself.
Question
Any idea why the remote server have problems but my local machine not?
Technical details
Local is a apache-tomcat-8.5.27.
Remote is a apache-tomcat-8.5.15.
Database to replicate schema is a postgres.
I have a method that I wish to run once using Spring and it needs to run on a given java.util.Date (or LocalDateTime alternatively). I am planning to persist all of the dates that the method should execute to a data source. It should run asynchronously.
One way is to check the DB every day for a date and execute the method if the date has passed and hasn't been executed. Is there a better way?
I know that Spring offers a ThreadPoolTaskScheduler and a ThreadPoolTaskExecutor. I am looking at ScheduledFuture schedule(Runnable task, Date startTime) from the TaskScheduler interface. Would I need to create a Runnable Spring managed bean just to call my method? Or is there a simpler annotation that would do this? An example would really help.
(Looked here too.)
By externalizing the scheduled date (to a database), the typical scheduling practices (i.e. cron based, or fixed scheduling) no longer apply. Given a target Date, you can schedule the task accurately as follows:
Date now = new Date();
Date next = ... get next date from external source ...
long delay = next.getTime() - now.getTime();
scheduler.schedule(Runnable task, delay, TimeUnit.MILLISECONDS);
What remains is to create an efficient approach to dispatching each new task.
The following has a TaskDispatcher thread, which schedules each Task based on the next java.util.Date (which you read from a database). There is no need to check daily; this approach is flexible enough to work with any scheduling scenario stored in the database.
To follow is working code to illustrate the approach.
The example Task used; in this case just sleeps for a fixed time. When the task is complete, the TaskDispatcher is signaled through a CountDownLatch.
public class Task implements Runnable {
private final CountDownLatch completion;
public Task(CountDownLatch completion) {
this.completion = completion;
}
#Override
public void run() {
System.out.println("Doing task");
try {
Thread.sleep(60*1000); // Simulate the job taking 60 seconds
} catch (InterruptedException e) {
e.printStackTrace();
}
completion.countDown(); // Signal that the job is complete
}
}
The dispatcher is responsible for reading the database for the next scheduled Date, launching a ScheduledFuture runnable, and waiting for the task to complete.
public class TaskDispatcher implements Runnable {
private static final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
private boolean isInterrupted = false;
#Override
public void run() {
while (!isInterrupted) {
Date now = new Date();
System.out.println("Reading database for next date");
Date next = ... read next data from database ...
//Date next = new Date(); // Used as test
//next.setTime(now.getTime()+10*1000); // Used as test
long delay = next.getTime() - now.getTime();
System.out.println("Scheduling next task with delay="+delay);
CountDownLatch latch = new CountDownLatch(1);
ScheduledFuture<?> countdown = scheduler.schedule(new Task(latch), delay, TimeUnit.MILLISECONDS);
try {
System.out.println("Blocking until the current job has completed");
latch.await();
} catch (InterruptedException e) {
System.out.println("Thread has been requested to stop");
isInterrupted = true;
}
if (!isInterrupted)
System.out.println("Job has completed normally");
}
scheduler.shutdown();
}
}
The TaskDispatcher was started as follows (using Spring Boot) - start the thread as you normally do with Spring:
#Bean
public TaskExecutor taskExecutor() {
return new SimpleAsyncTaskExecutor(); // Or use another one of your liking
}
#Bean
public CommandLineRunner schedulingRunner(TaskExecutor executor) {
return new CommandLineRunner() {
public void run(String... args) throws Exception {
executor.execute(new TaskDispatcher());
}
};
}
Let me know if this approach will work for your use case.
Take a look at the #Scheduled annotation. It may accomplish what you're looking for.
#Scheduled(cron="*/5 * * * * MON-FRI")
public void scheduledDateWork() {
Date date = new Date(); //or use DAO call to look up date in database
executeLogic(date);
}
Cron Expression Examples from another answer:
"0 0 * * * *" = the top of every hour of every day.
"*/10 * * * * *" = every ten seconds.
"0 0 8-10 * * *" = 8, 9 and 10 o'clock of every day.
"0 0/30 8-10 * * *" = 8:00, 8:30, 9:00, 9:30 and 10 o'clock every day.
"0 0 9-17 * * MON-FRI" = on the hour nine-to-five weekdays
"0 0 0 25 12 ?" = every Christmas Day at midnight
I have a Spring Batch application in which I want to schedule jobs calls.
The scheduling interval is not known at build so I can't just annotate my Job with #Scheduled.This led me to use a ThreadPoolTaskScheduler.
The thing is the method schedule takes a Runnable as a parameter. Is it possible to schedule jobs this way ?
I can call the job directly from the following service but I can't schedule it.
Here is my the background of my problem, I tried to make it simple :
#Service
public class ScheduledProcessor{
private final ThreadPoolTaskScheduler threadPoolTaskScheduler;
private Application application;
#Autowired
public ScheduledProcessor(ThreadPoolTaskScheduler threadPoolTaskScheduler, Application application){
this.threadPoolTaskScheduler = threadPoolTaskScheduler;
this.application = application;
scheduledTasks = new ArrayList();
Trigger trigger = new CronTrigger("0/6 * * * * *");
//Here I am trying to schedule my job.
//The following line is wrong because a Job can't be cast to a Runnable but I wanted to show the intended behaviour.
threadPoolTaskScheduler.schedule((Runnable) application.importUserjob, trigger);
System.out.println("Job launch !");
}
And here is the JobBuilderFactory :
#Bean
public Job importUserJob(JobBuilderFactory jobs, Step s1, Step s2) {
return jobs.get("importUserJob")
.incrementer(new RunIdIncrementer())
.flow(s1)
.end()
.build();
}
I understand (well, I'm even not sure about that) that I can't directly cast a Job to a Runnable but is it possible to convert it in any way ? Or can you give me some advice about what to use for being able to dynamically schedule spring batch jobs ?
In case that changes something, I also need to be able to restart / skip my steps, as I currently can with the threadPoolTaskScheduler.
Thank you in advance for any help or hint you could provide.
I finally got how to do it !
I created a class which implements Runnable (and for convenience, extends Thread, which avoid the need to implement all of Runnable classes).
#Component
public class MyRunnableJob extends Thread implements Runnable{
private Job job;
private JobParameters jobParameters;
private final JobOperator jobOperator;
#Autowired
public MyRunnableJob(JobOperator jobOperator) {
this.jobOperator = jobOperator;
}
public void setJob(Job job){
this.job=job;
}
#Override
public void run(){
try {
String dateParam = new Date().toString();
this.jobParameters = new JobParametersBuilder().addString("date", dateParam).toJobParameters();
System.out.println("jobName : "+job.getName()+" at "+dateParam);
jobOperator.start(job.getName(), jobParameters.toString());
} catch (NoSuchJobException | JobInstanceAlreadyExistsException | JobParametersInvalidException ex) {
Logger.getLogger(MyRunnableJob.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
In my ScheduledProcessor class, I set a Job to myRunnable class and then pass it as a parameter of the schedule method.
public class SchedulingProcessor {
//Autowired fields :
private final JobLauncher jobLauncher;
private final Job importUserJob;
private final ThreadPoolTaskScheduler threadPoolTaskScheduler;
private final MyRunnableJob myRunnableJob;
//Other fields :
private List<ScheduledFuture> scheduledTasks;
#Autowired
public SchedulingProcessor(JobLauncher jobLauncher, Job importUserJob, ThreadPoolTaskScheduler threadPoolTaskScheduler, MyRunnableJob myRunnableJob) throws Exception {
this.jobLauncher=jobLauncher;
this.importUserJob=importUserJob;
this.threadPoolTaskScheduler=threadPoolTaskScheduler;
this.myRunnableJob=myRunnableJob;
Trigger trigger = new CronTrigger("0/6 * * * * *");
myRunnableJob.setJob(this.importUserJob);
scheduledTasks = new ArrayList();
scheduledTasks.add(this.threadPoolTaskScheduler.schedule((Runnable) myRunnableJob, trigger));
}
}
The scheduledTasks list is just to keep a control over the tasks I just scheduled.
This trick enabled me to dynamically (thanks to ThreadPoolTaskScheduler) schedule Spring Batch Jobs encapsulated in a class implementing Runnable. I wish it can help someone in the same case as mine.
Heres another way to trigger them from your spring context.
Job emailJob = (Job) applicationContext.getBean("xyzJob");
JobLauncher launcher = (JobLauncher) applicationContext
.getBean("jobLauncher");
launcher.run(emailJob, new JobParameters());
I have an app that uses the "task:scheduler" and "task:scheduled-tasks" elements (the latter containing "task:scheduled" elements). This is all working fine.
I'm trying to write some code that introspects the "application configuration" to get a short summary of some important information, like what tasks are scheduled and what their schedule is.
I already have a class that has a bunch of "#Autowired" instance variables so I can iterate through all of this. It was easy enough to add a "List" to get all of the TaskScheduler objects. I only have two of these, and I have a different set of scheduled tasks in each of them.
What I can't see in those TaskScheduler objects (they are actually ThreadPoolTaskScheduler objects) is anything that looks like a list of scheduled tasks, so I'm guessing the list of scheduled tasks is recorded somewhere else.
What objects can I use to introspect the set of scheduled tasks, and which thread pool they are in?
This functionality will be available in Spring 4.2
https://jira.spring.io/browse/SPR-12748 (Disclaimer: I reported this issue and contributed code towards its solution).
// Warning there may be more than one ScheduledTaskRegistrar in your
// application context. If this is the case you can autowire a list of
// ScheduledTaskRegistrar instead.
#Autowired
private ScheduledTaskRegistrar scheduledTaskRegistrar;
public List<Task> getScheduledTasks() {
List<Task> result = new ArrayList<Task>();
result.addAll(this.scheduledTaskRegistrar.getTriggerTaskList());
result.addAll(this.scheduledTaskRegistrar.getCronTaskList());
result.addAll(this.scheduledTaskRegistrar.getFixedRateTaskList());
result.addAll(this.scheduledTaskRegistrar.getFixedDelayTaskList());
return result;
}
// You can this inspect the tasks,
// so for example a cron task can be inspected like this:
public List<CronTask> getScheduledCronTasks() {
List<CronTask> cronTaskList = this.scheduledTaskRegistrar.getCronTaskList();
for (CronTask cronTask : cronTaskList) {
System.out.println(cronTask.getExpression);
}
return cronTaskList;
}
If you are using a ScheduledMethodRunnable defined in XML:
<task:scheduled method="run" cron="0 0 12 * * ?" ref="MyObject" />
You can access the underlying target object:
ScheduledMethodRunnable scheduledMethodRunnable = (ScheduledMethodRunnable) task.getRunnable();
TargetClass target = (TargetClass) scheduledMethodRunnable.getTarget();
I have a snippet for pre spring 4.2 since it is still sitting at release candidate level.
The scheduledFuture interface is implemented by every runnable element in the BlockingQueue.
Map<String, ThreadPoolTaskScheduler> schedulers = applicationContext
.getBeansOfType(ThreadPoolTaskScheduler.class);
for (ThreadPoolTaskScheduler scheduler : schedulers.values()) {
ScheduledExecutorService exec = scheduler.getScheduledExecutor();
ScheduledThreadPoolExecutor poolExec = scheduler
.getScheduledThreadPoolExecutor();
BlockingQueue<Runnable> queue = poolExec.getQueue();
Iterator<Runnable> iter = queue.iterator();
while (iter.hasNext()) {
ScheduledFuture<?> future = (ScheduledFuture<?>) iter.next();
future.getDelay(TimeUnit.MINUTES);
Runnable job = iter.next();
logger.debug(MessageFormat.format(":: Task Class is {0}", JobDiscoverer.findRealTask(job)));
}
Heres a reflective way to get information about which job class is in the pool as threadPoolNamePrefix didn't return a distinct name for me:
public class JobDiscoverer {
private final static Field syncInFutureTask;
private final static Field callableInFutureTask;
private static final Class<? extends Callable> adapterClass;
private static final Field runnableInAdapter;
private static Field reschedulingRunnable;
private static Field targetScheduledMethod;
static {
try {
reschedulingRunnable = Class
.forName(
"org.springframework.scheduling.support.DelegatingErrorHandlingRunnable")
.getDeclaredField("delegate");
reschedulingRunnable.setAccessible(true);
targetScheduledMethod = Class
.forName(
"org.springframework.scheduling.support.ScheduledMethodRunnable")
.getDeclaredField("target");
targetScheduledMethod.setAccessible(true);
callableInFutureTask = Class.forName(
"java.util.concurrent.FutureTask$Sync").getDeclaredField(
"callable");
callableInFutureTask.setAccessible(true);
syncInFutureTask = FutureTask.class.getDeclaredField("sync");
syncInFutureTask.setAccessible(true);
adapterClass = Executors.callable(new Runnable() {
public void run() {
}
}).getClass();
runnableInAdapter = adapterClass.getDeclaredField("task");
runnableInAdapter.setAccessible(true);
} catch (NoSuchFieldException e) {
throw new ExceptionInInitializerError(e);
} catch (SecurityException e) {
throw new PiaRuntimeException(e);
} catch (ClassNotFoundException e) {
throw new PiaRuntimeException(e);
}
}
public static Object findRealTask(Runnable task) {
if (task instanceof FutureTask) {
try {
Object syncAble = syncInFutureTask.get(task);
Object callable = callableInFutureTask.get(syncAble);
if (adapterClass.isInstance(callable)) {
Object reschedulable = runnableInAdapter.get(callable);
Object targetable = reschedulingRunnable.get(reschedulable);
return targetScheduledMethod.get(targetable);
} else {
return callable;
}
} catch (IllegalAccessException e) {
throw new IllegalStateException(e);
}
}
throw new ClassCastException("Not a FutureTask");
}
With #Scheduled based configuration the approach from Tobias M’s answer does not work out-of-the-box.
Instead of autowiring a ScheduledTaskRegistrar instance (which is not available for annotation based configuration), you can instead autowire a ScheduledTaskHolder which only has a getScheduledTasks() method.
Background:
The ScheduledAnnotationBeanPostProcessor used to manage #Scheduled tasks has an internal ScheduledTaskRegistrar that’s not available as a bean. It does implement ScheduledTaskHolder, though.
Every Spring XML element has a corresponding BeanDefinitionReader. For <task:scheduled-tasks>, that's ScheduledTasksBeanDefinitionParser.
This uses a BeanDefinitionBuilder to create a BeanDefinition for a bean of type ContextLifecycleScheduledTaskRegistrar. Your scheduled tasks will be stored in that bean.
The tasks will be executing in either a default TaskScheduler or one you provided.
I've given you the class names so you can look at the source code yourself if you need more fine grained details.