Reset state before each Spring scheduled (#Scheduled) run - spring

I have a Spring Boot Batch application that needs to run daily. It reads a daily file, does some processing on its data, and writes the processed data to a database. Along the way, the application holds some state such as the file to be read (stored in the FlatFileItemReader and JobParameters), the current date and time of the run, some file data for comparison between read items, etc.
One option for scheduling is to use Spring's #Scheduled such as:
#Scheduled(cron = "${schedule}")
public void runJob() throws Exception {
jobRunner.runJob(); //runs the batch job by calling jobLauncher.run(job, jobParameters);
}
The problem here is that the state is maintained between runs. So, I have to update the file to be read, the current date and time of the run, clear the cached file data, etc.
Another option is to run the application via a unix cron job. This will obviously meet the need to clear state between runs but I prefer to tie the job scheduling to the application instead of the OS (and prefer it to OS agnostic). Can the application state be reset between #Scheduled runs?

You could always move the code that performs your task (and more importantly, keeps your state) into a prototype-scoped bean. Then you can retrieve a fresh instance of that bean from the application context every time your scheduled method is run.
Example
I created a GitHub repository which contains a working example of what I'm talking about, but the gist of it is in these two classes:
ScheduledTask.java
Notice the #Scope annotation. It specifies that this component should not be a singleton. The randomNumber field represents the state that we want to reset with every invocation. "Reset" in this case means that a new random number is generated, just to show that it does change.
#Component
#Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
class ScheduledTask {
private double randomNumber = Math.random();
void execute() {
System.out.printf(
"Executing task from %s. Random number is %f%n",
this,
randomNumber
);
}
}
TaskScheduler.java
By autowiring in ApplicationContext, you can use it inside the scheduledTask method to retrieve a new instance of ScheduledTask.
#Component
public class TaskScheduler {
#Autowired
private ApplicationContext applicationContext;
#Scheduled(cron = "0/5 * * * * *")
public void scheduleTask() {
ScheduledTask task = applicationContext.getBean(ScheduledTask.class);
task.execute();
}
}
Output
When running the code, here's an example of what it looks like:
Executing task from com.thomaskasene.example.schedule.reset.ScheduledTask#329c8d3d. Random number is 0.007027
Executing task from com.thomaskasene.example.schedule.reset.ScheduledTask#3c5b751e. Random number is 0.145520
Executing task from com.thomaskasene.example.schedule.reset.ScheduledTask#3864e64d. Random number is 0.268644

Thomas' approach seems to be a reasonable solution, that's why I upvoted it. What is missing is how this can be applied in the case of a spring batch job. Therefore I adapted his example little bit:
#Component
public class JobCreatorComponent {
#Bean
#Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public Job createJob() {
// use the jobBuilderFactory to create your job as usual
return jobBuilderFactory.get() ...
}
}
your component with the launch method
#Component
public class ScheduledLauncher {
#Autowired
private ... jobRunner;
#Autwired
private JobCreatorComponent creator;
#Scheduled(cron = "${schedule}")
public void runJob() throws Exception {
// it would probably make sense to check the applicationContext and
// remove any existing job
creator.createJob(); // this should create a complete new instance of
// the Job
jobRunner.runJob(); //runs the batch job by calling jobLauncher.run(job, jobParameters);
}
I haven't tried out the code, but this is the approach I would try.
When constructing the job, it is important to ensure that all reader, processors and writers used in this job are complete new instances as well. This means, if they are not instantiated as pure java objects (not as spring beans) or as spring beans with scope "step" you must ensure that always a new instance is used.
Edited:
How to handle SingeltonBeans
Sometimes singleton beans cannot be prevented, in these cases there must be a way to "reset" them.
An simple approach would be to define an interface "ResetableBean" with a reset method that is implemented by such beans. Autowired can then be used to collect a list of all such beans.
#Component
public class ScheduledLauncher {
#Autowired
private List<ResetableBean> resetables;
...
#Scheduled(cron = "${schedule}")
public void runJob() throws Exception {
// reset all the singletons
resetables.forEach(bean -> bean.reset());
...

Related

How do I get JobRunr to detect my scheduled background job in a Spring controller/service?

I have been looking into using JobRunr for starting background jobs on my Spring MVC application, as I really like the simplicity of it, and the ease of integrating it into an IoC container.
I am trying to create a simple test scheduled job that writes a line of text to my configured logger every minute, but I'm struggling to figure out how to get the JobRunr background job server to detect it and queue it up. I am not using Spring Boot so I am just using the generic jobrunr Maven artifact rather than the "Spring Boot Starter". My setup is as follows:
pom.xml
<dependency>
<groupId>org.jobrunr</groupId>
<artifactId>jobrunr</artifactId>
<version>2.0.0</version>
</dependency>
ApplicationConfig.java
#Bean
public JobMapper jobMapper() {
return new JobMapper(new JacksonJsonMapper());
}
#Bean
#DependsOn("jobMapper")
public StorageProvider storageProvider(JobMapper jobMapper) {
InMemoryStorageProvider storageProvider = new InMemoryStorageProvider();
storageProvider.setJobMapper(jobMapper);
return storageProvider;
}
#Bean
#DependsOn("storageProvider")
public JobScheduler jobScheduler(StorageProvider storageProvider, ApplicationContext applicationContext) {
return JobRunr.configure().useStorageProvider(storageProvider)
.useJobActivator(applicationContext::getBean)
.useDefaultBackgroundJobServer()
.useDashboard()
.useJmxExtensions()
.initialize();
}
BackgroundJobsController.java
#Controller
public class BackgroundJobsController {
private final Logger logger = LoggerFactory.getLogger(getClass());
private #Autowired JobScheduler jobScheduler;
#Job(name = "Test")
public void executeJob() {
BackgroundJob.scheduleRecurrently(Cron.minutely(), () -> logger.debug("It works!"));
jobScheduler.scheduleRecurrently(Cron.minutely(), () -> logger.debug("It works too!"));
}
}
As you can see, I have tried both methods of initiating the background job in the executeJob method. The issue is basically getting Jobrunr to detect the jobs - is it simply a case of somehow triggering the executeJob method upon startup of the application? If so, does anyone know the most simple way to do that? Previously I have used the Spring #Scheduled annotation to automatically run through methods in a Service/Controller class upon startup of the application, so I was hoping there was a straightforward way to get Jobrunr to pick up the scheduled tasks I am trying to create. Apologies if it is something stupid that I have overlooked. I've spent a good few hours trying different things and reading through the documentation!
Thanks in advance!
There are different ways for doing so:
This is one, annotating a method with #PostConstruct is indeed another.
#SpringBootApplication
#Import(JobRunrExampleConfiguration.class)
public class JobRunrApplication {
public static void main(String[] args) {
ConfigurableApplicationContext applicationContext = SpringApplication.run(JobRunrApplication.class, args);
JobScheduler jobScheduler = applicationContext.getBean(JobScheduler.class);
jobScheduler.<SampleJobService>scheduleRecurrently("recurring-sample-job", every5minutes(), x -> x.executeSampleJob("Hello from recurring job"));
}
}
You can see an example here: https://github.com/jobrunr/example-java-mag/blob/main/src/main/java/org/jobrunr/examples/JobRunrApplication.java
Have you tried annotating your executeJob Method with a #PostConstruct ? That way upon initialisation of your application, the jobs would be registered to the JobServer.
I believe the #Job annotation is meant fo the method of the job itself. (In your case the debug method).
There is now a new way to do so:
You can add #Recurring to any Spring Boot, Micronaut or Quarkus bean method. A Spring Boot example:
#Component
public class SomeService {
#Recurring(id="recurring-job-every-5-min" interval = "PT5M")
#Job(name="job name for the dashboard")
public void runEvery5Minutes() {
// business logic comes here
}
}
For more info, see the JobRunr documentation.

Spring-Boot: scalability of a component

I am trying Spring Boot and think about scalabilty.
Lets say I have a component that does a job (e.g. checking for new mails).
It is done by a scheduled method.
e.g.
#Component
public class MailMan
{
#Scheduled (fixedRateString = "5000")
private void run () throws Exception
{ //... }
}
Now the application gets a new customer. So the job has to be done twice.
How can I scale this component to exist or run twice?
Interesting question but why Multiple components per customer? Can scheduler not pull the data for every customer on scheduled run and process the record for each customer? You component scaling should not be decided based on the entities evolved in your application but the resources utilization by the component. You can have dedicated components type for processing the messages for queues and same for REST. Scale them based on how much each of them is getting utilized.
Instead of using annotations to schedule a task, you could do the same thing programmatically by using a ScheduledTaskRegistrar. You can register the same bean multiple time, even if it is a singleton.
public class SomeSchedulingConfigurer implements SchedulingConfigurer {
private final SomeJob someJob; <-- a bean that is Runnable
public SomeSchedulingConfigurer(SomeJob someJob) {
this.someJob = someJob;
}
#Override
public void configureTasks(#NonNull ScheduledTaskRegistrar taskRegistrar) {
int concurrency = 2;
IntStream.range(0, concurrency)).forEach(
__ -> taskRegistrar.addFixedDelayTask(someJob, 5000));
}
}
Make sure the thread executor you are using is large enough to process the amount of jobs concurrently. The default executor has exactly one thead :-). Be aware that this approach has scaling limits.
I also recommend to add a delay or skew between jobs, so that not all jobs run at exactly the same moment.
See SchedulingConfigurer
and
ScheduledTaskRegistrar
for reference.
The job needs to run only once even with multiple customers. The component itself doesn't need to scale at all. It just a mechanism to "signal" that some logic needs to be run at some moment in time. I would keep the component really thin and just call the desired business logic that handles all the rest e.g.
#Component
public class MailMan {
#Autowired
private NewMailCollector newMailCollector;
#Scheduled (fixedRateString = "5000")
private void run () throws Exception {
// Collects emails for customers
newMailCollector.collect();
}
}
If you want to check for new e-mails per customer you might want to avoid using scheduled tasks in a backend service as it will make the implementation very inflexible.
Better make an endpoint available for clients to call to trigger that logic.

Unable to set the cron job schedule in application properties. SpringBoot ;scheduling of cron job at runtime using Scheduler

I am trying to schedule my rest service(with GET method) using
#Scheduled(cron = xyzzy.getTimeSchedule())
The schedule details are expected to be obtained from application properties from cloud during the application start up. however I get "The value for Annotation attribute Scheduled.cron must be a constant expression" compile time error. Please suggest. Also what might be the underlying issue like with the Spring annotation and the properties being available during the time application start up. Please guide or direct me to understand.TIA.
Using #Scheduled annotation you cannot provide a method for cron-expression which comes from the cloud. As java annotation needs constant-expression, which is a variable whose value cannot change once it has been assigned. For that, you need to use final keyword.
public static final String TIME_SCHEDULE = "0 0/30 8-10 * * *";
Then use that constant expression in your scheduler method,
#Scheduled(cron = TIME_SCHEDULE)
In your case, you should go for #TaskScheduler (from doc)
Task scheduler interface that abstracts the scheduling of Runnables based on different kinds of triggers.
This interface is separate from SchedulingTaskExecutor since it usually represents for a different kind of backend, i.e. a thread pool with different characteristics and capabilities. Implementations may implement both interfaces if they can handle both kinds of execution characteristics.
Replacing #Scheduled annotation with #TaskScheduler
First, autowire TaskScheduler and make sure that you annotated your main class with #EnableScheduling annotation to provide bean for TaskScheduler.
#Autowired
private TaskScheduler scheduler;
Now you need to schedule providing Runnable and CronTrigger arguments. It schedules the given Runnable, invoking it whenever the trigger indicates a next execution time.
That means you need to wrap your logic(currently this is the code from your #Schduled method body) into Runnable instance. And your xyzzy.getTimeSchedule() should be provided to CronTrigger constructor.
Runnable runnableTask = () -> {
//call REST API here
};
scheduler.schedule(runnableTask, new CronTrigger(xyzzy.getTimeSchedule());
Now you get rid of "The value for Annotation attribute Scheduled.cron must be a constant expression"
Finally this is what worked for me.
I have my property stored as key : value pair over the cloud.like so..
xyz.Schedule = */5 * * * * ;
public Class testController {
#Autowired
private Type type;
#Scheduled(cron = "${type.getSchedule()}")
#GetMapping(path = "/", produces = "application/json")
public void getmethod() { blah blah } }
I ran my application successfully and was being able to populate the property configured over cloud at the start of application, and was able to get response for my api as well.
Things I tried:
As suggested by Shekhar Rai in this discussion chain, declaring it as final variable, but was unable to access it in my method.
tried wrapping the method as a runnable task, couldn't do so.
arrived at: #Scheduled.cron always expects a constant parameter(like a string), but get() is dynamic , therefore wrapped it as a constant parameter.
#ManagedConfiguration
private ConfigClass configClass;
#bean
public String getSchedulerValue() {
return configClass.getSchedule();}
#Scheduled(cron="#{getSchedulerValue}")

#EventListener(ApplicationReadyEvent.class) starts only one method?

I'm trying to run a few methods after the Spring Boot project starts. I'm using #EventListener(ApplicationReadyEvent.class) annotation above the methods I want ran after the project launches. But it's only starting for one method at a time. I want multiple methods started at once. Is that the expected behavior for #EventListener(ApplicationReadyEvent.class)?
Its OK to place several (more than one) methods annotated with #EventListener all of them will be executed:
#Configuration
public class SampleConfiguration {
#Bean
public SampleBean sampleBean() {return new SampleBean();}
#EventListener
public void onApplicationReadyEvent(ApplicationReadyEvent event) {
System.out.println("Hello");
}
#EventListener
public void onApplicationReadyEvent2(ApplicationReadyEvent event) {
System.out.println("How are you");
}
}
This will print both "Hello" and "How are you" upon the succesfull start of the Application Context.
Now, its true that spring doesn't invoke them concurrently, it resolves all the listeners and calls them sequentially.
If you need a parallel execution you can create one listener that will be an "entry point" for the logical tasks that must be run in parallel and use Threads / Thread Pool Executors to run the code of your choice in parallel
Did you try adding #Async also above the method?
This listener is invoked synchronously. You can make it asynchronous by simply adding #Async annotation.
You can have the event listeners execute asynchronously by adding the following bean to your #Configuration class.
#Bean(name = "applicationEventMulticaster")
public ApplicationEventMulticaster simpleApplicationEventMulticaster() {
SimpleApplicationEventMulticaster eventMulticaster =
new SimpleApplicationEventMulticaster();
eventMulticaster.setTaskExecutor(new SimpleAsyncTaskExecutor());
return eventMulticaster;
}
If you've defined a custom TaskExecutor then you should replace new SimpleAsyncTaskExecutor() with yourCustomTaskExecutorBeanMethod()
I had run into a similar issue where ApplicationReadyEvent was annoted for 3 function but in debugging we found it to fire for only 1 always. We added #Order and kept that function with the highest index(i.e. lowest priority), as in our case it was running for an indefinite while loop, and found that all the 3 functions were invoked seamlessly.

Accessing Spring objects from a dynamically created persistent Quartz job

Spring Boot 1.5
Quartz 2.2
I dynamically create and schedule Quartz jobs during runtime with a Quartz-configured as a jdbc-job-store. These jobs need to be persistent between app executions.
During the job execution, I need access to the full Spring context (Spring-managed beans and JPA transactions).
However, if I try to Autowire anything into my job, then I get an error like..
"Unsatisfied dependency expressed through field myAutowiredField"
I can't figure this out. I have found tons of examples of people showing how to get autowiring to work in a Quartz job, but almost all of these just have a static, hard-coded job. Not a job dynamically created at runtime.
The example at the following URL comes the closest. It dynamically creates jobs and autowiring works great in them. However, it's a ram job store. As soon as I switch to jdbc, I'm back to square one.
https://icecarev.com/2016/11/05/spring-boot-1-4-and-quartz-scheduling-runtime-created-job-instances-from-a-configuration-file/
I have also looked at these..
Spring + Hibernate + Quartz: Dynamic Job
inject bean reference into a Quartz job in Spring?
.... etc.
But again, their solutions all seem to be missing something. For example, they rely on static jobs, triggers, or just plain don't work, etc.
Anybody have any tips or links to up-to-date resources?
Thanks!
Edit 1
Something happens to the spring context when the job is fired. Here's some code to illustrate.
In the first autowireBean() call (this is done during the Spring Boot configuration), it doesn't throw an error. NOTE: At this point, there's no use for this, I'm just showing that it does 'work' here.
In the second autowireBean() call (this is when the job is fired), it fails. This is the 'real' call.
public class AutowiringSpringBeanJobFactory extends SpringBeanJobFactory
{
private transient AutowireCapableBeanFactory beanFactory;
public AutowiringSpringBeanJobFactory(ApplicationContext context)
{
super();
this.beanFactory = context.getAutowireCapableBeanFactory();
MyJobClass job = new MyJobClass();
beanFactory.autowireBean(new MyJobClass()); /** no problem **/
beanFactory.initializeBean(job, "job");
}
#Override
protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception
{
final Object job = super.createJobInstance(bundle); /* job is an instance MyJobClass.. same as above */
beanFactory.autowireBean(job); /** "Unsatisfied dependency" exception **/
beanFactory.initializeBean(job, "job");
return job;
}
}
Edit 2
Well, I seem to have got it working, however, I don't know if there will be consequences.
Here's the culprit in org.springframework.scheduling.quartz.AdaptableJobFactory
protected Object createJobInstance(TriggerFiredBundle bundle) throws Exception {
return bundle.getJobDetail().getJobClass().newInstance();
}
It seems that bundle.getJobDetail().getJobClass() returns some 'proxy' type of my job class. But basically it 'says' it's returning my correct Job class, but it's different. It has the same name, but I can't cast it. For example...
Object job = bundle.getJobDetail().getJobClass().newInstance();
MyJobClass myJob = (MyJobClass) job;
Throws an error saying that I can't cast com.company.MyJobClass to com.company.MyJobClass.
So here's my 'fix'...
#Override
protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception
{
String className = bundle.getJobDetail().getJobClass().getName();
Object job = Class.forName(className).newInstance();
beanFactory.autowireBean(job);
job = beanFactory.applyBeanPostProcessorsAfterInitialization(job, "job"); // Without this, #Transactional in my job bean doesn't work
beanFactory.initializeBean(job, "job");
return job;
}
Since I'm not calling super() anymore, I lose a handy feature of SpringBeanJobFactory, but I'm OK with that for the moment.
I guess bottom line is that autowireBean() needs to be called before this sucker gets wrapped in a transactional proxy.
When you mention "Not a job dynamically created at runtime.", I'm going to assume that at some point you do something similar to: MyJob job = new MyJob();
If you want to inject dependencies in MyJob using Spring's #Autowire, one approach that comes to my mind is:
Annotate MyJob class with: #Configurable(dependencyCheck = true)
Run the Java process using a Java agent: java -javaagent:<path to spring-agent-${spring.version}.jar> ...

Resources