Spring Boot 1.5
Quartz 2.2
I dynamically create and schedule Quartz jobs during runtime with a Quartz-configured as a jdbc-job-store. These jobs need to be persistent between app executions.
During the job execution, I need access to the full Spring context (Spring-managed beans and JPA transactions).
However, if I try to Autowire anything into my job, then I get an error like..
"Unsatisfied dependency expressed through field myAutowiredField"
I can't figure this out. I have found tons of examples of people showing how to get autowiring to work in a Quartz job, but almost all of these just have a static, hard-coded job. Not a job dynamically created at runtime.
The example at the following URL comes the closest. It dynamically creates jobs and autowiring works great in them. However, it's a ram job store. As soon as I switch to jdbc, I'm back to square one.
https://icecarev.com/2016/11/05/spring-boot-1-4-and-quartz-scheduling-runtime-created-job-instances-from-a-configuration-file/
I have also looked at these..
Spring + Hibernate + Quartz: Dynamic Job
inject bean reference into a Quartz job in Spring?
.... etc.
But again, their solutions all seem to be missing something. For example, they rely on static jobs, triggers, or just plain don't work, etc.
Anybody have any tips or links to up-to-date resources?
Thanks!
Edit 1
Something happens to the spring context when the job is fired. Here's some code to illustrate.
In the first autowireBean() call (this is done during the Spring Boot configuration), it doesn't throw an error. NOTE: At this point, there's no use for this, I'm just showing that it does 'work' here.
In the second autowireBean() call (this is when the job is fired), it fails. This is the 'real' call.
public class AutowiringSpringBeanJobFactory extends SpringBeanJobFactory
{
private transient AutowireCapableBeanFactory beanFactory;
public AutowiringSpringBeanJobFactory(ApplicationContext context)
{
super();
this.beanFactory = context.getAutowireCapableBeanFactory();
MyJobClass job = new MyJobClass();
beanFactory.autowireBean(new MyJobClass()); /** no problem **/
beanFactory.initializeBean(job, "job");
}
#Override
protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception
{
final Object job = super.createJobInstance(bundle); /* job is an instance MyJobClass.. same as above */
beanFactory.autowireBean(job); /** "Unsatisfied dependency" exception **/
beanFactory.initializeBean(job, "job");
return job;
}
}
Edit 2
Well, I seem to have got it working, however, I don't know if there will be consequences.
Here's the culprit in org.springframework.scheduling.quartz.AdaptableJobFactory
protected Object createJobInstance(TriggerFiredBundle bundle) throws Exception {
return bundle.getJobDetail().getJobClass().newInstance();
}
It seems that bundle.getJobDetail().getJobClass() returns some 'proxy' type of my job class. But basically it 'says' it's returning my correct Job class, but it's different. It has the same name, but I can't cast it. For example...
Object job = bundle.getJobDetail().getJobClass().newInstance();
MyJobClass myJob = (MyJobClass) job;
Throws an error saying that I can't cast com.company.MyJobClass to com.company.MyJobClass.
So here's my 'fix'...
#Override
protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception
{
String className = bundle.getJobDetail().getJobClass().getName();
Object job = Class.forName(className).newInstance();
beanFactory.autowireBean(job);
job = beanFactory.applyBeanPostProcessorsAfterInitialization(job, "job"); // Without this, #Transactional in my job bean doesn't work
beanFactory.initializeBean(job, "job");
return job;
}
Since I'm not calling super() anymore, I lose a handy feature of SpringBeanJobFactory, but I'm OK with that for the moment.
I guess bottom line is that autowireBean() needs to be called before this sucker gets wrapped in a transactional proxy.
When you mention "Not a job dynamically created at runtime.", I'm going to assume that at some point you do something similar to: MyJob job = new MyJob();
If you want to inject dependencies in MyJob using Spring's #Autowire, one approach that comes to my mind is:
Annotate MyJob class with: #Configurable(dependencyCheck = true)
Run the Java process using a Java agent: java -javaagent:<path to spring-agent-${spring.version}.jar> ...
Related
I have been looking into using JobRunr for starting background jobs on my Spring MVC application, as I really like the simplicity of it, and the ease of integrating it into an IoC container.
I am trying to create a simple test scheduled job that writes a line of text to my configured logger every minute, but I'm struggling to figure out how to get the JobRunr background job server to detect it and queue it up. I am not using Spring Boot so I am just using the generic jobrunr Maven artifact rather than the "Spring Boot Starter". My setup is as follows:
pom.xml
<dependency>
<groupId>org.jobrunr</groupId>
<artifactId>jobrunr</artifactId>
<version>2.0.0</version>
</dependency>
ApplicationConfig.java
#Bean
public JobMapper jobMapper() {
return new JobMapper(new JacksonJsonMapper());
}
#Bean
#DependsOn("jobMapper")
public StorageProvider storageProvider(JobMapper jobMapper) {
InMemoryStorageProvider storageProvider = new InMemoryStorageProvider();
storageProvider.setJobMapper(jobMapper);
return storageProvider;
}
#Bean
#DependsOn("storageProvider")
public JobScheduler jobScheduler(StorageProvider storageProvider, ApplicationContext applicationContext) {
return JobRunr.configure().useStorageProvider(storageProvider)
.useJobActivator(applicationContext::getBean)
.useDefaultBackgroundJobServer()
.useDashboard()
.useJmxExtensions()
.initialize();
}
BackgroundJobsController.java
#Controller
public class BackgroundJobsController {
private final Logger logger = LoggerFactory.getLogger(getClass());
private #Autowired JobScheduler jobScheduler;
#Job(name = "Test")
public void executeJob() {
BackgroundJob.scheduleRecurrently(Cron.minutely(), () -> logger.debug("It works!"));
jobScheduler.scheduleRecurrently(Cron.minutely(), () -> logger.debug("It works too!"));
}
}
As you can see, I have tried both methods of initiating the background job in the executeJob method. The issue is basically getting Jobrunr to detect the jobs - is it simply a case of somehow triggering the executeJob method upon startup of the application? If so, does anyone know the most simple way to do that? Previously I have used the Spring #Scheduled annotation to automatically run through methods in a Service/Controller class upon startup of the application, so I was hoping there was a straightforward way to get Jobrunr to pick up the scheduled tasks I am trying to create. Apologies if it is something stupid that I have overlooked. I've spent a good few hours trying different things and reading through the documentation!
Thanks in advance!
There are different ways for doing so:
This is one, annotating a method with #PostConstruct is indeed another.
#SpringBootApplication
#Import(JobRunrExampleConfiguration.class)
public class JobRunrApplication {
public static void main(String[] args) {
ConfigurableApplicationContext applicationContext = SpringApplication.run(JobRunrApplication.class, args);
JobScheduler jobScheduler = applicationContext.getBean(JobScheduler.class);
jobScheduler.<SampleJobService>scheduleRecurrently("recurring-sample-job", every5minutes(), x -> x.executeSampleJob("Hello from recurring job"));
}
}
You can see an example here: https://github.com/jobrunr/example-java-mag/blob/main/src/main/java/org/jobrunr/examples/JobRunrApplication.java
Have you tried annotating your executeJob Method with a #PostConstruct ? That way upon initialisation of your application, the jobs would be registered to the JobServer.
I believe the #Job annotation is meant fo the method of the job itself. (In your case the debug method).
There is now a new way to do so:
You can add #Recurring to any Spring Boot, Micronaut or Quarkus bean method. A Spring Boot example:
#Component
public class SomeService {
#Recurring(id="recurring-job-every-5-min" interval = "PT5M")
#Job(name="job name for the dashboard")
public void runEvery5Minutes() {
// business logic comes here
}
}
For more info, see the JobRunr documentation.
I am trying Spring Boot and think about scalabilty.
Lets say I have a component that does a job (e.g. checking for new mails).
It is done by a scheduled method.
e.g.
#Component
public class MailMan
{
#Scheduled (fixedRateString = "5000")
private void run () throws Exception
{ //... }
}
Now the application gets a new customer. So the job has to be done twice.
How can I scale this component to exist or run twice?
Interesting question but why Multiple components per customer? Can scheduler not pull the data for every customer on scheduled run and process the record for each customer? You component scaling should not be decided based on the entities evolved in your application but the resources utilization by the component. You can have dedicated components type for processing the messages for queues and same for REST. Scale them based on how much each of them is getting utilized.
Instead of using annotations to schedule a task, you could do the same thing programmatically by using a ScheduledTaskRegistrar. You can register the same bean multiple time, even if it is a singleton.
public class SomeSchedulingConfigurer implements SchedulingConfigurer {
private final SomeJob someJob; <-- a bean that is Runnable
public SomeSchedulingConfigurer(SomeJob someJob) {
this.someJob = someJob;
}
#Override
public void configureTasks(#NonNull ScheduledTaskRegistrar taskRegistrar) {
int concurrency = 2;
IntStream.range(0, concurrency)).forEach(
__ -> taskRegistrar.addFixedDelayTask(someJob, 5000));
}
}
Make sure the thread executor you are using is large enough to process the amount of jobs concurrently. The default executor has exactly one thead :-). Be aware that this approach has scaling limits.
I also recommend to add a delay or skew between jobs, so that not all jobs run at exactly the same moment.
See SchedulingConfigurer
and
ScheduledTaskRegistrar
for reference.
The job needs to run only once even with multiple customers. The component itself doesn't need to scale at all. It just a mechanism to "signal" that some logic needs to be run at some moment in time. I would keep the component really thin and just call the desired business logic that handles all the rest e.g.
#Component
public class MailMan {
#Autowired
private NewMailCollector newMailCollector;
#Scheduled (fixedRateString = "5000")
private void run () throws Exception {
// Collects emails for customers
newMailCollector.collect();
}
}
If you want to check for new e-mails per customer you might want to avoid using scheduled tasks in a backend service as it will make the implementation very inflexible.
Better make an endpoint available for clients to call to trigger that logic.
I am trying to schedule my rest service(with GET method) using
#Scheduled(cron = xyzzy.getTimeSchedule())
The schedule details are expected to be obtained from application properties from cloud during the application start up. however I get "The value for Annotation attribute Scheduled.cron must be a constant expression" compile time error. Please suggest. Also what might be the underlying issue like with the Spring annotation and the properties being available during the time application start up. Please guide or direct me to understand.TIA.
Using #Scheduled annotation you cannot provide a method for cron-expression which comes from the cloud. As java annotation needs constant-expression, which is a variable whose value cannot change once it has been assigned. For that, you need to use final keyword.
public static final String TIME_SCHEDULE = "0 0/30 8-10 * * *";
Then use that constant expression in your scheduler method,
#Scheduled(cron = TIME_SCHEDULE)
In your case, you should go for #TaskScheduler (from doc)
Task scheduler interface that abstracts the scheduling of Runnables based on different kinds of triggers.
This interface is separate from SchedulingTaskExecutor since it usually represents for a different kind of backend, i.e. a thread pool with different characteristics and capabilities. Implementations may implement both interfaces if they can handle both kinds of execution characteristics.
Replacing #Scheduled annotation with #TaskScheduler
First, autowire TaskScheduler and make sure that you annotated your main class with #EnableScheduling annotation to provide bean for TaskScheduler.
#Autowired
private TaskScheduler scheduler;
Now you need to schedule providing Runnable and CronTrigger arguments. It schedules the given Runnable, invoking it whenever the trigger indicates a next execution time.
That means you need to wrap your logic(currently this is the code from your #Schduled method body) into Runnable instance. And your xyzzy.getTimeSchedule() should be provided to CronTrigger constructor.
Runnable runnableTask = () -> {
//call REST API here
};
scheduler.schedule(runnableTask, new CronTrigger(xyzzy.getTimeSchedule());
Now you get rid of "The value for Annotation attribute Scheduled.cron must be a constant expression"
Finally this is what worked for me.
I have my property stored as key : value pair over the cloud.like so..
xyz.Schedule = */5 * * * * ;
public Class testController {
#Autowired
private Type type;
#Scheduled(cron = "${type.getSchedule()}")
#GetMapping(path = "/", produces = "application/json")
public void getmethod() { blah blah } }
I ran my application successfully and was being able to populate the property configured over cloud at the start of application, and was able to get response for my api as well.
Things I tried:
As suggested by Shekhar Rai in this discussion chain, declaring it as final variable, but was unable to access it in my method.
tried wrapping the method as a runnable task, couldn't do so.
arrived at: #Scheduled.cron always expects a constant parameter(like a string), but get() is dynamic , therefore wrapped it as a constant parameter.
#ManagedConfiguration
private ConfigClass configClass;
#bean
public String getSchedulerValue() {
return configClass.getSchedule();}
#Scheduled(cron="#{getSchedulerValue}")
I have a Spring Boot Batch application that needs to run daily. It reads a daily file, does some processing on its data, and writes the processed data to a database. Along the way, the application holds some state such as the file to be read (stored in the FlatFileItemReader and JobParameters), the current date and time of the run, some file data for comparison between read items, etc.
One option for scheduling is to use Spring's #Scheduled such as:
#Scheduled(cron = "${schedule}")
public void runJob() throws Exception {
jobRunner.runJob(); //runs the batch job by calling jobLauncher.run(job, jobParameters);
}
The problem here is that the state is maintained between runs. So, I have to update the file to be read, the current date and time of the run, clear the cached file data, etc.
Another option is to run the application via a unix cron job. This will obviously meet the need to clear state between runs but I prefer to tie the job scheduling to the application instead of the OS (and prefer it to OS agnostic). Can the application state be reset between #Scheduled runs?
You could always move the code that performs your task (and more importantly, keeps your state) into a prototype-scoped bean. Then you can retrieve a fresh instance of that bean from the application context every time your scheduled method is run.
Example
I created a GitHub repository which contains a working example of what I'm talking about, but the gist of it is in these two classes:
ScheduledTask.java
Notice the #Scope annotation. It specifies that this component should not be a singleton. The randomNumber field represents the state that we want to reset with every invocation. "Reset" in this case means that a new random number is generated, just to show that it does change.
#Component
#Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
class ScheduledTask {
private double randomNumber = Math.random();
void execute() {
System.out.printf(
"Executing task from %s. Random number is %f%n",
this,
randomNumber
);
}
}
TaskScheduler.java
By autowiring in ApplicationContext, you can use it inside the scheduledTask method to retrieve a new instance of ScheduledTask.
#Component
public class TaskScheduler {
#Autowired
private ApplicationContext applicationContext;
#Scheduled(cron = "0/5 * * * * *")
public void scheduleTask() {
ScheduledTask task = applicationContext.getBean(ScheduledTask.class);
task.execute();
}
}
Output
When running the code, here's an example of what it looks like:
Executing task from com.thomaskasene.example.schedule.reset.ScheduledTask#329c8d3d. Random number is 0.007027
Executing task from com.thomaskasene.example.schedule.reset.ScheduledTask#3c5b751e. Random number is 0.145520
Executing task from com.thomaskasene.example.schedule.reset.ScheduledTask#3864e64d. Random number is 0.268644
Thomas' approach seems to be a reasonable solution, that's why I upvoted it. What is missing is how this can be applied in the case of a spring batch job. Therefore I adapted his example little bit:
#Component
public class JobCreatorComponent {
#Bean
#Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public Job createJob() {
// use the jobBuilderFactory to create your job as usual
return jobBuilderFactory.get() ...
}
}
your component with the launch method
#Component
public class ScheduledLauncher {
#Autowired
private ... jobRunner;
#Autwired
private JobCreatorComponent creator;
#Scheduled(cron = "${schedule}")
public void runJob() throws Exception {
// it would probably make sense to check the applicationContext and
// remove any existing job
creator.createJob(); // this should create a complete new instance of
// the Job
jobRunner.runJob(); //runs the batch job by calling jobLauncher.run(job, jobParameters);
}
I haven't tried out the code, but this is the approach I would try.
When constructing the job, it is important to ensure that all reader, processors and writers used in this job are complete new instances as well. This means, if they are not instantiated as pure java objects (not as spring beans) or as spring beans with scope "step" you must ensure that always a new instance is used.
Edited:
How to handle SingeltonBeans
Sometimes singleton beans cannot be prevented, in these cases there must be a way to "reset" them.
An simple approach would be to define an interface "ResetableBean" with a reset method that is implemented by such beans. Autowired can then be used to collect a list of all such beans.
#Component
public class ScheduledLauncher {
#Autowired
private List<ResetableBean> resetables;
...
#Scheduled(cron = "${schedule}")
public void runJob() throws Exception {
// reset all the singletons
resetables.forEach(bean -> bean.reset());
...
I have used Spring in the past. I moved to a different team where I am getting familiarized with codebase. I found the following code and trying to understand how it works and how spring injects autowired objects in the case. From my basics of Spring, this is definitely not the right way to do. But surprisingly, this code is in production for a long time and no issues were identified.
#Controller
#RequestMapping("/start")
public class AController implements Runnable, InitializingBean {
#Autowired
private StartServiceImpl service = new StartServiceImpl(); // 1
Thread thread;
public void run() {
service.start();
}
public void stop() {
try {
thread.join();
} catch (InterruptedException e) {
}
}
}
#Override
public void afterPropertiesSet() throws Exception {
thread = new Thread(this);
thread.setPriority(Thread.MAX_PRIORITY);
thread.start();
}
}
#Component
public class StartServiceImpl {
//methods
}
Q1) What does localhost:8080/project/start is expected to do. There is NO GET or POST methods defined.
Q2) on the commented line 1, StartServiceImpl is both autowired and constructed with "new". So what happens here. Does the container inject bean or just an object is instantiated.
#Controller
#RequestMapping("/stop")
public class BController {
#Autowired
private StartServiceImpl service = new StartServiceImpl();
#RequestMapping(value = "**", method = RequestMethod.GET)
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
try {
service.shutdownRequested();
new AController().stop(); // 2
} catch (Exception e) {
}
}
}
Q3) Again in commented line 2, does calling stop, calls the stop on the bean in the application context or a new object gets created and the stop method is called. What would happen in the latter case? Are we really stopping the service that was started or not? I think we are not stopping the service.
I have read this post. It is very useful. But it did not answer my question.
I will try to answer the questions specifically, as the purpose of the code is hard to understand (for me at least).
Q1) It is unclear for me what this code tries to achieve. As you noticed, it is not a controller, and I suspect that the only reason why it is registered this way is so that it gets auto-scanned (which might as well get done by using #Controller. This is just a hunch, I don't quite understand its purpose.
Q2) The answer is that two instances will be created, one via new, the other as a bean. When running in Spring, the final value of the field is the bean, because dependency injection happens after the construction. Typically this is done when the class is envisioned to be used outside Spring (e.g. a unit test), so that the field can be initialized with a default value.
Q3) stop() will be invoked on a new instance, and not the bean. The service bean is stopped because of the direct call above that line to the injected bean, but the next one will be an NPE, I guess, because afterPropertiesSet is not invoked on the target object created via new. the only reason why this doesn't show an NPE in the logs is because the exception is swallowed below. The thread variable is not initialized and remains NULL.
Hope this helps,
This code is flawed on many levels.
Ever since Java 5, manually starting threads is an antipattern. It's messy and way too low-level. ExecutorServices should be used.
A Rest controller that is a Runnable? That's a monstrous mingling of concerns.
A service is created via new but then overwritten with an autowired dependency? WTF!
etc.
I'd keep the thread running all the time, scheduling the task with the #Scheduled annotation and use the controller to toggle a flag that decides if the thread actually does somethin, e.g.
#Service
class StartService{
private boolean active;
public void setActive(boolean active){this.active=active;}
#Scheduled(fixedRate=5000)
public void doStuff(){
if(!active)return;
// do actual stuff here
}
}
And now all the rest controllers do is toggle the value of the "active" field. Benefits:
every class does one thing only
you always know how many threads you have
The code you posted is very strange.
Q1 ) What does localhost:8080/project/start is expected to do. There
is NO GET or POST methods defined.
I think localhost:8080/project/start will return 404 error (The requested resource is not available). Because there is no mapped methods in AController. #RequestMapping annotation on class level is not enough for make request to controller. There is have to be mapped method.
But service will be started anyway. Because AController implements InitializingBean. Method afterPropertiesSet() will be invoked by Spring after controller will be created and all fields will be initialized.
Q2) on the commented line 1, StartServiceImpl is both autowired and
constructed with "new". so what happens here. does the container
inject bean or just an object is instantiated.
Another strange snippet. Java will create new instance of StartServiceImpl on creation of instance of AController class. But after that, Spring will assign it's own instance(declared as component) to this field. And reference to firs instance (created by constructor) will be lost.
Q3) Again in commented line 2, does calling stop, calls the stop on
the bean in the appication context or a new object gets created and
the stop method is called. what would happen in the latter case? Are
we really stopping the service that was started or not? I think we are
not stopping the service
Actually service will be stopped. Because of invocation of service.shutdownRequested();. But thread in AController bean will continue to work. new AController().stop(); will invoke method of just created instance, but not method of controller (instance created by Spring).
This code is totally wrong usage of Spring framework.