I am trying to schedule something than need to be ran every 200ms:
#Scheduled(every = "0.2s")
What's the recommended way to do it?
Is there a good reason why #Scheduled doesn't support rate under the second?
Unfortunately, this is not supported. The idea is that #Scheduled is only used for cron-like tasks with an accuracy of seconds (note that the Cron standard is designed to work with an accuracy of minutes).
However, you can inject a managed Vertx instance and make use of Vertx.setPeriodic(). But keep in mind that the handler is executed on an event loop thread and so the code should not block. If you really need to execute some blocking code then look at Vertx.executeBlocking().
The plan is to add an injectable managed ScheduledExecutorService for similar use cases.
Related
I am currently reviewing options to develop looped method execution and one suggestion was to make use of simple Spring #Scheduled annotation. So if I use this approach and use #Scheduled(fixedRate = 1000) in which task scheduled to run each second, If in case each task execution is taking more than second and if there is a huge backlog of scheduled tasks in millions, can it cause some sort of leak?
In my Spring Boot application, based on the Cron job(runs every 5 minutes) I need to process 2000 products in my database.
Right now the process time of these 2000 products takes more than 5 minutes. I ran into the issue where the second Cron job runs when the first one is not completed yet.
Is there in Spring/Cron out of the box functionality that will allow to synchronize these jobs and wait for the previous job completion before starting the next one?
Please advise how to properly implement such kind of system. Anyway, the following technologies are also available Neo4j, MongoDB, Kafka. Please advise how to properly design/implement this functionality using the Spring/Cron separately or even together with the mentioned technologies.
1) You may try to use #Scheduled(fixedDelay = 5*60*1000). It will guarantee that next invocation will happen strictly in 5 minutes after previous one is finished. But this may break your scheduling requirements
2) You can limit the underlying ThreadExecutor's pool size to 1 thread, so next invocation will have to wait until previous is finished, but this, again, can break the logic, since it would affect all periodic tasks invoked by #Scheduled
3) You can use Quartz instead of spring's native #Scheduled. It's more complicated to configure, but allows to achieve the desired behaviour via #DisallowConcurrentExecution annotation or via setting JobDetail::isConcurrentExectionDisallowed in your job details
I wonder how could I write spring tests to assert logic chain which is triggered by 'SourcePollingChannelAdapter'.
What comes to my mind:
use Thread.sleep() which is really bad idea for tests
Have another test version of spring context where I will replace all pollable channels with direct ones. This requires much work.
Are there any common ways to force trigger poller within test?
Typically we use QueueChannel in our tests and wait for the messages via its receive(10000) method. This way, independently of the source of data, our test method thread is blocked until data has arrived.
The SourcePollingChannelAdapter is triggered by the TaskScheduler, therefore the whole flow logic is done within a separate thread from the test method. I mean that your idea about replacing channels won't help. The Thread.sleep() might have value, but QueueChannel.receive(10000) is much reliable because we really maximum wait only for those 10 seconds.
Another way to block test-case comes from the standard CountDownLatch, which you would countDown() somewhere in the flow and wait for it in the test method.
There is some other way to test: have some loop with short sleep period in between iteration and check some condition to exit and verify. That may be useful in case of poller and database in the end. So, we would perform SELECT in that loop until desired state.
You can find some additional info in the Reference Manual.
I'm planning to use Quartz scheduler to process a one-time job.
My use case is, I need to migrate BLOB from one storage to another and blob's can be as big as 100GB, so a particular job can run really long enough to get the work done.
The reason I'm using Quartz because of its clustering support, fault tolerance and retry capabilities in case job fails etc. Only thing I'm concerned about is, I might have a lot of miss fire trigger scenario and a lot of database lock which can hamper live production traffic on those database hosts. I will probably be scheduling 10s of thousands of job in one shot.
Few of the things that I figured out is
I can set a high value for org.quartz.jobStore.misfireThreshold so that miss fire does not happen. I don't really care about the time when the job get's picked up as it's background job and no SLA as such. Only thing I care about is that eventually job getting picked up and getting work done.
I can also set batch mode properties org.quartz.scheduler.batchTriggerAcquisitionMaxCount and org.quartz.scheduler.batchTriggerAcquisitionFireAheadTimeWindow. I understand the batch max count property should be like equal to the thread pool size which can give the biggest bang on performance but what should be the value of fire ahead of time window be?
I'm using Quartz with Spring boot and will be leveraging org.quartz.impl.jdbcjobstore.JobStoreCMT. What I understand is execute method of the job get wrapped in the transaction, will this cause any problem since transaction will be open for a long time as the job might take hours to complete? Is this something ok? I will be using Oracle database.
Am I missing something here? Can someone share their experience with a similar use case?
Thanks!
I'v seen the term Scheduler very frequently in the documentation.
But, what does this term mean? I even don't know how to use a so called Scheduler. The official documentation didn't tell me what a Scheduler exactly is. Is this just a common concept or a specific concept in RxJS?
Rx schedulers provide an abstraction that allows work to be scheduled to run, possibly in the future, without the calling code needing to be aware of the mechanism used to schedule the work.
Whenever an Rx method needs to generate a notification, it schedules the work on a scheduler. By supplying a scheduler to the Rx method instead of using the default, you can subtly control how those notifications are sent out.
In server-side implementations of Rx (such as Rx.NET), schedulers play an important role. They allow you to schedule heavy duty work on the thread pool or dedicated threads, and run the final subscription on the UI thread so you can update your UI.
When using RxJs, it is actually pretty rare that you need to worry about the scheduler argument to most methods. Since JavaScript is essentially single-threaded, there are not a lot of options for scheduling and the default schedulers are usually the right choice.
The only real choices are:
immediateScheduler - Runs the work synchronously and immediately. Sort of like not using a scheduler at all. Work scheduled thus is guaranteed to run synchronously.
currentThreadScheduler - Similar to immediateScheduler in that the work is run immediately. However, it does not run work recursively. So, if the work is running and schedules more work, then that additional work is put into queue to be run after the current work finishes. Thus work sometimes runs synchronously and sometimes asynchronously. This scheduler is useful to avoid stack overflows or infinite recursion. For example Rx.Observable.of(42).repeat().subscribe() would cause infinite recursion if it ran on the immediate scheduler, but since return runs on the currentThread scheduler by default, infinite recursion is avoided.
timeoutScheduler - The only scheduler that supports work scheduled to be run in the future. Essentially uses setTimeout to schedule all work (though if you schedule the work to be run "now", then it uses other faster asynchronous methods to schedule the work). Any work scheduled on this scheduler is guaranteed to be run asynchronously.
There may be some more now, such as a scheduler that schedules work on the browser animation frames, etc.
If you are trying to write testable code, then you almost always want to supply the scheduler argument. This is because in your unit tests, you will be creating testScheduler instances, which will let your unit test control the clock used by your Rx code (and thus control the exact timing of the operations).