I want to schedule a job on a specific day at a specific time with some interval. I am using gocron scheduler for this. But I can't find a way to start a job on specific day. e.g. I want to execute a job on 7 Sept 2019 at 330pm. From 7 Sept, I want that job to be executed daily or weekly. How can I do that using gocron. or Any other packages available?
I tried passing UTC time to gocron.At() but its panics as it's expecting only "03:30" time formats and doesn't expect date.
When looking at the documentation for gocron, it does not seem to be designed to support scheduling things for specific days. It seems to be designed as a way to schedule things to run at various intervals, very similar to what the original cron utility was designed to do. So you would specify "I want this function to get called every 2 hours" or "I want this function to get called every Sunday at 3PM". There does not seem to be any documentation about starting jobs from a specific day.
The mentioned At(string) method is documented as allowing you to specify a time of day to run something. So you would use that to set that your job runs at 3:30PM.
If you wish to specify a start time, you would likely need to find another scheduling library or implement it yourself by creating a goroutine that sleeps until a specific time. The StackOverflow post mentioned by domcyrus looks like an excellent resource for implementing it yourself as well as listing some other scheduling libraries.
Related
I am trying to know if there is any way to use NIFI CORN Driven scheduler or other ways to run the process for a specific time a day, or to stop for a brief moment.
For Example:
Every day i want the scheduler to stop between 5am-6am & 5pm-6pm, and the rest of the time the continous.
hope my questions clear, and thanks for the read.
The CRON scheduler allows for specifically-timed discrete invocations, rather than "on during this period and off during this period". I've crafted a CRON schedule expression which sort of meets your request, but I have a different suggestion instead (CRON scheduling is also not an encouraged approach; it was implemented in NiFi for legacy reasons but isn't really actively developed or recommended).
The CRON expression
0-59 0-4,6-16,18-23 * * *
This expression means "run (the processor) at minute 0, 1, ..., 59 of hours 0-4, 6-16, 18-23 (midnight through 4:59am, 6:00am through 4:59pm, and 6:00pm through 11:59pm) everyday." Your processor will run 1320 times each day (60 times per hour for 22 hours). However, this means it will run once per minute, which may or may not be frequent enough for your needs.
Better suggestion
My actual solution would be to enable the processor to run on the actual Run Schedule you want (every 1 sec, every 3 min, etc.) and use a script to invoke the HTTP API (or Python API) to enable/disable the processor at the specified times. You can even use the CRON scheduling capability of your operating system to do so.
Example:
In NiFi, Processor X is enabled and running.
At 0500 daily, a script is run (via CRON or otherwise), which invokes curl -X PUT http://my.nifi.service:8443/nifi-api/processors/... and updates the state to STOPPED.
At 0600 daily, a script is run which updates the state to STARTED again.
Repeat at 1700 and 1800.
You can parameterize the script to accept an argument determining start/stop to reduce duplication of code. I think this is a more robust solution to your problem.
Other resources:
NiFi REST API
NiPyAPI (Unofficial but very good Python client library for NiFi)
Starting/stopping processors using NiFi API via curl - Cloudera Community and Stack Overflow
You can also make any specific request via the GUI and use the browser's developer tools to record the exact API invocation to capture the processor UUID, JSON values, etc.
Laravel is (correctly) running scheduled tasks via the App\Console\Kernel#schedule method. It does this without the need for a persistance layer. Previously ran scheduled tasks aren't saved to the database or stored in anyway.
How is this "magic" achieved? I want to have a deeper understanding.
I have looked through the source, and I can see it is somewhat achieved by rounding down the current date and diffing that to the schedule frequency, along with the fact that it is required to run every minute, it can say with a certain level of confidence that it should run a task. That is my interpretation, but I still can't fully grasp how it is guaranteeing to run on schedule and how it handles failure or things being off by a few seconds.
EDIT Edit due to clarity issue pointed out in comment.
By "a few seconds" I mean how does the "round down" method work, even when it is ran every minute, but not at the same second - example: first run 00:01.00, 00:01:02, 00:02:04
Maybe to clarify further, and to assist in understanding how it works, is there any boundary guarantees on how it functions? If ran multiple times per minute will it execute per minute tasks multiple times in the minute?
Cronjob can not guarantee seconds precisely. That is why generally no cronjob interval is less than a minute. So, in reality, it doesn't handle "things being off by a few seconds."
What happens in laravel is this, after running scheduling command for the first time the server asks "Is there a queued job?" every minute. If none, it doesn't do anything.
For example, take the "daily" cronjob. Scheduler doesn't need to know when was the last time it ran the task or something like this. When it encounters the daily cronjob it simply checks if it is midnight. If it is midnight it runs the job.
Also, take "every thirty minute" cronjob. Maybe you registered the cronjob at 10:25. But still the first time it will run on 10:30, not on 10:55. It doesn't care what time you registered or when was the last time it ran. It only checks if the current minute is "00" or divisible by thirty. So at 10:30 it will run. Again, it will run on 11:00. and so on.
Similarly a ten minute cronjob by default will only check if the current minute is divisible by ten or not. So, regardless of the time you registered the command it will run only on XX:00, XX:10, XX:20 and so on.
That is why by default it doesn't need to store previously ran scheduled task. However, you can store it into a file if you want for monitoring purpose.
My application has an Order model with an execution_datetime attribute. I'd like to send some distinct notifications. For example
execution_datetime minus 12 hours: email to carrier
execution_datetime minus 3 hours: sms to customer
execution_datetime plus 1 hour: email to customer
The above timings are not strict and can be approximated; slight deviations are acceptable. Also, the execution_datetime can change in the meantime...
I'm unsure whether to use cron or queued tasks for this. Some thoughts of my own:
Cron:
Business logic will need to be written to fetch applicable orders and execute accordingly
Is execution guaranteed? Should some sort of database flag be implemented to indicate a notification has been sent, and then perhaps fetch all due orders that are unflagged as some sort of failsafe?
Queued tasks:
Task is scheduled on creation of the order? If so, suppose the execution time is changed. How to modify the scheduled task? You'd need to somewhere keep track of the task ID?
Or perhaps a cron job that mass schedules applicable tasks every day?
I look forward to your suggestions.
Great question! I am interested in this discussion.Let me chip in with a scenario from my personal experience.
In my application, I have a Listing model and they have a promotion_ends_at column. Obviously, the listing promotion ends sometimes in the future.
So, like you also mentioned, there are two ways to do this.
When the listing is created, I could queue a job that will end the promotion on the listing in the future). The delay of that job would be the time the promotion has to end (and that could me months away).
I could also have a cron job that runs regularly that manages listings that their promotions should end on a specific date.
We were using SQS as our queue service and since the maximum delay on SQS is 15 mins, option 1 was not feasible. We, then, moved to Redis where we could queue delayed jobs with a long delay easily.
However, like you also said, the promotion_ends_at column could be updated during that time. So, either, you would have to keep track of the job to de-queue it or you could re-check whether the job should still run when it is about to execute.
For example, you could fresh() the model and check whether your condition is still valid. In my case, I would fresh my Listing and check if the promotion_ends_at is in the past. However, this means that we would have a lot of stale jobs that would probably be discarded anyway.
We finally went with a simple cron job that mass schedules the job on the day that they need to be run. I also think that running delayed jobs is a business logic and maybe the queue shouldn't be held responsible for running jobs delayed far too much in the future.
I have a script that must run at a certain hour for the amount of time I specify.
I'm looking at the clockwork gem (https://github.com/tomykaira/clockwork) which seems to be the closest piece of software I might eventually use to accomplish this, unfortunately it doesn't seem to give the ability to set a duration (start at 3PM stop 5PM), meaning I have to split the feature in 2, starting the script is going to be clockwork's job, stopping it is in the script itself with a custom solution.
Very suboptimal and messy.
How does people do this in Ruby? TIA
There is great gem called whenever for same job. With it you can set exact time for your task, like:
every 1.day, :at => '4:30 am' do
runner "MyModel.task_to_run_at_four_thirty_in_the_morning"
end
But you'll have to have two stages, one for starting one for stopping your job, which seems to be more natural than job which kills itself at some time by my opinion.
Somewhat janky, but there is another solution. I'm not sure what you are using to host your app, but on Heroku you can set up a scheduler to run every 10 minutes, on the hour, or daily. Then inside the method that the scheduler calls, you can determine the current time. Say you only want to run it between 3pm and 5pm, you would just wrap your code inside an if statement that verifies the current time is between 3pm and 5pm (watch out for time conversions with UTC).
Hope this helps.
I would like to do a cron job every 10 minutes, but my system only does 1 hour. So I'm looking for a method to do this. I've seen Timer and sleep but I'm not sure how to do this or even better yet a resource for achieving this.
Take a look at http://rufus.rubyforge.org/rufus-scheduler/
rufus-scheduler is a Ruby gem for scheduling pieces of code (jobs). It understands running a job AT a certain time, IN a certain time, EVERY x time or simply via a CRON statement.
rufus-scheduler is no replacement for cron/at since it runs inside of Ruby.
To do this reliably, invest in a VPS and create the 10-minute cron job as desired. Trying to emulate cron all on your own is very likely to fail in unforeseen ways.
Creating a sleeping process is not the way to go about this; if your server doesn't give you the freedom to make your own cron as you like it, you probably can't create your own background process for this sort of thing, either. You might be able to, on each request, take a look and see how many of the jobs need done (if it was 25 minutes since last request, you might have to do two), and go back and do them retroactively.
But, seriously. You need your own server to do this dependably.