I want to schedule jobs to happen at a specific time and date but I'm getting confused by the wide range of options for doing so.
My requirements:
These are not recurring jobs, they only need to happen once at a specified date and time
I'm the only user of the app so don't need to deal with heavy traffic
I would like to minimise the cost of running this on Heroku, i.e. not paying idle dynos
Any tips on which combinations of gems etc. to use?
Have you looked into using https://github.com/bvandenbos/resque-scheduler? You'll need the Redis To Go addon on Heroku. This will cost you $36 a month because you'll need a scheduler process running alongside your web process. However, I've done this for free. See the README here: https://github.com/austinthecoder/pinger.
Good luck!
Due to Heroku Scheduler (default Heroku add-on) doesn't allow you to schedule your job as specific time. It is best to rely on a clock process to do the job. Gem such has Clockwork could be set up please see Heroku's Clockwork guide. You need to combine Clock with a background queuer such as resque or sidekiq. I highly recommend you go for sidekiq. Please bear in mind that both resque and sidekiq requires redis which is offered by redistogo add-on and it will cost you money to run it.
Related
I'm building a monitoring service similar to pingdom but monitoring different aspects of a system and using sidekiq to queue the tasks which is working well. What I need to do is to schedule sending out pings every minute, rather than using a cron based system which would require spinning up a new ruby instance every minute I have gone down the route of using sidetiq (notice the different spelling with a "t") which uses sidekiq's own queue to schedule future tasks. This feels like a neat solution, however I am concerned this may not be the most reliable way of scheduling tasks? If there are issues with the system (as there inevitable will be at some point) will this method of scheduling tasks be less reliable than using a cron based method and why?
Thanks
You are giving too short description of your system needs but I'll try to guess how it could be:
In the first place using sidekiq means that you'll also need an instance of redis and also means that you'll need a way to monitor the sidekiq process and restart it in case of failure and possibly redis server.
A method based on cron tasks will have fewer requirements therefore much less possibilities of failing.
cron has been around for a long time and it's battle tested and it's very very reliable, but has it's drawbacks too.
Said that, you can build a system with separate instances of redis in a master/slave configuration and you can also use Redis sentinel to implement a failover in case of the master failure, implement a monitoring/alerting system on this setup (you can use something super simple like this http://contribsys.com/inspeqtor/ from the sidekiq author) and you can also start several instances of sidekiq in different machines.
With all of that, you can have a quite reliable system for running sidekiq with sidetiq.
Hope it helps
If I have an app on Heroku that consists of one worker and one or no web dynos, will it run? I'm unsure if the absent or idling web dynos will cause the worker dyno not to run.
Heroku doesn't just run web dynos, in fact, it makes no assumptions at all with regards to the processes you're running. There's absolutely nothing wrong with launching a single worker process.
This is actually a common scenario for me to deploy single cron-like tasks to Heroku, I've written about it here http://blog.y3xz.com/blog/2012/11/16/deploying-periodical-tasks-on-heroku/
If you are looking for cron-like tasks for simple jobs (like I am), now you have another alternative: Heroku Scheduler. It is easy to configure in a dashboard.
Advantage:
No need to choose and learn a new scheduler library. Configure it in seconds.
Same way for different platforms: Python, Ruby, etc.
Save Dyno Hours for Free Plan user. Only the actual working time counts. Some scheduler library (like Rufus Scheduler) will keep running between launches (so that it does not rely on cron to work).
Disadvantage:
Trivial options. You can only choose among "Daily"/"Hourly"/"Every 10 minutes".
Conclusion: Best for basic use.
I am working on a ruby application that creates todos and meetings.
There will be reminders that are sent out with respect to each meeting or todo as you would imagine.
We are already using sidekiq and it would be nice to use sidekiq to create the scheduled jobs in x number of days/hours etc.
My concern is that we will lose the jobs if redis restarts.
Am I write in assuming that if redis restarts, we lose the jobs and if so, is there anything that can be done about it?
If not sidekiq, what else could I use?
There are several ways of doing that, just go through the link http://redis.io/topics/persistence. Snapshotting is a technique to snapshots of the dataset on disk.
My understanding is that Heroku charges Cron tasks based on the actual amount of time the task runs, using a dyno (based on: https://devcenter.heroku.com/articles/scheduler#dynohour-costs).
So, if I need a quick task to run every X minutes, I could use the cron addon to process it, instead of a worker dyno and I would be charged a fraction of the cost.
So, if everything above is true, what is the use of the Iron.io workers? They charge (about) the same as dynos from the cron jobs and they can't connect to the DB.
I have a feeling that I am missing something.
I'm under the impression that IronWorker can use the databases for Heroku. I may be mistaken about this, however.
I can't claim a great deal of experience with cron tasks on Heroku, so forgive me if I get its limitations wrong. However, I'm under the impression that scaling from one dyno to many dynos is a little bit of a process. Where IronWorker shines is really in enabling the number of worker servers to fluctuate based on demand--you have the capacity to scale at a moment's notice, but are only paying for the scale you're actually using.
I have a Rails app running on heroku with Rufus Scheduler added on.
A 'once a day' task in the scheduler is running more often than once a day.
My guess would be something to do with the heroku app running on different dynos during the day, but I'm at a loss on how to confirm/fix the problem.
Has anyone else seen this/know of a solution?
Edit: I couldn't resolve the problem with the gem and have moved my app over to the heroku scheduler add on which does not experience this problem.
The Heroku scheduler isn't guaranteed, it's only a simple scheduling system designed to fill a gap. It's nothing to do with your application moving between dynos as it's a seperate management system spinning up one of processes.
If timeliness is essential to you, take a look at clockwork, which will let you configure all sorts of stuff, but also give you a bit more reliability (at the expense of having a clock process running).
If this won't do - simply rework your job so that it doesn't matter how often it runs.