Heroku's Scheduler is great. For tasks running daily, hourly or at 10 minute intervals I think it's excellent and simple.
I have a (very fast running) process I want to run every 10 seconds. I have it set up as a rake task, how should I efficiently and simply set this up on the Heroku platform while minimising my dyno usage?
Thanks
Maybe these articles give hints even though they are for Java and Python:
https://devcenter.heroku.com/articles/scheduled-jobs-custom-clock-processes-java-quartz-rabbitmq
https://devcenter.heroku.com/articles/clock-processes-python
Related
This is pretty simple to ask, but harder to solve. The heroku scheduler addon allows for a frequency of every 10 min. But how can I retrieve data from an API every, say, 30 seconds?
I've looked into
Whether the heroku scheduler can be tricked into doing it more frequently
Can't see a way
Whether I can schedule so many of the same task that at least one is bound to run every 30 seconds
This is incredibly unproductive and imprecise, but it's the leading candidate for what I need to do
App layer libraries, like ruby's whenever gem, which won't work on heroku because of some issues with cron/persistence of dynos
Running an infinite loop at app startup - this works, but typically crashes after about 4-8 hours, and I don't know how to get it to start up again without me doing so manually.
Question
How can I reliably run a task every 30 seconds on heroku?
A side note:
I need something that works in multiple languages, for example, both rails and node at the least
You might want to try the clockwork gem.
I've been using it on Heroku for a while now and it can run a task every 30 seconds (even less). The only downside of the gem is that you need to have a specific Dyno that runs the scheduler on your application.
You could use arask. It works in Heroku and is simple to setup.
arask.create task: 'my:awesome_task', interval: 30.seconds
If I have an app on Heroku that consists of one worker and one or no web dynos, will it run? I'm unsure if the absent or idling web dynos will cause the worker dyno not to run.
Heroku doesn't just run web dynos, in fact, it makes no assumptions at all with regards to the processes you're running. There's absolutely nothing wrong with launching a single worker process.
This is actually a common scenario for me to deploy single cron-like tasks to Heroku, I've written about it here http://blog.y3xz.com/blog/2012/11/16/deploying-periodical-tasks-on-heroku/
If you are looking for cron-like tasks for simple jobs (like I am), now you have another alternative: Heroku Scheduler. It is easy to configure in a dashboard.
Advantage:
No need to choose and learn a new scheduler library. Configure it in seconds.
Same way for different platforms: Python, Ruby, etc.
Save Dyno Hours for Free Plan user. Only the actual working time counts. Some scheduler library (like Rufus Scheduler) will keep running between launches (so that it does not rely on cron to work).
Disadvantage:
Trivial options. You can only choose among "Daily"/"Hourly"/"Every 10 minutes".
Conclusion: Best for basic use.
My understanding is that Heroku charges Cron tasks based on the actual amount of time the task runs, using a dyno (based on: https://devcenter.heroku.com/articles/scheduler#dynohour-costs).
So, if I need a quick task to run every X minutes, I could use the cron addon to process it, instead of a worker dyno and I would be charged a fraction of the cost.
So, if everything above is true, what is the use of the Iron.io workers? They charge (about) the same as dynos from the cron jobs and they can't connect to the DB.
I have a feeling that I am missing something.
I'm under the impression that IronWorker can use the databases for Heroku. I may be mistaken about this, however.
I can't claim a great deal of experience with cron tasks on Heroku, so forgive me if I get its limitations wrong. However, I'm under the impression that scaling from one dyno to many dynos is a little bit of a process. Where IronWorker shines is really in enabling the number of worker servers to fluctuate based on demand--you have the capacity to scale at a moment's notice, but are only paying for the scale you're actually using.
I want to schedule jobs to happen at a specific time and date but I'm getting confused by the wide range of options for doing so.
My requirements:
These are not recurring jobs, they only need to happen once at a specified date and time
I'm the only user of the app so don't need to deal with heavy traffic
I would like to minimise the cost of running this on Heroku, i.e. not paying idle dynos
Any tips on which combinations of gems etc. to use?
Have you looked into using https://github.com/bvandenbos/resque-scheduler? You'll need the Redis To Go addon on Heroku. This will cost you $36 a month because you'll need a scheduler process running alongside your web process. However, I've done this for free. See the README here: https://github.com/austinthecoder/pinger.
Good luck!
Due to Heroku Scheduler (default Heroku add-on) doesn't allow you to schedule your job as specific time. It is best to rely on a clock process to do the job. Gem such has Clockwork could be set up please see Heroku's Clockwork guide. You need to combine Clock with a background queuer such as resque or sidekiq. I highly recommend you go for sidekiq. Please bear in mind that both resque and sidekiq requires redis which is offered by redistogo add-on and it will cost you money to run it.
I have a Rails app running on heroku with Rufus Scheduler added on.
A 'once a day' task in the scheduler is running more often than once a day.
My guess would be something to do with the heroku app running on different dynos during the day, but I'm at a loss on how to confirm/fix the problem.
Has anyone else seen this/know of a solution?
Edit: I couldn't resolve the problem with the gem and have moved my app over to the heroku scheduler add on which does not experience this problem.
The Heroku scheduler isn't guaranteed, it's only a simple scheduling system designed to fill a gap. It's nothing to do with your application moving between dynos as it's a seperate management system spinning up one of processes.
If timeliness is essential to you, take a look at clockwork, which will let you configure all sorts of stuff, but also give you a bit more reliability (at the expense of having a clock process running).
If this won't do - simply rework your job so that it doesn't matter how often it runs.