I have a ruby on rails app (1.9.2 and 3.2) running on Heroku with Redis/Resque that requires a rake task be enqueued at regular intervals. Right now I am running 'heroku run rake update_listings' from my local machine once or twice a day....I would like to automate this. I've tried the whenever gem, but the task would not start up in the background. Heroku scheduler seems like the appropriate solution, but I am confused by the scheduler.rb file. I have:
desc "This task is called by the Heroku scheduler add-on"
task :hourly_feed => :environment do
Rake::Task[update_listings].execute
end
When I ran the :hourly_feed task from the Heroku Scheduler console and checked heroku logs, I saw several web dynos get spun up by hirefireapp, but the update_listings rake task was never invoked.
Update: I gave up on resque_scheduler. I am too green to make this work, so trying to use crontab and a sript file. Here is my update.sh script file:
Rake::Task["update_listings"].execute
I set cron using crontab-e and I have it executed every 5 minutes, but I get an error in mail logs:
Projects/livebytransit/update.sh: line 1: Rake::Task[update_listings].execute: command not found
It appears it is finding my update.sh script file and reading it, but it is not executing the code. I noticed the log entry dropped the quotes, so I also tried using single quotes in the shell script file, no change. I also tried changing the update.sh to this:
heroku run rake update_listings
error came back heroku: command not found
Personally, I used resque_scheduler, which will add jobs to the resque / redis queue using cron.
resque_schedule.yml
count_delayed_jobs_job:
cron: "0 */1 * * *"
class: Support::CountDelayedJobsResque
queue: CDJ
args:
description: "count_delayed_jobs_job, every 1hr"
alternatively, you could just chuck Rake::Task["update_listings"].execute in a shell script and use crontab to trigger the job.
It turns out the Heroku Scheduler works perfectly...I simply forgot the quotes around "updated listings".
Related
Currently I have a Ruby and Sinatra project in which I am trying to automate a task. I have followed all the installation steps for the Whenever gem and written a function in my schedule.rb file that looks like this:
set :output, "./cron_log.log"
every 1.minutes do
command "ruby ./scripts/my_script.rb"
end
My project will eventually be deployed to a remote server, but I do not have access to that at the moment. Instead, I am testing it all by running it on my local machine using the command bundle exec shotgun -o 0.0.0.0 -p 9393. When I run it, none of the functions in my schedule.rb file are run. I ran a test by putting require './scripts/my_script.rb' at the top of the schedule.rb file. Whenever I run bundle exec whenever, that outside function will execute, but not the function inside the every block. I figured maybe it only works on a live server, so I ran the bundle exec shotgun -o 0.0.0.0 -p 9393 command to start my server on localhost but nothing in the schedule.rb file was called, including the outside function. I tried putting the command system("bundle exec whenever") in my authentication file, which gets activated every time a user loads the home page of my website, and the outside function in schedule.rb does get called, but the every function does not. Even if it did work like this, I don't want the file to get called every single time a user accesses the home page.
I also tried putting require './config/schedule' in the authentication file and that just completely breaks the website and gives this error message:
Boot Error
Something went wrong while loading config.ru
NoMethodError: undefined method `every' for main:Object
Here is part of the output when running the crontab -l command:
# Begin Whenever generated tasks for: /file_path_redacted/config/schedule.rb at: 2022-10-21 18:50:21 -0500
* * * * * /bin/bash -l -c 'ruby ./scripts/my_script.rb >> ./cron_log.log 2>&1'
# End Whenever generated tasks for: /file_path_redacted/config/schedule.rb at: 2022-10-21 18:50:21 -0500
So, my question is this: How and when is the schedule.rb file supposed to get called? And is it important that I deploy the project to the remote server for the cron job to work? Again, it is not possible for me to deploy it, as it must be fully working before being deployed. If it is possible to run on localhost, what am I doing wrong?
So, if I run the command heroku ps:restart event_machine.1 --app app-name I get what I want. However, I'm trying to automate our travis-ci deploy process. What needs to happen is the following:
We have a successful test run.
Next, we deploy the code
If we deploy the code successfully, we need to execute a few rake tasks that tell an external service to rebuild it self.
Once this is fired off, we need to restart the heroku app. In travis, ideally, this would be executed on the heroku machine via a deploy run command. This would be done in much the same way that we run bundle exec db:migrate.
Does anyone have any thoughts on how we we can restart a particular dyno(s) via a command that can be ran via heroku run something as that is what travis is executing in the deploy run.
So, to answer this we had a procfile that is executing a rake command to spin up event machine. We've modified this at the proc file level to first tell the external service to rebuild it self, before starting the event machine. This takes travis completely out of the deployment loop, which is better because it allows Heroku and Travis to each do what they should be responsible for.
I've recently installed https://addons.heroku.com/scheduler in my heroku app, but I just cannot make any instruction get to work.
I think I don't know the correct syntax, for now I've tried with heroku pgbackups:capture --expire --app running-app command and selected frequency 10 mins.
It's been more than an hour and still hasn't done anything.
How can I get this to work?
Thanks
EDIT: This command is an example command, nor that I want to use that one specifically
You should be using the pgbackups addon, https://addons.heroku.com/pgbackups which is scheduled for you based on the level you pick.
Heroku Scheduler is more for if you need to run a rack task within your application codebase.
UPDATE BASED ON REVISED QUESTION:
You would write a rake task (assuming your using Rails?) which you could run locally using rake taskname and to schedule it on Heroku you would enter rake taskname as the command in the scheduler page for them to execute it.
I have some miniapp that use delayed_job. On my localhost everything works fine, but when I deploy my app to Heroku and click on the link that should be executed by delayed_job, so nothing happen, the "task" is just saved into the table delayed_job.
In this article on heroku blog is written, that the task from delayed_job table is executed, when is run this command rake jobs:work.
But how can I run this command? Where should be the command placed? In the code, or from terminal console?
If you are running the Cedar stack, run the following from the terminal console:
heroku run rake jobs:work
If you are running the older stacks (Bamboo, Aspen, etc.):
heroku rake jobs:work
see: https://devcenter.heroku.com/articles/rake
According to the delayed_job documentation, you can also start a worker programmatically:
#!/usr/bin/env ruby
require File.dirname(__FILE__) + '/../config/environment'
Delayed::Worker.new.start
You should use a Procfile to scpecify the commands for your dynos.
For example you would have something like this in your Procfile:
appDir/Procfile
web: bundle exec rails server -p $PORT
worker: bundle exec rake jobs:work
To use this on your development machine, you should use Foreman, it's all explained at the docs.
https://devcenter.heroku.com/articles/procfile
In our case we're only running a delayed job once a month, so didn't want to have a worker dyno running constantly.
To solve this we queue up the job (with .delayed) and then use the Heroku platform API to spawn rake jobs:workoff in a one-off worker. The API call returns relatively quickly.
PlatformAPI.connect_oauth(ENV["YOUR_HEROKU_KEY"]).dyno.create(ENV["YOUR_HEROKU_APP_NAME"],{command: 'rake jobs:workoff'})
Questions
1) When running cron tasks at Heroku, does Heroku have a time limit that cron tasks should last only X minutes or can the cron task run for whatever time it needs to complete processing?
2) To optimize my application, I want to move all my processing to batch mode. So I have lots of update queries. Basically that one Heroku cron file is getting messy. What can I do to clean up the code? Should I create multiple rake tasks and invoke the task tasks from the cron file?
2.1) If you agree with that then how do I invoke rake tasks from cron task? Let's say there are 3 independent rake tasks rake accounts:billing, rake accounts:collections, rake accounts:cleanup. How do I call them in cron file?
There aren't any time limits that I know of.
Using multiple tasks instead of one big cron task is a good idea. Not only is it easier to debug and maintain when they're separate but you can also easily run them individually when needed. And well factored code is a just a good idea all by itself.
Executing a task from another task is as simple as Rake::Task[task].execute. You'd have something like this:
desc 'Heroku cron job'
task :cron => :environment do
%w{accounts:billing accounts:collections accounts:cleanup}.each do |task|
Rake::Task[task].execute
end
end