I have been using resque for background processing, No my problem with code is :
- when I start rake task as "rake resque:work QUEUE=''" as per ryan bates episode no. 271. in remote server the code inside worker class for file maipulation works properly without any filepath issues and I/O errors.
- when i start rake task as "rake resque:work QUEUE='' BACKGROUND=yes" now, the code inside worker class gives "failed:Errno::EIO: Input/output error # io_write - >" error.
Now my question is I want to start the resque queue above rake command only one time and why second point giving error is this issue with filepaths if so then why it runs smoothly as mention in point first.
You can use god to manage your background process. Or nohup can be your solution too as below:
$ nohup bundle exec rake resque:work QUEUE=queue_name PIDFILE=tmp/pids/resque_worker_QUEUE.pid & >> log/resque_worker_QUEUE.log 2>&1
and even this command worked for me:
PIDFILE=./resque.pid BACKGROUND=yes QUEUE="*" rake resque:work >> worker1.log &
Hope that will help you too.
Related
I have a rake task like below. However, after the first system line runs and User.find_each(&:save) finishes the heroku console stays open and the ruby script does not proceed to the next system call as the first one is not "finished". How do I exit out of the first system call (and proceed to the next) after the User records are done saving?
task :production do
Bundler.with_clean_env do
system("echo 'User.find_each(&:save)' | heroku run console --remote production")
system("echo 'Post.find_each(&:save)' | heroku run console --remote production")
end
end
You're running something (rails console) that doesn't terminate.
What happens when you run rails console locally? It won't terminate until you type `exit'.
So try this:
echo 'User.find...; exit' | heroku run console
I have rake task which runs in every 5 mins with crontab. This rake task has to start one long running background process which creates a file as per task inputs.
For some reason long process is not running . I am using Backticks to run the command.
Any idea ?
rake task file
require 'json'
namespace :scrapper do
path = '/home/ubuntu/users'
def lockfile
Rails.root.join('tmp', 'pids', 'leads_task.lock')
end
def running!
`touch #{lockfile}`
end
def done!
`rm #{lockfile}`
end
def running?
File.exists?(lockfile)
end
task :get_profile => :environment do
unless running?
running!
user = User.last
`/usr/local/bin/casperjs #{path}/scrapper.coffee #{user.id}`
done!
end
end
end
cron syntax looks like this
*/5 * * * * /bin/bash -l -c 'cd /home/ubuntu/UserProfile && RAILS_ENV=staging bundle exec rake scrapper:get_profile --silent >> /home/ubuntu/UserProfile/scrapper_cron.output 2>&1'
few wried things
The cron job is running, i see scrapper_cron.out file get created when i delete it.
The rake task is running when i type it in shell and execute it.
I have rake timeout gem in my app which is giving timeout error to the rake task. Since, the cron command includes silent flag it is suppressing the error .
I have these rake tasks that will occasionally fail. I want to use monit to monitor them and to restart them if necessary.
I have read the other ruby/monit threads on StackOverflow. My case is different in that these programs require my Rails environment in order to work. That's why I have them as rake tasks now.
Here is one of the tasks I need to monitor, in it's entirety:
task(process_updates: :environment) do
`echo "#{Process.pid}" > #{Rails.root}/log/process_alerts.pid`
`echo "#{Process.ppid}" > #{Rails.root}/log/process_alerts.ppid`
SynchronizationService::process_alerts
end
My question is, do I leave this as a rake task, since SynchronizationService::process_alerts requires the Rails environment to work? Or is there some other wrapper I should invoke and then just run some *.rb file?
Monit can check for running pid, since you're creating pid when you run task you can create a monit config which should look something like this:
check process alerts with pidfile RAILSROOT/log/process_alerts.pid
start program = "cd PATH_TO_APP; rake YOURTASK" with timeout 120 seconds
alert your#mail.com on { nonexist, timeout }
Of course RAILSROOT, PATH_TO_APP, YOURTASK should correspond to your paths/rake task.
Monit then will check for running process in system using the pidfile value and will start the process using start program command if it can't find running process.
I'm trying to get whenever to work with sinatra. When I run the whenever command, I get the generated cron tab. But the problem is, that in my sinatra app, I don't have a script/runner file, which is present in Rails.
How do I get this runner, or is there a whenever command to generate one?
thx!
You can use a rake task in place of script/runner. The Whenever gem supports defining the job via a rake task (and more in fact)
Sample:
# config/schedule.rb
every 3.hours do
rake "destroy_all"
end
and in your Rakefile: (for lack of good examples)
task :destroy_all do
puts "Do not do this"
# sh "rm -rf ."
end
I have a foreman script starting up some workers on a standalone ruby app. Here's the script
Foreman script
worker: bundle exec rake resque:work BACKGROUND=true QUEUE=image VERBOSE=true
When I run the script this is the output I get.
$ foreman start
22:00:38 worker.1 | started with pid 882
22:00:38 worker.1 | exited with code 0
22:00:38 system | sending SIGTERM to all processes
SIGTERM received
The process seems to have exited but when I look at the ps -eaf | grep resque log it shows a resque worker running with pid 884. I've tested this and its always a pid +2 than the original.
When I run the bundle exec command straight from the terminal without foreman, the command executes just fine. Is there anything I'm missing with the foreman script?
So apparently, when running BACKGROUND=true the resque workers get daemonized and therefore the original pid gets deleted and a new one gets spanwed as an orphan process for the worker.
Still, there is an issue when creating 2 background workers with foreman because once one of the workers gets daemonized, foreman will end all processes and only one daemonized worker will be created instead of two.
You should not daemonize the workers with foreman - foreman needs to have all the processes running in the foreground. If you want multiple workers, simply use something like this:
image_worker: bundle exec rake resque:work QUEUE=image VERBOSE=true
other_worker: bundle exec rake resque:work QUEUE=other VERBOSE=true
To start multiple workers on the same queue:
foreman start -m image_worker=2