I have set up delayed paperclip and sidekiq, however when I try the photo upload again, the log's out put is below:
[ActiveJob] Enqueued DelayedPaperclip::Jobs::ActiveJob (Job ID: f9bf0100-2c5b-4c30-af04-9fb9b4ee8c5e) to Inline(paperclip) with arguments: "CarPhoto", 32, "image"
[ActiveJob] [DelayedPaperclip::Jobs::ActiveJob] [f9bf0100-2c5b-4c30-af04-9fb9b4ee8c5e] Performing DelayedPaperclip::Jobs::ActiveJob from Inline(paperclip) with arguments: "CarPhoto", 32, "image"
[ActiveJob] [DelayedPaperclip::Jobs::ActiveJob] [f9bf0100-2c5b-4c30-af04-9fb9b4ee8c5e] CarPhoto Load (0.5ms) SELECT "car_photos".* FROM "car_photos" WHERE "car_photos"."id" = $1 LIMIT 1 [["id", 32]]
which indicates that Activejob is doing the job.. and on the other side in the sidekiq console, I don't see anything being logged out.. so I am wondering if I am missing anything.
You haven't told ActiveJob to use Sidekiq so it is using the inline adapter.
# config/application.rb
module YourApp
class Application < Rails::Application
# Be sure to have the adapter's gem in your Gemfile and follow
# the adapter's specific installation and deployment instructions.
config.active_job.queue_adapter = :sidekiq
end
end
Related
I've spent several days on this and about 100 hours but can't get the fix.
Here's my setup (using Rails 4.2.8)
class CustomJob < ActiveJob::Base
def perform(*args)
filename = args.first
data = File.read(filename)
# process that data
end
end
When I run Delayed::Job.enqueue CustomJob.new('filename'), I get the error mentioned in the subject. The job is created and added to the db, but the error message is "Job failed..."
I have this line:
require 'custom_job'
in several places including script/delayed_job.rb, config/initializers/delayed_jobs.rb, config/initializers/custom_job.rb and the file in which I'm calling the job.
I also added this:
config.autoload_paths+=Dir[Rails.root.join('app','jobs')]
config.active_job.queue_adapter = :delayed_job
to my config/application.rb file
And this:
config.threadsafe! unless defined? ($rails_rake_task) && $rails_rake_task
I've also restarted my server after every change. And verified that delayed_job was running using:
dir$ RAILS_ENV=development script/delayed_job status
delayed_job: running [pid 64503]
Sources:
delayed_job fails jobs when running as a daemon. Runs fine when using rake jobs:work
DelayedJob: "Job failed to load: uninitialized constant Syck::Syck"
Rails Custom Delayed Job - uninitialized constant
model classes not loading in Delayed Job when using threadsafe
Can you also try adding this line into config/application.rb file
config.eager_load_paths+=Dir[Rails.root.join('app','jobs')]
I always feel like the answer is obvious...AFTER I figure it out.
The problem was that I was using a shared database and there were existing workers accessing this DB. Though I was restarting and refreshing my local instance of the server, the other instances were trying to run my jobs and the OTHER workers were causing the error, not my local instance.
Solution: Ensure that other instances of delayed_job are using the same table as the code you're testing/building/using. If so, use another DB if possible.
I'm trying to update an app (Rails 4.1.12; ruby 1.9.3p484) by deploying changes from development to production using capistrano. Below is log (I masked the IP address and suggested secret key).
Previously never had a problem running a deploy. Have searched and found posts with similar errors but have not found a solution. I did follow the instructions to add Devise.secret_key to the Devise initializer and restarted the application (dev and prod). Next attempt to deploy received same error with suggestion for different secret key.
All help, hints, suggestions are greatly appreciated. Thanks.
UPDATE:I just tried to deploy using
bundle exec cap production deploy
and did not get errors but still get errors when running
cap production deploy
UPDATE 2: I used Figaro to generate an application.yml file (and append to gitignore). In the devise.rb file I created a line config.secret_key = ENV["config.secret_key"] In the application.yml file I created a line config.secret_key: 'random_128_chr_string' I then restarted the development server and ran cap production deploy Unfortunately got same error.
cap production deploy
.
.
.
[deploy:migrate] Run `rake db:migrate`
[e0c2c251] Running bundle exec rake db:migrate as deployer#123.123.123.12
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as deployer#123.123.123.12: rake exit status: 1
rake stdout: Nothing written
rake stderr: rake aborted!
Devise.secret_key was not set. Please add the following to your Devise initializer:
config.secret_key = 'looong_string_of_numbers_and_letters'
Please ensure you restarted your application after installing Devise or setting the key.
deploy.rb:
lock '>=3.2.1'
set :application, 'pepperslice'
set :repo_url, 'git#github.com:jeffaltman/pepperslice.git'
set :linked_dirs, %w{log tmp public/images/uploaded}
set :linked_files, %w{config/database.yml config/initializers/devise.rb config/initializers/smtp_settings.rb}
set :format, :pretty
set :log_level, :info
namespace :deploy do
desc 'Restart application'
task :restart do
on roles(:app), in: :sequence, wait: 5 do
execute "sudo /usr/sbin/service unicorn restart"
end
end
after :publishing, :restart
after :restart, :clear_cache do
on roles(:web), in: :groups, limit: 3, wait: 10 do
# Here we can do anything such as:
# within release_path do
# execute :rake, 'cache:clear'
# end
end
end
end
capfile:
# Load DSL and Setup Up Stages
require 'capistrano/setup'
# Includes default deployment tasks
require 'capistrano/deploy'
# Includes tasks from other gems included in your Gemfile
#
# For documentation on these, see for example:
#
# https://github.com/capistrano/rvm
# https://github.com/capistrano/rbenv
# https://github.com/capistrano/chruby
# https://github.com/capistrano/bundler
# https://github.com/capistrano/rails
#
# require 'capistrano/rvm'
# require 'capistrano/rbenv'
# require 'capistrano/chruby'
require 'capistrano/bundler'
# require 'capistrano/rails/assets'
require 'capistrano/rails/migrations'
#require 'capistrano/ssh_doctor'
# Loads custom tasks from `lib/capistrano/tasks' if you have any defined.
Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }
Is there any way to debug a daemon in Ruby Mine. The way I am calling this file is like this
Daemons.run(file, options)
where file is the name of file and options is the bunch of options that I am passing to the file.
I'm also trying to debug a Rails daemon (gem daemons). The ActiveMessaging poller script in particular. However the ruby-debug-ide fails to attach due to various reasons (race conditions, etc).
The solution I came up with is to use gem rails-daemons. This works with ActiveMessaging and allows to me to create all kinds of daemons, and importantly, be able to debug them in RubyMine
Workaround
Use https://github.com/mirasrael/daemons-rails to create a new daemon. This works with RubyMine 6.3.3, Rails 4, Ruby 2.1.2 on OSX 10.8.x
Sample Daemon
mqpoller.rb (after running rails generate daemon mqpoller)
#!/usr/bin/env ruby
# You might want to change this
ENV["RAILS_ENV"] ||= "development"
root = File.expand_path(File.dirname(__FILE__))
root = File.dirname(root) until File.exists?(File.join(root, 'config'))
Dir.chdir(root)
require File.join(root, "config", "environment")
$running = true
Signal.trap("TERM") do
$running = false
end
while($running) do
# Replace this with your code
Rails.logger.auto_flushing = true
Rails.logger.info "This daemon is still running at #{Time.now}.\n"
# ADDED MY CODE BELOW HERE TO START ActiveMessaging
Rails.logger = Logger.new(STDOUT)
ActiveMessaging.logger = Rails.logger
# Load ActiveMessaging
ActiveMessaging::load_processors
# Start it up!
ActiveMessaging::start
sleep 10
end
I am developing a Ruby gem that I would like to add NewRelic monitoring to. The gem is used in a script that is run as a daemon and monitored by bluepill. I followed "Monitoring Ruby background processes and daemons" to get started.
I confirmed the gem is establishing a connection with NewRelic as the application shows up in my portal there, however, there is no transaction traces or any metrics breakdown of the code being invoked.
Here's the "entry" point of my gem as I tried to manually start the agent around the invoking method:
require 'fms/parser/version'
require 'fms/parser/core'
require 'fms/parser/env'
require 'mongoid'
ENV['NRCONFIG'] ||= File.dirname(__FILE__) + '/../newrelic.yml'
require 'newrelic_rpm'
module Fms
module Parser
def self.prepare_parse(filename)
::NewRelic::Agent.manual_start
Mongoid.load!("#{File.dirname(__FILE__)}/../mongoid.yml", :development)
Core.prepare_parse(filename)
::NewRelic::Agent.shutdown
end
end
end
I also tried adding this into the module:
class << self
include ::NewRelic::Agent::Instrumentation::ControllerInstrumentation
add_transaction_tracer :prepare_parse, :category => :task
end
I'm not entirely sure what else I can do. I confirmed the agent is able to communicate with the server and transaction traces are enabled. Nothing shows up in the background application tab either.
This is the most useful information I've gotten from the agent log so far:
[12/23/13 21:21:03 +0000 apivm (7819)] INFO : Environment: development
[12/23/13 21:21:03 +0000 apivm (7819)] INFO : No known dispatcher detected.
[12/23/13 21:21:03 +0000 apivm (7819)] INFO : Application: MY-APP
[12/23/13 21:21:03 +0000 apivm (7819)] INFO : Installing Net instrumentation
[12/23/13 21:21:03 +0000 apivm (7819)] INFO : Finished instrumentation
[12/23/13 21:21:04 +0000 apivm (7819)] INFO : Reporting to: https://rpm.newrelic.com/[MASKED_ACCOUNT_NUMBER]
[12/23/13 22:12:06 +0000 apivm (7819)] INFO : Starting the New Relic agent in "development" environment.
[12/23/13 22:12:06 +0000 apivm (7819)] INFO : To prevent agent startup add a NEWRELIC_ENABLE=false environment variable or modify the "development" section of your newrelic.yml.
[12/23/13 22:12:06 +0000 apivm (7819)] INFO : Reading configuration from /var/lib/gems/1.9.1/gems/fms-parser-0.0.6/lib/fms/../newrelic.yml
[12/23/13 22:12:06 +0000 apivm (7819)] INFO : Starting Agent shutdown
The only thing that's really concerning here is "No known dispatcher detected".
Is what I'm trying to do possible?
I work at New Relic and wanted to add some up-to-date details about the latest version of the newrelic_rpm gem. TrinitronX is on the right track, but unfortunately that code sample and blog post is based on a very old version of the gem, and the internals have changed significantly since then. The good news is that newer versions of the agent should make this simpler.
To start off, I should say I'm assuming that your process stays alive for a long time as a daemon, and makes repeated calls to prepare_parse.
Generally speaking, the explicit manual_start and shutdown calls you have inserted into your prepare_parse method should not be necessary - except for a few special cases (certain rake tasks and interactive sessions). The New Relic agent will automatically start as soon as it is required. You can see details about when the Ruby agent will automatically start and how to control this behavior here:
https://docs.newrelic.com/docs/ruby/forcing-the-ruby-agent-to-start
For monitoring background tasks like this, there are conceptually two levels of instrumentation that you might want: transaction tracers and method tracers. You already have a transaction tracer, but you may also want to add method tracers around the major chunks of work that happen within your prepare_parse method. Doing so will give you better visibility into what's happening within each prepare_parse invocation. You can find details about adding method tracers here:
https://docs.newrelic.com/docs/ruby/ruby-custom-metric-collection#method_tracers
With the way that you are calling add_transaction_tracer, your calls to prepare_parse should show up as transactions on the 'Background tasks' tab in the New Relic UI.
The one caveat here may be the fact that you're running this as a daemon. The Ruby agent uses a background thread to asynchronously communicate with New Relic servers. Since threads are not copied across calls to fork(), this means you will sometimes have to manually re-start the agent after a fork() (note that Ruby's Process.daemon uses fork underneath, so it's included as well). Whether or not this is necessary depends on the relative timing of the require of newrelic_rpm and the call to fork / daemon (if newrelic_rpm isn't required until after the call to fork / daemon, you should be good, otherwise see below).
There are two solutions to the fork issue:
Manually call NewRelic::Agent.after_fork from the forked child, right after the fork.
If you're using newrelic_rpm 3.7.1 or later, there's an experimental option to automatically re-start the background thread that you can enable in your newrelic.yml file by setting restart_thread_in_children: true. This is off by default at the moment, but may become the default behavior in future versions of the agent.
If you're still having trouble, the newrelic_agent.log file is your best bet to debugging things. You'll want to increase the verbosity by setting log_level: debug in your newrelic.yml file in order to get more detailed output.
For debugging this problem, try the following code:
require 'fms/parser/version'
require 'fms/parser/core'
require 'fms/parser/env'
require 'mongoid'
ENV['NRCONFIG'] ||= File.dirname(__FILE__) + '/../newrelic.yml'
# Make sure NewRelic has correct log file path
ENV['NEW_RELIC_LOG'] ||= File.dirname(__FILE__) + '/../log/newrelic_agent.log'
require 'newrelic_rpm'
::NewRelic::Agent.manual_start
# For debug purposes: output some dots until we're connected to NewRelic
until NewRelic::Agent.connected? do
print '.'
sleep 1
end
module Fms
module Parser
class << self
include ::NewRelic::Agent::Instrumentation::ControllerInstrumentation
add_transaction_tracer :prepare_parse, :category => :task
end
def self.prepare_parse(filename)
Mongoid.load!("#{File.dirname(__FILE__)}/../mongoid.yml", :development)
Core.prepare_parse(filename)
# Force the agent to prepare data before we shutdown
::NewRelic::Agent.load_data
# NOTE: Ideally you'd want to shut down the agent just before the process exits... not every time you call Fms::Parser#prepare_parse
::NewRelic::Agent.shutdown(:force_send => true)
end
end
end
I have a feeling that this probably has something to do with running your gem's code within the daemonized process that bluepill is starting. Ideally, we'd want to start the NewRelic agent within the process as soon after the daemon process is forked as we can get. Putting it after your library's requires should do this when the file is required.
We also would most likely want to stop the NewRelic agent just before the background task process exits, not every time the Fms::Parser#prepare_parse method is called. However, for our purposes this should get you enough debugging info to continue, so you can ensure that the task is contacting New Relic the first time it's run. We can also try using :force_send => true to ensure we send the data.
References:
Blog Post: Instrumenting your monitoring checks with New Relic
I'm trying to follow the AppFog guide on creating a background worker in ruby, and I'm running into some (probably noob) issues. The example uses Rufus-scheduler, which (according to the Ruby docs on AppFog) means I need to use Bundler to include/manage within my app. Nonetheless, I've run bundle install, pushed everything to AppFog in the appropriate ('standalone') fashion, and still can't seem to get it running.
my App & Gemfile:
...and via the AF CLI:
$ af push
[...creating/uploading/etc. etc... - removed to save space]
Staging Application 'chservice-dev': OK
Starting Application 'chservice-dev': .
Error: Application [chservice-dev] failed to start, logs information below.
====> /logs/staging.log <====
# Logfile created on 2013-06-27 20:22:23 +0000 by logger.rb/25413
Need to fetch tzinfo-1.0.1.gem from RubyGems
Adding tzinfo-1.0.1.gem to app...
Adding rufus-scheduler-2.0.19.gem to app...
Adding bundler-1.1.3.gem to app...
====> /logs/stdout.log <====
2013-06-27 20:22:28.841 - script executed.
Delete the application? [Yn]:
How can I fix (or troubleshoot) this? I'm probably missing a large step/concept... very new to ruby =)
Thanks in advance.
I think the app might be exiting immediately. The scheduler needs to be joined to the main thread in order to keep that app running.
require 'rubygems'
require 'rufus/scheduler'
scheduler = Rufus::Scheduler.start_new
scheduler.every '10s' do
puts 'Log this'
end
### join the scheduler to the main thread ###
scheduler.join
I created a sample rufus scheduler app that works on appfog: https://github.com/tsantef/appfog-rufus-example