unique_job_worker.rb
# -*- encoding : utf-8 -*-
require_relative 'logging_helper'
class UniqueJobWorker
include Sidekiq::Worker
include WorkerHelper
sidekiq_options retry: false,
backtrace: true,
queue: :sender,
failures: true
def perform(worker,campaign_guid, queue)
require'pry';binding.pry
end
end
unique_job_worker_test.rb
require 'test_helper'
require 'mocha/setup'
class UniqueJobWorkerTest < ActiveSupport::TestCase
def setup
require'pry';binding.pry
#worker = UniqueJobWorker.new
end
test "it exists" do
assert #worker
end
end
When enqueued through redis I get this response
INFO -- : Exception: uninitialized constant UniqueJobWorker
Any suggestions as to why my newly created worker, UniqueJobWorker, is not being found during runtime through redis or through a simple test?
Thanks ahead of time!
When you use sidekiq outside of Rails, you need to use the -r option to tell it how to load your workers. So (assuming that your worker is in a sub-directory called workers):
% sidekiq -r ./workers/unique_job_worker.rb
If you have multiple workers, an option is to create a loader file to ensure everything is loaded.
load_workers.rb
($LOAD_PATH << 'workers').uniq!
require 'unique_job_worker'
require 'other_worker'
...
Then require the loader file on the command line:
% sidekiq -r ./load_workers.rb
I had the same issue and ended up being a Redis namespace issue:
https://github.com/mperham/sidekiq/issues/2834#issuecomment-184800981
Adding that fixed it for me:
config.redis = {
url: ENV['REDIS_URL'],
namespace: "some_namespace_different_for_each_app"
}
You also need the redis-namespace gem BTW
Related
Is it possible to write MongoDB document via Mongoid from Sidekiq worker?
# lib/worker.rb
require 'sidekiq'
require 'model' # lib/model.rb
class AwesomeWorker
include Sidekiq::Worker
def perform
RandomNumberModel.create! { random_number: Random.new.rand(1..100) }
end
end
When I run Sidekiq it returns uninitialized constant AwesomeWorker::RandomNumberModel
What I'm doing wrong?
try ::RandomNumberModel.create! { random_number: Random.new.rand(1..100) }
if it fails, you did mistake in path to required file
Might it help of anyone :)
You have to create your worker under app > workers folder. Then every model will be easily accessible there.
# app/workers
class AwesomeWorker
include Sidekiq::Worker
def perform
RandomNumberModel.create! { random_number: Random.new.rand(1..100) }
end
end
So, I'm trying to build a quick console program for my development needs, akin to rails console (I'm using Sinatra + DataMapper + pry).
I run it and launch cat = Category.new(name: 'TestCat', type: :referential). It gives me the following error:
Error: Cannot open "/home/art-solopov/Projects/by-language/Ruby/billy-bones/=" for reading.
What could be the cause of the problem?
console:
#!/usr/bin/env ruby
$LOAD_PATH << 'lib'
require 'pry'
require 'config'
binding.pry
lib/config.rb:
# Configuration files and app-wide requires go here
require 'sinatra'
require 'data_mapper'
require 'model/bill'
require 'model/category'
configure :production do
DataMapper::Logger.new('db-log', :debug)
DataMapper.setup(:default,
'postgres://billy-bones:billy#localhost/billy-bones')
DataMapper.finalize
end
configure :development do
DataMapper::Logger.new($stderr, :debug)
DataMapper.setup(:default,
'postgres://billy-bones:billy#localhost/billy-bones-dev')
DataMapper.finalize
DataMapper.auto_upgrade!
end
configure :test do
require 'dm_migrations'
DataMapper::Logger.new($stderr, :debug)
DataMapper.setup(:default,
'postgres://billy-bones:billy#localhost/billy-bones-test')
DataMapper.finalize
DataMapper.auto_migrate!
end
lib/model/category.rb:
require 'data_mapper'
class Category
include DataMapper::Resource
property :id, Serial
property :name, String
property :type, Enum[:referential, :predefined, :computable]
has n, :bills
# has n, :tariffs TODO uncomment when tariff ready
def create_bill(params)
# A bill factory for current category type
case type
when :referential
ReferentialBill.new params
when :predefined
PredefinedBill.new params
when :computable
ComputableBill.new params
end
end
end
If I substitute pry with irb in the console script, it goes fine.
Thank you very much!
P. S.
Okay, yesterday I tried this script again, and it worked perfectly. I didn't change anything. I'm not sure whether I should remove the question now or not.
P. P. S.
Or actually not... Today I've encountered it again. Still completely oblivious to what could cause it.
** SOLVED **
DAMN YOU PRY!
Okay, so here's the difference.
When I tested it the second time, I actually entered a = Category.new(name: 'TestCat', type: :referential) and it worked. Looks like pry just thinks cat is a Unix command, not a valid variable name.
Not answer to the pry question I just generally hate case statements in ruby.
Why not change:
def create_bill(params)
# A bill factory for current category type
case type
when :referential
ReferentialBill.new params
when :predefined
PredefinedBill.new params
when :computable
ComputableBill.new params
end
end
to:
def create_bill(params)
# A bill factory for current category type
self.send("new_#{type}_bill",params)
end
def new_referential_bill(params)
ReferentialBill.new params
end
def new_predefined_bill(params)
PredefinedBill.new params
end
def new_computable_bill(params)
ComputableBill.new params
end
You could make this more dynamic but I think that would take away from readability in this case but if you'd like in rails this should do the trick
def create_bill(params)
if [:referential, :predefined, :computable].include?(type)
"#{type}_bill".classify.constantize.new(params)
else
#Some Kind of Handling for non Defined Bill Types
end
end
Or this will work inside or outside rails
def create_bill(params)
if [:referential, :predefined, :computable].include?(type)
Object.const_get("#{type.to_s.capitalize}Bill").new(params)
else
#Some Kind of Handling for non Defined Bill Types
end
end
I'm working in a Sinatra application using Sequel.
I want to make a transaction, according to the manual I have to use the DB object, how can I get this object from any part of my code?
You can define it in your base app.rb (or equivalent) or include a separate file where you configure the DB object if you wish.
For example, in one of my Sinatra apps, I have an app.rb that includes a
class App < Sinatra::Application
#lots of stuff here...
end
require_relative 'models/init'
In my models/init.rb I configure DB
require 'sequel'
conf = YAML.load(File.open(File.expand_path('./config/dbconn.yml')))
env = ENV['RACK_ENV'] || 'development'
DB = Sequel.connect(host:conf['database'][env]['host'],
port:conf['database'][env]['port'],
database:conf['database'][env]['schema'],
username:conf['database'][env]['username'],
password:conf['database'][env]['password'],
adapter:conf['database'][env]['adapter'],
encoding:conf['database'][env]['encoding'])
raise "Unable to connect to #{conf['database'][env]['host']}" unless DB.test_connection
...
That's one way. Hope it helps.
You mention that you want to reference from any part of your code; however I've found that encapsulated within the models is where I tend to wrap transactions; and from there it's relatively easy:
class X < Sequel::Model
def self.y
self.db.transaction {
...
end
end
def z
db.transaction {
...
}
end
end
I'm considering using IronWorker a project so that I can scale it easily (high traffic expected, with lots of background jobs).
In order to stay DRY, I'm trying to define workers using inheritance but I keep getting the following error:
/usr/local/lib/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- base_worker.rb (LoadError)
from /usr/local/lib/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from /task/child_worker.rb:3:in `<top (required)>'
from /task/runner.rb:344:in `require_relative'
from /task/runner.rb:344:in `<main>'
Here is the base worker class:
# app/workers/base_worker.rb
require 'net/http'
require 'uri'
require 'json'
class BaseWorker < IronWorker::Base
attr_accessor :params
# The run method is what IronWorker calls to run your worker
def run
data = custom_run(params)
common_post_process(data)
end
def custom_run(params)
#to be overwritten in the child class
end
def common_post_process(data)
# some common post processing => DRY
....
end
end
And here is a child class :
# app/workers/child_worker.rb
require 'net/http'
require 'uri'
require 'base_worker.rb'
class ChildWorker < BaseWorker
merge "base_worker.rb"
def custom_run(params)
#custom work
end
end
Any idea on how to fix this?
I'd recommend using our next generation gem, iron_worker_ng: https://github.com/iron-io/iron_worker_ruby_ng . The iron_worker gem is deprecated. And if you want to keep it similar style to what you have, your child_worker.rb might look like this:
require 'net/http'
require 'uri'
require_relative 'base_worker.rb'
class ChildWorker < BaseWorker
def custom_run(params)
#custom work
end
end
# NG gem doesn't run anything in particular, so to run your method:
cw = ChildWorker.new
cw.custom_run(params)
And in a child_worker.worker file:
runtime 'ruby'
file 'base_worker.rb'
exec 'child_worker.rb'
Then to upload it to IronWorker:
iron_worker upload child_worker
Then you can start queuing jobs for it:
worker = IronWorkerNG::Client.new
worker.tasks.create("child_worker", params)
If you use iron_worker_ng, it also possible to define a run method. This method will be called when the IronWorker runs. You have to specify the Class within the .worker file.
# child_worker.rb
class ChildWorker
def run
puts "doing the hard work"
end
end
And the child_worker.worker file:
# child_worker.worker
runtime 'ruby'
name 'ChildWorker'
exec 'child_worker.rb', 'ChildWorker'
I'm having trouble figuring out how to log messages with Sinatra. I'm not looking to log requests, but rather custom messages at certain points in my app. For example, when fetching a URL I would like to log "Fetching #{url}".
Here's what I'd like:
The ability to specify log levels (ex: logger.info("Fetching #{url}"))
In development and testing environments, the messages would be written to the console.
In production, only write out messages matching the current log level.
I'm guessing this can easily be done in config.ru, but I'm not 100% sure which setting I want to enable, and if I have to manually create a Logger object myself (and furthermore, which class of Logger to use: Logger, Rack::Logger, or Rack::CommonLogger).
(I know there are similar questions on StackOverflow, but none seem to directly answer my question. If you can point me to an existing question, I will mark this one as a duplicate).
Sinatra 1.3 will ship with such a logger object, exactly usable as above. You can use edge Sinatra as described in "The Bleeding Edge". Won't be that long until we'll release 1.3, I guess.
To use it with Sinatra 1.2, do something like this:
require 'sinatra'
use Rack::Logger
helpers do
def logger
request.logger
end
end
I personally log in Sinatra via:
require 'sinatra'
require 'sequel'
require 'logger'
class MyApp < Sinatra::Application
configure :production do
set :haml, { :ugly=>true }
set :clean_trace, true
Dir.mkdir('logs') unless File.exist?('logs')
$logger = Logger.new('logs/common.log','weekly')
$logger.level = Logger::WARN
# Spit stdout and stderr to a file during production
# in case something goes wrong
$stdout.reopen("logs/output.log", "w")
$stdout.sync = true
$stderr.reopen($stdout)
end
configure :development do
$logger = Logger.new(STDOUT)
end
end
# Log all DB commands that take more than 0.2s
DB = Sequel.postgres 'mydb', user:'dbuser', password:'dbpass', host:'localhost'
DB << "SET CLIENT_ENCODING TO 'UTF8';"
DB.loggers << $logger if $logger
DB.log_warn_duration = 0.2
If you are using something like unicorn logging or other middleware that tails IO streams, you can easily set up a logger to STDOUT or STDERR
# unicorn.rb
stderr_path "#{app_root}/shared/log/unicorn.stderr.log"
stdout_path "#{app_root}/shared/log/unicorn.stdout.log"
# sinatra_app.rb
set :logger, Logger.new(STDOUT) # STDOUT & STDERR is captured by unicorn
logger.info('some info') # also accessible as App.settings.logger
this allows you to intercept messages at application scope, rather than just having access to logger as request helper
Here's another solution:
module MySinatraAppLogger
extend ActiveSupport::Concern
class << self
def logger_instance
#logger_instance ||= ::Logger.new(log_file).tap do |logger|
::Logger.class_eval { alias :write :'<<' }
logger.level = ::Logger::INFO
end
end
def log_file
#log_file ||= File.new("#{MySinatraApp.settings.root}/log/#{MySinatraApp.settings.environment}.log", 'a+').tap do |log_file|
log_file.sync = true
end
end
end
included do
configure do
enable :logging
use Rack::CommonLogger, MySinatraAppLogger.logger_instance
end
before { env["rack.errors"] = MySinatraAppLogger.log_file }
end
def logger
MySinatraAppLogger.logger_instance
end
end
class MySinatraApp < Sinatra::Base
include MySinatraAppLogger
get '/' do
logger.info params.inspect
end
end
Of course, you can do it without ActiveSupport::Concern by putting the configure and before blocks straight into MySinatraApp, but what I like about this approach is that it's very clean--all logging configuration is totally abstracted out of the main app class.
It's also very easy to spot where you can change it. For instance, the SO asked about making it log to console in development. It's pretty obvious here that all you need to do is a little if-then logic in the log_file method.