How can I dynamically define Object methods inside Ruby modules? - ruby

I have a case where I'm trying to leverage a database configuration file used by Padrino for a custom worker I'm writing without having to load the Padrino environment and without having to modify the existing database configuration. In short, I want to write my code to work with the existing database configuration code rather than modifying the existing configuration.
The database configuration file in question looks like the following:
# This is just here for right now so I can test
# to see if logger is set to anything in DB config
puts "Logger in DB config: #{logger}"
ActiveRecord::Base.configurations[:development] = {
:adapter => 'sqlite3',
:database => Padrino.root('db', "app_development.db") }
ActiveRecord::Base.logger = logger
ActiveRecord::Base.include_root_in_json = true
ActiveRecord::Base.store_full_sti_class = true
ActiveSupport.use_standard_json_time_format = true
ActiveSupport.escape_html_entities_in_json = false
ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations[Padrino.env])
My custom worker looks like the following:
ROOT = File.expand_path(File.dirname(__FILE__))
$LOAD_PATH.unshift(ROOT)
require 'active_record'
require 'logger'
# Mock Padrino module with methods used by database config file
module Padrino
def self.root(*args)
return File.expand_path(File.join("#{ROOT}/..", *args))
end
def self.env
return :development
end
end
module Worker
class << self
attr_accessor :logger
def init
#logger = Logger.new(STDOUT)
#logger.level = Logger::DEBUG
require "#{ROOT}/../config/database"
end
end
end
Worker.init
When I require the database configuration file in my worker I get an error saying the following:
/home/user/devel/app/config/database.rb:3:in `<top (required)>': undefined local variable or method `logger' for main:Object (NameError)
from /home/user/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:55:in `require'
from /home/user/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:55:in `require'
from worker.rb:26:in `init'
from worker.rb:31:in `<main>'
Given this, I modified the Worker#init function to be the following:
def init
#logger = Logger.new(STDOUT)
#logger.level = Logger::DEBUG
Object.send(:define_method, :logger, lambda { #logger })
puts "Logger in Worker module: #{logger}"
require "#{ROOT}/../config/database"
end
This change resulted in the following output:
Logger in Worker module: #<Logger:0x9505088>
Logger in DB config:
I take this to mean the #logger is not in scope in the database configuration file, even though I'm using a lambda when I define the logger method on Object.
Oddly enough, if I pull the Object.send line of code out of the Worker#init function and instead call it right before I call Worker.init like below, I get the following result.
Object.send(:define_method, :logger, lambda { Worker.logger })
Worker.init
results in
Logger in Worker module: #<Logger:0x81bc8b4>
Logger in DB config: #<Logger:0x81bc8b4>
Can someone explain to me why if I make the Object.send call inside the Worker#init function it doesn't work like it does if I make it outside the Worker module?

In your first example, you try to use undefined variable 'logger', as you can see from error message. Try to use the logger feature this way:
ActiveRecord::Base.logger = Logger.new(STDOUT)

Related

How do I reference a method in a different class from a method in another class?

I have a module and class in a file lib/crawler/page-crawler.rb that looks like this:
require 'oga'
require 'net/http'
require 'pry'
module YPCrawler
class PageCrawler
attr_accessor :url
def initialize(url)
#url = url
end
def get_page_listings
body = Net::HTTP.get(URI.parse(#url))
document = Oga.parse_html(body)
document.css('div.result')
end
newpage = PageCrawler.new "http://www.someurl"
#listings = newpage.get_page_listings
#listings.each do |listing|
bizname = YPCrawler::ListingCrawler.new listing['id']
end
end
end
Then I have another module & class in another file lib/crawler/listing-crawler.rb that looks like this:
require 'oga'
require 'pry'
module YPCrawler
class ListingCrawler
def initialize(id)
#id = id
end
def extract_busines_name
binding.pry
end
end
end
However, when I try to run this script ruby lib/yp-crawler.rb which executes the page-crawler.rb file above and works without the YPCrawler call, I get this error:
/lib/crawler/page-crawler.rb:23:in `block in <class:PageCrawler>': uninitialized constant YPCrawler::ListingCrawler (NameError)
The issue is on this line:
bizname = YPCrawler::ListingCrawler.new listing['id']
So how do I call that other from within my iterator in my page-crawler.rb?
Edit 1
When I just do `ListingCrawler.new listing['id'], I get the following error:
uninitialized constant YPCrawler::PageCrawler::ListingCrawler (NameError)
Edit 2
Here is the directory structure of my project:
Edit 3
My yp-crawler.rb looks like this:
require_relative "yp-crawler/version"
require_relative "crawler/page-crawler"
require_relative "crawler/listing-crawler"
module YPCrawler
end
In your yp-crawler.rb file, based on the structure that you posted, you should have something like:
require 'yp-crawler/version'
require 'crawler/listing-crawler'
require 'crawler/page-crawler'
Try this, in your yp-crawler.rb add the line:
Dir["#{File.dirname(__FILE__)}/crawler/**/*.rb"].each { |file| load(file) }
That should automatically include all files in your /crawler directory at runtime. Might want to do the same for the other directories.
Let me know if that helps :)

Can't access custom helper functions made within the file - Sinatra

I have this helper within my 'app.rb' file, which is used to get the current user object.
helpers do
def get_current_user(column, value)
user = User.where(column => value).first
return user
end
end
get '/' do
user = get_current_user(:id, params[:id])
end
This is what i did in irb:
irb(main):001:0> require './app'
=> true
irb(main):007:0> user = get_current_user(:id, 2)
NoMethodError: undefined method `get_current_user' for main:Object
from (irb):7
from /usr/bin/irb:12:in `<main>'
I don't understand why i can't access the helper methods from irb. Should i explicitly include the helpers or something? If so, why? Because i put them under the same file, under the same class.
get_current_users is metaprogrammed through the helpers method to be an instance method of App. So, if app.rb looks something like this:
require 'sinatra/base'
class App < Sinatra::Base
helpers do
def get_current_user
puts "here!"
end
end
end
...then from irb you can invoke get_current_user on an instance of App like this:
>> require './app'
>> App.new!.get_current_user
here!
=> nil
>>
(If you're wondering why that's new! and not new like most sane ruby code, read this answer.)

Inheritance within iron workers when using the iron_worker_ruby gem

I'm considering using IronWorker a project so that I can scale it easily (high traffic expected, with lots of background jobs).
In order to stay DRY, I'm trying to define workers using inheritance but I keep getting the following error:
/usr/local/lib/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- base_worker.rb (LoadError)
from /usr/local/lib/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from /task/child_worker.rb:3:in `<top (required)>'
from /task/runner.rb:344:in `require_relative'
from /task/runner.rb:344:in `<main>'
Here is the base worker class:
# app/workers/base_worker.rb
require 'net/http'
require 'uri'
require 'json'
class BaseWorker < IronWorker::Base
attr_accessor :params
# The run method is what IronWorker calls to run your worker
def run
data = custom_run(params)
common_post_process(data)
end
def custom_run(params)
#to be overwritten in the child class
end
def common_post_process(data)
# some common post processing => DRY
....
end
end
And here is a child class :
# app/workers/child_worker.rb
require 'net/http'
require 'uri'
require 'base_worker.rb'
class ChildWorker < BaseWorker
merge "base_worker.rb"
def custom_run(params)
#custom work
end
end
Any idea on how to fix this?
I'd recommend using our next generation gem, iron_worker_ng: https://github.com/iron-io/iron_worker_ruby_ng . The iron_worker gem is deprecated. And if you want to keep it similar style to what you have, your child_worker.rb might look like this:
require 'net/http'
require 'uri'
require_relative 'base_worker.rb'
class ChildWorker < BaseWorker
def custom_run(params)
#custom work
end
end
# NG gem doesn't run anything in particular, so to run your method:
cw = ChildWorker.new
cw.custom_run(params)
And in a child_worker.worker file:
runtime 'ruby'
file 'base_worker.rb'
exec 'child_worker.rb'
Then to upload it to IronWorker:
iron_worker upload child_worker
Then you can start queuing jobs for it:
worker = IronWorkerNG::Client.new
worker.tasks.create("child_worker", params)
If you use iron_worker_ng, it also possible to define a run method. This method will be called when the IronWorker runs. You have to specify the Class within the .worker file.
# child_worker.rb
class ChildWorker
def run
puts "doing the hard work"
end
end
And the child_worker.worker file:
# child_worker.worker
runtime 'ruby'
name 'ChildWorker'
exec 'child_worker.rb', 'ChildWorker'

Logging in Sinatra?

I'm having trouble figuring out how to log messages with Sinatra. I'm not looking to log requests, but rather custom messages at certain points in my app. For example, when fetching a URL I would like to log "Fetching #{url}".
Here's what I'd like:
The ability to specify log levels (ex: logger.info("Fetching #{url}"))
In development and testing environments, the messages would be written to the console.
In production, only write out messages matching the current log level.
I'm guessing this can easily be done in config.ru, but I'm not 100% sure which setting I want to enable, and if I have to manually create a Logger object myself (and furthermore, which class of Logger to use: Logger, Rack::Logger, or Rack::CommonLogger).
(I know there are similar questions on StackOverflow, but none seem to directly answer my question. If you can point me to an existing question, I will mark this one as a duplicate).
Sinatra 1.3 will ship with such a logger object, exactly usable as above. You can use edge Sinatra as described in "The Bleeding Edge". Won't be that long until we'll release 1.3, I guess.
To use it with Sinatra 1.2, do something like this:
require 'sinatra'
use Rack::Logger
helpers do
def logger
request.logger
end
end
I personally log in Sinatra via:
require 'sinatra'
require 'sequel'
require 'logger'
class MyApp < Sinatra::Application
configure :production do
set :haml, { :ugly=>true }
set :clean_trace, true
Dir.mkdir('logs') unless File.exist?('logs')
$logger = Logger.new('logs/common.log','weekly')
$logger.level = Logger::WARN
# Spit stdout and stderr to a file during production
# in case something goes wrong
$stdout.reopen("logs/output.log", "w")
$stdout.sync = true
$stderr.reopen($stdout)
end
configure :development do
$logger = Logger.new(STDOUT)
end
end
# Log all DB commands that take more than 0.2s
DB = Sequel.postgres 'mydb', user:'dbuser', password:'dbpass', host:'localhost'
DB << "SET CLIENT_ENCODING TO 'UTF8';"
DB.loggers << $logger if $logger
DB.log_warn_duration = 0.2
If you are using something like unicorn logging or other middleware that tails IO streams, you can easily set up a logger to STDOUT or STDERR
# unicorn.rb
stderr_path "#{app_root}/shared/log/unicorn.stderr.log"
stdout_path "#{app_root}/shared/log/unicorn.stdout.log"
# sinatra_app.rb
set :logger, Logger.new(STDOUT) # STDOUT & STDERR is captured by unicorn
logger.info('some info') # also accessible as App.settings.logger
this allows you to intercept messages at application scope, rather than just having access to logger as request helper
Here's another solution:
module MySinatraAppLogger
extend ActiveSupport::Concern
class << self
def logger_instance
#logger_instance ||= ::Logger.new(log_file).tap do |logger|
::Logger.class_eval { alias :write :'<<' }
logger.level = ::Logger::INFO
end
end
def log_file
#log_file ||= File.new("#{MySinatraApp.settings.root}/log/#{MySinatraApp.settings.environment}.log", 'a+').tap do |log_file|
log_file.sync = true
end
end
end
included do
configure do
enable :logging
use Rack::CommonLogger, MySinatraAppLogger.logger_instance
end
before { env["rack.errors"] = MySinatraAppLogger.log_file }
end
def logger
MySinatraAppLogger.logger_instance
end
end
class MySinatraApp < Sinatra::Base
include MySinatraAppLogger
get '/' do
logger.info params.inspect
end
end
Of course, you can do it without ActiveSupport::Concern by putting the configure and before blocks straight into MySinatraApp, but what I like about this approach is that it's very clean--all logging configuration is totally abstracted out of the main app class.
It's also very easy to spot where you can change it. For instance, the SO asked about making it log to console in development. It's pretty obvious here that all you need to do is a little if-then logic in the log_file method.

Use Rack::CommonLogger in Sinatra

I have a small web-server that I wrote with Sinatra. I want to be able to log messages to a log file. I've read through http://www.sinatrarb.com/api/index.html and www.sinatrarb.com/intro.html, and I see that Rack has something called Rack::CommonLogger, but I can't find any examples of how it can be accessed and used to log messages. My app is simple so I wrote it as a top-level DSL, but I can switch to subclassing it from SinatraBase if that's part of what's required.
Rack::CommonLogger won't provide a logger to your main app, it will just logs the request like Apache would do.
Check the code by yourself: https://github.com/rack/rack/blob/master/lib/rack/common_logger.rb
All Rack apps have the call method that get's invoked with the HTTP Request env, if you check the call method of this middleware this is what happens:
def call(env)
began_at = Time.now
status, header, body = #app.call(env)
header = Utils::HeaderHash.new(header)
log(env, status, header, began_at)
[status, header, body]
end
The #app in this case is the main app, the middleware is just registering the time the request began at, then it class your middleware getting the [status, header, body] triple, and then invoke a private log method with those parameters, returning the same triple that your app returned in the first place.
The logger method goes like:
def log(env, status, header, began_at)
now = Time.now
length = extract_content_length(header)
logger = #logger || env['rack.errors']
logger.write FORMAT % [
env['HTTP_X_FORWARDED_FOR'] || env["REMOTE_ADDR"] || "-",
env["REMOTE_USER"] || "-",
now.strftime("%d/%b/%Y %H:%M:%S"),
env["REQUEST_METHOD"],
env["PATH_INFO"],
env["QUERY_STRING"].empty? ? "" : "?"+env["QUERY_STRING"],
env["HTTP_VERSION"],
status.to_s[0..3],
length,
now - began_at ]
end
As you can tell, the log method just grabs some info from the request env, and logs in on a logger that is specified on the constructor call, if there is no logger instance then it goes to the rack.errors logger (it seems there is one by default)
The way to use it (in your config.ru):
logger = Logger.new('log/app.log')
use Rack::CommonLogger, logger
run YourApp
If you want to have a common logger in all your app, you could create a simple logger middleware:
class MyLoggerMiddleware
def initialize(app, logger)
#app, #logger = app, logger
end
def call(env)
env['mylogger'] = #logger
#app.call(env)
end
end
To use it, on your config.ru:
logger = Logger.new('log/app.log')
use Rack::CommonLogger, logger
use MyLoggerMiddleware, logger
run MyApp
Hope this helps.
In your config.ru:
root = ::File.dirname(__FILE__)
logfile = ::File.join(root,'logs','requests.log')
require 'logger'
class ::Logger; alias_method :write, :<<; end
logger = ::Logger.new(logfile,'weekly')
use Rack::CommonLogger, logger
require ::File.join(root,'myapp')
run MySinatraApp.new # Subclassed from Sinatra::Application
I followed what I found on this blog post - excerpted below
require 'rubygems'
require 'sinatra'
disable :run
set :env, :production
set :raise_errors, true
set :views, File.dirname(__FILE__) + '/views'
set :public, File.dirname(__FILE__) + '/public'
set :app_file, __FILE__
log = File.new("log/sinatra.log", "a")
STDOUT.reopen(log)
STDERR.reopen(log)
require 'app'
run Sinatra.application
then use puts or print. It worked for me.
class ErrorLogger
def initialize(file)
#file = ::File.new(file, "a+")
#file.sync = true
end
def puts(msg)
#file.puts
#file.write("-- ERROR -- #{Time.now.strftime("%d %b %Y %H:%M:%S %z")}: ")
#file.puts(msg)
end
end
class App < Sinatra::Base
if production?
error_logger = ErrorLogger.new('log/error.log')
before {
env["rack.errors"] = error_logger
}
end
...
end
Reopening STDOUT and redirecting it to a file is not a good idea if you use Passenger. It causes in my case that Passenger does not start. Read https://github.com/phusion/passenger/wiki/Debugging-application-startup-problems#stdout-redirection for this issue.
This would be the right way instead:
logger = ::File.open('log/sinatra.log', 'a+')
Sinatra::Application.use Rack::CommonLogger, logger

Resources