I am using Mongoid for interacting with MongoDB. In development I usually like to see the logs of what Mongo is doing. However, there is one instance where there is an excessive amount of redundant logging that I simply don't want to see. How can I disable logging in this specific case?
There doesn't appear to be any clean way to control logging in Mongoid on a per collection basis.
However, for your purposes, if you are able to identify the individual calls,
you can turn off logging temporarily by raising the level.
Hope that this is good enough for your purposes.
require 'test_helper'
def dont_log(temp_level = Logger::Severity::UNKNOWN)
logger = Rails.logger
old_level, logger.level = logger.level, temp_level
yield
logger.level = old_level
end
class LogDoTest < ActiveSupport::TestCase
def setup
LogDo.delete_all
LogDont.delete_all
end
test "log do dont" do
LogDo.create(text: 'Log me')
dont_log{ LogDont.create(text: 'Dont log me') }
end
end
Related
The gist
Is it possible to bake feature-toggle-like functionality into a Sinatra application?
A bit about feature toggles, just in-case ;)
Back story
I've set up a modular Sinatra project, and I tend to implement a GET/POST/PUT/DELETE endpoint for all my resources; it makes it easier to test the app and manipulate the data while in development.
Problem
When I go into production I don't want the unneeded endpoints to exist (e.g DELETE '/users').
Question
Can I annotate the methods with some kind of a :development flag, or maybe intercept the request in a before block? Would you do this using a helper? I'm not sure if I'm heading down the right path here, I'm probably over complicating it(?)
How would one go about this?
If you've done something like this it would be great if you can share your findings with the nation.
You can use the current environment to decide whether you define an action. For example:
class MyApp < Sinatra::Application
if settings.development?
get '/admin' do
'VIPs only'
end
end
end
If you have a lot to toggle, you might want to isolate them in one file that you can decide to require or not:
# routes/init.rb
require_relative 'main'
require_relative 'debug' if settings.development?
# routes/main.rb
class MyApp < Sinatra::Application
get '/' do
'Hello!'
end
end
# routes/debug.rb
class MyApp < Sinatra::Application
get '/admin' do
'VIPs only'
end
end
Or if you want to list your development-only paths in one place, here's a filter version:
class MyApp < Sinatra::Application
DEVELOPMENT_PATHS = %w[
/admin
]
before do
unless settings.development? || !DEVELOPMENT_PATHS.include?(request.path)
halt 404
end
end
end
Then you could also build some decorator-like methods that add to the list:
class MyApp < Sinatra::Application
def self.development_only(path)
DEVELOPMENT_PATHS << path
end
get '/admin' do
'VIPs only'
end
development_only '/admin
end
In general, I'd recommend caution when introducing significant differences between the code that runs in development vs. production. Inevitably, the dev code is either left untested or becomes cumbersome to maintain properly. In this case, there's the danger that you miss a route you intended to hide and it becomes available to everyone in production. I'd tend towards not having these routes at all and manipulating my dev environment from a console, or going all the way to the other end and building fully-tested and production-ready user permissions with something like sinatra-authentication.
I have a running environment with a Rails application, Sidekiq and clockwork mod for scheduling purposes.
I have many different workers, filled with logger.debug and logger.info instructions, and I occasionally need to activate debug logging on some of them to know what's going on.
I like the Sidekiq logger, and I would like to utilize it because it just need a "logger.debug" instruction in the workers to do its job.
What I miss with my current setup is the possibility to activate the DEBUG level for some workers, while leaving the others in standard INFO.
Now in each of my workers I have this initialize method:
class SendMailOnStart
include Sidekiq::Worker
sidekiq_options :retry => false, :queue => :critical
def initialize
logger.level = Logger::INFO
end
.... ...
But if a change the level in one worker, this level will be overwritten by the level specified in the next one - e.g. if two workers are processed together, the second one will "win".
What's the best way to achieve this in an elegant way?
Coming from Java world, I can think only to create a custom logger and putting it in each worker, copying output format used by Sidekiq logger, adding a logger method in each worker like
def logger
logger = MyLogger.new
end
and changing the level when I neeed it in initialize method
Is this the best approach in Ruby?
I had a similar question and I found this thread more useful:
Log to different logger based on call from Sidekiq or Rails
You should be able set the log level for Sidekiq workers specifically in the block mentioned there by altering Rails.logger.
I have no clue if that’s the best approach, but I would do the following. First of all, let’s prepare the function to retrieve the caller’s filename and/or method:
def parse_caller
# magic number 7 below is the amount of calls
# on stack to unwind to get your caller
if /^(?<file>.+?):(?<line>\d+)(?::in `(?<method>.*)')?/ =~ caller(7).first
file = Regexp.last_match[:file]
line = Regexp.last_match[:line].to_i
method = Regexp.last_match[:method]
[file, line, method]
end
end
Then I would override the default formatter of Logger instance, compelling it to check the caller:
logger.formatter = lambda do |severity, datetime, progname, msg|
f,l,m = parse_caller
# HERE GOES YOUR CHECK
if f =~ /…/
…
end
end
I know it looks a weird hack, but it works fine for me.
Rails' ActiveRecord has a feature called Query Caching (ActiveRecord::QueryCache) which saves the result of SQL query for the life-span of a request. While I'm not very familiar with the internals of the implementation, I think that it saves the query results somewhere in the Rack env, which is discarded in the end of the request.
The Mongoid, unfortunately, doesn't currently provide such feature, and this is exacerbated by the fact, that some queries occur implicitly (references).
I'm considering to implement this feature, and I'm curious, where and how Mongoid (or, perhaps, mongo driver?) should be hooked in order to implement this.
Mongoid has caching, described under http://mongoid.org/en/mongoid/docs/extras.html
Also MongoDB itself has caching ability: http://www.mongodb.org/display/DOCS/Caching
The mongoid caching extra knows 2 different cases: Caching of all queries of a model or caching of a query.
Mongoid caching seems to work slightly different: it looks like mongoid delegates caching to mongodb. (In the sources of mongoid I only can find option settings for caching but no cache module.)
Finally would say, there is no real difference in the caching in general -- in memory is in fact in memory! No matter if it's in the app or in the database.
I don't prefer to implement an extra caching algorithm, because this seems to be redundant and a RAM killer.
BTW: If your really want to cache results in-app you could try Rails.cache or another cache gem as a workaround.
The other answer is obviously wrong. Not only mongoid or mongo driver doesn't cache the query, even if mongo would - it still might be on other machine across the network.
My solution was to wrap the receive_message in Mongo::Connection.
Pros: one definite place
Cons: deserialization still takes place
require 'mongo'
module Mongo
class Connection
module QueryCache
extend ActiveSupport::Concern
module InstanceMethods
# Enable the selector cache within the block.
def cache
#query_cache ||= {}
old, #query_cache_enabled = #query_cache_enabled, true
yield
ensure
clear_query_cache
#query_cache_enabled = old
end
# Disable the selector cache within the block.
def uncached
old, #query_cache_enabled = #query_cache_enabled, false
yield
ensure
#query_cache_enabled = old
end
def clear_query_cache
#query_cache.clear
end
def cache_receive_message(operation, message)
#query_cache[operation] ||= {}
key = message.to_s.hash
log = "[MONGO] CACHE %s"
if entry = #query_cache[operation][key]
Mongoid.logger.debug log % 'HIT'
entry
else
Mongoid.logger.debug log % 'MISS'
#query_cache[operation][key] = yield
end
end
def receive_message_with_cache(operation, message, log_message=nil, socket=nil, command=false)
if query_cache_enabled
cache_receive_message(operation, message) do
receive_message_without_cache(operation, message, log_message, socket, command)
end
else
receive_message_without_cache(operation, message, log_message, socket, command)
end
end
end # module InstanceMethods
included do
alias_method_chain :receive_message, :cache
attr_reader :query_cache, :query_cache_enabled
end
end # module QueryCache
end # class Connection
end
Mongo::Connection.send(:include, Mongo::Connection::QueryCache)
OK, Mongoid 4 supports QueryCache middleware.
Just add middleware in application.rb
config.middleware.use "Mongoid::QueryCache::Middleware"
And then profit:
MOPED: 127.0.0.1:27017 QUERY database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564dabb6d61631e21d70000')}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil runtime: 0.4397ms
MOPED: 127.0.0.1:27017 QUERY database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564dacf6d61631e21dc0000')}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil runtime: 0.4590ms
QUERY CACHE database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564c9596d61631e21d30000')}, "$orderby"=>{:_id=>1}}
QUERY CACHE database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564dabb6d61631e21d70000')}, "$orderby"=>{:_id=>1}}
Source:
Mongoid changelog
https://github.com/mongoid/mongoid/blob/master/CHANGELOG.md#new-features-2
3410 Mongoid now has a query cache that can be used as a middleware in Rack applications. (Arthur Neves)
For Rails:
config.middleware.use(Mongoid::QueryCache::Middleware)
Mongoid 4.0+ now has a QueryCaching module: http://www.rubydoc.info/github/mongoid/mongoid/Mongoid/QueryCache
You can use it on finds by wrapping your lookups like so:
QueryCache.cache { MyCollection.find("xyz") }
By default Selenium runs as fast as possible through the scenarios I defined using Cucumber.
I would like to set it to run at a lower speed, so I am able to capture a video of the process.
I figured out that an instance of Selenium::Client::Driver has a set_speed method. Which corresponds with the Java API.
How can I obtain an instance of the Selenium::Client::Driver class? I can get as far as page.driver, but that returns an instance of Capybara::Driver::Selenium.
Thanks to http://groups.google.com/group/ruby-capybara/msg/6079b122979ffad2 for a hint.
Just a note that this uses Ruby's sleep, so it's somewhat imprecise - but should do the job for you. Also, execute is called for everything so that's why it's sub-second waiting. The intermediate steps - wait until ready, check field, focus, enter text - each pause.
Create a "throttle.rb" in your features/support directory (if using Cucumber) and fill it with:
require 'selenium-webdriver'
module ::Selenium::WebDriver::Firefox
class Bridge
attr_accessor :speed
def execute(*args)
result = raw_execute(*args)['value']
case speed
when :slow
sleep 0.3
when :medium
sleep 0.1
end
result
end
end
end
def set_speed(speed)
begin
page.driver.browser.send(:bridge).speed=speed
rescue
end
end
Then, in a step definition, call:
set_speed(:slow)
or:
set_speed(:medium)
To reset, call:
set_speed(:fast)
This will work, and is less brittle (for some small value of "less")
require 'selenium-webdriver'
module ::Selenium::WebDriver::Remote
class Bridge
alias_method :old_execute, :execute
def execute(*args)
sleep(0.1)
old_execute(*args)
end
end
end
As an update, the execute method in that class is no longer available. It is now here only:
module ::Selenium::WebDriver::Remote
I needed to throttle some tests in IE and this worked.
The methods mentioned in this thread no longer work with Selenium Webdriver v3.
You'll instead need to add a sleep to the execution command.
module Selenium::WebDriver::Remote
class Bridge
def execute(command, opts = {}, command_hash = nil)
verb, path = commands(command) || raise(ArgumentError, "unknown command: #{command.inspect}")
path = path.dup
path[':session_id'] = session_id if path.include?(':session_id')
begin
opts.each { |key, value| path[key.inspect] = escaper.escape(value.to_s) }
rescue IndexError
raise ArgumentError, "#{opts.inspect} invalid for #{command.inspect}"
end
Selenium::WebDriver.logger.info("-> #{verb.to_s.upcase} #{path}")
res = http.call(verb, path, command_hash)
sleep(0.1) # <--- Add your sleep here.
res
end
end
end
Note this is a very brittle way to slow down the tests since you're monkey patching a private API.
I wanted to slow down the page load speeds in my Capybara test suite to see if I could trigger some intermittently failing tests. I achieved this by creating an nginx reverse proxy container and sitting it between my test container and the phantomjs container I was using as a headless browser. The speed was limited by using the limit_rate directive. It didn't help me to achieve my goal in the end, but it did work and it may be a useful strategy for others to use!
Are there any logging frameworks in ruby that allow you to log a specific event type only once?
logger = IdealLogger.new
logger.log(:happy_path, "We reached the happy path") # => logs this message
logger.log(:happy_path, "We reached the happy path yet again") # => Doesn't log this
logger.log(:sad_path, "We've encountered a sad path!") # => logs this message
Also, is there a term for the concept of logging a certain event type only once?
Edit: I'm using Plain Old Ruby Objects, not Rails. I had in mind "once per time the script is run" for "once".
I'm not aware of one, but extending Logger to make your own isn't too difficult. It's essentially implementing caching for your logging, but instead of fetching from the cache and returning it like you would with a normal app, you quash it when it's been cached. Implementation and expiration strategy of this log cache are left as an exercise for the reader.
something like:
class IdealLogger < Logger
def info(event = nil, progname = nil, &block)
super(progname, &block) unless event_is_cached(event)
end
# define debug, warn, error, fatal, and unknown the same way, override others
# as you wish.
end