Summary
I'm trying to use SemanticLogger 4.10.0 as a part of a dynamic tracing solution in a Ruby 3.1.1 app. However, I seem to be misunderstanding something about how or where to access the logger instance that SemanticLogger::Loggable is supposed to create on the class, and am getting NameError exceptions (documented below) when calling #logger_measure_method from inside the extended class because I can't seem to implement the class extension properly.
Code That Demonstrates the Problem
Module with CustomLogger
require 'json'
require 'semantic_logger'
module SomeGem
module CustomLogger
include SemanticLogger
class MyFormat < SemanticLogger::Formatters::Default
def call (log, logger, machine_name: nil, json: nil)
self.log = log
self.logger = logger
message = "[#{machine name}] #{message}" if machine_name
payload = JSON.parse(json)
[time, level, process.info, tags, named.tags, duration, name, message, payload, exception].compact!
end
end
def self.extended(klass)
prepend SemanticLogger, SemanticLogger::Loggable, SemanticLogger::Loggable::ClassMethods
SemanticLogger.default_level = :debug
SemanticLogger.add_appender(file_name: "/dev/stderr", formatter: SomeGem::CustomLogger::MyFormat.new,
level: :warn, filter: proc { %i[warn fatal unknown].include? _1.level }
)
SemanticLogger.add_appender(file_name: "/dev/stdout", formatter: SomeGem::CustomLogger::MyFormat.new,
level: :trace, filter: proc { %i[trace debug info error].include? _1.level }
)
define_method(:auto_measure_method_calls) do
public_instance_methods(false).
reject { _1.to_s.match? /^logger/ }.
each { logger_measure_method _1, level: :trace }
end
end
end
end
Module Extended by CustomLogger
module SomeGem
class Foo
extend CustomLogger
def alpha = 1
def beta = 2
def charlie = 3
auto_measure_method_calls if ENV['LOG_LEVEL'] == 'trace'
end
end
Exercising the Code
ENV['LOG_LEVEL'] = 'trace'
f = SomeGem::Foo.new
pp f.alpha, f.beta, f.charlie
Problems and Issues with Code
My issues are:
The underlying (and possibly X/Y) issue is that I want to DRY up the code of including SemanticLogger::Loggable and SemanticLogger::Loggable::ClassMethods in every class I'm tracing, and auto-measuring the extended classes' methods in development.
When I try to extend a class with a module that is intended to pull in SemanticLogger::Loggable, I don't seem to always have logger available as an accessor throughout the class.
I'm also concerned that including the module in multiple classes would result in duplicate appenders being added to the #appenders array, wherever that's actually stored.
Most importantly though, when I try to automagically add logging and method measurement through extending a class, I get errors like the following:
~/.gem/ruby/3.1.1/gems/semantic_logger-4.10.0/lib/semantic_logger/loggable.rb:96:in `alpha': undefined local variable or method `logger' for #SomeGem::Foo:0x00000001138f6a38 (NameError) from foo.rb:51:in `<main>'
Am I missing something obvious about how to extend the classes with SemanticLogger? If there's a better way to accomplish what I'm trying to do, that's great! If I'm doing something wrong, understanding that would be great, too. If it's a bug, or I'm using the feature wrong, that's useful information as well.
Related
I have problem stubbing external api, following is the example
require 'rspec'
require 'google/apis/storage_v1'
module Google
class Storage
def upload file
puts '#' * 90
puts "File #{file} is uploaded to google cloud"
end
end
end
class UploadWorker
include Sidekiq::Worker
def perform
Google::Storage.new.upload 'test.txt'
end
end
RSpec.describe UploadWorker do
it 'uploads to google cloud' do
google_cloud_instance = double(Google::Storage, insert_object: nil)
expect(google_cloud_instance).to receive(:upload)
worker = UploadWorker.new
worker.perform
end
end
I'm trying to stub Google::Storage class. This class is instantiated inside the object being tested. How can I verify the message expectation on this instance?
When I run above example, I get following output, and it seems logical, my double is not used by tested object
(Double Google::Storage).upload(*(any args))
expected: 1 time with any arguments
received: 0 times with any arguments
I'm new to Rspec and having hard time with this, any help will be appreciated.
Thanks!
Reaching for DI is always a good idea (https://stackoverflow.com/a/51401376/299774) but there are sometimes reasons you can't so it, so here's another way to stub it without changing the "production" code.
1. expect_any_instance_of
it 'uploads to google cloud' do
expect_any_instance_of(Google::Storage).to receive(:insert_object)
worker = UploadWorker.new
worker.perform
end
In case you just want to test that the method calls the method on any such objects.
2. bit more elaborated setup
In case you want to control or set up more expectations, you can do this
it 'uploads to google cloud' do
the_double = instance_double(Google::Storage)
expect(Google::Storage).to receive(:new).and_return(the_double)
# + optional `.with` in case you wanna assert stuff passed to the constructor
expect(the_double).to receive(:insert_object)
worker = UploadWorker.new
worker.perform
end
Again - Dependency Injection is clearer, and you should aim for it. This is presented as another possibility.
I would consider reaching for dependency injection, such as:
class UploadWorker
def initialize(dependencies = {})
#storage = dependencies.fetch(:storage) { Google::Storage }
end
def perform
#storage.new.upload 'test.txt'
end
end
Then in the spec you can inject a double:
storage = double
expect(storage).to receive(...) # expection
worker = UploadWorker.new(storage: storage)
worker.perform
If using the initializer is not an option then you could use getter/setter method to inject the dependency:
def storage=(new_storage)
#storage = new_storage
end
def storage
#storage ||= Google::Storage
end
and in the specs:
storage = double
worker.storage = storage
I'm writing Ruby Gem where I have Connection module for Faraday configuration
module Example
module Connection
private
def connection
Faraday.new(url: 'http://localhost:3000/api') do |conn|
conn.request :url_encoded # form-encode POST params
conn.response :logger # log requests to STDOUT
conn.adapter Faraday.default_adapter # make requests with Net::HTTP
conn.use Faraday::Response::ParseJson
conn.use FaradayMiddleware::RaiseHttpException
end
end
end
end
Second module which makes API requests looks like this:
module Example
module Request
include Connection
def get(uri)
connection.get(uri).body
end
def post(url, attributes)
response = connection.post(url) do |request|
request.body = attributes.to_json
end
end
def self.extended(base)
base.include(InstanceMethods)
end
module InstanceMethods
include Connection
def put(url, attributes)
response = connection.put(url) do |request|
request.body = attributes.to_json
end
end
end
end
end
Class Cusomer where I use Request looks like this:
module Example
class Customer
extend Request
attr_accessor :id, :name, :age
def initialize(attrs)
attrs.each do |key, value|
instance_variable_set("##{key}", value)
end
end
def self.all
customers = get('v1/customer')
customers.map { |cust| new cust }
end
def save
params = {
id: self.id,
age: self.age
name: self.name,
}
put("v1/customers/#{self.id}", params)
end
end
end
So here you see in Customer#all class method I'm calling Request#get method which is available because I extended Request in Customer. then I'm using self.extended method in Request module to be make Request#put available in Customer class, so I have question is this good approach to use mixins like this, or do you have any suggestion?
Mixins are a strange beast. Best practices vary depending on who you talk to. As far as reuse goes, you've achieved that here with mixins, and you have a nice separation of concerns.
However, mixins are a form of inheritance (you can take a peek at #ancestors). I would challenge you saying that you shouldn't use inheritance here because a Customer doesn't have an "is-a" relationship with Connection. I would recommend you use composition instead (e.g. pass in Connection/Request) as it makes more sense to me in this case and has stronger encapsulation.
One guideline for writing mixins is to make everything end in "-able", so you would have Enumerable, Sortable, Runnable, Callable, etc. In this sense, mixins are generic extensions that provide some sort of helpers that are depending on a very specific interface (e.g. Enumerable depends on the class to implement #each).
You could also use mixins for cross-cutting concerns. For example, we've used mixins in the past in our background jobs so that we could add logging for example without having to touch the source code of the class. In this case, if a new job wants logging, then they just mixin the concern which is coupled to the framework and will inject itself properly.
My general rule of thumb is don't use them if you don't have to. They make understanding the code a lot more complicated in most cases
EDIT: Adding an example of composition. In order to maintain the interface you have above you'd need to have some sort of global connection state, so it may not make sense. Here's an alternative that uses composition
class CustomerConnection
# CustomerConnection is composed of a Connection and retains isolation
# of responsibilities. It also uses constructor injection (e.g. takes
# its dependencies in the constructor) which means easy testing.
def initialize(connection)
#connection = connection
end
def all_customers
#connection.get('v1/customers').map { |res| Customer.new(res) }
end
end
connection = Connection.new
CustomerConnection.new(connection).all_customers
I know in ruby, when we call an instance method, we need to firstly instantiate a class object.
But when I see a open sourced code I got confused.
The code is like this:
File Message.rb
require 'json'
module Yora
module Message
def serialize(msg)
JSON.generate(msg)
end
def deserialize(raw, symbolized_key = true)
msg = JSON.parse(raw, create_additions: true)
if symbolized_key
Hash[msg.map { |k, v| [k.to_sym, v] }]
else
msg
end
end
end
end
File. Persistance.rb
require 'fileutils'
require_relative 'message'
module Yora
module Persistence
class SimpleFile
include Message
def initialize(node_id, node_address)
#node_id, #node_address = node_id, node_address
FileUtils.mkdir_p "data/#{node_id}"
#log_path = "data/#{node_id}/log.txt"
#metadata_path = "data/#{node_id}/metadata.txt"
#snapshot_path = "data/#{node_id}/snapshot.txt"
end
def read_metadata
metadata = {
current_term: 0,
voted_for: nil,
cluster: { #node_id => #node_address }
}
if File.exist?(#metadata_path)
metadata = deserialize(File.read(#metadata_path)) #<============
end
$stderr.puts "-- metadata = #{metadata}"
metadata
end
.....
You can see the line I marked with "<==="
It uses deserialize function that been defined in message class.
And from message class we can see that method is a instance method, not class method.
So why can we call it without instantiating anything like this?
thanks
Message ist an module. Your Class SimpleFile includes this module. so the module methods included in your class SimpleFile. that means, all module methods can now be used like as methods from SimpleFile
see http://ruby-doc.org/core-2.2.0/Module.html for more infos about module in ruby. it's a great feature.
It is being called on an instance. In Ruby, if you leave out the explicit receiver of the message send, an implicit receiver of self is assumed. So, deserialize is being called on an instance, namely self.
Note that this exact same phenomenon also occurs in other places in your code, much earlier (in line 1, in fact):
require 'fileutils'
require_relative 'message'
Here, you also have two method calls without an explicit receiver, which means that the implicit receiver is self.
I'm experimenting with RSpec.
Since I don't like mocks, I would like to emulate a console print using a StringIO object.
So, I want to test that the Logger class writes Welcome to the console. To do so, my idea was to override the puts method used inside Logger from within the spec file, so that nothing actually changes when using Logger elsewhere.
Here's some code:
describe Logger do
Logger.class_eval do
def puts(*args)
???.puts(*args)
end
end
it 'says "Welcome"' do
end
Doing this way, I need to share some StringIO object (which would go where the question marks are now) between the Logger class and the test class.
I found out that when I'm inside RSpec tests, self is an instance of Class. What I thought initially was to do something like this:
Class.class_eval do
attr_accessor :my_io
#my_io = StringIO.new
end
and then replace ??? with Class.my_io.
When I do this, a thousand bells ring in my head telling me it's not a clean way to do this.
What can I do?
PS: I still don't get this:
a = StringIO.new
a.print('a')
a.string # => "a"
a.read # => "" ??? WHY???
a.readlines # => [] ???
Still: StringIO.new('hello').readlines # => ["hello"]
To respond to your last concern, StringIO simulates file behavior. When you write/print to it, the input cursor is positioned after the last thing you wrote. If you write something and want to read it back, you need to reposition yourself (e.g. with rewind, seek, etc.), per http://ruby-doc.org/stdlib-1.9.3/libdoc/stringio/rdoc/StringIO.html
In contrast, StringIO.new('hello') establishes hello as the initial contents of the string while leaving in the position at 0. In any event, the string method just returns the contents, independent of position.
It's not clear why you have an issue with the test double mechanism in RSpec.
That said, your approach for sharing a method works, although:
The fact that self is an anonymous class within RSpec's describe is not really relevant
Instead of using an instance method of Class, you can define your own class and associated class method and "share" that instead, as in the following:
class Foo
def self.bar(arg)
puts(arg)
end
end
describe "Sharing stringio" do
Foo.class_eval do
def self.puts(*args)
MyStringIO.my_io.print(*args)
end
end
class MyStringIO
#my_io = StringIO.new
def self.my_io ; #my_io ; end
end
it 'says "Welcome"' do
Foo.bar("Welcome")
expect(MyStringIO.my_io.string).to eql "Welcome"
end
end
Logger already allows the output device to be specified on construction, so you can easily pass in your StringIO directly without having to redefine anything:
require 'logger'
describe Logger do
let(:my_io) { StringIO.new }
let(:log) { Logger.new(my_io) }
it 'says welcome' do
log.error('Welcome')
expect(my_io.string).to include('ERROR -- : Welcome')
end
end
As other posters have mentioned, it's unclear whether you're intending to test Logger or some code that uses it. In the case of the latter, consider injecting the logger into the client code.
The answers to this SO question also show several ways to share a common Logger between clients.
I'm trying to do some monkey patching in ActiveShipping UPS class .
I need to add a class level method (starting with .self), so here it's what I'm trying to do:
module ActiveMerchant
module Shipping
class UPS < Carrier
def self.process_request(receiver, sender, packages, options = {})
# some code
end
def regular_method
"foobar"
end
end
end
end
Unfortunately when I'm trying to use it:
ActiveMerchant::Shipping::UPS.process_request(receiver etc)
I get an error:
NoMethodError: undefined method `process_request' for ActiveMerchant::Shipping::UPS:Class
from (irb):6
from C:/Ruby19/bin/irb.bat:19:in `<main>'
There is no class method named process_request in original class.
In original UPS class provided in gem there is one static method defined self.retry_safe = true
and I can use it without errors.
I can also use regular_method after creating instance of UPS class.
More details provided:
I'm working with Rails 2.3 ( :-( ) and Ruby 1.9.2. I have no influce on environment.
Monkey patched code is under plugins/my_plugin/lib/active_shipping/ext/carriers/ups.rb
In /active_shipping I have file named extensions.rb in which i have:
require 'active_shipping'
require_relative 'ext/carriers'
require_relative 'ext/carriers/ups'
It deals with loading everything properly (I suppose basing on regular_method beheaviour from first chunk of code in my question).
I try to invoke process_request in one of my Controllers. This part is little tricky, beacuse i'm using sth like this:
MyModel.courier_service.process_request(parameters)
where courier_service, in this case holds the ActiveMerchant::Shipping::UPS class.
I'm still a newbie in Ruby and don't know what sort of details i should provide.
Maybe you want to do it in another way
File patch_classes.rb:
module ActiveMerchantExpand
module Shipping
module ClassMethods
def self.process_request(receiver, sender, packages, options = {})
# some code
end
end
module InstanceMethods
def regular_method
"foobar"
end
end
def self.included(receiver)
receiver.extend ClassMethods
receiver.send :include, InstanceMethods
end
end
end
Then you have to load your class "ActiveMerchant::Shipping::UPS"
and after that you can attach your methods to your class via
Rails.configuration.to_prepare do
require_dependency [[file for ActiveMerchant::Shipping::UPS]]
require 'patch_classes' )
ActiveMerchant::Shipping::UPS.send(:include, ::ActiveMerchantExpand::Shipping)
end
This is from rails plugin writing, i hope this helps.
regards tingel2k
Do you explicitly require file with your monkey patch? If you just put it under your app or lib path without requiring, it wouldn't load because constant ActiveMerchant::Shipping::UPS is defined in gem and it doesn't trigger dependency resolution mechanism.