How do you test custom Bugsnag meta_data (in Ruby, with Rspec)?
The code that I want to test:
def do_something
thing_that_could_error
rescue => e
Bugsnag.notify(e) do |r|
r.meta_data = { my_extra_data: "useful info" }
end
end
The test I want to write:
context "when there's an error" do
it "calls Bugsnag with my special metadata" do
expect(Bugsnag).to receive(:notify) # TODO test meta_data values contain "my useful info"
expect do
do_something() # exception is thrown and rescued and sent to Bugsnag
end.not_to raise_error
end
end
I am using:
Ruby 2.6.6
Rspec 3.9.0
Bugsnag 6.17.0 https://rubygems.org/gems/bugsnag
The data inside of the meta_data variable is considerably more complicated than in this tiny example, which is why I want to test it. In a beautiful world, I would extract that logic to a helper and test the helper, but right now it is urgent and useful to test in situ.
I've been looking at the inside of the Bugsnag gem to figure this out (plus some Rspec-fu to capture various internal state and returned data) but at some point it's a good idea to ask the internet.
Since the metadata is complicated, I'd suggest simplifying it:
def do_something
thing_that_could_error
rescue => e
Bugsnag.notify(e) do |r|
r.meta_data = error_metadata(e, self, 'foo')
end
end
# I assumed that you'd like to pass exception and all the context
def error_metadata(e, object, *rest)
BugsnagMetadataComposer.new(e, object, *rest).metadata
end
So now you can have a separate test for BugsnagMetadataComposer where you have full control (without mocking) over how you initialize it, and test for metadata output.
Now you only have to test that BugsnagMetadataComposer is instantiated with the objects you want, metadata is called and it returns dummy hash:
let(:my_exception) { StandardError.new }
let(:mock_metadata) { Hash.new }
before do
# ensure thing_that_could_error throws `my_exception`
expect(BugsnagMetadataComposer)
.to receive(new)
.with(my_exception, subject, anything)
.and_return(mock_metadata)
end
And the hard part, ensure that metadata is assigned. To do that you can cheat a little and see how Bugsnag gem is doing it
Apparently there's something called breadcrumbs:
let(:breadcrumbs) { Bugsnag.configuration.breadcrumbs }
Which I guess has all the Bugsnag requests, last one on top, so you can do something similar to https://github.com/bugsnag/bugsnag-ruby/blob/f9c539670c448f7f129a3f8be7d412e2e824a357/spec/bugsnag_spec.rb#L36-L40
specify do
do_something()
expect(breadcrumbs.last.metadata).to eq(expected_metadata)
end
And for clarity, the whole spec would look a bit like this:
let(:my_exception) { StandardError.new }
let(:mock_metadata) { Hash.new }
before do
# ensure thing_that_could_error throws `my_exception`
expect(BugsnagMetadataComposer)
.to receive(new)
.with(my_exception, subject, anything)
.and_return(mock_metadata)
end
specify do
do_something()
expect(breadcrumbs.last.metadata).to eq(expected_metadata)
end
Related
I would like to achieve 100% coverage on a module. My problem is that there is a variable (called data) within a method which I am trying to inject data in to test my exception handling. Can this be done with mocking? If not how can i fully test my exception handling?
module CSV
module Extractor
class ConversionError < RuntimeError; end
class MalformedCSVError < RuntimeError; end
class GenericParseError < RuntimeError; end
class DemoModeError < RuntimeError; end
def self.open(path)
data = `.\\csv2text.exe #{path} -f xml --xml_output_styles 2>&1`
case data
when /Error: Wrong input filename or path:/
raise MalformedCSVError, "the CSV path with filename '#{path}' is malformed"
when /Error: A valid password is required to open/
raise ConversionError, "Wrong password: '#{path}'"
when /CSVTron CSV2Text: This page is skipped when running in the demo mode./
raise DemoModeError, "CSV2TEXT.exe in demo mode"
when /Error:/
raise GenericParseError, "Generic Error Catch while reading input file"
else
begin
csvObj = CSV::Extractor::Document.new(data)
rescue
csvObj = nil
end
return csvObj
end
end
end
end
Let me know what you think! Thanks
===================== EDIT ========================
I have modified my methods to the design pattern you suggested. This method-"open(path)" is responsible for trapping and raising errors, get_data(path) just returns data, That's it! But unfortunately in the rspec I am getting "exception was expected to be raise but nothing was raised." I thought maybe we have to call the open method from your stub too?
This is what I tried doing but still no error was raised..
it 'should catch wrong path mode' do
obj = double(CSV::Extractor)
obj.stub!(:get_data).and_return("Error: Wrong input filename or path:")
obj.stub!(:open)
expect {obj.open("some fake path")}.to raise_error CSV::Extractor::MalformedCSVError
end
Extract the code that returns the data to a separate method. Then when you test open you can stub out that method to return various strings that will exercise the different branches of the case statement. Roughly like this for the setup:
def self.get_data(path)
`.\\csv2text.exe #{path} -f xml --xml_output_styles 2>&1`
end
def self.open(path)
data = get_data(path)
...
And I assume you know how to stub methods in rspec, but the general idea is like this:
foo = ...
foo.stub(:get_data).and_return("Error: Wrong input filename or path:")
expect { foo.get_data() }.to raise_error MalformedCSVError
Also see the Rspec documentation on testing for exceptions.
Problem with testing your module lies in the way you have designed your code. Think about splitting extractor into two classes (or modules, it's matter of taste -- I'd go with classes as they are a bit easier to test), of which one would read data from external system call, and second would expect this data to be passed as an argument.
This way you can easily mock what you currently have in data variable, as this would be simply passed as an argument (no need to think about implementation details here!).
For easier usage you can later provide some wrapper call, that would create both objects and pass one as argument to another. Please note, that this behavior can also be easily tested.
I am running rspec tests on a catalog object from within a Ruby app, using Rspec::Core::Runner::run:
File.open('/tmp/catalog', 'w') do |out|
YAML.dump(catalog, out)
end
...
unless RSpec::Core::Runner::run(spec_dirs, $stderr, out) == 0
raise Puppet::Error, "Unit tests failed:\n#{out.string}"
end
(The full code can be found at https://github.com/camptocamp/puppet-spec/blob/master/lib/puppet/indirector/catalog/rest_spec.rb)
In order to pass the object I want to test, I dump it as YAML to a file (currently /tmp/catalog) and load it as subject in my tests:
describe 'notrun' do
subject { YAML.load_file('/tmp/catalog') }
it { should contain_package('ppet') }
end
Is there a way I could pass the catalog object as subject to my tests without dumping it to a file?
I am not very clear as to what exactly you are trying to achieve but from my understanding I feel that using a before(:each) hook might be of use to you. You can define variables in this block that are available to all the stories in that scope.
Here is an example:
require "rspec/expectations"
class Thing
def widgets
#widgets ||= []
end
end
describe Thing do
before(:each) do
#thing = Thing.new
end
describe "initialized in before(:each)" do
it "has 0 widgets" do
# #thing is available here
#thing.should have(0).widgets
end
it "can get accept new widgets" do
#thing.widgets << Object.new
end
it "does not share state across examples" do
#thing.should have(0).widgets
end
end
end
You can find more details at:
https://www.relishapp.com/rspec/rspec-core/v/2-2/docs/hooks/before-and-after-hooks#define-before(:each)-block
I am printing some custom messages in my application using the puts command. However, I do not want these to be appearing in my Test Output. So, I tried a way to stub puts as shown below. But it still outputs my messages. What am I doing wrong ?
stubs(:puts).returns("") #Did not work out
Object.stubs(:puts).returns("") #Did not work out either
puts.stubs.returns "" #Not working as well
Kernel.stubs(:puts).returns "" #No luck
I am using Test::Unit
You probably need to stub it on the actual instance that calls puts. E.g. if you're calling puts in an instance method of a User class, try:
user = User.new
user.stubs(:puts)
user.some_method_that_calls_puts
This similarly applies to when you're trying to test puts in the top-level execution scope:
self.stubs(:puts)
What I would do is define a custom log method (that essentially calls puts for now) which you can mock or silence in test quite easily.
This also gives you the option later to do more with it, like log to a file.
edit: Or if you really want to stub puts, and you are calling it inside an instance method for example, you can just stub puts on the instance of that class.
Using Rails 5 + Mocha: $stdout.stubs(puts: '')
So the comments to the original post point to the answer:
Kernel.send(:define_method, :puts) { |*args| "" }
Instead of silencing all output, I would only silence output from the the particular objects that are putsing during your tests.
class TestClass
def some_method
...
puts "something"
end
end
it "should do something expected" do
TestClass.send(:define_method, :puts) { |*args| "" }
test_class.some_method.should == "abc123"
end
How do I stub a file.read call so that it returns what I want it to? The following does not work:
def write_something
File.open('file.txt') do |f|
return contents = f.read
end
end
# rspec
describe 'stub .read' do
it 'should work' do
File.stub(:read) { 'stubbed read' }
write_something.should == 'stubbed read'
end
end
It looks like the stub is being applied to the File class and not the file instance inside my block. So File.read returns stubbed read as expected. But when I run my spec it fails.
I should note that File.open is just one part of Ruby’s very large I/O API, and so your test is likely to be very strongly coupled to your implementation, and unlikely to survive much refactoring. Further, one must be careful with “global” mocking (i.e. of a constant or all instances) as it can unintentionally mock usages elsewhere, causing confusing errors and failures.
Instead of mocking, consider either creating an actual file on disk (using Tempfile) or using a broader I/O mocking library (e.g. FakeFS).
If you still wish to use mocking you can somewhat safely stub File.open to yield a double (and only when called with the correct argument):
file = instance_double(File, read: 'stubbed read')
allow(File).to receive(:open).and_call_original
allow(File).to receive(:open).with('file.txt') { |&block| block.call(file) }
or, somewhat dangerously, stub all instances:
allow_any_instance_of(File).to receive(:read).and_return('stubbed read')
The main point is to make File.open to return an object that will respond to read with the content you want, here's the code:
it "how to mock ruby File.open with rspec 3.4" do
filename = 'somefile.txt'
content = "this would be the content of the file"
# this is how to mock File.open:
allow(File).to receive(:open).with(filename, 'r').and_yield( StringIO.new(content) )
# you can have more then one of this allow
# simple test to see that StringIO responds to read()
expect(StringIO.new(content).read).to eq(content)
result = ""
File.open('somefile.txt', 'r') { |f| result = f.read }
expect(result).to eq(content)
end
This is how I'd do it
describe 'write_something' do
it 'should work' do
file_double = instance_double('File')
expect(File).to receive(:open).with('file.txt').and_yield(file_double)
expect(file_double).to receive(:read).and_return('file content')
content = write_something
expect(content).to eq('file content')
end
end
The following code doesn't work, but it best show what I'm trying to achieve
context "this context describes the class" do
subject do
# described class is actually a module here
c = Class.new.extend(described_class)
c.some_method_that_has_been_added_through_extension
c
end
# ... testing the class itself here ...
context "instances of this class" do
subject do
# this doesn't work because it introduces a endless recursion bug
# which makes perfectly sense
subject.new
end
end
end
I also tried to use a local variable in the inner context that I initialized
with the subject, but no luck. Is there any way I can access the subject of a outer scope from within my subject definition in the inner scope?
Using #subject can sometimes cause trouble. It is "primarily intended" for use with the short-hand checks like #its.
It also can make example harder to read, as it can work to mask the name/intent of what you testing. Here's a blog post that David Chelimsky wrote on the topic of #subject and #let and their role in revealing intention: http://blog.davidchelimsky.net/blog/2012/05/13/spec-smell-explicit-use-of-subject/
Try using let, instead
https://www.relishapp.com/rspec/rspec-core/v/2-10/docs/helper-methods/let-and-let
Here is how I would most likely write it.
context "this context describes the class" do
let(:name_of_the_module) { Class.new.extend(described_class) }
before do
c.some_method_that_has_been_added_through_extension
end
# ... testing the class itself here ...
context "instances of this class" do
let(:better_name_that_describes_the_instance) { klass.new }
# ... test the instance
end
end
SIDENOTE
You might want to revisit whether you want to use subject at all. I prefer using #let in almost all cases. YMMV
Something that obviously works is using an instance variable in the inner context and initializing it not with the subject but subject.call instead. Subjects are Procs. Hence, my first approach didn't work.
context "instances of this class" do
klass = subject.call
subject { klass.new }
end
I have been looking for a solution to this, but for different reasons. When I test a method that could return a value or raise an error, I often have to repeat the subject in two contexts, once as a proc for raise_error and once normally.
What I discovered is that you can give subjects names, like lets. This let's you reference an named subject from an outer scope within a new subject. Here's an example:
describe 'do_some_math' do
let!(:calculator) { create(:calculator) }
# proc to be used with raise_error
subject(:do_some_math) {
-> { calculator.do_some_math(with, complicated, args) }
}
context 'when something breaks' do
it { is_expected.to raise_error } # ok
end
context 'when everything works' do
# here we call the named subject from the outer scope:
subject { do_some_math.call } # nice and DRY
it { is_expected.to be_a(Numeric) } # also ok!
end
end