Why doesn't MiniTest::Spec have a wont_raise assertion? - ruby

Ruby's Test::Unit has assert_nothing_raised. Test::Unit has been replaced by MiniTest. Why don't MiniTest's assertions / expectations have anything parallel to this? For example you can expect must_raise but not wont_raise.

MiniTest does implement assert_nothing_raised in its Test::Unit compatibility layer, but in its own tests (MiniTest::Unit and MiniTest::Spec) it does not implement any test like this. The reason is, the programmer argues, that testing for nothing raised is not a test of anything; you never expect anything to be raised in a test, except when you are testing for an exception. If an unexpected (uncaught) exception occurs in the code for a test, you'll get an exception reported in good order by the test and you'll know you have a problem.
Example:
require 'minitest/autorun'
describe "something" do
it "does something" do
Ooops
end
end
Output:
Run options: --seed 41521
# Running tests:
E
Finished tests in 0.000729s, 1371.7421 tests/s, 0.0000 assertions/s.
1) Error:
test_0001_does_something(something):
NameError: uninitialized constant Ooops
untitled:5:in `block (2 levels) in <main>'
1 tests, 0 assertions, 0 failures, 1 errors, 0 skips
Which is exactly what you wanted to know. If you were expecting nothing to be raised, you didn't get it and you've been told so.
So, the argument here is: do not use assert_nothing_raised! It's just a meaningless crutch. See, for example:
https://github.com/seattlerb/minitest/issues/70
https://github.com/seattlerb/minitest/issues/159
http://blog.zenspider.com/blog/2012/01/assert_nothing_tested.html
On the other hand, clearly assert_nothing_raised corresponds to some intuition among users, since so many people expect a wont_raise to go with must_raise, etc. In particular one would like to focus an assertion on this, not merely a test. Luckily, MiniTest is extremely minimalist and flexible, so if you want to add your own routine, you can. So you can write a method that tests for no exception and returns a known outcome if there is no exception, and now you can assert for that known outcome.
For example (I'm not saying this is perfect, just showing the idea):
class TestMyRequire < MiniTest::Spec
def testForError # pass me a block and I'll tell you if it raised
yield
"ok"
rescue
$!
end
it "blends" do
testForError do
something_or_other
end.must_equal "ok"
end
end
The point is not that this is a good or bad idea but that it was never the responsibility of MiniTest to do it for you.

If you need it:
# test_helper.rb
module Minitest::Assertions
def assert_nothing_raised(*)
yield
end
end
And to use it:
def test_unknown_setter
assert_nothing_raised do
result.some_silly_column_name = 'value'
end
end

This bothered me enough to dig into the MiniTest sources and provide an implementation in my spec_helper.rb file:
module MiniTest
module Assertions
def refute_raises *exp
msg = "#{exp.pop}.\n" if String === exp.last
begin
yield
rescue MiniTest::Skip => e
return e if exp.include? MiniTest::Skip
raise e
rescue Exception => e
exp = exp.first if exp.size == 1
flunk "unexpected exception raised: #{e}"
end
end
end
module Expectations
infect_an_assertion :refute_raises, :wont_raise
end
end
Hope this proves helpful to someone else who also needs wont_raise. Cheers! :)

Related

(RSpec) How do I check if an object is created?

I want to test if expected exception handling is taking place in the following Ruby code through RSpec. Through testing I realized that I cannot use the raise_error matcher to test if the exception was raised, after rescuing it.
So, now I want to test whether objects of CustomError and StandardError are created to see if the error was raised as expected.
test.rb
module TestModule
class Test
class CustomError < StandardError
end
def self.func(arg1, arg2)
raise CustomError, 'Inside CustomError' if arg1 >= 10 && arg2 <= -10
raise StandardError, 'Inside StandardError' if arg1.zero? && arg2.zero?
rescue CustomError => e
puts 'Rescuing CustomError'
puts e.exception
rescue StandardError => e
puts 'Rescuing StandardError'
puts e.exception
ensure
puts "arg1: #{arg1}, arg2: #{arg2}\n"
end
end
end
test_spec.rb
require './test'
module TestModule
describe Test do
describe '#func' do
it 'raises CustomError when arg1 >= 10 and arg2 <= -10' do
described_class.func(11, -11)
expect(described_class::CustomError).to receive(:new)
end
end
end
end
When I run the above code I get the following error
Failures:
1) TestModule::Test#func raises CustomError when arg1 >= 10 and arg2 <= -10
Failure/Error: expect(described_class::CustomError).to receive(:new)
(TestModule::Test::CustomError (class)).new(*(any args))
expected: 1 time with any arguments
received: 0 times with any arguments
# ./test_spec.rb:8:in `block (3 levels) in <module:TestModule>'
The idea was that if CustomError is being raised, it's obejct must be created using new and I can test that using RSpec. However, as you can see, this isn't working.
What am I missing?
Is there a better approach?
What you are trying to do is to test implementation details (what happens inside the method), and usually it's a bad idea. If a particular path leads to an exception but you want this exception to be swallowed - test that.
consider
def weird_division(x, y)
x/y
rescue ZeroDivisionError => e
return "Boom!"
end
No need to test that ZeroDivisionError has been created, that's an implementation detail (akin to testing private methods). Test behavior that is "visible" from the outside.
expect(weird_division(1/0)).to return "Boom!"
Because you might change the implementation:
def weird_division(x, y)
return "Boom!" if y == 0
x/y
end
And your tests would start failing, even though the method behaves the same.
I see two possibilities.
If you want to continue with your assertion you can use have_received instead of receive. This will allow you to keep you assertion after you act.
expect(described_class::CustomError).to have_received(:new)
As mentioned in comments test your puts. Which is ultimately the way that you check if your method does what you want.
it 'puts the output' do
expect do
described_class.func(11, -11)
end.to output('Some string you want to print').to_stdout
end

Ruby minitest assert_output syntax

I am new to minitest and still new to ruby and really tired of trying to google this question without result. I would be really grateful for help:
What is the exact syntax of assert_output in ruby minitest?
All I find on github or elsewhere seems to use parentheses. Yet, I get an error message when I don't use a block with assert_output, which makes sense as the definition of this method contains a yield statement.
But I cannot make it work, whatever I try.
testclass.rb
class TestClass
def output
puts 'hey'
end
end
test_test.rb
require 'minitest/spec'
require 'minitest/autorun'
require_relative 'testclass'
class TestTestClass < MiniTest::Unit::TestCase
def setup
#test = TestClass.new
end
def output_produces_output
assert_output( stdout = 'hey' ) { #test.output}
end
end
What I get is:
Finished tests in 0.000000s, NaN tests/s, NaN assertions
0 tests, 0 assertions, 0 failures, 0 errors, 0 skips
What am I doing wrong?
It must be something totally obvious, but I cannot figure it out.
Thanks for your help.
In order for your test method to run, the method name needs to start with test_. Also, the way assert_output works is that the block will write to stdout/stderr, and the arguments will be checked if they match stdout/stderr. The easiest way to check this IMO is to pass in a regexp. So this is how I would write that test:
class TestTestClass < MiniTest::Unit::TestCase
def setup
#test = TestClass.new
end
def test_output_produces_output
assert_output(/hey/) { #test.output}
end
end

How do I test yield_control multiple times to a block that raises an error?

I'd like to use the yield_control matcher in RSpec like so
expect {|b| some_method(&b) }.to yield_control.exactly(5).times
However, I'd like the block to raise an error. Is there an RSpec construct that allows you to do this?
Basically, I'm looking for something like an RSpec mock object that allows you to raise an error whenever the block is yielded to and count the number of times it was yielded to.
I understand there are other ways of doing it by defining my own block that keeps track of how many times it has been called, but I was hoping there is a built-in that does something like this.
Example:
def some_method(&block)
attempt = 0
begin
attempt += 1
yield
rescue => e
retry if attempt < 3
end
end
I'd like to test that control is yielded a certain number of times when the block raises an error. I understand that this is one way to do it:
number_of_times = 0
some_method do
number_of_times += 1
raise StandardError
end
expect(number_of_times).to eq(3)
But I was hoping I could use the expect {..}.to yield_control syntax.
Are you expecting the block to raise an error? There is an Rspec matcher for that: https://www.relishapp.com/rspec/rspec-expectations/docs/built-in-matchers/raise-error-matcher

Weird error when trying to test method with argument in Mocha. Is it a bug or is it me?

It's rather hard to find any documentation on Mocha, so I'm afraid I'm totally at sea here. I have found a problem with stubbing methods that pass arguments. So for instance if I set up a class like this:
class Red
def gets(*args)
#input.gets(*args)
end
def puts(*args)
#output.puts(*args)
end
def initialize
#input = $stdin
#output = $stdout
end
private
def first_method
input = gets.chomp
if input == "test"
second_method(input)
end
end
def second_method(value)
puts value
second_method(value)
end
end
Yes it's contrived, but it's a simplification of the idea that you may have a method that you don't want called in the test.
So I might write a test such as:
setup do
#project = Red.new
#project.instance_variable_set(:#input, StringIO.new("test\n"))
#project.stubs(:second_method)
end
should "pass input value to second_method" do
#project.expects(:second_method).with("test").once
#project.instance_eval {first_method}
end
Now I would expect this to pass. But instead I get this rather arcane error message:
Errno::ENOENT: No such file or directory - getcwd
/Users/i0n/.rvm/gems/ruby-1.9.2-head/gems/mocha-0.9.8/lib/mocha/backtrace_filter.rb:12:in `expand_path'
/Users/i0n/.rvm/gems/ruby-1.9.2-head/gems/mocha-0.9.8/lib/mocha/backtrace_filter.rb:12:in `block in filtered'
/Users/i0n/.rvm/gems/ruby-1.9.2-head/gems/mocha-0.9.8/lib/mocha/backtrace_filter.rb:12:in `reject'
/Users/i0n/.rvm/gems/ruby-1.9.2-head/gems/mocha-0.9.8/lib/mocha/backtrace_filter.rb:12:in `filtered'
/Users/i0n/.rvm/gems/ruby-1.9.2-head/gems/mocha-0.9.8/lib/mocha/expectation_error.rb:10:in `initialize'
/Users/i0n/.rvm/gems/ruby-1.9.2-head/gems/mocha-0.9.8/lib/mocha/mockery.rb:53:in `new'
/Users/i0n/.rvm/gems/ruby-1.9.2-head/gems/mocha-0.9.8/lib/mocha/mockery.rb:53:in `verify'
/Users/i0n/.rvm/gems/ruby-1.9.2-head/gems/mocha-0.9.8/lib/mocha/api.rb:156:in `mocha_verify'
/Users/i0n/.rvm/gems/ruby-1.9.2-head/gems/mocha-0.9.8/lib/mocha/integration/mini_test/version_131_and_above.rb:27:in `run'
This means absolutely nothing to me, other than something deep in Mochas bowels has just gone clang. If I write the same sort of test without an argument passing to the second method I get no problem. Am I missing something?
I think it must be something in shoulda causing the problem. I use test/unit, and everything appears to be OK.
require 'rubygems'
require "test/unit"
require 'mocha'
require File.dirname(__FILE__) + '/../src/red'
class RedTest < Test::Unit::TestCase
def setup
#project = Red.new
#project.instance_variable_set(:#input, StringIO.new("test\n"))
#project.stubs(:second_method)
end
def test_description_of_thing_being_tested
#project.expects(:second_method).with("test").once
#project.instance_eval {first_method}
end
end
gives the following output:
stephen#iolanta:~/tmp/red/test # ruby red_test.rb
Loaded suite red_test
Started
.
Finished in 0.000679 seconds.
1 tests, 1 assertions, 0 failures, 0 errors
stephen#iolanta:~/tmp/red/test #
Sorry - I've only just seen this. It's better to submit bug reports to us in Lighthouse. What documentation have you found? Have you seen the RDoc on Rubyforge? What sort of documentation were you looking for that you did not find?
I've been unable to reproduce your bug. What version of Ruby, Rubygems, Shoulda & Mocha were you using?
You can see the results of me running your test in this Gist.

DRY way of re-raising same set of exceptions in multiple places

short:
Is there a way in Ruby to DRY-ify this:
def entry_point_one
begin
do_something
rescue MySyntaxErrorOne, MySyntaxErrorTwo, MySyntaxErrorEtc => syn_err
raise syn_err.exception(syn_err.message)
end
end
def entry_point_two
begin
do_something_else
rescue MySyntaxErrorOne, MySyntaxErrorTwo, MySyntaxErrorEtc => syn_err
raise syn_err.exception(syn_err.message)
end
end
longer:
I'm building an interpreter. This interpreter can be called using different entry points. If I feed this interpreter a 'dirty' string, I expect it to raise an error. However, it would be nice if I don't get spammed by the by the entire back trace of every method called directly or indirectly by do_something, especially since the interpreter makes use of recursion.
As you can see in the above snippet, I already know a way to re raise an error and thereby removing the back trace. What I would like do is remove the duplication in the above example. The closest I have come thus far is this:
def entry_point_one
re_raise_known_exceptions {do_something}
end
def entry_point_two
re_raise_known_exceptions {do_something_else}
end
def re_raise_known_exceptions
yield
rescue MySyntaxErrorOne, MySyntaxErrorTwo, MySyntaxErrorEtc => syn_err
raise syn_err.exception(syn_err.message)
end
But that makes the method re-raise-known-exceptions show up in the back trace.
edit: I guess what I want would be something like a C pre-processing macro
You can just use the splat on an array.
Straight from IRB:
COMMON_ERRORS = [ArgumentError, RuntimeError] # add your own
def f
yield
rescue *COMMON_ERRORS => err
puts "Got an error of type #{err.class}"
end
f{ raise ArgumentError.new }
Got an error of type ArgumentError
f{ raise 'abc' }
Got an error of type RuntimeError
while thinking about it a bit more, I came up with this:
interpreter_block {do_something}
def interpreter_block
yield
rescue ExceptionOne, ExceptionTwo, ExceptionEtc => exc
raise exc.exception(exc.message)
end
Although it's still not quiet what I would like to have, at least now the extra entry in the back trace has become somewhat better looking.
It might be slightly evil, but I think you can simply remove the line from the backtrace ;-)
COMMON_ERRORS = [ArgumentError, RuntimeError]
def interpreter_block
yield
rescue *COMMON_ERRORS => err
err.backtrace.delete_if{ |line| line=~/interpreter_block/ }
raise err
end
I'm not sure it's such a good idea though. You'll have a hell of a lot of fun debugging your interpreter afterward ;-)
Side note: Treetop may be of interest to you.
This is a touch hackish, but as far as cleaning up the backtrace goes, something like this works nicely:
class Interpreter
def method1
error_catcher{ puts 1 / 0 }
end
def error_catcher
yield
rescue => err
err.set_backtrace(err.backtrace - err.backtrace[1..2])
raise err
end
end
The main trick is this line err.set_backtrace(err.backtrace - err.backtrace[1..2]). Without it, we get the following (from IRB):
ZeroDivisionError: divided by 0
from (irb):43:in `/'
from (irb):43:in `block in method1'
from (irb):47:in `error_catcher'
from (irb):43:in `method1'
from (irb):54
from /Users/peterwagenet/.ruby_versions/ruby-1.9.1-p129/bin/irb:12:in `<main>'
What we don't want in there are the second and third lines. So we remove them, ending up with:
ZeroDivisionError: divided by 0
from (irb):73:in `/'
from (irb):73:in `method1'
from (irb):84
from /Users/peterwagenet/.ruby_versions/ruby-1.9.1-p129/bin/irb:12:in `<main>'
If you have all of the information you need in the exceptions, and you do not need the backtrace at all, you can just define your own error and raise that, instead of reraising the existing exception. This will give it a fresh backtrace. (Of course, presumably your sample code is incomplete and there is other processing happening in the rescue block -- otherwise your best bet is to just let the error bubble up naturally.)
class MyError < StandardError; end
def interpreter_block
yield
rescue ExceptionOne, ExceptionTwo, ExceptionEtc => exc
raise MyError
end

Resources