I've come across some counter intuitive behavior in RSpec (2.8.0) custom matchers functionality and I wonder whether it is a bug or feature or me being confused. Let's look at the code:
# matcher code
RSpec::Matchers.define :exist do
chain :with_start_time do |time|
#start_time = time
end
chain :with_end_time do |time|
#end_time = time
end
match do |subject|
result = true
result &&= subject.start_time == #start_time if #start_time
result &&= subject.end_time == #end_time if #end_time
result
end
failure_message_for_should do |subject|
"Failure!\n".tap do |msg|
if #start_time != subject.start_time
msg << "Expected start_time to be ##start_time but was #{subject.start_time}\n"
end
if #end_time != subject.end_time
msg << "Expected end_time to be ##end_time but was #{subject.end_time}\n"
end
end
end
end
#spec code
require 'ostruct'
describe 'RSpec custom matcher keeping state between tests' do
let(:time) { Time.now }
it 'passes the first spec' do
o = OpenStruct.new(start_time: time)
o.should exist.with_start_time(time)
end
it 'fails the second because matcher kept #start_time from the first test' do
o = OpenStruct.new(end_time: time)
o.should exist.with_end_time(time)
end
end
This fails (demonstrating the issue):
avetia01:~/projects/custom_matcher_bug% rspec test_spec.rb
.F
Failures:
1) RSpec custom matcher keeping state between tests fails the second because matcher kept #start_time from the first test
Failure/Error: o.should exist.with_end_time(time)
Failure!
Expected start_time to be 2012-02-27 12:20:25 +0000 but was
# ./test_spec.rb:41:in `block (2 levels) in <top (required)>'
Finished in 0.00116 seconds
2 examples, 1 failure
So the unexpected bit is that the same matcher instance seems to be used across multiple specs. Which, in this particular case leads, to #start_time being initialized with the value from the first spec, causing incorrect failure of the second spec.
This has reported and fixed, but not yet released:
https://github.com/rspec/rspec-expectations/issues/104
Related
The class being tested qa.rb contains the code:
class QA
def initialize(bugs: 0)
#bugs = bugs
end
def speak
"Hello!"
end
def happy?
#bugs > 0
end
def debug
#bugs = 0
end
end
The RSpec file qa_spec.rb contains the code:
require 'rspec'
require_relative 'qa'
RSpec.describe QA do
describe '#happy?' do
context 'when bugs are more than 0' do
it 'returns true' do
subject { described_class.new(bugs: 1) }
expect(subject).to be_happy
end
end
end
end
The test fails when I run it, and gives me this error:
PS C:\Users\Jobla\repos\TDD> rspec qa_spec.rb
F
Failures:
1) QA#happy? when bugs are more than 0 returns true
Failure/Error: expect(subject).to be_happy
expected `#<QA:0x2e0d640 #bugs=0>.happy?` to return true, got false
# ./qa_spec.rb:9:in `block (4 levels) in <top (required)>'
Finished in 0.02999 seconds (files took 0.16995 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./qa_spec.rb:7 # QA#happy? when bugs are more than 0 returns true
However, when I edit qa_spec.rb and I swap the it and subject lines, the test suddenly passes:
require 'rspec'
require_relative 'qa'
RSpec.describe QA do
describe '#happy?' do
context 'when bugs are more than 0' do
subject { described_class.new(bugs: 1) } #swapped with line below
it 'returns true' do #swapped with line above
expect(subject).to be_happy
end
end
end
end
Tests pass:
PS C:\Users\Jobla\repos\TDD> rspec qa_spec.rb
.
Finished in 0.01003 seconds (files took 0.17993 seconds to load)
1 example, 0 failures
Please could someone explain why does swapping the it and subject lines change the result of the test?
subject is designed to be set in context or describe block, but not in it.
If you do not set subject before it then subject would be set automatically by calling new without parameters on described_class. bugs will be set to default 0. After that, you call it with a block subject { described_class.new(bugs: 1) } inside it, it's the same as if you call described_class.new { described_class.new(bugs: 1) } because subject inside it is an instance of QA class.
I have a large amount of Minitest unit tests (methods), over 300. They all take some time, from a few milliseconds to a few seconds. Some of them hang up, sporadically. I can't understand which one and when.
I want to apply Timeout to each of them, to make sure anyone fails if it takes longer than, say, 5 seconds. Is it achievable?
For example:
class FooTest < Minitest::Test
def test_calculates_something
# Something potentially too slow
end
end
You can use the Minitest PLugin loader to load a plugin. This is, by far, the cleanest solution. The plugin system is not very well documented, though.
Luckily, Adam Sanderson wrote an article on the plugin system.
The best news is that this article explains the plugin system by writing a sample plugin that reports slow tests. Try out minitest-snail, it is probably almost what you want.
With a little modification we can use the Reporter to mark a test as failed if it is too slow, like so (untested):
File minitest/snail_reporter.rb:
module Minitest
class SnailReporter < Reporter
attr_reader :max_duration
def self.options
#default_options ||= {
:max_duration => 2
}
end
def self.enable!(options = {})
#enabled = true
self.options.merge!(options)
end
def self.enabled?
#enabled ||= false
end
def initialize(io = STDOUT, options = self.class.options)
super
#max_duration = options.fetch(:max_duration)
end
def record result
#passed = result.time < max_duration
slow_tests << result if !#passed
end
def passed?
#passed
end
def report
return if slow_tests.empty?
slow_tests.sort_by!{|r| -r.time}
io.puts
io.puts "#{slow_tests.length} slow tests."
slow_tests.each_with_index do |result, i|
io.puts "%3d) %s: %.2f s" % [i+1, result.location, result.time]
end
end
end
end
File minitest/snail_plugin.rb:
require_relative './snail_reporter'
module Minitest
def self.plugin_snail_options(opts, options)
opts.on "--max-duration TIME", "Report tests that take longer than TIME seconds." do |max_duration|
SnailReporter.enable! :max_duration => max_duration.to_f
end
end
def self.plugin_snail_init(options)
if SnailReporter.enabled?
io = options[:io]
Minitest.reporter.reporters << SnailReporter.new(io)
end
end
end
I'm writing some tests for my backend jobs and I'm having a weird issue with rspec not finding my methods.
I wrote a simple class & test to illustrate the issue :
app/interactors/tmp_test.rb :
class TmpTest
def call
a = 10
b = 5
b.substract_two
return a + b
end
def substract_two
c = self - 2
return c
end
end
spec/interactors/tmp_test.rb :
require 'rails_helper'
describe TmpTest do
context 'when doing the substraction' do
it 'return the correct number' do
expect(described_class.call).to eq(13)
end
end
end
output:
TmpTest
when doing the substraction
return the correct number (FAILED - 1)
Failures:
1) TmpTest when doing the substraction return the correct number
Failure/Error: expect(described_class.call).to eq(13)
NoMethodError:
undefined method `call' for TmpTest:Class
# ./spec/interactors/tmp_test.rb:6:in `block (3 levels) in <top (required)>'
Finished in 0.00177 seconds (files took 1.93 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/interactors/tmp_test.rb:5 # TmpTest when doing the substraction return the correct number
It's not a class method, it's an instance method. Your test should look like this:
describe TmpTest do
subject(:instance) { described_class.new }
context 'when doing the subtraction' do
it 'returns the correct number' do
expect(instance.call).to eq(13)
end
end
end
This is a complete mess. Corrected version with comments:
class TmpTest
def call
a = 10
b = 5
# b.substract_two # why do you call method of this class on b?!
a + subtract_two(b)
end
def substract_two(from)
from - 2
end
end
Also: don’t use return in the very last line of the method.
I have written a custom match method in Rspec that matches an object against a hash. What I am trying to do is set custom failure messages for each line of the expect.
describe "/cars" do
car = FactoryGirl.create(:car, name: 'Alpha')
describe car do
it "displays single items" do
get cars_path
parsed_response = JSON.parse(response.body)
record_hash = parsed_response['cars'][0]
is_expected.to be_a_car_match_of(record_hash)
end
end
end
RSpec::Matchers.define :be_a_car_match_of do |hash|
match do |car|
expect(car.id).to eq(hash['id'])
expect(car.name).to eq(hash['name'])
end
failure_message do |car|
"expected that #{car} would be a match of #{hash}"
end
end
So what I would like is to have something like the following:
RSpec::Matchers.define :be_a_car_match_of do |hash|
match do |car|
expect(car.id).to eq(hash['id']) 'ids did not match'
expect(car.name).to eq(hash['name']) 'names did not match'
end
end
This would print out a much clearer error message.
I was initially doing this in mini-test but for a variety of reasons (outside of my control) needed to change it to rspec. The code I had in mini-test was:
def assert_car(car, hash)
assert_equal car.id, hash['id'], "ids did not match"
assert_equal car.name, hash['name'], "names did not match"
end
This is what I am trying to replicate.
Here is another example that requires less setup:
require 'rspec/expectations'
RSpec::Matchers.define :be_testing do |expected|
match do |actual|
expect(5).to eq(5)
expect(4).to eq(5)
end
failure_message do
"FAIL"
end
end
describe 'something' do
it 'something else' do
expect("expected").to be_testing('actual')
end
end
When this example is run, "FAIL" is printed out. On the other hand if I had:
describe 'something' do
it 'something else' do
expect(4).to eq(5)
end
end
I would get the following error message:
expected: 5
got: 4
This is what I want. I want to know what part of the custom matcher failed.
You could call the low-level matches? method instead of expect. Something like this:
require 'rspec/expectations'
RSpec::Matchers.define :be_a_positive_integer do
m1, m2 = nil, nil # matchers
r1, r2 = false, false # results
match do |actual|
m1 = be_a Integer # this returns matcher instances
m2 = be > 0
r1 = m1.matches?(actual) # evaluate matchers
r2 = m2.matches?(actual)
r1 && r2 # true if both are true
end
failure_message do |actual| # collect error messages from matchers
messages = []
messages << m1.failure_message unless r1
messages << m2.failure_message unless r2
messages.join("\n")
end
end
describe -1 do
it { is_expected.to be_a_positive_integer }
end
describe 1.0 do
it { is_expected.to be_a_positive_integer }
end
describe -1.0 do
it { is_expected.to be_a_positive_integer }
end
Output:
Failures:
1) -1 should be a positive integer
Failure/Error: it { is_expected.to be_a_positive_integer }
expected: > 0
got: -1
# ./ruby_spec.rb:24:in `block (2 levels) in <top (required)>'
2) 1.0 should be a positive integer
Failure/Error: it { is_expected.to be_a_positive_integer }
expected 1.0 to be a kind of Integer
# ./ruby_spec.rb:28:in `block (2 levels) in <top (required)>'
3) -1.0 should be a positive integer
Failure/Error: it { is_expected.to be_a_positive_integer }
expected -1.0 to be a kind of Integer
expected: > 0
got: -1.0
# ./ruby_spec.rb:32:in `block (2 levels) in <top (required)
I think aggregate_failures is what you are looking for :
It wraps a set of expectations with a block. Within the block, expectation failures will not immediately abort like normal; instead, the failures will be aggregated into a single exception that is raised at the end of the block, allowing you to see all expectations that failed.
See : https://relishapp.com/rspec/rspec-expectations/docs/aggregating-failures
Starting using rspec I have difficulties trying to test threaded code.
Here is a simplicfication of a code founded, and I made it cause i need a Queue with Timeout capabilities
require "thread"
class TimeoutQueue
def initialize
#lock = Mutex.new
#items = []
#new_item = ConditionVariable.new
end
def push(obj)
#lock.synchronize do
#items.push(obj)
#new_item.signal
end
end
def pop(timeout = :never)
timeout += Time.now unless timeout == :never
#lock.synchronize do
loop do
time_left = timeout == :never ? nil : timeout - Time.now
if #items.empty? and time_left.to_f >= 0
#new_item.wait(#lock, time_left)
end
return #items.shift unless #items.empty?
next if timeout == :never or timeout > Time.now
return nil
end
end
end
alias_method :<<, :push
end
But I can't find a way to test it using rspec. Is there any effective documentation on testing threaded code? Any gem that can helps me?
I'm a bit blocked, thanks in advance
When unit-testing we don't want any non-deterministic behavior to affect our tests, so when testing threading we should not run anything in parallel.
Instead, we should isolate our code, and simulate the cases we want to test, by stubbing #lock, #new_item, and perhaps even Time.now (to be more readable I've taken the liberty to imagine you also have attr_reader :lock, :new_item):
it 'should signal after push' do
allow(subject.lock).to receive(:synchronize).and_yield
expect(subject.new_item).to receive(:signal)
subject.push('object')
expect(subject.items).to include('object')
end
it 'should time out if taken to long to enter synchronize loop' do
#now = Time.now
allow(Time).to receive(:now).and_return(#now, #now + 10.seconds)
allow(subject.items).to receive(:empty?).and_return true
allow(subject.lock).to receive(:synchronize).and_yield
expect(subject.new_item).to_not receive(:wait)
expect(subject.pop(5.seconds)).to be_nil
end
etc...