Why does changing the order of 'it' and 'subject' in RSpec change my test result? - ruby

The class being tested qa.rb contains the code:
class QA
def initialize(bugs: 0)
#bugs = bugs
end
def speak
"Hello!"
end
def happy?
#bugs > 0
end
def debug
#bugs = 0
end
end
The RSpec file qa_spec.rb contains the code:
require 'rspec'
require_relative 'qa'
RSpec.describe QA do
describe '#happy?' do
context 'when bugs are more than 0' do
it 'returns true' do
subject { described_class.new(bugs: 1) }
expect(subject).to be_happy
end
end
end
end
The test fails when I run it, and gives me this error:
PS C:\Users\Jobla\repos\TDD> rspec qa_spec.rb
F
Failures:
1) QA#happy? when bugs are more than 0 returns true
Failure/Error: expect(subject).to be_happy
expected `#<QA:0x2e0d640 #bugs=0>.happy?` to return true, got false
# ./qa_spec.rb:9:in `block (4 levels) in <top (required)>'
Finished in 0.02999 seconds (files took 0.16995 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./qa_spec.rb:7 # QA#happy? when bugs are more than 0 returns true
However, when I edit qa_spec.rb and I swap the it and subject lines, the test suddenly passes:
require 'rspec'
require_relative 'qa'
RSpec.describe QA do
describe '#happy?' do
context 'when bugs are more than 0' do
subject { described_class.new(bugs: 1) } #swapped with line below
it 'returns true' do #swapped with line above
expect(subject).to be_happy
end
end
end
end
Tests pass:
PS C:\Users\Jobla\repos\TDD> rspec qa_spec.rb
.
Finished in 0.01003 seconds (files took 0.17993 seconds to load)
1 example, 0 failures
Please could someone explain why does swapping the it and subject lines change the result of the test?

subject is designed to be set in context or describe block, but not in it.
If you do not set subject before it then subject would be set automatically by calling new without parameters on described_class. bugs will be set to default 0. After that, you call it with a block subject { described_class.new(bugs: 1) } inside it, it's the same as if you call described_class.new { described_class.new(bugs: 1) } because subject inside it is an instance of QA class.

Related

How to hook after example has execution result and status with :aggregated_failures flag

I'm maintaining a standalone test automation suite written in Rspec & Capybara and SitePrism (No Rails).
Recently I started integrating it with Testrail for reporting - I used rspec-testrail gem for that but I had to modify it a bit, because I also wanted to send Slack notifications with test progress and report.
Anyway, the integration works smoothly, except the fact that example processing is relying on example having an exception and lack of exception causes setting the test status as Passed on Testrail.
As it appears, after :each nor after :example in Rspec.configure doesn't guarantee that the example has finished running.
I also tried around :example and around :each as described here, but to no avail.
I inspected contents of example.metadata and it looks that example.metadata[:execution_result] has only started_at variable, but a finished example would have also finished_at and status variables, according to the docs
My suspicion (after reading the relish docs) is that :aggregated_failures is the cause of different metadata structure and multiple expects running in threads that are later merged into one backtrace.
Do you know how can I wait for the example to finish or how to hook into the state where it's finished?
Or maybe I should create a custom formatter where I would hook after example notifications printed to the console (I would like to keep the stacktrace there).
My code is as follows:
Test (both assertions are failing):
require 'spec_helper'
feature 'Sign in' do
let(:login_page) { LoginPage.new }
let(:user) { { email: ENV['ADMIN_EMAIL'], password: ENV['ADMIN_PASSWORD'] } }
scenario 'is successful and user is redirected to dashboard for user with correct credentials', testrail_id: 5694 do
login_page.load
login_page.form.email.set(user[:email])
login_page.form.password.set(user[:password])
login_page.form.submit_btn.click
expect(login_page.sidebar).to have_jobs(text: "some_nonexistenttext")
login_page.logout
expect(current_url).to have_content "google.com"
end
end
Console output from the above test:
Failures:
1) Sign in is successful and user is redirected to dashboard for user with correct credentials
Got 2 failures:
1.1) Failure/Error: expect(login_page.sidebar).to have_jobs(text: "blala")
expected #has_jobs?({:text=>"some_nonexistenttext"}) to return true, got false
# ./spec/auth/login_spec.rb:13:in `block (3 levels) in <top (required)>'
1.2) Failure/Error: expect(current_url).to have_content "google.com"
expected to find text "google.com" in "https://example.com/"
# ./spec/auth/login_spec.rb:15:in `block (3 levels) in <top (required)>'
Finished in 53.91 seconds (files took 1.45 seconds to load)
4 examples, 1 failure
Failed examples:
rspec ./spec/auth/login_spec.rb:8 # Sign in is successful and user is redirected to dashboard for user with correct credentials
Spec helper:
require 'rubygems'
require 'capybara/rspec'
require 'selenium-webdriver'
require 'site_prism'
require 'slack-notifier'
require_relative '../config'
require_relative '../lib/testrail'
...
RSpec.configure do |config|
config.define_derived_metadata do |meta|
meta[:aggregate_failures] = true
end
config.example_status_persistence_file_path = 'examples.txt'
config.before :all do
testrail_initialize_test_run!
end
config.after :example, testrail_id: proc { |value| !value.nil? } do |example|
RSpec::Testrail.process(example)
end
end
processing method (slightly modified from original)
def process(example)
if example.exception
status = 5
message = example.exception.message
slack_notifier.publish(message_text "Failed")
elsif example.skipped? || example.pending?
puts 'Example skipped or pending'
status = 10
message = 'Pending or not implemented'
else
status = 1
message = ''
end
client.send_post("add_result_for_case/#{testrun['id']}/#{example.metadata[:testrail_id]}",
status_id: status,
comment: message)
end
So basically all I had to was to use a reporter listener and process notifications inside it :)
config.reporter.register_listener RSpec::Testrail::Listener.new, :start, :example_failed, :example_passed, :example_pending, :stop

Rspec not finding class methods

I'm writing some tests for my backend jobs and I'm having a weird issue with rspec not finding my methods.
I wrote a simple class & test to illustrate the issue :
app/interactors/tmp_test.rb :
class TmpTest
def call
a = 10
b = 5
b.substract_two
return a + b
end
def substract_two
c = self - 2
return c
end
end
spec/interactors/tmp_test.rb :
require 'rails_helper'
describe TmpTest do
context 'when doing the substraction' do
it 'return the correct number' do
expect(described_class.call).to eq(13)
end
end
end
output:
TmpTest
when doing the substraction
return the correct number (FAILED - 1)
Failures:
1) TmpTest when doing the substraction return the correct number
Failure/Error: expect(described_class.call).to eq(13)
NoMethodError:
undefined method `call' for TmpTest:Class
# ./spec/interactors/tmp_test.rb:6:in `block (3 levels) in <top (required)>'
Finished in 0.00177 seconds (files took 1.93 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/interactors/tmp_test.rb:5 # TmpTest when doing the substraction return the correct number
It's not a class method, it's an instance method. Your test should look like this:
describe TmpTest do
subject(:instance) { described_class.new }
context 'when doing the subtraction' do
it 'returns the correct number' do
expect(instance.call).to eq(13)
end
end
end
This is a complete mess. Corrected version with comments:
class TmpTest
def call
a = 10
b = 5
# b.substract_two # why do you call method of this class on b?!
a + subtract_two(b)
end
def substract_two(from)
from - 2
end
end
Also: don’t use return in the very last line of the method.

Error in Ruby/RSpec/WebDriver > expected respond to 'has_content?'

Can anyone explain this error?
My spec_helper.rb requires capybara, rspec, and selenium-webdriver.
My test_spec.rb file contains the following:
require_relative 'spec_helper'
#browser = Selenium::WebDriver.for :firefox
#browser.get "http://www.google.com"
describe 'ErrorCheck' do
it 'should log in to Trialnet' do
expect(#browser).to have_content('Search')
end
end
My error:
expected to respond to `has_content?`
./spec/webdriver3_spec.rb:9:in `block (2 levels) in <top (required)>'
-e:1:in `load'
-e:1:in `<main>'
Any idea why this expectation is failing? Is it returning a Boolean without the proper syntax to accept it?
This is happening because the #browser instance variable is not available to the it statement. Notice that your error message doesn't have a reference to the object that it is performing an expectation on (i.e. expected to respond to 'has_content?'.
Here's a contrived demonstration that shows it fail:
require 'rspec'
#x = 1
describe 'One' do
it 'should print 1' do
expect(#x).to eq 1
end
end
Failures:
1) One should print 1
Failure/Error: expect(#x).to eq 1
expected: 1
got: nil
And--by moving the instance variable into the it statement to available--the example passes:
require 'rspec'
describe 'One' do
it 'should print 1' do
#x = 1
expect(#x).to eq 1
end
end
1 example, 0 failures
And--if you use let--you can created a memoized variable that can be shared across examples:
require 'rspec'
describe 'One' do
let(:x) { 1 }
it 'should print 1' do
expect(x).to eq 1
end
it 'should print 2' do
expect(x+1).to eq 2
end
end
Based on your example code, you could use a before block for setup and then use subject, which is probably more appropriate than let (NOTE: snippet below is untested, and the difference between let and subject is covered in other SO answers, various blog posts, and rdoc):
describe 'ErrorCheck' do
before :all do
#browser = Selenium::WebDriver.for :firefox
#browser.get "http://www.google.com"
end
subject(:browser) {#browser} # or let(:browser) {#browser}
it 'should log in to Trialnet' do
expect(#browser).to have_content('Search')
end
end

Can rspec's "expect" parse a block to confirm a nested array/grid?

How can the nested array be checked using rspecs expect syntax?
This code block works using rspecs should syntax:
subject.cell_grid.each do |row|
row.is_a?(Array).should be_true
end
...and I think I've got the syntax correct on lines 22 & 23 of the "spec_game_of_life.rb" file, but, when I check the file with rspec I get the following error:
user#ubuntu:~/Ruby/GameOfLife$ rspec spec_game_of_life.rb
..F
Failures:
1) Game of Life world should create proper cell_grid upon initialization
Failure/Error:
expect(subject.cell_grid.each) do |row|
row.is_a?(Array).to be true
end
ArgumentError:
You cannot pass both an argument and a block to `expect`.
# ./spec_game_of_life.rb:22:in `block (3 levels) in <top (required)>'
Finished in 0.0019 seconds (files took 0.11777 seconds to load)
3 examples, 1 failure
Failed examples:
rspec ./spec_game_of_life.rb:19 # Game of Life world should create proper cell_grid upon initialization
I understand that rspec's "should" has been replaced with "expect" per: http://rspec.info/blog/2012/06/rspecs-new-expectation-syntax/
Initializing a class with a (rows, cols) cell grid - Ruby script, "game_of_life.rb":
1 # basic file
2
3 class World
4 attr_accessor :rows, :cols, :cell_grid
5 def initialize(rows=3, cols=3)
6 #rows = rows
7 #cols = cols
8 #cell_grid = Array.new(rows) do |row|
9 Array.new(cols) do |col|
10 end
11 end
12 end
13 end
Ruby spec file, "spec_game_of_life.rb":
1 # spec file
2
3 require 'rspec'
4 require_relative 'game_of_life.rb'
5
6 describe 'Game of Life' do
7
8 context 'world' do
9 subject { World.new }
10
11 it 'should create a new world object' do
12 expect(subject.is_a?(World)).to be true
13 end
14 it 'should respond to proper methods' do
15 expect(subject.respond_to?(:rows))
16 expect(subject.respond_to?(:cols))
17 expect(subject.respond_to?(:cell_grid))
18 end
19 it 'should create proper cell_grid upon initialization' do
20 expect(subject.cell_grid.is_a?(Array)).to be true
21
22 expect(subject.cell_grid.each) do |row|
23 row.is_a?(Array).to be true
24 end
25 end
26
27 end
28
29 end
FWIW: Ubuntu 14.04, Ruby 2.3.0, rspec 3.5.3 & I'm following along with this "Game of Life" tutorial which uses "should":
https://www.youtube.com/watch?v=Tzs3_pl410M&list=PLMC91Ry9EhRKUn0MIdgXrZiptF7nVyYoQ&index=4
EDIT per answer from Bartek Gladys:
expect{ subject.cell_grid.all? { |k| k.is_a?(Array) } }.to eq true
..F
Failures:
1) Game of Life world should create proper cell_grid upon initialization
Failure/Error: expect{ subject.cell_grid.all? { |k| k.is_a?(Array) } }.to eq true
You must pass an argument rather than a block to use the provided matcher (eq true), or the matcher must implement `supports_block_expectations?`.
# ./spec_game_of_life.rb:22:in `block (3 levels) in <top (required)>'
NOTE:
expect{ subject.cell_grid.all? { |k| k.is_a?(Array) } }.to be true
Failure/Error: expect{ subject.cell_grid.all? { |k| k.is_a?(Array) } }.to be true
You must pass an argument rather than a block to use the provided matcher (equal true), or the matcher must implement `supports_block_expectations?`.
So... how to implement supports_block_expectations??
Researching per: http://rspec.info/blog/2014/05/notable-changes-in-rspec-3/
https://www.relishapp.com/rspec/rspec-expectations/docs/custom-matchers/define-a-matcher-supporting-block-expectations
http://www.relishapp.com/rspec/rspec-expectations/v/3-5/docs
http://rspec.info/
The expect version of your spec is
subject.cell_grid.each do |row|
expect(row.is_a?(Array)).to be_truthy
end
(The be_true matcher no longer exists)
Slightly more naturally you would write
subject.cell_grid.each do |row|
expect(row).to be_an(Array)
end
You could use all? to have only one call to expect but this will lead to less helpful failure messages.
You wouldn't usually pass a block to expect just to make an assertion about a value - typically this is used to check for side effects (such as raising an exception).
try this:
expect{ subject.cell_grid.all? { |k| k.is_a?(Array) } }.to eq true
Not exactly the answer, but as I struggled to find how to define the supports_block_expectations thing, it's below:
Say, we are to give block support to matcher be (can be anything like eq, or equal or even your own matcher thing)
CAUTION: Using existing matchers kinda redefine it. Better name it with something new to avoid trouble.
Definition:
Short hand definition:
RSpec::Matchers.define :be do
match do |actual|
actual.is_a? Proc
end
supports_block_expectations
end
Full definition
RSpec::Matchers.define :equal do
match do |actual|
actual.is_a? Proc
end
def supports_block_expectations?
true # or some logic
end
end
Usage:
RSpec.describe "a custom block matcher" do
it { expect { subject.name }.to be('abc') }
end

RSpec custom matchers keep state between specs

I've come across some counter intuitive behavior in RSpec (2.8.0) custom matchers functionality and I wonder whether it is a bug or feature or me being confused. Let's look at the code:
# matcher code
RSpec::Matchers.define :exist do
chain :with_start_time do |time|
#start_time = time
end
chain :with_end_time do |time|
#end_time = time
end
match do |subject|
result = true
result &&= subject.start_time == #start_time if #start_time
result &&= subject.end_time == #end_time if #end_time
result
end
failure_message_for_should do |subject|
"Failure!\n".tap do |msg|
if #start_time != subject.start_time
msg << "Expected start_time to be ##start_time but was #{subject.start_time}\n"
end
if #end_time != subject.end_time
msg << "Expected end_time to be ##end_time but was #{subject.end_time}\n"
end
end
end
end
#spec code
require 'ostruct'
describe 'RSpec custom matcher keeping state between tests' do
let(:time) { Time.now }
it 'passes the first spec' do
o = OpenStruct.new(start_time: time)
o.should exist.with_start_time(time)
end
it 'fails the second because matcher kept #start_time from the first test' do
o = OpenStruct.new(end_time: time)
o.should exist.with_end_time(time)
end
end
This fails (demonstrating the issue):
avetia01:~/projects/custom_matcher_bug% rspec test_spec.rb
.F
Failures:
1) RSpec custom matcher keeping state between tests fails the second because matcher kept #start_time from the first test
Failure/Error: o.should exist.with_end_time(time)
Failure!
Expected start_time to be 2012-02-27 12:20:25 +0000 but was
# ./test_spec.rb:41:in `block (2 levels) in <top (required)>'
Finished in 0.00116 seconds
2 examples, 1 failure
So the unexpected bit is that the same matcher instance seems to be used across multiple specs. Which, in this particular case leads, to #start_time being initialized with the value from the first spec, causing incorrect failure of the second spec.
This has reported and fixed, but not yet released:
https://github.com/rspec/rspec-expectations/issues/104

Resources