I know. This is discouraged. For reasons I won't get into, I need to run my tests in the order they are written. According to the documentation, if my test class (we'll call it TestClass) extends Minitest::Unit::TestCase, then I should be able to call the public method i_suck_and_my_tests_are_order_dependent! (Gee - do you think the guy who created Minitest had an opinion on that one?). Additionally, there is also the option of calling a method called test_order and specifying :alpha to override the default behavior of :random. Neither of these are working for me.
Here's an example:
class TestClass < Minitest::Unit::TestCase
#override random test run ordering
i_suck_and_my_tests_are_order_dependent!
def setup
...setup code
end
def teardown
...teardown code
end
def test_1
test_1 code....
assert(stuff to assert here, etc...)
puts 'test_1'
end
def test_2
test_2_code
assert(stuff to assert here, etc...)
puts 'test_2'
end
end
When I run this, I get:
undefined method `i_suck_and_my_tests_are_order_dependent!' for TestClass:Class (NoMethodError)
If I replace the i_suck method call with a method at the top a la:
def test_order
:alpha
end
My test runs, but I can tell from the puts for each method that things are still running in random order each time I run the tests.
Does anyone know what I'm doing wrong?
Thanks.
If you just add test_order: alpha to your test class, the tests will run in order:
class TestHomePage
def self.test_order
:alpha
end
def test_a
puts "a"
end
def test_b
puts "b"
end
end
Note that, as of minitest 5.10.1, the i_suck_and_my_tests_are_order_dependent! method/directive is completely nonfunctional in test suites using MiniTest::Spec syntax. The Minitest.test_order method is apparently not being called at all.
EDIT: This has been a known issue since Minitest 5.3.4: see seattlerb/minitest#514 for the blow-by-blow wailing and preening.
You and I aren't the ones who "suck". What's needed is a BDD specification tool for Ruby without the bloat of RSpec and without the frat-boy attitude and contempt for wider community practices of MiniTest. Does anyone have any pointers?
i_suck_and_my_tests_are_order_dependent! may be a later addition to minitest & not available as a Ruby core method. In that case, you'd want to force use of your gem version:
require 'rubygems'
gem 'minitest'
I think that the method *test_order* should be a class method and not a instance method like so:
# tests are order dependent
def self.test_order
:alpha
end
The best way to interfere in this chain may be to override a class method runnable_methods:
def self.runnable_methods
['run_first'] | super | ['run_last']
end
#Minitest version:
def self.runnable_methods
methods = methods_matching(/^test_/)
case self.test_order
when :random, :parallel then
max = methods.size
methods.sort.sort_by { rand max }
when :alpha, :sorted then
methods.sort
else
raise "Unknown test_order: #{self.test_order.inspect}"
end
end
You can reorder test any suitable way around. If you define your special ordered tests with
test 'some special ordered test' do
end
, don't forget to remove them from the results of super call.
In my example I need to be sure only in one particular test to run last, so I keep random order on whole suite and place 'run_last' at the end of it.
Related
Lets say I have the following module:
module SillyDemo
class Monkey
def screech(sound)
sound
end
end
class Ape < Monkey
def process(sound)
sound
end
def screech(sound)
process(sound)
super
sound
end
end
end
And then the following minitest:
require_relative 'sillydemo'
require "minitest/spec"
require "minitest/autorun"
describe "Ape" do
before do
#ape = Ape.new
#screech = "YEEEEEEE"
end
it "screeches" do
#ape.screech(#screech)
must_send [#ape, :process, #screech]
must_send [#ape, :super, #screech]
end
end
This errors out with:
NoMethodError: undefined method `super' for #<SillyDemo::Ape:0x007feeb10943c0>
(eval):4:in `must_send'
I have also tried:
must_send [#ape, :"SillyDemo::Monkey.screech", #screech]
which errors out with:
NoMethodError: undefined method `SillyDemo::Ape.run' for #<SillyDemo::Ape:0x007fc5a1874e20>
(eval):4:in `must_send'
My question is, how can I use minitest to test a call to super?
In Ruby super is a keyword, not a method. Also, the must_send expectation isn't verifying that the method was called, it just verifies that the return value from the method is truthy.
http://www.ruby-doc.org/stdlib-2.0.0/libdoc/minitest/rdoc/MiniTest/Expectations.html#method-i-must_send
http://www.ruby-doc.org/stdlib-2.0.0/libdoc/minitest/rdoc/MiniTest/Assertions.html#method-i-assert_send
Mocks are usually used to verify that a method was called. However, Minitest::Mock doesn't allow for this type of check very easily by design. Here is how you can do this though.
it "screeches" do
# 1) Create mock
sound_mock = Minitest::Mock.new
sound_mock.expect :process, true, [String]
# 2) Place mock
#ape.instance_exec(sound_mock) do |sound_mock|
#mock = sound_mock
def process sound
#mock.process sound
end
end
# 3) Verify mock was called
#ape.screech(#screech)
sound_mock.verify
end
Pretty ugly, right? This is by design. Sort of like syntactic vinegar. The reason is that this use of mocks isn't very informative. It is checking the implementation of the code. You would like to be able to refactor the code without changing behavior and have the tests continue to pass. However, this test will very likely fail when the implementation is changed. To discourage folks from making this kind of mistake it was decided by the Minitest authors to keep this type of check difficult.
Other mocking libraries such as RR or Mocha make this type of check much easier.
I've extracted a single class from a Rails app into a gem. It's very, very simple, but of course I'd like to fully test it (I'm using rspec).
The class does some simple date-calculation. It's not dependent on Rails, but since it started out in a Rails app, and is still used there, it uses ActiveSupport's time zone-aware methods when it can. But, if ActiveSupport isn't available, it should use the std-lib Date methods.
Specifically, it only does this in one single place: Defaulting an optional argument to "today's date":
arg ||= if Date.respond_to?(:current)
Date.current # use ActiveSupport's time zone-aware mixin if possible
else
Date.today # stdlib fallback
end
Question is: How do I properly test this? If I require ActiveSupport in my spec_helper.rb, it'll obviously always use that. If I don't require it anywhere, it'll never use it. And if I require it for a single example group, rspec's random execution order makes the testing unpredictable, as I don't know when AS will be required.
I can require maybe it in a before(:all) in a nested group, as nested groups are (I believe) processed highest to deepest. But that seems terribly inelegant.
I could also split the specs into two files, and run them separately, but again, that seems unnecessary.
I could also disable rspec's random ordering, but that's sort of going against the grain. I'd rather have it as randomized as possible.
Any ideas?
Another solution is to mock the current and today methods, and use those for testing. Eg:
# you won't need these two lines, just there to make script work standalone
require 'rspec'
require 'rspec/mocks/standalone'
def test_method(arg = nil)
arg ||= if Date.respond_to?(:current)
Date.current # use ActiveSupport's time zone-aware mixin if possible
else
Date.today # stdlib fallback
end
arg
end
describe "test_method" do
let(:test_date) { Date.new(2001, 2, 3) }
it "returns arg unchanged if not nil" do
test_method(34).should == 34
end
context "without Date.current available" do
before(:all) do
Date.stub(:today) { test_date }
end
it "returns Date.today when arg isn't present" do
test_method.should == test_date
end
end
context "with Date.current available" do
before(:all) do
Date.stub(:current) { test_date }
end
it "returns Date.current when arg isn't present" do
test_method.should == test_date
end
end
end
Running with rspec test.rb results in the tests passing.
Also, the stubs are present only in each context, so it doesn't matter what order the specs are run in.
This is more than a little perverse, but it should work. Include ActiveSupport, and then:
context "without ActiveSupport's Date.current" do
before(:each) do
class Date
class << self
alias_method :current_backup, :current
undef_method :current
end
end
end
# your test
after(:each) do
class Date
class << self
alias_method :current, :current_backup
end
end
end
end
I can't really recommend this; I would prefer to split out this one spec and run it separately as you suggested.
I have two classes:
1.Sale is a subclass of ActiveRecord; its job is to persist sales data to the database.
class Sale < ActiveRecord::Base
def self.total_for_duration(start_date, end_date)
self.count(conditions: {date: start_date..end_date})
end
#...
end
2.SalesReport is a standard Ruby class; its job is to produce and graph information about Sales.
class SalesReport
def initialize(start_date, end_date)
#start_date = start_date
#end_date = end_date
end
def sales_in_duration
Sale.total_for_duration(#start_date, #end_date)
end
#...
end
Because I want to use TDD and I want my tests to run really fast, I have written a spec for SalesReport that doesn't doesn't load Rails:
require_relative "../../app/models/sales_report.rb"
class Sale; end
# NOTE I have had to re-define Sale because I don't want to
# require `sale.rb` because it would then require ActiveRecord.
describe SalesReport do
describe "sales_in_duration" do
it "calls Sale.total_for_duration" do
Sale.should_receive(:total_for_duration)
SalesReport.new.sales_in_duration
end
end
end
This test works when I run bundle exec rspec spec/models/report_spec.rb.
However this test fails when I run bundle exec rake spec with the error superclass mismatch for class Sale (TypeError). I know the error is happening because Tap is defined by sale.rb and inline within the spec.
So my question is there a way to Stub (or Mock or Double) a class if that class isn't defined? This would allow me to remove the inline class Sale; end, which feels like a hack.
If not, how do I set up my tests such that they run correctly whether I run bundle exec rspec or bundle exec rake spec?
If not, is my approach to writing fast tests wrong?!
Finally, I don't want to use Spork. Thanks!
RSpec's recently added stub_const is specifically designed for cases like these:
describe SalesReport do
before { stub_const("Sale", Class.new) }
describe "sales_in_duration" do
it "calls Sale.total_for_duration" do
Sale.should_receive(:total_for_duration)
SalesReport.new.sales_in_duration
end
end
end
You may also want to use rspec-fire to use a test double in place of Sale that automatically checks all the mocked/stubbed methods exist on the real Sale class when running your tests with the real Sale class loaded (e.g. when you run your test suite):
require 'rspec/fire'
describe SalesReport do
include RSpec::Fire
describe "sales_in_duration" do
it "calls Sale.total_for_duration" do
fire_replaced_class_double("Sale")
Sale.should_receive(:total_for_duration)
SalesReport.new.sales_in_duration
end
end
end
If you rename total_for_duration on the real Sale class, rspec-fire will give you an error when you mock the method since it doesn't exist on the real class.
A simple way would be to check if "Sale" has already been defined
unless defined?(Sale)
class Sale; end
end
Sale need not be a class either in your test so:
unless defined?(Sale)
Sale = double('Sale')
end
I've got an interesting conundrum. I'm in the midst of developing a library to parse PSDs in Ruby. Also, a buddy is simultaneously working on a library to parse PSDs in JavaScript. We would like to share the same unit tests via a git submodule.
We've decided to use a simple JSON DSL to define each test. A single test might look like:
{
"_name": "Layer should render out",
"_file": "test/fixtures/layer_out.psd",
"_exports_to": "test/controls/layer_out_control.png"
}
So, now it's up to us to build the appropriate test harnesses to translate the JSON into the appropriate native unit tests. I've been using MiniTest to get myself up to speed, but I'm running into a few walls.
Here's what I've got so far. The test harness is named TargetPractice for the time being:
# run_target_practice.rb
require 'target_practice'
TargetPractice.new(:test) do |test|
test.pattern = "test/**/*.json"
end
and
# psd_test.rb
class PSDTest < MiniTest::Unit::TestCase
attr_accessor :data
def tests_against_data
# do some assertions
end
end
and
# target_practice.rb
class TargetPractice
attr_accessor :libs, :pattern
def initialize(sym)
#libs = []
#pattern = ""
yield self
run_tests
end
def run_tests
FileList[#pattern].to_a.each do |file|
test_data = JSON.parse(File.open(file).read)
test = PSDTest.new(test_data["_name"]) do |t|
t.data = test_data
end
end
end
end
Unfortunately, I'm having trouble getting a yield in the initialize to stick in my PSDTest class. Also, it appears that a test will run immediately on initialization.
I would like to dynamically create a few MiniTest::Unit::TestCase objects, set their appropriate data properties and then run the tests. Any pointers are appreciated!
I think you are overcomplicating things a bit here. What you need is a parameterized test, which is pretty trivial to implement using mintest/spec:
describe "PSD converter" do
def self.tests(pattern = 'test/**/*.json')
FileList[pattern].map{|file| JSON.parse(File.read(file))}
end
tests.each do |test|
it "satisfies test: " + test["_name"] do
# some assertions using test["_file"] and test["_exports_to"]
end
end
end
Is there an alternative to RSpec's before(:suite) and after(:suite) in MiniTest?
I suspect that a custom test runner is in order, however I cannot imagine it is not a common requirement, so somebody has probably implemented in. :-)
There are setup() and teardown() methods available. The documentation also lists before() and after() as being available.
Edit: Are you looking to run something before each test or before or after the whole suite is finished?
As noted above in Caley's answer and comments, MiniTest::Unit contains the function after_tests. There is no before_tests or equivalent, but any code in your minitest_helper.rb file should be run before the test suite, so that will do the office of such a function.
Caveat: Still relatively new at Ruby, and very new at Minitest, so if I'm wrong, please correct me! :-)
To get this to work with the current version of Minitest (5.0.6) you need to require 'minitest' and use Minitest.after_run { ... }.
warn "MiniTest::Unit.after_tests is now Minitest.after_run. ..."
https://github.com/seattlerb/minitest/blob/master/lib/minitest.rb
https://github.com/seattlerb/minitest/blob/master/lib/minitest/unit.rb
To run code before each test, use before. You're operating here in the context of an instance, possibly of a class generated implicitly by describe, so instance variables set in before are accessible in each test (e.g. inside an it block).
To run code before all tests, simply wrap the tests in a class, a subclass of MiniTest::Spec or whatever; now, before the tests themselves, you can create a class or module, set class variables, call a class method, etc., and all of that will be available in all tests.
Example:
require "minitest/autorun"
class MySpec < MiniTest::Spec
class MyClass
end
def self.prepare
puts "once"
##prepared = "prepared"
##count = 0
end
prepare
before do
puts "before each test"
#local_count = (##count += 1)
end
describe "whatever" do
it "first" do
p MyClass
p ##prepared
p #local_count
end
it "second" do
p MyClass
p ##prepared
p #local_count
end
end
end
Here's the output, along with my comments in braces explaining what each line of the output proves:
once [this code, a class method, runs once before all tests]
Run options: --seed 29618 [now the tests are about to run]
# Running tests:
before each test [the before block runs before each test]
MySpec::MyClass [the class we created earlier is visible in each test]
"prepared" [the class variable we set earlier is visible in each test]
1 [the instance variable from the before block is visible in each test]
before each test [the before block runs before each test]
MySpec::MyClass [the class we created earlier is visible in each test]
"prepared" [the class variable we set earlier is visible in each test]
2 [the instance variable from the before block is visible each test]
(Note that I do not mean this output to imply any guarantee about the order in which tests will run.)
Another approach is to use the existing before but wrap code to run only once in a class variable flag. Example:
class MySpec < MiniTest::Spec
##flag = nil
before do
unless ##flag
# do stuff here that is to be done only once
##flag = true
end
# do stuff here that is to be done every time
end
# ... tests go here
end
One simple way to do this is to write a guarded class method, and then call that in a begin.
A Minitest::Spec example:
describe "my stuff" do
def self.run_setup_code
if #before_flag.nil?
puts "Running the setup code"
#before_flag = true
end
end
before do
self.class.run_setup_code
end
it "will only run the setup code once" do
assert_equal 1, 1
end
it "really only ran it once" do
assert_equal 1,1
end
end
...to get
Run options: --seed 11380
# Running:
Running the setup code
..
Finished in 0.001334s, 1499.2504 runs/s, 1499.2504 assertions/s.
2 runs, 2 assertions, 0 failures, 0 errors, 0 skips
You can just place the code outside of the class.
This is what I do to have a banner.
require 'selenium-webdriver'
require 'minitest/test'
require 'minitest/autorun'
class InstanceTest < Minitest::Test
def setup
url = ARGV.first
#url = self.validate_instance(url)
#driver = Selenium::WebDriver.for :firefox
end
Nice thing about minitest is its flexibility. I've been using a custom MiniTest Runner with a +before_suite+ callback. Something like in this example - Ruby Minitest: Suite- or Class- level setup?
And then tell minitest to use the custom runner
MiniTest::Unit.runner = MiniTestSuite::Unit.new
You can also add an after test callback by updating your test_helper.rb (or spec_helper.rb) like this
# test_helper.rb
class MyTest < Minitest::Unit
after_tests do
# ... after test code
end
end