edit: this problem only happens sometimes
This only appears to happen when I run the test from within TextMate (even when I specify the ruby to run it from by hand with a shebang). If I run it from the terminal then everything is peachy…
Here's some code:
require 'test/unit'
require 'shoulda'
class TestingTest < Test::Unit::TestCase
context "My thing" do
should "always have this test fail, and give me this message" do
assert false
end
end
end
I'm expecting it to tell me something like:
1) Failure:
test: My thing should always have this test fail, and give me this message (TestingTest)
# etc
An assert message, if one was given
But I'm getting:
1) Failure:
test:8
Failed assertion, no message given.
So what am I missing? The example code above is as simple as I think I can make it and I can't see the problem!
try inheriting from ActiveSupport::TestCase instead of the Test::Unit::TestCase
Related
I'm trying to write a custom tool that runs ruby unit tests with my customizations.
What I need it to do is to load a certain TestCase from given file(through require or whatever), and then run it after doing some calculations and initializations.
Problem is, the moment I require "test/unit" and a test case, it runs immediately.
What can I do with this?
Thanks.
Since you're running 1.9 and test/unit in 1.9 is merely a wrapper for MiniTest, the following approach should work:
implement your own custom Runner
set MiniTest's runner to your custom runner
Something like (shameless plug from EndOfLine Custom Test Runner, adjusted to Ruby 1.9):
fastfailrunner.rb:
require 'test/unit'
class FastFailRunner19 < MiniTest::Unit
def _run args = []
puts "fast fail runner"
end
end
~
example_test.rb:
require 'test/unit'
class ExampleTest < Test::Unit::TestCase
def test_assert_equal
assert_equal 1, 1
end
def test_lies
assert false
end
def test_exceptions
raise Exception, 'Beware the Jubjub bird, and shun the frumious Bandersnatch!'
end
def test_truth
assert true
end
end
run.rb:
require_relative 'fast_fail_runner'
require_relative 'example_test'
MiniTest::Unit.runner= FastFailRunner19.new
If you run this with
ruby run.rb
the custom FastFailRunner19 will be used, which does nothing.
What about reading file content as a regular text file and doing eval on its content after you initialize/calculate things you say? It may not be sufficient for your needs and may require manual setup and execution of testing framework.
Like that (I put heredoc instead of reading file). Basically content is just a string containing your test case code.
content = <<TEST_CASE
class YourTestCase
def hello
puts 'Hello from eval'
end
end
YourTestCase.new.hello
TEST_CASE
eval content
Note: Altough I'd rather not use eval if there is another way. One should be extra careful when evaling code from string manually in any language.
You could collect the test cases you want to deferred its executions and store them in an array. Afterwards you would create a block execution code. For instance:
test_files = ['test/unit/first_test.rb'] #=> Testcases you want to run
test_block = Proc.new {spec_files.each {|f|load f} } #=> block storing the actual execution of those tests.
Once you're ready to call those testcases you just do test_block.call.
To generalize a bit, when thinking about deferring or delaying code executions, closures are a very elegant and flexible alternative.
I have a Ruby program that fails at runtime, but works when I test it with RSpec. I know the cause of the bug and how to fix it (see below), but I can't figure out how to build a failing RSpec test which proves the existence of the bug.
Imagine the following Ruby:
foobar.rb
class Foobar
attr_reader :fruit
def initialize
#fruit = Set.new ["Apple", "Banana", "Kiwi"]
end
end
The above code uses a Set, but it fails to "require 'set'". This causes it to fail at runtime:
$ irb
> require './foobar.rb'
> f = Foobar.new
NameError: uninitialized constant Foobar::Set
Before fixing the oversight, I wanted to build a simple RSpec test that proves the bug. My test looks like this:
foobar_spec.rb
require 'rspec'
require './foobar.rb'
describe Foobar do
it "can be initialized" do
expect { Foobar.new }.to_not raise_error
end
end
Running the test, I was surprised to see that it passes:
$ rspec foobar_spec.rb
.
Finished in 0.00198 seconds
1 example, 0 failures
After a little digging, I learned that RSpec loads Set for itself. This has the consequence of making Set available to the code it tests, and in my case concealing a bug.
I had the idea of "unloading/unrequiring" Set in my test. The closest I came was this code:
Object.send(:remove_const, :Set)
That indeed causes the test to fail, but unfortunately it also prevents Set from being loaded again by a future 'require', meaning it continued to fail even after I added require 'set' inside foobar.rb.
Is there a better way to unload gems at runtime? If not, what can I do to make this test fail as it should?
require 'rspec'
describe 'foobar.rb' do
it "can instantiate Foobar" do
`ruby -e 'Foobar.new' -r./foobar.rb`
$?.exitstatus.should == 0
end
end
works for the one case you mentioned. That said, I wouldn't recommend this approach. To cover all the cases where a class is referenced, you'd need to run all your specs this way, since the class reference could appear anywhere in your code.
I'm clearly doing something wrong. I'm trying to write and test plain ruby in a single file. I want guard to watch the file and the test file and run minitest any time either file changes.
So, two files: game.rb and game_test.rb
game.rb
class Game
end
game_test.rb
require 'rubygems'
require 'minitest/autorun'
require './game'
class GameTest < MiniTest::Unit::TestCase
def test_truth
assert true
end
end
I also have a Guardfile that looks like this:
notification :terminal_notifier
guard 'minitest', test_folders: '.' do
watch('game.rb')
watch('game_test.rb')
end
Now, I'm probably forgetting something, but I can't for the life of me figure out what it is.
If I start guard and press Enter, "Run All" happens and the tests run.. at least most of the time. However, I have to press Enter for it to happen.
Also, if I make a change to the files nothing happens. I've tried putting gem 'rb-fsevent' in a Gemfile and running with "bundle exec guard" but that doesn't seem to help either.
Any help would be much appreciated. I'm going nuts.
Thanks,
Jeremy
Your first "watch" definition will simply pass "game.rb", which is not a test file so it won't be run.
The second "watch" is correct so when you save "game_test.rb", the tests should run.
This should be a more correct Guardfile:
notification :terminal_notifier
guard 'minitest', test_folders: '.' do
watch('game.rb') { 'game_test.rb' }
watch('game_test.rb')
end
I have some unit tests written using Test::Unit::TestCase, with XML generated by ci_reporter. However, due to circumstances beyond my control, they may occasionally fluctuate, and randomly fail. I'd like to detect when a test fails, and attempt to re-run it.
I tried doing this by monkey-patching 'teardown' to check 'passed?', and re-running the tests on a failure. However, the XML output will still show the first failed case, and not the second (now passing) run.
This sounds a bit like the opposite of Multiple tests with minitest
Perhaps this is a possibility: Copy your test case in an own file. As an example, try the following test:
#store it as file 'testcase.rb'
gem 'test-unit'
require 'test/unit'
class X < Test::Unit::TestCase
def test_1
num = rand(10)
assert_true( num < 2, "Value is #{num}")
end
end
Then define your test call in a rake task:
require 'rake'
TEST_REPETION = 10
task :test do
success = false
TEST_REPETION.times{
stdout = `ruby testcase.rb`
if stdout =~ /Failure/
puts "Failure occured - redo the test"
else
puts 'Tests ok'
success = true
exit
end
}
puts "Stopped after #{TEST_REPETION} tries" unless success
end
Now the test is called, until the test succeed or TEST_REPETION are done.
Remarks:
Rake isn't needed, you may do the call without rake (My template was a rake task)
This works only, if your xml changes for each run (it must be regenerated before the test. else you test always the same).
You may store the test result (stdout) in a file and use it later to analyze, which tests failed and try to retest them.
I'd like to test class and gem loading. Have a look at the following stupid test case:
require 'rubygems'
require 'shoulda'
class SimpleTest < Test::Unit::TestCase
context 'This test' do
should 'first load something' do
require 'bundler'
assert Object.const_defined? :Bundler
end
should 'second have it reset again' do
assert !Object.const_defined?(:Bundler)
end
teardown do
# This works, but is tedious and unclean
#Object.send :remove_const, :Bundler rescue nil
# Instead I want something like this ;)
magic_reset
end
end
end
How about creating a subclass of Test::Unit::TestCase which runs the test method in a forked process?
class ForkingTestCase < Test::Unit::TestCase
def run(...)
fork do
super.run(...)
# somehow communicate the result back to the parent process
# this is the hard part
end
end
end
If this can be implemented, it should then just be a matter of changing the base class of your test case.
AFAIK, you cannot unload a file that you have loaded. You need to start a separate Ruby process for every test. (Or a separate Ruby instance if you are running on a Ruby implementation which supports multiple instances in the same process.)
Try using Kernel#load with wrap set to true:
load(filename, wrap=false) → true
Loads and executes the Ruby program in the file filename. If the
filename does not resolve to an absolute path, the file is searched
for in the library directories listed in $:. If the optional wrap
parameter is true, the loaded script will be executed under an
anonymous module, protecting the calling program’s global namespace.
In no circumstance will any local variables in the loaded file be
propagated to the loading environment.
each time you want to do a test of bundler, load it into a new anonymous module, do your tests on the bundler class within that module, and then go onto your next test.
If your code refers to the Bundler constant, then you'd have to set and unset that constant, though.
I haven't tried it myself, but can't see why it wouldn't work.
Try keeping track of what constants were defined before your tests started, and what constants are defined after your test finished, and remove the constants that were defined during the test.
I guess this isn't so much telling you how to call magic_reset as how to implement magic_reset.