I am using Minitest as the runner for my functional tests, using Selenium as the driver to run the browser. Each test is modeled as MiniTest::Unit::TestCase.
Minitest reports summary of execution when it completes executing all tests. The Exceptions that were encountered are also printed towards the end of the execution. I find it difficult to debug when something unexpected fails as the context of execution is lost. The exceptions I am running into are not deterministic.
Is there a way to make Minitest runner to stop execution of tests on exception or assertion failure?
I am using minitest (2.11.2) and ruby 1.9.2p290 (2011-07-09) [i386-mingw32]
I think you mean to have a "fast fail" option available. I found the fail_fast for minitest (Test::Unit): Immediate backtrace & exit article, but that's out of date (still covers what can be done). I think you'll need to monkeypatch your testing library to enable this option. I found a Gist showing how to add a simple fail-fast option to minitest/turn/minitest-rails so that might get you on the right track. I understand your problem is to do with the first article I've referenced:
When I run the Test::Unit suite in my Ruby on Rails 3 project through
rake test and a test is failing, the default behavior is to just
print “F” or “E”, keep running until all the tests are finished (while
I twiddle my thumbs), and only then print out a stack trace.
try rails test -f will do it. It means abort test run of first failure or error.
As I answered here, I found a gem for: minitest-fail-fast. It works with Rails 4.2 and Minitest 5.6.1
It does not directly answer the question, but might be very useful.
You could use pry-rescue gem so that your tests will launch a pry session whenever something goes wrong. All you have to do is to add the gem into your Gemfile:
group :development, :test do
gem 'pry-rescue'
end
Then launch your tests with the following flag:
PRY_RESCUE_RAILS=1 rails test rails test test/integration/agendas_test.rb
Use -f.
In pure ruby, as you ask for, this means ruby <file> -f.
In rails, this means rails test -f.
Minitest will check the terminal's arguments for the -f flag, so how you call it appears to not be relevant to the -f flag.
Related
I used jruby-1.7.0 and testing frame work Test::Unit::TestCase. Now I have updated some gems which caused me to use MiniTest::Test for testing, so I also upgraded to jruby-1.7.13. I have a rake task too which builds my gems with java *.class files instead of ruby *.rb files. Now many things dont work any more like 'rake test', or require "my_gem_xyz" from irb, getting errors like load error and ArrayIndexOutOfBoudsException. I have rvm installed to switch between rubies.
How can I use *.class files in my self made gems like it worked before under jruby-1.7.0? (If I include the *.rb files in the gem it seems to work ok)
Some time ago I tried with jruby-1.7.4 but that also failed with similar reasons (at that time I still used Test::Unit::TestCase, so the test frame work is not the problem)
Any ideas?
Frank
I found a surprising solution to the problem. I had compiled the *.rb classes with my jruby version 1.7.0's jrubyc. This caused the tests to fail with the load error. I had to compile all ruby classes with the jrubyc version 1.7.13 where I also build the gems.
update:
'rake test' still does not work comming up with errors like Mocha::ExpectationError: unexpected invocation: ...
what works is to use test options specifiying test file and test name (also with reg.exp.) like
rake test TEST=test/test_bla_bla.rb TESTOPTS="--name=/test_should_read/ -v"
I have the feeling the randomization with the seed parameter causes the problem. How can I influence the seed parameter?
update:
It looks like if there are too many tests in the test suite, MiniTest cannot handle this. In some cases I could run about 10 tests in some up to 35, using the TESTOPTS switch. I always fail if I run all tests in my test library with 'rake test'. This looks like a bug to me. Is there anybody who can help?
The documentation for the minitest-ci gem (seemingly the only option for producing test results for a CI tool such as Jenkins) has the extremely annoying habit of not preserving the results of rake minitest:models when invoked as rake minitest - the test results from running minitest:models are deleted prior to running the rest of the tests. minitest-ci's barely-extant documentation claims adding this to test_helper.rb will disable the troublesome auto-clean behavior, but it doesn't:
# Why do SO and GitHub have to use completely different ways of indicating inline code?
# test/helper.rb
MiniTest::Ci.auto_clean = false
Has anyone out there managed to get minitest-ci to preserve all the test result files? I'm reaching wits' end here.
I think ci_reporter gem supports miniTest. This could be another option.
This may seem obvious but I don't find a way to run Rcov or coverMe outside a Rails project.
I would like to use it with Rspec 2.5
I am using Ruby 1.9.2 so I guess this may be the problem.
I also would like not to use rake tasks but a command on the command line. I have tried several things and the best result I got is Rcov report for :
/var/lib/gems/1.9.1/gems/rcov-0.9.9/lib/rcov/code_coverage_analyzer.rb
and
/var/lib/gems/1.9.1/gems/rcov-0.9.9/lib/rcov/code_coverage_analyzer.rb
No idea why
Check out http://ruby-toolbox.com/categories/code_metrics.html for some alternatives to RCov if using Ruby 1.9 is the problem.
SimpleCov, it runs awesome outside the Rails box.
http://rubydoc.info/gems/simplecov/0.5.4/frames
The latest available version of rcov (0.9.8) still doesn't have good support for 1.9.2. You can try to run it on your project by doing:
rcov spec/*.rb
But you'll most likely get something like:
** WARNING: Ruby 1.9 Support is experimental at best. Don't expect correct results! **
And then some errors after that.
I'm trying to debug a simple BDD test using cucumber. In order to do that, I inserted a debugger statement where I would like to break the control flow. But it seems that cucumber ignores this statement.
I'm running the tests executing:
rake cucumber:wip
It's rather simple, so I think the code isn' t worth pasting here. Thanks in advance
You don't have the rails-debug gem installed and/or included.
Make sure rails-debug listed in your Gemfile
If it's still not being included, make sure you run cucumber like:
bundle exec cucumber ...
When running tests in Ruby's unit::test framework, is there a really easy way to specify, from the command-line, that only one test should be run (that is, specify the test class and test member variable)? If not, is there another framework that has this feature?
ruby /path/to/foo_test.rb --name test_should_do_something_really_spiffy
That will call the test defined by the method test_should_do_something_really_spiffy in that file.
EDIT: That's for the Test::Unit framework that most ruby tests are written with. I am assuming you meant the same.
If you have the full Test::Unit framework, you can do
ruby /path/to/foo_test.rb --help
to get the command line options. I don't think it works if you just have the minitest version though.