Re-run failed unit tests in ruby - ruby

Question

I have some unit tests written using Test::Unit::TestCase, with XML generated by ci_reporter. However, due to circumstances beyond my control, they may occasionally fluctuate, and randomly fail. I'd like to detect when a test fails, and attempt to re-run it.
I tried doing this by monkey-patching 'teardown' to check 'passed?', and re-running the tests on a failure. However, the XML output will still show the first failed case, and not the second (now passing) run.

Answer 1

This sounds a bit like the opposite of Multiple tests with minitest
Perhaps this is a possibility: Copy your test case in an own file. As an example, try the following test:
#store it as file 'testcase.rb'
gem 'test-unit'
require 'test/unit'
class X < Test::Unit::TestCase
def test_1
num = rand(10)
assert_true( num < 2, "Value is #{num}")
end
end
Then define your test call in a rake task:
require 'rake'
TEST_REPETION = 10
task :test do
success = false
TEST_REPETION.times{
stdout = `ruby testcase.rb`
if stdout =~ /Failure/
puts "Failure occured - redo the test"
else
puts 'Tests ok'
success = true
exit
end
}
puts "Stopped after #{TEST_REPETION} tries" unless success
end
Now the test is called, until the test succeed or TEST_REPETION are done.
Remarks:
Rake isn't needed, you may do the call without rake (My template was a rake task)
This works only, if your xml changes for each run (it must be regenerated before the test. else you test always the same).
You may store the test result (stdout) in a file and use it later to analyze, which tests failed and try to retest them.