Ruby Test:Unit, how to know fail/pass status for each test case in a test suite? - ruby

This question sounds stupid, but I never found an answer online to do this.
Assume you have a test suite like this page:
http://en.wikibooks.org/wiki/Ruby_Programming/Unit_testing
or code:
require "simpleNumber"
require "test/unit"
class TestSimpleNumber < Test::Unit::TestCase
def test_simple
assert_equal(4, SimpleNumber.new(2).add(2) )
assert_equal(4, SimpleNumber.new(2).multiply(2) )
end
def test_typecheck
assert_raise( RuntimeError ) { SimpleNumber.new('a') }
end
def test_failure
assert_equal(3, SimpleNumber.new(2).add(2), "Adding doesn't work" )
end
end
Running the code:
>> ruby tc_simpleNumber2.rb
Loaded suite tc_simpleNumber2
Started
F..
Finished in 0.038617 seconds.
1) Failure:
test_failure(TestSimpleNumber) [tc_simpleNumber2.rb:16]:
Adding doesn't work.
<3> expected but was
<4>.
3 tests, 4 assertions, 1 failures, 0 errors
Now, how to use a variable (what kind?) to save the testing results?
e.g., an array like this:
[{:name => 'test_simple', :status => :pass},
{:name => 'test_typecheck', :status => :pass},
{:name => 'test_failure', :status => :fail},]
I am new to testing, but desperate to know the answer...

you need to execute your test script file, that's it, the result will display pass or fails.
Suppose you save file test_unit_to_rspec.rb, after that execute below command
ruby test_unit_to_rspec.rb

Solved the problem with setting a high verbose level, in a test runner call.
http://ruby-doc.org/stdlib-1.8.7/libdoc/test/unit/rdoc/Test/Unit/UI/Console/TestRunner.html
require 'test/unit'
require 'test/unit/ui/console/testrunner'
class MySuperSuite < Test::Unit::TestSuite
def self.suite
suites = self.new("My Super Test Suite")
suites << TestSimpleNumber1
suites << TestSimpleNumber2
return suites
end
end
#run the suite
# Pass an io object
#new(suite, output_level=NORMAL, io=STDOUT)
runner = Test::Unit::UI::Console::TestRunner.new(MySuperSuite, 3, io)
results will be saved in the io stream in a nice format fo each test case.

What about using '-v' (verbose):
ruby test_unit_to_rspec.rb -v
This should show you a lot more information

You can check out another of Nat's posts for a way to capture the results. The short answer to your question is that there is no variable for capturing the results. All you get is:
Loaded suite My Special Tests
Started
..
Finished in 1.000935 seconds.
2 tests, 2 assertions, 0 failures, 0 errors
Which is not very helpful if you want to report to someone else what happened. Nat's other post shows how to wrap the Test::Unit in rspec to get a better result and more flexibility.

class Test::Unit::TestCase
def setup
#id = self.class.to_s()
end
def teardown
#test_result = "pass"
if(#_result.failure_count > 0 || #_result.error_count > 0)
#test_result = "fail"
# making sure no errors/failures exist before the next test case runs.
i = 0
while(i < #_result.failures.length) do
#_result.failures.delete_at(i)
i = i + 1
end
while(i < #_result.errors.length) do
#_result.errors.delete_at(i)
i = i + 1
end
#test_result = "fail"
end # if block ended
puts"#{#id}: #{#test_result}"
end # teardown definition ended
end # class Test::Unit::TestCase ended
Example Output :
test1: Pass
test2: fail
so on....

Related

Capturing and logging cucumber test results

I'm adding logging to a test suite. It uses Ruby, Cucumber and Rspec to run. I've already set up an env.rb to log each test and can capture Scenarios and Steps and drop them into the logger but I am at a loss how to grab the actual results.
Each test run will put something like
1 scenario (1 passed)
27 steps (27 passed)
0m0.176s
into the console.
I know I can grab feature name, scenario name, tags and steps using the Before do |scenario| below but I cannot find anyway of accessing the results.
$logger.level = :info
$logger.add_appenders \
Logging.appenders.stdout,
Logging.appenders.file("logging/#{Time.now.strftime("%m-%d-%Y--%H:%M")}.log")
user = `id -un` #Mac specific, sorry windows users
user.delete!("\n")
$logger.debug "Test run by #{user} on #{Time.now.strftime("%m/%d/%Y")}"
Before do |scenario|
#feature_name = scenario.feature.name
#scenario_name = scenario.name
#scenario_tags = scenario.source_tag_names
$logger.info "FEATURE: #{#feature_name}"
$logger.info "SCENARIO: #{#scenario_name}"
$step_index = 0
$stop_count = scenario.test_steps.count
#scenario = scenario
end
AfterStep do |step|
if $step_index < $stop_count
$logger.info "#{#scenario.test_steps[$step_index].text}\n"
end
$step_index += 2
end
It needs to output to a .log which is why I've so far avoided the html report builders.
Found that this does exactly what I needed
AfterConfiguration do |config|
config.on_event :test_case_finished do |event|
p event.result
end
end

Using Rspec in one file like test/unit

When I have time I like to take on a challenge at codewars.
Until now I used test/unit to do my unit testing but I would like to use Rspec now without changing my way of working. These are small methods/files/tests so I like to keep everything together in one script.
I run almost all of my code using Sublime Text and get the result in a window at the bottom of the editor.
Here my working test/unit example
require 'test/unit'
def anagrams(word, words)
words.select { |w| w.chars.sort == word.chars.sort }
end
class MyTest < Test::Unit::TestCase
def test_fail
assert_equal(['aabb', 'bbaa'], anagrams('abba', ['aabb', 'abcd', 'bbaa', 'dada']) )
assert_equal(['carer', 'racer'], anagrams('racer', ['crazer', 'carer', 'racar', 'caers', 'racer']) )
assert_equal([], anagrams('laser', ['lazing', 'lazy', 'lacer']) )
end
end
This gives in Sublime the following output
Loaded suite C:/Users/.../codewars/anagram
Started
.
Finished in 0.001 seconds.
------
1 tests, 3 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
------
1000.00 tests/s, 3000.00 assertions/s
[Finished in 0.3s]
And here what I tried for Rspec
require 'rspec'
describe "Anagrams" do
def anagrams(word, words)
words.select { |w| w.chars.sort == word.chars.sort }
end
it "should only match the anagrams" do
anagrams('abba', ['aabb', 'abcd', 'bbaa', 'dada']) == ['aabb', 'bbaa']
end
end
In Sublime text, I get no output, just a empty black window with the time it took to execute the script, if I use the console and run rspec anagram.rb I get
.
Finished in 0.001 seconds (files took 0.10601 seconds to load)
1 example, 0 failures
How do I have my code and test in the same file and do the test just by running the script in my Sublime Text editor (and get the output) and how could I better rephrase this test ?
You just need to tell the RSpec::Core::Runner to run your spec.
Adding RSpec::Core::Runner.run([$__FILE__]) at the end of your fill should work.
Updated code:
require 'rspec'
describe "Anagrams" do
def anagrams(word, words)
words.select { |w| w.chars.sort == word.chars.sort }
end
it "should only match the anagrams" do
anagrams('abba', ['aabb', 'abcd', 'bbaa', 'dada']) == ['aabb', 'bbaa']
end
end
RSpec::Core::Runner.run([$__FILE__])
I don't know what magic you used for sublime text to auto run your unit tests.
What I can answer is how to phrase the test better.
require 'rspec'
def anagrams(word, words)
words.select { |w| w.chars.sort == word.chars.sort }
end
RSpec.describe "Anagrams" do
it "matches words that are anagrams" do
# 2 different ways to do this
expect(anagrams('abba', ['aabb', 'abcd', 'bbaa', 'dada'])).to match_array(['aabb', 'bbaa'])
expect(anagrams('abba', ['aabb', 'abcd', 'bbaa', 'dada'])).to contain_exactly('aabb', 'bbaa')
end
end
match_array & contain_exactly are identical, except that match_array needs 1 parameter: array and contain exactly needs no array, instead you list all memebers of array.
https://www.relishapp.com/rspec/rspec-expectations/docs/built-in-matchers/contain-exactly-matcher
You could also break this into 2 or more specs if you want to. I'd do that if logic was more complicated. Going to do it here anyway so you can see more examples of rspec messages. Using should in spec name is no longer recommended.
RSpec.describe "Anagrams" do
it "when no annagrams found returns empty array" do
expect(anagrams('abba', ['abcd', 'dada'])).to eq([])
end
it "recognizes itself as annagram" do
expect(anagrams('abba', ['abba'])).to eq(['abba'])
end
it "returns array containing words that are anagrams" do
expect(anagrams('abba', ['aabb', 'abcd', 'bbaa', 'dada'])).to contain_exactly('aabb', 'bbaa')
end
end

Is it possible to write a spec for infinite-loop issues using Rspec?, Ruby

Listen I've an interesting question here, the other day I ran into an "infinite-loop" problem using Rspec, Rspec couldn't even go through the spec related to other methods inside the loop and even the comp was almost crashing. Very funny.
I'd like to test my future loops (While-loop in this case) against infinite loop-code.
How I can test this while-loop and catch up this problem like this one and make the proper correction?
Thanks!
This is my code from other day:
i = 0
while i <= Video.all.count do
if ( #sampler = Video.find_next_sampler(#samplers[-1].end_time, #samplers[-1].end_point) )
#samplers << #sampler
else
flash[:error] = 'There is not any more match for this video-sampler'
end
i + 1 #Now Here is the bug!! IT should be: i += 1
end
require 'timeout'
it 'should not take too long' do
Timeout.timeout(20) do
... blah ...
end
end
Or even
# spec_helper.rb
require 'timeout'
RSpec.configure do |c|
c.around(:example, finite: true) do |example|
Timeout.timeout(20) do
example.run
end
end
end
# my_spec.rb
it "should work really fast", finite: true do
... blah ...
end
In this particular example is doesn't make sense to run the loop more often that the total number of all videos in the database.
Therefore I would try something like this:
let(:videos_count) { Video.count }
before do
allow(Video).to receive(:find_next_sampler).and_call_original
end
it 'is not an infinite loop' do
except(Video).to receive(:find_next_sampler).at_most(videos_count).times
# call your method
end

Capybara redirect stderr

While running automated test scripts I will oftentimes get the following warning messages and others:
QFont::setPixelSize: Pixel size <= 0 (0)
QNetworkReplyImplPrivate::error: Internal problem, this method must only be called once.
I've searched around and these outputs have nothing to do with my test scripts. They don't impact the results in any way. Thus, I don't wish to see them.
While looking for a way to solve this, I found code which is supposed to redirect stderr to a class which will filter out specific messages. However, when I try to use this code, none of my scripts work.
The class that suppresses the warnings:
class WarningSuppressor
# array to hold warnings to suppress
SUPPRESS_THESE_WARNINGS = [
'QFont::setPixelSize: Pixel size <= 0 (0)',
'QNetworkReplyImplPrivate::error: Internal problem, this method must only be called once.'
]
class << self
def write(message)
if suppress_warning? message
0
end
end
def suppress_warning? message
SUPPRESS_THESE_WARNINGS.any? { |suppressable_warning| message.chomp.include? suppressable_warning }
end
end
end
The configuration code that is supposed to redirect stderr:
Capybara.configure do |config|
config.default_driver = :webkit
config.javascript_driver = :webkit
config.run_server = false # prevents Capybara from booting up a rack application automatically
config.app_host = 'http://local.xxxxxxxx.com'
# Sends output to a custom warning supressor
config.register_driver :webkit do |app|
Capybara::Driver::Webkit.new(app, stderr: WarningSuppressor)
end
# 10 second wait for ajax to finish
config.default_wait_time = 10
end
If I insert a binding.pry in the block statement, app is nil but Capybara::Driver::Webkit exists.
Does anyone have a better way/way to fix this; a method of hiding certain warnings from being displayed while running my automated scripts?

Mongo / Ruby driver output specific number of documents at a time?

Ruby Mongo Driver question:
How do I output 5_000 document batches from the collection at a time until I read the last document in the collection without dumping the entire database into memory first?
This is really bad method for me:
mongo = MongoClient.new('localhost', 27017)['sampledb']['samplecoll']
#whois.find.to_a....
Mongo::Collection#find returns a Mongo::Cursor that is Enumerable. For batch processing Enumerable#each_slice is your friend and well worth adding to your toolkit.
Hope that you like this.
find_each_slice_test.rb
require 'mongo'
require 'test/unit'
class FindEachSliceTest < Test::Unit::TestCase
def setup
#samplecoll = Mongo::MongoClient.new('localhost', 27017)['sampledb']['samplecoll']
#samplecoll.remove
end
def test_find_each_slice
12345.times{|i| #samplecoll.insert( { i: i } ) }
slice__max_size = 5000
#samplecoll.find.each_slice(slice__max_size) do |slice|
puts "slice.size: #{slice.size}"
assert(slice__max_size >= slice.size)
end
end
end
ruby find_each_slice_test.rb
Run options:
# Running tests:
slice.size: 5000
slice.size: 5000
slice.size: 2345
.
Finished tests in 6.979301s, 0.1433 tests/s, 0.4298 assertions/s.
1 tests, 3 assertions, 0 failures, 0 errors, 0 skips

Resources