I'm using Ruby 2.2. I need to run a unit test and get information if it succeeded or failed. I'm browsing through docs of both test-unit and minitest (suggested gems for unit testing in Ruby 2.2) but I can't seem to find a method that would return or store somewhere information about the test result.
All I need is information whether the test failed/succeeded, and I need to access it from the level of Ruby. I imagine I would have to use a specific method to run the test - so far, I was only able to run a single test by running the test file, not by invoking any method.
Maybe it's just my poor knowledge of Ruby, anyway I would appreciate any help.
May be you can run the tests using Ruby's ability to run shell command and return results.
Here is an example for test-unit:
test_output = `ruby test.rb --runner console --verbose=progress`
failed_tests = test_output.chomp.split('').count('F')
passed_tests = test_output.chomp.split('').count('.')
puts "P: #{passed_tests}, F: #{failed_tests}"
We are using --verbose=progress option so that we get minimum output. It will look something like below:
.F...F
We count number of F to figure out how many tests failed.
For about test output, the sample program will print:
P: 4, F: 2
Another option is to use passed? method:
https://ruby-doc.org/stdlib-2.1.1/libdoc/minitest/unit/rdoc/MiniTest/Unit/TestCase.html#method-i-passed-3F
Not sure if it's still available in the latest versions of Ruby, so please check that before using.
Related
Currently I am using
parallel tests
rspec
allure 0.8.0
After I run the tests I get the following error:
RSpec::Core::MultipleExceptionError
I need the whole backtrace of the error. Is it some parameter that I need to pass to the command that I use to run and is there a permanent way so that it always prints the whole error
You have two options, to define a helper with that attribute or in command line
## spec/spec_helper.rb
config.full_backtrace = true # false to dismiss
or in the command line
$ rspec spec/folder/your_spec.rb --backtrace
If you get this error you need to find the specific test in the rspec log (scroll up). There you will see details about the issue.
In my case error was shown only on CI and I needed some time in order to figure out that everything I need is "hidden" somewhere in the super long output.
I have an RSpec suite, run via Bundler, that is testing a number of different command-line applications using Aruba. It works fine ... as long as the command being tested is not itself written in Ruby using Bundler. But I cannot figure out how to prevent the RSpec suite's bundler config from interfering with the execution of commands that themselves use Bundler - at least, not without extreme measures.
I have tried various permutations of unset_bundler_env_vars and with_clean_env, to no avail. Here's an example of a technique I thought would work:
describe 'my ruby app' do
before :each { unset_bundler_env_vars }
it 'should work' do
Bundler.with_clean_env { run_simple ruby_command_name }
end
end
I also tried unset_bundler_env_vars without with_clean_env, and vice-versa, in case they interfered with each other. No dice.
The only way I've gotten it to work is to massage Aruba's copy of the environment manually, like this:
before :all do
aruba.environment.tap do |env|
if env.include? 'BUNDLE_ORIG_PATH' then
env['PATH'] = env['BUNDLE_ORIG_PATH']
%w(BUNDLE_BIN_PATH BUNDLE_GEMFILE BUNDLE_ORIG_PATH GEM_HOME RBENV_DIR
RBENV_HOOK_PATH RUBYLIB RUBYOPT).each do |key|
env.delete key
end
end
end
end
There must be a better way. Neither the test suite nor the command being tested should know or care what language the other is written in. And my test code that uses Aruba and Bundler should not need to know the details of how bundle exec affects the process environment.
So what am I doing wrong? How should I be doing this?
It looks like unset_bundler_env_vars is deprecated and replaced by delete_by_environment_variable which requires a string param (source).
You might try before :each { delete_environment_variable('BUNDLE_GEMFILE') } in your spec. If that does not work, you may need to iterate through the PATH variable list to delete each one.
In the deprecation notice, there is a work-around, though I am not sure how brittle that would be moving forward.
unset_bundler_env_vars
aruba.environment.clear.update(ENV)
Hope this helps.
I'm trying to use RSpec from within existing ruby runtime and run specs every time when file changes. This is because of JRuby and JVM startup time. To eliminate this on every run I'd like to start ruby once, then only reload changed files and run specs. I was using guard (with diffrent extensions) and watchr but all seem to suffer from an issue described below.
I nailed the issue down to RSpec itself. The problem is, when running RSpec via RSpec::Core::Runner.run several times it works fine until spec file is reloaded using load. Then RSpecs starts running specs twice.
I've created sample project showing this issue live: https://github.com/mostr/rspec_double_run_issue
Below is sample output:
ruby run_spec_in_loop.rb
Running spec from within ruby runtime
.
Finished in 0.00047 seconds
1 example, 0 failures
loading spec file via 'load' as if it was changed and we wanted changes to be picked up
Running spec from within ruby runtime
..
Finished in 0.001 seconds
2 examples, 0 failures
Is there any way to tell RSpec to clear its context between subsequent runs when run from within existing ruby runtime? I've also raised this as an issue #826 for RSpec Core project.
Summarizing the answer here in order to remove this question from the "Unanswered" filter...
Per RJHunter's observation, the explanation has been documented on the GitHub RSpec Core project here:
https://github.com/rspec/rspec-core/issues/826#issuecomment-15089030
For posterity (in case the above link dies), here are the details:
The RSpec runner is already calling load internally, your second load is what's causing the double run issue.
I quickly knocked up a script based off your example which reruns a single spec file, changes the specs to something else, then reruns them, work's correctly without the second load
See: https://gist.github.com/JonRowe/5192007
The aforementioned Gist contains:
require 'rspec'
spec_file = 'spec/sample_spec.rb'
File.open(spec_file, 'w') { |file| file.write 'describe { specify { expect(true).to eq false } }' }
1.upto(5) do |i|
puts "Running spec from within ruby runtime"
::RSpec::Core::Runner.run([spec_file], STDERR, STDOUT)
#rewriting the spec file
File.open(spec_file, 'w') { |file| file.write "describe { specify { expect(#{i}).to eq false } }" }
end
I've been playing around with autotest trying to make it work all day.. but am having some problems...
I've been following https://github.com/rspec/rspec/wiki/autotest, I'm running with:
Ruby 1.9.3-p194
rspec 2.10.0
ZenTest 4.8.1
I also created a .rspec file.
So with this setup, I run autotest, and it works - my test runs, it passes, hooray!. When I stick a failure into my test e.g. false.should == true, then the test starts looping, over and over again.
what happens is that it's an integration test, and I'm writing to an sqlite db. If I run find . -mmin -1 then I'm able to see that my db folder has changed - so I figured this is the problem.
So I edit .autotest and add the following:
Autotest.add_hook :initialize do |autotest|
%w{db}.each { |exception| autotest.add_exception(exception) }
false
end
But now when I run autotest, it just says the following:
loading autotest/rspec2
and that's it, it won't do anything anymore. Previously the output was:
loading autotest/rspec2
/home/me/.rbenv/versions/1.9.3-p194/bin/ruby -rrubygems -S '/home/me/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.10.1/exe/rspec' ``--tty '/home/me/Workspace/myproject/spec/integration/db/lead_spec.rb'
and then it'd run my test and show the result...
Anyone know what could be going on? it's very frustrating, and I feel like I've come to a road block....
Thanks for your help!
Autotest checks if defined exceptions match any part of the filename. Your spec has db in it's path so it is ignored by autotest.
If you want to ignore db folder, then do the following:
Autotest.add_hook :initialize do |a|
a.add_exception %r{^\./db}
end
I'm using CruiseControl.rb (2.0.0pre1) for CI with a Rails 3 app and I'm trying to get it to work with rspec. Inside the cruise_control.rb I'm calling the spec rake task
Project.configure do |project|
project.rake_task = 'spec'
end
Very basic and easy configuration. But even when a test is failing, the build passes. It doesn't detect wether the tests passes or fail.
Furthermore I want to know if I can grab the exit status from the rake task, and call a ruby script depending if a build fails or passes.
I found out, that it was an error in Rails 3.2.1. The status code returned from the tests was always 0. (see https://github.com/rails/rails/issues/4923 and http://github.com/rails/rails/commit/abe4a8d070d069f24f7befd9a8da25c40f4c2a6d). Upgrading to Rails 3.2.2.rc1 fixed the problem.