Currently I am using
parallel tests
rspec
allure 0.8.0
After I run the tests I get the following error:
RSpec::Core::MultipleExceptionError
I need the whole backtrace of the error. Is it some parameter that I need to pass to the command that I use to run and is there a permanent way so that it always prints the whole error
You have two options, to define a helper with that attribute or in command line
## spec/spec_helper.rb
config.full_backtrace = true # false to dismiss
or in the command line
$ rspec spec/folder/your_spec.rb --backtrace
If you get this error you need to find the specific test in the rspec log (scroll up). There you will see details about the issue.
In my case error was shown only on CI and I needed some time in order to figure out that everything I need is "hidden" somewhere in the super long output.
Related
Normally Ruby test-unit will display a summary of tests run after they are finished, something like this:
Finished in 0.117158443 seconds.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10 tests, 10 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
298.74 tests/s, 0.00 assertions/s
This was working, but now something has changed and when the unit tests are run it shows the dots but then stops. I tried re-organizing some test file into different directories and made absolutely sure to change the filepaths in the testrunner. Also, the dots do not match the number of tests/assertions.
Loaded suite test
Started
.................$prompt> // <<-- does not put newline here.
I notice that if I run the testrunner from another directory, the summary will show, but it will cause errors with the test dependencies. I should be able to run the testrunner from the same directory. This is an example of the testrunner I am using: https://test-unit.github.io/test-unit/en/file.how-to.html. What are the reasons that this would not display at the end?
Seems like it could be an issue with not having the test-unit.yml file in the same directory as which you run the script.
See here in the code or in the same section in that document you posted.
See how it's configured here, for example:
runner: console
console_options:
color_scheme: inverted
color_schemes:
inverted:
success:
name: red
bold: true
failure:
name: green
bold: true
This part of the code documentation really stuck out:
# ## Test Runners
#
# So, now you have this great test class, but you still
# need a way to run it and view any failures that occur
# during the run. There are some test runner; console test
# runner, GTK+ test runner and so on. The console test
# runner is automatically invoked for you if you require
# 'test/unit' and simply run the file. To use another
# runner simply set default test runner ID to
# Test::Unit::AutoRunner:
Maybe you need to have that runner specified in your YML file?
Without seeing how you are calling your script and your directory organization, it will be hard to tell what is causing that issue but I think it begins with it not reading that yaml file for instance.
If all else fails, let me recommend two great unit testing libraries for Ruby if you feel compelled to switch to a more widely-used library:
minitest
RSpec
Edit: You could also revert your directories to be in the same order as before and hardcode in your Gemfile for that test-unit gem the last version that worked for you, like `"test-unit": "3.4.0".
So check this out: it appears as though, upon running bundle exec rspec, there's a race between jekyll serve and puma/rspec's boot up. Sometimes I run the command, and my tests run fine. Other times, I get the error for each of my spec files: cannot load such file -- /path/to/project/sitename_jekyll/_layouts/spec/form_spec.rb which is interesting cause that's not where my spec files are located. They're in /path/to/project/sitename_jekyll/spec/form_spec.rb.
What's crazy is that I can literally just re-run the command over and over and over again and sometimes it'll go through and run the spec tests in the correct location, and sometimes it'll look for them in _layouts and error out. It probably runs correctly maybe once out of ever three or five attempts. All the other times I get the following errors:
Here's what my spec_helper.rb looks like: https://gist.github.com/johnhutch/2cddfafcde0485ff021501d5696c0c2d
And here's an example test file:
https://gist.github.com/johnhutch/a35d15c170f5fd9ca07998bf035d111d
My .rspec only contains two lines:
--color
--require spec_helper
And here's the output, both successful and unsuccesful, back to back:
https://gist.github.com/johnhutch/7927d609170ef5c70a595735502b128d
HEEELLLLLP!
This sounds like jekyll is changing the current directory while building the site, which since it is being run in a thread also affects the tests RSpec is trying to run (See https://bugs.ruby-lang.org/issues/9785 for why Dir.chdir is not threadsafe) - leading to attempts to load things from incorrect locations.
A potential solution to this would be to wait for the Jekyll site to be built before actually running your tests. A comment in your spec_helper seems to state that someone thought passing force_build: true would do this but from a quick perusal of the jekyll-rack code I don't think that's true and you actually need to wait for compiling? to return false (v 0.5) (complete? to return true in the current master branch) to ensure building has finished (as well as passing force_build). This could either be done in a loop sleeping and checking (simpler)
sleep 0.1 while <jekyll app>.compiling?
or (if using the master branch) via the mutex/conditional Rack::Jekyll exposes like in its test suite - https://github.com/adaoraul/rack-jekyll/blob/master/test/helper.rb#L49
Note: Also check my comment about your tests that aren't actually testing anything.
As per Thomas Walpole's super helpful responses this ended up working:
sleep 0.1 while Capybara.app.compiling?
inserted right after:
51 Capybara.app = Rack::Jekyll.new(force_build: true)
in my spec_helper.rb
Thanks again, Thomas!
I have a Logstash plugin written in Ruby that has started failing on TravisCI, but it works locally. Any ideas what this means?
$ bundle exec rspec spec
The signal EXIT is in use by the JVM and will not work correctly on this platform
Coverage may be inaccurate; set the "--debug" command line option, or do JRUBY_OPTS="--debug" or set the "debug.fullTrace=true" option in your .jrubyrc
W, [2016-12-02T14:12:20.127000 #5894] WARN -- : This usage of the Code Climate Test Reporter is now deprecated. Since version
1.0, we now require you to run `SimpleCov` in your test/spec helper, and then
run the provided `codeclimate-test-reporter` binary separately to report your
results to Code Climate.
More information here: https://github.com/codeclimate/ruby-test-reporter/blob/master/README.md
The command "bundle exec rspec spec" exited with 1.
From here:
https://travis-ci.org/mikebski/logstash-filter-datepart
As you can see on the code-climate/ruby-test-reporter CHANGELOG, you need to replace these lines on your test helper file (in my case spec/spec_helper.rb):
require 'codeclimate-test-reporter'
CodeClimate::TestReporter.start
for these ones:
require 'simplecov'
SimpleCov.start
Henceforth, you also have to call codeclimate-test-reporter explicitly.
Hope it helps!
I'm using Ruby 2.2. I need to run a unit test and get information if it succeeded or failed. I'm browsing through docs of both test-unit and minitest (suggested gems for unit testing in Ruby 2.2) but I can't seem to find a method that would return or store somewhere information about the test result.
All I need is information whether the test failed/succeeded, and I need to access it from the level of Ruby. I imagine I would have to use a specific method to run the test - so far, I was only able to run a single test by running the test file, not by invoking any method.
Maybe it's just my poor knowledge of Ruby, anyway I would appreciate any help.
May be you can run the tests using Ruby's ability to run shell command and return results.
Here is an example for test-unit:
test_output = `ruby test.rb --runner console --verbose=progress`
failed_tests = test_output.chomp.split('').count('F')
passed_tests = test_output.chomp.split('').count('.')
puts "P: #{passed_tests}, F: #{failed_tests}"
We are using --verbose=progress option so that we get minimum output. It will look something like below:
.F...F
We count number of F to figure out how many tests failed.
For about test output, the sample program will print:
P: 4, F: 2
Another option is to use passed? method:
https://ruby-doc.org/stdlib-2.1.1/libdoc/minitest/unit/rdoc/MiniTest/Unit/TestCase.html#method-i-passed-3F
Not sure if it's still available in the latest versions of Ruby, so please check that before using.
I've been playing around with autotest trying to make it work all day.. but am having some problems...
I've been following https://github.com/rspec/rspec/wiki/autotest, I'm running with:
Ruby 1.9.3-p194
rspec 2.10.0
ZenTest 4.8.1
I also created a .rspec file.
So with this setup, I run autotest, and it works - my test runs, it passes, hooray!. When I stick a failure into my test e.g. false.should == true, then the test starts looping, over and over again.
what happens is that it's an integration test, and I'm writing to an sqlite db. If I run find . -mmin -1 then I'm able to see that my db folder has changed - so I figured this is the problem.
So I edit .autotest and add the following:
Autotest.add_hook :initialize do |autotest|
%w{db}.each { |exception| autotest.add_exception(exception) }
false
end
But now when I run autotest, it just says the following:
loading autotest/rspec2
and that's it, it won't do anything anymore. Previously the output was:
loading autotest/rspec2
/home/me/.rbenv/versions/1.9.3-p194/bin/ruby -rrubygems -S '/home/me/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.10.1/exe/rspec' ``--tty '/home/me/Workspace/myproject/spec/integration/db/lead_spec.rb'
and then it'd run my test and show the result...
Anyone know what could be going on? it's very frustrating, and I feel like I've come to a road block....
Thanks for your help!
Autotest checks if defined exceptions match any part of the filename. Your spec has db in it's path so it is ignored by autotest.
If you want to ignore db folder, then do the following:
Autotest.add_hook :initialize do |a|
a.add_exception %r{^\./db}
end