I'm using CruiseControl.rb (2.0.0pre1) for CI with a Rails 3 app and I'm trying to get it to work with rspec. Inside the cruise_control.rb I'm calling the spec rake task
Project.configure do |project|
project.rake_task = 'spec'
end
Very basic and easy configuration. But even when a test is failing, the build passes. It doesn't detect wether the tests passes or fail.
Furthermore I want to know if I can grab the exit status from the rake task, and call a ruby script depending if a build fails or passes.
I found out, that it was an error in Rails 3.2.1. The status code returned from the tests was always 0. (see https://github.com/rails/rails/issues/4923 and http://github.com/rails/rails/commit/abe4a8d070d069f24f7befd9a8da25c40f4c2a6d). Upgrading to Rails 3.2.2.rc1 fixed the problem.
Related
So check this out: it appears as though, upon running bundle exec rspec, there's a race between jekyll serve and puma/rspec's boot up. Sometimes I run the command, and my tests run fine. Other times, I get the error for each of my spec files: cannot load such file -- /path/to/project/sitename_jekyll/_layouts/spec/form_spec.rb which is interesting cause that's not where my spec files are located. They're in /path/to/project/sitename_jekyll/spec/form_spec.rb.
What's crazy is that I can literally just re-run the command over and over and over again and sometimes it'll go through and run the spec tests in the correct location, and sometimes it'll look for them in _layouts and error out. It probably runs correctly maybe once out of ever three or five attempts. All the other times I get the following errors:
Here's what my spec_helper.rb looks like: https://gist.github.com/johnhutch/2cddfafcde0485ff021501d5696c0c2d
And here's an example test file:
https://gist.github.com/johnhutch/a35d15c170f5fd9ca07998bf035d111d
My .rspec only contains two lines:
--color
--require spec_helper
And here's the output, both successful and unsuccesful, back to back:
https://gist.github.com/johnhutch/7927d609170ef5c70a595735502b128d
HEEELLLLLP!
This sounds like jekyll is changing the current directory while building the site, which since it is being run in a thread also affects the tests RSpec is trying to run (See https://bugs.ruby-lang.org/issues/9785 for why Dir.chdir is not threadsafe) - leading to attempts to load things from incorrect locations.
A potential solution to this would be to wait for the Jekyll site to be built before actually running your tests. A comment in your spec_helper seems to state that someone thought passing force_build: true would do this but from a quick perusal of the jekyll-rack code I don't think that's true and you actually need to wait for compiling? to return false (v 0.5) (complete? to return true in the current master branch) to ensure building has finished (as well as passing force_build). This could either be done in a loop sleeping and checking (simpler)
sleep 0.1 while <jekyll app>.compiling?
or (if using the master branch) via the mutex/conditional Rack::Jekyll exposes like in its test suite - https://github.com/adaoraul/rack-jekyll/blob/master/test/helper.rb#L49
Note: Also check my comment about your tests that aren't actually testing anything.
As per Thomas Walpole's super helpful responses this ended up working:
sleep 0.1 while Capybara.app.compiling?
inserted right after:
51 Capybara.app = Rack::Jekyll.new(force_build: true)
in my spec_helper.rb
Thanks again, Thomas!
UPDATE: when I run rake test from root, I get:
C:\workspace\faker>rake test
DL is deprecated, please use Fiddle
C:/RailsInstaller/Ruby2.1.0/bin/ruby.exe -w -I"lib;test;." -I"C:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib" "C:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/rake_test_loader.r
b" "test/test_array_sample_method_compat.rb" "test/test_avatar.rb" "test/test_en_au_ocker_locale.rb" "test/test_en_ca_locale.rb" "test/test_en_locale.rb" "test/test_en_ug_locale.rb" "test/test_en_us_locale.rb" "test/test_es_l
ocale.rb" "test/test_faker.rb" "test/test_faker_app.rb" "test/test_faker_bitcoin.rb" "test/test_faker_book.rb" "test/test_faker_business.rb" "test/test_faker_city.rb" "test/test_faker_code.rb" "test/test_faker_color.rb" "test
/test_faker_commerce.rb" "test/test_faker_company.rb" "test/test_faker_date.rb" "test/test_faker_hacker_talk.rb" "test/test_faker_internet.rb" "test/test_faker_lorem.rb" "test/test_faker_name.rb" "test/test_faker_number.rb" "
test/test_faker_shakespear.rb" "test/test_faker_shakespeare.rb" "test/test_faker_slack_emoji.rb" "test/test_faker_street.rb" "test/test_faker_team.rb" "test/test_faker_time.rb" "test/test_faker_university.rb" "test/test_flexi
ble.rb" "test/test_helper.rb" "test/test_locale.rb" "test/test_pl_locale.rb" "test/test_uk_locale.rb"`
I don't fully understand what rake is doing at this point, but the system hangs infinitely. What am I missing?
So I have built some added functionality for the faker gem, and want to run the tests I wrote before submitting a pull request. However, all my attempts to run the existing tests, and my new tests, have failed. I am worried that it is a Windows machine related issue.
I have tried running rake test:units but it gives me an error and then hangs, while also ringing the Bell on my system 5 times. Strange right?
I have also tried running bundle exec rspec spec, but no tests are found becasue they are not RSpec unit tests, or at least not in the right location.
To see how the gem is formatted, check it out at the repo
All the test files, including my new one, live in the test directory. But for the life of me, I can't run them.
I tried this and works fine:
bundle install
bundle exec ruby test
I'm using Ruby 2.2. I need to run a unit test and get information if it succeeded or failed. I'm browsing through docs of both test-unit and minitest (suggested gems for unit testing in Ruby 2.2) but I can't seem to find a method that would return or store somewhere information about the test result.
All I need is information whether the test failed/succeeded, and I need to access it from the level of Ruby. I imagine I would have to use a specific method to run the test - so far, I was only able to run a single test by running the test file, not by invoking any method.
Maybe it's just my poor knowledge of Ruby, anyway I would appreciate any help.
May be you can run the tests using Ruby's ability to run shell command and return results.
Here is an example for test-unit:
test_output = `ruby test.rb --runner console --verbose=progress`
failed_tests = test_output.chomp.split('').count('F')
passed_tests = test_output.chomp.split('').count('.')
puts "P: #{passed_tests}, F: #{failed_tests}"
We are using --verbose=progress option so that we get minimum output. It will look something like below:
.F...F
We count number of F to figure out how many tests failed.
For about test output, the sample program will print:
P: 4, F: 2
Another option is to use passed? method:
https://ruby-doc.org/stdlib-2.1.1/libdoc/minitest/unit/rdoc/MiniTest/Unit/TestCase.html#method-i-passed-3F
Not sure if it's still available in the latest versions of Ruby, so please check that before using.
I have tests in the folder:
test/models/person/helper/age_calculator_test.rb
These test should fail because of:
assert_equal false, true
they do fail when I run:
bundle exec m test/models/person/helper/age_calculator_test.rb
But when Im running:
bundle exec m test/models
These Tests are not run! What do I wrong?
How can I test all of my Model-Test with Bundler in one command? Thanks!!
It appears that you use the Metal test runner. I cannot see anywhere on its documentation supporting path testing such as test/models. Its main feature is to run tests by line number.
If you would like to execute all of your model tests you could use Rails'
rake test:models
to do it. See the RoR Guide on Rake Tasks for Running your Tests for more options.
I am using buildr for some time now, but today I came over a little problem in connection to unit testing of ruby code.
So in my buildfile I have these lines:
define "ruby-project" do
project.version = VERSION_NUMBER
project.group = GROUP
Rake::TestTask.new(:test_rb) do |t|
t.warning = true
t.verbose = true
t.test_files = FileList['test/*.rb']
end
task test => [:test_rb]
end
running buildr test actually runs the tests, what is nice. The test is actually just that:
require 'test/unit'
class TestFileParse < Test::Unit::TestCase
def test_fail
assert(false, 'test to fail')
end
end
As expected it fails, BUT what is strange for me is that buildr quits the build with that message:
sh: 2: Syntax error: Unterminated quoted string
Buildr aborted!
RuntimeError : Command failed with status (1): [/usr/bin/ruby1.9.1 -w -I"lib" -I"/var/lib/...]
Running ruby file-with-failing-test-from-above.rp does not throw a runtime error, instead it prints the test report on screen, what is what is what I want.
Question
How can I make Buildr run the unit tests without quitting with an RuntimeError if a test fails?
Greetings Philipp
Since I am not too familiar with Ruby development, what involves Rake, I was looking for the wrong question. Instead of looking for: »how to run unit test with buildr«, the question should have been »how to run unit tests with rake«, because buildr is a kind of extended Rake (similar to the »maven-ant-relationship«). So everything one can do in Rake, one can do in buildr, too. Therefore on good SO answer to run ruby unit tests in buildr is here.
Additionally it is possible to run RSpec's with buildr, therefore one has two options:
set project.test.using :rspec, what involves the use of JRuby, so one has to set JRUBY_HOME (in my case ~/.rvm/rubies/jruby-1.7.9/), or run buildr within jruby. In my case this slowed test execution down, because each time a jvm needed to be started.
Or one can use rspec's rake task in this manner. I choose this method since my tests run much faster without the jvm overhead.
n.b. as the answer implies, I switched over to rspec.
Happy Testing/Speccing!