Rake tests running very slow - ruby

Question

After running some tests, I'm convinced there has to be something wrong with my setup (windows, rubymine and latest ruby versions). My times right now are:
Finished tests in 14.289817s, 0.0700 tests/s, 0.3499 assertions/s.
1 tests, 5 assertions, 0 failures, 0 errors, 0 skips
Process finished with exit code 0
With 5 VERY easy tests (just checking if validation on empty fields works). The total time for these 5 unit tests is 160 seconds, over 2 minutes.
What could I do to improve this speed?
Here are the tests:
require 'test_helper'
class ItemTest < ActiveSupport::TestCase
test 'item attributes must not be empty' do
item = Item.new
assert item.invalid?
assert item.errors[:name].any?
assert item.errors[:description].any?
assert item.errors[:image_url].any?
assert item.errors[:rating].any?
end
end

Answer 1

Your problem is Windows. We use JRuby on Windows and it actually runs faster than RubyInstaller(mingw) ruby on Windows but we do see very slow results when running test suites or starting a rails server. About 1 minute for a single test run due to the loading of the Rails environment.
You have a few options:
Switch to linux / osx
Use spork to keep a couple rails environments pre-loaded for your tests. Note that this isn't perfect but it will reduce your times substantially. With this option you'll probably want to use minitest or rspec, I had trouble getting spork to work on Windows with testunit. With spork you should be able to get your single test run time down to about 10 seconds.
Write as many of your tests to run outside of Rails, in other words to not require the Rails stack. This will be very fast, you should be able to run a test in only a few seconds but as you could guess, it's hard to test a lot of things (controllers, views) outside of rails. Works perfectly though for functions you've broken out into modules that already do not require anything from rails.
Good luck!

Answer 2

What's the rest of your gem stack? Sometimes third-party gems are initialized by rails and will try to phone home (New Relic, Airbrake) which can inflate your test times (though probably not by this much). If something isn't strictly required for your test suite, you should try to pull it into the proper env group, or set require :false via bundler:
group :production do
gem 'newrelic_rpm'
end

Answer 3

Startup time appears to be killing you, so you and I are probably in the same boat. Between Jodell and Dark Castle a lot of this is covered already, but here's my nearly-whole list of things that helped, in descending order of efficacy.
Get a patched 1.9.3 with the 2.0 filesystem improvements backported. The first gets 2x better numbers but I'm using the second because I felt the first was unstable
https://groups.google.com/forum/?fromgroups=#!topic/thecodeshop/WVA4N2lJoM4
https://raw.github.com/gist/3242245/5cdb932c0a544cfbb51fdee5759942d4d2c263f5/1-complete.diff
Set your GC options
http://snaprails.tumblr.com/post/241746095/rubys-gc-configuration (I'm using the last one)
Turn off collecting coverage data (My IDE kept volunteering this)
Run Spork, and set SPEC_OPTS=--drb
Turn off virus scanner (don't actually do this, it's only worth 10% for me anyway)
Double-check your Gems, delaying the loading of gems with require: false
6 didn't actually buy me very much. The biggest thing we had wrong was loading Thin unconditionally (which pulls in Eventmachine, which 21 megabytes installed), but the next 3 on the stack were actually being used by RSpec. I have not looked at network traffic, but Jodell is probably on to something there.