I am learning Rails the age old way. By reading Agile Web Development with Rails (3rd Edition) as a starting point. I am currently in the chapter that teaches Testing. I am also aware of other BDD Testing framework such as RSPec. So I was wondering if frameworks such as RSpec, Cucumber, Shoulda replace the need for knowing/using Test::Unit? Are they both relevant in their own right and used for different purposes?
UPDATE: Since I am new to Testing, I would also love to get feedback on what resources are useful to get my feet wet with different frameworks.
Thanks!
The world has changed! Check down and upvote #prusswan
I think Test::Unit is still relevant even though there is a lot of hype surrounding BDD and other testing tools.
That being said, if you are up for it, you could bypass learning Test::Unit and start with something like RSpec and Shoulda right away, but there is something to be said about following through the examples in the Agile Web Development Book to see where the ideas from BDD came from.
I find myself still using Test::Unit for some projects since it comes with Rails and is still a very great testing framework.
So long story short, I don't think it's obsolete but it's not the cutting edge any more. BDD is a testing paradigm shift especially if you start using Cucumber and Webrat, but it's fantastic once you get into it. Shoulda is the easiest to make the transition to, so I would start with Test::Unit, then move to Shoulda, then Give RSpec and Cucumber a try.
You are testing or at least interested in testing! That's the best part. In the end it doesn't matter what you use as long as you are happy with it.
Good luck!
DHH himself is one of the avid users of Test::Unit and avoider of RSpec / Cucumber for their relative complexity.
I respect the guys behind it and I'm all for experimentation, but the proliferation of rSpec and Cucumber makes me sad.
RSpec offends me aesthetically with no discernible benefit for its added complexity over test/unit.
Cucumber makes no sense to me unless you have clients reading the tests. Why would you build a test-specific parser for English?
The important thing is of course that we get people testing, so tools shouldn't matter too much. But the added complexity still upsets me.
"something.should be_true" sums it up the aesthetic argument for me pretty well. That call hurts my eyes vs "assert something".
If you are interested, keep reading the twitter as of these twits.
Just thought I would post an update from 2012:
Out of the three main and current (usable with Rails 3.2 with no more than some minor modifications) introductory books on RoR, two of them are using Rspec (with at most a short mention for Test::Unit) and one of them is planning on even moar Rspec for the next edition, so it is clear who would be the winner of this "battle" for now
And allow me to selectively quote dhh:
The important thing is of course that we get people testing, so tools shouldn't matter too much.
Revisit in 2014:
Minitest has now superceded Test::Unit as a default in the current versions of Rails. Personally I feel it deserves a closer look compared to its predecessor for a number of reasons -
Its status as a default
Better support for capybara and selenium, a large part due to gems like this
Relatively lightweight
Mostly backward compatible with Test::Unit
Rspec is an entirely separate testing framework. Shoulda is an enhancement to Rails's built-in framework, Test::Unit. If you're using Shoulda, you're using Test::Unit but with more capabilities and simpler, more readable syntax.
I've tried Rspec and Shoulda and for me, Shoulda wins hands down. I like it way better. But that may be a matter of taste.
Note that you can use Cucumber with Shoulda.
Other resources? I recommend the ZenTest and RedGreen gems. ZenTest provides autotest, which lets you run your tests automatically every time you change a file. It's a big help.
Regarding fixtures vs. factories, if you need to set up a bunch of interrelated objects where you're testing both sides or a parent-child relationship and/or testing many-to-many relationships, fixtures work a lot better. Actually I'm not even sure you can do that with factories. So don't dismiss fixtures -- they have their uses.
There are a lot of people who still like Test::Unit, and to some extent, it's a personal preference. However, on balance, you will find far more activity on the RSpec front. The really cool stuff is all being done with RSpec and Cucumber, so if you don't have a personal preference yourself, I'd probably skip Test::Unit. You should, however, be familiar enough with it to read someone else's tests that are written with it, but I wouldn't foresee that ever being a problem.
Related
Basically are there things I should avoid doing in ruby that may break cucumber.
Bonus points if you can make me understand cucumber more as someone who went from c# to picking up ruby.
Things "I should avoid doing in ruby that may break cucumber"? No. Cucumber is a language-agnostic testing framework. Cucumber's goal is to express use cases in human-readable language that serves as both business specifications and executable test scenarios. In fact, cucumber was initially forked out of rspec, so it's a product of the the ruby open-source community. You'll likely run into some gotchas along the way, but there's no magic bullet. You just have to read the docs (of which there are plenty) and ride the learning curve.
I've been reading the README for the MSpec project, and though it does a lot of explaining about what it is and (what it is not) with several contrasts between itself and RSpec, there's nothing about why it exists. Would using RSpec (at the time of starting MSpec) have caused problems in some way, or was it missing some features? Are these things still true? Could an extension have been (or be) written for RSpec that would do this? Is it something political?
There's obviously a lot of documentation and examples for RSpec, more features and more updates to the library, and since MSpec seems harder to use IMO (considering the differences in feature set and my own comfort level with RSpec) I'd be very interested if anyone knows the reasons. Perhaps that sounds critical, but that's not my point, I'm just trying to supply some context - there are likely to be good reasons for all of this and that's what I wish to find out.
From the README:
MSpec attempts to use the simplest Ruby language features so that beginning
Ruby implementations can run the Ruby specs.
This was designed for incomplete implementations (Specifically Rubinius) of the base Ruby language. It doesn't use all the language features of Ruby, so it's easier to bootstrap your implementation to the point where you can run mspec's.
If you aren't creating a new implementation for the Ruby language, then you shouldn't use this.
I am new to the unit testing scene; I have only been using unit tests for about 2 months now. When I unit test in Ruby I currently follow the TDD style and use Test::Unit::TestCase. I have also read about RSpec and how it follows the BDD methodology. I also have read about Shoulda which is a in between the two frameworks. My question is, which framework should I devote my time to RSpec or Shoulda, or should I stick with Test::Unit::TestCase? Any opinions on the subject are appreciated.
Shoulda compliments Test::Unit and RSpec so I don't think it's a case of choosing Shoulda over RSpec, or vice-versa. I use Shoulda on multiple projects, some tested with Test::Unit, others with RSpec, and it works great alongside both frameworks.
I'd recommend trying Shoulda on your existing Test::Unit project. It can save you a lot of key strokes and keep your test classes a bit tidier and more consistent.
As Andrew Vit rightly says, the Shoulda team are favouring the RSpec way of doing things more and more (see http://robots.thoughtbot.com/post/701863189/shoulda-rails3-and-beyond). It's worth trying RSpec and see how you like it. It's not to everyone's taste, but I prefer it to Test::Unit for its syntax, expressiveness, nested contexts, custom matchers and just all round coolness! ;-)
RSpec has a lot of nice features, more than just the syntax too. I would recommend it over Shoulda.
Even Thoughtbot, the authors of Shoulda, are using RSpec now. Shoulda now works as an RSpec plugin and it provides a number of useful matchers.
There's no either-or. Use the different testing tools where they fit best.
Shoulda is a great single-line testing tool. It helps you to keep your tests clean and concise. If you like this style, too, I'd also recommend the Zebra gem. It's a nice little tool to write one-liner tests.
What I mean by that is, which tools are for unit testing, which for BDD, is there even a clear divide?
I'm just learning Ruby from C#, and the tools I'm familiar with there are xUnit-style for pure unit testing/TDD, and Cucumber/SpecFlow for BDD/integration testing. Of course, you can use unit testing tools for integration testing if you want to.
The main testing tools I've come across for Ruby are Test::Unit, RSpec, and Cucumber (there are obviously plenty more but these seem to be the most popular, and hence probably a good bet to start off with).
I'm not looking for a long list of tools here - if there's a main contender I've missed then please tell me, but I'd rather keep it simple at this stage than start using 'Tool Z beta with the Library X extensions running on the MegaChode platform' or whatever. I'm only a poor confused .Net guy, after all!
It's clear that Test::Unit was designed for unit testing, and Cucumber is for BDD. But I'm not sure about RSpec - some people seem to use it as a low-level unit testing tool, and others for higher-level BDD. At the same time, some people seem to use Test::Unit (and similar) with additional libraries for BDD.
To give you an idea of where I'm coming from, the way I see it is that you can write more functional, testable specifications (which a business guy might understand) with your BDD tool and then do low-level, developer-focussed TDD with your unit testing tool as you implement the functionality required for your Cuke (or whatever).
Is there a generally accepted way of doing these things in Ruby, or do I need to forget about such a crazy notion and use whatever seems to do the job?
Cheers,
Grant
I think most Rubyists have standardized on RSpec rather than Test::Unit for testing. If you're doing Rails apps, there is RSpec-Rails and then Cucumber you can layer on top of it.
Grab a copy of The Rspec Book # http://www.pragprog.com/titles/achbd/the-rspec-book
MiniTest contains both an xUnit and spec style testing interface (with the ability to combine them both in the same file).
It is included as part of the standard gems since Ruby2.2.
Both can be included in the same file which makes it easy to convert between the styles or other testing tools like RSpec and Test::Unit
require 'minitest/autorun'
describe Math do
describe 'addition' do
it '1+1=2' do
(1+1).must_equal 2
end
end
end
class MathTest < Minitest::Test
def test_1_plus_1_equal_2
assert_equal 2, 1+1
end
end
MiniTest also includes a mocking framework (although I prefer rr)
Does Ruby have any tools along the lines of pylint for analyzing source code for errors and simple coding standards?
It would be nice if it could be integrated with cruisecontrolrb for continuous integration.
Or does everyone write such good tests that they don't need source code checkers!
I reviewed a bunch of Ruby tools that are available here
http://devver.wordpress.com/2008/10/03/ruby-tools-roundup/
most of the tools were mentioned by webmat, but if you want more information I go pretty in depth with examples.
I also highly recommend using Metric-Fu it gives you a on gem/plugin install of 3 of the more popular tools and is built with cruisecontrolrb integration in mind.
The creator has a great post that should help get you up and running in no time.
http://jakescruggs.blogspot.com/2008/04/dead-simple-rails-metrics-with-metricfu.html
There has been a lot of activity in Ruby tools lately which I think is a good sign of a growing and maturing language.
Check these out:
on Ruby Inside, an article presenting Towelie, Flay and Simian, all tools to find code duplication
Towelie
flay
Simian (a more general tool that supports Ruby among other languages)
reek: a code smell detector for Ruby
Roodi: checks the style of your Ruby code
flog: a code complexity analyzer
rcov: will give you a C0 (if I remember correctly) code coverage analysis. But be careful though. 100% coverage is very costly and doesn't mean perfect code.
heckle: changes your code in subtle manners and runs your test suite to see if it catches it. Brutal :-)
Since they're all command-line tools they can all be integrated simply as cc.rb tasks. Just whip out your regex skillz to pick the important part of the output.
I recommend you first try them out by hand to see if they play well with your codebase and if you like the info they give you. Once you find a few that give you value, then spend time integrating them in your cc.
One recently-updated interesting-looking tool is Ruby Object Oriented Design Inferometer - roodi for short. It's at v1.3.0, so I'm guessing it's fairly mature.
I haven't tried it myself, because my code is of course already beyond reproach (hah).
As for test coverage (oh dear, I haven't tried this one either) there's rcov
Also (look, I am definitely going to try some of these today. One at least) you might care to take a look at flog and flay for another style checker and a refactoring candidate finder.
There's also the built-in warnings you can enable with a quick:
ruby -w
Or setting the global variable $VERBOSE to true at any point.
Code metrics on ruby toolbox website.
Rubocop is a widely-used static code analyzer.
I just released Excellent which implements several checks on Ruby code. It combines roodi, reek and flog and also adds some Rails specific checks:
gem sources -a http://gems.github.com
sudo gem install simplabs-excellent
May be helpful...