One test suite for functional and performance test - ruby

I have been playing around watir-webdriver-performance gem of late and I am quiet interested to test the functionality and performance of the website using single test suite.
Thanks to Alister Scott for this
Now I want to include this into my env.rb file and run after each scenario.
Any inputs in this regards would be of great help.

Related

Distinguish logical and structural/exception errors

I am learning Ruby developement with Rspec and Cucumber. I have a hard time knowing when I have to switch from one to another. I know Rspec is used for logical errors while Cucumber, for structural/exceptions errors.
How can I know what type of error it gives? Is there a certain pattern of error reporting.
For example, expected ... is a logical error.
Cucumber is usually used for acceptance tests, but under the hoods you're using Capybara steps for browser automation, which of course are available to Rspec too.
SO it's up to you. You can use either Cucumber for acceptance tests and Rspec for unit tests, or use Rspec for everything. I personally go with the latter, but it's really up to you. Try both, and see how it works for you.

How do you get StoryTest to work?

Does StoryTest (http://storytestiq.solutionsiq.com/wiki/Main_Page) work? Everything on the wiki is under construction.
If you have gotten it to work, can you share how you did it?
While StoryTest appears to be a very cool approach to managing Selenium and FitNesse type tests, there has been no activity on that project in a year. I would chalk it up to abandoned.
I would consider using Selenium with SpecFlow to achieve a similar result that could be run from your continuous integration server.
Selenium
SpecFlow
Steven Sanderson has an awesome article on his blog about using SpecFlow with WatiN, his technique should be in the neighborhood of what you are attempting to accomplish.
http://blog.stevensanderson.com/2010/03/03/behavior-driven-development-bdd-with-specflow-and-aspnet-mvc/

What are some good example open source Ruby projects that use Cucumber and RSpec well?

What are some good example open source Ruby projects that use Cucumber and RSpec well?
Update: While the suggestions below where quite impressive, I wanted to see how others are using RSpec and Cucumber to drive product development.
From Quora:
Teng Siong Ong, Ruby, Python, etc.
https://github.com/teambox/teambox
I find it really nice that the RSpec project is tested with both Cucumber and RSpec itself:
https://github.com/rspec/rspec-core/tree/master/features
https://github.com/rspec/rspec-core/tree/master/spec
Cucumber is also self-hosted test-wise:
https://github.com/aslakhellesoy/cucumber/tree/master/features
I've also found it very useful that Aruba, a library of steps for testing CLI apps with Cucumber, is itself tested with Cucumber:
https://github.com/aslakhellesoy/aruba/tree/master/features
RubySpec is a rather ambitious effort to write specs for the Ruby language.

RSpec vs. Shoulda?

I am new to the unit testing scene; I have only been using unit tests for about 2 months now. When I unit test in Ruby I currently follow the TDD style and use Test::Unit::TestCase. I have also read about RSpec and how it follows the BDD methodology. I also have read about Shoulda which is a in between the two frameworks. My question is, which framework should I devote my time to RSpec or Shoulda, or should I stick with Test::Unit::TestCase? Any opinions on the subject are appreciated.
Shoulda compliments Test::Unit and RSpec so I don't think it's a case of choosing Shoulda over RSpec, or vice-versa. I use Shoulda on multiple projects, some tested with Test::Unit, others with RSpec, and it works great alongside both frameworks.
I'd recommend trying Shoulda on your existing Test::Unit project. It can save you a lot of key strokes and keep your test classes a bit tidier and more consistent.
As Andrew Vit rightly says, the Shoulda team are favouring the RSpec way of doing things more and more (see http://robots.thoughtbot.com/post/701863189/shoulda-rails3-and-beyond). It's worth trying RSpec and see how you like it. It's not to everyone's taste, but I prefer it to Test::Unit for its syntax, expressiveness, nested contexts, custom matchers and just all round coolness! ;-)
RSpec has a lot of nice features, more than just the syntax too. I would recommend it over Shoulda.
Even Thoughtbot, the authors of Shoulda, are using RSpec now. Shoulda now works as an RSpec plugin and it provides a number of useful matchers.
There's no either-or. Use the different testing tools where they fit best.
Shoulda is a great single-line testing tool. It helps you to keep your tests clean and concise. If you like this style, too, I'd also recommend the Zebra gem. It's a nice little tool to write one-liner tests.

Is Test::Unit still relevant in rails?

I am learning Rails the age old way. By reading Agile Web Development with Rails (3rd Edition) as a starting point. I am currently in the chapter that teaches Testing. I am also aware of other BDD Testing framework such as RSPec. So I was wondering if frameworks such as RSpec, Cucumber, Shoulda replace the need for knowing/using Test::Unit? Are they both relevant in their own right and used for different purposes?
UPDATE: Since I am new to Testing, I would also love to get feedback on what resources are useful to get my feet wet with different frameworks.
Thanks!
The world has changed! Check down and upvote #prusswan
I think Test::Unit is still relevant even though there is a lot of hype surrounding BDD and other testing tools.
That being said, if you are up for it, you could bypass learning Test::Unit and start with something like RSpec and Shoulda right away, but there is something to be said about following through the examples in the Agile Web Development Book to see where the ideas from BDD came from.
I find myself still using Test::Unit for some projects since it comes with Rails and is still a very great testing framework.
So long story short, I don't think it's obsolete but it's not the cutting edge any more. BDD is a testing paradigm shift especially if you start using Cucumber and Webrat, but it's fantastic once you get into it. Shoulda is the easiest to make the transition to, so I would start with Test::Unit, then move to Shoulda, then Give RSpec and Cucumber a try.
You are testing or at least interested in testing! That's the best part. In the end it doesn't matter what you use as long as you are happy with it.
Good luck!
DHH himself is one of the avid users of Test::Unit and avoider of RSpec / Cucumber for their relative complexity.
I respect the guys behind it and I'm all for experimentation, but the proliferation of rSpec and Cucumber makes me sad.
RSpec offends me aesthetically with no discernible benefit for its added complexity over test/unit.
Cucumber makes no sense to me unless you have clients reading the tests. Why would you build a test-specific parser for English?
The important thing is of course that we get people testing, so tools shouldn't matter too much. But the added complexity still upsets me.
"something.should be_true" sums it up the aesthetic argument for me pretty well. That call hurts my eyes vs "assert something".
If you are interested, keep reading the twitter as of these twits.
Just thought I would post an update from 2012:
Out of the three main and current (usable with Rails 3.2 with no more than some minor modifications) introductory books on RoR, two of them are using Rspec (with at most a short mention for Test::Unit) and one of them is planning on even moar Rspec for the next edition, so it is clear who would be the winner of this "battle" for now
And allow me to selectively quote dhh:
The important thing is of course that we get people testing, so tools shouldn't matter too much.
Revisit in 2014:
Minitest has now superceded Test::Unit as a default in the current versions of Rails. Personally I feel it deserves a closer look compared to its predecessor for a number of reasons -
Its status as a default
Better support for capybara and selenium, a large part due to gems like this
Relatively lightweight
Mostly backward compatible with Test::Unit
Rspec is an entirely separate testing framework. Shoulda is an enhancement to Rails's built-in framework, Test::Unit. If you're using Shoulda, you're using Test::Unit but with more capabilities and simpler, more readable syntax.
I've tried Rspec and Shoulda and for me, Shoulda wins hands down. I like it way better. But that may be a matter of taste.
Note that you can use Cucumber with Shoulda.
Other resources? I recommend the ZenTest and RedGreen gems. ZenTest provides autotest, which lets you run your tests automatically every time you change a file. It's a big help.
Regarding fixtures vs. factories, if you need to set up a bunch of interrelated objects where you're testing both sides or a parent-child relationship and/or testing many-to-many relationships, fixtures work a lot better. Actually I'm not even sure you can do that with factories. So don't dismiss fixtures -- they have their uses.
There are a lot of people who still like Test::Unit, and to some extent, it's a personal preference. However, on balance, you will find far more activity on the RSpec front. The really cool stuff is all being done with RSpec and Cucumber, so if you don't have a personal preference yourself, I'd probably skip Test::Unit. You should, however, be familiar enough with it to read someone else's tests that are written with it, but I wouldn't foresee that ever being a problem.

Resources