How would you make a decision between cucumber and shoulda if you were about to choose a testing framework?
What differentiates these two frameworks primarily?
As everybody has pointed out, Cucumber and Shoulda have fairly different objectives. You can think of Cucumber as being the "view from 10,000 feet" testing framework - you define a broad feature or specific user interaction and make sure everything works together. Shoulda is for unit testing - you pick out a specific model and thoroughly test all the picky little bits of functionality for that individual piece.
Usually, you would want to use these sort of frameworks in conjunction. For example, most of your broad, high level tests can be written in Cucumber, but when there's a particularly complex or opaque bit of code in your system, you can drill down with Shoulda or RSpec to test the particulars.
They have completely different objectives. Shoulda is a unit testing extension built on top of Test::Unit.
Cucumber is a Acceptance/Functional Testing framework that can use Test::Unit/RSpec/Whatever for doing the assertions.
Shoulda can be directly compared to RSpec, for instance.
I don't see anyone else mentioning that you actually can use Shoulda as the "test-engine" for Cucumber.
Cucumber is targeting Acceptance Testing.
Shoulda is a Unit Testing framework.
Shoulda is an extension of the Test::Unit framework that consists of test macros, assertions, and helpers. Shoulda is a prettier way to write unit tests.
Cucumber - a rewrite of RSpec's "Story runner" - is a tool for Behaviour-Driven Development. It allows you to write executable specifications in a business-readable domain-specific language. Cucumber is more an acceptance testing tool.
Cucumber and Shoulda have thus different objective (even if Shoulda can be used for BDD).
Related
I was working on setting up Selenium in a project today, and the thought came to mind: "Should I be using the NUnit framework in correlation with my Selenium tests?"
Here's my concern with using the NUnit framework: From the NUnit website, it states that: "NUnit is a unit-testing framework for all .Net languages". The purpose of the framework is to build unit tests, not integration tests.
Selenium tests are typically (I don't know of any instance when they aren't) integration tests. So, going back to my question, is it good practice to use a unit testing framework to do integration tests? Are there integration test frameworks that are robust enough to compete with NUnit, of which would be more appropriate?
The two main options for .NET development are, for the most part, NUnit, and MSTest. IMHO, neither are very well optimized for automated browser tests.
You can force each to do what you want, but features such as TestNG's #DataProvider are a pain to implement, especially if you are dynamically changing your data provider for each test (say, loading your browsers to test through a properties file) -- this is trivial with TestNG, but NUnit and MSTest take a significant amount of "hacking" to make it work.
TL;DR version: Someone really needs to port TestNG over to .NET :)
Which is the difference among spec / rspec and mspec?
In ruby spec home, it said that we should use mspec command.
But some other guys said it should be 'spec kind_of_spec.rb'.
Also someones said we should use 'rspec'.
Which is the difference of these three modules?
In short - these modules have the same aim (testing) but different kinds of toolsets to reach that.
RubySpec . RubySpec is a project to write an executable specification for the Ruby Programming Language.
RSpec - RSpec is a Behaviour-Driven Development tool for Ruby programmers. BDD is an approach to software development that combines Test-Driven Development, Domain Driven Design, and Acceptance Test-Driven Planning. RSpec helps you do the TDD part of that equation, focusing on the documentation and design aspects of TDD.
MSpec - MSpec is a specialized framework that is syntax-compatible with RSpec for
basic things like +describe+, +it+ blocks and +before+, +after+ actions. MSpec
contains additional features that assist in writing the RubySpecs used by
multiple Ruby implementations.
These are different tools / project with very simliar names but completely different aims!
1) rspec
RSpec is a Tool you use to test your own ruby code. It's a
replacement for the built in testing framework of Rails described
in the Rails Guide here.
If you switch to rspec, you run the command
rake spec
to run your whole test suite, or you run a command like
rspec spec/models/course_spec.rb
to run one specific part of the test.
2) rubyspec and mspec
RubySpec is a project to write an executable specification for the Ruby Programming Language. There is a need for such a specification because there a several different ruby interpreters: Matz's Ruby Interpreter (called MRI) is the de facto standard, but there are also Rubinius, JRuby, IronRuby, MacRuby, HotRuby,...
MSpec is a tool used in developin the Ruby Specs. It's similar in usage zu rspec.
If you want to write a new Ruby interpreter you can use rubyspec to test if you
are doing it right.
But RubySpec is not something need if you're just using ruby to develop your own web project.
RubySpec isn't the same as RSpec
Starting with RSpec2 you should use rspec <path> command instead of rake spec to run your specs
I just upgraded to XCode 4 and I was wondering if I need to 'include unit tests' when setting up an application? Also, what does that mean exactly?
You do not need to include unit tests.
What does "unit testing" mean? (from the unit-testing FAQ)
Unit testing is a method by which individual units of source code are tested to determine if they are fit for use. A unit is the smallest testable part of an application. In procedural programming a unit may be an individual function or procedure. Unit tests are created by programmers or occasionally by white box testers.
Ideally, each test case is independent from the others: substitutes like method stubs, mock objects, fakes and test harnesses can be used to assist testing a module in isolation. Unit tests are typically written and run by software developers to ensure that code meets its design and behaves as intended.Wikipedia
Unit testing is closely related to Test Driven Development.
#ToddH points out:
It's easier to include [unit tests] when you setup the project. If you do it later there are quite a few steps involved in doing it correctly: http://twobitlabs.com/2011/06/...
Thanks for the protip, Todd!
I have a system thats highly reliant on various web APIs. I would like to run my API specific tests at least once per day to make sure all API's are still playing nicely and alive. I have a set of unit tests (just plain rb files that test API calls for expected data) and would like to run these every 24 hours. If something breaks, I would like to take an action (e.g. email or sms me).
How best to setup automated Ruby tests and parse the result? Can I just setup a cron job to handle the .rb files? How would I take an action and detect programmatically if the tests are failing? Maybe there is some kind of continuous integration solution for RUby that can handle this?
I've just gone through the process of setting up Hudson CI as my integration server, using this amazing tutorial from Dr. Nic. It installs through a gem, coming pretty much preconfigured, and was extremely simple to get working.
I'm using rspec and cucumber, and Hudson runs all tests when it sees a new commit on my git repository. If all tests pass, it merges the code into my master branch. If any test fails, it holds its horses and sends me an email.
EDIT:
I also want to give ten thumbs up to the ChuckNorris plugin for Hudson. Agile doesn't get better than pair programming with Walker, Texas Ranger.
Ruby has Test::Unit built-in, RSpec, ZenTest, shoulda, cucumber and probably many more tools to help test. Being built-in, Test::Unit is used a lot and is the target to be beaten by the other tools.
ZenTest and RSpec can do continuous testing: You make a change and save a file and they'll see it and run the test suite. I like that because then I know the state of things right away.
I haven't used cucumber, but have used the rest. I heard cucumber's emphasis is on integration testing but that might have been the commenter's feelings rather than by design of the developers. The list of tutorials for cucumber is interesting browsing. In particular there's webrat: Automated Acceptance Testing with RSpec or Cucumber.
Any of these could be wired up with cron to run periodically; Just treat them as you would any other set of command-line apps.
It should be easy to tie in web-testing too, but you'll have to identify the gems/modules needed and write the glue code. I haven't had need for such a beast as I'd go at it using Mechanize and/or one of the other HTTP gems plus Nokogiri to ransack the pages.
Is there a facility similar to SeleniumGrid that I can use to run webrat (or other, similar framework) browser automation tests in parallel across a farm of coordinated agents?
Coordinated via TeamCity with rake?
Edit: We're looking at using cucumber+webrat to do functional and acceptance testing as described in Testing ASP.NET Web Applications
I've worked on just this project actually. If you're working on rails, check out http://github.com/sgrove/spec_storm . It's only setup to run rspec + selenium tests in parallel, but it can be extended to others depending on the demand. And of course if you have any questions, I'm more than happy to help out. The more people using it, the happier I am :D