A BDD example - why test "happy path" only? - tdd

I've accidentally stumbled upon an old article by Luke Redpath that presents a very simple BDD example (very short and easy to follow even for non-Ruby programmers like me). I found the end result very incomplete, thus making the example pretty useless.
The end result is a single test which verifies that a user with preset attributes is valid. In my view, this is simply not enough to verify the validation rules correctly. For example, if you change
validates_length_of :password, :in => 6..12, :allow_nil => :true
to
validates_length_of :password, :in => 7..8, :allow_nil => :true
(or even remove password length validation completely) the test will still pass, but you can obviously see the code is now violating the initial requirements.
I just think the last refactoring of putting all the individual tests into a single one is simply not enough. He tests only the "happy path" which doesn't guarantee much. I would absolutely have all the tests that verify that the correct error is triggered with certain values. In the case of the password, I would test that a password of length less than 6 and greater than 12 is invalid and triggers the appropriate error. The "happy path" test would be there as well, but not alone by itself as it's in the article.
What's your opinion? I'm just trying to figure out why the guy did it the way he did and whether he simply overlooked the problem or it was his intention. I may be missing something.

I don't quite understand your question. The specs do contain expectations about the password lenght, both for the happy path and the two different failure modes (password too long and password too short):
specify "should be valid with a full set of valid attributes" do
#user.attributes = valid_user_attributes
#user.should_be_valid
end
This takes care of the happy path, since valid_user_attributes contains a valid password.
specify "should be invalid if password is not between 6 and 12 characters in length" do
#user.attributes = valid_user_attributes.except(:password)
#user.password = 'abcdefghijklm'
#user.should_not_be_valid
#user.password = 'abcde'
#user.should_not_be_valid
end
And this tests the two failure modes.
Granted, there is one boundary case missing (12 characters), but that's not too bad.

I don't have time to read the article, so I can't verify your claims, but the general answer in my opinion is that if the password validation rule is a concrete requirement, it should be verified with one or more tests for that specific requirement (at least one per "part" of the requirement).

BDD (and TDD) are design activities. The tests are meant to drive the design of the code, not guarantee that it is completely bug-free. There should be independent testers for that. So we need a decent degree of coverage, to ensure that our code works as expected and handles exceptions in a clean fashion. But TDD doesn't demand that we write unit tests for every conceivable edge case.
With regard to the specific example you cite, perhaps he should have coded two tests, one with a password of six characters, one with a passowrd of twelve characters. But what would be the point? We know that the requirement is the password must be between six and twelve characters in length. If we have misunderstood the requirements and think the rule ought to be ...
validates_length_of :password, :in => 7..8, :allow_nil => :true
... then we're going to write our test data to make a test which passes our incorrect interpretation. So writing more tests would only give us misplaced confidence. That's why proponents of TDD and BDD favour other XP techniques like pair programming as well: to catch the errors we introduce into our unit tests.
Similarly, we could remove the test validating the password length altogether, but what would be the point? The tests are there to help us correctly implement the spceification. If we don't have tests for every piece of code we write then we are not doing TDD/BDD.

Related

Is there a better way to assign a Ruby hash while avoiding RuboCop's ABC Size warnings?

I have a method that builds a laptop's attributes, but only if the attributes are present within a row that is given to the method:
def build_laptop_attributes desk_id, row, laptop
attributes = {}
attributes[:desk_number] = room_id if laptop && desk_id
attributes[:status] = row[:state].downcase if row[:state]
attributes[:ip_address] = row[:ip_address] if row[:ip_address]
attributes[:model] = row[:model] if row[:model]
attributes
end
Currently, RuboCop is saying that the Metric/AbcSize is too high, and I was wondering if there is an obvious and clean way to assign these attributes?
Style Guides Provide "Best Practices"; Evaluate and Tune When Needed
First of all, RuboCop is advisory. Just because RuboCop complains about something doesn't mean it's wrong in some absolute sense; it just means you ought to expend a little more skull sweat (as you're doing) to see if what you're doing makes sense.
Secondly, you haven't provided a self-contained, executable example. That makes it impossible for SO readers to reliably refactor it, since it can't currently be tested without sample inputs and expected outputs not provided in your original post. You'll need those things yourself to evaluate and refactor your own code, too.
Finally, the ABC Metric looks at assignments, branches, and conditionals. You have five assignments, four conditionals, and what looks liks a method call. Is that a lot? If you haven't tuned Rubocop, the answer is "RuboCop thinks so." Whether or not you agree is up to you and your team.
If you want to try feeding Rubocop, you can do a couple of things that might help reduce the metric:
Refactor the volume and complexity of your assignments. Some possible examples include:
Replace your postfix if-statements with safe navigators (&.) to guard against calling methods on nil.
Extract some of your branching logic and conditionals to methods that "do the right thing", potentially reducing your current method to a single assignment with four method calls. For example:
attributes = { desk_number: location, status: laptop_status, ... }
Replace all your multiple assignments with a deconstructing assignment (although Rubocop often complains about those, too).
Revisit whether you have the right data structure in the first place. Maybe you really just want an OpenStruct, or some other data object.
Your current code seems readable, so is the juice really worth the squeeze? If you decide that RuboCop is misguided in this particular case, and your code works and passes muster in your internal code reviews, then you can tune the metric's sensitivity in your project's .rubocop.yml or disable that particular metric for just that section of your source code.
After reading #Todd A. Jacobs answer, you may want (or not) to write something like this:
def build_laptop_attributes desk_id, row, laptop
desk_number = room_id if laptop && desk_id
{
desk_number: desk_number,
status: row[:state]&.downcase,
ip_address: = row[:ip_address],
model: row[:model]
}.compact
end
This reduces has the advantage of reducing the number of calls to []=, as well as factorizing many ifs in a single compact.
In my opinion, it is more readable because it is more concise and because the emphasis is completely on the correspondence between your keys and values.
Alternative version to reduce the amount of conditionals (assuming you are checking for nil / initialized values):
def build_laptop_attributes desk_id, row, laptop
attributes = {}
attributes[:desk_number] = room_id if laptop && desk_id
attributes[:status] = row[:state]&.downcase
attributes[:ip_address] = row[:ip_address]
attributes[:model] = row[:model]
attributes.compact
end
There is an additional .compact as a cost of removing assignments checks.

Using proper grammar in Gherkin

It seems to be very difficult to look up documentation about Gherkin, so I was wondering if there was a way to augment step definitions to enable the tester to use proper grammar. One example that shows what I mean is:
...Testing...
Then I see there is 1 item
...More testing...
Then I see there are 2 items
Obviously, these two steps would use the same code. I defined a step definition like this which almost works:
Then(/^I see there (is|are) (\d+) item(s)?$/) do |item_count|
...code...
end
Except the problem is that it interprets is/are and the optional plural s as arguments. Is there any way to signal to Gherkin that these are just for allowing proper grammar?
Use ?: at the start of the group marks it as noncapturing, Cucumber won’t pass it as an argument.
/^I see there (?:is|are) (\d+) item(?:s)?$/
These steps don't have to use the same code. Instead they can call the same code. If you apply this pattern you can then concentrate on your steps doing just the single thing they should be doing which is using well expressed natural language to fire code. So ...
module ItemStepHelper
def see_items(count:)
...
end
World ItemStepHelper
Then 'I see there is one item' do
see_items(count: 1)
end
Then 'I see there are \d+ items' do |count|
see_items(count: count)
end
Now obviously with this example thats quite a bit more boilerplate for very little benefit, but when you apply the pattern on more realistic examples then the benefits really begin to kick in. In particular you never have to write a really complex regex for step definitions (in practice 90% or more of my step defs don't even use a regex).

What Is the correct syntax for writing test definitions in Cucumber?

What is the actual syntax for writing step definitions in Cucumber? I have seen it being written in different ways. Is there no definite syntax? I know the anchors are not compulsory, but is there a basic rule?
I am new to Cucumber and will appreciate baby step information to help me understand the basics. Thanks guys!
I was planning to point you to online documentation, but the online documentation I know about (at cucumber.io and at relishapp.com) doesn't actually answer your question well. (It does contain many examples, though, and is well worth reading.)
In the Ruby implementation of Cucumber, step definition files are .rb files in the features/step_definition directory. They contain a series of calls to methods that each define an implementation of a Gherkin step. Here's an example:
Given /^there is a user named "(.*)"$/ do |username|
# code that creates a user with the given username
end
There are several methods that define steps: Given, When, Then, And and But. They all do exactly the same thing, and you can use any one to define any step. The best practice is to use the one that reads best with the step you're defining (never And or But).
The argument passed to the step-defining method is a regular expression intended to match one or more steps in Gherkin .feature files. The above example matches the following step:
Given there is a user named "Ade Tester"
("Ade Tester" could be anything).
The block passed to the step-defining method is run when Cucumber executes a step which the regular expression matches. It can contain any Ruby code you like.
Matching groups (enclosed in parentheses) in the regular expression are passed to the block as block parameters. The number of matching groups must match the number of block parameters, or you'll get an error. A common convention is to enclose matching groups that match strings in quotes to visually separate them from the fixed part of the step, as I did above, but this is purely convention and you can choose not to do it.
The regexp need not match the entire step by default. If you want a definition to match only the entire step you must enforce that in the regular expression, as I did in the example above with ^ and $. Do that unless you have a good reason not to. This step definition (without $)
Given /^there is a user named "(.*)"/ do |username|
create :user, username: username
end
would match
Given there is a user named "Ade Tester" on weekdays but "Dave Schweisguth" on weekends
which would probably be a bad idea. Worse, if you had definitions for both steps, Cucumber would not be able to tell which definition to use and you'd get an error.
In features/step_definitions/documentation.rb:
When /^I go to the "([^"]+)" documentation$/ do |section|
path_part =
case section
when "Documentation"
"documentation"
else
raise "Unknown documentation section: #{section}"
end
visit "/documentation/#{path_part}/topics"
end
Then /^I should see the "([^"]+) documentation"$/ do |section|
expect(page).to have_css('h2.doctag_title a', text: section)
end
These steps exercise a web application. They are about as simple as they can be while still being practical.
A good step definition should have in its block a single method call e.g.
When "Frank logs in" do
login user: #frank
end
Bad step definitions have lots of code in their block e.g
When "I login as Frank" do
visit root_path
fill_in login_email, with:
# lots of other stuff about HOW to login
...
end
Other bad step definitions use really complicated regex's with lots of arguments.
Terrible step definitions call other step definitions and do the things bad step definitions do.

Step definitions for cucumber data tables for storing correct answers

I am trying to find a way to test multiple choice questions. The structure is that a lesson has 8 stages and each stage contains multiple choice questions with only one correct answer. The loading of the questions is random each time so i have been looking for the best way to test whether the correct answer is clicked. For this reason i have created a data table with two columns which is obviously more extensive than this but is along these lines:
| what is the opposite of true | false |
| what comes after tuesday | wednesday |
In the feature test i have written that it is testing a correct answer match. Later I was then hoping to find a way to test that if the question and answer match were not in the data table then it is incorrect. Would someone be able to explain how I would go about doing the test definitions for this?
I have tried to use the rows_hash method but I am getting the following error
undefined method `rows_hash' for -3634850196505698949:Fixnum (NoMethodError)
Given(/^a list of answer\-value pairs$/) do |table|
#question_answer_table = hash
end
When(/^I choose a match$/) do
hash = #question_answer_table
#question_answer_table.rows_hash
return false if hash[question].nil?
return hash[question] == answer
end
I think the rows_hash method will help you.
def question_correct?(cucumber_table, question, answer)
hash = cucumber_table.rows_hash
return false if hash[question].nil?
return hash[question] == answer
end
The code works by converting a two column data table into a hash, where the first column is the key and the second is the value.
Keep in mind that this method requires your data table to be limited to two columns.
You'll find this much easier, if you don't try and do this in Cucumber using data tables. Instead push all that detail down to helper methods that are called by the step definitions. To start doing this you need to write a more abstract feature.
The first thing you need to do is to get the simplest possible lesson to work with. So
Given a simple lesson
When I answer the questions correctly
Then I should see I passed the lesson
This is the scenario you'll use to 'drive' your development.
You can implement these steps really easily by delegating e.g.
Given "a simple lesson" do
#lesson = create_simple_lesson
end
When "I answer the questions correctly" do
answer_questions lesson: #lesson
end
Then "I should see I passed the lesson" do
expect(page).to have_content "You passed"
end
To get this to work you'll have to implement some helper methods
module QuestionaireStepHelper
def create_simple_lesson
...
def answer_questions lesson: nil
...
end
World QuestionaireStepHelper
What this does is move your technical problem into a new domain. Here you have the full power of a programming language to do whatever you want: so you can do things like
create Questions that have answers and know what the correct answer is
ask Questions for thier correct answer, so you can answer them correctly
add questions to a lesson
...
Remember because you are still in the Cucumber::World, you have the full power to
drive your browser
access your database
...
When you have finished this you'll have lots of tools to write scenarios like
Given a simple lesson
When I answer the questions incorrectly
Then I should see I failed the lesson
and so on.

Ruby/RSpec randomization seems not-so-random

I'm using RSpec to test my site. I have a method that generates random email addresses for users to sign up with:
def random_email
alphabet = [('a'..'z'),('A'..'Z')].map{|i| i.to_a}.flatten
(0...15).map{ alphabet[rand(alphabet.length)] }.join << "#example.com"
end
This method is called multiple times throughout the test suite. Many times, it ends up generating either A) multiple identical email addresses (actual example: tqCXuHCfITEUJBh#example.com), or B) email addresses that add a few new characters to the beginning of a previously used email address, and chop a few off the end (actual example: tqCXuHCfITEUJBh#example.com and vtqCXuHCfITEUJB#example.com).
I found that pseudo random generators work in a sequence, such that when the randomness is re-seeded with the same number, it will do the same sequence over again. I can't help wondering if RSpec is re-seeding Ruby's randomness between tests. (Perhaps it's worth noting that the tests that reuse email addresses often occur in different RSpec test files; maybe RSpec is re-seeding in between files?)
Any ideas?
Even if RSpec isn't re-seeding the RNG, something else may, and it'll break your specs. You're making your specs fragile by tying their behavior to some greater system state.
Often unique email address are just generated sequentially, either using something like FactoryGirl or just in a helper method. If you run out of those, congratulate yourself on writing the largest test suite in history.

Resources