I'm using RSpec to test my site. I have a method that generates random email addresses for users to sign up with:
def random_email
alphabet = [('a'..'z'),('A'..'Z')].map{|i| i.to_a}.flatten
(0...15).map{ alphabet[rand(alphabet.length)] }.join << "#example.com"
end
This method is called multiple times throughout the test suite. Many times, it ends up generating either A) multiple identical email addresses (actual example: tqCXuHCfITEUJBh#example.com), or B) email addresses that add a few new characters to the beginning of a previously used email address, and chop a few off the end (actual example: tqCXuHCfITEUJBh#example.com and vtqCXuHCfITEUJB#example.com).
I found that pseudo random generators work in a sequence, such that when the randomness is re-seeded with the same number, it will do the same sequence over again. I can't help wondering if RSpec is re-seeding Ruby's randomness between tests. (Perhaps it's worth noting that the tests that reuse email addresses often occur in different RSpec test files; maybe RSpec is re-seeding in between files?)
Any ideas?
Even if RSpec isn't re-seeding the RNG, something else may, and it'll break your specs. You're making your specs fragile by tying their behavior to some greater system state.
Often unique email address are just generated sequentially, either using something like FactoryGirl or just in a helper method. If you run out of those, congratulate yourself on writing the largest test suite in history.
Related
I have a method that builds a laptop's attributes, but only if the attributes are present within a row that is given to the method:
def build_laptop_attributes desk_id, row, laptop
attributes = {}
attributes[:desk_number] = room_id if laptop && desk_id
attributes[:status] = row[:state].downcase if row[:state]
attributes[:ip_address] = row[:ip_address] if row[:ip_address]
attributes[:model] = row[:model] if row[:model]
attributes
end
Currently, RuboCop is saying that the Metric/AbcSize is too high, and I was wondering if there is an obvious and clean way to assign these attributes?
Style Guides Provide "Best Practices"; Evaluate and Tune When Needed
First of all, RuboCop is advisory. Just because RuboCop complains about something doesn't mean it's wrong in some absolute sense; it just means you ought to expend a little more skull sweat (as you're doing) to see if what you're doing makes sense.
Secondly, you haven't provided a self-contained, executable example. That makes it impossible for SO readers to reliably refactor it, since it can't currently be tested without sample inputs and expected outputs not provided in your original post. You'll need those things yourself to evaluate and refactor your own code, too.
Finally, the ABC Metric looks at assignments, branches, and conditionals. You have five assignments, four conditionals, and what looks liks a method call. Is that a lot? If you haven't tuned Rubocop, the answer is "RuboCop thinks so." Whether or not you agree is up to you and your team.
If you want to try feeding Rubocop, you can do a couple of things that might help reduce the metric:
Refactor the volume and complexity of your assignments. Some possible examples include:
Replace your postfix if-statements with safe navigators (&.) to guard against calling methods on nil.
Extract some of your branching logic and conditionals to methods that "do the right thing", potentially reducing your current method to a single assignment with four method calls. For example:
attributes = { desk_number: location, status: laptop_status, ... }
Replace all your multiple assignments with a deconstructing assignment (although Rubocop often complains about those, too).
Revisit whether you have the right data structure in the first place. Maybe you really just want an OpenStruct, or some other data object.
Your current code seems readable, so is the juice really worth the squeeze? If you decide that RuboCop is misguided in this particular case, and your code works and passes muster in your internal code reviews, then you can tune the metric's sensitivity in your project's .rubocop.yml or disable that particular metric for just that section of your source code.
After reading #Todd A. Jacobs answer, you may want (or not) to write something like this:
def build_laptop_attributes desk_id, row, laptop
desk_number = room_id if laptop && desk_id
{
desk_number: desk_number,
status: row[:state]&.downcase,
ip_address: = row[:ip_address],
model: row[:model]
}.compact
end
This reduces has the advantage of reducing the number of calls to []=, as well as factorizing many ifs in a single compact.
In my opinion, it is more readable because it is more concise and because the emphasis is completely on the correspondence between your keys and values.
Alternative version to reduce the amount of conditionals (assuming you are checking for nil / initialized values):
def build_laptop_attributes desk_id, row, laptop
attributes = {}
attributes[:desk_number] = room_id if laptop && desk_id
attributes[:status] = row[:state]&.downcase
attributes[:ip_address] = row[:ip_address]
attributes[:model] = row[:model]
attributes.compact
end
There is an additional .compact as a cost of removing assignments checks.
I'm attempting to perform a mutation-test on some ruby code using rspec. I am just learning ruby and I don't really know if I'm doing it right. Part of the code I'm trying to test is:
class Cipher
def initialize()
#a_offset = 65 #'A'.unpack('C').first
#z_offset = 90 #'Z'.unpack('C').first
end
def encode(key, plain_text)
key_offset = key.upcase.unpack('C').first
cipher_text = plain_text.upcase.split('').collect do |letter|
cipher_letter = (letter.unpack('C').first + key_offset - #a_offset)
if cipher_letter > #z_offset
cipher_letter -= ( #z_offset - #a_offset + 1 )
end
cipher_letter.chr
end
return cipher_text.join
end
So far my test suite looks like this:
require 'rspec'
require 'Cipher'
describe "#initialize" do
it "should have correct #a_offset" do
encoder = Cipher.new()
expect(encoder.instance_variable_get(:#a_offset)).to eq 65
end
it "should have correct #z_offset" do
encoder = Cipher.new()
expect(encoder.instance_variable_get(:#z_offset)).to eq 90
end
end
describe "#encode" do
it "It should correctly encode Caesar with key = A"do
encoder = Cipher.new()
expect(encoder.encode('R', 'CAESAR')).to eq ("TRVJRI")
end
end
When running rspec my 3 tests pass. However when I use mutation testing on this suite I only kill 3/343 which is not very good.
Since your goal is to develop a mutation-adequate test suite, the unkilled mutants essentially provide guidance for what to test for. Ideally, your question should have provided some examples for generated mutants that are not killed by your test suite so that others can comment on how to write tests that kill them.
Let me try to best answer your question with general recommendations and examples.
I am not familiar with the particular mutation testing tool you are using, but in general you should follow a process similar to the following*:
Select a mutant that is not killed by your test suite and determine whether it can be killed (a mutant might be semantically equivalent to your program -- if so, discard it from the pool of mutants).
Write a test that kills the selected mutant -- that is, write a test that passes on your original program but fails on the mutant.
Determine suitable inputs that trigger the faulty behavior, introduced by the mutant.
Determine a suitable test oracle that asserts on the expected behavior/output of your program (i.e., a test oracle that passes on your original program but fails on the mutant).
Please keep in mind that your goal should not be to write minimal tests that simply kill all mutants, but rather tests that make sense in the context of your project and as a byproduct kill the mutants. Again, if you could provide some concrete examples for mutants that your test suite does not kill, I and others can better comment on what tests are missing.
Example: Suppose you have a mutant that introduces the following fault (i.e., changes the relational operator in the if statement of your original program):
- if cipher_letter > #z_offset
+ if cipher_letter >= #z_offset
Killing this mutant requires a test that checks for the boundary condition -- that is, a test that encodes at least one character to a 'Z'. Your current test suite does not have such a test.
*This process describes traditional mutation testing. Recent research [1], [2] suggests that (1) not all killable mutants should be killed (some mutants elicit tests that simply don't make sense) and (2) equivalent mutants may indicate a problem in your program (e.g., unwanted redundancy or ambiguity) that you might want to fix instead of discarding the equivalent mutant.
It seems to be very difficult to look up documentation about Gherkin, so I was wondering if there was a way to augment step definitions to enable the tester to use proper grammar. One example that shows what I mean is:
...Testing...
Then I see there is 1 item
...More testing...
Then I see there are 2 items
Obviously, these two steps would use the same code. I defined a step definition like this which almost works:
Then(/^I see there (is|are) (\d+) item(s)?$/) do |item_count|
...code...
end
Except the problem is that it interprets is/are and the optional plural s as arguments. Is there any way to signal to Gherkin that these are just for allowing proper grammar?
Use ?: at the start of the group marks it as noncapturing, Cucumber won’t pass it as an argument.
/^I see there (?:is|are) (\d+) item(?:s)?$/
These steps don't have to use the same code. Instead they can call the same code. If you apply this pattern you can then concentrate on your steps doing just the single thing they should be doing which is using well expressed natural language to fire code. So ...
module ItemStepHelper
def see_items(count:)
...
end
World ItemStepHelper
Then 'I see there is one item' do
see_items(count: 1)
end
Then 'I see there are \d+ items' do |count|
see_items(count: count)
end
Now obviously with this example thats quite a bit more boilerplate for very little benefit, but when you apply the pattern on more realistic examples then the benefits really begin to kick in. In particular you never have to write a really complex regex for step definitions (in practice 90% or more of my step defs don't even use a regex).
I recently asked how to test in RSpec if a block was called and the answers to that question seem to work in a simple case. The problem is when the initialization with the block is more complex. Then it is done in before and reused by a number of different tests in the context, among them the one testing if the block was evaluated. See the example:
context "the node definition using block of code" do
before do
#n=node do
# this block should be called
end
# some more complex setup concerning #n
end
it "should call the block" do
# how to test it?
end
# here a bunch of other tests using #n
end
In this case the solution with side effect changing value of a local variable does not work. Raising an exception from the block is useless since the whole statement must be properly evaluated to be used by the other tests.
Obviously I could do the tests separately, but it seems to stink, since I must copy-paste the initialization part and since the was-the-block-called test inherently belongs to this very context.
How to test if the block was evaluated in such a case?
Explanation for question asked by #zetetic below.
The context is that I'm implementing a kind of DSL, with nodes defined by their parameters and blocks of code (that can define something else in the scope of node). Since the things defined by the node's block can be pretty generic, at least for the first attempt I just need to be sure the block is evaluated and that what a user provides there will be considered. For now does not matter what it is.
Probably I should refactor my tests now and using mocks make them test behaviors rather then implementation. However it will be a little bit tricky, for the sake of some mixins and dynamic handling of messages sent to objects. For now the cincept of such tests is a little bit fuzzy in my head ;-)
Anyway your answers and comments helped me to better understand how RSpec works and explained why what I'm trying to do looks as if it did not fit to the RSpec.
Try something like this (untested by me):
context "the node definition using block of code" do
let(:node){
node = Node.new "arg1", "arg2", node_block
# more complex stuff here
node
}
context "checking the block is called" do
let(:node_block) {
double = double("node_block")
double.should_receive("some kind of arg").and_return("something")
# this will now cause a fail if it isn't called
double
}
it "should call the block" do
node.blah()
end
end
let(:node_block) {
# some real code
}
subject { node.blah() }
it { should == 2 }
# ...
end
So that's a very shaky piece of code (you'll have to fill in the gaps as you didn't give very much to go on, and let is obviously a lambda too, which could mean you've got to play around with it a bit) that uses let and a double to check it's called, and avoids using before, which is really for side effects not setting up variables for use in the specs.
#zetetic makes a very insightful comment that you're not testing behaviour here. I'm not against using rspec for doing more unit test style stuff (guidelines are made to be broken), but you might ask how later tests will pass when using a real block of code if that block isn't being called? In a way, I'm not even sure you need to check the block is called, but only you know.
I've accidentally stumbled upon an old article by Luke Redpath that presents a very simple BDD example (very short and easy to follow even for non-Ruby programmers like me). I found the end result very incomplete, thus making the example pretty useless.
The end result is a single test which verifies that a user with preset attributes is valid. In my view, this is simply not enough to verify the validation rules correctly. For example, if you change
validates_length_of :password, :in => 6..12, :allow_nil => :true
to
validates_length_of :password, :in => 7..8, :allow_nil => :true
(or even remove password length validation completely) the test will still pass, but you can obviously see the code is now violating the initial requirements.
I just think the last refactoring of putting all the individual tests into a single one is simply not enough. He tests only the "happy path" which doesn't guarantee much. I would absolutely have all the tests that verify that the correct error is triggered with certain values. In the case of the password, I would test that a password of length less than 6 and greater than 12 is invalid and triggers the appropriate error. The "happy path" test would be there as well, but not alone by itself as it's in the article.
What's your opinion? I'm just trying to figure out why the guy did it the way he did and whether he simply overlooked the problem or it was his intention. I may be missing something.
I don't quite understand your question. The specs do contain expectations about the password lenght, both for the happy path and the two different failure modes (password too long and password too short):
specify "should be valid with a full set of valid attributes" do
#user.attributes = valid_user_attributes
#user.should_be_valid
end
This takes care of the happy path, since valid_user_attributes contains a valid password.
specify "should be invalid if password is not between 6 and 12 characters in length" do
#user.attributes = valid_user_attributes.except(:password)
#user.password = 'abcdefghijklm'
#user.should_not_be_valid
#user.password = 'abcde'
#user.should_not_be_valid
end
And this tests the two failure modes.
Granted, there is one boundary case missing (12 characters), but that's not too bad.
I don't have time to read the article, so I can't verify your claims, but the general answer in my opinion is that if the password validation rule is a concrete requirement, it should be verified with one or more tests for that specific requirement (at least one per "part" of the requirement).
BDD (and TDD) are design activities. The tests are meant to drive the design of the code, not guarantee that it is completely bug-free. There should be independent testers for that. So we need a decent degree of coverage, to ensure that our code works as expected and handles exceptions in a clean fashion. But TDD doesn't demand that we write unit tests for every conceivable edge case.
With regard to the specific example you cite, perhaps he should have coded two tests, one with a password of six characters, one with a passowrd of twelve characters. But what would be the point? We know that the requirement is the password must be between six and twelve characters in length. If we have misunderstood the requirements and think the rule ought to be ...
validates_length_of :password, :in => 7..8, :allow_nil => :true
... then we're going to write our test data to make a test which passes our incorrect interpretation. So writing more tests would only give us misplaced confidence. That's why proponents of TDD and BDD favour other XP techniques like pair programming as well: to catch the errors we introduce into our unit tests.
Similarly, we could remove the test validating the password length altogether, but what would be the point? The tests are there to help us correctly implement the spceification. If we don't have tests for every piece of code we write then we are not doing TDD/BDD.