I have a method I want to test that is being called with a particular object. However, identifying this object is somewhat complex because I'm not interested in a particular instance of the object, but one that conforms to certain conditions.
For example, my code might be like:
some_complex_object = ComplexObject.generate(params)
my_function(some_complex_object)
And in my test I want to check that
test_complex_object = ComplexObject.generate(test_params)
subject.should_receive(:my_function).with(test_complex_object)
But I definitely know that ComplexObject#== will return false when comparing some_complex_object and test_complex_object because that is desired behavior (elsewhere I rely on standard == behavior and so don't want to overload it for ComplexObject).
I would argue that it's a problem that I find myself needing to write a test such as this, and would prefer to restructure the code so that I don't need to write such a test, but that's unfortunately a much larger task that would require rewriting a lot of existing code and so is a longer term fix that I want to get to but can't do right now within time constraints.
Is there a way with Rspec to be able to do a more complex comparison between arguments in a test? Ideally I'd like something where I could use a block so I can write arbitrary comparison code.
See https://www.relishapp.com/rspec/rspec-mocks/v/2-7/docs/argument-matchers for information on how you can provide a block to do arbitrary analysis of the arguments passed to a method, as in:
expect(subject).to receive(:my_function) do |test_complex_object|
# code setting expectations on test_complex_object
end
You can also define a custom matcher which will let you evaluate objects to see if that satisfy the condition, as described in https://www.relishapp.com/rspec/rspec-expectations/v/2-3/docs/custom-matchers/define-matcher
Related
I am submitting solutions to Ruby puzzles on codewars.com and experimenting with how locked into the testing enviroment I am for one of the challenges.
I can redefine the classes used to test my solution but they are defined by the system after I submit my code. If I freeze these objects, the system cannot write over them but a RunTime error is raised when it tries to.
I'm fairly new to Ruby, so I'm not sure which parts (other than falsiness and truthiness) are impossible to override. Can I use Ruby code to force modification of frozen objects to silently fail instead of terminate the program or is that bound up in untouchable things like the assignment operator or similar?
The real answer here is that if you might want to modify an object later, you shouldn't freeze it. That's inherent in the whole concept of "freezing" an object. But since you asked, note that you can test whether an object is frozen with:
obj.frozen?
So if those pesky RuntimeErrors are getting you down, one solution is to use a guard clause like:
obj.do_something! if !obj.frozen?
If you want to make the guard clauses implicit, you can redefine the "problem" methods using a monkey patch:
class Array
# there are a couple other ways to do this
# read up on Ruby metaprogramming if you want to know
alias :__pop__ :pop
def pop
frozen? ? nil : __pop__
end
end
If you want your code to work seamlessly with any and all Ruby libraries/gems, adding behavior to built-in methods like this is probably a bad idea. In this case, I doubt it will cause any problems, but whenever you choose to start hacking on Ruby's core classes, you have to be ready for the possible consequences.
I have a function which receives a complex argument (an HTML string). I want to check multiple conditions about this string, i.e.:
receiver.should_receive(:post_data).with(json_content).with(id_matching(5))
Multiple with arguments doesn't work, any alternatives? I'm happy to define custom matchers if it's possible to make a compound one in some way.
Obviously I could run the same test multiple times and test different things about the result, however this is an integration test which takes several seconds to run, so I don't want to make it even slower.
Thanks
EDIT:
At time of writing, the accepted answer (use a custom matcher with custom description), appears to be the best option. However it isn't perfect, ideally with would support a concept of 'this was an item of the expected type, but wasn't the one we expected', instead of a pure binary match.
Maybe you don't even need a custom matcher and the block form is sufficient for you.
receiver.should_receive(:post_data) do |*args|
json_content = args.first
json_content.should_not be_empty
json_content.should include "some string"
end
See RSpec Mocks documentation, section Arbitrary Handling
You need to provide a custom matcher, but you can readily define your error reporting so that you can give specifics about what failed and why. See https://github.com/dchelimsky/rspec/wiki/Custom-Matchers .
In particular, the custom matcher would be supplied as the argument to with, as mentioned in the last sentence of the first paragraph of the "Argument Matchers" section of https://github.com/rspec/rspec-mocks.
As for error reporting, there are no custom failure methods that apply to this use case, but the description method of the custom matcher is used generate the string shown as the "expected" value and, though not its purpose, can be defined to output anything you want regarding the failed match.
I tend to use before blocks to set instance variables. I then use those variables across my examples. I recently came upon let(). According to RSpec docs, it is used to
... to define a memoized helper method. The value will be cached across multiple calls in the same example but not across examples.
How is this different from using instance variables in before blocks? And also when should you use let() vs before()?
I always prefer let to an instance variable for a couple of reasons:
Instance variables spring into existence when referenced. This means that if you fat finger the spelling of the instance variable, a new one will be created and initialized to nil, which can lead to subtle bugs and false positives. Since let creates a method, you'll get a NameError when you misspell it, which I find preferable. It makes it easier to refactor specs, too.
A before(:each) hook will run before each example, even if the example doesn't use any of the instance variables defined in the hook. This isn't usually a big deal, but if the setup of the instance variable takes a long time, then you're wasting cycles. For the method defined by let, the initialization code only runs if the example calls it.
You can refactor from a local variable in an example directly into a let without changing the
referencing syntax in the example. If you refactor to an instance variable, you have to change
how you reference the object in the example (e.g. add an #).
This is a bit subjective, but as Mike Lewis pointed out, I think it makes the spec easier to read. I like the organization of defining all my dependent objects with let and keeping my it block nice and short.
A related link can be found here: http://www.betterspecs.org/#let
The difference between using instances variables and let() is that let() is lazy-evaluated. This means that let() is not evaluated until the method that it defines is run for the first time.
The difference between before and let is that let() gives you a nice way of defining a group of variables in a 'cascading' style. By doing this, the spec looks a little better by simplifying the code.
I have completely replaced all uses of instance variables in my rspec tests to use let(). I've written a quickie example for a friend who used it to teach a small Rspec class: http://ruby-lambda.blogspot.com/2011/02/agile-rspec-with-let.html
As some of the other answers here says, let() is lazy evaluated so it will only load the ones that require loading. It DRYs up the spec and make it more readable. I've in fact ported the Rspec let() code to use in my controllers, in the style of inherited_resource gem. http://ruby-lambda.blogspot.com/2010/06/stealing-let-from-rspec.html
Along with lazy evaluation, the other advantage is that, combined with ActiveSupport::Concern, and the load-everything-in spec/support/ behavior, you can create your very own spec mini-DSL specific to your application. I've written ones for testing against Rack and RESTful resources.
The strategy I use is Factory-everything (via Machinist+Forgery/Faker). However, it is possible to use it in combination with before(:each) blocks to preload factories for an entire set of example groups, allowing the specs to run faster: http://makandra.com/notes/770-taking-advantage-of-rspec-s-let-in-before-blocks
It is important to keep in mind that let is lazy evaluated and not putting side-effect methods in it otherwise you would not be able to change from let to before(:each) easily.
You can use let! instead of let so that it is evaluated before each scenario.
In general, let() is a nicer syntax, and it saves you typing #name symbols all over the place. But, caveat emptor! I have found let() also introduces subtle bugs (or at least head scratching) because the variable doesn't really exist until you try to use it... Tell tale sign: if adding a puts after the let() to see that the variable is correct allows a spec to pass, but without the puts the spec fails -- you have found this subtlety.
I have also found that let() doesn't seem to cache in all circumstances! I wrote it up in my blog: http://technicaldebt.com/?p=1242
Maybe it is just me?
Dissenting voice here: after 5 years of rspec I don't like let very much.
1. Lazy evaluation often makes test setup confusing
It becomes difficult to reason about setup when some things that have been declared in setup are not actually affecting state, while others are.
Eventually, out of frustration someone just changes let to let! (same thing without lazy evaluation) in order to get their spec working. If this works out for them, a new habit is born: when a new spec is added to an older suite and it doesn't work, the first thing the writer tries is to add bangs to random let calls.
Pretty soon all the performance benefits are gone.
2. Special syntax is unusual to non-rspec users
I would rather teach Ruby to my team than the tricks of rspec. Instance variables or method calls are useful everywhere in this project and others, let syntax will only be useful in rspec.
3. The "benefits" allow us to easily ignore good design changes
let() is good for expensive dependencies that we don't want to create over and over.
It also pairs well with subject, allowing you to dry up repeated calls to multi-argument methods
Expensive dependencies repeated in many times, and methods with big signatures are both points where we could make the code better:
maybe I can introduce a new abstraction that isolates a dependency from the rest of my code (which would mean fewer tests need it)
maybe the code under test is doing too much
maybe I need to inject smarter objects instead of a long list of primitives
maybe I have a violation of tell-don't-ask
maybe the expensive code can be made faster (rarer - beware of premature optimisation here)
In all these cases, I can address the symptom of difficult tests with a soothing balm of rspec magic, or I can try address the cause. I feel like I spent way too much of the last few years on the former and now I want some better code.
To answer the original question: I would prefer not to, but I do still use let. I mostly use it to fit in with the style of the rest of the team (it seems like most Rails programmers in the world are now deep into their rspec magic so that is very often). Sometimes I use it when I'm adding a test to some code that I don't have control of, or don't have time to refactor to a better abstraction: i.e. when the only option is the painkiller.
let is functional as its essentially a Proc. Also its cached.
One gotcha I found right away with let... In a Spec block that is evaluating a change.
let(:object) {FactoryGirl.create :object}
expect {
post :destroy, id: review.id
}.to change(Object, :count).by(-1)
You'll need to be sure to call let outside of your expect block. i.e. you're calling FactoryGirl.create in your let block. I usually do this by verifying the object is persisted.
object.persisted?.should eq true
Otherwise when the let block is called the first time a change in the database will actually happen due to the lazy instantiation.
Update
Just adding a note. Be careful playing code golf or in this case rspec golf with this answer.
In this case, I just have to call some method to which the object responds. So I invoke the _.persisted?_ method on the object as its truthy. All I'm trying to do is instantiate the object. You could call empty? or nil? too. The point isn't the test but bringing the object ot life by calling it.
So you can't refactor
object.persisted?.should eq true
to be
object.should be_persisted
as the object hasn't been instantiated... its lazy. :)
Update 2
leverage the let! syntax for instant object creation, which should avoid this issue altogether. Note though it will defeat a lot of the purpose of the laziness of the non banged let.
Also in some instances you might actually want to leverage the subject syntax instead of let as it may give you additional options.
subject(:object) {FactoryGirl.create :object}
"before" by default implies before(:each). Ref The Rspec Book, copyright 2010, page 228.
before(scope = :each, options={}, &block)
I use before(:each) to seed some data for each example group without having to call the let method to create the data in the "it" block. Less code in the "it" block in this case.
I use let if I want some data in some examples but not others.
Both before and let are great for DRYing up the "it" blocks.
To avoid any confusion, "let" is not the same as before(:all). "Let" re-evaluates its method and value for each example ("it"), but caches the value across multiple calls in the same example. You can read more about it here: https://www.relishapp.com/rspec/rspec-core/v/2-6/docs/helper-methods/let-and-let
Note to Joseph -- if you are creating database objects in a before(:all) they won't be captured in a transaction and you're much more likely to leave cruft in your test database. Use before(:each) instead.
The other reason to use let and its lazy evaluation is so you can take a complicated object and test individual pieces by overriding lets in contexts, as in this very contrived example:
context "foo" do
let(:params) do
{ :foo => foo, :bar => "bar" }
end
let(:foo) { "foo" }
it "is set to foo" do
params[:foo].should eq("foo")
end
context "when foo is bar" do
let(:foo) { "bar" }
# NOTE we didn't have to redefine params entirely!
it "is set to bar" do
params[:foo].should eq("bar")
end
end
end
I use let to test my HTTP 404 responses in my API specs using contexts.
To create the resource, I use let!. But to store the resource identifier, I use let. Take a look how it looks like:
let!(:country) { create(:country) }
let(:country_id) { country.id }
before { get "api/countries/#{country_id}" }
it 'responds with HTTP 200' { should respond_with(200) }
context 'when the country does not exist' do
let(:country_id) { -1 }
it 'responds with HTTP 404' { should respond_with(404) }
end
That keeps the specs clean and readable.
I wonder if one should validate that the arguments passed to a method is of a certain class.
eg.
def type(hash = {}, array = [])
# validate before
raise "first argument needs to be a hash" unless hash.class == Hash
raise "second argument needs to be an array" unless array.class == Array
# actual code
end
Is it smart to do this or is it just cumbersome and waste of time to validate all passed in arguments?
Are there circumstances when you would like to have this extra security and circumstances when you won't bother?
Share your experiences!
I wouldn't recommend this specific approach, as you fail to accommodate classes that provide hash or array semantics but are not that class. If you need this kind of validation, you're better off using respond_to? with a method name. Arrays implement the method :[], for what it's worth.
OpenStruct has hash semantics and attribute-accessor method semantics, but won't return true for the condition hash.class==Hash. It'll work just like a hash in practice, though.
To put it in perspective, even in a non-dynamic language you wouldn't want to do it this way; you'd prefer to verify that an object implements IDictionary<T>. Ruby would idiomatically prefer that, when necessary, you verify that the method exists, because if it does, the developer is probably intending their object to act alike. You can provide additional sanity with unit tests around the client code as an alternative to forcing things to be non-dynamic.
There's usually no need to validate that arguments are of a certain class. In Ruby, you're encouraged to use Duck Typing.
I have found that validating that the input parameters meet your preconditions is a very valuable practice. The stupid person that its saves you from is you. This is especially true for Ruby as it has no compile time checks. If there are some characteristics of the input of your method that you know must be true it makes senes to check them at run time and raise errors with meaningful error messages. Otherwise, the code just starts producing garbage out for garbage in and you either get a wrong answer or an exception down the line.
I think that it is unnecessary. I once read on a blog something like "If you need to protect your code from stupid people, ruby isn't the language for you."
If you want compiler/runtime-enforced code contracts, then Ruby isn't for you. If you want a malleable language and easy testability, then Ruby is.
I've run into this dilemma several times. Should my unit-tests duplicate the functionality of the method they are testing to verify its integrity? OR Should unit tests strive to test the method with numerous manually created instances of inputs and expected outputs?
I'm mainly asking the question for situations where the method you are testing is reasonably simple and its proper operation can be verified by glancing at the code for a minute.
Simplified example (in ruby):
def concat_strings(str1, str2)
return str1 + " AND " + str2
end
Simplified functionality-replicating test for the above method:
def test_concat_strings
10.times do
str1 = random_string_generator
str2 = random_string_generator
assert_equal (str1 + " AND " + str2), concat_strings(str1, str2)
end
end
I understand that most times the method you are testing won't be simple enough to justify doing it this way. But my question remains; is this a valid methodology in some circumstances (why or why not)?
Testing the functionality by using the same implementation, doesn't test anything. If one has a bug in it, the other will as well.
But testing by comparing with an alternative implementation is a valid approach. For example, you might test a iterative (fast) method of calculating fibonacci numbers by comparing it with a trivial recursive, yet slow implementation of the same method.
A variation of this is using an implementation, that only works for special cases. Of course in that case you can use it only for such special cases.
When choosing input values, using random values most of the time isn't very effective. I'd prefer carefully chosen values anytime. In the example you gave, null values and extremely long values which won't fit into a String when concatenated come to mind.
If you use random values, make sure, you have a way to recreate the exact run, with the same random values, for example by logging the seed value, and having a way to set that value at start time.
It's a controversial stance, but I believe that unit testing using Derived Values is far superior to using arbitrary hard-coded input and output.
The issue is that as an algorithm becomes even slightly complex, the relationship between input and output becomes obscure if represented by hard-coded values. The unit test ends up being a postulate. It may work technically, but hurts test maintenability because it leads to Obscure Tests.
Using Derived Values to test against the result establishes a much clearer relationship between test input and expected output.
The argument that this doesn't test anything is simply not true, because any test case will exercise only a part of a path through the SUT, so no single test case will reproduce the entire algorithm being tested, but the combination of tests will do so.
An additional benefit is that you can use fewer unit tests to cover the desired functionality, and even make them more communicative at the same time. The end result is terser and more maintainable unit tests.
In unit testing you should definitely manually come up with test cases (so input, output and what side-effects you are expecting - these will be expectations on your mock objects). You come up with these test cases in a way so that they cover all functionality of your class (e.g. all methods are covered, all branches of all if statements, etc.). Think about it more along the lines of creating documentation of your class by showing all possible usages.
Reimplementing the class is not a good idea, because not only you get obvious code/functionality duplication, but also it is likely that you will introduce the same bugs in this new implementation.
to test the functionality of a method i'd use input and output pairs wherever possible. otherwise you might be copy&pasting the functionality as well as the errors in its implementation. what are you testing then? you would be testing if the functionality (including all of its errors) hasn't changed over time. but you wouldn't be testing the correctness of the implementation.
testing if the functionality hasn't changed over time might (temporarily) be useful during refactoring. but how often do you refactor such small methods?
also unit tests can be seen as documentation and as specification of a method's inputs and expected outputs. both should be as simple as possible so others can easily read and comprehend it. as soon as you introduce additional code/logic into a test it becomes harder to read.
your test actually looks like a fuzz test. fuzz tests can be very useful, but in unit tests randomness should be avoided due to reproducibility.
A Unit-Test should exercise your code, not something as part of the language you are using.
If the code's logic is to concatenate strings in a special way, you should be testing for that - otherwise you need to rely on your language/framework.
Finally, you should create your unit tests to fail first "with meaning". In other words, random values shouldn't be used (unless you're testing your random number generator isn't returning the same set of random values!)
Yes. It bothers me too.. although I'd say that it is more prevalent with non-trivial computations. In order to avoid updating the test when the code changes, some programmers write a IsX=X test, which always succeeds irrespective of the SUT
About duplicating functionality
You don't have to. Your test can state what the expected output is, not how you derived it.
Although in some non-trivial cases, it may make your test more readable as to how you derived the expected value - test as a spec. You shouldn't refactor away this duplication
def doubler(x); x * 2; end
def test_doubler()
input, expected = 10, doubler(10)
assert_equal expected, doubler(10)
end
Now if I change doubler(x) to be a tripler, the above test won't fail.
def doubler(x); x * 3; end
However this one would:
def test_doubler()
assert_equal(20, doubler(10))
end
randomness in unit tests - Don't.
Instead of random datasets, choose static representative data points for testing and use a xUnit RowTest/TestCase to run the test with diff data inputs. If n input-sets are identical for the unit, choose 1.
The test in the OP could be used as a exploratory test/ or to determine additional representative input-sets. Unit tests need to be repeatable (See q#61400) - Using random values defeats this objective.
Never use random data for input. If your test reports a failure, how are you ever going to be able to duplicate it? And don't use the same function to generate the expected result. If you have a bug in your method you're likely to put the same bug in your test. Compute the expected results by some other method.
Hard-coded values are perfectly fine, and make sure inputs are picked to represent all of the normal and edge cases. At the very least test the expected inputs as well as inputs in the wrong format or wrong size (eg: null values).
It's really quite simple -- a unit test must test whether the function works or not. That means you need to give a range of known inputs that have known outputs and test against that. There is no universal right way to do that. However, using the same algorithm for the method and the verification proves nothing but that you're adept at copy/paste.