TDD and Mockito testing newb here... I have recently discovered about injection of mocks of private fields and am trying to get my ideas straight.
It may well be that my thinking on this particular matter is all wrong, but it's just that I often find a situation where I want to test that calling a particular method has resulted in a private field being assigned: before the call it was null, and after the call it is set to an instance of whatever class.
A mocked private field is a subclass of the class in question by the time you start to run your test method, so obviously testing for null is meaningless/doesn't work. Equally, if your method results in an assignment of this field this means that the mock variable has been replaced by a genuine instance. Thus, even if methods are called on it it won't lead to any interaction with the mock variable. In any event, if the test is just to check on the creation and assignment there is no reason you would want to have any interaction with it in the test.
Obviously one answer is to create a public get() method or a public create() method in the app class. But this gives me a bad feeling as it is clearly just being done for testing purposes.
Incidentally there will be those, many no doubt much more experienced than me at TDD, who will say that I'm getting too fine-grained and shouldn't be testing something as specific as assignment of a field. But my understanding of the most extreme approach to TDD is "don't write a line of code without a test". I really am trying to stick with this at this stage as far as possible. With experience maybe I will have greater understanding of how and when to depart from this principle.
It's just that assignment of a field seems to me to be a "class event" (of the host class) as worthy as any other type of event, such as a call of a method on the field once an instance has been assigned to the field. Shouldn't we be able to test for it in some way...? What do the Mockito pros who accept that this may be a legitimate thing to test for do in these circumstances?
later...
The comment from Fabio highlights the tension which, as a newb, I find puzzling, between "never write a line without a test" and "test only public methods": the effect of this is inevitably to expose much more stuff than I want, purely for the sake of testing.
And what he says about "changing tests all the time" I also find problematic: if we inject mocked private fields, and use them, then we are going to have to change the tests if we change the "implementation details". Nor does this idea fill me with dread: a need for change is perceived, you look at the very explanatory names of your existing test methods and decide which ones no longer apply, and of course you create new ones... the testing code, like the app code, will have to change all the time, right?
Isn't the testing code like some sort of die or mould, which guides and constrains the app code but at the same time can be detached from it? By Mockito allowing us to mock private fields a privileged form of access is already granted... who's to say how far this privileged access should rightly extend?
even later...
Jeff Bowman suggests this is a dup with this. I'm not entirely convinced, since I am asking a very specific thing for which it's just possibly there may be a specific answer (some Mockito means of detecting assignment). But his answer there seems very comprehensive and I'm trying to fully understand it all... including the whole dependency injection thing... and the compromises you seem have to make in "distorting" your public interface to make an app class as testable as you want to make it.
Simple example: you need to write a class Class1 with two methods Method1 and Method2 which return values provided by some Helper class.
[Test] Start writing test for first method - it will not compile
[Production] Create class Class1 and Method1 - compile ok and
test fail
[Production] In Method1 you create new instance of Helper and call some of this methods which return expected value - test pass
[Test] Create another test case for same Method1 - test fail
[Production] Make changes in Method1 to satisfy both test cases - tests are green
[Production] Make some refactorings in Method1 if you see possibility - test must stay green
[Test] Create test for Method2 which will update some public field Field1 of Class1 - will not compile
[Production] Create method Method2 and field Field1 in Class1 - compile - ok, test fail
[Production] Create new instance of Helper class in Method2, call it method and update Field1 with expected value - test pass
[Test] Create another test case for Method2 - test fail
[Production] Make changes in Method2 to satisfy both test cases - tests are green
Now you have implemented needed behavior and noticed that both Method1 and Method2 create instance of Helper class.
[Production] So you decide to declare private field of type Helper and use it in both methods - all tests are failed - because your private field not instantiated.
[Production] - you instantiate new instance in the constructor - all tests are green
Later if you decide to move creation of Helper class outside and pass it to the Class1 as constructor's parameter, you just change your production code and tests will show you does you break anything without changing tests.
Main point of my answer, that if you think like "I want write this line of code - How I can test it" - it will goes you in thats kind of situation you got right now.
Try to first think what you want to test and after writing the test think how you will implement it.
Related
My understanding is that unit testing should test classes in isolation, focusing on granular behavior and substituting objects of other classes using doubles/mocks wherever possible. (Please correct me if I'm wrong here.)
I'm writing a gem with a class called MatchList. MatchList::new takes two arguments, each an instance of another class called MatchPhrase. MatchPhrase contains some behavior that MatchList depends heavily on (i.e., if you feed anything other than a MatchPhrase to MatchList::new, you're going to get a bunch of "undefined method" errors).
My current (naive?) test setup uses let statements to assign variables for use in my examples:
let(:query) { MatchPhrase.new('Good Eats') }
let(:candidate) { MatchPhrase.new('Good Grief') }
let(:match_list) { MatchList.new(query, candidate) }
How do I write this unit test? Am I right in thinking it should be done without invoking the MatchPhrase class? Is that even possible?
For reference, here is what the MatchList class looks like:
class MatchList < Array
attr_reader :query, :this_phrase, :that_phrase
def initialize(query, candidate)
super(query.length)
#query = query
#this_phrase = query.dup
#that_phrase = candidate
find_matches until none?(&:nil?)
end
private
def find_matches
query.each.with_index do |this_token, i|
next unless self[i].nil?
that_token = this_token.best_match_in(that_phrase)
next if that_token.match?(that_token) &&
this_token != that_token.best_match_in(this_phrase)
self[i] = this_token.match?(that_token) ? that_token : NilToken.new
this_phrase.delete_once(this_token)
that_phrase.delete_once(that_token)
end
end
end
My understanding is that unit testing should test classes in isolation, focusing on granular behavior and substituting objects of other classes using doubles/mocks wherever possible. (Please correct me if I'm wrong here.)
In my understanding this is not true.
Using doubles/mocks has advantages and disadvantages.
Advantage is that you can take a slow service like database, email and mock it with fast performing object.
Disadvantage is that object that you are mocking is not "real" object and might surprise you and behave differently than real object would.
That's why it's always better to use real objects if practical.
Only use mocks if you want to speed up your tests or if it leads to much simpler tests. Even then have one test using real object to verify that it all works. This is called integration test.
Considering your case:
let(:query) { MatchPhrase.new('Good Eats') }
let(:candidate) { MatchPhrase.new('Good Grief') }
let(:match_list) { MatchList.new(query, candidate) }
There is really no advantage to mock query or candidate.
Mocking should be done for legitimate reasons and not as a matter of principle.
If there is only one collaborator class and your primary class is heavily coupled to it, mocking out the collaborator as a matter of principle may result in more fragility than benefit as the mock will not reflect the behavior of the collaborator.
Mocks and stubs are good candidates when you can reason against the mock's interface instead of an implementation. Let's ignore the existing code and look at the interfaces in use here:
MatchList.new takes a query and candidate
query is an Enumerable containing objects which implements best_match_in?(something)
The objects in query also implement delete_once(something)
candidate also implements delete_once(something)
best_match_in? returns something that implements match? and best_match_in?
Looking at the interfaces in use, MatchList appears to rely pretty heavily on the implementation of the query and candidate objects. Smells like a case of feature envy to me. Perhaps this functionality should reside within MatchPhrase instead.
In this case, I would write unit tests using actual MatchPhrase objects with a note to refactor this code.
Your understanding of using test part is correct. Its about focusing on granular behavior. E.g individual methods.
However to test the individual methods, try using doubles/mocks is not advisable. Marko ^^ has outlined the advantages / disadvantages of the mocks. I personally prefer not to not to use doubles/mocks as much as I can.
Its always a balance between the speed of your tests and the objects you create.
Before moving to doubles/mocks, its a good idea to see if you can write your test without saving the values to the DB. Like you have done before. That is faster than saving and retrieving the values from the DB.
One more thing is, private methods and not generally unit tested. This is with understanding of, the caller of your private method will have a unit test.
let(:query) { MatchPhrase.new('Good Eats') }
let(:candidate) { MatchPhrase.new('Good Grief') }
it 'check your expectation' do
expect(MatchList.new(query, candidate).query).to <check the expectation>
end
However I will re-evaluate the following points.
1 - do u want to call find_matches from initializer
2 - Let find_matches to return a value and then assign it to the #query variable (so that its easy to test the method with a return value)
3 - rename the query param in init to something else (just to avoid the confusion)
And the golden rule is If its hard to test (specially unit test), maybe your are doing something wrong.
HTH
First of all, I have to say, I'm new to mocking. So maybe I'm missing a point.
I'm also just starting to get used to the TDD approach.
So, in my actual project I'm working on a class in the business layer, while the data layer has yet to be deployed. I thought, this would be a good time to get started with mocking. I'm using Rhino Mocks, but I've come to the problem of needing to know the implementation details of a class before writing the class itself.
Rhino Mocks checks if alle the methods expected to be called are actually called. So I often need to know which mocked method is being called by the tested method first, even though they could be called in any order. Because of that I'm often writing complicated methods before I test them, because then I know already in which order the methods are being called.
simple example:
public void CreateAandB(bool arg1, bool arg2) {
if(arg1)
daoA.Create();
else throw new exception;
if(arg2)
daoB.Create();
else throw new exception;
}
if I want to test the error handling of this method, I'd have to know which method is being called first. But I don't want to be bugged about implementation details when writing the test first.
Am I missing something?
You have 2 choices. If the method should result in some change in your class the you can test the results of your method instead. So can you call CreateAandB(true,false) then then call some other method to see if the correct thing was created. In this situation your mock objects will probably be stubs which just provide some data.
If the doaA and doaB are objects which are injected into your class that actually create data in the DB or similar, which you can't verify the results of in the test, then you want to test the interaction with them, in which case you create the mocks and set the expectations, then call the method and verify that the expectations are met. In this situation your mock objects will be mocks and will verify the expected behaviour.
Yes you are testing implementation details, but your are testing the details of if your method is using its dependencies correctly, which is what you want to test, not how it is using them, which are the details you are not really interested in.
EDIT
IDao daoA = MockRepository.GenerateMock<IDao>(); //create mock
daoA.Expect(dao=>dao.Create); //set expectation
...
daoA.VerifyExpectations(); //check that the Create method was called
you can ensure that the expectations happen in a certain order, but not using the AAA syntax I believe (source from 2009, might have changed since,EDIT see here for an option which might work), but it seems someone has developed an approach which might allow this here. I've never used that and can't verify it.
As for needing to know which method was called first so you can verify the exception you have a couple of choices:
Have a different message in your exception and check that to determine which exception was raised.
Expect a call to daoA in addition to expecting the exception. If you don't get the call to daoA then the test fails as the exception must have been the first one.
Often times you just need fake objects, not mocks. Mock objects are meant to test component interaction, and often you can avoid this by querying the state of SUT directly. Most practical uses of mocks are to test interaction with some external system (DB, file system, webservice, etc.), and for other things you should be able to query system state directly.
I have a class API which has full code coverage and uses DI to mock out all the logic in the main class function (Job.Run) which does all the work.
I found a bug in production where we werent doing some validation on one of the data input fields.
So, I added a stub function called ValidateFoo()... Wrote a unit test against this function to Expect a JobFailedException, ran the test - it failed obviously because that function was empty. I added the validation logic, and now the test passes.
Great, now we know the validation works. Problem is - how do I write the test to make sure that ValidateFoo() is actually called inside Job.Run()? ValidateFoo() is a private method of the Job class - so it's not an interface...
Is there anyway to do this with NMock2.0? I know TypeMock supports fakes of non interface types. But changing mock libs right now is not an option. At this point if NMock can't support it, I will simply just add the ValidateFoo() call to the Run() method and test things manually - which obviously I'd prefer not to do considering my Job.Run() method has 100% coverage right now. Any Advice? Thanks very much it is appreciated.
EDIT: the other option I have in mind is to just create an integration test for my Job.Run functionality (injecting to it true implementations of the composite objects instead of mocks). I will give it a bad input value for that field and then validate that the job failed. This works and covers my test - but it's not really a unit test but instead an integration test that tests one unit of functionality.... hmm..
EDIT2: IS there any way to do tihs? Anyone have ideas? Maybe TypeMock - or a better design?
The current version of NMock2 can mock concrete types (I don't remember exactly which version they added this, but we're using version 2.1) using the mostly familiar syntax:
Job job = mockery.NewMock<Job>(MockStyle.Transparent);
Stub.On(job).Method("ValidateFoo").Will(Return.Value(true));
MockStyle.Transparent specifies that anything you don't stub or expect should be handled by the underlying implementation - so you can stub and set expectations for methods on an instance you're testing.
However, you can only stub and set expectations on public methods (and properties), which must also be virtual or abstract. So to avoid relying on integration testing, you have two options:
Make Job.ValidateFoo() public and virtual.
Extract the validation logic into a new class and inject an instance into Job.
Since all private are all called by public methods (unless relying on reflection runtime execution), then those privates are being executed by public methods. Those private methods are causing changes to the object beyond simply executing code, such as setting class fields or calling into other objects. I'd find a way to get at those "results" of calling the private method. (Or mocking the things that shouldn't be executed in the private methods.)
I can't see the class under test. Another problem that could be pushing you to want access to the private methods is that it's a super big class with a boatload of private functionality. These classes may need to be broken down into smaller classes, and some of those privates may turn into simpler publics.
I have a class called FooJob() which runs on a WCF windows service. This class has only 2 public methods, the constructor, and a Run() method.
When clients call my service, a Dim a new instance of the Job class, pass in some parameters to the ctor, then call Run()...
Run() will take the parameters, do some logic, send a (real time) request to an outside data vendor, take the response, do some business logic, then put it in the database...
Is it wise to only write a single unit test then (if even possible) on the Run() function? Or will I wind up killing myself here? In this case then should I drill into the private functions and unit test those of the FooJob() class? But then won't this 'break' the 'only test behavior' / public interface paradigm that some argue for in TDD?
I realize this might be a vague question, but any advice / guidance or points in the right direction would be much appreciated.
Drew
do some logic, send a (real time) request to an outside data vendor, take the response, do some business logic, then put it in the database
The problem here is that you've listed your class as having multiple responsibilities... to be truly unit testable you need to follow the single responsibility principle. You need to pull those responsibilities out into separate interfaces. Then, you can test your implementations of these interfaces individually (as units). If you find that you can't easily test something your class is doing, another class should probably be doing that.
It seems like you'd need at least the following:
An interface for your business logic.
An interface defining the request to the outside vendor.
An interface for your data repository.
Then you can test that business logic, the process of communicating with the outside vendor, and the process of saving to your database separately. You can then mock out those interfaces for testing your Run() method, simply ensuring that the methods are called as you expect.
To do this properly, the class's dependencies (the interfaces defined above) should ideally be passed in to its constructor (i.e. dependency injection), but that's another story.
My advice would be to let your tests help with the design of your code. If you are struggling to execute statements or functions then your class is doing too much. Follow the single-responsibility-priciple, add some interfaces (allowing you to mock out the complicated stuff), maybe even read Fowler's 'Refactoring' or Feather's 'Working With Legacy Code', these taught me more about TDD than any other book to date.
It sounds like your run method is trying to do too much I would separate it up but if you're overall design won't allow it.
I would consider making the internal members protected then inheriting from the class in your test class to test them. Be careful though I have run into gotchas doing this because it doesn't reset the classes state so Setup and TearDown methods are essential.
Simple answer is - it depends. I've written a lot of unit tests that test the behaviour of private methods; I've done this so that I can be happy that I've covered various inputs and scenarios against the methods.
Now, many people think that testing private methods is a bad idea, since it's the public methods that matter. I get this idea, but in my case the public method for these private calls was also just a simple Run() method. The logic of the private methods included reading from config files and performing tasks on the file system, all "behind the scenes".
Had I just created a unit test that called Run() then I would have felt that my tests were incomplete. I used MSTest to create accessors for my class, so that I could call the private methods and create various scenarios (such as what happens when I run out of disk space, etc).
I guess it's each to their own with this private method testing do/or don't do argument. My advice is that, if you feel that your tests are incomplete, in other words, require more coverage, then I'd recommend testing the private methods.
Thanks everyone for the comments. I believe you are right - I need to seperate out into more seperate classes. This is one of the first projects im doing using true TDD, in that I did no class design at all and am just writing stub code... I gotta admit, I love writing code like this and the fact I can justify it to my mangagment with years of backed up successful results is purely friggin awesome =).
The only thing I'm iffy about is over-engineering and suffering from class-bloat, when I could have just written unit tests against my private methods... I guess common sense and programmers gut have to be used here... ?
I've been told not to use implementation details. A dependency seems like an implementation detail. However I could phrase it also as a behavior.
Example: A LinkList depends on a storage engine to store its links (eg LinkStorageInterface). The constructor needs to be passed an instance of an implemented LinkStorageInterface to do its job.
I can't say 'shouldUseLinkStorage'. But maybe I can say 'shouldStoreLinksInStorage'.
What is correct to 'test' in this case? Should I test that it stores links in a store (behavior) or don't test this at all?
The dependency itself is not an expected behavior, but the actions called on the dependency most certainly are. You should test the stuff you (the caller) know about, and avoid testing the stuff that requires you to have intimate knowledge of the inner workings of the SUT.
Expanding your example a little, lets imagine that our LinkStorageInterface has the following definition (Pseudo-Code):
Interface LinkStorageInterface
void WriteListToPersistentMedium(LinkList list)
End Interface
Now, since you (the caller) are providing the concrete implementation for that interface it is perfectly reasonable for you to test that WriteListToPersistentMedium() gets called when you invoke the Save() method on your LinkList.
A test might look like this, again using pseudo-code:
void ShouldSaveLinkListToPersistentMedium()
define da = new MockLinkListStorage()
define list = new LinkList(da)
list.Save()
Assert.Method(da.WriteListToPersistentMedium).WasCalledWith(list)
end method
You have tested expected behavior without testing implementation specific details of either your SUT, or your mock. What you want to avoid testing (mostly) are things like:
Order in which methods were called
Making a method, or property public just so you can check it
Anything that does not directly involve the expected behavior you are testing
Again, a dependency is something that you as the consumer of the class are providing, so you expect it to be used. Otherwise there is no point in having that dependency in the first place.
LinkStorageInterface is not an implementation detail - its name suggests an interface to to an engine. In which case the name shouldUseLinkStorage has more value than shouldStoreLinksInStorage.
That's my 2 pennies worth!