I was just reading Martin Fowler's post Mocks Aren't Stubs. He defines the different test doubles (or rather references Gerard Meszaros's xUnit patterns book):
Dummy objects are passed around but never actually used. Usually they
are just used to fill parameter lists.
Fake objects actually have
working implementations, but usually take some shortcut which makes
them not suitable for production (an in memory database is a good
example).
Stubs provide canned answers to calls made during the test,
usually not responding at all to anything outside what's programmed in
for the test. Stubs may also record information about calls, such as
an email gateway stub that remembers the messages it 'sent', or maybe
only how many messages it 'sent'.
Mocks are ... objects pre-programmed with expectations which form a
specification of the calls they are expected to receive.
Part one of my question would be, is this even authoritative? Is this language widely used and understood?
The second part of my question is that it seems that my mocking framework of choice, Mockito, makes it easy to blur the line, certainly between mocks and stubs.
Everything is called "mock". Either by calling the Mockito.mock() method or with a #Mock annotation, you use the word "mock" to create mocks, stubs, and sometimes dummies (if a simple "new" won't do). The exception is a "spy" which might be used to make something like a "fake", but can also be used to wrap your system under test.
Even if you didn't care about the name of the method to create a test double, the double can be verified (or not) and you can include a capture in the verification step, which seem to include some things that a stub would do (remembering calls that were made) and mocks (verifying that certain expected calls were made).
The reason I ask is that I try to name my doubles according to the four things I see above, but then get confused sometimes whether something really has the role of stub or a mock. So, is this a deficiency of Mockito, or is this just how things have evolved and the distinction is not really important?
Actually, it's a strength of Mockito. A Mockito mock is an object on which you can either "stub" the methods, or "verify" the methods, or both. (Doing both for the same method is kind of a code smell, but that's another topic). So a Mockito mock is both a "stub" (in the Martin Fowler sense) and a "mock" (in the Martin Fowler sense); but you don't have to use it as both. Usually, a Mockito mock will act as EITHER a "stub", OR as a "mock"; less often as both.
In some other mocking frameworks, you don't have quite this level of flexibility. If you want to do "verifying" on a mock, you also have to do "stubbing". In these frameworks, the mocks MUST act as both a "stub" and as a "mock". As far as I understand, one of the factors that motivated Szczepan Faber to create Mockito was a desire to be able to separate "stub" behaviour, and "mock" behaviour (in the strict Martin Fowler senses of both words).
The English word "mock" means "an imitation of lesser quality than the original". This is why even hand-rolled mocks (written without the aid of a framework like Mockito) are sometimes called mocks.
The language which Martin used is now a little bit out of date. He wrote it in the context of old mocking frameworks like JMock, before the "nice mocks" came along. In that era, mocks used to be strict; any interactions which hadn't been set up and weren't expected would fail.
Nowadays we tend to think of it a different way. If I'm a class, I have some other classes that I need to help me. They're either providing information, or doing some work for me - for instance, a repository might provide a list of employees, or save a new employee.
Mocks stand in for these collaborators, and we don't tend to use expectations on mocks any more. Instead, we set up mocks to provide information, then verify that they were asked to do any jobs that need to be done. Mockito was the first framework to work this way, and that's why the distinction is blurred - because whatever you're doing, you're mocking out a collaborator, and you no longer need to set up expectations. Moq works the same way in .NET.
Mockito's mocks by default don't even care if you use them and don't check (although you'll need to set up any information that they have to provide before-hand - the equivalent of a "stub").
Additionally, because Mockito provides "nice" mocks, you don't need to worry about setting expectations in case a dummy object is used somewhere - you can just use Mockito to create those, as well. And, just in case you want to add some simple behavior, Mockito lets you do callbacks easily on the arguments which are passed to it, so you can create "Fakes".
It doesn't really matter what they are - you're just mocking out a collaborating class, and the flexibility means that you don't need to worry about how you do that.
Early frameworks didn't provide this flexibility, hence Martin's differentiation, intended to help you use mocks appropriately. Hope this helps clarify things and explain why Mockito's flexibility isn't a deficiency, but - as David Wallace pointed out - a strength.
According to what I understand from Gerard Meszaros' X-Unit test patterns, a mock is a test-double that includes the functionality of a dummy, a stub and a spy. You can check out the actual comparison he draws about them in pg.742 of that book.
Also this article might throw some light on your question. This article clearly states that
"A mock is dynamically created by a mock library (the others are
typically produced by a test developer using code). The test developer
never sees the actual code implementing the interface or base class,
but can configure the mock to provide return values, expect particular
members to be invoked, and so on. Depending on its configuration, a
mock can behave like a dummy, a stub, or a spy."
Both the quote and the image were taken from this article. IMHO, a mock is intended to blur the line between a dummy, a stub and a spy.
Related
First, Please bear with me with all my questions. I have never used TDD before but more and more I come to realize that I should. I have read a lot of posts and how to guides on TDD but some things are still not clear. Most example used for demonstration are either math calculation or some other simple operations. I also started reading Roy Osherove's book about TDD. Here are some questions I have:
If you have an object in your solution, for instance an Account class, what is the benefit of testing setting a property on it, for example an account name, then you Assert that whatever you set is right. Would this ever fail?
Another example, an account balance, you create an object with balance 300 then you assert that the balance is actually 300. How would that ever fail? What would I be testing here? I can see testing a subtraction operation with different input parameters would be more of a good test.
What should I actually test my objects for? methods or properties? sometime you also have objects as service in an infrastructure layer. In the case of methods, if you have a three tier app and the business layer is calling the data layer for some data. What gets tested in that case? the parameters? the data object not being null? what about in the case of services?
Then on to my question regarding real life project, if you have a green project and you want to start it with TDD. What do you start with first? do you divide your project into features then tdd each one or do you actually pick arbitrarily and you go from there.
For example, I have a new project and it requires a login capability. Do I start with creating User tests or Account tests or Login tests. Which one I start with first? What do I test in that class first?
Let's say I decide to create a User class that has a username and password and some other properties. I'm supposed to create the test first, fix all build error, run the test for it to fail then fix again to get a green light then refactor. So what are the first tests I should create on that class? For example, is it:
Username_Length_Greater_Than_6
Username_Length_Less_Than_12
Password_Complexity
If you assert that length is greater than 6, how is that testing the code? do we test that we throw an error if it's less than 6?
I am sorry if I was repetitive with my questions. I'm just trying to get started with TDD and I have not been able to have a mindset change. Thank you and hopefully someone can help me determine what am I missing here. By the way, does anyone know of any discussion groups or chats regarding TDD that I can join?
Have a look at low-level BDD. This post by Dan North introduces it quite well.
Rather than testing properties, think about the behavior you're looking for. For instance:
Account Behavior:
should allow a user to choose the account name
should allow funds to be added to the account
User Registration Behavior:
should ensure that all usernames are between 6 and 12 characters
should ask the password checker if the password is complex enough <-- you'd use a mock here
These would then become tests for each class, with the "should" becoming the test name. Each test is an example of how the class can be used valuably. Instead of testing methods and properties, you're showing someone else (or your future self) why the class is valuable and how to change it safely.
We also do something in BDD called "outside-in". So start with the GUI (or normally the controller / presenter, since we don't often unit-test the GUI).
You already know how the GUI will use the controller. Now write an example of that. You'll probably have more than one aspect of behavior, so write more examples until the controller works. The controller will have a number of collaborating classes that you haven't written yet, so mock those out - just dependency inject them via an interface. You can write them later.
When you've finished with the controller, replace the next thing you've mocked out in the real system by real code, and test-drive that. Oh, and don't bother mocking out domain objects (like Account) - it'll be a pain in the neck - but do inject any complex behavior into them and mock that out instead.
This way, you're always writing the interface that you wish you had - something that's easy to use - for every class. You're describing the behavior of that class and providing some examples of how to use it. You're making it safe and easy to change, and the appropriate design will emerge (feel free to be guided by patterns, thoughtful common sense and experience).
BTW, with Login, I tend to work out what the user wants to log in for, then code that first. Add Login later - it's usually not very risky and doesn't change much once it's written, so you may not even need to unit-test it. Up to you.
Test until fear is replaced by boredom. Property accessors and constructors are high cost to benefit to write tests against. I usually test them indirectly as part of some other (higher) test.
For a new project, I'd recommend looking at ATDD. Find a user-story that you want to pick first (based on user value). Write an acceptance test that should pass when the user story is done. Now drill down into the types that you'd need to get the AT to pass -- using TDD. The acceptance test will tell you which objects and what behaviors are required. You then implement them one at a time using TDD. When all your tests (incl your acc. test) pass - you pick up the next user story and repeat.
Let's say you pick 'Create user' as your first story. Then you write examples of how that should work. Turn them into automated acceptance tests.
create valid user -> account should be created
create invalid user ( diff combinations that show what is invalid ) -> account shouldn't be created, helpful error shown to the user
AccountsVM.CreateUser(username, password)
AccountsVM.HasUser(username)
AccountsVM.ErrorMessage
The test would show that you need the above. You then go test-drive them them out.
Don't test what is too simple to break.
getters and setters are too simple to be broken, so said, the code is so simple that an error can not happen.
you test the public methods and assert the response is as expected. If the method return void you have to test "collateral consequences" (sometimes is not easy, eg to test a email was sent). When this happens you can use mocks to test not the response but how the method executes (you ask the mockk if the Class Under Test called him the desired way)
I start doing Katas to learn the basics: JUnit and TestNG; then Harmcrest; then read EasyMock or Mockito documentation.
Look for katas at github, or here
http://codekata.pragprog.com
http://codingdojo.org/
The first test should be the easiest one! Maybe one that just force you to create the CUT (class under test)
But again, try katas!
http://codingdojo.org/cgi-bin/wiki.pl?KataFizzBuzz
I'm reading through the (still beta) rspec book by the prag progs as I'm interested in behavioral testing on objects. From what I've gleaned so far (caveat: after only reading for 30 min), the basic idea is that I want ensure my object behaves as expected 'externally' i.e. in its output and in relation to other objects.
Is it true then that I should just be black box testing my object to ensure the proper output/interaction with other objects?
This may be completely wrong, but given all of the focus on how my object behaves in the system, it seems this is ideology one would take. If that's so, how do we focus on the implementation of an object? How do I test that my private method is doing what I want it to do for all different types of input?
I suppose this question is maybe valid for all types of testing?? I'm still fairly new to TDD and BDD.
If you want to understand BDD better, try thinking about it without using the word "test".
Instead of writing a test, you're going to write an example of how you can use your class (and you can't use it except through public methods). You're going to show why your class is valuable to other classes. You're defining the scope of your class's responsibilities, while showing (through mocks) what responsibilities are delegated elsewhere.
At the same time, you can question whether the responsibilities are appropriate, and tune the methods on your class to be as intuitively usable as possible. You're looking for code which is easy to understand and use, rather than code which is easy to write.
If you can think in terms of examples and providing value through behaviour, you'll create code that's easy to use, with examples and descriptions that other people can follow. You'll make your code safe and easy to change. If you think about testing, you'll pin it down so that nobody can break it. You'll make it hard to change.
If it's complex enough that there are internal methods you really want to test separately, break them out into another class then show why that class is valuable and what it does for the class that uses it.
Hope this helps!
I think there are two issues here.
One is that from the BDD perspective, you are typically testing at a higher level than from the TDD perspective. So your BDD tests will assert a bigger piece of functionality than your TDD tests and should always be "black box" tests.
The second is that if you feel the need to test private methods, even at the unit test level, that could be a code smell that your code is violating the Single Responsibilty Principle
and should be refactored so that the methods you care about can be tested as public methods of a different class. Michael Feathers gave an interesting talk about this recently called "The Deep Synergy Between Testability and Good Design."
Yes, focus on the exposed functionality of the class. Private methods are just part of a public function you will test. This point is a bit controversial, but in my opinion it should be enough to test the public functionality of a class (everything else also violates the OOP principle).
I have created a factory and a set of specifications to create and validate an aggregate root. Currently I have some tests for the factory that call the specifications on the product of the factory, but I'm wondering if that's enough. It might be better from a design perspective to couple the factory to the specifications of it's product, since they are closely interrelated.
If a specification for an aggregate root product is being used for validation, rather than for creation, does it make sense to call it from inside the factory?
Or is a unit test good enough?
The answer probably depends on how you are using your specifications, and whether the code is breaking a lot during the creation process.
Specifications can be used for almost anything you can think of. At a basic level specifications are merely controllable conditional statements encapsulated into objects. Wherever the code uses conditional logic one could probably refactor that logic into specifications, if the developer felt there was some justification.
There is nothing wrong with using specifications in the actual code, so long as it makes the code more usable, maintainable, or readable. There is also nothing wrong with creating specifications that are only used in tests. Specifications are simple objects, coupling code to specifications in one way or another doesn't seem to have much of a negative impact on maintenance or reusability due to the relative simplicity of most specifications.
If a specification for an aggregate
root product is being used for
validation, rather than for creation,
does it make sense to call it from
inside the factory?
Yes, but probably only if you are having trouble or a lack of confidence in the product of the factory.
Or is a unit test good enough?
Yeah calling a specification from a unit test can be good enough to prove the validity of a factory's product (at least in regard to what the specification covers). I don't often use specifications in my unit tests however, only when I'm having a tough time with something, or it's part of the logic that I'm testing.
A very specific question from a novice to TDD:
I separate my tests and my application into different packages. Thus, most of my application methods have to be public for tests to access them. As I progress, it becomes obvious that some methods could become private, but if I make that change, the tests that access them won't work. Am I missing a step, or doing something wrong, or is this just one downfall of TDD?
This is not a downfall of TDD, but rather an approach to testing that believes you need to test every property and every method. In fact you should not care about private methods when testing because they should only exist to facilitate some public portion of the API.
Never change something from private to public for testing purposes!
You should be trying to verify only publicly visible behavior. The rest are implementation details and you specifically want to avoid testing those. TDD is meant to give you a set of tests that will allow you to easily change the implementation details without breaking the tests (changing behavior).
Let’s say I have a type: MyClass and I want to test the DoStuff method. All I care about is that the DoStuff method does something meaningful and returns the expected results. It may call a hundred private methods to get to that point, but I don't care as the consumer of that method.
You don't specify what language you are using, but certainly in most of them you can put the tests in a way that have more privileged access to the class. In Java, for example, the test can be in the same package, with the actual class file being in a different directory so it is separate from production code.
However, when you are doing real TDD, the tests are driving the class design, so if you have a method that exists just to test some subset of functionality, you are probably (not always) doing something wrong, and you should look at techniques like dependency injection and mocking to better guide your design.
This is where the old saying, "TDD is about design," frequently comes up. A class with too many public methods probably has too many responsibilities - and the fact that you are test-driving it only exposes that; it doesn't cause the problem.
When you find yourself in this situation, the best solution is frequently to find some subset of the public methods that can be extracted into a new class ("sprout class"), then give your original class an instance variable of the sprouted class. The public methods deserve to be public in the new class, but they are now - with respect to the API of the original class - private. And you now have better adherence to SRP, looser coupling, and higher cohesion - better design.
All because TDD exposed features of your class that would otherwise have slid in under the radar. TDD is about design.
At least in Java, it's good practice to have two source trees, one for the code and one for the tests. So you can put your code and your tests in the same package, while they're still in different directories:
src/org/my/xy/X.java
test/org/my/xy/TestX.java
Then you can make your methods package private.
I have a class called FooJob() which runs on a WCF windows service. This class has only 2 public methods, the constructor, and a Run() method.
When clients call my service, a Dim a new instance of the Job class, pass in some parameters to the ctor, then call Run()...
Run() will take the parameters, do some logic, send a (real time) request to an outside data vendor, take the response, do some business logic, then put it in the database...
Is it wise to only write a single unit test then (if even possible) on the Run() function? Or will I wind up killing myself here? In this case then should I drill into the private functions and unit test those of the FooJob() class? But then won't this 'break' the 'only test behavior' / public interface paradigm that some argue for in TDD?
I realize this might be a vague question, but any advice / guidance or points in the right direction would be much appreciated.
Drew
do some logic, send a (real time) request to an outside data vendor, take the response, do some business logic, then put it in the database
The problem here is that you've listed your class as having multiple responsibilities... to be truly unit testable you need to follow the single responsibility principle. You need to pull those responsibilities out into separate interfaces. Then, you can test your implementations of these interfaces individually (as units). If you find that you can't easily test something your class is doing, another class should probably be doing that.
It seems like you'd need at least the following:
An interface for your business logic.
An interface defining the request to the outside vendor.
An interface for your data repository.
Then you can test that business logic, the process of communicating with the outside vendor, and the process of saving to your database separately. You can then mock out those interfaces for testing your Run() method, simply ensuring that the methods are called as you expect.
To do this properly, the class's dependencies (the interfaces defined above) should ideally be passed in to its constructor (i.e. dependency injection), but that's another story.
My advice would be to let your tests help with the design of your code. If you are struggling to execute statements or functions then your class is doing too much. Follow the single-responsibility-priciple, add some interfaces (allowing you to mock out the complicated stuff), maybe even read Fowler's 'Refactoring' or Feather's 'Working With Legacy Code', these taught me more about TDD than any other book to date.
It sounds like your run method is trying to do too much I would separate it up but if you're overall design won't allow it.
I would consider making the internal members protected then inheriting from the class in your test class to test them. Be careful though I have run into gotchas doing this because it doesn't reset the classes state so Setup and TearDown methods are essential.
Simple answer is - it depends. I've written a lot of unit tests that test the behaviour of private methods; I've done this so that I can be happy that I've covered various inputs and scenarios against the methods.
Now, many people think that testing private methods is a bad idea, since it's the public methods that matter. I get this idea, but in my case the public method for these private calls was also just a simple Run() method. The logic of the private methods included reading from config files and performing tasks on the file system, all "behind the scenes".
Had I just created a unit test that called Run() then I would have felt that my tests were incomplete. I used MSTest to create accessors for my class, so that I could call the private methods and create various scenarios (such as what happens when I run out of disk space, etc).
I guess it's each to their own with this private method testing do/or don't do argument. My advice is that, if you feel that your tests are incomplete, in other words, require more coverage, then I'd recommend testing the private methods.
Thanks everyone for the comments. I believe you are right - I need to seperate out into more seperate classes. This is one of the first projects im doing using true TDD, in that I did no class design at all and am just writing stub code... I gotta admit, I love writing code like this and the fact I can justify it to my mangagment with years of backed up successful results is purely friggin awesome =).
The only thing I'm iffy about is over-engineering and suffering from class-bloat, when I could have just written unit tests against my private methods... I guess common sense and programmers gut have to be used here... ?