A very specific question from a novice to TDD:
I separate my tests and my application into different packages. Thus, most of my application methods have to be public for tests to access them. As I progress, it becomes obvious that some methods could become private, but if I make that change, the tests that access them won't work. Am I missing a step, or doing something wrong, or is this just one downfall of TDD?
This is not a downfall of TDD, but rather an approach to testing that believes you need to test every property and every method. In fact you should not care about private methods when testing because they should only exist to facilitate some public portion of the API.
Never change something from private to public for testing purposes!
You should be trying to verify only publicly visible behavior. The rest are implementation details and you specifically want to avoid testing those. TDD is meant to give you a set of tests that will allow you to easily change the implementation details without breaking the tests (changing behavior).
Let’s say I have a type: MyClass and I want to test the DoStuff method. All I care about is that the DoStuff method does something meaningful and returns the expected results. It may call a hundred private methods to get to that point, but I don't care as the consumer of that method.
You don't specify what language you are using, but certainly in most of them you can put the tests in a way that have more privileged access to the class. In Java, for example, the test can be in the same package, with the actual class file being in a different directory so it is separate from production code.
However, when you are doing real TDD, the tests are driving the class design, so if you have a method that exists just to test some subset of functionality, you are probably (not always) doing something wrong, and you should look at techniques like dependency injection and mocking to better guide your design.
This is where the old saying, "TDD is about design," frequently comes up. A class with too many public methods probably has too many responsibilities - and the fact that you are test-driving it only exposes that; it doesn't cause the problem.
When you find yourself in this situation, the best solution is frequently to find some subset of the public methods that can be extracted into a new class ("sprout class"), then give your original class an instance variable of the sprouted class. The public methods deserve to be public in the new class, but they are now - with respect to the API of the original class - private. And you now have better adherence to SRP, looser coupling, and higher cohesion - better design.
All because TDD exposed features of your class that would otherwise have slid in under the radar. TDD is about design.
At least in Java, it's good practice to have two source trees, one for the code and one for the tests. So you can put your code and your tests in the same package, while they're still in different directories:
src/org/my/xy/X.java
test/org/my/xy/TestX.java
Then you can make your methods package private.
Related
At my workplace there seems to currently be a crusade against static classes. I can understand part of that, they do sort of break the whole unit testing modularity thing. However, I am seeing an influx of code reviews that call for removing static classes.
A common case is a utility class this is spring-injected with a few other objects that are "known" at compile-time. It has no other member variables. If class M is calling this static class, I always see the suggestion to make this utility class non-static and inject it into class M.
This doesn't make sense to me. I don't particularly see anything wrong with it other than that it seems a waste of time and makes utility class less easily usable. As far as I can tell the justification is usually for unit testing, but I don't like the idea that valid code has to be changed to conform to a testing paradigm. Admittedly mocking a simple static utility class would probably be overkill.
Are static classes appropriate in this use case, or best avoided?
I think the differences in the two approaches are small, but as long as the class contain no state it is slightly better to make it static. Let's say I have this code in class A:
StaticClass.utilMethod()
If I want to use this code in class B I can copy and paste. That's it. No adding member variables, injection, etc. cmd-c cmd-v.
Considering your existing code uses static classes and modifying that will take work, it's definitely best to continue using static classes.
I vote for using the static classes... i.e. A class with just static methods for purely Utility purposes. Even java has provided us such classes like java.util.Collections and java.util.Arrays
If you pretend for a moment that your static class did not belong to you, that it was, say, part of the .NET framework, how would your team handle it? Their answer would be my answer. In other words, if their answer to that question is inconsistent with what they're asking you to do, then they should probably either change how they work with .NET static classes or with how they work with yours.
I avoid using static classes (assuming we are actually talking about classes that contain static methods), and I do it for the sake of testability.
If you are using static methods, you will have a difficult time mocking/stubbing the portion of your code that uses said static for your unit tests.
Consider this:
public String myMethod() {
String complicatedStringOutput = MyUtility.createComplicatedStringOutput();
//do some more complicated work on this String
}
To write a unit test for this method, how would you go about making it a 'true unit test' without needing to also test the creation of complicatedStringOutput? In my unit tests, I prefer to test only the method that is the focus of the unit test.
Change it to this:
public String myMethod(MyNonStaticUtility util) {
String complicatedStringOutput = util.createComplicatedStringOutput();
//do some more complicated work on this String
}
Suddenly, this class is much easier to write a 'true unit test' for. You can control the behavior of MyNonStaticUtility by either using a stub or mock.
All that said, it is really up to you (or your Business Unit). If you value unit tests and feel that it is important to have good test coverage of your complicated code, this is the preferred approach. If you do not have time/money to invest in 'fixing' your code, then it just won't happen.
Depends, naturally.
How do you want to test the code that uses the static classes?
Do the static classes encapsulate behavior you'll need to mock often?
Will anybody ever need to modify the behavior of those static classes?
Finally:
Is there a compelling reason not to inject a Spring-managed singleton bean?
Admittedly mocking a simple static utility class would probably be overkill.
You're absolutely right on this. Static classes should only be used for utility classes that are extremely simple, where there is no benefit of mocking such a class. If you're using them for any other purpose, you should rethink your design.
Is it reasonable to use a non-static class if that class does not contain state?
This really has nothing to do with state. For example, strategy objects often contain no state, yet they are not static; they usually implement a common interface and need to be interchangeable / mockable.
I was just reading Martin Fowler's post Mocks Aren't Stubs. He defines the different test doubles (or rather references Gerard Meszaros's xUnit patterns book):
Dummy objects are passed around but never actually used. Usually they
are just used to fill parameter lists.
Fake objects actually have
working implementations, but usually take some shortcut which makes
them not suitable for production (an in memory database is a good
example).
Stubs provide canned answers to calls made during the test,
usually not responding at all to anything outside what's programmed in
for the test. Stubs may also record information about calls, such as
an email gateway stub that remembers the messages it 'sent', or maybe
only how many messages it 'sent'.
Mocks are ... objects pre-programmed with expectations which form a
specification of the calls they are expected to receive.
Part one of my question would be, is this even authoritative? Is this language widely used and understood?
The second part of my question is that it seems that my mocking framework of choice, Mockito, makes it easy to blur the line, certainly between mocks and stubs.
Everything is called "mock". Either by calling the Mockito.mock() method or with a #Mock annotation, you use the word "mock" to create mocks, stubs, and sometimes dummies (if a simple "new" won't do). The exception is a "spy" which might be used to make something like a "fake", but can also be used to wrap your system under test.
Even if you didn't care about the name of the method to create a test double, the double can be verified (or not) and you can include a capture in the verification step, which seem to include some things that a stub would do (remembering calls that were made) and mocks (verifying that certain expected calls were made).
The reason I ask is that I try to name my doubles according to the four things I see above, but then get confused sometimes whether something really has the role of stub or a mock. So, is this a deficiency of Mockito, or is this just how things have evolved and the distinction is not really important?
Actually, it's a strength of Mockito. A Mockito mock is an object on which you can either "stub" the methods, or "verify" the methods, or both. (Doing both for the same method is kind of a code smell, but that's another topic). So a Mockito mock is both a "stub" (in the Martin Fowler sense) and a "mock" (in the Martin Fowler sense); but you don't have to use it as both. Usually, a Mockito mock will act as EITHER a "stub", OR as a "mock"; less often as both.
In some other mocking frameworks, you don't have quite this level of flexibility. If you want to do "verifying" on a mock, you also have to do "stubbing". In these frameworks, the mocks MUST act as both a "stub" and as a "mock". As far as I understand, one of the factors that motivated Szczepan Faber to create Mockito was a desire to be able to separate "stub" behaviour, and "mock" behaviour (in the strict Martin Fowler senses of both words).
The English word "mock" means "an imitation of lesser quality than the original". This is why even hand-rolled mocks (written without the aid of a framework like Mockito) are sometimes called mocks.
The language which Martin used is now a little bit out of date. He wrote it in the context of old mocking frameworks like JMock, before the "nice mocks" came along. In that era, mocks used to be strict; any interactions which hadn't been set up and weren't expected would fail.
Nowadays we tend to think of it a different way. If I'm a class, I have some other classes that I need to help me. They're either providing information, or doing some work for me - for instance, a repository might provide a list of employees, or save a new employee.
Mocks stand in for these collaborators, and we don't tend to use expectations on mocks any more. Instead, we set up mocks to provide information, then verify that they were asked to do any jobs that need to be done. Mockito was the first framework to work this way, and that's why the distinction is blurred - because whatever you're doing, you're mocking out a collaborator, and you no longer need to set up expectations. Moq works the same way in .NET.
Mockito's mocks by default don't even care if you use them and don't check (although you'll need to set up any information that they have to provide before-hand - the equivalent of a "stub").
Additionally, because Mockito provides "nice" mocks, you don't need to worry about setting expectations in case a dummy object is used somewhere - you can just use Mockito to create those, as well. And, just in case you want to add some simple behavior, Mockito lets you do callbacks easily on the arguments which are passed to it, so you can create "Fakes".
It doesn't really matter what they are - you're just mocking out a collaborating class, and the flexibility means that you don't need to worry about how you do that.
Early frameworks didn't provide this flexibility, hence Martin's differentiation, intended to help you use mocks appropriately. Hope this helps clarify things and explain why Mockito's flexibility isn't a deficiency, but - as David Wallace pointed out - a strength.
According to what I understand from Gerard Meszaros' X-Unit test patterns, a mock is a test-double that includes the functionality of a dummy, a stub and a spy. You can check out the actual comparison he draws about them in pg.742 of that book.
Also this article might throw some light on your question. This article clearly states that
"A mock is dynamically created by a mock library (the others are
typically produced by a test developer using code). The test developer
never sees the actual code implementing the interface or base class,
but can configure the mock to provide return values, expect particular
members to be invoked, and so on. Depending on its configuration, a
mock can behave like a dummy, a stub, or a spy."
Both the quote and the image were taken from this article. IMHO, a mock is intended to blur the line between a dummy, a stub and a spy.
I'm reading through the (still beta) rspec book by the prag progs as I'm interested in behavioral testing on objects. From what I've gleaned so far (caveat: after only reading for 30 min), the basic idea is that I want ensure my object behaves as expected 'externally' i.e. in its output and in relation to other objects.
Is it true then that I should just be black box testing my object to ensure the proper output/interaction with other objects?
This may be completely wrong, but given all of the focus on how my object behaves in the system, it seems this is ideology one would take. If that's so, how do we focus on the implementation of an object? How do I test that my private method is doing what I want it to do for all different types of input?
I suppose this question is maybe valid for all types of testing?? I'm still fairly new to TDD and BDD.
If you want to understand BDD better, try thinking about it without using the word "test".
Instead of writing a test, you're going to write an example of how you can use your class (and you can't use it except through public methods). You're going to show why your class is valuable to other classes. You're defining the scope of your class's responsibilities, while showing (through mocks) what responsibilities are delegated elsewhere.
At the same time, you can question whether the responsibilities are appropriate, and tune the methods on your class to be as intuitively usable as possible. You're looking for code which is easy to understand and use, rather than code which is easy to write.
If you can think in terms of examples and providing value through behaviour, you'll create code that's easy to use, with examples and descriptions that other people can follow. You'll make your code safe and easy to change. If you think about testing, you'll pin it down so that nobody can break it. You'll make it hard to change.
If it's complex enough that there are internal methods you really want to test separately, break them out into another class then show why that class is valuable and what it does for the class that uses it.
Hope this helps!
I think there are two issues here.
One is that from the BDD perspective, you are typically testing at a higher level than from the TDD perspective. So your BDD tests will assert a bigger piece of functionality than your TDD tests and should always be "black box" tests.
The second is that if you feel the need to test private methods, even at the unit test level, that could be a code smell that your code is violating the Single Responsibilty Principle
and should be refactored so that the methods you care about can be tested as public methods of a different class. Michael Feathers gave an interesting talk about this recently called "The Deep Synergy Between Testability and Good Design."
Yes, focus on the exposed functionality of the class. Private methods are just part of a public function you will test. This point is a bit controversial, but in my opinion it should be enough to test the public functionality of a class (everything else also violates the OOP principle).
I have a class called FooJob() which runs on a WCF windows service. This class has only 2 public methods, the constructor, and a Run() method.
When clients call my service, a Dim a new instance of the Job class, pass in some parameters to the ctor, then call Run()...
Run() will take the parameters, do some logic, send a (real time) request to an outside data vendor, take the response, do some business logic, then put it in the database...
Is it wise to only write a single unit test then (if even possible) on the Run() function? Or will I wind up killing myself here? In this case then should I drill into the private functions and unit test those of the FooJob() class? But then won't this 'break' the 'only test behavior' / public interface paradigm that some argue for in TDD?
I realize this might be a vague question, but any advice / guidance or points in the right direction would be much appreciated.
Drew
do some logic, send a (real time) request to an outside data vendor, take the response, do some business logic, then put it in the database
The problem here is that you've listed your class as having multiple responsibilities... to be truly unit testable you need to follow the single responsibility principle. You need to pull those responsibilities out into separate interfaces. Then, you can test your implementations of these interfaces individually (as units). If you find that you can't easily test something your class is doing, another class should probably be doing that.
It seems like you'd need at least the following:
An interface for your business logic.
An interface defining the request to the outside vendor.
An interface for your data repository.
Then you can test that business logic, the process of communicating with the outside vendor, and the process of saving to your database separately. You can then mock out those interfaces for testing your Run() method, simply ensuring that the methods are called as you expect.
To do this properly, the class's dependencies (the interfaces defined above) should ideally be passed in to its constructor (i.e. dependency injection), but that's another story.
My advice would be to let your tests help with the design of your code. If you are struggling to execute statements or functions then your class is doing too much. Follow the single-responsibility-priciple, add some interfaces (allowing you to mock out the complicated stuff), maybe even read Fowler's 'Refactoring' or Feather's 'Working With Legacy Code', these taught me more about TDD than any other book to date.
It sounds like your run method is trying to do too much I would separate it up but if you're overall design won't allow it.
I would consider making the internal members protected then inheriting from the class in your test class to test them. Be careful though I have run into gotchas doing this because it doesn't reset the classes state so Setup and TearDown methods are essential.
Simple answer is - it depends. I've written a lot of unit tests that test the behaviour of private methods; I've done this so that I can be happy that I've covered various inputs and scenarios against the methods.
Now, many people think that testing private methods is a bad idea, since it's the public methods that matter. I get this idea, but in my case the public method for these private calls was also just a simple Run() method. The logic of the private methods included reading from config files and performing tasks on the file system, all "behind the scenes".
Had I just created a unit test that called Run() then I would have felt that my tests were incomplete. I used MSTest to create accessors for my class, so that I could call the private methods and create various scenarios (such as what happens when I run out of disk space, etc).
I guess it's each to their own with this private method testing do/or don't do argument. My advice is that, if you feel that your tests are incomplete, in other words, require more coverage, then I'd recommend testing the private methods.
Thanks everyone for the comments. I believe you are right - I need to seperate out into more seperate classes. This is one of the first projects im doing using true TDD, in that I did no class design at all and am just writing stub code... I gotta admit, I love writing code like this and the fact I can justify it to my mangagment with years of backed up successful results is purely friggin awesome =).
The only thing I'm iffy about is over-engineering and suffering from class-bloat, when I could have just written unit tests against my private methods... I guess common sense and programmers gut have to be used here... ?
C#, nUnit, and Rhino Mocks, if that turns out to be applicable.
My quest with TDD continues as I attempt to wrap tests around a complicated function. Let's say I'm coding a form that, when saved, has to also save dependent objects within the form...answers to form questions, attachments if available, and "log" entries (such as "blahblah updated the form." or "blahblah attached a file."). This save function also fires off emails to various people depending on how the state of the form changed during the save function.
This means in order to fully test out the form's save function with all of its dependencies, I have to inject five or six data providers to test out this one function and make sure everything fired off in the right way and order. This is cumbersome when writing the multiple chained constructors for the form object to insert the mocked providers. I think I'm missing something, either in the way of refactoring or simply a better way to set the mocked data providers.
Should I further study refactoring methods to see how this function can be simplified? How's the observer pattern sound, so that the dependent objects detect when the parent form is saved and handle themselves? I know that people say to split out the function so it can be tested...meaning I test out the individual save functions of each dependent object, but not the save function of the form itself, which dictates how each should save themselves in the first place?
First, if you are following TDD, then you don't wrap tests around a complicated function. You wrap the function around your tests. Actually, even that's not right. You interweave your tests and functions, writing both at almost exactly the same time, with the tests just a little ahead of the functions. See The Three Laws of TDD.
When you follow these three laws, and are diligent about refactoring, then you never wind up with "a complicated function". Rather you wind up with many, tested, simple functions.
Now, on to your point. If you already have "a complicated function" and you want to wrap tests around it then you should:
Add your mocks explicitly, instead of through DI. (e.g. something horrible like a 'test' flag and an 'if' statement that selects the mocks instead of the real objects).
Write a few tests in order to cover the basic operation of the component.
Refactor mercilessly, breaking up the complicated function into many little simple functions, while running your cobbled together tests as often as possible.
Push the 'test' flag as high as possible. As you refactor, pass your data sources down to the small simple functions. Don't let the 'test' flag infect any but the topmost function.
Rewrite tests. As you refactor, rewrite as many tests as possible to call the simple little functions instead of the big top-level function. You can pass your mocks into the simple functions from your tests.
Get rid of the 'test' flag and determine how much DI you really need. Since you have tests written at the lower levels that can insert mocks through areguments, you probably don't need to mock out many data sources at the top level anymore.
If, after all this, the DI is still cumbersome, then think about injecting a single object that holds references to all your data sources. It's always easier to inject one thing rather than many.
Use an AutoMocking container. There is one written for RhinoMocks.
Imagine you have a class with a lot of dependencies injected via constructor injection. Here's what it looks like to set it up with RhinoMocks, no AutoMocking container:
private MockRepository _mocks;
private BroadcastListViewPresenter _presenter;
private IBroadcastListView _view;
private IAddNewBroadcastEventBroker _addNewBroadcastEventBroker;
private IBroadcastService _broadcastService;
private IChannelService _channelService;
private IDeviceService _deviceService;
private IDialogFactory _dialogFactory;
private IMessageBoxService _messageBoxService;
private ITouchScreenService _touchScreenService;
private IDeviceBroadcastFactory _deviceBroadcastFactory;
private IFileBroadcastFactory _fileBroadcastFactory;
private IBroadcastServiceCallback _broadcastServiceCallback;
private IChannelServiceCallback _channelServiceCallback;
[SetUp]
public void SetUp()
{
_mocks = new MockRepository();
_view = _mocks.DynamicMock<IBroadcastListView>();
_addNewBroadcastEventBroker = _mocks.DynamicMock<IAddNewBroadcastEventBroker>();
_broadcastService = _mocks.DynamicMock<IBroadcastService>();
_channelService = _mocks.DynamicMock<IChannelService>();
_deviceService = _mocks.DynamicMock<IDeviceService>();
_dialogFactory = _mocks.DynamicMock<IDialogFactory>();
_messageBoxService = _mocks.DynamicMock<IMessageBoxService>();
_touchScreenService = _mocks.DynamicMock<ITouchScreenService>();
_deviceBroadcastFactory = _mocks.DynamicMock<IDeviceBroadcastFactory>();
_fileBroadcastFactory = _mocks.DynamicMock<IFileBroadcastFactory>();
_broadcastServiceCallback = _mocks.DynamicMock<IBroadcastServiceCallback>();
_channelServiceCallback = _mocks.DynamicMock<IChannelServiceCallback>();
_presenter = new BroadcastListViewPresenter(
_addNewBroadcastEventBroker,
_broadcastService,
_channelService,
_deviceService,
_dialogFactory,
_messageBoxService,
_touchScreenService,
_deviceBroadcastFactory,
_fileBroadcastFactory,
_broadcastServiceCallback,
_channelServiceCallback);
_presenter.View = _view;
}
Now, here's the same thing with an AutoMocking container:
private MockRepository _mocks;
private AutoMockingContainer _container;
private BroadcastListViewPresenter _presenter;
private IBroadcastListView _view;
[SetUp]
public void SetUp()
{
_mocks = new MockRepository();
_container = new AutoMockingContainer(_mocks);
_container.Initialize();
_view = _mocks.DynamicMock<IBroadcastListView>();
_presenter = _container.Create<BroadcastListViewPresenter>();
_presenter.View = _view;
}
Easier, yes?
The AutoMocking container automatically creates mocks for every dependency in the constructor, and you can access them for testing like so:
using (_mocks.Record())
{
_container.Get<IChannelService>().Expect(cs => cs.ChannelIsBroadcasting(channel)).Return(false);
_container.Get<IBroadcastService>().Expect(bs => bs.Start(8));
}
Hope that helps. I know my testing life has been made a whole lot easier with the advent of the AutoMocking container.
You're right that it can be cumbersome.
Proponent of mocking methodology would point out that the code is written improperly to being with. That is, you shouldn't be constructing dependent objects inside this method. Rather, the injection API's should have functions that create the appropriate objects.
As for mocking up 6 different objects, that's true. However, if you also were unit-testing those systems, those objects should already have mocking infrastructure you can use.
Finally, use a mocking framework that does some of the work for you.
I don't have your code, but my first reaction is that your test is trying to tell you that your object has too many collaborators. In cases like this, I always find that there's a missing construct in there that should be packaged up into a higher level structure. Using an automocking container is just muzzling the feedback you're getting from your tests. See http://www.mockobjects.com/2007/04/test-smell-bloated-constructor.html for a longer discussion.
In this context, I usually find statements along the lines of "this indicates that your object has too many dependencies" or "your object has too many collaborators" to be a fairly specious claim. Of course a MVC controller or a form is going to be calling lots of different services and objects to fulfill its duties; it is, after all, sitting at the top layer of the application. You can smoosh some of these dependencies together into higher-level objects (say, a ShippingMethodRepository and a TransitTimeCalculator get combined into a ShippingRateFinder), but this only goes so far, especially for these top-level, presentation-oriented objects. That's one less object to mock, but you've just obfuscated the actual dependencies via one layer of indirection, not actually removed them.
One blasphemous piece of advice is to say that if you are dependency injecting an object and creating an interface for it that is quite unlikely to ever change (Are you really going to drop in a new MessageBoxService while changing your code? Really?), then don't bother. That dependency is part of the expected behavior of the object and you should just test them together since the integration test is where the real business value lies.
The other blasphemous piece of advice is that I usually see little utility in unit testing MVC controllers or Windows Forms. Everytime I see someone mocking the HttpContext and testing to see if a cookie was set, I want to scream. Who cares if the AccountController set a cookie? I don't. The cookie has nothing to do with treating the controller as a black box; an integration test is what is needed to test its functionality (hmm, a call to PrivilegedArea() failed after Login() in the integration test). This way, you avoid invalidating a million useless unit tests if the format of the login cookie ever changes.
Save the unit tests for the object model, save the integration tests for the presentation layer, and avoid mock objects when possible. If mocking a particular dependency is hard, it's time to be pragmatic: just don't do the unit test and write an integration test instead and stop wasting your time.
The simple answer is that code that you are trying to test is doing too much. I think sticking to the Single Responsibility Principle might help.
The Save button method should only contain a top-level calls to delegate things to other objects. These objects can then be abstracted through interfaces. Then when you test the Save button method, you only test the interaction with mocked objects.
The next step is to write tests to these lower-level classes, but thing should get easier since you only test these in isolation. If you need a complex test setup code, this is a good indicator of a bad design (or a bad testing approach).
Recommended reading:
Clean Code: A Handbook of Agile Software Craftsmanship
Google's guide to writing testable code
Constructor DI isn't the only way to do DI. Since you're using C#, if your constructor does no significant work you could use Property DI. That simplifies things greatly in terms of your object's constructors at the expense of complexity in your function. Your function must check for the nullity of any dependent properties and throw InvalidOperation if they're null, before it begins work.
When it is hard to test something, it is usually symptom of the code quality, that the code is not testable (mentioned in this podcast, IIRC). The recommendation is to refactor the code so that the code will be easy to test. Some heuristics for deciding how to split the code into classes are the SRP and OCP. For more specific instructions, it would be necessary to see the code in question.