I have a class with tests after many red-green cycles.
I see that the class is to big and make different tasks.
Now i want refactor it... after refactoring i have 3 classes and still 100% coverage and all tests are still green.
But the problem is, i test the new two classes over the tests from the "old big class", should i split the tests now?? Or should i write extra tests for each class before i refactore the code in classes?
should i split the tests now??
A quick implication of "refactoring" -- your tests didn't need to change. The public API and the observable behaviors of your implementation were not changed. You've just created some new module boundaries within the implementation.
(If the changes you made forced you to modify the existing tests, then the activity you were doing was not "refactoring").
Since the public API hasn't changed, the existing tests still have value - they describe the required behaviors of your original API. So you shouldn't "split" those tests.
If the new classes that you created during refactoring are going to be lifted into the public API, then you should introduce new tests for the new public API.
Your existing tests of the original API remain untouched until you have announced that API's end of life, deprecated it, and reached the end of life deadline. At that point, you can delete the tests of the old API.
Related
A simple question: How do you differentiate between a feature, unit and integration test?
There are a lot of differing opinions, but I'm specifically trying to determine how to organise a Laravel test which touches a model's relationship. Here is an example if some PHP code which would require testing:
public function prices()
{
return $this->hasMany(Prices::class);
}
public function getPriceAttribute($)
{
return $this->prices()->first() * 2;
}
The test descriptions as I understand them (feel free to correct me):
Unit test
Tests the smallest part of your code
Does not touch the database
Does not interact with any other part of the system
Integration test
Tests part of the system working together
e.g controllers which call helper functions which need to be tested together
Feature test
Blackbox test
e.g. Call an api end point, see that it has returned the correct JSON response
Here is my issue given those descriptions:
My Laravel model test needs to test the smallest unit of code - the calculated accessor of a model, which makes it feel like a Unit test
But, it touches the database when it loads the model's relationship
It doesnt feel like an Integration test, because it is only touching other related models, not internal or external services
Other property accessor tests in Laravel would fall under Unit tests when they do not touch the database or the model's relationships
Separating these types of tests into integration tests would mean that a single model's tests against its properties are fragmented between integration and unit tests
So, without mocking relationships between models, where would my test belong?
If I’m interpreting your original question correctly, I think the killer constraint here is:
So, without mocking relationships between models, where would my test belong?
If mocking isn't allowed and you're required to touch a DB then, by your/and google's definition, it has to belong as an integration/medium size test :)
The way I think of this is get price attribute functionality is separate from the DB. Even though it's in the model the prices could come from anywhere. Right now its a RDBMS but what if your org go really big and it split into another service? Basically, I believe, that the capability of getPriceAttributes is distinct from the storage of attributes:
public function getPriceAttribute($)
{
return $this->prices()->first() * 2;
}
If you buy into this reasoning, it creates a logical separation that supports unit tests. prices() can be mocked to returns a collection of 0, 1 & many (2) results. This test can be executed as a unit tests (for orders of magnitude faster test execution (ie on the order of 1ms vs potentially 10s or 100s of ms talking to a local DB)
I am not familiar with php test ecosystem but one way to do this could be with a test specific subclass (not sure if the following is valid PHP :p ):
class PricedModel extends YourModel {
function __construct($stub_prices_supporting_first) {
$this->stub_prices = $stub_prices_supporting_first;
}
public function prices() {
return $this->stub_prices;
}
}
tests
function test_priced_model_0_prices() {
p = new PricedModel(new Prices(array()));
assert.equal(null, p.getPriceAttribute());
}
function test_priced_model_1_price() {
p = new PricedModel(new Prices(array(1)));
assert.equal(2, p.getPriceAttribute());
}
function test_priced_model_2_prices() {
p = new PricedModel(new Prices(array(5, 1)));
assert.equal(10, p.getPriceAttribute());
}
The above should hopeuflly allow you to fully control input into the getPriceAttribute method to support direct IO-free unit testing.
——
Also all the unit tests above can tell you is that you’re able to process prices correctly , it doesn’t price any feedback on if you’re able to query prices !
What distinguishes the tests is their respective goal:
Unit-testing aims at findings those bugs that can be found in isolated small parts of the software. (Note that this does not say you must isolate - it only means your focus is on the isolated code. Isolation and mocking often enough are not needed to reach this goal: Think of a call to a sin function - you almost never need to mock this, you let your system under test just call the original one.)
Integration testing aims at findings bugs in the interaction of two or more components, for example mutual misconceptions about an interface. These bugs can not be found in the isolated software: If you test code in isolation, you also write your tests on your (possibly wrong) understanding of the other components.
Feature tests as you describe them will then have the goal to find further bugs, which the other tests so far could not detect. One example for such a bug could be, that an old version of the feature was integrated (which was correct at that time, but lacked some functionality).
The conclusion, although it may be surprising, is, that it is not in the stricter sense forbidden to make data base accesses in unit-testing. Consider the following scenario: You start writing unit-tests and mock the data base accesses. Later, you realize you can be more lazy and just use the data base without mocking - but otherwise leave all the tests as they are. Your tests have not changed, and they will continue finding the bugs in the isolated code as before. They may run a bit slower now, and the setup may be more complex than with the mocked data base. However, the goal of the test suite was the same - with and without mocking the data base.
This scenario simplifies things a bit, because there may be test cases that can only be done with a mock: For example, testing the case that the data base gets corrupted in a specific way and your code handles this properly. With the real data base such test cases may be practically impossible to set up.
I have an algorithm implemented by a number of classes, all covered by unit test.
I would like to refactor it, which will change behavior of two classes.
When I change one class and its tests, all unit tests pass, though the algorithm becomes incorrect until refactoring is done.
This example illustrates that complete coverage by unit tests is sometimes not enough and I need "integration" tests for the whole algorithm in terms of input-output. Ideally, such tests should cover the behavior of my algorithm completely.
My question: looks like by adding such integration tests I make unit tests unnecessary and superfluous. I don't want to support duplicated test logic.
Should I remove my unit tests or leave them as is, e.g. for easier bug location?
This is part of the problem with tests which are too fine grained and are tightly coupled with the implementation.
Personally I would write tests which focus on the behaviour of the algorithm and would consider this 'a unit'. The fact that it is broken into several classes is an implementation detail, in the same way that breaking down a public method's functionality into several smaller private methods is also an implementation detail. I wouldn't write tests for the private methods separately, they would be tested by the tests of the functionality of the public method.
If some of those classes are generically useful and will be reused elsewhere then I would consider writing unit tests for them at that point as then they will have some defined behaviour on their own.
This would result in some duplication but this is ok as those classes now have a public contract to uphold (and which is used by both components which use it), which those tests can define.
Interestingly, see the definition of Unit in this article
Recently, I got started reading on BDD and TDD and I got hooked. I got lost with the amount of unorganized sources of information and different opinions of what's best and what not. At the end I settled on xBehave & xUnit. I like the fluent syntax and the ease of defining the behaviors with Fluent Assertions and Fluent Validation.
I'm also trying to implement the onion architecture with a test project I'm working on for learning. Here's my scenario: The project, to make it simple, is a product tracker. I can create products and track who owns it. I want to implement two specs:
when a new product is created without a name then an error should be displayed
when a new product is created without an owner assigned then an error should be displayed.
I created the spec which instantiated a new Product and a new ProductService which in turns creates the Product. The spec passes and the validation is occuring now the question is:
How do I test my ProductRepository class? Do I test it next or mock it and finish all specs first then come back and test repository classes?
Should I have mocked the ProductService class in the first spec?
Is that done at the unit test level? should i create a unit test class?
Wouldn't testing the repository make it an integration test?
so far, I don't have a UI and i'm writing my specs for the domain, service, and infrastructure layers.
do i need to use watin for my UI tests?
would switching to watin/specflow makes more sense and would save on efforts to have fully tested layers from top to bottom?
Here's one of the specs I worked on:
[Scenario]
public void creating_new_product_without_a_name_should_throw_error()
{
var productService = default(IProductService);
var action = default(Action);
_
.Given("a new product", () =>
productService = new ProductService() as IProductService)
.When("creating the new product without a name", () =>
action = () => productService.Create(new Product()))
.Then("it should should display an error", () =>
action.ShouldThrow<ValidationException("Name is required."));
}
Thank you for your reply in advance and, please, if you are answering this thread back up with some materials/articles/sample code on to why your suggestion would be better to follow.
It sounds like you are testing small parts and then plan to glue them together into something big. This is (IMO, againt) not TDD (and certainly not BDD): you are not letting the tests drive the design/architecture.
To start with, don't think that much about the design. Don't think onion architecture, don't think repositories, don't think services.
Start by writing a test that verifies the whole solution, from end to end. Make that test as small as possible. To start with, just verify that e.g. a label is displayed. Then write the smallest solution that you can think of to make the test pass. Then write another test and the solution for that.
After a while, have a look at the code (production, but also test) to find resposibilities. Is there an embryo of a service in there somewhere? Extract it! But don't do it prematurely. Let the code tell you what it wants to look like.
Start thinking about the domain (Product, Owner, etc) up front and include it early in your code. Wait a little longer with persistance (repositories), but not too long.
Keep testing at this level (end-to-end). Add micro tests when necessary. So my answer to the question "how do I test my ProductRepository/Service class" is a question to you: do you need to? Or is it sufficiently covered by the end-to-end tests? If not, why?
In TDD how should you continue when you know what your final outcome should be, but not the processing steps you need to get there?
For example your class is being passed an object whose API is completely new to you, You know the class has the information you need but you don't know how to retrieve it yet: How would you go about testing this?
Do you just focus on the desired result ignoring the steps?
Edit 1
package com.wesley_acheson.codeReview.annotations;
import com.sun.mirror.apt.AnnotationProcessor;
import com.sun.mirror.apt.AnnotationProcessorEnvironment;
public class AnnotationPresenceWarner implements AnnotationProcessor {
private final AnnotationProcessorEnvironment environment;
public AnnotationPresenceWarner(AnnotationProcessorEnvironment env) {
environment = env;
}
public void process() {
//This is what I'm testing
}
}
I'm trying to test this incomplete class. I want to test I have the right interactions with AnnotationProcessorEnvironment within the process method. However I'm unsure from the API docs what the right interaction is.
This will produce a file that contains details on the occurrence of each annotation within a source tree.
The actual file writing will probably be delegated to another class however. So this class' responsiblity is to create a representation of the annotation occurrences and pass that to whatever classes need to move it.
In non TDD I'd probably invoke a few methods set a breakpoint and see what they return.
Anyway I'm not looking for a solution to this specific example more sometimes you don't know how to get from A to B and you'd like your code to be test driven.
I'm basing my answer on this video:
http://misko.hevery.com/2008/11/11/clean-code-talks-dependency-injection/
If you have a model/business logic class that's supposed to get some data from a service then I'd go about this way:
Have your model class take the data that it needs in the constructor, rather than the service itself. You could then mock the data and unit test your class.
Create a wrapper for the service, you can then unit test then wrapper.
Perform a fuller test where you actually pass the data from the wrapper to the model class.
General Answer
TDD can be used to solve a number of issues, the first and foremost is to ensure that code changes do not break existing code in regards to their expected behavior. Thus, if you've written a class with TDD, you write some code first, see that it fails, then write the behavior to make it green without causing other tests to become red.
The side-effect of writing the test cases is that now you have Documentation. This means that TDD actually provides answers to two distinct problems with code. When learning a new API, regardless of what it is, you can use TDD to explore it's behavior (granted, in some frameworks this can be very difficult). So, when you are exploring an API, it's ok to write some tests to provide documentation to it's use. You can consider this a prototyping step as well, just that prototyping assumes you throw it away when complete. With the TDD approach, you keep it, so you can always return back to it long after you've learned the API.
Specific Answer to the Example Given
There are a number of approaches which attempt to solve the problem with the AnnotationProcessor. There is an Assertion framework which addresses the issue by loading the java code during the test and asserting the line which the error/warning occurs. And here on Stack overflow
I would create a prototype without the testing to get knowledge of how the api is working. When I got that understanding, I would continue on the TDD cycle on my project
I agree with Bassetassen. First do a spike to understand what is this external API call does and what you need for your method. Once you are comfortable with the API you know how to proceed with TDD.
Never ever Unit Test against an unknown API. Follow the same principle is if you didn't own the code. Isolate all the code you are writing from the unknown or unowned.
Write your unit tests as if the environmental processor was going to be code that you were going to TDD later.
Now you can follow #Tom's advice, except drop step 1. Step 2's unit tests now are just a matter of mapping the outputs of the wrapper class to calls on the API of the unknown. Step two is more along the lines of an integration test.
I firmly believe changing your flow from TDD to Prototyping to TDD is a loss in velocity. Stay with the TDD until you are done, then prototype.
C#, nUnit, and Rhino Mocks, if that turns out to be applicable.
My quest with TDD continues as I attempt to wrap tests around a complicated function. Let's say I'm coding a form that, when saved, has to also save dependent objects within the form...answers to form questions, attachments if available, and "log" entries (such as "blahblah updated the form." or "blahblah attached a file."). This save function also fires off emails to various people depending on how the state of the form changed during the save function.
This means in order to fully test out the form's save function with all of its dependencies, I have to inject five or six data providers to test out this one function and make sure everything fired off in the right way and order. This is cumbersome when writing the multiple chained constructors for the form object to insert the mocked providers. I think I'm missing something, either in the way of refactoring or simply a better way to set the mocked data providers.
Should I further study refactoring methods to see how this function can be simplified? How's the observer pattern sound, so that the dependent objects detect when the parent form is saved and handle themselves? I know that people say to split out the function so it can be tested...meaning I test out the individual save functions of each dependent object, but not the save function of the form itself, which dictates how each should save themselves in the first place?
First, if you are following TDD, then you don't wrap tests around a complicated function. You wrap the function around your tests. Actually, even that's not right. You interweave your tests and functions, writing both at almost exactly the same time, with the tests just a little ahead of the functions. See The Three Laws of TDD.
When you follow these three laws, and are diligent about refactoring, then you never wind up with "a complicated function". Rather you wind up with many, tested, simple functions.
Now, on to your point. If you already have "a complicated function" and you want to wrap tests around it then you should:
Add your mocks explicitly, instead of through DI. (e.g. something horrible like a 'test' flag and an 'if' statement that selects the mocks instead of the real objects).
Write a few tests in order to cover the basic operation of the component.
Refactor mercilessly, breaking up the complicated function into many little simple functions, while running your cobbled together tests as often as possible.
Push the 'test' flag as high as possible. As you refactor, pass your data sources down to the small simple functions. Don't let the 'test' flag infect any but the topmost function.
Rewrite tests. As you refactor, rewrite as many tests as possible to call the simple little functions instead of the big top-level function. You can pass your mocks into the simple functions from your tests.
Get rid of the 'test' flag and determine how much DI you really need. Since you have tests written at the lower levels that can insert mocks through areguments, you probably don't need to mock out many data sources at the top level anymore.
If, after all this, the DI is still cumbersome, then think about injecting a single object that holds references to all your data sources. It's always easier to inject one thing rather than many.
Use an AutoMocking container. There is one written for RhinoMocks.
Imagine you have a class with a lot of dependencies injected via constructor injection. Here's what it looks like to set it up with RhinoMocks, no AutoMocking container:
private MockRepository _mocks;
private BroadcastListViewPresenter _presenter;
private IBroadcastListView _view;
private IAddNewBroadcastEventBroker _addNewBroadcastEventBroker;
private IBroadcastService _broadcastService;
private IChannelService _channelService;
private IDeviceService _deviceService;
private IDialogFactory _dialogFactory;
private IMessageBoxService _messageBoxService;
private ITouchScreenService _touchScreenService;
private IDeviceBroadcastFactory _deviceBroadcastFactory;
private IFileBroadcastFactory _fileBroadcastFactory;
private IBroadcastServiceCallback _broadcastServiceCallback;
private IChannelServiceCallback _channelServiceCallback;
[SetUp]
public void SetUp()
{
_mocks = new MockRepository();
_view = _mocks.DynamicMock<IBroadcastListView>();
_addNewBroadcastEventBroker = _mocks.DynamicMock<IAddNewBroadcastEventBroker>();
_broadcastService = _mocks.DynamicMock<IBroadcastService>();
_channelService = _mocks.DynamicMock<IChannelService>();
_deviceService = _mocks.DynamicMock<IDeviceService>();
_dialogFactory = _mocks.DynamicMock<IDialogFactory>();
_messageBoxService = _mocks.DynamicMock<IMessageBoxService>();
_touchScreenService = _mocks.DynamicMock<ITouchScreenService>();
_deviceBroadcastFactory = _mocks.DynamicMock<IDeviceBroadcastFactory>();
_fileBroadcastFactory = _mocks.DynamicMock<IFileBroadcastFactory>();
_broadcastServiceCallback = _mocks.DynamicMock<IBroadcastServiceCallback>();
_channelServiceCallback = _mocks.DynamicMock<IChannelServiceCallback>();
_presenter = new BroadcastListViewPresenter(
_addNewBroadcastEventBroker,
_broadcastService,
_channelService,
_deviceService,
_dialogFactory,
_messageBoxService,
_touchScreenService,
_deviceBroadcastFactory,
_fileBroadcastFactory,
_broadcastServiceCallback,
_channelServiceCallback);
_presenter.View = _view;
}
Now, here's the same thing with an AutoMocking container:
private MockRepository _mocks;
private AutoMockingContainer _container;
private BroadcastListViewPresenter _presenter;
private IBroadcastListView _view;
[SetUp]
public void SetUp()
{
_mocks = new MockRepository();
_container = new AutoMockingContainer(_mocks);
_container.Initialize();
_view = _mocks.DynamicMock<IBroadcastListView>();
_presenter = _container.Create<BroadcastListViewPresenter>();
_presenter.View = _view;
}
Easier, yes?
The AutoMocking container automatically creates mocks for every dependency in the constructor, and you can access them for testing like so:
using (_mocks.Record())
{
_container.Get<IChannelService>().Expect(cs => cs.ChannelIsBroadcasting(channel)).Return(false);
_container.Get<IBroadcastService>().Expect(bs => bs.Start(8));
}
Hope that helps. I know my testing life has been made a whole lot easier with the advent of the AutoMocking container.
You're right that it can be cumbersome.
Proponent of mocking methodology would point out that the code is written improperly to being with. That is, you shouldn't be constructing dependent objects inside this method. Rather, the injection API's should have functions that create the appropriate objects.
As for mocking up 6 different objects, that's true. However, if you also were unit-testing those systems, those objects should already have mocking infrastructure you can use.
Finally, use a mocking framework that does some of the work for you.
I don't have your code, but my first reaction is that your test is trying to tell you that your object has too many collaborators. In cases like this, I always find that there's a missing construct in there that should be packaged up into a higher level structure. Using an automocking container is just muzzling the feedback you're getting from your tests. See http://www.mockobjects.com/2007/04/test-smell-bloated-constructor.html for a longer discussion.
In this context, I usually find statements along the lines of "this indicates that your object has too many dependencies" or "your object has too many collaborators" to be a fairly specious claim. Of course a MVC controller or a form is going to be calling lots of different services and objects to fulfill its duties; it is, after all, sitting at the top layer of the application. You can smoosh some of these dependencies together into higher-level objects (say, a ShippingMethodRepository and a TransitTimeCalculator get combined into a ShippingRateFinder), but this only goes so far, especially for these top-level, presentation-oriented objects. That's one less object to mock, but you've just obfuscated the actual dependencies via one layer of indirection, not actually removed them.
One blasphemous piece of advice is to say that if you are dependency injecting an object and creating an interface for it that is quite unlikely to ever change (Are you really going to drop in a new MessageBoxService while changing your code? Really?), then don't bother. That dependency is part of the expected behavior of the object and you should just test them together since the integration test is where the real business value lies.
The other blasphemous piece of advice is that I usually see little utility in unit testing MVC controllers or Windows Forms. Everytime I see someone mocking the HttpContext and testing to see if a cookie was set, I want to scream. Who cares if the AccountController set a cookie? I don't. The cookie has nothing to do with treating the controller as a black box; an integration test is what is needed to test its functionality (hmm, a call to PrivilegedArea() failed after Login() in the integration test). This way, you avoid invalidating a million useless unit tests if the format of the login cookie ever changes.
Save the unit tests for the object model, save the integration tests for the presentation layer, and avoid mock objects when possible. If mocking a particular dependency is hard, it's time to be pragmatic: just don't do the unit test and write an integration test instead and stop wasting your time.
The simple answer is that code that you are trying to test is doing too much. I think sticking to the Single Responsibility Principle might help.
The Save button method should only contain a top-level calls to delegate things to other objects. These objects can then be abstracted through interfaces. Then when you test the Save button method, you only test the interaction with mocked objects.
The next step is to write tests to these lower-level classes, but thing should get easier since you only test these in isolation. If you need a complex test setup code, this is a good indicator of a bad design (or a bad testing approach).
Recommended reading:
Clean Code: A Handbook of Agile Software Craftsmanship
Google's guide to writing testable code
Constructor DI isn't the only way to do DI. Since you're using C#, if your constructor does no significant work you could use Property DI. That simplifies things greatly in terms of your object's constructors at the expense of complexity in your function. Your function must check for the nullity of any dependent properties and throw InvalidOperation if they're null, before it begins work.
When it is hard to test something, it is usually symptom of the code quality, that the code is not testable (mentioned in this podcast, IIRC). The recommendation is to refactor the code so that the code will be easy to test. Some heuristics for deciding how to split the code into classes are the SRP and OCP. For more specific instructions, it would be necessary to see the code in question.