Related
I am quite new to TDD and the first question which came into my mind is whether I should apply unit tests to every developed component. I am asking it since I observed that unit testing takes a lot of time, especially when some changes into the requirements are provided. So, could you suggest something like best practices in TDD regarding unit testing?
A short description of TDD expressed in three rules:
You are not allowed to write any production code unless it is to make a failing unit test pass.
You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
And a longer description: http://jamesshore.com/Agile-Book/test_driven_development.html
And a more in-depth description: http://www.growing-object-oriented-software.com/
Just an observation - you said
I observed that unit testing takes a
lot of time
which is true in the short view. But for code that's going to be around a while, the extra work saves time. I stressed the value of unit testing on one project I was on about years ago, telling everyone that would listen, "We don't have time to skip that step." And it's true. There are many places that time will be saved - testers will spend less time testing the app, kicking bugs back through whatever bug-tracking process you use, you'll spend less time remembering what you did weeks or months later so you can fix the bug, users will see fewer bugs, meaning they will spend less time yelling at you for broken apps. You'll spend less time on the phone at 2 AM hoping you can get the app fixed before the users come in the next day.
It all comes to to economics, in a way. For any code that's going in to production, trust me, you don't have time to skip that step. It'll cost you more time and your company more cash.
If the code's not going to prod, that is, it's a utility you wrote to help with some task, or see how the network layer really works, you want to adjust the amount of testing you do to suit the need. Experience will help guide you there to know the right amount.
TDD means that the tests drive the development of your components. You start by writing a unit test that specifies the behavior of the component, then you implement the component. So to answer your question:
should apply unit tests to every
developed component
No, because the unit tests should already be written before developing the component.
The tests drive the development of your code. TDD is really all about defining the desired behaviour of your software, the fact that's it's all testable is just a good side effect.
A good essay on TDD is available here
I don't know so much about Test-Driven Development (TDD), but I always hear that i need to start the development with some test cases. Then, I need to make this tests pass with the most simple solution. And then create more tests to make my tests fail again...
But the question is: When stop creating new tests? When I know that my application is in agreement with the requirements?
Shamelessly copying Kent Beck's answer to this question.
I get paid for code that works, not
for tests, so my philosophy is to test
as little as possible to reach a given
level of confidence (I suspect this
level of confidence is high compared
to industry standards, but that could
just be hubris). If I don't typically
make a kind of mistake (like setting
the wrong variables in a constructor),
I don't test for it. I do tend to make
sense of test errors, so I'm extra
careful when I have logic with
complicated conditionals. When coding
on a team, I modify my strategy to
carefully test code that we,
collectively, tend to get wrong.
Different people will have different
testing strategies based on this
philosophy, but that seems reasonable
to me given the immature state of
understanding of how tests can best
fit into the inner loop of coding. Ten
or twenty years from now we'll likely
have a more universal theory of which
tests to write, which tests not to
write, and how to tell the difference.
In the meantime, experimentation seems
in order.
Code coverage tools can provide useful information about how well tested your code is. Such tools will identify code paths that have not been exercised by your tests.
In TDD, you stop writing tests when you stop writing code (or just so slightly before the last code is written), unless (as mentioned), your code coverage is too low.
Lifecycle
If you follow Test Driven Development to the letter, you have a 5 step cycle:
Write a test: for each unit (the smallest piece of code you can test) you write a test, where you determine what that unit will be responsible for. You need to follow the so called Right-BICEP checklist (right results, boundary conditions, inverse relationships, cross-check results, error conditions, performance characteristics).
Run tests and see them fail: in this step the newly written tests should fail. This is the so called red step, as the unit tests should show up in red. If the tests do not fail, you probably didn't write them correct.
Implement unit: write the code, even if you hardcode it, the point of this step is to get to the next green step.
Run the tests and see them pass: the green step as all the test should pass. If they don't you're not done with writing code.
Done? No, refactor!
(source: wikimedia.org)
What to test
Test all units until you reach complete code coverage (wishful thinking in most cases, you would have to have a unit test for severe fail scenarios like tripping over the power cable, no more disk space, flood etc). If you reach the 90% ballpark you're more than done.
If you find a bug in your code, create a unit test and fix the code. Repeat.
If your code has a GUI try any automated functional testing you can find. In my case Selenium or JMeter would do the trick. Selenium is a good tool as it allows you to record your tests with Firefox and them replay them on demand.
Continuous integration
Because running all the tests all the time is time consuming, you can delegate most of this mundane tasks to a continuous integration server that will do them for you at predefined time intervals. This does not mean that you do not have to run tests before you commit your code. You still need to run the tests for the part of the system you were fixing, if the system is large running all unit tests would be counterproductive. The CI server would inform you of any failures and you would need to buy drinks for all of your colleagues on top of fixing the code you broke ;)
You stop writing tests when you have no more functionality to add to your code. There may be some additional edge cases you want to make sure are covered, but beyond that, when you don't have anything more to have your code do, you don't have any more TDD tests to write (Acceptance and QA tests are a different story).
There are certain areas you may find difficult to test, such as gui and data access but apart from that you write test until you objectives are met.
In an ideal world where I would follow eXtreme Programming practices, (not just TDD) my customer is supposed to provide me with some automated functional test. When such test goes green I stop writing tests and go to my customer to ask for some more functional tests that do not pass (because tests are specification and if my customer does not provide me with failing tests I won't know what to do)
I could explain it another way, aimed at a more prectical world. At XP France we organize TDD Dojo's on a regular basis (once a week). You could call taht TDD training sessions. There we use to practice TDD on some toy problems. When doing this the idea is to propose a test that fail, then write code to make it pass. Never propose a test that works without coding.
Whoever propose a test that goes green without any code should pay beers to others. So that's a way to know it's time to stop testing: when you are not able any more to write tests that fails you are finished. (Anyway coding after drinking is bad practice).
Can TDD be successful as a defect-reduction strategy without incorporating guidance on test case construction and evaluation?
IMO, my answer would be no. For TDD to be effective, there has to be guidelines around what is a test and what it means to have something be reasonably tested. Without a guideline, there may be some developers that end up with tons of defects because their initial tests cover a very small set of inputs,e.g. only the valid ones, which can cause the idea of using TDD to become worthless.
Test driven development can reduce defects in a QA cycle simply because testing allows developers to find defects prior to releasing their code to the QA team.
But without guidance on how to test you really won't be able to create any kind of long-term benefit since haphazard testing will only prevent defects by blind luck. Good tests based on good guidance can go a long way towards reducing defects.
if you don't have tests to reproduce defects, how do you know that "defect reduction" has taken place?
of course you do have tests - they're just manual, and thus tedious and time-consuming to reproduce...
Here's a study (warning: link to PDF file) done by microsoft on some of their internal teams.
A quote from it:
The results of the case studies indicate that the pre-release defect density of the four products decreased between 40% and 90% relative to similar projects that did not use the TDD practice. Subjectively, the teams experienced a 15–35% increase in initial development time after adopting TDD
That's the only actual empirical study done on TDD/Unit testing that I'm aware of, but there are plenty of people (including myself) that will anecdotally tell you that TDD (and unit testing in general) will definitely provide an increase in the quality of your code.
From my own experience, there is definitely a reduction in the number of defects, but the numbers feel like they would be far less than even the 40% from the Microsoft study; This is (again, based solely on what I've seen) primarily because most corporate developers have little to no experience with Unit Testing (let alone TDD), and will invariably do a bad job of it while they are learning. Actually learning how to do TDD well requires at least a solid year of experience, and I've never worked in (or even seen) a team which actually had a full complement of developers with that experience.
You may want to pickup a copy of Gerard Meszaros' xUnit Test Patterns. Specifically, Chapter 5 might apply most directly to your question where it covers the Principles of Test Automation. Some of those principles that I think apply to your scenario where there needs to be some sort of guidance, common interest, or some sort of implied do this or fear the wrath of :
Principle: Communicate Intent
Tests need to be easy to maintain, readily apparent what the test is doing.
Principle: Keep Tests Independent
Small tests that cover one small piece. Tests should not interfere with each other.
Principle: Minimize Test Overlap
Need to design tests that cover a specific piece, and do not create tests that exercise the same paths repeatedly.
Principle: Verify One Condition Per Test
This one seemed simple enough for me, but is one of the most challenging in my experiences for people to grasp. I may write tests that have some multiple asserts, but I try to keep all those together around the specific area. When it comes to hunting down failures and other test issues, it is MUCH easier to fiddle with a single test that is testing a specific path instead of several different paths all clumped into a single test method.
Further relating to my experiences, if we want to talk about the corporate developer, I have seen some folks that are interested and take the initiative to learn something new, but more often than not, you have folks that like to go with the flow, and like to have things laid out for them. Without some sort of direction, be it a mandate from a senior engineer-type, or some sort of joint-team discussions (see Practices of an Agile Developer for some ideas such as lunch time meetings once a week), I think your chance of success would be limited.
In a team situation, where your code is likely to be used by someone else, the tests have a fringe benefit that can reduce defects without necessarily even improving anyone's code.
Where documentation is poor (which during development is "often"), the tests act a crib for how you expect your code to be called. So, even in cases where the code is really very fragile, TDD can still reduce the number of defects raised against the end-product -- by ensuring your colleagues can see well-written tests before they can use your code, you've ensured they know how you intend your code to be used before they call it. They are thus less-likely to call your code in an unexpected sequence / without having configured something you expected (but forgot to write a check for) as a prerequisite. Thus they are less likely to trigger the failure condition, and you are less likely to see them or the (human) test team raising a defect because something crashed.
Of course, whether you see that "there's a hidden bug in there, it's just not being called yet" as a problem itself is another good question.
I'm in the process of doing a study on Test-Driven Development and one of the discussion points is the "Barrier to Entry" associated with TDD. Does anyone have any experience around this area, on any projects you've worked on that decided not to use TDD because the barrier to entry was too high?
From what I can tell the only barrier to entry is knowledge (and as such experience) of individual developers, with most not being entirely accustomed to the process and it being slightly alien. Financially it seems to be very appealing given most of the market leading tools are open source, freely available, well documented and well supported.
Thoughts/feelings appreciated.
Thanks,
EDIT - does anyone know of any high profile quotes of people advocating TDD? Would love to see how high it goes up the chain. Cheers.
Some barriers include:
An existing code base which doesn't lend itself to unit testing.
A problem domain that is hard to unit test meaningfully, such as GUI work or integrations with third party systems.
A perception of integration problems over unit problems (in other words, if it doesn't work end to end it doesn't do anything, so what is the point of testing the unit).
A mindset that wants to design ahead of time and have a clear system design rather than have tests drive design
A political culture where design is done by a different person/group than development, and that design is not unit-test friendly.
An inability to get over the fact that TDD is not about testing for conformance (arguments like "the one who writes the tests shouldn't be the one who codes it, they will be too lenient on themselves" and such variants).
It isn't they way they have coded until now, so the shift is harder.
Sometimes a certain test can be hard to set up, so the method will get abandoned because it "feels" slower.
Design requirements that don't lend themselves to evolving design well or at all (think Nuclear Plant control software or other systems were actual lives depend on their functioning correctly).
If everyone isn't running the test before checking in code, tests start to break often for wrong reasons (that is the intended behavior of the code changed, but the test didn't keep up, so the test is wrong, not the code) so they can be perceived as a drag.
In terms of barriers to entry, effectively, because you are explicitly writing tests that must pass before code is considered to be complete, the lead time in the dev cycle involved in getting functional code is longer. Now, when using TDD, you're effectively guaranteeing a certain level of quality on the code (whatever level of quality you choose to test against) and so that is generally more than enough compensation for the lag in lead time, but strictly speaking, there IS a greater lead time to getting functional code using TDD.
Effectively, if you have coders that write bug-free code, TDD will be a drag on your development cycle. The value of TDD, of course, is that there aren't any coders who can always write bug-free code, and so the cost of fixing bugs has to be factored in somewhere; in TDD, the cost of the test infrastructure is front-loaded.
Note that this is not in any way a negative thing about TDD; I'm just saying, that front-loading COULD be considered to be a "barrier to entry". Personally, as a coder, I would say that the Return on Investment is more than worth the effort, and I think most experienced dev managers would as well.
Team and/or management buy-in is the biggest obstacle in some companies. If you're the lone developer trying to use TDD and you can't get others on the project interested, it can be very frustrating.
Of course that's not a financial barrier at all. The biggest perceived financial barrier is probably time. If you have a large code base that you need to write unit tests for, it can seem quite daunting. Your manager (or someone above them) will question why you want to spend time writing code that will not add features/functionality to the code. Many people don't realize that writing the tests up front (as you do in TDD) can actually save you time, both immediately and in the long run when you're maintaining that code.
I think one major barrier is how it requires you to change the way you think.
Before I tried TDD, I would create a class, say Employee, then I would stub in things like FirstName, LastName, Email, etc. Then I would write some logic and forget that I missed a few fields or something else. And before I knew it I had a pretty complex class without knowing if those fields were ever necessary.
Also, it's a complete change from how we are used to writing software. We are used to writing software as we receive features from the guys who sign our checks. We are not used to writing code which doesn't compile, making it compile, then making it work to make our tests pass.
The first time you do this, you feel a bit.. well silly and stupid. Why am I making my code intentionally fail? It seems illogical to the "make it work" philosophy we've all been taught for so long.
A few reasons why it has failed so far where I work:
Most of the project at work on are older apps. Not pre-.NET but, .NET 2.0 and in some cases .NET 1.0.
Some of these projects are not well factored, either because the technology wasn't there in 1.0, or it was built quickly because they needed something NOW..
As Jon pointed out, some things are still a PIA (pain-in-the-***) to unit test, UI, database, etc.
Expensive tools. If you are only allowed to Microsoft tools, it's a high price tag to do this the "right way". We use resharper, so it really isn't a problem.
Time. I'm in a team of three guys supporting a department of 30 people. We are considered overhead, and many of our development consists of interfacing systems together
Yes the main barrier to entry is in your head or in the head of other programmers.
In the beginning you don't know what to srite in your tests.
The trick is to think about how your code will be used instead of focusing on how you re going to write it. Easier said than done ...
When you start to "get it", it's a bit hard to know where to stop writing tests.
You have to remember that tests prove nothing so you just can't write tests to covers all cases, you have to select the most useful ones ... and that's already a lot !
I've certainly seen plenty of resistance. The barriers I've encountered are:
Unit testing user interfaces (web or thick client) is tricky. I know there are lots of attempts to solve the problem, but I don't think any of them have made it really simple - because it's a naturally hard problem.
At the other end, although there are various ways of making it easier to test the code involved with the database, it's still tricky and time-consuming.
While good tests definitely speed up development overall, testing is a skill - and while you suck at it, unit testing may well be more trouble than it's worth, which means you never build up the skill...
Managers often see it as an optional extra to development - a nice to have rather than critical. This means it's the first thing to go when the project inevitably has a resource squeeze.
I wrote a long-ish article about this a few weeks back, "Why I write tests first".
I think the biggest barrier is building the discipline to start with tests first, but I don't believe the TDD (or any practice for that matter) should be approached as an always, absolutely, 100% of the time solution.
TDD is a tool in each developer's arsenal. I tend to think it works well for me most of the time. A developer that isn't as accustomed to writing tests (first or otherwise) may it difficult to get anything done if TDD is forced on them because they can't think in terms of writing the test first.
I consider myself an experienced test-writer, but I can't always think in terms of tests. Some problems don't lend themselves well to it, or at least my head doesn't get wrapped around it some days. And some types of code (such as UI and client-side code) doesn't lend itself well to always writing tests.
If you have a team of developers that do not write tests as a matter of habit, I'd push that first. I have no problem requiring that all new code have accompanying unit tests where possible/practical. Once testing is a discipline, converting developers to TDD individually or as a team is much easier.
One non-obvious barrier (non-obvious to me, at least) is the build infrastructure. If developers don't have control over the build process, or if the infrastructure is too baroque to be manageable, then integrating tests into the build process is going to be shunted to the side in the name of "efficiency". (Of course, in these situations it's the build infrastructure that should be shunted aside in the name of efficiency.)
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I was watching Rob Connerys webcasts on the MVCStoreFront App, and I noticed he was unit testing even the most mundane things, things like:
public Decimal DiscountPrice
{
get
{
return this.Price - this.Discount;
}
}
Would have a test like:
[TestMethod]
public void Test_DiscountPrice
{
Product p = new Product();
p.Price = 100;
p.Discount = 20;
Assert.IsEqual(p.DiscountPrice,80);
}
While, I am all for unit testing, I sometimes wonder if this form of test first development is really beneficial, for example, in a real process, you have 3-4 layers above your code (Business Request, Requirements Document, Architecture Document), where the actual defined business rule (Discount Price is Price - Discount) could be misdefined.
If that's the situation, your unit test means nothing to you.
Additionally, your unit test is another point of failure:
[TestMethod]
public void Test_DiscountPrice
{
Product p = new Product();
p.Price = 100;
p.Discount = 20;
Assert.IsEqual(p.DiscountPrice,90);
}
Now the test is flawed. Obviously in a simple test, it's no big deal, but say we were testing a complicated business rule. What do we gain here?
Fast forward two years into the application's life, when maintenance developers are maintaining it. Now the business changes its rule, and the test breaks again, some rookie developer then fixes the test incorrectly...we now have another point of failure.
All I see is more possible points of failure, with no real beneficial return, if the discount price is wrong, the test team will still find the issue, how did unit testing save any work?
What am I missing here? Please teach me to love TDD, as I'm having a hard time accepting it as useful so far. I want too, because I want to stay progressive, but it just doesn't make sense to me.
EDIT: A couple people keep mentioned that testing helps enforce the spec. It has been my experience that the spec has been wrong as well, more often than not, but maybe I'm doomed to work in an organization where the specs are written by people who shouldn't be writing specs.
First, testing is like security -- you can never be 100% sure you've got it, but each layer adds more confidence and a framework for more easily fixing the problems that remain.
Second, you can break tests into subroutines which themselves can then be tested. When you have 20 similar tests, making a (tested) subroutine means your main test is 20 simple invocations of the subroutine which is much more likely to be correct.
Third, some would argue that TDD addresses this concern. That is, if you just write 20 tests and they pass, you're not completely confident that they are actually testing anything. But if each test you wrote initially failed, and then you fixed it, then you're much more confident that it's really testing your code. IMHO this back-and-forth takes more time than it's worth, but it is a process that tries to address your concern.
A test being wrong is unlikely to break your production code. At least, not any worse than having no test at all. So it's not a "point of failure": the tests don't have to be correct in order for the product to actually work. They might have to be correct before it's signed off as working, but the process of fixing any broken tests does not endanger your implementation code.
You can think of tests, even trivial tests like these, as being a second opinion what the code is supposed to do. One opinion is the test, the other is the implementation. If they don't agree, then you know you have a problem and you look closer.
It's also useful if someone in future wants to implement the same interface from scratch. They shouldn't have to read the first implementation in order to know what Discount means, and the tests act as an unambiguous back-up to any written description of the interface you may have.
That said, you're trading off time. If there are other tests you could be writing using the time you save skipping these trivial tests, maybe they would be more valuable. It depends on your test setup and the nature of the application, really. If the Discount is important to the app, then you're going to catch any bugs in this method in functional testing anyway. All unit testing does is let you catch them at the point you're testing this unit, when the location of the error will be immediately obvious, instead of waiting until the app is integrated together and the location of the error might be less obvious.
By the way, personally I wouldn't use 100 as the price in the test case (or rather, if I did then I'd add another test with another price). The reason is that someone in future might think that Discount is supposed to be a percentage. One purpose of trivial tests like this is to ensure that mistakes in reading the specification are corrected.
[Concerning the edit: I think it's inevitable that an incorrect specification is a point of failure. If you don't know what the app is supposed to do, then chances are it won't do it. But writing tests to reflect the spec doesn't magnify this problem, it merely fails to solve it. So you aren't adding new points of failure, you're just representing the existing faults in code instead of waffle documentation.]
All I see is more possible points of failure, with no real beneficial return, if the discount price is wrong, the test team will still find the issue, how did unit testing save any work?
Unit testing isn't really supposed to save work, it's supposed to help you find and prevent bugs. It's more work, but it's the right kind of work. It's thinking about your code at the lowest levels of granularity and writing test cases that prove that it works under expected conditions, for a given set of inputs. It's isolating variables so you can save time by looking in the right place when a bug does present itself. It's saving that suite of tests so that you can use them again and again when you have to make a change down the road.
I personally think that most methodologies are not many steps removed from cargo cult software engineering, TDD included, but you don't have to adhere to strict TDD to reap the benefits of unit testing. Keep the good parts and throw out the parts that yield little benefit.
Finally, the answer to your titular question "How do you unit test a unit test?" is that you shouldn't have to. Each unit test should be brain-dead simple. Call a method with a specific input and compare it to its expected output. If the specification for a method changes then you can expect that some of the unit tests for that method will need to change as well. That's one of the reasons that you do unit testing at such a low level of granularity, so only some of the unit tests have to change. If you find that tests for many different methods are changing for one change in a requirement, then you may not be testing at a fine enough level of granularity.
Unit tests are there so that your units (methods) do what you expect. Writing the test first forces you to think about what you expect before you write the code. Thinking before doing is always a good idea.
Unit tests should reflect the business rules. Granted, there can be errors in the code, but writing the test first allows you to write it from the perspective of the business rule before any code has been written. Writing the test afterwards, I think, is more likely to lead to the error you describe because you know how the code implements it and are tempted just to make sure that the implementation is correct -- not that the intent is correct.
Also, unit tests are only one form -- and the lowest, at that -- of tests that you should be writing. Integration tests and acceptance tests should also be written, the latter by the customer, if possible, to make sure that the system operates the way it is expected. If you find errors during this testing, go back and write unit tests (that fail) to test the change in functionality to make it work correctly, then change your code to make the test pass. Now you have regression tests that capture your bug fixes.
[EDIT]
Another thing that I have found with doing TDD. It almost forces good design by default. This is because highly coupled designs are nearly impossible to unit test in isolation. It doesn't take very long using TDD to figure out that using interfaces, inversion of control, and dependency injection -- all patterns that will improve your design and reduce coupling -- are really important for testable code.
How does one test a test? Mutation testing is a valuable technique that I have personally used to surprisingly good effect. Read the linked article for more details, and links to even more academic references, but in general it "tests your tests" by modifying your source code (changing "x += 1" to "x -= 1" for example) and then rerunning your tests, ensuring that at least one test fails. Any mutations that don't cause test failures are flagged for later investigation.
You'd be surprised at how you can have 100% line and branch coverage with a set of tests that look comprehensive, and yet you can fundamentally change or even comment out a line in your source without any of the tests complaining. Often this comes down to not testing with the right inputs to cover all boundary cases, sometimes it's more subtle, but in all cases I was impressed with how much came out of it.
When applying Test-Driven Development (TDD), one begins with a failing test. This step, that might seem unecessary, actually is here to verify the unit test is testing something. Indeed, if the test never fails, it brings no value and worse, leads to wrong confidence as you'll rely on a positive result that is not proving anything.
When following this process strictly, all ''units'' are protected by the safety net the unit tests are making, even the most mundane.
Assert.IsEqual(p.DiscountPrice,90);
There is no reason the test evolves in that direction - or I'm missing something in your reasoning. When the price is 100 and the discount 20, the discount price is 80. This is like an invariant.
Now imagine your software needs to support another kind of discount based on percentage, perhaps depending on the volume bought, your Product::DiscountPrice() method may become more complicated. And it is possible that introducing those changes breaks the simple discount rule we had initially. Then you'll see the value of this test which will detect the regression immediately.
Red - Green - Refactor - this is to remember the essence of the TDD process.
Red refers to JUnit red bar when a tests fails.
Green is the color of JUnit progress bar when all tests pass.
Refactor under green condition: remove any dupliation, improve readability.
Now to address your point about the "3-4 layers above the code", this is true in a traditional (waterfall-like) process, not when the development process is agile. And agile is the world where TDD is coming from ; TDD is the cornerstone of eXtreme Programming.
Agile is about direct communication rather than thrown-over-the-wall requirement documents.
While, I am all for unit testing, I
sometimes wonder if this form of test
first development is really beneficial...
Small, trivial tests like this can be the "canary in the coalmine" for your codebase, alerting of danger before it's too late. The trivial tests are useful to keep around because they help you get the interactions right.
For example, think about a trivial test put in place to probe how to use an API you're unfamiliar with. If that test has any relevance to what you're doing in the code that uses the API "for real" it's useful to keep that test around. When the API releases a new version and you need to upgrade. You now have your assumptions about how you expect the API to behave recorded in an executable format that you can use to catch regressions.
...[I]n a real process, you have 3-4
layers above your code (Business
Request, Requirements Document,
Architecture Document), where the
actual defined business rule (Discount
Price is Price - Discount) could be
misdefined. If that's the situation,
your unit test means nothing to you.
If you've been coding for years without writing tests it may not be immediately obvious to you that there is any value. But if you are of the mindset that the best way to work is "release early, release often" or "agile" in that you want the ability to deploy rapidly/continuously, then your test definitely means something. The only way to do this is by legitimizing every change you make to the code with a test. No matter how small the test, once you have a green test suite you're theoretically OK to deploy. See also "continuous production" and "perpetual beta."
You don't have to be "test first" to be of this mindset, either, but that generally is the most efficient way to get there. When you do TDD, you lock yourself into small two to three minute Red Green Refactor cycle. At no point are you not able to stop and leave and have a complete mess on your hands that will take an hour to debug and put back together.
Additionally, your unit test is another
point of failure...
A successful test is one that demonstrates a failure in the system. A failing test will alert you to an error in the logic of the test or in the logic of your system. The goal of your tests is to break your code or prove one scenario works.
If you're writing tests after the code, you run the risk of writing a test that is "bad" because in order to see that your test truly works, you need to see it both broken and working. When you're writing tests after the code, this means you have to "spring the trap" and introduce a bug into the code to see the test fail. Most developers are not only uneasy about this, but would argue it is a waste of time.
What do we gain here?
There is definitely a benefit to doing things this way. Michael Feathers defines "legacy code" as "untested code." When you take this approach, you legitimize every change you make to your codebase. It's more rigorous than not using tests, but when it comes to maintaining a large codebase, it pays for itself.
Speaking of Feathers, there are two great resources you should check out in regard to this:
Working Effectively with Legacy Code
Brownfield Application Development in .NET
Both of these explain how to work these types of practices and disciplines into projects that aren't "Greenfield." They provide techniques for writing tests around tightly coupled components, hard wired dependencies, and things that you don't necessarily have control over. It's all about finding "seams" and testing around those.
[I]f the discount price is wrong, the
test team will still find the issue,
how did unit testing save any work?
Habits like these are like an investment. Returns aren't immediate; they build up over time. The alternative to not testing is essentially taking on debt of not being able to catch regressions, introduce code without fear of integration errors, or drive design decisions. The beauty is you legitimize every change introduced into your codebase.
What am I missing here? Please teach
me to love TDD, as I'm having a hard
time accepting it as useful so far. I
want too, because I want to stay
progressive, but it just doesn't make
sense to me.
I look at it as a professional responsibility. It's an ideal to strive toward. But it is very hard to follow and tedious. If you care about it, and feel you shouldn't produce code that is not tested, you'll be able to find the will power to learn good testing habits. One thing that I do a lot now (as do others) is timebox myself an hour to write code without any tests at all, then have the discipline to throw it away. This may seem wasteful, but it's not really. It's not like that exercise cost a company physical materials. It helped me to understand the problem and how to write code in such a way that it is both of higher quality and testable.
My advice would ultimately be that if you really don't have a desire to be good at it, then don't do it at all. Poor tests that aren't maintained, don't perform well, etc. can be worse than not having any tests. It's hard to learn on your own, and you probably won't love it, but it is going to be next to impossible to learn if you don't have a desire to do it, or can't see enough value in it to warrant the time investment.
A couple people keep mentioned that
testing helps enforce the spec. It has
been my experience that the spec has
been wrong as well, more often than
not...
A developer's keyboard is where the rubber meets the road. If the spec is wrong and you don't raise the flag on it, then it's highly probable you'll get blamed for it. Or at least your code will. The discipline and rigor involved in testing is difficult to adhere to. It's not at all easy. It takes practice, a lot of learning and a lot of mistakes. But eventually it does pay off. On a fast-paced, quickly changing project, it's the only way you can sleep at night, no matter if it slows you down.
Another thing to think about here is that techniques that are fundamentally the same as testing have been proven to work in the past: "clean room" and "design by contract" both tend to produce the same types of "meta"-code constructs that tests do, and enforce those at different points. None of these techniques are silver bullets, and rigor is going to cost you ultimately in the scope of features you can deliver in terms of time to market. But that's not what it's about. It's about being able to maintain what you do deliver. And that's very important for most projects.
Unit testing works very similar to double entry book keeping. You state the same thing (business rule) in two quite different ways (as programmed rules in your production code, and as simple, representative examples in your tests). It's very unlikely that you make the same mistake in both, so if they both agree with each other, it's rather unlikely that you got it wrong.
How is testing going to be worth the effort? In my experience in at least four ways, at least when doing test driven development:
it helps you come up with a well decoupled design. You can only unit test code that is well decoupled;
it helps you determine when you are done. Having to specify the needed behavior in tests helps to not build functionality that you don't actually need, and determine when the functionality is complete;
it gives you a safety net for refactorings, which makes the code much more amenable to changes; and
it saves you a lot of debugging time, which is horribly costly (I've heard estimates that traditionally, developers spend up to 80% of their time debugging).
Most unit tests, test assumptions. In this case, the discount price should be the price minus the discount. If your assumptions are wrong I bet your code is also wrong. And if you make a silly mistake, the test will fail and you will correct it.
If the rules change, the test will fail and that is a good thing. So you have to change the test too in this case.
As a general rule, if a test fails right away (and you don't use test first design), either the test or the code is wrong (or both if you are having a bad day). You use common sense (and possilby the specs) to correct the offending code and rerun the test.
Like Jason said, testing is security. And yes, sometimes they introduce extra work because of faulty tests. But most of the time they are huge time savers. (And you have the perfect opportunity to punish the guy who breaks the test (we are talking rubber chicken)).
Test everything you can. Even trivial mistakes, like forgetting to convert meters to feet can have very expensive side effects. Write a test, write the code for it to check, get it to pass, move on. Who knows at some point in the future, someone may change the discount code. A test can detect the problem.
I see unit tests and production code as having a symbiotic relationship. Simply put: one tests the other. And both test the developer.
Remember that the cost of fixing defects increases (exponentially) as the defects live through the development cycle. Yes, the testing team might catch the defect, but it will (usually) take more work to isolate and fix the defect from that point than if a unit test had failed, and it will be easier to introduce other defects while fixing it if you don't have unit tests to run.
That's usually easier to see with something more than a trivial example ... and with trivial examples, well, if you somehow mess up the unit test, the person reviewing it will catch the error in the test or the error in the code, or both. (They are being reviewed, right?) As tvanfosson points out, unit testing is just one part of an SQA plan.
In a sense, unit tests are insurance. They're no guarantee that you'll catch every defect, and it may seem at times like you're spending a lot of resources on them, but when they do catch defects that you can fix, you'll be spending a lot less than if you'd had no tests at all and had to fix all defects downstream.
I see your point, but it's clearly overstated.
Your argument is basically: Tests introduce failure. Therefore tests are bad/waste of time.
While that may be true in some cases, it's hardly the majority.
TDD assumes: More Tests = Less Failure.
Tests are more likely to catch points of failure than introduce them.
Even more automation can help here !
Yes, writing unit tests can be a lot of work, so use some tools to help you out.
Have a look at something like Pex, from Microsoft, if you're using .Net
It will automatically create suites of unit tests for you by examining your code. It will come up with tests which give good coverage, trying to cover all paths through your code.
Of course, just by looking at your code it can't know what you were actually trying to do, so it doesn't know if it's correct or not. But, it will generate interesting tests cases for you, and you can then examine them and see if it is behaving as you expect.
If you then go further and write parameterized unit tests (you can think of these as contracts, really) it will generate specific tests cases from these, and this time it can know if something's wrong, because your assertions in your tests will fail.
I've thought a bit about a good way to respond to this question, and would like to draw a parallel to the scientific method. IMO, you could rephrase this question, "How do you experiment an experiment?"
Experiments verify empirical assumptions (hypotheses) about the physical universe. Unit tests will test assumptions about the state or behavior of the code they call. We can talk about the validity of an experiment, but that's because we know, through numerous other experiments, that something doesn't fit. It doesn't have both convergent validity and empirical evidence. We don't design a new experiment to test or verify the validity of an experiment, but we may design a completely new experiment.
So like experiments, we don't describe the validity of a unit test based on whether or not it passes a unit test itself. Along with other unit tests, it describes the assumptions we make about the system it is testing. Also, like experiments, we try to remove as much complexity as we can from what we are testing. "As simple as possible, but no simpler."
Unlike experiments, we have a trick up our sleeve to verify our tests are valid other than just convergent validity. We can cleverly introduce a bug we know should be caught by the test, and see if the test does indeed fail. (If only we could do that in the real world, we'd depend much less on this convergent validity thing!) A more efficient way to do this is watch your test fail before implementing it (the red step in Red, Green, Refactor).
You need to use the correct paradigm when writing tests.
Start by first writing your tests.
Make sure they fail to start off with.
Get them to pass.
Code review before you checkin your code (make sure the tests are reviewed.)
You cant always be sure but they improve overall tests.
Even if you do not test your code, it will surely be tested in production by your users. Users are very creative in trying to crash your soft and finding even non-critical errors.
Fixing bugs in production is much more costly than resolving issues in development phase.
As a side-effect, you will lose income because of an exodus of customers. You can count on 11 lost or not gained customers for 1 angry customer.