I tend to write a lot of applications that, say, will expose API to a logic, which will call other APIs within the same logic, and then persist some form of data to a database at the end. So, in other words, I tend to have a method call that encapsulates calling other APIs, processing data and then persisting to the database.
I'm familiar with TDD (test driven development), but I find it difficult to practice, because the last thing I want is persisting useless data in production database while running mandatory tests, and also calling external APIs that I don't control and messing up their database in turn.
I'm probably looking at it the wrong way.
How do I practice efficient TDD and evade this problem?
How do I practice efficient TDD and evade this problem?
Design.
Roughly, we arrange our code so that it is a composition of modules of two flavors
Modules that are easy to test
Modules that simple and stable
So the code that actually needs to talk to the database should be of the second type; no branching, trivial operations, and most importantly it doesn't need to change very often.
The code that we write that feels inherently risky? that goes into an "easy to test" module. These modules are only loosely coupled to their simpler cousins - it should be easy to substitute a "test" implementation for the "real" one. Thus, instead of talking to "the database" during our testing, we talk to some in memory collection that just gives us scripted responses.
In short, we arrange our code so that it is easy to test where the mistakes are.
As for the hard to test bits; well, perhaps we measure the risks in other ways, or we run those tests at a different cadence, or we defer the testing of them to functional tests, or....
Related
Background:
I'm currently creating my first game engine project for learning purposes. This is the first larger project I've ever done. I knew going in to the project the sort of higher level details of what's involved in a game engine (such as separate systems - input, graphics, audio, physics, etc.) but when getting into the nitty gritty I'm kindof figuring things out as I go. My coding process right now looks like this:
1.) Figure out some quick, higher level details of the system I want to design
2.) Start some experimental coding to see how exactly things need to work
3.) Add tests afterwards once I'm feeling good about what I have.
Since I'm pretty unfamiliar with the problem domain (game engine programming) I find I really need to experiment in code to see what functions I need and what design suites best. However, I know most of the community (so it seems anyway) usually advocates more of a TDD approaches to building applications. I can see the benefit of this but I'm not quite sure how I would apply the "Write test first, fail, then pass" when I'm not really even sure what functions I really need to be testing yet. Even if I can think of 1 or 2 definite functions, what if during the experimenting phase I find it's better to split those functions out into more functions across different classes. Then I would have to redesign both my code and my tests constantly.
My Issue/Question(s):
So, is there a good way to go about using a TDD approach when your experimenting in code? Or is TDD generally meant for those who are familiar with the projects their working on, and have an idea of what the design needs to be or what functions they will be needing to test?
A common approach when using TDD in unfamiliar territory:
Do some experimental coding to learn more
Set aside the experimental code and start the TDD "test first" approach
As you write production code to make your tests pass, you'll be harvesting pieces of your experimental code.
The benefit of this approach is that with the TDD "red,green,refactor" cycle, you'll usually be improving the design of your experimental code. Without it, your experimental code can often end up as a "big ball of mud".
I have found that when I let "I'm not comfortable writing unit tests in this situation" be an excuse for not writing unit tests... I never write any unit tests.
If you know enough about the behavior of something to code it, you know enough to test it.
Even if I can think of 1 or 2 definite functions, what if during the
experimenting phase I find it's better to split those functions out
into more functions across different classes
If you already have the original function tested, and you're only changing the interface, then there should be little test logic that would have to change. And there's no risk of breaking something when you change the interface afterwards. Further, this concern is not specific to experimenting with a new domain.
Perhaps an unfamiliar domain is not the best place to start learning TDD. But I would certainly not say it is inherently unsuitable to the process.
I've been reading about TDD lately, and it is advocated because it supposedly results in code that is more testable and less coupled (and a bunch of other reasons).
I haven't been able to find much in the way of practical examples, except for a Roman numeral conversion and a number-to-English converter.
Observing these two examples, I observed the typical red-green-refactor cycles typical of TDD, as well as the application of the rules of TDD. However, this seemed like a big waste of time when normally I would observe a pattern and implement it in code, and then write tests for it after. Or possibly write a stub for the code, write the unit tests, and then write the implementation - which might arguably be TDD - but not this continuous case-by-case refactoring.
TDD appears to incite developers to jump right into the code and build their implementation inductively rather than designing a proper architecture. My opinion so far is that the benefits of TDD can be achieved by a proper architectural design, although admittedly not everyone can do this reasonably well.
So here I have two questions:
Am I right in understanding that using TDD pretty much doesn't allow you to design first (see the rules of TDD)?
Is there anything that TDD gives you that you can't get from doing proper design before you start coding?
well, I was in your shoes some time ago and had the same questions. Since then I have done quite some reading about TDD and decided to mess with it a little.
I can summarize my experience about TDD in these points:
TDD is unit testing done right, ATDD/BDD is TDD done right.
Whether you design beforehand or not is totally up to you. Just make sure you don't do BDUF. Believe me you will end up changing most of it midways because you can never fully understand the requirements until your hands get dirty.
OTOH, you can do enough design to get you started. Class diagrams, sequence diagrams, domain models, actors and collaborators are perfectly fine as long as you don't get hung up in the design phase trying to figure everything out.
Some people don't do any design at all. They just let the code talk and concentrate on refactoring.
IMHO, balance your approach. Do some design till you get the hang of it then start testing. When you reach a dead end then go back to the white board.
One more thing, some things can't be solved by TDD like figuring out an algorithm. This is a very interesting post that shows that some things just need to be designed first.
Unit testing is hard when you have the code already. TDD forces you to think from your API users perspective. This way you can early on decide if the public interface from your API is usable or not. If you decide to do unit testing after implementing everything you will find it tedious and most probably it will be only for some cases and I know some people who will right only passing test cases just to get the feature done. I mean who wants to break his own code after all that work?
TDD breaks this mentality. Tests are first class citizens. You aren't allowed to skip tests. You aren't allowed to postpone some tests till the next release because we don't have enough time.
Finally to answer your question if there anything that TDD gives you that you can't get from doing proper design before you start coding, I would say commitment.
As long as your doing TDD you are committed to apply good OO principles, so that your code is testable.
To answer your questions:
"Test Driven Development" (TDD) is often referred to as "Test Driven Design", in that this practice will result in a good design of the code. When you have written a failing unit test, you are forced into a test driven design approach, so that you can implement just what is needed to make the test pass i.e. you have to consider the design of the code you are writing to make the test pass.
When using a TDD approach a developer will implement the minimum amount of code required to pass the test. Doing proper design beforehand usually results in waste if the requirements change once the project has started.
You say "TDD appears to incite developers to jump right into the code and build their implementation inductively rather than designing a proper architecture" - If you are following an Agile approach to your software development, then you do still need to do some up front architectural investigation (e.g. if you were using the Scrum methodology you would create a development "Spike" at the start of a project) to ascertain what the minimum amount of architecture needed to start the project. At this stage you make decisions based on what you know now e.g. if you had to work with a small dataset you'd choose to use a regular DB, if you have a huge DB you might to choose to use a NoSQL big data DB etc.
However, once you have a general idea of the architecture, you should let the design evolve as the project progresses leaving further architectural decisions as late in the process as possible; Invariably as a project progresses the architecture and requirements will change.
Further more this rather popular post on SO will give you even more reasons why TDD is definetly worth the effort.
The idea of TDD is great, but i'm trying to wrap my head around how to implement a complex system if a design is not proposed upfront.
For example, let's say I have multiple services for an payment processing application. I'm not sure I understand how development would/can proceed across multiple developers if there is not a somewhat solid design upfront.
It would be great if someone can provide an example and high level steps to putting together a system in this manner. I can see how TDD can lead to simpler and more robust code, I'm just not sure how it can bring together 1) different developers to a common architectural vision and 2) result in a system that can abstract out behavior in order to prevent having to refactor large chunks of code (e.g. accept different payment methods or pricing models based on a long term development roadmap).
I see the refactoring as a huge overhead in a production system where data model changes increase risks for customers and the company.
Clearly i'm probably missing something that TDD gurus have discovered....
IMHO, It depends on the the team's composition and appetite for risk.
If the team consists of several experienced and good designers, you need a less formal 'architecture' phase. It could be just a back of the napkin doodle or a a couple of hours on the whiteboard followed by furious coding to prove the idea. If the team is distributed and/or contains lots of less skilled designers, you'd need to put more time/effort (thinking and documenting) in the design phase before everyone goes off on their own path
The next item that I can think of is to be risk first. Continually assess what are the risks to your project, calculate your exposure/impact and have mitigation plans. Focus on risky and difficult to reverse decisions first. If the decision is easily reversible, spend less time on it.
Skilled designers are able to evolve the architecture in tiny steps... if you have them, you can tone down the rigor in an explicit design phase
TDD can necessitate some upfront design but definitely not big design upfront. because no matter how perfect you think your design is before you start writing code, most of the time it won't pass the reality check TDD forces on it and will blow up to pieces halfway through your TDD session (or your code will blow up if you absolutely want to bend it to your original plan).
The great force of TDD is precisely that it lets your design emerge and refine as you write tests and refactor. Therefore you should start small and simple, making the least assumptions possible about the details beforehand.
Practically, what you can do is sketch out a couple of UML diagrams with your pair (or the whole team if you really need a consensus on the big picture of what you're going to write) and use these diagrams as a starting point for your tests. But get rid of these models as soon as you've written your first few tests, because they would do more harm than good, misleading you to stick to a vision that is no longer true.
First of all, I don't claim to be a TDD guru, but here are some thoughts based on the information in your question.
My thoughts on #1 above: As you have mentioned, you need to have an architectural design up-front - I can't think of a methodology that can be successful without this. The architecture provides your team with the cohesion and vision. You may want to do just-enough-design up front, but that depends on how agile you want to be. The team of developers needs to know how they are going to put together the various components of the system before they start coding, otherwise it will just be one big hackfest.
It would be great if someone can provide an example and high level
steps to putting together a system in this manner
If you are putting together a system that is composed of services, then I would start by defining the service interfaces and any messages that they will exchange. This defines how the various components of your system will interact (this would be an example of your up-front design). Once you have this, you can allocate various development resources to build the services in parallel.
As for #2; one of the advantages of TDD is that it presents you with a "safety net" during refactoring. Since your code is covered with unit tests, when you come to change some code, you will know pretty soon if you have broken something, especially if you are running continuous integration (which most people do with a TDD approach). In this case you either need to adapt your unit tests to cover the new behavior OR fix your code so that your unit tests pass.
result in a system that can abstract out behavior in order to prevent
having to refactor large chunks of code
This is just down to your design, using e.g. a strategy pattern to allow you to abstract and replace behavior. TDD does not prescribe that your design has to suffer. It just asks that you only do what is required to satisfy some functional requirement. If the requirement is that the system must be able to adapt to new payment methods or pricing models, then that is then a point of your design. TDD, if done correctly, will make sure that you are satisfying your requirements and that your design is on the right lines.
I see the refactoring as a huge overhead in a production system where
data model changes increase risks for customers and the company.
One of the problems of software design is that it is a wicked problem which means that refactoring is pretty much inevitable. Yes, refactoring is risky in production systems, but you can mitigate that risk and TDD will help you. You also need to have a supple design and a system with low coupling. TDD will help reduce your coupling since you are designing your code to be testable. And one of the by-products of writing testable code is that you reduce your dependencies on other parts of the system; you tend to code to interfaces which allows you to replace an implementation with a mock or stub. A good example of this is replacing a call to a database with a mock/stub that returns some known data - you don't want to hit a database in your unit tests. I guess I can mention here that a good mocking framework is invaluable with a TDD approach (Rhino mocks and Moq are both open source).
I am sure there are some real TDD gurus out there who can give you some pearls of wisdom...Personally, I wouldn't consider starting a new project with out a TDD approach.
I usually use ASP.net web forms for GUI, maybe one of most "stateful" technologies. But it applies to any technology which has states. Sometimes forms are tricky and complex, with >30 elements and > 3 states of each element. Intuitive way of designing such a form usually works for 90%. Other 10% usually find testers or end-users:).
The problem as i see it that we should imagine a lot of scenarios on the same object, which is much harder than a consequence of independent operations.
From functional programming courses I know that best way is not to use state management and use pure functions and variable passing by value and all these stuff, which is greatly formalized. Sometimes, we cannot avoid it.
Do you use any math formalisms and approaches to state management of complex objects? Not like monads in Haskell, but which can be used in more traditional business applications and languages - Java, C#, C++.
It may be not Turing-complete formalism, but 99% will be great also:).
Sorry if it is just another tumbleweed question:)
Use message-passing as an abstraction. Advantages:
The difficulty with complex state is complex interactions, which are especially hairy in concurrent systems like typical GUIs. Message-passing, by eliminating shared state, stops the complexity of state in one process from being infectious.
Message-passing concurrency has nice foundational models: e.g., the Actor model, CSP, both of which influenced Erlang.
It integrates well with functional programming: check out Erlang again. Peter van Roy's book *Concepts, Techniques, and Models of Computer Programming is an excellent text that shows the fundamental ingredients of programming languages, such as pure functions, and message-passing, and how they can be combined. The text is avilable as a free PDF.
It may be not Turing-complete formalism, but 99% will be great also:).
Sorry, but I'd rather provide NP-complete solution :)
Quick answer from me would be Test-Driven Approach. But read further for more.
The problem as i see it that we should
imagine a lot of scenarios on the same
object, which is much harder than a
consequence of independent operations.
In such cases the decomposition (not only in computer science sense, but in mathematical too) is very useful.
You decompose complex scenario in many simpler ones, which in turn can still be complex by themselves and can be decomposed further.
As a result of such a process you should end up with a number of simple functions (tasks) mostly independent of each ones.
This is very important because then you can UNIT TEST those simple scenarios.
Additionally, it is much easier and better to follow test-first approach which allows to see the decomposition in the very beginning of the development process.
Do you use any math formalisms and approaches to state management of complex objects?
To continue what I said, for me the most important thing is to make a good decomposition so that I can ensure the quality and being able to easily reproduce errors in an automated manner.
To give you an abstract example:
You have a complex scenario A. You always need to write at least 3 tests for each scenario: correct input, incorrect input and corner case(s).
Starting to write first test (correct input) I realize that the test becomes too complex.
As a result, I decompose scenario A into less complex A1, A2, A3. Then I start writing tests for each of them again (I should end up with at least 3*3=9 tests).
I realise that A1 is still too complex to test, so I decompose it again into A1-1, A1-2. Now I have 4 different scenarios (A1-2, A1-2, A2, A3) and 3*4=12 potential tests. I continue writing the tests.
After I am done. I start implementation, so all my tests pass. After that you have 12 proves that scenario A (more precisely its parts) works correctly. Additionally, you might write another 3 tests for the scenario A that combines all of its decomposed parts - this kind of testing is often (but not always!) can be seen as Integration testing.
Then let's assume a bug is found in scenario A. You are not sure which part it belongs to, but you suspect that it is related to A1-2 or A3. So you write 2 more tests for each of the scenario to reproduce the bug (write such a test that fails not meeting your expectations). After you have reproduced the bug you fix it and make ALL tests pass.
Now you have 2 more proves of correctly working system that ensures all the previous functionality is working the same way.
There are 2 main major problems with this approach IMO.
You need to write a lot of tests and support them. Many developers just do not want to do that.
Additionally, the process of decomposition is more art than science. Good decomposition will result in a good structure, tests and supportability while a bad one will result in a lot of pain and wasted time. And it is hard to tell if the decomposition is good or bad at first.
This process is called Test-Driven-Development. I find it to be the closest "formalization" of development process that plays nice between science and real world.
So I do not really talk about state here but rather behavior and proving it works correctly.
From personal experience, I should mention that ASP.NET WebForm is technically VERY hard to test.
To overcome that, I would suggest to apply MVP pattern for ASP.NET WebForms.
As opposed to WebForms, ASP.NET MVC is so much easier to test.
But still, you should have set of so called "services" (our scenarios) and (unit) test them separately, then test the UI integration in the environment close to Integration tests.
A number of developers are designing their applications using Test Driven Development (TDD) but I would like to know at which stage should I incorporate TDD into my projects? Should I first design my classes, do some unit tests and determine what methods are needed or design methods first and then do some unit tests after?
What is the best way to go about this?
TDD is a coding and design-in-the-small technique. It is not a big-picture envisioning technique. If you starting to write an application, you want to do some storyboards, or wireframes, or even some simple sketches. You should have an idea about the larger-scale design i.e. the classes and relationships in your system. Before you get to the point where you'd start to do interaction design (e.g. methods and arguments) you start doing TDD.
The wireframes you have done will give you an idea of how the system should appear. The large scale design will give you an idea of what classes to create. However, none of these models should be considered correct or permanent. As you write tests, you'll find better design ideas that will change your high-level design, and even your wireframe design.
The point of it being test driven is that the tests drive the design as well as the implementation.
In some cases that's not appropriate - it may be blatantly obvious what the design should look like, especially if it's implementing an existing interface (or some similarly restricted choice) but when you've got a large "design space" TDD encourages you to writes tests demonstrating how you want to be able to use the class, rather than starting off with you how you think you want to implement it.
Generally if something is easy to test, it will be easy to use.
The mantra for test driven design:
create the test.
compile test (and verify that it fails)
write the interface
compile test (it should compile now)
run test (it should fail now)
write the code
compile/run test (it should do both now)
Repeat for each item.
I'm very skeptical about writing unit tests before implementation. Maybe it would be efficient if you had a good idea about the exact design about the classes and methods required but generally at the start of a new project you don't.
My preferred approach is to acknowledge that the start of a new project can be a bit of organised chaos. One part of the design process is writing code to validate certain ideas or to find the best way. A lot of time the code hits a dead end and gets thrown away and any unit tests written for it would have been a waste of time.
Don't get me wrong I'm all for well designed software and at a certain point it's essential that the organised chaos morphes into a well understood design. At that point the class structure and UML diagrams start to solidify. This is the point TDD becomes important to me. Every class should be able to be tested in isolation. If the design is good with the right degree of decoupling that is easy to achieve usually with a sprinkling of mock objects around the place. If the design is bad testing becomes a real nightmare and consequently gets ignored.
At the end of the day it's high quality productive code that you're after. Unit tests are one of the best tools to help achieve that but they shouldn't take on a life of their own and become more important than the code they are testing which you sometimes get the feeling of from the TDD mantra.
Start small
a) The next time you add a method to a Utility class, write the tests first for the method. Methods that do string handerling are a good place to start, as they tend to have lot of bugs, but are don’t need complex dataset to test them.
b) Then once you get confidant with writing unit tests, move on the classes that don’t touch UI and don’t talk directly or indirectly to the database.
c) You have then got to decide if you will design your application to make unit testing easier. E.g
it is hard to test UI code so look at
MVP etc,
it is head to test code
that access the database, so separate
your logic into method that don’t need
to access the database to run.
a) and b) has a big payback without having to change how you design the overall system. You have to consider if c) is worth the cost.... (I think it is, but a lot of people don't and often you can't control the overall system design.)
Ideally you will start to apply TDD when you begin coding. Think about your design first, you can draw diagrams on paper ; no need to use CASE tool, just envision the main classes and their interactions, pick one, write a first simple test, write the code to make it pass, write another test, make it pass, refactor ...
If your process mandates documenting and reviewing the design before the coding stage, then you cannot follow TDD and let your design emerge. You can however make your design first, and then do the implementation test-first. You'll get unit-tests as if you applied TDD, but still applying your process.
Keep your design as hi-level as possible, so that you can let the low-level details of the design emerge.
Please look at
http://en.wikipedia.org/wiki/Test-driven_development
They have stated a high level tasks
TDD ensures that developers give equal importance to test cases as compared to their business logic. I followed it in my project and i could see the advantages: Since i started with writing my test cases first, i ensured that my test cases handle all the possible coverages of a business logic, thereby catching/reducing the number of bugs. Some times its not practical since it requires enough time to write these test cases.
The bottom line is you need to unit test your code really well.So either you start with implementation and then ensure that there are enough test cases for all the scenarios (Most people dont hence this practice has emerged.) or write the test methods do a dry run and then write your implementation. Your implementation is said to be complete only when all your testcases pass successfully.
You should listen to stack overflow podcast 41 where they talk about TTD. It's hard to say everybody is using TDD and it would be take a lot of time and resources to have a high code coverage percentage. Unit testing could effectively double the development time if you write tests for all of your code.
One of the points from the podcast is that your time and resources could be better used doing other tasks such as usability and new features.