I understand the general idea about unit testing and have used it in scenarios where complex interactions were happening in a system, but I still have a question about all of those principles coming together.
We are cautioned about not testing the framework, or the database. Good UI design does not lend itself to non-human testing. UI interaction in general is excluded in MVC frameworks. In many apps what is left?. 37signals talks about extensive unit testing, but in an app like Basecamp or Backpack, what exactly are the types of things being tested through appropriate unit tests? What would 100% code coverage mean in that scenario?
EDIT: I am not tearing down apps like Backpack - they are awesome, but the work seems to go more into the design and interaction instead of complex logic (in fact, they espouse that very idea). In those areas of that app, where CRUD and the hierarchy of which object is related to which pretty much cover it, is the appropriate amount of unit tests zero? Is the point of tests in that case to be another copy of the validation code (required, regex, etc.)?
TDD for business apps works like this.
Write down a business requirement.
Write down a test for that requirement.
Write code that passes the test.
The trick is that there are many junky non-business requirements that don't need extensive testing.
"saves to a database" isn't a business requirement. It's technical.
"activates a button on the GUI for some situation" isn't a business requirement, it's part of the interface.
"backup" isn't a business requirement; it's security or business continuity or something.
Consider a concrete example.
Requirement -- "You can't borrow books until you've paid your fines."
Tests.
Attempt to borrow book with fine.
Attempt to borrow book with no fine.
Code.
class FinesNotPaid( unittest.TestCase ):
def setUp( self ):
# load some fixtures with users, books and fines outstanding.
def test_should_not_checkout_book( self ):
x = TheCheckoutClass()
x.checkoutBook( someBook )
self.fail( "Should have thrown error" )
class FinesPaid( unittest.TestCase ):
def setUp( self ):
# load some fixtures with users, books and fines paid.
def test_should_not_checkout_book( self ):
x = TheCheckoutClass()
x.checkoutBook( someBook )
self.success( )
class NoFines( unittest.TestCase ):
etc.
These are Business Rules implemented by classes that are separate from your database and your GUI. These are classes in an application domain layer.
Your CRUD rules are business rules. You need to test them. However, you don't need to test every database-related feature. You need a few "can I create and persist an object?" tests. You have to trust that the ORM, the Data Access layer and the Database actually work. You don't write tests to exhaustively test the built-in features of the ORM.
Code coverage (100% or 80% or 10%) is largely meaningless. Is software that has tests with 80% code coverage actually 20% more likely to fail? It doesn't work that way. Testing every line of code doesn't cover every logic path, so stop worrying and start testing.
Test the business use cases. If they pass and there's untested code, then -- perhaps -- you wrote too much code.
You're testing to ensure that the business logic (in many apps this is the "service" or "logic" layer) matches up with your business users' description of how the business really works. You're testing to make sure that, for instance, just because you add 6% sales tax to all purchases from Pennsylvania, you aren't also giving a bonus 6% credit when you give someone a gift card.
There is (or there should be) a whole lot of brain juice in the layers of the application that stand between the UI and the database. That's the stuff to test the heck out of.
Related
I am trying to make sense of TDD. Say I am going to write multiple test classes along with tens of methods for a small project. There will be hundreds of combinations of test classes and methods that work.Are there best practices, standards or guidance on what test classes there should be? Similarly, any on methods of each class?
I involves in the full lifecycle of web application development.Any recommendations on books or training? Thank you in advance!
In TDD, you don't "plan" to write hundreds of combinations of test classes. What you do is take one of your requirements (often just a tiny subset of a requirement), and write one test that proves it's been met. Your test will fail, because your code doesn't do that yet. Next, you write a little bit of code, just enough to pass the test. And then you refactor that code you just wrote - the goal of this refactoring is to remove duplication, to follow good OO design principles, and to make your code fit your application's architecture.
What you'll find is that you'll think about coding differently. You stop trying to plan out every class in advance. You don't generate huge UML diagrams. Instead, you grow your product to match more and more of your requirements. When all your requirements are met, the code is done.
Getting to your more specific questions, what we do is to have a one-to-one relationship between test classes and production classes. I'm a fan of naming the test classes by suffixing _Test to the class name: the tests for ShippingClass become ShippingClass_Test. That makes the tests for a class easy to find. Inside ShippingClass_Test, there will be several test methods for each method of ShippingClass. Each test method exercises a different flow through the production method. Let's say there's a CalculateShippingFees() method and this requirement has to deal with taxes, tax exempt, and international shipping. Each part of the requirements will drive a new little bit of logic that will result in another test. I might get as far as writing a CalculateShippingFees_TaxExempt_Test, CalculateShippingFees_Taxable_Test, and a CalculateShippingFees_TaxServiceOffline_Test (always write tests for your error handling paths, too.)
But when I get to CalculateShippingFees_International_Test, as I'm refactoring I realize that there are more destinations than I thought, and the method is growing too big. So maybe I extract the destinations into a new class using the Strategy pattern, and then I start writing tests for each of those. Or maybe I discover that our shipping provider has a fee calculator that I can leverage, so I change the code to use it. Making these changes is quick and easy, because I know that as long as all my tests pass, I can change anything I want!
And that's the point of TDD: you evolve the software to meet your needs as you need them. If I built up a giant mental model that said "I'll need a shipping fee tax class, and an international destination, a domestic destination, a customs fee calculator, blah blah blah", I may spend a lot of time building all those classes only to later discover that the shipping provider has an API that gives me all that stuff for free, and I wasted my time writing it. Or maybe I screwed up the initial design, thinking that customs fees would be a part of shipping instead of a part of international destinations. With TDD, as I add the new methods and classes, I put them where I need them when I need them.
There are several good books on the topic. Like #Franck, I also recommend Mezaros' xUnit Test Patterns. Roy Osherove has a pretty good book The Art of Unit Testing. And Freeman and Pryce have Growing Object Oriented Software Guided By Tests. Kent Beck's book, Test Driven Development by Example was kind of the spark and it's pretty easy to follow, but it's getting old and the practices of TDD have evolved past it.
I guess would be better if you start to write your code and you show us where do you think you will have tons of combinations, the recommended books are really awesome and also you will have to take advantage of what unit testing framework are you using, junit(Java) supports parameters for those combinations and Spock(groovy) uses parameter tables that are really awesome
I've watched and read a handful of tutorials on PHPUnit and Test Driven Development and have recently begun working with Laravel which extends the PHPUnit Framework with it's TestCase class. All of these things make sense to me, as far as, creating tests as you develop. And I find Laravel's extensions particularly intuitive (especially in regards to testing Controller routes)
However, I've recently been tasked with creating unit tests for a sizable app that's near completion. The app is built in Codeigniter, and it was not built with any tests
I find that I'm not entirely sure where to begin, or what steps to take in order to determine the tests I should create.
Should I be looking to test each controller method? Or do I need to break it down more than that? Admittedly, many of these controller methods are doing more than one task.
It is really difficult to write tests for existing project. I will suggest you to first start with writing tests for classes which are not dependent on other classes. Then you can continue to write tests to classes which coupled with classes for which you wrote tests. You will increase your test coverage step by step by repeating this process.
Also don't forget that some times you will need to refactor your code to make it testable. You should improve design of code for example if your controller methods doing more than one task you should divide this method to sub methods and test each of these methods independently.
I also will suggest you to look at this question
You are in a bit of a tight spot, but here is what I would do in your situation. You need to refactor (ie. change) the existing code so that you end up with three types of functions.
The first type are those that deal with the outside world. By this I mean anything that talks to I/O, or your framework or your operating system or even libraries or code from stable modules. Basically everything that has a dependency on code that you can not, or may not change.
The second group of functions are where you transform or create data structures. The only thing they should know about are the data structures that they receive as parameters and the only way they communicate back is by changing those structures or by creating and populating a new structure.
The third group consists of co-ordinating functions which make the calls to the outside world functions, get their returned data structures and pass those structures to the transforming functions.
Your testing strategy is then as follows: the second group can be tested by creating fake data structures, passing them in and checking that the transforms were done correctly. The third group of co-ordinating functions can be tested by dependency injection and mocking to see that they call the outside world and transform functions correctly. Finally the last group of functions should not be tested. You follow the maxim - "make it so simple that their is obviously nothing wrong". See if you can keep it to a single line of code. If you go over four lines of code for these then you are probably doing it wrong.
If you are completely new to TDD I do however strongly suggest that you first get used to doing it on green field projects/modules. I made a couple of false starts on unit testing because I tried to bolt it onto projects afterwards. TDD is really a joy when you finally grok it so it would not be good if you get discouraged early on because of a too steep learning curve.
I have a lengthy but simple form which allows users to update products, nothing fancy, just displays the fields to enter mostly text values & then updates the corresponding database table.
Is it regular practice to write tests for CRUD operations such as this? Its important the form works properly as it will be used everyday - however there is really not much to go wrong. I have the time to do this, but don't want to be wasting time either or make future maintenance of my test suite overly difficult.
If I was to cover this with tests, should I use integration (cucumber) or unit (rspec) tests?
Thanks for any advice!
In theory I believe you are supposed to use both: Rspec to test each layer in isolation, then Cucumber to test the whole stack. But if you think this piece of your app is too straight-forward to warrent slavish adherence to those principles, I would recommend you stick with the integration tests. It's important to know if something isn't working right, and IMO Cucumber alone should be suffient for that purpose.
Of course for more complex areas of your app, you probably want to use some mixture of each.
A simple billing system (on top of ColdBox MVC) is ballooning into a semi-enterprisey inventory + provisioning + issue-tracking + profit tracking app. They seem to be doing their own thing yet they share many things including a common pool of Clients and Staff (login's), and other intermingled data & business logic.
How do you keep such system modular? from a maintenance, testability & re-usability stand point?
single monolithic app? (i.e. new package for the base app)
ColdBox module? not sure how to make it 'installable' and what benefits does it bring yet.
Java Portlet? no idea, just thinking outside the box
SOA architecture? through webservice API calls?
Any idea and/or experience you'd like to share?
I would recommend you break the app into modular pieces using ColdBox Modules. You can also investigate on separate business logic into a RESTful ColdBox layer also and joining the system that way also. Again, it all depends on your requirements and needs at the moment.
Modules are designed to break monolithic applications into more manageable parts that can be standalone or coupled together.
Stop thinking about technology (e.g. Java Portals, ColdBox modules, etc...) and focus on architecture. By this I mean imagining how you can explain your system to an observer. Start by drawing a set of boxes on a whiteboard that represent each piece - inventory, clients, issue tracking, etc... - and then use lines to show interactions between those systems. This focuses you on a separation of concerns, that is grouping together like functionality. To start don't worry about the UI, instead focus on algorithms and data.
If you we're talking about MVC, that step is focusing on the model. With that activity complete comes the hard part, modifying code to conform to that diagram (i.e the model). To really understand what this model should look like I suggest reading Domain Driven Design by Eric Evans. The goal is arriving at a model whose relationships are manageable via dependency injection. Presumably this leaves you with a set of high level CFCs - services if you will - with underlying business entities and persistence management. Their relationships are best managed by some sort of bean container / service locator, of which I believe ColdBox has its own, another example is ColdSpring.
The upshot of this effort is a model that's unit testable. Independent of of the user interface. If all of this is confusing I'd suggest taking a look at Working Effectively with Legacy Code for some ideas on how to make this transition.
Once you have this in place it's now possible to think about a controller (e.g. ColdBox) and linking the model to views through it. However, study whatever controller carefully and choose it because of some capability it brings to the table that your application needs (caching is an example that comes to mind). Your views will likely need to be reimagined as well to interact with this new design, but what you should have is a system where the algorithms are now divorced from the UI, making the views' job easy.
Realistically, the way you tackle this problem is iteratively. Find one system that can easily be teased out in the fashion I describe, get it under unit tests, validate with people as well, and continue to the next system. While a tedious process, I can assure it's much less work than trying to rewrite everything, which invites disaster unless you have a very good set of automated validation ahead of time.
Update
To reiterate, the tech is not going to solve your problem. Continued iteration toward more cohesive objects will.
Now as far as coupled data, with an ORM you've made a tradeoff, and monolithic systems do have their benefits. Another approach would be giving one stateful entity a reference to another's service object via DI, such that you retrieve it through that. This would enable you to mock it for the purpose of unit testing and replace it with a similar service object and corresponding entity to facilitate reuse in other contexts.
In terms of solving business problems (e.g. accounting) reuse is an emergent property where you write multiple systems that do roughly the same thing and then figure out how to generalize. Rarely if ever in my experience do you start out writing something to solve some business problem that becomes a reusable component.
I'd suggest you invest some time in looking at Modules. It will help with partitioning your code into logical features whilst retaining the integration with the Model.
Being ColdBox there is loads of doc's and examples...
http://wiki.coldbox.org/wiki/Modules.cfm
http://experts.adobeconnect.com/p21086674/
You need to get rid of the MVC and replace it with an SOA architecture that way the only thing joining the two halves are the service requests.
So on the server side you have the DAO and FACADE layers. And the client side can be an MVC or what ever architecture you want to use sitting somewhere else. You can even have an individual client for each distinct business.
Even for the server side you can break the project down into multiple servers: what's common between all businesses and then what's distinct between all of them.
The problem we're facing here luckily isn't unique.
The issue here seems not to be the code itself, or how to break it apart, but rather to understand that you're now into ERP design and development.
Knowing how best to develop and grow an ERP which manages the details of this organization in a logical manner is the deeper question I think you're trying to get at. The design and architecture itself of how to code from this flows from an understanding of the core functional areas you need.
Luckily we can study some existing ERP systems you can get a hold of to see how they tackled some of the problems. There's a few good open source ERP's, and what brought this tip to my mind is a full cycle install of SAP Business One I oversaw (a small-mid size ERP that bypasses the challenges of the big SAP).
What you're looking for is seeing how others are solving the same ERP architecture you're facing. At the very least you'll get an idea of the tradeoffs between modularization, where to draw the line between modules and why.
Typically an ERP system handles everything from the quote, to production (if required), to billing, shipping, and the resulting accounting work all the way through out.
ERPS handle two main worlds:
Production of goods
Delivery of service
Some businesses are widget factories, others are service businesses. A full featured out of the box ERP will have one continuous chain/lifecycle of an "order" which gets serviced by a number of steps.
If we read a rough list of the steps an ERP can cover, you'll see the ones that apply to you. Those are probably the modules you have or should be breaking your app into. Imagine the following steps where each is a different document, all connected to the previous one in the chain.
Lead Generation --> Sales Opportunities
Sales Opportunities --> Quote/Estimate
Quote Estimate --> Sales Order
Sales Order --> Production Order (Build it, or schedule someone to do the work)
Production order --> Purchase orders (Order required materials or specialists to arrive when needed)
Production Order --> Production Scheduling (What will be built, when, or Who will get this done, when?)
Production Schedule --> Produce! (Do the work)
Produced Service/Good --> Inventory Adjustments - Convert any raw inventory to finished goods if needed, or get it ready to ship
Finished Good/Service --> Packing Slip
Packing Slip items --> Invoice
Where system integrators come in is using the steps required, and skipping over the ones that aren't used. This leads to one thing for your growing app:
Get a solid data security strategy in place. Make sure you're confortable that everyone can only see what they should. Assuming that is in place, it's a good idea to break apart the app into it's major sections. Modules are our friends. The order to break them up in, however, will likely have a larger effect on what you do than anything.
See which sections are general, (reporting, etc) that could be re-used between multiple apps, and which are more specialized to the application itself. The features that are tied to the application itself will likely be more tightly coupled already and you may have to work around that.
For an ERP, I have always preferred a transactional "core" module, which all the other transaction providers (billing pushing the process along once it is defined).
When I converted a Lotus Notes ERP from the 90's to the SAP ERP, the Lotus Notes app was excellent, it handled everything as it should. THere were some mini-apps built on the side that weren't integrated as modules which was the main reason to get rid of it.
If you re-wrote the app today, with today's requirements, how would you have done it differently? See if there's any major differences from what you have. Let the app fight for your attention to decide what needs overhauling / modularization first. ColdBox is wonderful for modularization, whether you're using plugin type modules or just using well separated code you won't go wrong with it, it's just a function of developer time and money available to get it done.
The first modules I'd build / automate unit testing on are the most complex programatically. Chances are if you're a decent dev, you don't need end to end unit testing as of yesterday. Start with the most complex, move onto the core parts of the app, and then spread into any other areas that may keep you up at night.
Hope that helped! Share what you end up doing if you don't mind, if anything I mentioned needs further explanation hit me up on here or twitter :)
#JasPanesar
Assuming the you are implementing a user story that requires changes in all layers from UI (or service facade) to DB.
In what direction do you move?
From UI to Business Layer to Repository to DB?
From DB to Repository to Business Layer to UI?
It depends. (On what ?)
The best answer I've seen to this sort of question was supplied by the Atomic Object guys and their Presenter First pattern. Basically it is an implementation of the MVP pattern, in which (as the name suggests) you start working from the Presenter.
This provides you with a very light-weight object (since the presenter is basically there to marshal data from the Model to the View, and events from the View to the Model) that can directly model your set of user actions. When working on the Presenter, the View and Model are typically defined as interfaces, and mocked, so your initial focus is on defining how the user is interacting with your objects.
I generally like to work in this way, even if I'm not doing a strict MVP pattern. I find that focusing on user interaction helps me create business objects that are easier to interact with. We also use Fitnesse in house for integration testing, and I find that writing the fixtures for Fitnesse while building out my business objects helps keep things focused on the user's perspective of the story.
I have to say, though, that you end up with a pretty interesting TDD cycle when you start with a failing Fitnesse test, then create a failing Unit Test for that functionality, and work your way back up the stack. In some cases I'm also writing Database unit tests, so there is another layer of tests that get to be written, failed, and passed, before the Fitnesse tests pass.
If change is likely, start in the front. You can get immediate feedback from shareholders. Who knows? Maybe they don't actually know what they want. Watch them use the interface (UI, service, or otherwise). Their actions might inspire you to view the problem in a new light. If you can catch changes before coding domain objects and database, you save a ton of time.
If requirements are rigid, it's not as important. Start in the layer that's likely to be the most difficult - address risk early. Ultimately, this is one of those "more an art than a science" issues. It's probably a delicate interplay between layer design that creates the best solution.
Cheers.
I'd do it bottom up, since you'll have some working results fast (i. e. you can write unit tests without a user interface, but can't test the user interface until the model is done).
There are other opinions, though.
I would start modeling the problem domain. Create relevant classes representing the entities of the system. Once I feel confident with that, I'd try to find a feasible mapping for persisting the entities to the database. If you put too much work into the UI before you have a model of the domain, there is a significant risk that you need to re-work the UI afterwards.
Thinking of it, you probably need to do some updates to all of the layers anyway... =)