Related
Ideally, using the outside-in development style of BDD, you have to write scenarios first with associated step definition. Then have it failed. Then write enough application code to have it passed.
My question is, in a real scenario, how is it possible to write step definitions of integration tests when there isn't any UI developed which can be used by selenium to implement page object on OR any application classes which can be used in the step definition to perform business logic layer or model testing.
Am I missing something here? I am new to BDD and this is really confusing. Please share your practical experience, no theories please.
Edit:
Example Scenario:
A client wants an e-commerce web app. Initially asks for the usual features such as:
Product Listing Gallery
A Shopping Cart
Checkout with various payment options
Email notifications
As per my understanding(which is probably wrong) of outside-in development process:
1) I would be writing scenarios specifications using Gherkin after analysis of above mentioned user stories
2) Then writing step definitions for these scenarios (How do I write them since there isn't any UI or application classes to use from)
3) I run these scenarios which fail
4) Then the developers write some application code which makes my scenarios pass
5) Then refactoring my step definition.
Your so called scenario description is just a draft about what features the application must contain. It should be something like this instead using the gherkin syntax:
Feature: Product Listing Gallery
To be able to order
The Customers should be able to
Browse a product list
Scenario: Listing products without filters
Given Susan is a Customer
When Susan asks for a product list
And does not give any filters
Then she should get a list of products
But only products the store has in stock will be listed
...
So you can have many scenarios in your feature file, and each of them describes an use case. After you have written the use cases into these files, you can add step definitions.
This part of the story is similar to polymorphism. By polymorphism you define an interface, which describes the abstraction (what to do) and you have classes, which implement the interface in certain ways (how to do it). If you don't feel the analogy already: feature descriptions are about what to do, and step definitions are about how to do it. I think it is evident, that you can use many different kind of step definitions with the same feature description, just like many classes can implement the same interface by polymorphism. So you can write e2e test step definitions if you want to test the whole application. You can write GUI test step definitions and mocking out the business logic if you want to test the GUI in separation. You can write domain model test step definitions and mock out both the GUI and the persistence if you want to test the business logic in separation, and so on...
You can use all of these different step definitions with the same feature files. So if an implementation detail changes, e.g. you change GUI layout, then the only thing you have to modify are the step definitions which use the GUI and not the feature files, which contain only abstractions.
To answer the question, you don't need to develop outside-in. If you use BDD with DDD, then you can develop the domain model first and use the same feature files with different step definitions to develop the GUI later. If you want to start with the GUI, then you should mock out the business logic and test the GUI in separation first. After that you can write the step definitions for testing the business logic in separation. It is better this way than using e2e tests for developing the whole app, because e2e tests are slow. I don't support the latter approach, because I think it is better to think about the domain model without the influence of the GUI, database structure and other things. Ofc. these are just my impressions about how we should develop with BDD.
I want to ask this because I want to know developer perspective about developing from scratch mobile games.
Let's say the project is already going to start its development.
I am already aware of the specification of the project example.
Logic, Browser/devices to support, IDE to be used.
What should be the first thing to do.
Create the UI/UX designs for all devices with no functionality
and after the designs are finalized then integrate functionality
Create functionality logic then do design
Do design and functionality together
which of these will save much more time.
please share your experiences about this development and how to make out of it
so that failure of delivering the project will be at its minimum errors
Every of these aproaches may work, however I will tell you about my experience.
I did a lot of different projects using points you provide in the question, but for me the easiest and most successful approach was logic first, design second.
How to start using this approach?
Think about use case at first. For example, user should be able to register in the game.
Implement database structure(if needed). For example, create table 'Users'.
Implement data access logic. For example, write code that saves a username and a password.
Create service layer with use case's (business) logic implementation . For example, you can write saveUser method.
Create a simple(ugly) UI that allows you to feel the use case and to test it.
Pay guru designer to create clean design. For example, a design of the registration page.
Сontinue with step 1.
Advantages?
Allows to create really working MVP with ease.
UI manipulates only real and dynamic data.
Allows you to build foundation first.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have a good grasp of unit testing, DI, mocks, and all the design principal goodness required to have as close to full code coverage as humanly possible (single responsibility principal, think 'how will i test this' as I code, etc...).
My most recent app, I did not code doing true TDD. I kept unit-testing in mind as I coded, and wrote my tests after writing the code, refactoring, etc.. I did TDD when it was 'easy' to do... however I did not have as good of a grasp as I do now... That was the first project I made full use of DI, mocking frameworks, etc, and the first which had full code coverage - and I learned a lot from it as I went along. I'm itching to get assigned to my next project so I can code it completely doing TDD from scratch.
I know this is a broad question, and I've already ordered TDD by example and XP Unleashed, but I'm hoping for a brief overview of how you all design / write a large application doing TDD.
Do you write the entire application, using nothing but stubbed out code? (e.g., write all the function signatures, interfaces, structures, and write the entire application but without writing any actual implementation)? I could picture it working on small-mid sized, but is this even possible on large applications?
If not, how the heck would you write your first unit test for the highest level function in your system? Lets say for example - on a web service where you have a function called DoSomethingComplicated(param1,...,param6) exposed to the world. Obviously, writing the test first for a simple function like AddNumbers() is trivial - but when the function is at the top of the call stack such as this?
Do you still do design up-front? Obviously you still want to do 'architecture' design - e.g., a flow chart showing IE talking to IIS which talks to a windows service via WCF which talks to the SQL Database... an ERD which shows all your SQL tables and their fields, etc... but what about class design? Interactions between the classes, etc? Do you design this up-front, or just keep writing stub code, refactoring the interactions as you go along, until the whole thing connects and looks like it will work?
Any advice is much appreciated
Do you do design up front?
Of course you do. You've got a big application in front of you. You've got to have some idea of the structure it will have before you start writing tests and code. You don't have to have it all worked out in detail, but you should have some basic idea of the layers, components, and interfaces. For example, if you are working on a web services system, you ought to know what the top level services are, and have a good first approximation of their signatures.
Do you write the entire application using nothing but stubbed out code?
No. You stub things out only if they are really difficult to control in a test. For example, I like to stub out the database, and the UI. I will also stub out third party interfaces. Sometimes I will stub out one of my own components if it vastly increases the test time, or it forces me to create test data that is too complicated. But most of the time I let my tests work on a pretty well integrated system.
I have to say I really dislike the style of testing that relies heavily on mocks and stubs. Don't get me wrong, I think mocks and stubs are very useful for decoupling from things that are hard to test. But I don't like writing things that are hard to test, and so I don't use a lot of mocks and stubs.
How do you write your first unit test for a high level function?
Most high level functions have degenerate behavior. For example, login is a pretty high level function and can be very complicated. But if you try to log in with no user name and no password, the response from the system is going to be pretty simple. Writing that tests will also be very simple. So you start with the degenerate cases. Once you have exhausted them, you move on to the next level of complexity. For example, what if a user tries to log in with a username but no password? Bit by bit you climb the ladder of complexity, never tackling the more complex aspects until the less complex aspects are all passing.
It is remarkable how well this strategy works. You might think that you'd just be climbing around the edges all the time and never getting to the meat; but that's not what happens. Instead you find yourself designing the internal structure of the code based on all the degenerate and exceptional cases. When you finally get around to the primary flow, you find that the structure of the code you are working on has a nice hole of just the right shape to plug the main flow in.
Please don't create your UI first.
UIs are misleading things. They make you focus on the wrong aspects of the system. Instead, imagine that your system must have many different UIs. Some will be web, some will be thick client, some will be pure text. Design your system to work properly irrespective of the UI. Get all the business rules working first, with all tests passing. Then plug the UI in later. I know this flies in the face of a lot of conventional wisdom, but I wouldn't do it any other way.
Please don't design the database first.
Databases are details. Save the details for later. Rather, design your system as though you had no idea what kind of database you were using, Keep any notion of schema, tables, rows, and columns out of the core of the system. Implement your business rules as though all the data were kept in memory all the time. Then add the database later, once you've gotten all the business rules working. Again, I know this flies in the face of some conventional wisdom, but coupling systems to databases too early is a source of a lot of badly warped designs.
Do I write the entire application, using nothing but stubbed out code?
No, not in the slightest sense - that sounds like a very wasteful approach. We must always keep in mind that the underlying reason for doing TDD is rapid feedback. An automated test suite can tell us if we broke anything much faster than a manual test can. If we wait wiring things together until the last moment, we don't get rapid feedback - while we may get rapid feedback from our unit tests, we wouldn't know if the application works as a whole. Unit tests are only one form of test we need to perform to verify the application.
A better approach is to start with the most important feature and work your way in from there, using an outside-in approach. This often means starting with some UI.
The way I do it is by creating the desired UI. Since we normally can't develop UI with TDD, I simply create the View with the technology of choice. No tests there, but I wire up the UI to some API (preferrably using declarative databinding), and that's when the testing begins.
In the beginning, I would then TDD my ViewModels/Presentation Models and corresponding Controllers, possibly hard-coding some responses to see that the UI works. As soon as I have something that doesn't explode when you run it, I check in the code (remember, many small incremental check-ins).
I subsequently work my way vertically down that feature and ensure that this particular piece of UI can go all the way to the data source (or whatever), ignoring all other features.
When the feature is done, I can start on the next feature. The way I picture this process is that I fill out the application by doing one vertical slice at a time until all features are done.
Kick-starting a greenfield app this way always takes extra long time for the first feature since this is where you have to wire up everything, so pick something simple (like the initial View of the app) to keep things as simple as possible. Once the first feature is done, the next ones become much easier because the foundations are now in place.
Do I still design up-front?
Not much, no. I normally have an overall design in mind before I start, and when I work in a team, we sketch this overall architecture on a whiteboard or a slide deck before we start.
This is more or less limited to
The number and names of layers (UI, Presentation Logic, Domain Model, Data Access, etc).
The technologies used (WPF, ASP.NET MVC, SQL Server, .NET 3.5 or whatnot)
How we structure production code and test code, and which test technologies we use
Quality requirements for the code (pair programming, static code analysis, coding standards, etc.)
The rest we figure out as we go, but we use many ad-hoc design sessions at the whiteboard as we go along.
+1 Good question
I truly don't know the answer, but I would start with building blocks of classes that I could test then build into the application, not with the top-level stuff. And yes I would have a rough up-front design of the interfaces, otherwise I think you would find those interfaces changing so often as you refactor that it would be a real hinderance.
TDD By Example won't help I don't think. IIRC it goes through a simple example. I am reading Roy Osherove's The Art of Unit Testing and while it seems to comprehensively cover tools and techniques like mocks and stubs, the example so far seem also pretty simple and I don't see that it tells you how to approach a large project.
Do you write the entire application, using nothing but stubbed out code?
To test our systems we mainly do unit, integration and remote services testing. In unit tests we stub out all long running, time consuming, and external services, i.e. database operations, web services connection or any connection to external services. This is to make sure that our tests are fast, independent and not relying on the response of any external service to provide us quick feedback. We have learnt this the hard way because we do have some tests that do database operations which makes it really slow that goes against the principle "Unit tests must be fast to run"
In integration tests, we test the database operations but still not the web services and external services because that can make the test brittle depending on their availability and we use autotest to run the tests in the background all the while we are coding.
However, to test any kind of remote services, we have tests that connect to the external services, do the operation on them and get the response. What matters to the test is their response and their end state if it is important for the test. The important thing here is, we keep these kind of tests in another directory called remote (that's a convention we created and follow) and these remote tests are only run by our CI (continuous integration) server when we merge any code to the master/trunk branch and push/commit it to the repo so that we know quickly if there has been any changes in those external services that can affect our application.
Do I still design up-front?
Yes but we don't do big design up front basically what uncle Bob (Robert C. Martin) said.
In addition, we get to the whiteboard before immersing ourself into coding and create some Class Collaboration Diagrams just to make it clear and sure that everyone in the team is on the same page and this also helps us to divide the work amongst the team members.
I have recently designed a web application that I would like to write in Ruby. Coming from a ASP background I designed it with method and fields and linked them together (in my diagram and UML) like I would do it in C#.
However, now that I've moved from a single app to MVC I have no idea where my code goes or how the pieces are linked.
For example, my application basically collects information from various sources for users, and when they log in the information is presented to them with "new" information (information collected since last login) is tagged specially in the interface.
In C# I would have a main loop that waits let's say 5 minutes and does the collection, then when a client tries to connect it would spawn a new thread that generates the page with the new information. Now that I'm moving to Ruby I'm not sure how to achieve the same result.
I understand that the controller connects the model to the view and I thus assume this is where my code goes yet I've haven't seen a tutorial that talks about doing what I've mentioned. If someone could point me to one or tell me precisely what I need to do to turn my pseudocode into production code I'd be extremely grateful and probably will still have hair: D
EDIT: Somehow I forgot to mention that I'll be using the Rails framework. I don't really like Ruby but RoR is so nice together that I think I can put up with it.
The part of your application that is retrieving the data at certain interval shouldn't be, strictly speaking, part of the web application. In Unix world (including Rails), it would be implemented either as a daemon process, or a cron job. On Windows, I presume that Windows service is the right tool.
Regarding C# -> Ruby transition, if that's purely for Rails, I'd listen to the George's advice and give ASP.NET MVC a shot, as it resembles Rails logic pretty closely (some would call it a ripoff, I guess ;)). However, learning a new language, especially so different than C# as Ruby is, is always a good idea and a way to improve yourself as a developer.
I realize you want to move to Ruby; but you may want to give ASP.NET MVC a shot. It's the MVC framework on the ASP.NET platform.
Coming from ASP, you're going to have to do a lot of conversion to change your code to become more modular. Much more than any one post on Stack Overflow will do justice.
MVC is made up into 'tiers':
Model - Your Data
View - What the user Sees
Controller - Handles requests and communicates with the View and Model.
Pick up a book on ASP.NET MVC 1.0, and do some research on the MVC pattern. It's worth it.
Whatever Ruby web framework you plan to use (Rails, merb, Sinatra), it sounds like the portion that collects this data would typically be handled by a background task. Your models would be representations of this data, and the rest of your web app would be pretty standard.
There are some good Railscast episodes on performing tasks in the background:
Rake in Background
Starling and Workling
Custom Daemon
Delayed Job
There are other options for performing tasks in the background (such as using a message queue and the ActiveMessaging plugin) but these screen casts will at least give you a feel for how background jobs are generally approached in Rails.
If you perform these tasks on a regular schedule, there are tools for that as well.
I hope this is of some help.
Check out Rails for .NET Developers. I've heard good things about this book and it sounds like it's just what you're looking for.
I routinely run into this problem, and I'm not sure how to get past this hurdle. I really want to start learning and applying Test-Driven-Development (or BDD, or whatever) but it seems like every application I do where I want to apply is it pretty much only standard database CRUD stuff, and I'm not sure how to go about applying it. The objects pretty much don't do anything apart from being persisted to a database; there is no complex logic that needs to be tested. There is a gateway that I'll eventually need to test for a 3rd-party service, but I want to get the core of the app done first.
Whenever I try to write tests, I only end up testing basic stuff that I probably shouldn't be testing in the first place (e.g. getters/setters) but it doesn't look like the objects have anything else. I guess I could test persistence but this never seems right to me because you aren't supposed to actually hit a database, but if you mock it out then you really aren't testing anything because you control the data that's spit back; like I've seen a lot of examples where there is a mock repository that simulates a database by looping and creating a list of known values, and the test verifies that the "repository" can pull back a certain value... I'm not seeing the point of a test like this because of course the "repository" is going to return that value; it's hard-coded in the class! Well, I see it from a pure TDD standpoint (i.e. you need to have a test saying that your repository needs a GetCustomerByName method or whatever before you can write the method itself), but that seems like following dogma for no reason other than its "the way" - the test doesn't seem to be doing anything useful apart from justifying a method.
Am I thinking of this the wrong way?
For example take a run of the mill contact management application. We have contacts, and let's say that we can send messages to contacts. We therefore have two entities: Contact and Message, each with common properties (e.g. First Name, Last Name, Email for Contact, and Subject and Body and Date for Message). If neither of these objects have any real behavior or need to perform any logic, then how do you apply TDD when designing an app like this? The only purpose of the app is basically to pull a list of contacts and display them on a page, display a form to send a message, and the like. I'm not seeing any sort of useful tests here - I could think of some tests but they would pretty much be tests for the sake of saying "See, I have tests!" instead of actually testing some kind of logic (While Ruby on Rails makes good use of it, I don't really consider testing validation to be a "useful" test because it should be something the framework takes care of for you)
"The only purpose of the app is basically to pull a list of contacts"
Okay. Test that. What does "pull" mean? That sounds like "logic".
" display them on a page"
Okay. Test that. Right ones displayed? Everything there?
" display a form to send a message,"
Okay. Test that. Right fields? Validations of inputs all work?
" and the like."
Okay. Test that. Do the queries work? Find the right data? Display the right data? Validate the inputs? Produce the right error messages for the invalid inputs?
I am working on a pure CRUD application right now
But I see lots of benefits of Unit test cases (note- I didn't say TDD)
I write code first and then the test cases- but never too apart- soon enough though
And I test the CRUD operations - persistence to the database as well.
When I am done with the persistence - and move on to the UI layer- I will have fair amount of confidence that my service\persistence layer is good- and I can then concentrate on the UI alone at that moment.
So IMHO- there is always benefit of TDD\Unit testing (whatever you call it depending on how extreme you feel about it)- even for CRUD application
You just need to find the right strategy for- your application
Just use common sense....and you will be fine.
I feel like we are confusing TDD with Unit Testing.
Unit Testing are specific tests which tests units of behaviors. These tests are often included in the integration build. S.Lott described some excellent candidates for just those types of tests.
TDD is for design. I find more often then not that my tests I write when using TDD will either be discarded or evolve into a Unit Test. Reason behind this is when I'm doing TDD I'm testing my design while I'm designing my application, class, method, domain, etc...
In response to your scenario I agree with what S.Lott implied is that what you are needing is a suite of Unit tests to test specific behaviors in your application.
TDDing a simple CRUD application is in my opinion kind of like practicing scales on a guitar- you may think that it's boring and tedious only to discover how much your playing improves. In development terms - you would be likely to write code that's less coupled - more testable. Additionally you're more likely to see things from the code consumer's perspective - you'll actually be using it. This can have a lot of interesting side effects like more intuitive API's, better segregation of concerns etc. Granted there are scaffold generators that can do basic CRUD for you and they do have a place especially for prototyping, however they are usually tied to a framework of sorts. Why not focus on the core domain first, deferring the Framework / UI / Database decisions until you have a better idea of the core functionality needed - TDD can help you do that as well.
In your example: Do you want messages to be a queue or a hierarchical tree etc?
Do you want them to be loaded in real time? What about sorting / searching? do you need to support JSON or just html? it's much easier to see these kinds of questions with BDD / TDD. If you're doing TDD you may be able to test your core logic without even using a framework (and waiting a minute for it to load / run)
Skip it. All will be just fine. I'm sure you have a deadline to meet. (/sarcasm)
Next month, we can go back and optimize the queries based on user feedback. And break things that we didn't know we weren't supposed to break.
If you think the project will last 2 weeks and then never be reopened, automated testing probably is a waste of time. Otherwise, if you have a vested interest in "owning" this code for a few months, and its active, build some tests. Use your judgement as to where the most risk is. Worse, if you plan on being with the company for a few years, and have other teammates who take turns whacking on various pieces of a system, and it may be your turn again a year from now, build some tests.
Don't over do it, but do "stick a few pins in it", so that if things start to "move around", you have some alarms to call attention to things.
Most of my testing has been JUnit or batch "diff" type tests, and a rudimentaryy screen scraper type tool I wrote a few years ago (scripting some regex + wget/curl type stuff). I hear Selenium is supposed to be a good tool for web app UI testing, but have not tried it. Anybody have available tools for local GUI apps???
Just an idea...
Take the requirements for the CRUD, use tools like watij or watir or AutoIt to create test cases. Start creating the UI to pass the test cases. Once you have the UI up and passing maybe just one test, start writing the logic layer for that test, and then the db layer.
For most users, the UI is the system. Remember to write test cases for each new layer that you are building. So instead of starting from the db to app to ui layer, start in the reverse direction.
At the end of the day, you would probably have a accumulated a powerful set of regression test set, to give you some confidence in doing refactoring safely.
this is just an idea...
I see what you are saying, but eventually your models will become sufficiently advanced that they will require (or be greatly augmented by) automated testing. If not, what you are essentially developing is a spreadsheet which somebody has already developed for you.
Since you mentioned Rails, I would say doing a standard create/read/update/delete test is a good idea for each property, especially because your test should note permissions (this is huge I think). This also ensures that your migrations work as you expected them to.
I am working on a CRUD application now. What I am doing at this point is writing unit tests on my Repository objects and test that the CRUD features are working as they should. I have found that this has inherently unit tested the actual database code as well. We have found quite a few bugs in the database code this way. So I would suggest you push ahead and keep going with unit tests. I know applying TDD on CRUD apps is not as glamorous as things you might read about in blogs or magazines, but it is serving its purpose and you will be that much better when you work on a more complex application.
These days you should not need much hand written code for a CRUD app apart from the UI, as there are a 101 frameworks that will generate the database and data access code.
So I would look at reducing the amount of hand written code, and automating the testing of the UI. Then I would use TDD of the odd bits of logic that need to be written by hand.