I am coming from a TDD mindset into BDD. I understand that using BDD is to focus on ensuring the behaviours and business goals of software are being met.
What confuses me is that if I start using BDD in place of TDD, it seems I'm not able to test at such a low-level. For example, when writing a test with a TDD mindset, I might test that properties have been attached to the scope:
it('should attach properties to scope', function () {
expect(MainCtrl.items.length).toEqual(1);
});
In doing this, another developer knows the assignment to scope was expected and required for future use, saving them some debugging should they have removed the assignments or changed defaults etc.
This example doesn't define a behavior so can't be considered BDD. Of course, I could rewrite the test description to make it more behavior orientated, such as, "when the page loads, setup properties for later use so the user can do so and so..." but this seems too abstract.
Is it a done-thing to use both TDD and BDD at the same time? Have BDD available to define the behaviors for all parties involved in the project (including non-technical) and TDD specifically for developers?
Short answer, yes.
However, the distinction between BDD and TDD is not as you've mentioned and I'd like to clear up what "ensuring the behaviours and business goals of software are being met" really means :)
BDD precedes, envelopes and goes beyond the development stage. BDD is far more than a syntax and not just a change from the word "test" to the word "should". BDD is actually a paradigm to writing software that involves the end-to-end business. This page explains that:
BDD is a second-generation, outside–in, pull-based, multiple-stakeholder, multiple-scale, high-automation, agile methodology. It describes a cycle of interactions with well-defined outputs, resulting in the delivery of working, tested software that matters.
BDD starts with deliberate discovery, where a group of people get together and flesh out the details of a story using scenarios. The group of people is typically made up of 1+ of each of the following disciplines: product, development and QA - also referred to as the three amigos. This exercise precedes development and is very good at identifying defects before you even start development and it creates better specifications through a shared understanding.
Once you have a good story with good scenarios, you can drive development using an outside-in testing approach:
This means you start with the acceptance test (avoid the UI if you can), and as you drive development into the system, you employ TDD at the integration and unit levels for things like your services and domain objects. Here you can choose any syntax you like, and what you have used above with "should" is fine.
If you're using Javascript, we created an open source tool called Chimp to make the process of outside-in testing easy.
Finally, I recommend you look at the following links:
Specification by Example
Shallow Depth of Test
Related
I want to know that,does BDD just work in acceptance test level? If not, does it work in unit test level as well? Does BDD have any suggestion For Unit tests?
thank you
BDD is just a way of defining the specifications for an area of functionality. The idea is to bridge the gap between technical and non-technical people by using human readable syntax of some kind and to use specific examples to define the desired behaviour rather than talking in the abstract. As such it is a tool to help people work together and define the business' requirements for the new functionality. This is the primary point of BDD. Not testing.
The definitions that come out of BDD are however, useful for acceptance testing as they define the expected behaviour that was agreed upon. As such many great tools like cucumber are available to facilitate automation of these scenarios to cut down on your testing time.
With regard to using BDD for something like unit tests, the idea of using BDD and non-technical descriptions is to help involve non-technical people. If there are no non technical people involved in the creation of your unit tests (which I guess is the most likely case) then why bother with it? Technical people can read unit tests that are properly written, just fine. The unit tests that you're writing will come out of the functionality that is described by your BDD scenarios anyway.
However, if there is some technical detail of what you're working on that you're having trouble describing, and your team are comfortable working in a BDD way then definitely give the non-technical language and specific examples approach a go. I just wouldn't bother using the the human readable language version of the example in your unit test.
Edit: After reading xmojmr 's comment about your question, I can absolutely see the benefit of using BDD tools and syntax to make your unit tests more readable/easier to plan out, but I think this is quite different from BDD in general, which is more about bridging a communication gap.
BDD actually started at the class level. The first incarnation of JBehave was a replacement for JUnit that avoided the word "test". It was only later that the system-level stuff came along after Dan North explained mock objects to Chris Matts, an analyst at the time who was learning how to code.
These days, even unit testing frameworks don't insist that you start your test methods with the word "test", and the frameworks for the dynamic languages are pretty much derived from RSpec, which was a port of JBehave's early functionality to Ruby anyway.
So, yes, it's perfectly possible to do BDD at that level.
Of course, alannichols is right in saying that the audience is different, being non-technical.
So why would you want to do BDD?
As Dan says in that first link, it turns out that talking about behaviour is more useful than talking about tests. In BDD, we just avoid the word test. We prefer talking about examples and how things should behave rather than pass or fail.
By having conversations around the desired behaviour of a system or class, and providing examples of that behaviour, you can explore it more easily than when you're talking about testing.
However, because it's a non-technical audience, I find it's enough to put the "given, when, then" in comments. You can see an example here. You don't need an explicit BDD tool.
If you can't find another dev to talk about the behaviour with, I suggest you find a rubber duck.
I have a question about test driven development, which is something I am trying to learn.
I have been reviewing a project delivered by a team of my company to see what kind of stuff they include in their tests.
Basically, for every project in the solution, there is a corresponding test project. This includes the projects in the data layer. I have found that the tests in that project are actually hitting the database and making assertions based on retrieved data.
In fact, the tests of the classes in the Services layer are also hitting the database.
Is this normal in TDD?
If not, and if those classes being tested did nothing other than retrieve data, then what would be the best way of testing them?
Dare I say, should they be tested at all? If TDD helps drive out the design, arguably they should.
What do the TDD kings out there say?
Is hitting the database from most tests normal in TDD? Yes, unfortunately, it's more normal than it ought to be. That doesn't mean it's right, though.
There are various styles of TDD:
Outside-In TDD, popularized by GOOS.
Bottom-Up TDD, which is the 'older' style, where you develop 'leaf' components first.
When you use the Outside-In approach, you'd typically start with a few coarse-grained tests to flesh out the behaviour of the system. These might very well hit the a database. That's OK.
However, you can't properly test for basic correctness from the boundary of a complex application alone, because the combinatorial explosion of required test cases is prohibitive. You would literally have to write tens of thousands, or hundreds of thousands of test cases.
Therefore, even with the Outside-In approach, you should write most of the tests at the level of a unit. These tests should not hit the database.
If you're using the Bottom-Up style, you shouldn't hit the database in most cases.
In short, TDD means Test-Driven Development, not necessarily Unit Test-Driven Development, so it may be fine with a few tests hitting a database. However, such tests tend to be slow and fragile, so there should be only a few of them. The concept of the Test Pyramid nicely explains this.
If you want to learn more, you may want to watch my Pluralsight courses
Outside-In Test-Driven Development
Advanced Unit Testing
Has anyone come across any workable approaches to implementing test-driven development (and potentially behavior-driven development) in/for COBOL applications?
An ideal solution would enable both unit and integration testing of both transactional (CICS) and batch-mode COBOL code, sitting atop the usual combination of DB2 databases and various fixed width datasets.
I've seen http://sites.google.com/site/cobolunit/, and it looks interesting. Has anyone seen this working in anger? Did it work? What were the gotchas?
Just to get your creative juices flowing, some 'requirements' for an ideal approach:
Must allow an integration test to exercise an entire COBOL program.
Must allow tests to self-certify their results (i.e. make assertions a la xUnit)
Must support both batch mode and CICS COBOL.
Should allow a unit test to exercise individual paragraphs within a COBOL program by manipulating working storage before/after invoking the code under test.
Should provide an ability to automatically execute a series of tests (suite) and report on the overall result.
Should support the use of test data fixtures that are set up before a test and torn down afterwards.
Should cleanly separate test from production code.
Should offer a typical test to production code ratio of circa 1:1 (i.e., writing tests shouldn't multiply the amount of code written by so much that the overall cost of maintenance goes up instead of down)
Should not require COBOL developers to learn another programming language, unless this conflicts directly with the above requirement.
Could support code coverage reporting.
Could encourage that adoption of different design patterns within the code itself in order to make code easier to test.
Comments welcome on the validity/appropriateness of the above requirements.
Just a reminder that what I'm looking for here is good practical advice on the best way of achieving these kinds of things - I'm not necessarily expecting a pre-packaged solution. I'd be happy with an example of where someone has successfully used TDD in COBOL, together with some guidance and gotchas on what works and what doesn't.
Maybe check out QA Hiperstation. It could cost a lot though (just like every other mainframe product).
It only used it briefly a long time ago, so I cannot claim to be an expert. I used it to run and verify a battery of regression tests in a COBOL/CICS/DB2/MQ-SERIES type environment and found it to be quite effective and flexible.
I would say this could be one of the pieces of your puzzle, but certainly not the whole thing.
Regardless of how you build/run unit tests, you likely need a summary of how well the tests are doing and how well tested the resulting software is.
See our SD COBOL Test Coverage tool, specifically designed for IBM COBOL.
This answer may not be as easy as you (and I) had hoped.
I have heard about COBOLunit before, but I also don't think it's currently being maintained.
Our team develops an enterprise software product for managing Auto/Truck/Ag dealerships the vast majority of which is in AcuCOBOL.
We were able to break some ground in possibly using JUnit (unit testing for Java) to execute and evaluate COBOL unit tests.
This required a custom test adapter that can serve as the piping and wiring for data between the COBOL unit tests and the JUnit framework. In the application to be tested we will then need to add/design hooks that will evaluate the input as test case data, perform the test to which the data relates, and report results to the adapter.
We are at the beginning of this experiment and haven't gotten much past the "it's possible" phase into "it's valuable". The first foreseeable snag (which I think exists in all TDD) is how to build harnesses into the program.
I a bit familiar with rspec [Ruby] and specs [Scala]. Yesturday I passed a tutor for Cucumber. What I disliked about Cucumber is that additionally to describing scenarios (like you would do with spec- or xUnit-style testing) you have to implement extra layer of indirection: translating scenario steps into ruby expressions. For me creating unnecessary (?) extra layers of indirection feels like "heavy-weight" J2EE-way, not like "light-weight" ruby way. Is understandability by "domain experts" the only advantage of Cucumber? Or is there some non-obvious (technical?) advantages for developer/tester as well?
BDD, from a practical standpoint, is highly synonymous with TDD. Rspec is a BDD test framework, as well as Cucumber.
That stakeholders can read and understand Cucumber acceptance specifications is certainly a key advantage, but this fact alone really doesn't get at the real benefit of Cucumber. Your features and scenarios ought to grow somewhat in specificity as the work being done for them moves through the value stream of your team's development cycle.
Some teams may have an analyst scoping out work at the beginning of the cycle. Sometimes this analyst writes gherkin acceptance specifications, but whoever writes the first draft, you would expect them to be fairly coarse grained. They may not cover every un-happy path.
As the developers take up the work, they often discover edge cases and missing scenarios. At this point they can touch base with the analyst, and the results of such conversations should be written into the cucumber features.
In my experience the testers have cultivated an even more critical eye, and thus it is not uncommon to see them add even more scenarios and features. The testers may also uncover defects, which should be added to the cukes to protect us from regression.
The point is that, in addition to providing executable documentation for our code, Cucumber also provides a repository for the state of the team's conversations vis a vis the features being developed.
There is certainly extra overhead to all of this. However, it's worth considering how much overhead is already in your team's process, which Cucumber might serve to streamline. I find that Cucumber helps reduce the amount of thrash that happens in communication about features both within and outside of the team room.
I should also mention that cucumber is intended as full-stack acceptance testing, and therefore should be less fine-grained, relative to your unit tests. And cucumber is not a good substitute for unit and integration tests. I also would never recommend using cucumber to verify aesthetic aspects of your app's UI. Just use it to validate the actions which a user might take when using your app.
Cucumber is designed to help engage business stakeholders in refining developers' and testers' understanding of the system by collaborating to create scenarios that everyone can understand.
The act of engaging business stakeholders pays off because everyone gets a better understanding, they start sharing the same language and carrying that language into the code (see Domain Driven Design's "Ubiquitous Language"), which can lead to better estimates, appreciation of scope, conversations around options for achieving the same goal, etc. etc.
There are certainly other ways of achieving the goals. For instance, on our C# project we talk through the scenarios, write them on a wiki then implement them using a little custom Domain Specific Language rather like this one. The same thing could be done in Ruby.
BDD is the process of learning the places where we thought we knew what we were doing, but it turned out we were wrong - the discovery of ignorance. With Feature Injection and unit-level examples, this happens at multiple levels of granularity, all the way up to the project vision itself. It tends to pay back for itself, but you don't need a BDD framework to do BDD.
The conversations in BDD are the important bits, not the tool you use to capture them (I helped write JBehave and still believe this is true). Automating regression tests is also important to cut down the manual effort which rises as the codebase grows, and Cucumber, DSLs and other BDD tools give you this as a very nice by-product which also help you trace, and drive out, the shared understanding.
Edit: I should mention that the reuse of steps is also important, but it doesn't make much difference whether you use a BDD framework or a DSL for that. It does make the difference between a DSL and just procedurally mimicking every user interaction.
It depends what you want to achieve.
Cucumber adds overhead, can be tricky to get used to, takes time to master.
If you want your domain experts to be able to read your tests, you should definitely give it a try.
On the other hand, if developers are the only people reading your tests, you can probably stick to rspec/unit-test/etc. and write your integration tests in those frameworks. However you might achieve more readable high level documentation by using cucumber.
See for example rspec 2 core features' descriptions in cucumber.
My question is same as the title. I have not done Test driven development yet,I have performed unit testing and I am aware of testing in general but I hear it is much advantageous to follow TDD?. So should I opt for it everytime or are there some preconditions... i just need to know some basic or important requirements when i should opt for TDD. Sorry if this question is too trivial.
I would say whenever you are coding for a project. By this I mean where you are hoping to produce code that will be used by people. This is basically all code apart from research where you are learning and discovering new techniques.
Even if you think the project is just a small one often things spiral up out of control without care. You wil be glad for the tests when you find yourself having to refactor a big sprawl of code.
Also note that tdd isn't just about testing. It is a methodology of development that encourages you to create clean and solid designs.
As you are starting out tdd everything. Once you have more experience then perhaps you can back off and determine when to not tdd.
IMHO if you are not comfortable with TDD, trying to apply it in projects where you will need to interact/use legacy code it's much more complex than applying it in a project from scratch. In fact, there is a whole book about this.
HTH
My advice would be to use unit testing as often as possible.
Caveat: In my experience TDD works best when working with technologies you already have some experience with. It's often hard to write test assertions if you do not know exactly what the desired result looks like (For example, try writing the test for an ASP.NET MVC action method if you never wrote an action method in your entire life). In those cases you're probably better off writing unit tests after implementing the code.
If I am developing something without a user interface, I always use TDD these days. After all, you have to test the software. You can either do the extra work and do TDD, or you can do the extra work and cludge together a user client just for testing. The former tests more completely and in a more repeatable fashion.
Doing TDD against user interface code, on the other hand, hasn't really delivered much value for me. For various business reasons I'm restricted to Visual Studio out of the box for my work, and "recording" tests with VS is a huge time sink, especially when you have to re-record them if you change the UI. I do TDD for the business logic behind the UI, but not the UI itself.