As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I have no experience neither in TDD nor in BDD. Yes I've created unit tests for existing code a lot, but it's not relevant here. Also I cann't use TDD/BDD at my job but want to try in some hobby project.
I am not sure if I currently grasped the difference between TDD and BDD correctly. For now I just see BDD as evolved TDD with the most destinctive feature being the ability to work on higher level of abstraction (user stories) then TDD. In TDD you basically get same user stories but they are not as explicit as in BDD. Is it correct?
In terms of tools, assuming that statements above are correct, for TDD I should use something like TestNG or JUnit, and for BDD I'll benefit from tools like JBehave.
Now the question is should I first start with TestNG and TDD and only after some succesfull experience with it migrate to JBehave and BDD? Or this is just waste of time and there are no reasons at all preventing me from trying to use Jbehave and BDD from the very beginning?
UPDATE:
After receiving two great answers on my question, and spending some time on additional reading on the topic, I couldn't help myself not to add link to a great article I found. It just repeats same ideas as in two answers to this question below, but maybe with more details. My favorite part of the article:
The best way to do that is to leverage BDD and TDD. Here is an approach:
1. Write requirements as user stories using the BDD grammar/structure.
Do this collaboratively with the key stakeholders.
2. Enter the User
Stories (feature + scenarios) in a BDD tool.
3. Write code to map the
User Stories to tests.
4. Write production code using TDD to make the
tests pass.
As you can see, BDD is not just TDD done right. You could use just
the vocabulary of BDD to improve TDD but that would be like using
only some of the benefits that BDD has to offer us. When we use the
the strengths of both these techniques we will have “Software that
matters” along with “Software that works”.
BDD concentrates on testing the functionality of the app, and ensuring that it meets the stated acceptance criteria of the application as described in the user story.
TDD tends to concentrate on the lower-level code, testing on a per-class basis rather than on a per-functional-unit basis.
If you have a good idea of what you want to achieve in the user story - enough to enumerate a list of things the feature should & shouldn't do - you're in a good position to start BDD. You can write a BDD feature file from that list, and doing this before writing a scrap of code will allow you think about the feature and crystallise your thinking on it. This will help you write more concise correct code.
I would suggest looking at Cucumber-JVM as your tool for implementing the BDD tests, but I'm told JBehave is functionally similar.
Finally, to me BDD and TDD are techniques that compliment each other, not compete - doing the BDD beforehand makes it easier to think about what is required, which makes it easier to think of what low-level functionality is required, which in turn makes it easier to write your unit tests up front. Once the code is written, you then have a suite of tests allowing you to prove the app meets the acceptance criteria.
The way I do BDD is this:
1) Examine the user story for User-Acceptance Criteria
2) Write a feature file (English) containing test scenarios to test each of these conditions.
3) Implement the feature
4) Use Cucumber/JBehave to programmatically implement the tests from the feature file
5) Ensure all the tests pass
We then maintain a library of these tests, and run them as part of our Continuous Integration
A feature file contains instructions in a specific wording, such as:
Given a user goes to our app
When the user clicks on the search link
Then the search window opens
And the user enters XXX in the search box
And the user presses enter
The search results are displayed
Although most of the literature focuses on TDD as unit tests and BDD as acceptance tests, this is not what distinguishes them.
On the implementation-level, BDD is basically TDD with extra attention to language. BDD is not significantly harder to learn, but you will be missing most of the value of BDD as it is geared towards team communication.
My advice would be to learn JUnit if you haven't already because it is the most ubiquitous framework. The hardest part of starting TDD/BDD is figuring out how write tests first and design for testability.
If you want to learn pure TDD/BDD then write acceptance tests (executable specifications) first and let them drive your unit tests and code. This is probably overkill for a small project, but a good exercise.
Related
I want to know that,does BDD just work in acceptance test level? If not, does it work in unit test level as well? Does BDD have any suggestion For Unit tests?
thank you
BDD is just a way of defining the specifications for an area of functionality. The idea is to bridge the gap between technical and non-technical people by using human readable syntax of some kind and to use specific examples to define the desired behaviour rather than talking in the abstract. As such it is a tool to help people work together and define the business' requirements for the new functionality. This is the primary point of BDD. Not testing.
The definitions that come out of BDD are however, useful for acceptance testing as they define the expected behaviour that was agreed upon. As such many great tools like cucumber are available to facilitate automation of these scenarios to cut down on your testing time.
With regard to using BDD for something like unit tests, the idea of using BDD and non-technical descriptions is to help involve non-technical people. If there are no non technical people involved in the creation of your unit tests (which I guess is the most likely case) then why bother with it? Technical people can read unit tests that are properly written, just fine. The unit tests that you're writing will come out of the functionality that is described by your BDD scenarios anyway.
However, if there is some technical detail of what you're working on that you're having trouble describing, and your team are comfortable working in a BDD way then definitely give the non-technical language and specific examples approach a go. I just wouldn't bother using the the human readable language version of the example in your unit test.
Edit: After reading xmojmr 's comment about your question, I can absolutely see the benefit of using BDD tools and syntax to make your unit tests more readable/easier to plan out, but I think this is quite different from BDD in general, which is more about bridging a communication gap.
BDD actually started at the class level. The first incarnation of JBehave was a replacement for JUnit that avoided the word "test". It was only later that the system-level stuff came along after Dan North explained mock objects to Chris Matts, an analyst at the time who was learning how to code.
These days, even unit testing frameworks don't insist that you start your test methods with the word "test", and the frameworks for the dynamic languages are pretty much derived from RSpec, which was a port of JBehave's early functionality to Ruby anyway.
So, yes, it's perfectly possible to do BDD at that level.
Of course, alannichols is right in saying that the audience is different, being non-technical.
So why would you want to do BDD?
As Dan says in that first link, it turns out that talking about behaviour is more useful than talking about tests. In BDD, we just avoid the word test. We prefer talking about examples and how things should behave rather than pass or fail.
By having conversations around the desired behaviour of a system or class, and providing examples of that behaviour, you can explore it more easily than when you're talking about testing.
However, because it's a non-technical audience, I find it's enough to put the "given, when, then" in comments. You can see an example here. You don't need an explicit BDD tool.
If you can't find another dev to talk about the behaviour with, I suggest you find a rubber duck.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
From the agile manifesto, agile values:
Individuals and interactions over processes and tools,
Working software over comprehensive documentation,
Customer collaboration over contract negotiation,
Responding to change over following a plan
Yet doesn't TDD create a plan and almost structure out a contract negotiation?
"What are the features you want?"
"1,2,3"
Developer writes tests for 1,2,3 -> Team delivers code
"Here's 1,2,3 give us our money"
It's also a form of comprehensive documentation and also a process. Once the tests are written individuals and interactions no longer matter as much because the "source of truth" is no longer with people but ironed out in the code.
Just wondering how they fit together, if they're opposed or do they work together?
TDD is more like a practice for individual contributors, instead of a process. Test here usually refers to unit test, which is part of development work, instead of comprehensive test suits such as performance, functional, and integration tests.
TDD in certain cases should help individual contributor really think about requirement and implementation (respond to change and come up with working software). I personally do not adopt this practice, but it is an agile practice that can be adopted by a single contributor. Do not confuse it with higher level tests and related documents.
Yet doesn't TDD create a plan
Nope. TDD does not mean "write tests up front" it means "write tests before writing code". The whole "Do as much as you need and no more" comes into play. You are not expected to write the tests for all your features before writing any code, just the feature you are currently on. And then (depending on the level of testing) just a small subset of the feature will need tests now.
It's also a form of comprehensive documentation and also a process
It also aids with working software.
Working software over comprehensive documentation,
Over, not instead of. If you can get both, great.
the tests are written individuals and interactions no longer matter as much because the "source of truth" is no longer with people but ironed out in the code.
The oracle for what it does is always the code. The oracle for what it should do is always people.
TDD done well also aids with the communication.
Any insight as to why some people seem to be getting mad at the question?
The question comes off as very troll-y. You are twisting the manifesto to make it sound like anything that aids the latter is "bad" and you are twisting the definition of TDD to be an all-encompassing, completely up-front process. Neither of which are true.
Individuals and interactions over processes and tools,
BDD is a great tool for aiding interactions at a dev/BA/stake holder level. TDD (xUnit and alikes) are great tools for aiding interactions at a dev level.
Working software over comprehensive documentation
TDD helps create working software.
Customer collaboration over contract negotiation
(BDD) Being able to describe in a common language the specification and have that execute is awesome.
Responding to change over following a plan
A well tested code base can change with ease. An untested or badly tested code base is fixed.
That is, while there is value in the items on
the right, we value the items on the left more.
I also agree with Tom’s answer, ‘if it is possible to do agile well, then I believe that is always a good thing. If it is not possible to do it well, then I believe that it can be harmful.’ Agile simply isn’t the right answer for every company. It is different to do well in a large company and it’s lack of focus on software architecture can really affect the usefulness of the resulting technology. Digital Animal have written an interesting article on Agile and why it doesn’t work for them. http://digitalanimal.com/blog/slaying-the-agile-dragon-the-game-of-thrones-methodology/?AT=D8c953
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
The other day I was showing one of the testers at my company some tests I had written in cucumber (2 features, 5 scenarios).
Then he asked me question that I could not answer:
How is this better than selenium or any other functionality test recording tool?
I understand that cucumber is a different technology and it's placed at a different level of testing, but I can't understand why I should bother to write and maintain Cucumber/Capybara tests.
Can someone give me a reasonable explanation for using Cucumber/Capybara instead of just Selenium?
This question is borderline asking for an opinion. Your question actually reads to me, "What tool is right for me?" I say this because you don't give a reason for why you chose Cucumber and Capybara. I believe to answer that tester's question, you need to answer a couple more questions first:
1.) What stage in the process are you going to be writing these tests?
Cucumber may not be the right choice for unit tests, depending on the language you're using. But it can be used for any level of testing, from unit to integration to end-user.
2.) Who is going to maintaining your tests? You? Other developers? Testers? Business Analysts? Project Managers?
Automated tests must be maintained, and knowing who will be doing that can help you decide on a tool - as some will be too technical for certain users.
3.) Who is going to be defining new tests?
Cucumber is meant to be used collaboratively between development, QA and business owners. It is the perfect tool for leveraging everyone's knowledge into the automated testing process. It requires the development of an ubiquitous language to be effect however. You can read up on that on James Shore's Art of Agile page.
Once you've answered these questions, you're ready to address the tester's question.
However, there are a couple of points to keep in mind when comparing recording tools (such as Selenium IDE, HP Quick Test Pro, IBM Rational Functional Tester) vs. development tools (nUnit, jUnit, RSpec, Selenium webdriver, Capybara) is that they are targeted towards different audiences. They also have different plusses and minuses.
Recording tools are easy for anyone to use, but the scripts they create are fragile. They break easily and require more maintenance. They are great for one-off automated testing, where you need to get it done quickly and have non-technical manpower.
Development tools have a larger learning curve and require programming (or at the least scripting) experience. The scripts are generally more robust, but require more technical knowledge to maintain. They are a good solution when you want repeatability and plan to use tests for a long time.
I strongly suggest you read The Cucumber Book. It will really help you decide if Cucumber is the right choice for you.
Cucumber isn't only a testing tool. Besides testing Cucumber features also take a role of documentation, a mechanism to collaborate with stakeholders and requirements storage (if you write them in declarative style).
You don't have to use Cucumber with Capybara. You can use selenium directly. But Capybara has the same high-level API for all supporting drivers. It's more high-level than Selenium's and allows to write tests a bit faster. You don't have to change code when you switch from one driver to another.
Tests built using test recording tools are generally much worse. Selenium IDE may produce valid programming code but it's not good-looking and thus quiet difficult to maintain.
Cucumber is tool used to make tests readable to business users. It consists of plain English sentences that are matched using regex to your Capybara steps.
Using recording tools won't do you any good in the long run. They were meant for beginners and aren't that powerful so I recommend you go straight to coding.
You can use Selenium alone for your tests, but I would recommend you continue to use Cucumber for documentation purposes, if you find them useful and easy to work with. After all, Cucumber can use Capybara or the Selenium web driver.
Selenium ide is good for testing features that have mostly visual elements (links, text and etc.). But often web apps have features that don't represent itself as visual elements, like sending emails, queueing jobs, communicating with 3rd party services and etc. You could, for example, test if an 'Email has been sent' message is present after submitting a form. But it doesn't really tell you if an email is actually sent and therefore you aren't really testing the whole feature here.
I a bit familiar with rspec [Ruby] and specs [Scala]. Yesturday I passed a tutor for Cucumber. What I disliked about Cucumber is that additionally to describing scenarios (like you would do with spec- or xUnit-style testing) you have to implement extra layer of indirection: translating scenario steps into ruby expressions. For me creating unnecessary (?) extra layers of indirection feels like "heavy-weight" J2EE-way, not like "light-weight" ruby way. Is understandability by "domain experts" the only advantage of Cucumber? Or is there some non-obvious (technical?) advantages for developer/tester as well?
BDD, from a practical standpoint, is highly synonymous with TDD. Rspec is a BDD test framework, as well as Cucumber.
That stakeholders can read and understand Cucumber acceptance specifications is certainly a key advantage, but this fact alone really doesn't get at the real benefit of Cucumber. Your features and scenarios ought to grow somewhat in specificity as the work being done for them moves through the value stream of your team's development cycle.
Some teams may have an analyst scoping out work at the beginning of the cycle. Sometimes this analyst writes gherkin acceptance specifications, but whoever writes the first draft, you would expect them to be fairly coarse grained. They may not cover every un-happy path.
As the developers take up the work, they often discover edge cases and missing scenarios. At this point they can touch base with the analyst, and the results of such conversations should be written into the cucumber features.
In my experience the testers have cultivated an even more critical eye, and thus it is not uncommon to see them add even more scenarios and features. The testers may also uncover defects, which should be added to the cukes to protect us from regression.
The point is that, in addition to providing executable documentation for our code, Cucumber also provides a repository for the state of the team's conversations vis a vis the features being developed.
There is certainly extra overhead to all of this. However, it's worth considering how much overhead is already in your team's process, which Cucumber might serve to streamline. I find that Cucumber helps reduce the amount of thrash that happens in communication about features both within and outside of the team room.
I should also mention that cucumber is intended as full-stack acceptance testing, and therefore should be less fine-grained, relative to your unit tests. And cucumber is not a good substitute for unit and integration tests. I also would never recommend using cucumber to verify aesthetic aspects of your app's UI. Just use it to validate the actions which a user might take when using your app.
Cucumber is designed to help engage business stakeholders in refining developers' and testers' understanding of the system by collaborating to create scenarios that everyone can understand.
The act of engaging business stakeholders pays off because everyone gets a better understanding, they start sharing the same language and carrying that language into the code (see Domain Driven Design's "Ubiquitous Language"), which can lead to better estimates, appreciation of scope, conversations around options for achieving the same goal, etc. etc.
There are certainly other ways of achieving the goals. For instance, on our C# project we talk through the scenarios, write them on a wiki then implement them using a little custom Domain Specific Language rather like this one. The same thing could be done in Ruby.
BDD is the process of learning the places where we thought we knew what we were doing, but it turned out we were wrong - the discovery of ignorance. With Feature Injection and unit-level examples, this happens at multiple levels of granularity, all the way up to the project vision itself. It tends to pay back for itself, but you don't need a BDD framework to do BDD.
The conversations in BDD are the important bits, not the tool you use to capture them (I helped write JBehave and still believe this is true). Automating regression tests is also important to cut down the manual effort which rises as the codebase grows, and Cucumber, DSLs and other BDD tools give you this as a very nice by-product which also help you trace, and drive out, the shared understanding.
Edit: I should mention that the reuse of steps is also important, but it doesn't make much difference whether you use a BDD framework or a DSL for that. It does make the difference between a DSL and just procedurally mimicking every user interaction.
It depends what you want to achieve.
Cucumber adds overhead, can be tricky to get used to, takes time to master.
If you want your domain experts to be able to read your tests, you should definitely give it a try.
On the other hand, if developers are the only people reading your tests, you can probably stick to rspec/unit-test/etc. and write your integration tests in those frameworks. However you might achieve more readable high level documentation by using cucumber.
See for example rspec 2 core features' descriptions in cucumber.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I've been using TDD for server-side development. I'm not really sure if the benefits of having all of my production code surrounded by unit tests outweigh the disadvantage of spending 4x more time than needed on refactoring.
But when I'm developing UI code, I simply cannot apply TDD. To all fundamentalists out there, the first law of TDD states that "you may not write production code until you have written a failing unit test". But how can this be if you are developing UIs?
(One can use acceptance test frameworks like Selenium, but that doesn't count, because you don't interact directly with the source code.)
So, can I tell my manager that because of the new >90% code coverage policy I cannot write user interface code?
If you find that writing TDD causes you to spend 4x more time on refactoring, you need to be writing better, more isolated tests, and really let the tests drive the design, as intended. You are also not counting the time you spend in the debugger when you refactor without tests, not to mention how much time everyone else spends on bugs you introduce when you refactor.
Anyway, here is some good advice about what TDD means for UI development. How much that will translate into code coverage depends heavily on the UI framework.
Definitely don't tell your manager you can't do it, he may just replace you with someone who can.
First off, even Robert Martin has testing challenges with UIs.
When TDDing a UI, you write "behavioral contracts" that get as close to the action as possible. Ideally that means unit tests. But some UI frameworks make that inordinately difficult, requiring that you step back and use integration or "acceptance" tests to capture how you expect the UI to behave.
Does it not count if you can't use unit tests? That depends on which rules you're using to keep score. The "only unit tests count" rule is a good one for beginners to try to live with, in the same vein as "don't split infinitives" or "avoid the passive voice". Eventually, you learn where the boundaries of that rule are. In one podcast, Kent Beck talks about using combinations of unit and integration tests, as appropriate (adding, if I recall correctly, that it doesn't bother him).
And if TDD is your goal, you can most certainly write Selenium tests first, though that can be a slow way to proceed. I've worked on several projects that have used Selenium RC to great effect (and great pain, because the tests run so slowly).
Whatever your framework, you can Google around for TDD tips from people who've fought the same battles.
TDD is about testing methods in isolation. If you want to test your UI you are doing integration tests and not unit tests. So if you carefully separate the concerns in your application you will be able to successfully apply TDD to ANY kind of project.
That policy sounds a little artificial, but I would agree with the answer that UIs require functional test cases, not unit test. I disagree however with the point about which comes first. I've worked in an environment where the UI functional tests had to be written before the UI was developed and found it to work extremely well. Of course, this assumes that you do some design work up front too. As long as the test case author and the developer agree on the design it's possible for someone to write the test cases before you start coding; then your code has to make all the test cases pass. Same basic principle but it doesn't follow the law to the letter.
Unit tests are inappropriate for UI code. Functional tests are used to test the UI, but you cannot feasibly write those first. You should check with your manager to see if the >90% code coverage policy covers UI code as well. If it does, he should probably seriously rethink that move.
Separate the business logic from the UI and ensure that the UI code takes less than 10% of the total? Separation of concerns is the main goal of TDD, so that's actually a good thing.
As far as 90% coverage goes ... well, best course is to review extant literature (I'd focus on Kent Beck and Bob Martin), and I think you'll find support for not following a mindless coverage percentage (in fact, I think Uncle Bob wrote a blog post on this recently).
A having a >90% code coverage is dumb because a smart dev can get 100% coverage in one test. ;)
If you are using WPF, you can test your UI code if you use the MVVM pattern. By test your UI code, i mean you can test the ModleView but there is nothing that I know that can test XAML.
Read Phlip's book