What are some strategies for writing BDD tests, which can test behaviour that relies on certain data being in the system?
For example, say I was working with the following scenario:
Feature: Search for friend
In order to find a friend
As a user
I want to search my list of friends
And filter by 'first name'
How could this test ever succeed unless/until some "dummy" friends had been entered into the system?
More to the point, what "dummy" criteria would the test utilize?
Should I hard-code the name of a friend, assuming it to already exist in the database?
But what if I move my code to a new environment with a fresh database?
Or, should I write code to manually insert dummy data into the system prior to executing each test?
But this would be modifying the internal state of the application from within a test framework, which seems like a bad approach, since we're supposed to be treating the program as a black-box, and only dealing with it through an interface.
Or, would I create other scenarios/tests, in which the data is created using an interface of the program?
For example, 'Feature: Add a new friend to my list'. Then I could run that test, to add a user called 'Lucy', then run the 'Search for friend' tests to search for 'Lucy', which would now exist in the database.
But, then I'd be introducing dependencies between my scenarios, which contradicts the common advice that tests should be independently runnable.
Which one the best strategy? Or is there a better way?
You would use the Given clause in your scenario to get the system into the appropriate state for the test. The actual implementation of this would be hidden in the step definition.
If the data is going to shared across your scenarios then you could have in a background step:
Background:
Given I have the following friends:
| andy smith |
| andy jones |
| andrew brown |
To add these friends you could either insert records directly into the database:
def add_friend(name)
Friend.create!(:name => name)
end
or automate the UI, e.g.:
def add_friend(name)
visit '/friends/new'
fill_in 'Name', :with => name
click_button 'Add'
end
For the scenarios themselves, you would need to think of key examples to validate the behaviour, e.g.:
Scenario: Searching for a existing person by first name
When I search for 'andy'
Then I should see the friends:
| andy smith |
| andy jones |
But I should not see "andrew brown"
Scenario: Searching for a non-existing person by first name
When I search for 'peter'
Then I should not see any friends
You're correct that tests should be independent, so you shouldn't rely on other scenarios to leave the database in a particular state. You will probably need some mechanism to clean-up after each test. For example, the 'database-cleaner' gem if you're using Cucumber and Rails.
You are referring to BDD and Integration style of testing. If you use a decent ORM (NHibernate?) you can create an in-memory database before each test runs and clean it up after the test succeeds and since the db is in memory, it won't take much time comparing to running it on a real database.
You can use the pre/post test hooks to fit in the data necessary for your scenario and clean it up afterwards so that your tests can be run without depending on each other.
Related
I understand that within this framework users are unable to call scenarios within scenarios.
I am trying to create End to End test cases where there are validation points at multiple stages of the test.
Using 'bad' gherkin syntax the process for the scenario would be something like:
Given Item A exists
When User processes Item
Then Warning is displayed
When User accesses Item
Then Warning is not displayed
When User finalises Item
Then Item status = "CURRENT"
And Record Status = "COMPLETED"
The first thing I considered was breaking the scenario into 3 distinct GWT scenarios.
That is fine.
However suppose I now want to create a new end to end scenario that can re-use one of the 3 scenarios created (as you would re-use a function).
How do I do this without duplication of Gherkin code?
I cannot use Background as the sections that I require to re-use are in the middle of the execution.
Steps prior and after may be different.
IN SUMMARY: I am trying to re-use a GWT scenario that is common for many end to end scenarios where the end to end scenarios are inherently different.
Any feedback or assistance you can provide would be greatly appreciated.
Cheers and thanks,
I think I know what you mean. I have done some "meta" scenario's. For example there are ones that test login, maintaining details and creating an account that are very fine grained. Then I have scenario's that test things further along the way, assuming that the happy path has been taken up to that point. So imagine having a scebnario called:
The user is logged in and has successfully created an account and is at the add product screen
For that, in the step definition, I would just combine function calls with hardcoded values:
#When("^The user is logged in and has successfully created an account and is at the add product screen$")
def addProducts(String val) {
navLibrary.loginAsUser("user", "password")
navLibrary.createAccount("some", "params", "you", "need")
navLibrary.navToProducts("some param")
}
And then start with the finer details in new separate steps. You have to think system design, and cascading. For me it meant a lot of initial rework as the tests started growing, but now it's a breeze. Pick the right level of reusability. It is testing code, so it doesn't have to subscribe perfectly to all the programming precepts. It must work, and it must be low maintenance in the long run.
I didn't use Cypress though. My tests are in Groovy with Selenium and Cucumber.
I have a set of functionally similar websites that I want to write cucumber specs for to drive both development, and selennium browser tests. The site are in different languages and will have different URLs, but will have mainly the same features.
An example scenario might be
Scenario Outline: Photo Gallery Next Action
Given I visit a "<photo-gallery-page>"
When I click "<next-button>" in the gallery
Then the photo should advance
Examples:
| photo-gallery-page | next-button |
| www.site1.com/photo-gallery | Next |
| www.site2.com/la-galerie-de-photos | Suivant |
This is fine when I have a small number of scenarios and examples. However I'm anticipating hundred of scenarios and fairly regular launch of new sites. I want to avoid having to edit each scenario to add examples when launching new sites.
I think I need to store all my example variables in a per site configuration, so that I can run the same scenario against all sites. Then I can add new configurations fairly easily and avoid editing all the scenario examples and making them unreadable.
site[:en].photo-gallery-page = 'www.site1.com/photo-gallery'
site[:fr].photo-gallery-page = 'www.site2.com/la-galerie-de-photos'
site[:en].next-button = 'Next'
site[:fr].next-button = 'Suivant'
One option would be to store this config somewhere, then generate the site specific gherkin files using a script. I could then run these generated gherkins which would contain the required examples
I'm wondering if there's an easier way. My other idea was if I can use table transforms to replace the example blocks. I've had a read, but as far as I can tell I can only transform a table (and replace it with a custom code block) if it's an inline table within a step. I can't transform an examples block in the same way.
Have I understood that correctly? Any other suggestions on how best to achieve this?
I wonder if there's a better way... This all feels very brittle.
What if:
Given I follow a link to the gallery "MyGallery"
And the gallery "MyGallery" contains the following photos:
|PhotoID|PhotoName|
|1 |MyPhoto1 |
|2 |MyPhoto2 |
And the photo "MyPhoto1" is displayed
When I view the next photo
Then the next photo "MyPhoto2" should be displayed
Note that you've taken out the notion of button names, etc. - implementation details that are presumably better defined in your step definitions. The behaviour you're defining is simply going to a gallery, viewing an image, requesting the next one, viewing the next image. Define how in your step definitions.
There's some reading I found very useful on this topic at http://cuke4ninja.com/. Download the PDF and check out the web automation section (it details the web automation pyramid).
To address your configuration problem, maybe you could define some kind of config. class and supply it to the step definition files via dependency injection. You could make it site specific by loading from different config. files as you suggested in its constructor. Step definitions could pull the relevant site specific data from the config. class' properties. I think this would make your scenario is more readable and less brittle.
I am new to Cucumber testing.
I have created two features files:
events.feature
partner.feature
and have my step definitions in a step_definitions folder:
./step_definitions/
events.rb
partner.rb
It seems that Cucumber looks in all the .rb files for the step information.
Is there anyway of restricting the feature to look at a specific step definition file?
The reason as to why I want to do this, is because I am getting Ambiguous match errors, even when I use the --guess flag.
The reason as to why I want to do this is for the following reasons. I am testing a CMS, and want to test each of the different content types (events & partners) in separate features.
events.feature
Feature: Add partner
As an administrator I can add a new partner
Scenario: Create partner
Given I am logged in
When I create a partner
Then I should see content
partner.feature
Feature: Add event
As an administrator I can add a new event
Scenario: Create event
Given I am logged in
When I create an event
Then I should see content
Just focusing on the 'then I should see content' which is in both scenarios, the error occurs because in the .rb files I need to include:
partners.rb
Then /^I should see content$/ do
BROWSER.html.should include('SOME PARTNER CONTENT')
end
events.rb
Then /^I should see content$/ do
BROWSER.html.should include('SOME EVENT CONTENT')
end
which means there is an Ambiguous match of "I should see content".
I understand there are different ways of structuring this, i.e. I could create a content feature, and use scenario outlines:
Feature: Add content
As an administrator I can add a new content
Scenario Outline: Create content
Given I am logged in
When I create an <content type>
Then I should see <example content>
Examples:
|event |March Event |
|partner |Joe Blogs |
But I don't want to do this because I want to encapsulate each content type in their own test feature.
So essentially I need to work out how to run specific step files according to the feature I am testing.
Cucumber always loads all files and I don't think that there is a way to override this behavior. Regarding your problem with ambiguous steps - the solution is easy - add parameters to your steps
Then /^(?:|I )should see "([^"]*)"$/ do |text|
page.should have_content(text)
end
And in scenarios just call it like this
Then I should see "PARTNER CONTENT"
free bonus - your scenario is now much more readable
I don't see anything wrong with the alternative approach that you suggested. Separating out the step definitions into logical domains makes sense. However, it seems like you may be trying to take it too far, and that's going to lead to a good deal of duplicated code and issues with ambiguous matches like you're seeing now. I'd recommend doing something like this:
Feature: Add partner
As an administrator I can add a new partner
Scenario: Create partner
Given I am logged in
When I create a partner
Then I should see "partner content"
And, similarly, in your event feature:
...
Then I should see "event content"
Then you could the following in a separate step_definitions/common_steps.rb file:
Then /I should see "(.*)"$/ do |content|
BROWSER.html.should include(content)
end
This step doesn't have anything partner/event specific about it. Instead, the scenarios contain the data-specific strings for your features.
If you are working on a Rails app, the cucumber-rails gem will actually create a bunch of common steps for web application testing for you. Even if you aren't using Rails, it might be useful to take a look at some of these steps.
I've been looking for this, but it appears not to be possible "out of the box".
My solution is to differentiate steps always using some kind of additional description, such as class name, for example:
Scenario: Buildings List
Given I have a Building with code "B1"
And I have a Building with code "B2"
When I go to the list of buildings
Then I should see "B1" building code
And I should see "B2" building code
These "building code" descriptions are all you need not to reuse steps between different files / domains.
I'm writing a web app using Ruby/Sinatra and Datamapper. I have some tables that I want to preload data into. Is there a best practice for that? E.g. I have a table called references
id name url
-- ---- -------------------
1 AOL http://www.aol.com
2 Dell http://www.dell.com
I want to ensure that this data is always loaded when someone starts up the app for the first time and and ensure the IDs do not changes....
Thoughts?
There may be a gem out there to do this for you which I'm not aware of, and if there is, I'd love to hear about it. But, for my last project I simply had my import data in a yaml file and wrote a Ruby class that parsed that yaml file and used DataMapper to insert the data for me.
I then created a Rake task that allowed me to run this Ruby class.
Is this the sort of solution you might be after? If so, I can give you more info on it.
For my RSpec tests I would to automatically associate data files with each test. To clarify, if my tests each require an xml file as input data and then some xpath statements to validate the responses they get back I would like to externalize the xml and xpath as files and have the testing framework easily associate them with the particular test being run by using the unique ID of the test as the file(s) name. I tried to get this behavior but the solution isn't very clean. I wrote a helper method that takes the value of "description" and combines it with FILE to create a unique identifier which is set into a global variable that other utilities can access. The unique identifier is used to associate the data files I need. I have to call this helper method as the first line of every test, which is ugly.
If I have an RSpec example that looks like this:
describe "Basic functions of this server I'm testing" do
it "should give me back a response" do
# Sets a global var to: "my_tests_spec.rb_should_give_me_back_a_response"
TestHelper::who_am_i __FILE__, description
...
end
end
Is there some better/cleaner/slicker way I can get an unique ID for each test that I could use to associate data files with? Perhaps something build into RSpec I'm unaware of?
Thank you,
-Bill
I just learned about the nifty global before and after hooks. I can hide the unique ID creation code there. It makes things much cleaner. I'll probably go with this solution unless there's an even slicker way to acquire a unique ID for each test. Thanks