I have a scenario outline that has two different variables. One variable has about 20 different values and the other has 3.
I need to be able to test every single combination and I need each in a separate scenario since they must be tested independently.
Currently, I just write it out by hand(they are integers in the example but not in my test):
Scenario Outline: Test my stuff
Given first var is <var_a>
And second var is <var_b>
When I do stuff
Then good stuff should happen
Examples:
| var_a | var_b |
| 1 | 1 |
| 1 | 2 |
| 1 | 3 |
| 2 | 1 |
| 2 | 2 |
etc...
Question: Is there a way to run this where every combination doesn't have to be written out? The variables are bound to change in size and content later on and I would prefer to have a single data structure to deal with.
Scenario: Test my stuff
Given first var combinations is #var_a_combos
Given second var combinations is #var_b_combos
When I run every combination
Then good stuff should happen
#var_a_combos and #var_b_combos
are an Enumerable that contains every
different possibility for that type.
The When I run every combination
will iterate over both variables and
test every single combination.
Instead of using an assertion, it
will instead create a
#results(String) variable that will
list any failures and their required
debugging data.
The Then good stuff should happen
will run the code #results.should
eql "" to determine whether the test
case passed/failed.
The purpose of using a String variable to hold the results(instead of an assertion) is to ensure that testing does not stop when the first failure is encountered.
I really don't like this solution but it is the best I could come up with so far. I appreciate the other answers but this testing really is integration testing and I really do need to test every combination.
If anyone posts further answers I will review them and change the accepted answer if I think it is better.
Since the tests are for a UI widget, and you're trying to test it at the unit level, you should check out QUnit (or another javascript unit testing framework, but this one's my personal preference). You can even run QUnit tests via cucumber, as in Test sorting with Cucumber and Capybara
Are you sure you really need to test every combination? Could you explain some context around the problem?
Anyway, regarding your query about running everything in a separate scenario, a Scenario Outline with examples is what you need. See the Cucumber docs.
Related
I have a class named QuestionGroup which contains a list of Questions. supposing it has the following structure:
+QuestionGroup
-Title
And a class named Question like this:
+Question
-Title
-Description
-QuestionGroupId
And a class named InteractionGroup which contains a list of Employees:
+InteractionGroup
-Title
and a class named Employee with:
+Employee
-FirstName
-LastName
-InteractionGroupId
and I have a class named AppraisalTemplate . it contains a list of QuestionGroups and a list of InteractionGroups.
+AppraisalTemplate
-Title
-List<QuestionGroup>
-List<InteractionGroup>
now I want to write a specflow feature in which I need to create an AppraisalTemplate before my scenario runs.
my scenario depends on this AppraisalTemplate because in order to execute my When step in the scenario those steps should be processed already.
now this is the feature I have written (steps are not written yet):
Feature: Appraisals
Background:
Given I have an QuestionGroup with Title '<QuestionGroupTitle>' and following Questions
| Title | Description |
| Question 1 | Desc Test 1 |
| Question 2 | Desc Test 2 |
And an InteractionGroup with Title '<InteractionGroupTitle>' and following employees
| FirstName | LastName |
| Clubber | Lang |
| Mickey | Goldmill |
And an AppraisalTemplate with Title '<AppraisalTemplateTitle>' and following QuestionGroup and InteractionGroup
#What should I write here?
Scenario Outline: [add a new appraisal]
Given [given]
When [when]
Then [then]
I was wondering how should I write the 3rd Given step in Background section?
I want to say
Given I have created an AppraisalTemplate with the **mentioned QuestionGroup(s) and InteractionGroup(s) in the last two steps**
How can I do it?
How can I execute a step and use a previously created objects in it?
Maybe I've misunderstood the whole story. If you would please explain me if there is a mistake in my feature as well.
thank you for your time
First of all you should try and simplify you scenario to
Not use example groups
Not use scenario outlines
This will remove lots of complexity which distracts from your core question
Then you should focus on the behaviour you are trying to exercise, which seems to be about filling in an Appraisal.
Then write your Givens and Thens
So ...
When I fill in my appraisal
To do this we might have something like
Given my appraisal is due
and we would implement this with something like
Given "my appraisal is due" do
#appraisal = create_appraisal(user: #I)
end
Now this requires me to be somebody, perhaps an employee
Given I am an employee
And my appraisal is due
When I fill in my appraisal
Then ...
and we can implement that Given with
Given "I am an employee" do
#I = create_employee
end
So now we have left just the idea of creating an appraisal
module AppraisalStepHelper
def create_appraisal(user: )
...
end
end
World AppraisalStepHelper
Notice how there is nothing to do with HOW you interact with an appraisal in our feature or step definitions.
So overall we have
Feature: Employee appraisals
...
Scenario: Fill in appraisal
Given I am an employee
And my appraisal is due
When I fill in my appraisal
Then ...
Now you will probably need a load of over features to allow you to get to here. For example
Feature: Schedule An Appraisal
Feature: Create Appraisal Template
...
There are several important points to note here
cuking is about specifying WHAT you are doing and WHY its important. Its not about specifying HOW things are done
good cukes are simple and easy to read
you want to deal with one small piece of business behaviour in each feature e.g. fill in my appraisal
you simplify by
abstracting
pushing the HOW down (into step definitions and helper methods)
choosing smaller chunks
each Given builds upon a previous when
You need to change your approach to Cuke effectively. Your current approach will overwhelm you with complexity. Hopefully the stuff above helps. Good luck
I have a single test case to test a particular scenario in a particular environment everyday, this is automatically done by a jenkins job.
Scenario Outline: Verify a user can book
Given I navigate to the "xxxxx" Page
And I set the "Location" field with "<location>" value
And I click on the "Search" button on "xxxxx" page
Then I verify the "Results" page is displayed
Examples:
| location |
|Boston |
I need to internally have a list of 20 locations and everytime the test case is executed it changes the location some how, can be ramdon or in any order, but always changing.
I'm using cucumber, capybara and of course ruby
Thoughts please?
Cucumber has a lot of limitations in terms of being used as a programming language. It's easier to do this kind of thing if you move it into a ruby file (cucumber files aren't ruby).
One option would be to make a single step that calls these other steps internally. Some people might say it's better to call methods rather than steps from inside other steps, but if you already have your cases written as steps than this will be quicker to do it this way, because you don't have to rewrite the code into methods. It is a good idea to write test code in methods and then call them from steps, by the way, rather than putting all the logic in the test cases.
Cucumber file:
Scenario Outline: Verify a user can book
Given I navigate to the "xxxxx" Page
Then the search bar works
Ruby file:
Then /the search bar works/ do
locations = ["Boston", "Berkeley"].shuffle
locations.each do |location|
step %{I set the "Location" field with "#{location}" value}
step %{I click on the "Search" button on "xxxxx" page}
step %{I verify the "Results" page is displayed}
end
end
Another reason this could be considered nonidiomatic is because it's packing too much into a single test case. However I'm not sure a good way to get around this other than simply copy-pasting the original step definitions in the cucumber file with different hard-coded values.
It's possible
locations = ["Boston", ...]
day_of_the_month = Date.new(2001,2,3).mday
today_location = locations[(day_of_the_month - 1) % locations.count]
I use - 1 in the third line since #mday returns integer from 1 to 31.
I have a registration form and need to test it using cucumber and ruby.
I decided to user Scenario Outline with different values in table:
Scenario Outline: Log in with valid data
Given I am on the Sign up Form
When I provide <Email>
And I provide Confirm <СEmail>
And I provide <Password>
And I provide Confirm <СPassword>
And I click on Register button
Then I registered to the site
Examples:
| Email | CEmail | Password | CPassword |
| vip17041#yopmail.com |vip17041#yopmail.com | 123 | 123 |
| vip17042#yopmail.com |vip17042#yopmail.com |123 | 123 |
Now I need create steps definition. In step definition I need to put into the fields values from the table.
How could I do that? Previously I used the next method:
When(/^I provide vip(\d+)#yopmail\.com$/) do |email|
browser.text_field(:name, "Email").set("email#yopmail.com")
But how could I set instead of hard coded email - email from my table?
Thanks
If you're looking to merge the capture with the email address:
When(/^I provide (vip\d+)#yopmail\.com$/) do |email|
browser.text_field(:name, "Email").set("#{email}#yopmail.com")
end
This will concatenate the captured text ("vip" literally plus any number of numerical values with a length of one or more) with the string "#yopmail.com"
A note on how Scenario Outline works
Scenario Outline will grab the lines from the examples table, and simply use the columns to create individual scenarios that use the values that match the column headers in place of the placeholders.
For instance:
Scenario Outline: A note
Given I am logged in as <user>
When I go to the homepage
Then I should see "Welcome Back, <display_name>"
Examples:
| user | display_name |
| rick#stley.com | Rick Astley |
| tammy1992 | Tammy Holmes |
Would be converted into two scenarios:
Scenario: A note
Given I am logged in as rick#stley.com
When I go to the homepage
Then I should see "Welcome Back, Rick Astley"
Scenario: A note
Given I am logged in as tammy1992
When I go to the homepage
Then I should see "Welcome Back, Tammy Holmes"
Which makes it no different to writing your normal scenario, the place holders that you use simply complete the step that you are writing.
How I would write your Scenario
Cucumber is a tool meant to bridge the conversational gap between testers, developers and management.
Scenario Outline: Log in with valid data
Given I am on the Sign up Form
When I sign up with the email "<Email>" and password "<Password>"
Then I should be able to log in as "<Email>" with password "<Password>"
Examples:
| Email | Password |
| vip17041#yopmail.com | 123 |
| vip17042#yopmail.com | 123 |
We don't necessarily have to know each individual step of the process, and the feature file shows the intent of the test.
What this seems to be looking for is whether you can log in after registering a new account, so why not write it as such?
As someone who has used Cucumber for many years I would advise you to avoid using Scenario Outlines. Features and scenarios are for expressing intent in simple clear terms, not programming things using tables.
You can write your scenario as
Scenario: I should be welcomed when I sign in
Given I am registered
When I sign in
Then I should be welcomed
Good scenarios state what behaviour they are trying to verify in their title, and then have steps that are consistent with this behaviour. They have no need to explain HOW your application implements that behaviour. Putting that information in your scenarios makes them longer, harder to implement, and much more difficult to maintain.
A side effect of such simple scenarios is that the step definitions are much simpler and easier to write. No regex's params or table parsing needed here.
You can see a simple example of this approach here (https://github.com/diabolo/cuke_up/tree/master/features),
I have a few tests in feature files that use the Scenario Template method to plug in multiple parameters. For example:
#MaskSelection
Scenario Template: The Mask Guide Is Available
Given the patient is using "<browser>"
And A patient registered and logged in
And A patient selected the mask
When the patient clicks on the "<guide>"
Then the patient should see the "<guide>" guide for the mask with "<guideLength>" slides
Examples:
| browser | guide | guideName | guideLength |
| chrome | mask | Mask | 5 |
| firefox | replacement | Mask Replacement Guide | 6 |
| internetexplorer | replacement | Mask Replacement Guide | 6 |
Currently, this is exporting test results with names like "TheMaskGuideIsAvailableVariant3". Is there any way to have it instead export something like "TheMaskGuideIsAvailable("chrome", "mask", "Mask", "5")"? I have a few tests which export 50+ results, and it's a pain to count the list to figure out exactly which set of parameters failed. I could have sworn the export used to work like this at one time, but I can't seem to replicate that behavior.
Possibly tied to it, recently, I've lost the ability to double-click on the test instance in Test Explorer in Visual Studio and go to the test outline in its file. Instead, nothing happens and I have to manually go to that file.
The answer to the Variant situation is that the part that gets appended is the first column of the table. If there are non-unique items in the first column, it gets exported as numbered "Variants".
The answer I found to exporting the list is to use vstest.console with the "/ListTests" option. As per the prior paragraph, since the first column is the one to be used for naming, a column can be established with a concatenated list of parameters.
I'm trying to run a feature file like this:
Feature: my feature
Background:
When I do something
And I choose from a <list>
Scenario Outline: choice A
And I click on <something> after the choice A is clicked
Examples:
| list | something |
| a | 1 |
| b | 2 |
| c | 3 |
But what happens is when the second Background step runs, in the step definition, list is a String with the value <list>, and the first Scenario line something is 1, so can Background not use the variables from Examples? Putting a copy of Examples before Scenario Outline does not work.
The answer to your question is: no. Background is not a scenario outline. It does not take values from Examples, which is exclusively for the Scenario Outline where it is included.
Let's suposse you have several Scenario Outlines. Each of them should have its own Examples sections and it is not shared between them. Consequently, it is not shared with Background either.
That is why it does not work when you move Examples before Scenario Outline, as you mentioned in your question.