Background
I'm writing both a cucumber library with a variety of different steps to use across multiple projects and attempting to reduce complexity of step definitions by splitting them into 3 different modules, for iOS, Android and Web - specifically projects that include all 3.
One of the steps libraries will include a load of security-based steps that I want to explicitly include into a project before using the steps.
The project split library will be explicitly included depending on what has been specified in the configuration:
if $profile[:app_type] == 'android'
World(ProjectSteps::Android)
else
if $profile[:app_type] == 'ios'
World(ProjectSteps::IOS)
else
World(ProjectSteps::Web)
end
end
These are not to replace the helper methods, but to save us time when starting up on new projects and will also allow us to have project specific step definitions written in different ways depending on whether we are testing the web, native iOS app or native Android app built to have the exact same functionality, but are different enough to require a different step definition
The Issue
After defining the step within a module, the feature files can still execute this happily, even if the module has not been included in the "World" like this: World(CommonSteps::Security), which is what you would usually use to let cucumber know about helper methods hidden away inside of modules.
When 'I provide my personal details' do
select :title, 'Mr'
fill :first_name, 'John'
fill :last_name, 'Doe'
unless $profile[:app_type] == 'web'
click :progress
end
if $profile[:app_type] == 'android'
fill :postcode, 'TE37ER'
select :address, '1 Test Street'
click :progress
fill :occupation, 'Tester'
fill :company, 'Test Company'
click :progress
else
fill :occupation, 'Tester'
fill :company, 'Test Company'
unless $profile[:app_type] == 'web'
click :progress
end
fill :postcode, 'TE37ER'
select :address, '1 Test Street'
click :progress
end
end
This step definition is trying to test 3 apps at once, but it is testing the exact same functionality, the exact same scenarios and the exact same features. If this was split into 3 step definitions, then it would be simpler to debug in the future, but the same feature file would be able to be used for each of them (which isn't out of the question, as there are many apps that share the exact same functionality across web and native mobile variants). In my opinion, this type of step definition is trying to achieve too much.
This would be easier to maintain despite being more, as it is simpler:
module ProjectSteps::IOS
When 'I provide my personal details' do
select :title, $user[:title]
fill :first_name, $user[:first_name]
fill :last_name, $user[:last_name]
click :progress
fill :occupation, $user[:occupation]
fill :company, $user[:company]
click :progress
fill :postcode, $user[:postcode]
select :address, $user[:line1]
click :progress
end
end
module ProjectSteps::Android
When 'I provide my personal details' do
select :title, $user[:title]
fill :first_name, $user[:first_name]
fill :last_name, $user[:last_name]
click :progress
fill :postcode, $user[:postcode]
select :address, $user[:line1]
click :progress
fill :occupation, $user[:occupation]
fill :company, $user[:company]
click :progress
end
end
module ProjectSteps::Web
When 'I provide my personal details' do
select :title, $user[:title]
fill :first_name, $user[:first_name]
fill :last_name, $user[:last_name]
fill :occupation, $user[:occupation]
fill :company, $user[:company]
fill :postcode, $user[:postcode]
select :address, $user[:line1]
click :progress
end
end
When 'some thing that is the same across platforms' do
# Some stuff
end
Bear in mind, this is a simple version of what I'm trying to achieve, and doesn't show the full complexity of some of the issues that I'm trying to resolve. In this case, I would most likely go with the if/unless version rather than the split, but there are a few cases that are drastically more complex, and would benefit from being split into 3 sections.
We could also add silent checks for bugs that we found in development to make sure that these don't regress, and as the web, android and ios apps feature different bugs, we'd end up with a large amount of if/unless statements.
What have I tried? - I hear you ask
Well I'm either really close, or really far off.
Given, When and Then don't work as expected when within a different module, which is why I had to search for the method that I believe them to be an alias of.
Here is the resulting code:
require_relative 'xss.rb'
require 'cucumber'
module CommonSteps
module Security
Cucumber::RbSupport::RbDsl.register_rb_step_definition(
'I attempt to write a step definition that has to be included to work',
Proc.new {
# Some stuff here
})
end
end
Registers the step definition absolutely fine. But that's part of the problem. I only want to register this step definition if the module has been included in the world of the project that I'm working on.
It would also mean that we can switch out the Web steps for the iOS and Android steps if we need to, while keeping the feature files the exact same. (Yes I know if statements are a thing, but too many, and step defs get complicated really fast.)
Edit
What I'm trying to achieve is not like the "Web steps" that we have seen in the past. No generic steps show code and are only in the language that we have agreed with the businesses that we work with, alongside the development team. As a lot of the projects we work on are cross platform, I'm in essence trying to achieve something that will switch which type of step definition is being used. - if you're using Chrome, use the Web version of this step definition, if you're using iOS, use the iOS version of this step definition, but also a means to include a variety of generic steps that are powered by text, which can link back to our page object model - keeping the scenarios completely business-based.
Given I am on the "Personal Details" page # (generic)
When I provide my personal details # (non-generic, but Web, iOS and Android versions exist)
But leave the "First Name" field blank # (generic)
And I attempt to continue to the next page # (generic)
Then I should see a validation error for the "First Name" text box stating: "Please provide your first name" # (generic)
For example, and if the validation is something that the business wants to know is working, and it'a part of the requirements that have been agreed with the business, is there a more business understandable way to communicate that information? - the why in this, is that we want to make sure the user fills in the information so that the validation does not show, but in the case that they don't provide that information, we should be also testing for the validation appearing in scenarios where it is needed.
We use the page object model in such a way that "Personal Details" will find the :personal_details key in the urls map. The key can be passed down to the next steps, linking to the Pages.personal_details method, which contains the :first_name key. All of our projects use this setup, and it's part of the documentation that goes with our core helper method library.
What I'm trying to achieve is not necessarily bad practice when used in the way that I'm suggesting, but could, if used incorrectly, be used as such.
There have been quite a few times in Cucumber's history when things like this have been done Cucumber itself used to have web steps, which was removed some time ago. These are now in a Gem https://github.com/cucumber/cucumber-rails-training-wheels. You might get some tips from that for your library.
That said, I would strongly recommend not writing a library of step definitions. Instead I would write a library of methods that can be used by step definitions. A crude example might help illustrate this.
Lets say you have a really complex method for logging in to an application. You could write a really commplicated step definition that deals with all sorts of logging in things such as
When I login (hugely complex regex to deal with things like ...
# loads of code to deal with you params and regex's and make things work with lots of different scenarios
or you could write a method like
def login(as:, args={})
and let people use this method when they write stuff e.g.
When 'I login' do
login as: #i
end
When 'I login as Fred' do
login as: create_or_find_user(firstname: 'Fred')
end
or
When 'I login as Fred with Jill's password' do
login as: #fred, password: #jill.password
end
The helper methods provide utility to help you write simple step definitions that are appropriate to your individual context. A shared step definition restricts you to using something that is highly complex and cannot have anything context specific.
Scenarios should be context specific and allow flexible simple language that is specific to the context of the individual World they are part of. They should be all about Why something is being done and What that thing is, and have nothing about How something is being done. By definition they do not share and so by definition neither to step definitions.
Once you've left a step definition by making a call you are into the realm of code, and code is really effective at sharing
Cucumber has learnt the lesson that shared step definitions are a really bad idea (see http://aslakhellesoy.com/post/11055981222/the-training-wheels-came-off). Be wary of repeating past mistakes.
Related
I understand that within this framework users are unable to call scenarios within scenarios.
I am trying to create End to End test cases where there are validation points at multiple stages of the test.
Using 'bad' gherkin syntax the process for the scenario would be something like:
Given Item A exists
When User processes Item
Then Warning is displayed
When User accesses Item
Then Warning is not displayed
When User finalises Item
Then Item status = "CURRENT"
And Record Status = "COMPLETED"
The first thing I considered was breaking the scenario into 3 distinct GWT scenarios.
That is fine.
However suppose I now want to create a new end to end scenario that can re-use one of the 3 scenarios created (as you would re-use a function).
How do I do this without duplication of Gherkin code?
I cannot use Background as the sections that I require to re-use are in the middle of the execution.
Steps prior and after may be different.
IN SUMMARY: I am trying to re-use a GWT scenario that is common for many end to end scenarios where the end to end scenarios are inherently different.
Any feedback or assistance you can provide would be greatly appreciated.
Cheers and thanks,
I think I know what you mean. I have done some "meta" scenario's. For example there are ones that test login, maintaining details and creating an account that are very fine grained. Then I have scenario's that test things further along the way, assuming that the happy path has been taken up to that point. So imagine having a scebnario called:
The user is logged in and has successfully created an account and is at the add product screen
For that, in the step definition, I would just combine function calls with hardcoded values:
#When("^The user is logged in and has successfully created an account and is at the add product screen$")
def addProducts(String val) {
navLibrary.loginAsUser("user", "password")
navLibrary.createAccount("some", "params", "you", "need")
navLibrary.navToProducts("some param")
}
And then start with the finer details in new separate steps. You have to think system design, and cascading. For me it meant a lot of initial rework as the tests started growing, but now it's a breeze. Pick the right level of reusability. It is testing code, so it doesn't have to subscribe perfectly to all the programming precepts. It must work, and it must be low maintenance in the long run.
I didn't use Cypress though. My tests are in Groovy with Selenium and Cucumber.
I am a big advocate of the Page Object Pattern (POP) as defined by the experts at Selenium:
https://code.google.com/p/selenium/wiki/PageObjects
A key view of theirs that I have always followed when using Appium with Java is:
"Methods return other PageObjects"
e.g. LoginPage loginPage = homePage.gotoLoginPage();
I am now trying to following POP using Calabash with Ruby and so have been writing code like this:
e.g. #login_page = #home_page.goto_login_page
However, since Ruby doesn't know what type of object #login_page is or #home_page is, you dont get any of the benefits of intellisense showing what methods are available for a given page.
Anyone know a good way around this?
As much as I appreciate and apply PO design pattern, as much I disagree with returning page object by page object. Page object should be independent and don't need to know about other page objects. Look at two examples:
You test form validation. Click on submit button returns page object which is subsequent in the workflow, but in this case you remain on page with validation errors. Your page object won't know about it and will return the other page.
Page which you get to after clicking a button may differ depending on the context (e.g. from what other page you got to current page). It can lead to having multiple versions of actually same method, which will return different page objects depending on context. This is not good and overcomplicates simple thing.
If you want to return current page object, you can benefit from it e.g. in Java, when you return this at the end of the method. Then you can chain all methods you execute as long as you are on the same page. But when it comes to the question 'how to implement returning different page objects' - answer is simple - 'just don't'. Please note wiki entry you quoted has not been updated for a good while and best practices has evolved since it was originally published.
It seems like you already have your solution. However for others and perhaps also for you the x-platform approach to calabash uses page objects so you could check out that implementation https://github.com/calabash/x-platform-example
An alternative method would be as follows. Not as neat as I would like (given the need to manually create new instances of subsequent pages), but available as an alternative option:
When(/^I buy a movie from the movie page$/) do
movie_page = MoviePage.new
movie_page.buyMovie("Test Movie")
purchase_page = PurchasePage.new
purchase_page.confirmPurchase
end
Found a way of getting this to work after much research and applying well known Java/C#/Obj-c principles to Ruby:
Given(/^I am on the launch page$/) do
#launch_page ||= LaunchPage.new
end
When(/^I open the set alarm time page$/) do
#set_alarm_page = #launch_page.goto_set_alarm_page
end
When(/^I open our apps from the home page$/) do
#launch_page.navigation_toolbar.open_our_apps
end
Then(/^I should see the homepage alarm time is (\d+)$/) do |alarm_time|
alarm_time_actual = #launch_page.get_alarm_time
assert_equal(alarm_time, alarm_time_actual)
end
As long as somewhere on the step definition class you explicitly create a new page object (in the above example: LaunchPage.new), then all subsequent pages will provide intellisense method/property values, since the resulting page types returned will be known by RubyMine.
Context:
I'm trying to make reusable step definitions that click on page objects on the current page,
e.g. (cucumber step def follows):
When(/^the user clicks the "([^"]*)" button$/) do |button|
click_button = button.downcase.gsub(" ","_")
#current_page #somehow get current page object on this line
#current_page.click_button
end
Problem statement:
I can't find anything that returns the current page object.
An explanation for why the obvious solution didn't work:
I thought #current_page was already there as something I could use. I looked in the source code for page object, and the variable #current_page does exist. Not sure how to use it if I can...
BTW, in this case, I have a bunch of testers that can write Gherkin but not necessarily step definitions. We are trying to rapidly finish a bunch of regression tests for an in house app with an unchanging interface.
This is somewhat at odds with what page-object is trying to provide.
Page object attempts to provide well named actions for interacting with a specific page. If you are wanting to make something that works in general against any page, it will be much easier to write it with watir-webdriver directly.
That said, I agree that a specification based heavily on implementation like that is likely to change. I also would add that it doesn't add much value. I would only continue down this path if you understand and accept that you are using cucumber as a test templating tool instead of a requirements communication tool.
As Justin Ko mentioned, #current_page gets set when you call the on or visit methods. Its not a good idea to lump something that changes the page object in a step that performs a specific action (in this case clicking a button). You might want a different step that indicates the behavior of the application, such as
the application lands on the <your page> page
Then you're can use the name of the page object class to load #current_page via the on method in that step definition. This also gives the benifit (or curse of having your step having more lower level details) of indicating expected page navigation behavior.
I am new to Cucumber testing.
I have created two features files:
events.feature
partner.feature
and have my step definitions in a step_definitions folder:
./step_definitions/
events.rb
partner.rb
It seems that Cucumber looks in all the .rb files for the step information.
Is there anyway of restricting the feature to look at a specific step definition file?
The reason as to why I want to do this, is because I am getting Ambiguous match errors, even when I use the --guess flag.
The reason as to why I want to do this is for the following reasons. I am testing a CMS, and want to test each of the different content types (events & partners) in separate features.
events.feature
Feature: Add partner
As an administrator I can add a new partner
Scenario: Create partner
Given I am logged in
When I create a partner
Then I should see content
partner.feature
Feature: Add event
As an administrator I can add a new event
Scenario: Create event
Given I am logged in
When I create an event
Then I should see content
Just focusing on the 'then I should see content' which is in both scenarios, the error occurs because in the .rb files I need to include:
partners.rb
Then /^I should see content$/ do
BROWSER.html.should include('SOME PARTNER CONTENT')
end
events.rb
Then /^I should see content$/ do
BROWSER.html.should include('SOME EVENT CONTENT')
end
which means there is an Ambiguous match of "I should see content".
I understand there are different ways of structuring this, i.e. I could create a content feature, and use scenario outlines:
Feature: Add content
As an administrator I can add a new content
Scenario Outline: Create content
Given I am logged in
When I create an <content type>
Then I should see <example content>
Examples:
|event |March Event |
|partner |Joe Blogs |
But I don't want to do this because I want to encapsulate each content type in their own test feature.
So essentially I need to work out how to run specific step files according to the feature I am testing.
Cucumber always loads all files and I don't think that there is a way to override this behavior. Regarding your problem with ambiguous steps - the solution is easy - add parameters to your steps
Then /^(?:|I )should see "([^"]*)"$/ do |text|
page.should have_content(text)
end
And in scenarios just call it like this
Then I should see "PARTNER CONTENT"
free bonus - your scenario is now much more readable
I don't see anything wrong with the alternative approach that you suggested. Separating out the step definitions into logical domains makes sense. However, it seems like you may be trying to take it too far, and that's going to lead to a good deal of duplicated code and issues with ambiguous matches like you're seeing now. I'd recommend doing something like this:
Feature: Add partner
As an administrator I can add a new partner
Scenario: Create partner
Given I am logged in
When I create a partner
Then I should see "partner content"
And, similarly, in your event feature:
...
Then I should see "event content"
Then you could the following in a separate step_definitions/common_steps.rb file:
Then /I should see "(.*)"$/ do |content|
BROWSER.html.should include(content)
end
This step doesn't have anything partner/event specific about it. Instead, the scenarios contain the data-specific strings for your features.
If you are working on a Rails app, the cucumber-rails gem will actually create a bunch of common steps for web application testing for you. Even if you aren't using Rails, it might be useful to take a look at some of these steps.
I've been looking for this, but it appears not to be possible "out of the box".
My solution is to differentiate steps always using some kind of additional description, such as class name, for example:
Scenario: Buildings List
Given I have a Building with code "B1"
And I have a Building with code "B2"
When I go to the list of buildings
Then I should see "B1" building code
And I should see "B2" building code
These "building code" descriptions are all you need not to reuse steps between different files / domains.
My app has 10 features that are enabled/disabled depending upon which of the 3 'types' of account a user has.
Currently, I have 10 methods (one per feature) along the lines of:
def is_FEATURENAME_enabled
case currentuser.accounttype
when "A", "C" # account types allow to see that feature
return true
else
return false
end
end
Then, in each place where I potentially disable a feature, I do
if foo.is_SOMEFEATURE_enable
do stuff to enable that feature
end
It works. It's not that hard to maintain. But there should be a better way. I suspect the right solution is to define some sort of structure (hash? I dunno) in one place that maps enabled features to accounttypes, then have a single method that I call something like:
if foo.is_feature_enabled(:FEATURENAME)
do stuff to enable feature
end
where the method is_feature_enabled looks at currentuser.accountype and checks the mapping structure to see if the identified feature is enabled.
And I suspect the DRY way to define that mapping (given I have WAY more features than account types) is to list all the features ONCE then for each feature list the accounttypes that have access to that feature (not the other way around). That way when I add a new feature I only have to edit ONE line in the mapping. Something like:
FeatureA: usertype1
FeatureB: usertype1, usertype3
FeatureC: usertype2
...
seems more logical and easier to maintain than:
usertype1: FeatureA, FeatureB, FeatureD, FeatureG
usertype2: FeatureC, FeatureD
usertype3: FeatureB, FeatureD, FeatureG, FeatureH
Any suggestions would be appreciated, and instructive for learning The Right Way to do stuff in ruby.
I think you've pretty much discovered the best way to do it on your own-- what you suggest is wise. Just use the feature name as a lookup key for your hash, then take the resulting list and check whether that list contains the account type of the current user.
E.g.,
# For example...
$AllowedUserCastes = {
:CanLogin => ["admin", "paiduser", "crazyuser", "anonymous"],
:CanDrink => ["admin", "21yearolduser", "crazyuser"],
:CanArrest => ["admin", "police"]
}
def featureAllowed?( whichFeature )
$AllowedUserCastes[whichFeature].include? currentUserCaste()
end
It sounds like you're looking for some kind of event dispatcher. I've yet to bump into a very good one in ruby. But I'm sure I've missed a few, so I'll be happy to be stood corrected in the comments.