I need to be able to prompt for user input (username, password and authorisation code) so my tests can access a GUI. These details cannot be stored as test data, so they'll have to be input part way through a test.
I've tried the following, but it's not working how I want:
Feature file:
Feature: user input as part of a test
Scenario: user input at the start
Given the test requires a name
Step definition:
Given(/^the test requires a name$/) do
get_a_name
end
Method:
def get_a_name
puts "Gimme a name"
#input_name = gets.chomp
puts "Hello #{#input_name}"
end
Result:
Gimme a name
Hello Feature: user input as part of a test
Any help would be much appreciated. Thanks.
You have a number of options when dealing with external services in test automation. In this particular case you can
Change your source so the authentication behaves differently when testing
OR
Record a response from the external service and use that response instead of going to the external service (see https://github.com/vcr/vcr). As the response is recorded you will know the authorisation code
OR
Use a test version of the external service which gives back know responses
I suspect there are a number of other solutions, but all three of the above are used widely and work fine.
There is certainly no need to have manual interactions to test your system unless you are running your tests against a production system (which is a very bad idea!).
Related
I have been implementing a basic slack bot to a channel and as the code is very simple I have no public methods. Just commands from slack-ruby-bot gem. But as project requirement I need to add several unit testing to my project. Basically when I give command of #ruby_codes strings it gives me the wiki page of that area. how can I test this? attached my version but tests fail everytime.
My command example is as follows:
module RubyCodes
# SlackBot command methods#
class GetWiki < SlackRubyBot::Commands::Base
command 'strings' do |client, data, _match|
client.say(channel: data.channel, text: 'https://en.wikipedia.org/wiki/Ruby_(programming_language)#Strings')
end
My test case is as follows:
describe SlackRubyBot::Commands do
it 'returns ruby strings wiki page' do
expect(message: "#{SlackRubyBot.config.user} strings").to respond_with_slack_message('https://en.wikipedia.org/wiki/Ruby_(programming_language)#Strings')
end
end
Thanks in advance!
also, this is the error I get:
expected to receive message with text: https://en.wikipedia.org/wiki/Ruby_(programming_language)#Strings once,
received:[{:text=>"Sorry <#user>, I don't understand that command!", :channel=>"channel"}]
Is it possible to use JGiven (with or without Spring support) to retrieve the statements before / during execution? For example, if we had a fairly typical login acceptance test i.e.
public class LoginFeatureTest extends SpringScenarioTest<GivenIAmAtTheLoginPage, WhenILogin, ThenTheLoginActionWillBeSuccessful> {
#Test
public void my_login_test() {
given().I_am_a_new_user()
.and().I_am_at_the_login_page();
when().I_login_with_username_$_and_password_$("dave", "dave123");
then().the_home_page_is_visible();
}
}
Is it possible to get something access to the following information?
My Login Test (start)
Given I am a new user
and I am at the login page
When I login with username dave and password dave123
Then the home page is visible
My Login Test (end)
i.e. what i'm looking for is: -
The name of a scenario method + all it's given, when, then and and statement calls _(formatted via JGiven formatting).
When each scenario method starts at run-time.
When each given, when, then and and executes at run-time.
When the scenario method ends.
This will give me the ability to visually show in a UI (a) exactly what is going to execute and (2) it's current position during execution (with durations).
12:00:01.012 [ My Login Test (start) ]
12:00:02.035 23ms Given I am a new user
12:00:02.051 16ms and I am at the login page
----> When I login with username dave and password dave123
Then the home page is visible
[ end ]
I'm thinking Spring AOP might come to the rescue here? Or does JGiven provide anything useful buried in it's code?
At the moment there is no possibility to do that. I have created an issue for that: https://github.com/TNG/JGiven/issues/328.
It should not be too difficult to implement this as there is internally already a listener concept. If you are still interested you could propose an API and integrate the mechanism in JGiven.
Right now I am trying to grab the information from the current scenario (description, file path, name, etc.). I try to get the scenario information in the before hook like this
Before do |scenario|
puts scenario.name
puts scenario.description
#etc.
end
however things like scenario.description are not available for use. This is because when running in both Before and After hook, scenario is being called as Cucumber::RunningTestCase::Scenario instead part of the core AST module, like feature. I want to be able to access the scenario from Cucumber::Core::Ast::Scenario and Cucumber::Core::Ast::OutlineTable::ExampleRow, where values like current scenario description is available. Is it possible to get access to Cucumber::Core::Ast::Scenario in before and after hooks instead of Cucumber::RunningTestCase::Scenario, or another way to grab information like scenario name, description, file path, etc.? This is using the latest version of the cucumber and cucumber core gems (2.4 and 1.5). In addition, I am also no longer to able to access information about the current step, and step count.
Before do |scenario|
all_sources = scenario.outline? ? scenario.scenario_outline.all_source : scenario.all_source
end
From the all_source variable you will have access to several objects from ::Ast
Scenario Outline:
Cucumber::Core::Ast::Feature
Cucumber::Core::Ast::ScenarioOutline
Cucumber::Core::Ast::Examples
Cucumber::Core::Ast::ExamplesTable::Row:
Cucumber::Core::Ast::ExpandedOutlineStep
Scenario:
Cucumber::Core::Ast::Feature
Cucumber::Core::Ast::Scenario
Cucumber::Core::Ast::Step
From these you should be able to get data about file path, descriptions, comments, tags, hooks, line numbers and so on.
Here as you can see I am trying fetch a value from .yml file located in config/environments in Examples: table.
But instead of fetching it is sending the value as it is?
Is it possible to pass parameter like this? If Yes, how?
If not, which Ruby or Cucumber feature/concept refrains user to do so and why?
Feature: Verify login of all test users
I want to verify all test users can login.
Scenario Outline: Login as different users on the website
Given I am on login page
When I enter "<username>" and password
Then I click Login button
And I see "<user>" successfully logged in
Examples:
|user|username|
|testuser1|#{FigNewton.test1_email}|
|testuser2|FigNewton.test2_email|
First of all this is a pretty poor feature, better would be
Scenario: Test Users can login
Given there are some test users
When the test users login
Then all test users should be logged in
or something like that. Features are for stating what you want to do and why, not how you do things.
IF you do the above then all the programming will be done in the step definitions. This will allow you do do whatever you want.
You can implement this quite easily e.g
Given 'there are some test users' do
#test_users = create_test_users
end
When 'the test users login' do
#login_results = login_each(#test_users)
end
Then 'all test users should be logged in' do
expect(check_for_errors(#login_results).count).to eql 0
end
then implement the methods you need in a step helper e.g
module TestUsersLoginStepHelper
def create_test_users
...
def login_each(users)
users.each do
...
...
end
World TestUsersLoginStepHelper
By putting all the work in the step definitions, you make your live much easier, as you can use the full power of ruby to do what you need
Answer to query1:
You can parametrize via Examples: table but not directly passing value using FigNewton gem because it is a .feature file not a Ruby .rb file.
Answer to query2:
How you do it:
Parametrize and Loop it on username and in you steps definition mention what to do when particular user name found. By this you can easily parametrize.
Examples:
|user|username|
|testuser1|test1|
|testuser2|test2|
Step definition
When(/^I enter "([^"]*)" and password$/) do |username|
case username
when 'test1'
on(LoginPage).user_email = FigNewton.test1_email
when 'test'
on(LoginPage).user_email = FigNewton.test2_email
end
....
....
end
You can use this DDD scenario in project whenever its needed - by using this we do not need to create multiple test cases, it will fetch data value from Example outline.
Feature file : Test case
Scenario Outline: Login to application
When I enter "username>" and "password>"
Then I click Login button
And I see user successfully logged in
Examples:
|username|password|
|abc#gmail.com|Test1234!|
|abc#yahoo.com|Test1234!|
Step definition:
When(/^I enter "([^"])" and "([^"])"$/) do |username,password|
sleep 20
on(Login).email_edit_text_element.send_keys username
on(Login).password_edit_text_element.send_keys password
end
Then(/^I click Login button$/) do
sleep 20
on(Login).login_button_element.click
end
Then(/^I see user successfully logged in$/) do
expect(on(Login).account_bg_cover_element.displayed?).to be_truthy
puts 'Login Success'
end
In ruby file, you have created methods, you are calling that methods in step definition.
It will work. make sure about the name of parameter you are passing.
Trying to write up cucumber feature steps for REST API test.
I am not sure which approach is better:
Given I log in with username and password
When I add one "tv" into my cart
And I check my cart
Then I should see the item "tv" is in my cart
or
Given the client authenticate with username and password
When the client send POST to "/cart/add" with body "{item: body}"
Then the response code should be "200"
And the response body should expect "{success: true}"
When the client send GET to "/cart"
Then the response code should be "200"
And the response body should expect "{"items": ["tv"]}"
Is there any convention to follow when people trying to write cucumber steps for REST API?
I just stumbled on this helpful article: https://www.gregbeech.com/2014/01/19/effective-api-testing-with-cucumber/
To summarize...
Scenario: List fruit
Given the system knows about the following fruit:
| name | color |
| banana | yellow |
| strawberry | red |
When the client requests a list of fruit
Then the response is a list containing 2 fruits
And one fruit has the following attributes:
| attribute | type | value |
| name | String | banana |
| color | String | yellow |
And one fruit has the following attributes:
| attribute | type | value |
| name | String | strawberry |
| color | String | red |
Validating a result against JSON is tricky business because if the result is an array, the elements may not be the same order as how you are validating in the test.
Edit: Updated link using finderAUT's comment. Thanks!
Here's a (close enough) example to what the Pragmatic Programmer's "the Cucumber Book" says about testing REST APIs via Cuke,and it seems to more closely relate to your second example:
Feature: Addresses
In order to complete the information on the place
I need an address
Scenario: Addresses
Given the system knows about the following addresses:
[INSERT TABLE HERE or GRAB FROM DATABASE]
When client requests GET /addresses
Then the response should be JSON:
"""
[
{"venue": "foo", "address": "bar"},
{ more stuff }
]
"""
STEP DEFINITION:
Given(/^the system knows about the following addresses:$/) do |addresses|
# table is a Cucumber::Ast::Table
File.open('addresses.json', 'w') do |io|
io.write(addresses.hashes.to_json)
end
end
When(/^client requests GET (.*)$/) do |path|
#last_response = HTTParty.get('local host url goes here' + path)
end
Then /^the response should be JSON:$/ do |json|
JSON.parse(#last_response.body).should == JSON.parse(json)
end
ENV File:
require File.join(File.dirname(__FILE__), '..', '..', 'address_app')
require 'rack/test'
require 'json'
require 'sinatra'
require 'cucumber'
require 'httparty'
require 'childprocess'
require 'timeout'
server = ChildProcess.build("rackup", "--port", "9000")
server.start
Timeout.timeout(3) do
loop do
begin
HTTParty.get('local host here')
break
rescue Errno::ECONNREFUSED => try_again
sleep 0.1
end
end
end
at_exit do
server.stop
end
I've been using cucumber to test and more importantly to document the API that I created using rails-api in my current project. I looked around for some tools to use and I ended up using a combination of cucumber-api-steps and json_spec. It worked well for me.
There is no convention on how to write the cucumber steps. The way you write your steps depends on how you want to use your cucumber suite. I used the cucumber output as the reference for our Angular JS client devs to implement the API client. So my cucumber steps contained the actual JSON requests and responses along with the status code for each scenario. This made it really easy to communicate with a client side team whenever something changed ( especially when the client side team was not physically present at my workplace ).
Everytime I would create or update an API, the CI server would run cucumber as part of the build and move the HTML formatted output to a "build_artifacts" location that can be opened in the browser. The client side devs would always get the most recent reference that way.
I've written all of this down in a blog post about creating a tested, documented and versioned JSON API, hope it helps you in some way.
One of Cucumber's original intents, which contributes to its design, is to bridge the gap between technical implementation, and people who know the business needs, so that the test descriptions could be written and/or understood by non-developers. As such, it's not a great fit to detailed technical specs, or blow-by-blow unit testing.
So that would point me to your first test description, if that's also the reason you are using Cucumber.
There is no major problem with implementing tests like the second version, Cucumber can support it. There probably are not a large number of statement types you would need to parse either. But you could end up fighting the test framework a little, or going against your rationale for using Cucumber in the first place.
As for a convention, I am not aware of enough REST API tests in practice to comment, and none that I have seen tested have used Cucumber as the framework.
Update: Browsing around SO on the subject, I did find a link to this: https://github.com/jayzes/cucumber-api-steps which is more similar to your second format.
There are a few libraries now for server side REST testing with cucumber in Ruby. Here's a couple:
Cucumber-API-Steps. (Recommended)
Cucumber-API. A small tutorial for that is here.
The library I've been using for server side REST testing with cucumber is Cucumber-API-Steps.
Cucumber-API-Steps
Here's how I'd write your test using 'cucumber-api-steps' (Recommended):
#success
Scenario: Successfully add to cart
Given I am logged in
When I send a POST request to “/cart/add” with the following:
| item | body |
Then the response status should be “200”
And the JSON response should have "success" with the text "true"
When I send a GET request to “/cart”
Then the response status should be “200”
And the JSON response should be "{'items': ['tv']}"
And here's what my tests look like using 'cucumber-api-steps':
#success
Scenario: Successfully log in
Given I am logged out
When I send a POST request to “/login” with:
| username | katie#gmail.com |
| password | mypassword |
Then the response status should be “200”
And the JSON response should have "firstName" with the text "Katie"
Cucumber-API
Here's how I'd write your test using 'cucumber-api':
#success
Scenario: Successfully add to cart
Given I am logged in
When I send a POST request to “/cart/add”
And I set JSON request body to '{item: body}'
Then the response status should be “200”
And the response should have key “success” with value “true”
When I send a GET request to “/cart”
Then the response status should be “200”
And the response should follow "{'items': ['tv']}"
And here's what my tests look like using 'cucumber-api':
#success
Scenario: Successfully log in
Given I am logged out
When I send a POST request to “/login” with:
| username | katie#gmail.com |
| password | mypassword |
Then the response status should be “200”
And the response should have key “firstName”
Note about Cucumber-API: There is no way currently to do should have key “firstName” with value “Katie”. The "with value" part has not been done yet.
Also "Follow" expects a JSON file
Another resource is here, but it is old (2011).
I would recommend your first scenario.
From my own experiences I personally feel that the biggest value you get from using BDD as a software delivery method, is when you place the emphasis on business value.
In other words the scenarios should be examples of what behaviour the business wants, rather than technical implementation. This ensures that the development is driven by the goals of the business and the deliverables match their expectations.
This is known as outside-in development.
Additional tests of the system behaviour can and should be used to cover the technical requirements but I think there's less value in spending effort writing these up in natural language, which is often time consuming and laborious across a large number of scenarios.
I recommend the following approach:
1) Work with the BAs and POs to develop examples of the behaviour they want using non-implementation specific language (like your first example).
2) Engineers use these to drive the development from a test first approach, automating them as integration tests - with the majority below the browser (against your REST API for example) and the most core scenarios also through the browser(if you are developing one).
3) Engineers TDD the feature code with unit tests until both the unit tests and BDD examples pass.
I think the first one is better. I would put the technical in the ruby classes and modules. E.g like module cart.add(items) in the when step and in the then step put expect(cart.item).to include('items' => a_string_matching(item))
By this, the ruby classes and modules can be reuse in another features steps. E.g like maybe you have another scenario which would add multiple items into the cart then validate the total amount.
However, the second 1 I think can make it like technical features. E.g like common/global header or body request is expected across all the api.
See here: https://github.com/ctco/cukes-rest. It provides a Cucumber DSL to test RESTful APIs.