How to run Cucumber headless tests on Jenkins Server (Linux)?
What's the proper way to use the 'headless' gem executing Cucumber/Watir-Webdriver tests on Jenkins?
First attempt:
I have this in my env.rb:
require 'rubygems'
require 'cucumber'
require 'rest_client'
require 'json'
require 'watir-webdriver'
require 'page-object'
require 'page-object/page_factory'
require 'allure-cucumber'
require 'rspec'
require 'data_magic'
require 'fig_newton'
require 'yaml'
require 'headless'
require 'phantomjs'
ENVT = FigNewton.load('staging.yml')
DATA = DataMagic.load('data.yml')
KBA = YAML.load_file('config/data/kba.yml')
#Actions performed before each scenario
headless = Headless.new
headless.start
browser = Watir::Browser.start
Before do
#browser = browser
end
at_exit do
browser.close
headless.destroy
end
World(PageObject::PageFactory)
And this is the output I get:
wrong number of arguments (0 for 1+) (ArgumentError)
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/watir-webdriver-0.6.11/lib/watir-webdriver/browser.rb:28:in `start'
/var/lib/jenkins/jobs/idme_revelator/workspace/features/support/env.rb:24:in `<top (required)>'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/rb_support/rb_language.rb:95:in `load'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/rb_support/rb_language.rb:95:in `load_code_file'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime/support_code.rb:180:in `load_file'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime/support_code.rb:83:in `block in load_files!'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime/support_code.rb:82:in `each'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime/support_code.rb:82:in `load_files!'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime.rb:184:in `load_step_definitions'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/runtime.rb:42:in `run!'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/lib/cucumber/cli/main.rb:47:in `execute!'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/cucumber-1.3.18/bin/cucumber:13:in `<top (required)>'
/var/lib/jenkins/.rbenv/versions/2.1.0/bin/cucumber:23:in `load'
/var/lib/jenkins/.rbenv/versions/2.1.0/bin/cucumber:23:in `<main>'
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Second Attempt:
When I make ONE change in the env file on this line:
browser = Watir::Browser.start
to
browser = Watir::Browser.new
Then I get this output, but still error:
Could not find Firefox binary (os=linux). Make sure Firefox is installed or set the path manually with Selenium::WebDriver::Firefox::Binary.path= (Selenium::WebDriver::Error::WebDriverError)
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/firefox/binary.rb:127:in `path'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/firefox/binary.rb:60:in `execute'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/firefox/binary.rb:34:in `start_with'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/firefox/launcher.rb:70:in `start_silent_and_wait'
Third Attempt:
If I change my env.rb file to:
require 'rubygems'
require 'cucumber'
require 'rest_client'
require 'json'
require 'watir-webdriver'
require 'page-object'
require 'page-object/page_factory'
require 'allure-cucumber'
require 'rspec'
require 'data_magic'
require 'fig_newton'
require 'yaml'
require 'headless'
require 'phantomjs'
ENVT = FigNewton.load('staging.yml')
DATA = DataMagic.load('data.yml')
KBA = YAML.load_file('config/data/kba.yml')
#Actions performed before each scenario
headless = Headless.new
headless.start
# browser = Watir::Browser.start
Before do
#browser = headless
end
at_exit do
#browser.close
headless.destroy
end
World(PageObject::PageFactory)
It looks a lot better but still fails:
WARN: Unresolved specs during Gem::Specification.reset:
mini_portile (~> 0.6.0)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
Using the default profile...
#regression #hlp #hlp_admin
Feature: Test the HLP DocumentPage functionality
Background: # features/HLP_features/hlp_DocumentPage.feature:4
* I visit the Hosted Landing Page Admin tool # features/step_definitions/hlp/hlp_admin/hlp_admin_step_defs.rb:1
Unable to pick a platform for the provided browser. (RuntimeError)
./features/step_definitions/hlp/hlp_admin/hlp_admin_step_defs.rb:2:in `/^I visit the Hosted Landing Page Admin tool$/'
features/HLP_features/hlp_DocumentPage.feature:5:in `* I visit the Hosted Landing Page Admin tool'
* I login to the Hosted Landing Page Admin # features/step_definitions/hlp/hlp_admin/hlp_admin_step_defs.rb:5
* I visit the Add DocumentPage page # features/step_definitions/hlp/hlp_admin/hlp_admin_document_step_defs.rb:1
undefined method `window' for #<Headless:0x007f47cb1854b0> (NoMethodError)
/var/lib/jenkins/jobs/revelator/workspace/features/support/hooks.rb:23:in `After'
undefined method `driver' for #<Headless:0x007f47cb1854b0> (NoMethodError)
/var/lib/jenkins/jobs/revelator/workspace/features/support/hooks.rb:16:in `After'
Scenario: Add a new DocumentPage # features/HLP_features/hlp_DocumentPage.feature:9
* I create a new DocumentPage # features/step_definitions/hlp/hlp_admin/hlp_admin_document_step_defs.rb:5
* I should see DocumentPage edit page elements # features/step_definitions/hlp/hlp_admin/hlp_admin_document_step_defs.rb:10
* I delete the "DocumentPage" # features/step_definitions/hlp/hlp_admin/hlp_admin_edit_shared_step_defs.rb:1
So what's the proper way to use the 'headless' gem executing Cucumber/Watir-Webdriver tests on Jenkins?
UPDATE: How to use Xvfb and Watir?
my env.rb:
require 'rubygems'
require 'cucumber'
require 'json'
require 'watir-webdriver'
require 'page-object'
require 'page-object/page_factory'
require 'allure-cucumber'
require 'rspec'
require 'data_magic'
require 'fig_newton'
require 'yaml'
require 'phantomjs'
require 'headless'
ENVT = FigNewton.load('staging.yml')
DATA = DataMagic.load('data.yml')
KBA = YAML.load_file('config/data/kba.yml')
headless = Headless.new
headless.start
browser = Watir::Browser.new
Before do
#browser = browser
end
at_exit do
#browser.close
end
World(PageObject::PageFactory)
The last place I worked we did not use the headless gem, we just started up XVFB in the Jenkins script that ran the tests, before it issued the cucumber command that started the tests. And used Firefox as the browser. Thaat also allowed us to take screenshots on failing tests to aid with debugging the failures.
If I still worked there I would try using phantomjs also and see if it worked, then use whichever was more reliable, and/or faster.
Nothing was different in env.rb, same file we used for local runs we used on jenkins also.
EDIT: Update for late 2017.. with recent changes the current recommended best solution for headless browsing is using chrome along with chromedriver and chrome's new headless option.
The third error is because Headless is not a browser, just a virtual frame buffer. #browser = headless will not work, because it should be a Watir::Browser object. The virtual frame buffer is where your browser, like Firefox or Chrome draws objects.
Based on your list of requires, it looks like you're trying to run with PhantomJS. Watir::Browser.start requires a URL to navigate to as an argument, and will try to start Firefox by default, if you do not pass in :phantomjs, in addition to a url as an argument.
#base_url = "https://my-test-domain.com"
#browser = Watir::Browser.start #base_url, :phantomjs
Watir::Browser.new runs the browser as :firefox by default. If you want to use PhantomJS, pass :phantomjs as an argument.
browser = Watir::Browser.new :phantomjs
If you intend to use PhantomJS as your browser, you likely don't need the Headless gem, and can remove that from your env.rb.
Related
I am continually getting flakey errors on CI where my Selenium session is not restarting. I'm not having this issue when I run tests locally. It only ever happens on my CI server.
I'm using Capybara 2.18.0, rspec 3.7.0, slenium-webdriver 3.9.0 and site_prism 2.13. Is there
Capybara::ExpectationNotMet: Timed out waiting for Selenium session reset
Failure/Error: raise Capybara::ExpectationNotMet.new('Timed out waiting for Selenium session reset') if (Capybara::Helpers.monotonic_time - start_time) >= 10
Capybara::ExpectationNotMet:
Timed out waiting for Selenium session reset
C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/capybara-2.18.0/lib/capybara/selenium/driver.rb:145:in `reset!'
C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/capybara-2.18.0/lib/capybara/session.rb:127:in `reset!'
C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/capybara-2.18.0/lib/capybara.rb:314:in `block in reset_sessions!'
my spec_helper.rb looks like this:
require 'rspec'
require 'capybara/rspec'
require 'capybara-screenshot/rspec'
require 'capybara/dsl'
require 'selenium-webdriver'
require 'site_prism'
Capybara.register_driver :selenium do |app|
Capybara::Selenium::Driver.new(app, browser: :chrome)
end
Capybara.save_path = "#{Dir.pwd}/screenshots"
Capybara.default_driver = :selenium
Capybara.app_host = ENV['url']
if ENV['timeout']
Capybara.default_max_wait_time = ENV['timeout'].to_i
else
Capybara.default_max_wait_time = 15
end
RSpec.configure do |config|
config.full_backtrace = true
config.before(:each) do
config.include Capybara::DSL
end
config.after(:each) do |example|
if example.exception
name = example.example_group.to_s
name.slice!('RSpec::ExampleGroups::')
name.gsub!('::', '#')
whole_page.save_screenshot("#{name}##{example.description.tr(' ', '_')}-#{Time.now.strftime('%H_%M_%S')}.png")
end
end
end
When Capybara resets the driver, it tells the browser to go to about:blank and then waits up to 10 seconds for there to be no elements on the page matching CSS '/html/body/*'. In your case that isn't happening within 10 seconds.
One reason for this could be you having onunload handlers that are doing something that could take longer than 10 seconds on your CI hardware (or opening alert messages, etc)?? If that's the case a workaround is to have the test visit a page that doesn't have an onunload handler and check for something visible on that page at the end of the test (potentially in an after block to keep the test clean). Another thing to verify is that the versions of Chrome and chromedriver are the same between local and CI.
I have written a small piece of code just for launch a browser but I'm getting this error:
C:/Users/KASTURIPARIDA/RubymineProjects/Project1/features/support/driversettings.rb:6:in
<top (required)>'
C:/Ruby193/lib/ruby/gems/1.9.1/gems/cucumber-1.3.17/bin/cucumber:13:in
' C:/Ruby193/bin/cucumber:23:in load'
C:/Ruby193/bin/cucumber:23:in'
-e:1:in `load'
Here below are all the code:
Support:
require 'rubygems'
require 'watir-webdriver'
Selenium::WebDriver::Chrome::Service.executable_path = 'C:\Ruby193\bin\chromedriver.exe'
$driver=watir::Browser.new :Chrome
wait 5
Feature:
Feature: Application
Scenario: Login to Open
And I have account with Opencart and launch page
Step definition:
And(/^I have account with Opencart and launch page$/) do
puts "browser loading"
$driver.goto("https://www.facebook.com/")
end
Try this:
Support
require 'rubygems'
require 'watir'
Selenium::WebDriver::Chrome.driver_path = 'C:\Ruby193\bin\chromedriver.exe'
$driver = Watir::Browser.new('chrome')
Hope it helps.
I'm working on website test automation using Cucumber/Ruby/Selenium-Webdriver/Capybara. I want to switch to Watir-Webdriver in combination with Cucumber and Ruby, but I'm struggling with the following:
Every time I run my cucumber test, Watir opens two browser windows, a blank screen to the site I configurated as default, plus another in which the actual test steps are executed.
My 'Support/env.rb' file has:
require 'allure-cucumber'
require 'rspec'
require 'watir-webdriver'
AllureCucumber.configure do |c|
c.output_dir = 'D:\Test\result'
c.clean_dir = true
c.tms_prefix = '#PRACTEST--'
c.issue_prefix = '#JIRA++'
c.severity_prefix = '#URGENCY:'
c.tms_prefix = ''
end
My steps file begins with:
require 'watir-webdriver'
require 'cucumber'
require 'rspec'
require_relative 'D:\EntelTest\src\PageObject\home_page.rb'
Before do
#test = AbstractPage.new(Watir::Browser.new :ff)
#test.full_size
end
After do
#test.quit
end
home_page = nil
When(/^Go to home page$/) do
home_page = #test.goToHomePage
end
Can you put these before do and after do in hooks.rb? In the steps.rb file, just mention the code for your cucumber steps, and before that declare browser = Watir::Browser.new :ff
The best practice is to put it in hooks.rb. env.rb usually should consist the desired capabilities plus server environment codes. :)
What you have put in step file should go in hooks.rb file.
Please install gem called testnow.
It will help you to create most standard and easy to use watir-webdriver framework with all pre-configured browsers.
Steps:
1) gem install testnow
2) testnow watir_cucumber_now
It will as you to install dependecies, enter Y to set it up completely.
It will create robust framework with a sample scenario.
Just run the sample scenario with any of the following commands.
rake testnow BROWSER=firefox
rake testnow BROWSER=chrome
rake testnow BROWSER=opera
This will only work provided you have browsers pre-installed and webdriver present in PATH variable.
Please comment for more information.
Hope it helps!!
Trying to run Cucumber/Watir-Webdriver on Jenkins using Xvfb (headless gem)
This is my env.rb:
require 'rubygems'
require 'cucumber'
require 'json'
require 'watir-webdriver'
require 'page-object'
require 'page-object/page_factory'
require 'allure-cucumber'
require 'rspec'
require 'data_magic'
require 'fig_newton'
require 'yaml'
require 'headless'
ENVT = FigNewton.load('staging.yml')
DATA = DataMagic.load('data.yml')
KBA = YAML.load_file('config/data/kba.yml')
headless = Headless.new
headless.start
browser = Watir::Browser.new :firefox
Before do
#browser = browser
end
at_exit do
#browser.close
headless.destroy
end
World(PageObject::PageFactory)
This is the error I'm getting:
end of file reached (EOFError)
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/2.1.0/net/protocol.rb:153:in `read_nonblock'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/2.1.0/net/protocol.rb:153:in `rbuf_fill'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/2.1.0/net/protocol.rb:134:in `readuntil'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/2.1.0/net/protocol.rb:144:in `readline'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/2.1.0/net/http/response.rb:39:in `read_status_line'
/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/remote/bridge.rb:640:in `raw_execute'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/remote/bridge.rb:618:in `execute'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/remote/bridge.rb:112:in `get'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/selenium-webdriver-2.45.0.dev3/lib/selenium/webdriver/common/navigation.rb:14:in `to'
/var/lib/jenkins/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/watir-webdriver-0.6.11/lib/watir-webdriver/browser.rb:77:in `goto'
/var/lib/jenkins/jobs/idme_revelator/workspace/features/support/hooks.rb:32:in `Before'
* I resize the browser # features/step_definitions/misc_step_defs.rb:1
Connection refused - connect(2) for "127.0.0.1" port 7055 (Errno::ECONNREFUSED)
./features/step_definitions/misc_step_defs.rb:2:in `/^I resize the browser$/'
How do I need to setup my env file to correctly run Cucumber/Watir-Webdriver using the headless gem (Xvfb)?
In some other attempts, errors have been related to undefined method 'goto', etc. In my tests other places like hooks.rb I use the variable #browser or #browser.driver, so I need those to work as well.
I would take a look at the answer to this question as it your error is the same save for the different port number
Opening several threads with watir-webdriver results in 'Connection refused' error
It looks like you maybe have multiple browsers trying to access the same port at the same time. Adding a sleep might fix it, but is not a best-practice solution. I would more clearly define your port numbers that each browser connects to.
Setup with cucumber, capybara and selenium but some scenarios works only randomly.
Running
ruby 1.8.6 on rvm
rails 2.3.8
selenium pops open firefox 3.6
I have tried to add this with no luck:
with_scope(selector) do
click_button(button)
selenium.wait_for_page_to_load
end
The error output is sometimes:
> Given I am logged in and have created newsletter and subscribers # features/step_definitions/newsletter_send_steps.rb:108
end of file reached (EOFError)
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:133:in `sysread'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:133:in `rbuf_fill'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/timeout.rb:62:in `timeout'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/timeout.rb:93:in `timeout'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:132:in `rbuf_fill'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:126:in `readline'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:2020:in `read_status_line'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:2009:in `read_new'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:1050:in `request_without_fakeweb'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:1037:in `request_without_fakeweb'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:543:in `start'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:1035:in `request_without_fakeweb'
./features/step_definitions/web_steps.rb:24:in `__instance_exec2'
./features/step_definitions/web_steps.rb:9:in `with_scope'
./features/step_definitions/web_steps.rb:9:in `with_scope'
./features/step_definitions/web_steps.rb:23:in `/^(?:|I )press "([^\"]*)"(?: within "([^\"]*)")?$/'
features/enhanced/newsletter_send1.feature:7:in `Given I am logged in and have created newsletter and subscribers'
And othertimes:
> no button with value or id or text 'create_user_button' found (Capybara::ElementNotFound)
./features/step_definitions/web_steps.rb:24:in `__instance_exec2'
./features/step_definitions/web_steps.rb:9:in `with_scope'
./features/step_definitions/web_steps.rb:9:in `with_scope'
./features/step_definitions/web_steps.rb:23:in `/^(?:|I )press "([^\"]*)"(?: within "([^\"]*)")?$/'
features/enhanced/newsletter_send1.feature:7:in `Given I am logged in and have created newsletter and subscribers'
And sometimes it just works....
This is how my env.rb looks like
ENV["RAILS_ENV"] ||= "cucumber"
require File.expand_path(File.dirname(__FILE__) + '/../../config/environment')
require 'cucumber/formatter/unicode' # Remove this line if you don't want Cucumber Unicode support
require 'cucumber/rails/world'
require 'cucumber/rails/active_record'
require 'cucumber/web/tableish'
require 'capybara/rails'
require 'capybara/cucumber'
require 'capybara/session'
require 'cucumber/rails/capybara_javascript_emulation'
require "selenium-webdriver"
Capybara.default_driver = :selenium
Capybara.default_wait_time = 5
Capybara.ignore_hidden_elements = false
Capybara.default_selector = :css
ActionController::Base.allow_rescue = false
require 'database_cleaner'
DatabaseCleaner.strategy = :truncation
Before do
Capybara.reset_sessions!
DatabaseCleaner.clean
end
Cucumber::Rails::World.use_transactional_fixtures = false
Cucumber-steps:
Given I am on the signup page
And I fill in "user_login" with "jeppsipeppsi#arcticelvis.com" within "body"
And I fill in "user_password" with "secret" within "body"
And I fill in "user_password_confirmation" with "secret" within "body"
And I check "terms_of_use" within "body"
And I press "create_user_button" within "body"
Any insight would be great :)
It's HTTP mocking, if you remove fakeweb or webmock from your environment (entirely), it should all work again.
The last comment by Adam Greene DOES WORK regarding setting up Curb with:
Selenium::WebDriver.for :firefox, :http_client => Selenium::WebDriver::Remote::Http::Curb
Read the thread on the Capybara group.
The problem we're having is playing back recorded http traffic using fakeweb or webmock since web driver is now Curb. So if you're goal was to fake out traffic over Capybara, you'll get browser testing to work again but you won't be able to play the traffic back over the same browser. (We're using VCR to record.)
Adding Curb support is listed as a "ticket" on the Fakeweb's Github Issues site.
I bumped into this in a Rails 2.3 app with cucumber/capybara/akephalos/fakeweb recently, but ultimately got to resolve this by completely killing all gems in my bundle (they where kept in .bundle/ and reinstalling.