Capybara/Selenium - force to reload Selenium Driver on each call - ruby

I have a solution to parse some data from a remote websites using Firefox(46) + Capybara + Selenium. I pass path_to_firefox_profile argument on initialize, which is some real Firefox profile folder.
The problem is that if I execute this code:
a = MyParser.new(profile_a)
a.parse_something
b = MyParser.new(profile_b)
b.parse_something
... the instance b will have profile A loaded despite I specified another profile.
But, if I run these 2 lines in a separate processes, I'll get what I want. So I assume one of them - either Capybara or Selenium - stores profiles settings once per ruby process, and won't change it on demand.
Are there are ideas how to change profile within the same process?
I tries to .quit Firefox, but it doesn't help, on visiting new URL Selenium opens another Firefox window with the exact same profile instead of new one.
class MyParser
def initialize(path_to_firefox_profile)
Capybara.register_driver(:selenium) do |app|
client = Selenium::WebDriver::Remote::Http::Default.new
client.timeout = 150
Capybara::Selenium::Driver.new(app,
profile: path_to_firefox_profile,
http_client: client)
end
end
def parse_something
# perform some parsings & return result
end
end

Capybara's register_driver registers the driver by the name you assign globally, and will use those drivers (by name) in its sessions. It will also automatically manage sessions in a manner designed for ease of use by people testing web-apps. The issue here is that a new session with the newly registered driver isn't being created. This is because you're not really using Capybara for what it was designed, and therefore need to pick up a bit more of the weight of session management. If you're ever going to have more than 2 of your MyParser objects around you probably should be creating a new session per MyParser instance you create and then using that session in all of the MyParser methods. Since you're using different driver settings per MyParser instance you should probably also be naming each driver registered differently.
class MyParser
def initialize(path_to_firefox_profile)
Capybara.register_driver(self.object_id.to_s) do |app|
client = Selenium::WebDriver::Remote::Http::Default.new
client.timeout = 150
Capybara::Selenium::Driver.new(app,
profile: path_to_firefox_profile,
http_client: client)
end
#session = Capybara::Session.new(self.object_id.to_s)
end
def parse_something
#session.visit("some url")
#session.find(...) # perform some parsings & return result
end
end
You will also need to handle cleaning up the session when you're done with each MyParser instance.

Related

DRuby with Selenium WebDriver

I'm using distributed ruby so that I can save the selenium web-driver object in one script and use the same object in the next script when I run the clients I get an error indicating that #<Drb::DRbConnError: connection closed>.
Has anyone tried this or how do we overcome this issue?
Below are my scripts
Server.rb
require 'drb/drb'
# The URI for the server to connect to
URI="druby://localhost:8787"
# Allows sharing of variables between component runs
class TestScope
# class variable
##variables = {}
def save_variable(key, value)
##variables.store(key, value)
end
def get_variable(key)
return ##variables[key]
end
def get_size
return ##variables.size
end
end
# The object that handles requests on the server
FRONT_OBJECT=TestScope.new
DRb.start_service(URI, FRONT_OBJECT, safe_level: 1, verbose: true)
# Wait for the drb server thread to finish before exiting.
DRb.thread.join
Client1.rb
require 'drb/drb'
require 'selenium-webdriver'
# The URI to connect to
SERVER_URI="druby://localhost:8787"
# Start a local DRbServer to handle callbacks.
# Not necessary for this small example, but will be required
# as soon as we pass a non-marshallable object as an argument
# to a dRuby call.
# Note: this must be called at least once per process to take any effect.
# This is particularly important if your application forks.
DRb.start_service
test_scope = DRbObject.new_with_uri(SERVER_URI)
driver = Selenium::WebDriver::Driver.for :firefox
driver.navigate.to "http://www.google.com"
puts "Saving the driver object to the test scope"
test_scope.save_variable('driver', driver)
Client2.rb
require 'drb/drb'
require 'selenium-webdriver'
# The URI to connect to
SERVER_URI="druby://localhost:8787"
# Start a local DRbServer to handle callbacks.
# Not necessary for this small example, but will be required
# as soon as we pass a non-marshallable object as an argument
# to a dRuby call.
# Note: this must be called at least once per process to take any effect.
# This is particularly important if your application forks.
DRb.start_service
test_scope = DRbObject.new_with_uri(SERVER_URI)
puts "Fetching the driver object from the test scope"
driver1 = test_scope.get_variable('driver')
driver1.navigate.to "http://www.yahoo.com"
In order to share an object using DRb, the class must be defined in the dRb server as it allows an object in one Ruby process to invoke methods on an object in another Ruby process on the same or a different machine.
If there is a scenario where one needs to create an object on the dRb client and use that object in other DRb clients. We need to use ruby script runner and define object in the scriptrunner.rb so that multiple clients can use it.

How to store secret strings on a GEM?

I wrote a ruby gem to interact with a webservice. The gem contains the hardcoded value of client id and application secret.
So anyone else can use my data on his script and I will be charged for his requests. :(
On other hands it would be bad for the users if they have to create a new app in order to get a new personal application secret.
Any idea on how to store these info in a secure way?
While it might be OK to hardcode values in internal-only, early stage prototypes or temporary solution it is usually not advisable to hardcode values that normally change from application to application and most certainly not password or security tokens!
I would recommend that you made these values configurable in ways that people can easily set them via standard environment variables (very common in PaaS environments such as Heroku) or via initializers if need be. Exceptions are sane default values such as development database URLs (ex: redis' default 127.0.0.1:6379)
Example
module MyGem
class Configuration
attr_writer :client_id, :application_secret, :url
# Expects a MY_GEM_CLIENT_ID env var to be set in production
def client_id
#client_id ||= ENV.fetch('MY_GEM_CLIENT_ID', 'CLIENT_ID')
end
# Expects a MY_GEM_APPLICATION_SECRET env var to be set in production
def application_secret
#application_secret ||= ENV.fetch('MY_GEM_APPLICATION_SECRET', 'SECRET')
end
# ENV driven config but with working default
def url
#url ||= ENV.fetch('MY_GEM_URL', 'https://myservice.com/v1/api')
end
end
def self.configure(&block)
block.call(configuration)
end
def self.configuration
#configuration ||= Configuration.new
end
end
Initializer-based config of the code above
# config/initializers/my_gem.rb in Rails
# CustomConfigObject could be a Mixlib configuration object
MyGem.configure do |config|
config.client_id = CustomConfigObject.my_gem.client_id
config.application_sercret = CustomConfigObject.my_gem.application_secret
config.url = CustomConfigObject.my_gem.url
end
If you absolutely need to provide a default "sandbox" set of Client ID and Application Secret you can provide those values as defaults in the code above (the second argument to ENV.fetch()), but I'd recommend either adding a sandbox boolean setting that can be set to true/false and sent with API calls, at which point your API would process those as sandbox / dev calls or, preferably, add support for account-specific sandbox credentials.

Simulate network failure / offline mode in Capybara?

I have some JavaScript in my app that detects when the network connection goes away and temporarily caches data in local storage, to be synced with the server when the connection is re-established.
I've been trying to find a way to test this end-to-end using Capybara, but I can't seem to find any way of either temporarily disabling the app server or switching the headless browser into offline mode. FWIW I'm using Poltergeist as the driver.
Does anyone have an idea how this could be tested? (I can test the JavaScript app using sinon to fake the server going away, but I'd like to be able to test it end-to-end with a headless browser if possible).
If you stumbled on this question looking for a way to test offline / progressive web apps with Capybara and Chrome Headless, here's how:
params = {
cmd: 'Network.emulateNetworkConditions',
params: {
offline: true,
latency: 0,
downloadThroughput: 0,
uploadThroughput: 0
}
}
page.driver.browser.send(:bridge).send_command(params)
My team has stubbed out the Rack app to simulated errors from the server. It works well enough (in Firefox). Here are some relevant excerpts from the code:
class NoResponseRack
attr_reader :requests
def initialize disconnected_mode
#disconnected_mode = disconnected_mode
#requests = []
#sleeping_threads = []
end
def call(env)
#requests.push(env)
case #disconnected_mode
when :elb_pool_empty
#sleeping_threads << Thread.current
sleep 65
#sleeping_threads.delete Thread.current
[504, {}, ['']]
when :server_maintenance
[200, {}, ['status_message=Atlas is down for maintenance.']]
else
[999, {}, [""]]
end
end
def wakeup_sleeping_threads
#sleeping_threads.each &:wakeup
#sleeping_threads.clear
end
end
def go_disconnected_with_proxy disconnected_mode=:server_error
if $proxy_server_disconnected
puts 'going DISconnected'
$current_proxy = NoResponseRack.new disconnected_mode
rack_mappings.unshift([nil, "", /^(.*)/n, $current_proxy])
$proxy_server_disconnected = false
end
end
def rack_app
Capybara::current_session.driver.app
end
def rack_mappings
rack_app.instance_variable_get(:#mapping)
end
About the only way I can think of would be to allow the host to be overridden in your tests, and give it a bogus host (something like localhost:31337).
Maybe have a look at http://robots.thoughtbot.com/using-capybara-to-test-javascript-that-makes-http and see if anything jumps out.
I found also another option. To start Chrome with
options.add_argument('--host-resolver-rules=MAP * ~NOTFOUND')
I saw this in chromium docs because this flag was missing from google-chrome --help.
Which link also means that you can also use your own proxy server to simulate different conditions as well.

rails persist objects over requests in development mode

I am trying to interact with Matlab.Application.Single win32ole objects in my rails application. The problem I am running into is that while I am developing my application, each separate request reloads my win32ole objects so I loose the connection to my matlab orignal instances and new instances are made. Is there a way to persist live objects between requests in rails? or is there a way to reconnect to my Matlab.Application.Single instances?
In production mode I use module variables to store my connection between requests, but in development mode Module variables are reloaded every request.
here is a snippet of my code
require 'win32ole'
module Calculator
#engine2 = nil
#engine3 = nil
def self.engine2
if #engine2.nil?
#engine2 = WIN32OLE.new("Matlab.Application.Single")
#engine2.execute("run('setup_path.m')")
end
#engine2
end
def self.engine3
if #engine3.nil?
#engine3 = WIN32OLE.new("Matlab.Application.Single")
#engine3.execute("run('setup_path.m')")
end
#engine3
end
def self.load_CT_image(file)
Calculator.engine2.execute("spm_image('Init','#{file}')")
end
def self.load_MR_image(file)
Calculator.engine3.execute("spm_image('Init','#{file}')")
end
end
I am then able to use my code in my controllers like this:
Calculator.load_CT_image('Post_Incident_CT.hdr')
Calculator.load_MR_image('Post_Incident_MRI.hdr')
You can keep an app-wide object in a constant that won't be reset for every request. Add this to a new file in config/initializers/:
ENGINE_2 = WIN32OLE.new("Matlab.Application.Single")
You might also need to include the .execute("run('setup_path.m')") line here as well (I'm not familiar with WIN32OLE). You can then assign that object to your instance variables in your Calculator module (just replace the WIN32OLE.new("Matlab.Application.Single") call with ENGINE_2, or simply refer to them directly.
I know this is beyond the scope of your question, but you have a lot of duplicated code here, and you might want to think about creating a class or module to manage your Matlab instances -- spinning up new ones as needed, and shutting down old ones that are no longer in use.

How can I run Selenium (used through Capybara) at a lower speed?

By default Selenium runs as fast as possible through the scenarios I defined using Cucumber.
I would like to set it to run at a lower speed, so I am able to capture a video of the process.
I figured out that an instance of Selenium::Client::Driver has a set_speed method. Which corresponds with the Java API.
How can I obtain an instance of the Selenium::Client::Driver class? I can get as far as page.driver, but that returns an instance of Capybara::Driver::Selenium.
Thanks to http://groups.google.com/group/ruby-capybara/msg/6079b122979ffad2 for a hint.
Just a note that this uses Ruby's sleep, so it's somewhat imprecise - but should do the job for you. Also, execute is called for everything so that's why it's sub-second waiting. The intermediate steps - wait until ready, check field, focus, enter text - each pause.
Create a "throttle.rb" in your features/support directory (if using Cucumber) and fill it with:
require 'selenium-webdriver'
module ::Selenium::WebDriver::Firefox
class Bridge
attr_accessor :speed
def execute(*args)
result = raw_execute(*args)['value']
case speed
when :slow
sleep 0.3
when :medium
sleep 0.1
end
result
end
end
end
def set_speed(speed)
begin
page.driver.browser.send(:bridge).speed=speed
rescue
end
end
Then, in a step definition, call:
set_speed(:slow)
or:
set_speed(:medium)
To reset, call:
set_speed(:fast)
This will work, and is less brittle (for some small value of "less")
require 'selenium-webdriver'
module ::Selenium::WebDriver::Remote
class Bridge
alias_method :old_execute, :execute
def execute(*args)
sleep(0.1)
old_execute(*args)
end
end
end
As an update, the execute method in that class is no longer available. It is now here only:
module ::Selenium::WebDriver::Remote
I needed to throttle some tests in IE and this worked.
The methods mentioned in this thread no longer work with Selenium Webdriver v3.
You'll instead need to add a sleep to the execution command.
module Selenium::WebDriver::Remote
class Bridge
def execute(command, opts = {}, command_hash = nil)
verb, path = commands(command) || raise(ArgumentError, "unknown command: #{command.inspect}")
path = path.dup
path[':session_id'] = session_id if path.include?(':session_id')
begin
opts.each { |key, value| path[key.inspect] = escaper.escape(value.to_s) }
rescue IndexError
raise ArgumentError, "#{opts.inspect} invalid for #{command.inspect}"
end
Selenium::WebDriver.logger.info("-> #{verb.to_s.upcase} #{path}")
res = http.call(verb, path, command_hash)
sleep(0.1) # <--- Add your sleep here.
res
end
end
end
Note this is a very brittle way to slow down the tests since you're monkey patching a private API.
I wanted to slow down the page load speeds in my Capybara test suite to see if I could trigger some intermittently failing tests. I achieved this by creating an nginx reverse proxy container and sitting it between my test container and the phantomjs container I was using as a headless browser. The speed was limited by using the limit_rate directive. It didn't help me to achieve my goal in the end, but it did work and it may be a useful strategy for others to use!

Resources