Watir/Selenium - browser.goto keep getting TimeOut error on Chrome and Firefox - ruby

Got a very annoying problem with Watir webdriver..
I have debugged a little, and found out I always get TimeOut::Error on a simple #browser.goto line, even I can visually see the page has loaded fully...
The scenario is like this:
Open a browser, goto a url, click a few links, and then suddenly at one point, the script stops continue browsing, waiting about 30+ seconds and throw errors.
Tried both Chrome and FF: Chrome is much worse, normally a 2nd or 3nd link clicking will trigger; for FF, sometimes takes 10+ page browsing...
Bet there is some environment or comparability issue:
jd#deskbox:~$ uname -am
Linux deskbox 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
jd#deskbox:~$ ruby -v
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-linux]
jd#deskbox:~$ rails -v
Rails 3.2.1
jd#deskbox:~$ gem -v
1.8.15
jd#deskbox:~$ gem list|grep webdriver
selenium-webdriver (2.12.0)
watir-webdriver (0.5.3)
Can someone help on this? Source code here:
#!/usr/bin/env ruby
require 'watir-webdriver'
require 'pry'
class Search
attr_accessor :browser, :company_url, :company_name
def initialize()
#browser = Watir::Browser.start 'http://www.google.com', :chrome
end
def visit_company_home_via_google(company)
#company_name = company
#browser.goto "http://www.google.com/search?q=#{company}"
link = #browser.div(:id=>'ires').link
return nil unless link.exists?
#browser.goto link.href
#company_url ||= #browser.url
#company_url
end
def logoff()
#browser.close if #browser
end
end
s = Search.new
puts s.visit_company_home_via_google("github")
puts s.visit_company_home_via_google("Mashable")
puts s.visit_company_home_via_google("Barracuda Networks")
s.logoff
My result is like:
jd#deskbox:~/cuda$ ./search.rb
https://github.com/
/usr/local/lib/ruby/1.9.1/net/protocol.rb:140:in `rescue in rbuf_fill': Timeout::Error (Timeout::Error)
from /usr/local/lib/ruby/1.9.1/net/protocol.rb:134:in `rbuf_fill'
from /usr/local/lib/ruby/1.9.1/net/protocol.rb:116:in `readuntil'
from /usr/local/lib/ruby/1.9.1/net/protocol.rb:126:in `readline'
from /usr/local/lib/ruby/1.9.1/net/http.rb:2219:in `read_status_line'
from /usr/local/lib/ruby/1.9.1/net/http.rb:2208:in `read_new'
from /usr/local/lib/ruby/1.9.1/net/http.rb:1191:in `transport_request'
from /usr/local/lib/ruby/1.9.1/net/http.rb:1177:in `request'
from /usr/local/lib/ruby/1.9.1/net/http.rb:1170:in `block in request'
from /usr/local/lib/ruby/1.9.1/net/http.rb:627:in `start'
from /usr/local/lib/ruby/1.9.1/net/http.rb:1168:in `request'
from /usr/local/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.12.0/lib/selenium/webdriver/remote/http/default.rb:81:in `response_for'
from /usr/local/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.12.0/lib/selenium/webdriver/remote/http/default.rb:43:in `request'
from /usr/local/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.12.0/lib/selenium/webdriver/remote/http/common.rb:39:in `call'
from /usr/local/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.12.0/lib/selenium/webdriver/remote/bridge.rb:450:in `raw_execute'
from /usr/local/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.12.0/lib/selenium/webdriver/remote/bridge.rb:428:in `execute'
from /usr/local/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.12.0/lib/selenium/webdriver/remote/bridge.rb:99:in `get'
from /usr/local/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.12.0/lib/selenium/webdriver/common/navigation.rb:14:in `to'
from /usr/local/lib/ruby/gems/1.9.1/gems/watir-webdriver-0.5.3/lib/watir-webdriver/browser.rb:61:in `goto'
from ./search.rb:17:in `visit_company_home_via_google'
from ./search.rb:33:in `<main>'

I think there's a bug in Chromedriver that doesn't return the URL correctly. I got your example to work using:
require 'watir-webdriver'
class Search
attr_accessor :browser, :company_name
def initialize
#browser = Watir::Browser.start 'http://www.google.com', :chrome
end
def get_company_url(company, url=nil)
#company_name = company
#company_url = url
#browser.goto "http://www.google.com/search?q=#{company}"
link = #browser.div(:id=>'ires').link
return nil unless link.exists?
#company_url ||= #browser.driver.current_url
end
def logoff()
#browser.close if #browser
end
end
s = Search.new
puts s.get_company_url 'Barracuda Networks'

Related

Ruby not breaking out of while loop

I'm trying to get this to loop while a specific element exists on the page. The code runs and grabs the urls I want, however, when the next button is no longer on the page it wont break out of the loop and throws the following error.
/Users/someone/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/rspec-expectations-3.2.0/lib/rspec/matchers.rb:926:in `method_missing': undefined method `each' for nil:NilClass (NoMethodError)
from /something/something/something.rb:30:in `block in <top (required)>'
from /something/something/something.rb:28:in `open'
from /something/something/something.rb:29:in `<top (required)>'
from -e:1:in `load'
from -e:1:in `<main>'</code>
Brand new to Ruby, so please be gentle ;)
require 'capybara/poltergeist'
require 'capybara/dsl'
require 'csv'
require 'rspec'
include RSpec::Matchers
include Capybara::DSL
Capybara.register_driver :poltergeist do |app|
Capybara::Poltergeist::Driver.new(app,timeout: 60, :phantomjs_options => ['--debug=no', '--load-images=yes', '--ignore-ssl-errors=yes', '--ssl-protocol=TLSv1'], :debug => false)
end
Capybara.default_driver = :poltergeist
Capybara.javascript_driver = :poltergeist
Capybara.default_wait_time = 20
Capybara.ignore_hidden_elements = true
Capybara.current_session.driver.resize(1200, 1000)
visit('site.com')
while page.find(:xpath, 'html/body/div[4]/div[6]/div[1]/div[2]/div[2]/div[1]/div[2]/button[1]') do
page.find(:xpath, 'html/body/div[4]/div[6]/div[1]/div[2]/div[2]/div[1]/div[2]/button[1]').click
urls = page.all('.author-name>a').map { |a| a['href'] }.uniq
puts urls
end
puts urls
f = File.open("profiles.txt", "a") do |f|
urls.each { |element| f.puts(element) }
end

How to skip SSL certificate verification with Capybara

I'm trying to test a webapp that is served over HTTPS, but I'm not able to skip certificate verification:
#!/usr/bin/env ruby
require "rubygems"
require "bundler/setup"
require "capybara"
require "capybara/dsl"
require "capybara-webkit"
Capybara.run_server = false
Capybara.register_driver :webkit do |app|
Capybara::Driver::Webkit.new(app, :ignore_ssl_errors => true)
end
Capybara.current_driver = :webkit
Capybara.app_host = "https://foo.bar.com"
module Test
class Net
include Capybara::DSL
def get_results
visit('/index.jsp')
fill_in "#UserId", :with => "sheldon"
fill_in "#Pwd", :with => "cooper"
click_button "Enter"
page.save_screenshot('screenshot.png')
end
end
end
spider = Test::Net.new
spider.get_results
I get:
net.rb:10:in `initialize': wrong number of arguments (2 for 0) (ArgumentError)
from net.rb:10:in `new'
from net.rb:10:in `block in <main>'
from /usr/local/lib/ruby/gems/2.0.0/gems/capybara-2.2.0/lib/capybara/session.rb:69:in `call'
from /usr/local/lib/ruby/gems/2.0.0/gems/capybara-2.2.0/lib/capybara/session.rb:69:in `driver'
from /usr/local/lib/ruby/gems/2.0.0/gems/capybara-2.2.0/lib/capybara/session.rb:197:in `visit'
from /usr/local/lib/ruby/gems/2.0.0/gems/capybara-2.2.0/lib/capybara/dsl.rb:51:in `block (2 levels) in <module:DSL>'
from net.rb:20:in `get_results'
from net.rb:31:in `<main>'
How can I skip it?
I believe the API has changed and that you need to do:
Capybara.register_driver :webkit do |app|
Capybara::Webkit::Driver.new(app).tap {|d| d.browser.ignore_ssl_errors }
end

`block in non_options': file not found: (ArgumentError)

I'm trying to open browser url based on argument passed to script. Hence I wrote following ruby code:
require 'selenium-webdriver'
require 'test/unit'
class TestTitle < Test::Unit::TestCase
def setup
$driver = Selenium::WebDriver.for :firefox
if ARGV[0] == 'google'
$driver.get 'http://www.google.com'
elsif ARGV[0] == 'twitter'
$driver.get 'http://www.twitter.com'
end
end
def test_title
puts $driver.title
end
def teardown
$driver.quit
end
end
When I passed argument: ruby test.rb 'google', it results into following error:
c:/Ruby193/lib/ruby/1.9.1/test/unit.rb:167:in `block in non_options': file not found: google (ArgumentError)
from c:/Ruby193/lib/ruby/1.9.1/test/unit.rb:146:in `map!'
from c:/Ruby193/lib/ruby/1.9.1/test/unit.rb:146:in `non_options'
from c:/Ruby193/lib/ruby/1.9.1/test/unit.rb:207:in `non_options'
from c:/Ruby193/lib/ruby/1.9.1/test/unit.rb:52:in `process_args'
from c:/Ruby193/lib/ruby/1.9.1/minitest/unit.rb:891:in `_run'
from c:/Ruby193/lib/ruby/1.9.1/minitest/unit.rb:884:in `run'
from c:/Ruby193/lib/ruby/1.9.1/test/unit.rb:21:in `run'
from c:/Ruby193/lib/ruby/1.9.1/test/unit.rb:326:in `block (2 levels) in autorun'
from c:/Ruby193/lib/ruby/1.9.1/test/unit.rb:27:in `run_once'
from c:/Ruby193/lib/ruby/1.9.1/test/unit.rb:325:in `block in autorun'
Please help me understand what I'm doing wrong.
It appears that test-unit (as of 1.9.1) grabs command line options in its GlobOptions module. You are using ARGV[0] to pass browser name, but it thinks you're passing a file name. A workaround is to capture the value of ARGV[0] and then clear it before your test case runs:
browser = ARGV[0]
ARGV[0] = nil

How do I use selenium with Ruby?

I made some tests with the Firefox Selenium and then had it exported to Ruby. Although the tests all ran fine in Firefox, I am having trouble running the same suite in Ruby.
I tried to run one of the example programs they have and I also get the same connection refused error. Here is the error I got when trying to run their google_test suite.
tellingsen$ ruby google_test.rb
Loaded suite google_test
Started
E
Finished in 0.001558 seconds.
1) Error:
test_page_search(ExampleTest):
Errno::ECONNREFUSED: Connection refused - connect(2)
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:560:in `initialize'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:560:in `open'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:560:in `connect'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:62:in `timeout'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:93:in `timeout'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:560:in `connect'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:553:in `do_start'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:542:in `start'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:1035:in `request'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:845:in `post'
/Users/tellingsen/.gem/ruby/1.8/gems/selenium-client-1.2.18/lib/selenium/client/protocol.rb:89:in `http_post'
/Users/tellingsen/.gem/ruby/1.8/gems/selenium-client-1.2.18/lib/selenium/client/protocol.rb:12:in `remote_control_command'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:62:in `timeout'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:93:in `timeout'
/Users/tellingsen/.gem/ruby/1.8/gems/selenium-client-1.2.18/lib/selenium/client/protocol.rb:11:in `remote_control_command'
/Users/tellingsen/.gem/ruby/1.8/gems/selenium-client-1.2.18/lib/selenium/client/protocol.rb:19:in `string_command'
/Users/tellingsen/.gem/ruby/1.8/gems/selenium-client-1.2.18/lib/selenium/client/base.rb:85:in `start_new_browser_session'
google_test.rb:21:in `setup'
1 tests, 0 assertions, 0 failures, 1 errors
Can someone help me with this?
Note:
Mac OS: 10.6.4
Macbook Pro
Ruby: 1.8.7
gem: selenium-client 1.2.18
EDIT
Here is the google_test.rb that I tried
#!/usr/bin/env ruby
#
# Sample Test:Unit based test case using the selenium-client API
#
require "test/unit"
require "rubygems"
gem "selenium-client", ">=1.2.18"
require "selenium/client"
class ExampleTest < Test::Unit::TestCase
attr_reader :browser
def setup
#browser = Selenium::Client::Driver.new \
:host => "localhost",
:port => 4444,
:browser => "*firefox",
:url => "http://www.google.com",
:timeout_in_second => 60
browser.start_new_browser_session
end
def teardown
browser.close_current_browser_session
end
def test_page_search
browser.open "/"
assert_equal "Google", browser.title
browser.type "q", "Selenium seleniumhq"
browser.click "btnG", :wait_for => :page
assert_equal "Selenium seleniumhq - Google Search", browser.title
assert_equal "Selenium seleniumhq", browser.field("q")
assert browser.text?("seleniumhq.org")
assert browser.element?("link=Cached")
end
end
I figured it out after a few hours of searching on forums and through google.
What I needed to do was have the selenium server running for it to work. I was able to download it from this site http://seleniumhq.org/download/ (current: Selenium RC February 23, 2010 1.0.3).
From there I opened up a new terminal and did
cd Downloads/selenium-remote-control-1.0.3/selenium-server-1.0.3
java -jar selenium-server.jar
Then ran my ruby generated script with another terminal window
ruby google_test.rb
And it worked!
This is Selenium Webdriver example for simple google search
Save as google_search.rb
require "selenium-webdriver"
require "test/unit"
class GoogleSearch < Test::Unit::TestCase
def setup
#driver = Selenium::WebDriver.for :firefox
#base_url = "http://www.google.com/"
#driver.manage.timeouts.implicit_wait = 30
#verification_errors = []
end
def teardown
#driver.quit
assert_equal [], #verification_errors
end
def test_google_search
#driver.get(#base_url)
#driver.find_element(:name, "q").clear
#driver.find_element(:name, "q").send_keys "Thiyagarajan Veluchamy"
#driver.find_element(:name, "btnK").click
end
def element_present?(how, what)
#driver.find_element(how, what)
true
rescue Selenium::WebDriver::Error::NoSuchElementError
false
end
def verify(&blk)
yield
rescue Test::Unit::AssertionFailedError => ex
#verification_errors << ex
end
end
$ruby google_search.rb
Here is a much simpler version of the script:
require "selenium-webdriver"
#driver = Selenium::WebDriver.for :chrome
#base_url = "http://www.google.com/"
#driver.get(#base_url)
#driver.find_element(:name, "q").send_keys "Stack Overflow"
Methods available on the #driver object can be found here: http://selenium.googlecode.com/svn/trunk/docs/api/rb/Selenium/WebDriver/Driver.html
find_element gives you access to the Element class. Methods available on the Element class can be found here:
http://selenium.googlecode.com/svn/trunk/docs/api/rb/Selenium/WebDriver/Element.html

Ruby Net::HTTP time out

I'm trying to write my first Ruby program, but have a problem. The code has to download 32 MP3 files over HTTP. It actually downloads a few, then times-out.
I tried setting a timeout period, but it makes no difference. Running the code under Windows, Cygwin and Mac OS X has the same result.
This is the code:
require 'rubygems'
require 'open-uri'
require 'nokogiri'
require 'set'
require 'net/http'
require 'uri'
puts "\n Up and running!\n\n"
links_set = {}
pages = ['http://www.vimeo.com/siai/videos/sort:oldest',
'http://www.vimeo.com/siai/videos/page:2/sort:oldest',
'http://www.vimeo.com/siai/videos/page:3/sort:oldest']
pages.each do |page|
doc = Nokogiri::HTML(open(page))
doc.search('//*[#href]').each do |m|
video_id = m[:href]
if video_id.match(/^\/(\d+)$/i)
links_set[video_id[/\d+/]] = m.children[0].to_s.split(" at ")[0].split(" -- ")[0]
end
end
end
links = links_set.to_a
p links
cookie = ''
file_name = ''
open("http://www.tubeminator.com") {|f|
cookie = f.meta['set-cookie'].split(';')[0]
}
links.each do |link|
open("http://www.tubeminator.com/ajax.php?function=downloadvideo&url=http%3A%2F%2Fwww.vimeo.com%2F" + link[0],
"Cookie" => cookie) {|f|
puts f.read
}
open("http://www.tubeminator.com/ajax.php?function=convertvideo&start=0&duration=1120&size=0&format=mp3&vq=high&aq=high",
"Cookie" => cookie) {|f|
file_name = f.read
}
puts file_name
Net::HTTP.start("www.tubeminator.com") { |http|
#http.read_timeout = 3600 # 1 hour
resp = http.get("/download-video-" + file_name)
open(link[1] + ".mp3", "wb") { |file|
file.write(resp.body)
}
}
end
puts "\n Yay!!"
And this is the exception:
/Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/protocol.rb:140:in `rescue in rbuf_fill': Timeout::Error (Timeout::Error)
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/protocol.rb:134:in `rbuf_fill'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/protocol.rb:116:in `readuntil'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/protocol.rb:126:in `readline'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:2138:in `read_status_line'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:2127:in `read_new'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:1120:in `transport_request'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:1106:in `request'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:312:in `block in open_http'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:564:in `start'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:306:in `open_http'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:767:in `buffer_open'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:203:in `block in open_loop'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:201:in `catch'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:201:in `open_loop'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:146:in `open_uri'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:669:in `open'
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:33:in `open'
from test.rb:38:in `block in <main>'
from test.rb:37:in `each'
from test.rb:37:in `<main>'
I'd also appreciate your comments on the rest of the code.
For Ruby 1.8 I used this to solve my time-out issues. Extending the Net::HTTP class in my code and re-initialized with default parameters including an initialization of my own read_timeout should keep things sane I think.
require 'net/http'
# Lengthen timeout in Net::HTTP
module Net
class HTTP
alias old_initialize initialize
def initialize(*args)
old_initialize(*args)
#read_timeout = 5*60 # 5 minutes
end
end
end
Your timeout isn't in the code you set the timeout for. It's here, where you use open-uri:
open("http://www.tubeminator.com/ajax.php?function=downloadvideo&url=http%3A%2F%2Fwww.vimeo.com%2F" + link[0],
You can set a read timeout for open-uri like so:
#!/usr/bin/ruby1.9
require 'open-uri'
open('http://stackoverflow.com', 'r', :read_timeout=>0.01) do |http|
http.read
end
# => /usr/lib/ruby/1.9.0/net/protocol.rb:135:in `sysread': \
# => execution expired (Timeout::Error)
# => ...
# => from /tmp/foo.rb:5:in `<main>'
:read_timeout is new for Ruby 1.9 (it's not in Ruby 1.8). 0 or nil means "no timeout."

Resources