I am using chrome headless on my linux server for web scraping.
I am getting Selenium::WebDriver::Error::TimeOutError: timeout Error
Here's my code:
Selenium::WebDriver::Chrome.driver_path = "/usr/local/bin/chromedriver"
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('--ignore-certificate-errors')
options.add_argument('--disable-popup-blocking')
options.add_argument('--disable-translate')
options.add_argument('--headless')
browser = Watir::Browser.new :chrome, options: options
browser.goto 'www.abc.com'
browser.link(text: 'hello').click! # opens in new window
browser.windows.last.use
browser.url # FAILING AT THIS POINT
Error
Selenium::WebDriver::Error::TimeOutError: timeout
(Session info: headless chrome=84.0.4147.105)
(Driver info: chromedriver=2.41.578700
Related
When I set the option "--remote-debugging-port", it will throw an error. And without this option, it can work properly. However, I want to debug when using the headless chrome. How can I do ?
.rvm/rubies/ruby-2.2.5/lib/ruby/2.2.0/net/protocol.rb:158:in `rescue in rbuf_fill': Net::ReadTimeout (Net::ReadTimeout)
Here is my code:
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--remote-debugging-port') # debug
#driver = Selenium::WebDriver.for(:chrome, options: options)
puts #driver.manage.logs.get :browser
As per Getting Started with Headless Chrome to enable remote-debugging you can add the argument remote-debugging-port through Selenium::WebDriver::Chrome::Options.new which will help in:
Navigating to http://localhost:9222 in another browser to open the DevTools interface or use a tool such as Selenium to drive the headless browser.
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--remote-debugging-port=9222')
#driver = Selenium::WebDriver.for(:chrome, options: options)
puts #driver.manage.logs.get :browser
I am using windows 10 32 bit ruby version 233, I am facing these issues with running the tests on a ie 11 browser and the chrome browser for the tests(using page object) that are already running on firefox browser,
ISSUE with IE: Watir::Exception::NoMatchingWindowFoundException:
browser window was closed (eval):1:in `process_watir_call'
ISSUE with Chrome: Errno::ECONNREFUSED: Failed to open TCP connection
to 127.0.0.1:9515 (No connection could be made because the target
machine actively refused it. - connect(2) for "127.0.0.1" port 9515)
i have used the following hooks for ie and chrome:
Before do
case $browser
when 'mozilla'
#browser = Watir::Browser.new :firefox
#browser.window.maximize
when 'chrome'
#browser = Watir::Browser.new :chrome, :profile => "default"
when 'ie'
#browser = Watir::Browser.new :ie
# #browser.window.maximize
# #browser.visible = true
else
#browser = Watir::Browser.new :firefox
#browser.window.maximize
end
# #browser = Watir::Browser.new :firefox
# #browser.window.maximize
# this file contains test data that needs to be changed if tests are being executed in a different environment
$test_data = YAML.load_file('features/support/input_data/data/login_information.yml')
# this file contains base URL that needs to be changed if tests are being executed in a different environment
FigNewton.load('default.yml')
end
I was able to run a small sample test on a separate project from ruby mine with Watir gem.
Is there any way to make it work on the existing firefox tests?
The problem with chrome has been fixed by using the right chromedriver version, but still having problems with the IE using watir.
Found the solution for IE too its something to do with the internet options security and lower down the security level and uncheck the Enable Protected Mode.
Recently I went through the chrome headless browser selenium automation. I use Selenium-cucumber with ruby.Now I wanted to run my whole project in chrome headless mode using Selenium::WebDriver::Chrome::Options.
I updated selenium-webdriver, chrome driver and my chrome version is 61.0.3163.100
In order to get to know its working, I created test.rb with below code:
require "selenium-webdriver"
#configure the driver to run in headless mode
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('--headless')
options.add_argument('--disable-gpu')
driver = Selenium::WebDriver.for :chrome, options: options
driver.get('http://stackoverflow.com/')
puts driver.title
driver.quit
I was able to get the response as below:
>>ruby test.rb
[1012/112029.980:ERROR:devtools_http_handler.cc(786)]
DevTools listening on 127.0.0.1:12703
Stack Overflow - Where Developers Learn, Share, & Build Careers
Now as I got the response correctly I want to configure the driver to run my project in headless mode.
so I edited the env.rb file of the support folder:
Imported require 'selenium-webdriver' and edited the below part where we create driver instance:
else # else create driver instance for desktop browser
begin
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('headless')
options.add_argument('--disable-gpu')
driver = Selenium::WebDriver.for :chrome, options: options
driver.get('http://stackoverflow.com/')
puts driver.title
rescue Exception => e
puts e.message
Process.exit(0)
end
end
I ran the command >>cucumber BROWSER=chrome
uninitialized constant Selenium::WebDriver::Chrome::Options
Did you mean? Selenium::WebDriver::Options
Please help me to resolve this issue and How can I run my whole selenium-cucumber project in headless mode?
I'm learning to use Selenium for basic things like taking a screenshot, scraping, and testing and would like to use it with headless Chrome, which is now stable as of Chrome 59.
I have been able to take a screenshot using the 'selenium-webdriver' gem and chromedriver, but not headless.
Here is the ruby script that I am running which hangs after starting to initialize the driver
require 'rubygems'
require 'selenium-webdriver'
Selenium::WebDriver.logger.level = :debug
p 'initializing driver'
driver = Selenium::WebDriver.for :chrome, switches: %w[--headless --disable-gpu --screenshot --hide-scrollbars]
p 'navigating to Google'
driver.navigate.to "http://google.com"
driver.save_screenshot("./screen.png")
driver.quit
and the output from the logs:
:> ruby rubytest.rb
"initializing driver"
2017-06-07 15:55:43 DEBUG Selenium Executing Process
["/Users/name/Documents/scrapings/python/env/bin/chromedriver", "--port=9515"]
2017-06-07 15:55:43 DEBUG Selenium polling for socket on ["127.0.0.1", 9515]
Starting ChromeDriver 2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b) on port 9515
Only local connections are allowed.
2017-06-07 15:55:43 INFO Selenium -> POST session
2017-06-07 15:55:43 INFO Selenium >>> http://127.0.0.1:9515/session | {"desiredCapabilities":{"browserName":"chrome","version":"","platform":"ANY","javascriptEnabled":true,"cssSelectorsEnabled":true,"takesScreenshot":false,"nativeEvents":false,"rotatable":false,"chromeOptions":{"args":["--headless","--disable-gpu","--screenshot","--hide-scrollbars"]}}}
2017-06-07 15:55:43 DEBUG Selenium > {"Accept"=>"application/json", "Content-Type"=>"application/json; charset=utf-8", "Content-Length"=>"284"}
[RUBY BACKTRACE TO DRIVER INITIALIZATION]
I have tried using the JavaScript and Python drivers with similar code and nothing works. When I try this with Python, the error message is
WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.5 x86_64)
I found this blog post useful for setting up headless chrome with selenium in ruby
require "selenium-webdriver"
# configure the driver to run in headless mode
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('--headless')
driver = Selenium::WebDriver.for :chrome, options: options
driver.navigate.to "https://www.google.com"
# resize the window and take a screenshot
driver.manage.window.resize_to(800, 800)
driver.save_screenshot "screenshot.png"
Managed to work through this in the end via various docs, blog posts and gists.
caps = Selenium::WebDriver::Remote::Capabilities.chrome("desiredCapabilities" => {"takesScreenshot" => true}, "chromeOptions" => {"binary" => "/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary"})
driver = Selenium::WebDriver.for :chrome, desired_capabilities: caps, switches: %w[--headless --no-sandbox --disable-gpu --remote-debugin-port=9222 --screen-size=1200x800]
You need to use a very recent version of Chrome (I'm using Canary) and tell Selenium the path to the binary. You also need to set the desired capabilities for 'takesScreenshot' to true.
I've written a blog post about how to do it. Summing up:
1) Make sure you have Chrome version 57+ on Linux, 59+ on macOS or 60+ on Windows;
2) Add/update the gem selenium-webdriver;
3) Make sure you’re using ChromeDriver version 2.30 or higher;
4) Add the following driver to your spec_helper.rb or rails_helper.rb:
Capybara.register_driver :chrome do |app|
Capybara::Selenium::Driver.new app, browser: :chrome,
options: Selenium::WebDriver::Chrome::Options.new(args: %w[headless disable-gpu])
end
Capybara.javascript_driver = :chrome
I'm trying to change webdriver in ruby to open a tor browser instead of the default firefox broswer. I'm using the following code and I have a tor browser open before I run this code.
path='C:\Users\Bonnnie\Downloads\Tor Browser\App\tor.exe'
Selenium::WebDriver::Firefox.path = path
driver = Selenium::WebDriver.for :firefox
I get the following error:
unable to obtain stable firefox connection in 60 seconds
I think I might be linking to the wrong tor file.
The following worked for me with selenium-webdriver 2.48.1 and Tor Browser Bundle 5.0.3 on Ubuntu Linux 15.04.
require 'selenium-webdriver'
tor_dir = '/opt/tor-browser_en-US'
# The Tor binary relies on these shared libraries
ENV['LD_LIBRARY_PATH']= [
File.join(tor_dir, 'Browser/'),
File.join(tor_dir, 'Browser/TorBrowser/Tor/')
].join(':')
Selenium::WebDriver::Firefox::Binary.path =
File.join(tor_dir, 'Browser/firefox')
profile = Selenium::WebDriver::Firefox::Profile.new(
File.join(tor_dir, 'Browser/TorBrowser/Data/Browser/profile.default'))
driver = Selenium::WebDriver.for :firefox, :profile => profile
driver.get('https://check.torproject.org/')
require 'selenium-webdriver'
tor_dir = '/home/me/tor-browser_en-US' # change as necessary
options = Selenium::WebDriver::Firefox::Options.new
options.binary = File.join(tor_dir, 'Browser/firefox')
proxy = Selenium::WebDriver::Proxy.new(socks: '127.0.0.1:9050')
proxy.socks_version = 5
options.proxy = proxy
driver = Selenium::WebDriver.for :firefox, options: options
driver.get('https://check.torproject.org')
driver.title
=> "Congratulations. This browser is configured to use Tor."
I had to set the browser proxy to use my locally running tor, as Selenium evidently doesn't use the one bundled with Tor Browser Bundle. With no proxy set I get Selenium::WebDriver::Error::UnknownError: Reached error page: about:neterror?e=proxyConnectFailure..