I have a problem setting the preferred (accepted language) within headless Chrome using Selenium Webdriver and Ruby. I use the following WebDriver settings:
Selenium::WebDriver::Chrome.driver_path = #config[<path to the Chrome Driver>]
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('--headless')
options.add_argument('--disable-translate')
options.add_argument("--lang=de")
The driver is then initialized with:
#selenium_driver = Selenium::WebDriver.for :chrome, options: options
Everything works fine but at some pages Chrome returns English content even when I navigate to the German page URL (e.g. page.de). In these cases the Chrome driver returns the English content due to an internal forwarding to page.de/en. I do not specify the en path in my queried URL.
I have tried to set the language using the Webdriver preference:
options.add_preference('accept_languages', 'de')
instead of the add_argument but it doesn't change anything of the behavior.
Does anyone have an idea how to force a headless Chrome controlled by Selenium Webdriver within Ruby to request page content in a defined language or - not optimal but it might help as a workaround - to stop the forwarding?
Any help greatly appreciated
Best
Krid
I found a solution that works for me. As in many cases the problem was sitting in front of the screen and simply doesn't work precisely enough ;-)
Instead of using
options.add_argument("--lang=de")
you have to use
options.add_argument("--lang=de-DE")
When I use an IETF language tag the code I initially posted works as expected.
I'am using this in my test_helper.rb Works fine for me.
Capybara.register_driver :selenium do |app|
Chromedriver.set_version "2.36"
desired_capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
'chromeOptions' => {
'prefs' => {
'intl.accept_languages' => 'en-US'
},
args: ['disable-gpu', 'headless']
}
)
Capybara::Selenium::Driver.new(app, { browser: :chrome, desired_capabilities: desired_capabilities })
end
Capybara.javascript_driver = :chrome
Capybara.default_driver = :selenium
This prefs hash inside an options hash did the trick for me. It's at the end of the driven_by :selenium line.
(Inside test/application_syste_test_case.rb)
# frozen_string_literal: true
require 'test_helper'
require 'capybara/rails'
class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
driven_by :selenium, using: :chrome, screen_size: [1400, 1400], options: { prefs: { 'intl.accept_languages' => 'de,de-DE;q=0.9,en;q=0.1' } }
# ...
2021-06-14 UPDATE:
The previous example produces this deprecation warning:
WARN Selenium [DEPRECATION] :prefs is deprecated. Use Selenium::WebDriver::Chrome::Options#add_preference instead.
IMO, the solution below is uglier, but I'm posting it for when it's fully deprecated and the original stops working.
class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
driven_by(:selenium,
using: :chrome,
screen_size: [1400, 1400],
options: {
options: Selenium::WebDriver::Chrome::Options.new(
prefs: { 'intl.accept_languages' => 'de,de-DE;q=0.9,en;q=0.1' }
)
},
)
You should be able to solve your problem by adding an experimental option:
options.add_option('prefs', {'intl.accept_languages': 'en,en_US'})
I'm sure it works with Python, but I've not tried with Ruby: this approach is the correct one, not sure about the implementation.
You can find in this repository the code which handles your problem in Python code, and in this Q&A how to implement experimental_options in Ruby
For me works:
options = Selenium::WebDriver::Firefox::Options.new
options.add_preference("intl.accept_languages", 'de-DE')
Capybara::Selenium::Driver.new(app, browser: :firefox, options: options)
Related
Super beginner here. Trying to update this test with Selenium WebDriver using Ruby the course I was going through uses the below which is deprecated.
driver = Selenium::WebDriver.for :remote, desired_capabilities: :firefox
The error I'm getting in cmd when I try to run the tests is
"WARN Selenium [DEPRECATION] [:desired_capabilities] :desired_capabilities as a parameter for driver initialization is deprecated. Use :capabilities with an Array value of capabilities/options if necessary instead."
So I've tried to find examples of how to suppress the error mentioned in This Link, but I'm having trouble finding examples of how to implement that.
I've also tried to look up several ways to just use capabilities: as suggested, but I'm having trouble finding resources for that also, so I've just messed around and tried different combinations to no avail.
Curious if anyone knows of anything that would help me find the answer for this?
Also looked at these sources
https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities
https://www.lambdatest.com/blog/desired-capabilities-in-selenium-testing/
Selenium include firefox profile into DesiredCapabilities for Remote Driver
https://www.selenium.dev/documentation/webdriver/remote_webdriver/
Based off the last link, I believe the below should work? But I'm sure I'm just missing something with syntax.
driver = Selenium::WebDriver.for :Remote::Capabilities.firefox
Selenium capabilities are not where they should be in Ruby. You want to avoid using Capabilities entirely now.
Here's an example in the Selenium documentation with a before/after for how to use Options properly: https://www.selenium.dev/documentation/webdriver/getting_started/upgrade_to_selenium_4/#before
It does not match how the other Selenium languages do things, so I'm planning to change the deprecations in Selenium 4.3 to get them to match.
# spec/rails_helper.rb
Capybara.server = :puma, { Silent: true }
Capybara.server_port = 9887
Capybara.register_driver :headless_chrome do |app|
options = ::Selenium::WebDriver::Chrome::Options.new.tap do |opts|
opts.args << '--headless'
opts.args << '--disable-site-isolation-trials'
opts.args << '--no-sandbox'
end
options.add_preference(:download, prompt_for_download: false, default_directory: Rails.root.join('tmp/capybara/downloads'))
options.add_preference(:browser, set_download_behavior: { behavior: 'allow' })
service_options = ::Selenium::WebDriver::Service.chrome(
args: {
port: 9515,
read_timeout: 120
}
)
remote_caps = Selenium::WebDriver::Remote::Capabilities.chrome(
'goog:loggingPrefs': {
browser: ENV['JS_LOG'].to_s == 'true' ? 'ALL' : nil
}.compact
)
Capybara::Selenium::Driver.new(
app,
browser: :chrome,
capabilities: [remote_caps, options],
service: service_options,
timeout: 120
)
end
Capybara::Screenshot.register_driver(:headless_chrome) do |driver, path|
driver.browser.save_screenshot(path)
end
Capybara.javascript_driver = :headless_chrome
I'd hope this be helpfull for you.
I have code with the Chrome browser and this works, but I never worked with Firefox, but an Ubuntu server normally works only with Firefox, and now I have a question: How can I work with a proxy on the Firefox browser using the proxy_chain_rb gem?
I think my code for the Chrome browser will work in Firefox if you tell me how I can make Firefox options. My problem - I don’t know how I can use Firefox options and manuals are old. How can I replace my code for Google in Firefox?
Code
require 'watir'
require 'proxy_chain_rb'
require 'selenium-webdriver'
time2 = Time.now
file = File.new("report.json", "a:UTF-8")
myuseragent = File.readlines("user_agents.txt").sample
options = Selenium::WebDriver::Chrome::Options.new
options.add_emulation(user_agent: (myuseragent))
options.add_argument('--headless')
puts "Work started: " + time2.inspect
u_proxy = File.readlines("proxy.txt").sample
real_proxy = u_proxy
server = ProxyChainRb::Server.new
generated_proxy = server.start(real_proxy)
proxy = {
http: generated_proxy,
ssl: generated_proxy
}
caps = Selenium::WebDriver::Remote::Capabilities.chrome(:proxy => proxy)
driver = Selenium::WebDriver.for :chrome, :desired_capabilities => caps, options: options
driver.execute_script('return navigator.userAgent')
driver.get "https://raskruty.ru/"
The below java code works fine in my machine to disable automation extension.
How I can write a replica of this code in ruby
Java
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("useAutomationExtension", false);
In Watir Ruby
this code doesn't work
Watir::Browser.new(:chrome, :switches => %w[--disable-popup-blocking --disable-extensions --disable-infobars] )
Maybe try passing it as a raw option:
b = Watir::Browser.new :chrome, options: {options: {'useAutomationExtension' => false}}
You should be able to pass them along with args: See the update of Watir 6.6: http://watir.com/watir-6-6/
In short:
Watir::Browser.new :chrome, options: {args: %w[--disable-popup-blocking --disable-extensions --disable-infobars]}
This is how I like to start the browser keeping all options variable.
browser_name = ENV['BROWSER'] || 'chrome'
settings = {}
if ENV['DEVICE'] == 'remote'
settings[:url] = 'http://selenium__standalone-chrome:4444/wd/hub/'
end
if browser_name == 'chrome'
args = ['--ignore-certificate-errors', '--disable-popup-blocking', '--disable-translate', '--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream']
settings[:options] = {args: args}
end
Watir::Browser.new browser_name.to_sym, settings
I found the root cause which is not allowing the script to run.
I navigated to below path in "regedit" and deleted the "ExtensionInstallBlacklist" folder but this is a temporary solution the registry will be auto created in some time.
Path:
“HKEY_CURRENT_USER\Software\Policies\Google\Chrome\ExtensionInstallBlacklist”
My search has only turned up answers for Java - this and this
The selenium gem doesn't include addCommandLineOptions as far as I can tell, but it does have WebDriver::Remote::Capabilities.
How do I use it to add arguments? I know you pass it as desired_capabilities: to the driver constructor, but in what format?
Unfortunately the documentation has been especially useless
You can set --start-maximizedby following for Chrome. See this post for more details.
Capybara.register_driver :chrome_maximize do |app|
caps = Selenium::WebDriver::Remote::Capabilities.chrome(
'chromeOptions' => {
"args" => [ "--start-maximized", "--otherthings" ]
}
)
$driver = Capybara::Selenium::Driver.new(app, {:browser => :chrome, :desired_capabilities => caps})
end
I finally figured out!!! Here is a working example for gem "selenium-webdriver". Should work too for Capybara as well.
First line is if you want to run a custom binary. In case of --headless command line argument, support starts from firefox version 55. Don't forget to make firefox-nightly available to $PATH Env Var.
Selenium::WebDriver::Firefox.path = "/home/user/bin/firefox-nightly"
caps = Selenium::WebDriver::Remote::Capabilities.firefox(
"moz:firefoxOptions" => {
args: ["--headless"] # and other arguments...
}
)
driver = Selenium::WebDriver.for :firefox, desired_capabilities: caps
# do stuff here ....
driver.quit
I learned how to use Firefox 4 with watir and webdriver (on Win7 x64), setting profile items. Example:
profile = Selenium::WebDriver::Firefox::Profile.new
profile["browser.download.useDownloadDir"] = true
profile["browser.download.dir"] = 'D:\\FirefoxDownloads'
profile["browser.helperApps.neverAsk.saveToDisk"] = "application/csv"
driver = Selenium::WebDriver.for :firefox, :profile => profile
browser = Watir::Browser.new(driver)
What I try to do with the example below, is setting CSV files to be always downloaded to a specific directory, never opened.
The code above succeeds in setting all the files automatically downloaded to the specified directory, but setting browser.helperApps.neverAsk.saveToDisk has no effect: I still get the open/save question.
After the script runs, the Firefox window is still open, and I enter the URL about:config.
I can see that browser.helperApps.neverAsk.saveToDisk was correctly set to application.csv , but in firefox/options/options/applications I don't see the entry for CSV files.
It seems that the menu setting, that is really effective, is not really bound with the about:config setting.
What am I doing wrong?
I've done some testing of this for you, unfortunately there doesn't seem to be a standard content-type for CSV files. You can try passing a comma separated list of content-types, hopefully one of those work for you. For me it was application/octet-stream that did the trick...
require 'watir-webdriver'
require 'selenium-webdriver'
profile = Selenium::WebDriver::Firefox::Profile.new
profile["browser.download.useDownloadDir"] = true
profile["browser.download.dir"] = '/tmp'
profile["browser.helperApps.neverAsk.saveToDisk"] = "text/plain, application/vnd.ms-excel, text/csv, text/comma-separated-values, application/octet-stream"
driver = Selenium::WebDriver.for :firefox, :profile => profile
browser = Watir::Browser.new(driver)
browser.goto "http://altentee.com/test/test.csv"
In Firefox 6+, I couldn't get this to work without specifically setting the 'browser.download.folderList' value:
profile = Selenium::WebDriver::Firefox::Profile.new
profile['browser.download.folderList'] = 2 #custom location
profile['browser.download.dir'] = download_directory
profile['browser.helperApps.neverAsk.saveToDisk'] = "text/csv, application/csv"
b = Watir::Browser.new :firefox, :profile => profile
See: http://watirwebdriver.com/browser-downloads/