Chrome headless download pdf - ruby

Question

I've got a script which downloads a pdf from a site which keeps updating every month and I want to automate this. It works but I cannot get it to work headless and I think it's because it's not handling the downloads correctly. It seems to start the chrome in headless ok and my navigation commands seem to work, but when it goes to download nothing happens.
#!/usr/bin/env ruby
#
require 'capybara'
require 'rb-inotify'
require 'webdrivers/chromedriver'
def initialise
Capybara.register_driver :chrome do |app|
Capybara::Selenium::Driver.new(app, :browser => :chrome, options: chrome_options)
end
#session = Capybara::Session.new(:chrome)
end
# Settings and profile for the Chrome Browser
# NOTE: still cannot get headless working
def chrome_options
opts = Selenium::WebDriver::Chrome::Options.new
opts.add_argument('--headless') unless ENV['UI']
opts.add_argument('--no-sandbox')
opts.add_argument('--disable-gpu')
opts.add_argument('--disable-dev-shm-usage')
opts.add_argument('--window-size=1920,1080')
opts.add_preference(:download,
directory_upgrade: true,
prompt_for_download: false,
default_directory: "~/Downloads")
opts.add_preference(:plugins,
plugins_disabled: ["Chrome PDF Viewer"])
opts.add_preference(:browser, set_download_behavior: { behavior: 'allow' })
opts
end
Update I'm using Chrome version 81.0.4044.113-1

Answer 1

Across different versions of Chrome and selenium-webdriver the settings required to get downloads working have changed/grown. It looks like you're missing one of them.
opts.add_preference('download.default_directory', '~/Downloads')
Another thing you can also do, depending on versions, is
def initialise
Capybara.register_driver :chrome do |app|
Capybara::Selenium::Driver.new(app, :browser => :chrome, options: chrome_options).tap do |driver|
driver.browser.download_path = '~/Downloads'
end
#session = Capybara::Session.new(:chrome)
end

Answer 2

I added some configurations to your code, probably it will work:
#!/usr/bin/env ruby
#
require 'capybara'
require 'rb-inotify'
require 'webdrivers/chromedriver'
def initialise
Capybara.register_driver :chrome do |app|
Capybara::Selenium::Driver.new(app, :browser => :chrome, options: chrome_options)
end
#session = Capybara::Session.new(:chrome)
end
# Settings and profile for the Chrome Browser
# NOTE: still cannot get headless working
def chrome_options
download_directory = "~/Downloads"
opts = Selenium::WebDriver::Chrome::Options.new
opts.add_argument('--headless') unless ENV['UI']
opts.add_argument('--no-sandbox')
opts.add_argument('--disable-gpu')
opts.add_argument('--disable-dev-shm-usage')
opts.add_argument('--window-size=1920,1080')
opts.add_preference(:download,
directory_upgrade: true,
prompt_for_download: false,
default_directory: download_directory)
opts.add_preference(:browser, set_download_behavior: { behavior: 'allow' })
driver = Capybara::Selenium::Driver.new(app, browser: :chrome,
options: options)
bridge = driver.browser.send(:bridge)
path = '/session/:session_id/chromium/send_command'
path[':session_id'] = bridge.session_id
bridge.http.call(:post, path, cmd: 'Page.setDownloadBehavior',
params: {
behavior: 'allow',
downloadPath: download_directory
})
driver
opts.add_preference(:plugins,
always_open_pdf_externally: true)
opts.add_preference(:browser, set_download_behavior: { behavior: 'allow' })
opts
end