I'm trying to get Ruby's Net::HTTP implementation to work with SNI.
Both mail.google.com and gmail.com live on the same IP address, so when connecting via SSL, the Google server needs to know which certificate to use. By default, it returns the mail.google.com certificate, which is a problem if you're trying to implement WebFinger.
WebFinger requires you to retrieve https://gmail.com/.well-known/host-meta to get the LRDD information, however, for security reasons, it's critical to verify the SSL certificate information.
Since Google serves up the default mail.google.com certificate in this case, the SSL post_connection_check fails. The correct solution here would be to enable Server Name Indication for Net::HTTP, but it's not clear to me how to get that working with the Ruby bindings for OpenSSL. Anyone else have an idea?
You should be able to see the problem by running:
require 'open-uri'
open('https://gmail.com/.well-known/host-meta') { |f| f.read }
I've also created a gist that exhibits the problem using an earlier version of curl and OpenSSL:
https://gist.github.com/7936ef38787092a22897
For SNI support, you need a newer OpenSSL release (0.9.8f with --enable-tlsext or 0.9.8j or later) and call OpenSSL::SSL::SSLSocket#hostname = 'hostname' before SSLSocket#connect. Net::HTTPS does not support SNI yet, and open-uri doesn't.
Checking out httpclient development repository should support SNI.
https://github.com/nahi/httpclient
Let me know if you need released gem real soon now...
Ruby 2.0 will address the TLS SNI (Server Name Indication) issue:
from net/http..
# ...
# s.session = #ssl_session if #ssl_session
# # Server Name Indication (SNI) RFC 3546
# s.hostname = #address if s.respond_to? :hostname=
# Timeout.timeout(#open_timeout, Net::OpenTimeout) { s.connect }
# if #ssl_context.verify_mode != OpenSSL::SSL::VERIFY_NONE
# s.post_connection_check(#address)
# end
# ...
To make this work in 1.9.2 (or higher )
apply similar patch to net/http
# ...
# BEGIN: SNI PATCH http://bugs.ruby-lang.org/issues/4351
# s.hostname = #address if s.respond_to? :hostname=
# END: SNI PATCH http://bugs.ruby-lang.org/issues/4351
# timeout(#open_timeout) { s.connect }
# if #ssl_context.verify_mode != OpenSSL::SSL::VERIFY_NONE
# s.post_connection_check(#address)
# end
# ...
see also:
http://bugs.ruby-lang.org/issues/4351
http://en.wikipedia.org/wiki/Server_Name_Indication
Related
I'm using a local server to test an application, and make requests to that server from my own machine.
The test server's SSL is bad, and HTTParty throws errors because of that. From what I read, HTTParty should ignore SSL by default, but when I try to do this:
HTTParty.get( "#{ #settings.api_server }#{ url }" ).parsed_response
It throws this error:
OpenSSL::SSL::SSLError at /
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
How do I make it ignore the SSL?
In the latest HTTParty, you can use the verify option to disable SSL verification;
HTTParty.get( "#{ #settings.api_server }#{ url }", :verify => false ).parsed_response
To make HTTParty always skip SSL cert verification, and not have to specify this in every call:
require 'httparty'
HTTParty::Basement.default_options.update(verify: false)
HTTParty.get("#{#settings.api_ssl_server}#{url1}")
HTTParty.get("#{#settings.api_ssl_server}#{url2}")
HTTParty.get("#{#settings.api_ssl_server}#{url3}")
# ...
You can also do this scoped to a class when including HTTParty as a module:
require 'httparty'
class Client
include HTTParty
default_options.update(verify: false)
end
Client.get("#{#settings.api_ssl_server}#{url1}")
Client.get("#{#settings.api_ssl_server}#{url2}")
Client.get("#{#settings.api_ssl_server}#{url3}")
Or
require 'httparty'
module APIHelpers
class Client
include HTTParty
default_options.update(verify: false)
end
end
World(APIHelpers)
Client.get("#{#settings.api_ssl_server}#{url1}")
Client.get("#{#settings.api_ssl_server}#{url2}")
Client.get("#{#settings.api_ssl_server}#{url3}")
If you want to still send your certificates, use this flag:
verify_peer: false
This may be totally off base, as I'm new to Ruby, but this is what worked for me when other solutions wouldnt
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE
Additional ways of doing this, if you get a 'dynamic constant assignment' (pulled from here)
OpenSSL::SSL.const_set(:VERIFY_PEER, OpenSSL::SSL::VERIFY_NONE)
A lot of what I've seen around SO and blog posts is that
By default, Net::HTTP does not verify peer SSL certificates
To force Net::HTTP to verify peer, you need to to it too and supply a CA cert file containing trusted authorities
However, I've found that this is not completely necessary. On my Mac, I have the following code
http = Net::HTTP.new('www.google.com', 443)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
http.start do
http.request_get('/') do |response|
puts response.body
end
end
This works. But what is Ruby using to verify the peer with? When I look at the output of http.ca_path and http.ca_file they are both blank.
I'm running this code in Ruby 1.9.3p374. Perhaps when Ruby was compiled on my Mac, it pulled in some default location of CA cert chains from openssl or something?
It looks like if no verify callback is set then OpenSSL library default callback is used. See line 217 in the openssl module verify callback wrapper (the ok argument is the result of the default callback).
On OS X, Apple has apparently customized the default callback in their OpenSSL library to hook in to the OS X Keychain facility. The root CA of Google's certificate is Equifax Secure Certificate Authority. If you change the trust setting in Keychain (e.g. using the Keychain Access utility) for this CA then your ruby test behaves accordingly.
I believe the answer to your question is that when peer certificate verification is required but not configured, the ruby shipped with OS X verifies using Keychain.
This is the code I'm using to connect to the SSL site.
require 'mechanize'
a = Mechanize.new
page = a.get 'https://site.com'
I"m using using Ruby 1.9.3 and Mechanize 2.1pre1 + dependencies. On Mac the above code works and returns the page. On windows 7 running the same versions it gives me the following error:
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3
read server certificate B: certificate verify failed
Reverting to Mechanize 2.0.1 seems to solve this problem, but I then get plagued with the too many connections reset by peer problem. Thus that is not a solution.
I've tried doing a.verify_mode = false, but that does not do anything. I have read that you can turn off SSL verification by using:
open(uri,:ssl_verify_mode => OpenSSL::SSL::VERIFY_NONE)
How can I turn it off in Mechanize ? Why am I only getting this error on Windows ?
The version of OpenSSL (the library used to establish secure connections with Net::HTTPS) is not able to properly find the certificate chain in your computer.
To our bad, OpenSSL was never able to use the Windows installed cert storage to validate remote servers so is failing because of that.
From your example, you can do:
a.agent.http.verify_mode = OpenSSL::SSL::VERIFY_NONE
To avoid the verification, however that is far from ideal (due clear security issues)
I recommend you download some cert bundles (like the ones from curl):
http://curl.haxx.se/ca
And modify your code to something like this:
require "rbconfig"
require "mechanize"
a = Mechanize.new
# conditionally set certificate under Windows
# http://blog.emptyway.com/2009/11/03/proper-way-to-detect-windows-platform-in-ruby/
if RbConfig::CONFIG["host_os"] =~ /mingw|mswin/
# http://curl.haxx.se/ca
ca_path = File.expand_path "~/Tools/bin/curl-ca-bundle.crt"
a.agent.http.ca_file = ca_path
end
page = a.get "https://github.com/"
That seems to work, Ruby 1.9.3-p0 (i386-mingw32), Windows 7 x64 and mechanize 2.1.pre.1
Hope that helps.
Luis' answer looks fine but more generally:
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE
You can simply do the following:
agent = Mechanize.new
agent.verify_mode = OpenSSL::SSL::VERIFY_NONE
This worked on the latest version 2.8
I am attempting to use a remote system for user authentication. This chunk of code gets a response when I run it on MacOSX, but fails on my machine:
def create
uri = URI.parse('https://ourclient.example.com/')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new('/login.jsp')
request.set_form_data({'login_name' => params[:login], 'password' => params[:password]})
response = http.request(request)
puts "Response BODY: #{response.body.inspect}"
end
Turning verify off gets rid of a warning on the Macs. On my machine, the http.request raises this exception:
OpenSSL::SSL::SSLError (SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: sslv3 alert illegal parameter):
app/controllers/sessions_controller.rb:16:in `create'
I get the same behavior using IRB without Rails. I did a clean install of Fedora 14 yesterday, installed the required development tools and libraries. I'm using Ruby 1.9.2-p180, and Rails 3.0.4. I thought I might have had my libraries misconfigured (I had Fedora 12 that had been upgraded a few times), but this is now a new install.
The remote system is probably Microsoft's IIS, but I'm not certain of that. Perhaps I can use an older SSL protocol, but my Google-fu can't find the incantation.
I would appreciate any tips on resolving this issue. Thanks,
Chris
how was created the ssl cert on the server? self-signed or ca-signed?
which is the error you receive if you remove the VERIFY_NONE?
use Mechanize gem for it
gem install mechanize
and try
require 'mechanize'
page = Mechanize.new{|a| a.ssl_version, a.verify_mode = 'SSLv3', OpenSSL::SSL::VERIFY_NONE}.get "**YOUR HTTPS LINK HERE**"
I am trying to access an HTTPS web service that uses SSL cert authentication using Ruby EventMachine but I am not getting it to work.
I have written the following simple code block to test it end-to-end:
require 'rubygems'
require 'em-http'
EventMachine.run do
url = 'https://foobar.com/'
ssl_opts = {:private_key_file => '/tmp/private.key',
:cert_chain_file => '/tmp/ca.pem',
:verify_peer => false}
http = EventMachine::HttpRequest.new(url).get :ssl => ssl_opts
http.callback do
p http.response_header.status
p http.response_header
p http.response
EventMachine.stop
end
http.errback do
EventMachine.stop
fail "Request failed"
end
end
Running the above outputs <SSL_incomp> followed by the raised RuntimeError message. I have tried running with :verify_peer set to both true and false and it gives me the same error. Running EventMachine::HttpRequest#get without the :ssl option does the same.
I have also tried sending the request to GMail (https://mail.google.com) without the :ssl option (i.e. plain HTTPS without cert) and that works, outputting status code 200, the headers and the body.
I have tried doing the same request to the web service with curl and that works:
curl --silent --cert /tmp/private.key --cacert /tmp/ca.pem https://foobar.com/
I am thinking that I am either using the em-http-request gem or EventMachine incorrectly or that the SSL files are in a format that works with curl but not EventMachine.
I someone knows how to solve the example above or provide a similar example using EventMachine directly would be much appreciated!
The file passed to curl's --cert contains both the cert and the key (unless you pass in a --key separately). Just use /tmp/private.key as the argument to both :private_key_file and :cert_chain_file
See http://github.com/eventmachine/eventmachine/issues/#issue/115 for more details about the issue and a patch that exposes the underlying error (instead of just printing out SSL_incomp).