Ruby net/imap getting OpenSSL::SSL::SSLError - self-signed certificate? - ruby

In trying to develop a tool for that would use IMAP to access Gmail, I'm running into difficulties with even this simple start-up code:
require 'net/imap'
imap = Net::IMAP.new('imap.gmail.com', ssl: true)
Running that, it fails as follows (Note: lightly edited for easier display):
Traceback (most recent call last):
5: from test-imap:2:in `<main>'
4: from test-imap:2:in `new'
3: from /.../.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/net/imap.rb:1092:in `initialize'
2: from /.../.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/net/imap.rb:1533:in `start_tls_session'
1: from /.../.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/net/protocol.rb:44:in `ssl_socket_connect'
/.../.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/net/protocol.rb:44:in
`connect_nonblock': SSL_connect returned=1 errno=0 state=error:
certificate verify failed (self signed certificate) (OpenSSL::SSL::SSLError)
Searching around, I found some somewhat-similar issues, e.g. imap-backup issue #57, ruby/openssl issue #238 (still open, as of this writing closed after sharing the answer below), and rbenv/ruby-build issue #380... but nothing on SO.
Piecing info from the above resources together, I came up with this command to try:
openssl s_client -connect imap.gmail.com:993 \
-CAfile $(ruby -ropenssl -e 'puts OpenSSL::X509::DEFAULT_CERT_FILE') \
< /dev/null > /dev/null
Which reports:
depth=2 /OU=GlobalSign Root CA - R2/O=GlobalSign/CN=GlobalSign
verify return:1
depth=1 /C=US/O=Google Trust Services/CN=Google Internet Authority G3
verify return:1
depth=0 /C=US/ST=California/L=Mountain View/O=Google LLC/CN=imap.gmail.com
verify return:1
DONE
So, it seems that the SSL certificate does actually verify as OK in that way (as one might expect).
I did find some instructions for using net/imap with SSL with host-checking disabled, which works... but I'd really prefer not to do that.
I also found a non-IMAP gmail interface, but my intention is a tool that could also be used with other IMAP providers, so I'm specifically aiming to stick to IMAP here. So:
How can I get net/imap to connect successfully using SSL, and still validate the certificate (given that it is, in fact, valid)?

Ideal: upgrade to Ruby 2.6.3 or later
Upgrading to Ruby version 2.6.3 or later should fix things, as ruby pull #2077, which provides a fix for related issue #15594, was incorporated into the 2.6.3 change list. With 2.6.3, the original test snippet in the question now works.
If you can't upgrade:
That said, if upgrading is impractical for some reason, the commit that fixed imap-backup issue #57 offers a workaround-type solution. Instead of the following:
imap = Net::IMAP.new('imap.gmail.com', ssl: true)
Try this:
imap = Net::IMAP.new('imap.gmail.com', ssl: {ssl_version: :TLSv1_2})
That seems to make things work, even in Ruby 2.6.0.

In Ruby 2.4, I had to use this patch:
require 'net/imap'
class Net::IMAP
module UseSNI
def start_tls_session(*)
#sock.instance_variable_set(:#hostname, #host)
super
end
end
prepend UseSNI
end
class OpenSSL::SSL::SSLSocket
module UseSNI
def connect(*)
self.hostname = io.instance_variable_get(:#hostname)
super
end
end
prepend UseSNI
end

Related

Error Certificate verify failed (certificate has expired)): in Mac OSX 11.6.1 and ruby 3.0.3

I have a ruby on rails webapp sending requests to a third party SOAP API. When I request like:
endpoint = "https://www.booking-manager.com/cbm_web_service2/services/CBM?wsdl"
client = Savon.client(wsdl: endpoint,
#log_level: :info,
log_level: :debug,
log: true,
pretty_print_xml: true,
open_timeout: 300,
read_timeout: 300)
message = {'in0' => xxx,
'in1' => 'xxxx',
'in2' => 'xxx'}
response = client.call(:get_bases, message: message)
I´m getting next error:
HTTPI::SSLError (SSL_connect returned=1 errno=0 state=error: certificate verify failed (certificate has expired)):
The webapp is running under:
Mac OSX Big Sur 11.6.1
ruby 3.0.3p157 (2021-11-24 revision 3fb7d2cadc) [x86_64-darwin20]
I have this issue for weeks and I don´t know what else to do. According to many posts, I tested
openssl s_client -showcerts -host valid-isrgrootx1.letsencrypt.org -port 443
and got:
CONNECTED(00000005)
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
depth=3 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
---
Certificate chain
0 s:/CN=origin.letsencrypt.org
i:/C=US/O=Let's Encrypt/CN=R3
so, according to this: https://community.letsencrypt.org/t/help-thread-for-dst-root-ca-x3-expiration-september-2021/149190/970
I manually updated the file /etc/ssl/cert.pem to remove the DST Root CA X3 certificate. After that, I think that I moved one step forward. When running:
openssl s_client -showcerts -host valid-isrgrootx1.letsencrypt.org -port 443
Now, I don´t get the error and I think looks good:
CONNECTED(00000005)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = origin.letsencrypt.org
verify return:1
---
Certificate chain
0 s:/CN=origin.letsencrypt.org
i:/C=US/O=Let's Encrypt/CN=R3
However, unfortunately the error in my ruby app still remains the same. According to this, I understand ruby is running an openssl that is not getting the information from this certs. I´m not skilled with this at all and don´t know if this makes sense.
I just read other posts and checking openssl version
I got LibreSSL 2.8.3
which openssl
/usr/bin/openssl
In my /usr/local/opt I see three openssl versions folders:
openssl
openssl#1.1
openssl#3
I updated my .zshrc file and now openssl version notifies
OpenSSL 3.0.1 14 Dec 2021 (Library: OpenSSL 3.0.1 14 Dec 2021)
and ruby seems to be using:
ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.1.1l 24 Aug 2021
I´m aware that the ruby OpenSSL version is 1.1.1 and the system is running 3.0.1. I don´t know how to update ruby to run OpenSSL 3.0.1, although I´m not sure if this can be the root problem. I´m lost at this point.
UPDATE
I think I´m narrowing the issue down. My guess is that Ruby is using a version of openSSL, in this case 1.1.1, that is pointing to /Users/Rober/.rbenv/versions/3.0.3/openssl/ssl/certs bundler instead of pointing to /etc/ssl/cert.pem
irb
irb(main):001:0> require "openssl"
=> true
irb(main):002:0> puts OpenSSL::OPENSSL_VERSION
OpenSSL 1.1.1l 24 Aug 2021
=> nil
irb(main):003:0> puts "SSL_CERT_FILE: %s" % OpenSSL::X509::DEFAULT_CERT_FILE
irb(main):004:0> puts "SSL_CERT_DIR: %s" % OpenSSL::X509::DEFAULT_CERT_DIR
SSL_CERT_FILE: /Users/Rober/.rbenv/versions/3.0.3/openssl/ssl/cert.pem
SSL_CERT_DIR: /Users/Rober/.rbenv/versions/3.0.3/openssl/ssl/certs
This file /Users/Rober/.rbenv/versions/3.0.3/openssl/ssl/cert.pem , unfortunately when I check the content is in the format:
-----BEGIN CERTIFICATE-----
certificate chain
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
I mean, in this file /etc/ssl/cert.pem I could read some readable headers that helped identify the certificate to remove, but in this case the headers are not present, so it´s not possible.
I think that I probably just need to config ruby to run openssl to point to this file /etc/ssl/cert.pem. According to mamy posts, I just added export SSL_CERT_FILE="/etc/ssl/cert.pem" to my .zshrc file, but still getting
OpenSSL::X509::DEFAULT_CERT_FILE
SSL_CERT_FILE: /Users/Rober/.rbenv/versions/3.0.3/openssl/ssl/cert.pem
SOLUTION
Thanks to #JanGaraj that provided the right solution to this problem in my other production post: SSL_connect returned=1 errno=0 state=error: certificate verify failed in ruby and Ubuntu 14.04
Just to summarize, apart from the points depicted above, I just needed to update my web service request specifying my ca-certificates file, like: Savon.client(ssl_ca_cert_file: "/etc/ssl/certs/ca-certificates.crt ")
The solution to this question was provided in another post by #jangaraj
It looks like you are using Ubuntu 14 and Savon 2 client. Savon 2 client doc: https://www.savonrb.com/version2/globals.html
ssl_ca_cert_file
Sets the SSL ca cert file to use.
Savon.client(ssl_ca_cert_file: "lib/ca_cert.pem")
I would point ssl_ca_cert_file to /etc/ssl/certs/ca-certificates.crt explicitly.

How can I convert this openssl command in ruby?

I am trying to convert this command to ruby code using OpenSSL:
openssl s_client -verify_hostname www.example.com -connect example.com:443
I've pieced together that openssl has a verify_hostname method but I dont think that I am using this properly. Contextually I have a problem verifying a domain's SAN field. For a given domain I get wrong results. I provide a domain that is clearly a name mismatch buy I get a verify_result code of 0 which is ok. The command above gives me the right information but I can't seem to get this to translate in ruby. Is there an online tool that can convert this for me?
-verify_hostname is a parameter and s_client is a method name in your command. Check the output of openssl s_client --help to get more information. OpenSSL gem has a verify_hostname method but its source code looks like it's only checking if provided strings are correct, it does not call the provided host.
All http clients will do the SSL verification for you by default (unless you disable it):
require "net/http"
begin
Net::HTTP.get(URI("https://expired.badssl.com"))
rescue OpenSSL::SSL::SSLError
puts "bad ssl cert"
end
If you'd like to to the exact thing as your command does you'd probably need to download host's certificate first and then instantiate it with OpenSSL::X509::Certificate.new verify it with the check_validity method.

API Request - OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server hello A

Another of of these questions, I know this question has been asked (and answered) a lot on StackOverflow, but I can't get any of those to work for me and I also have a few questions I would like to learn.
Here is my error:
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server hello A
To start, here is my system settings.
I am on OSX El Capitan version 10.11.6
openssl version
OpenSSL 0.9.8zh 14 Jan 2016
which openssl
/usr/bin/openssl
ruby -v
ruby 2.1.6p336 (2015-04-13 revision 50298) [x86_64-darwin14.0]
rbenv -v
rbenv 0.4.0
My questions are these:
1) Does this error mean that a certificate was sent back to me, and then my OpenSSL version was unable to verify it? Did the other server have a chance to read mine, or even see it yet? Is there a way to dig into this request using Net::HTTP and inspect this other than opening up a program like Wireshark? Once I call net::HTTP.new.request(request) I seem to lose control and it just errors.
2) Did I even successfully talk to the other server, and it denied me?
3) At what point in the request am I in when I get this message?
and most of all
4) What are my options to get past this point
4a. So far i'm seeing a possible brew solution, but I haven't been able to get brew to link
4b. I could manually install Mozilla's CA (Or any other CA) into my Mac OSX Keychain
4c. Can I attach the file using the request.ca_file = "file" as I tried in my code? (see below)
4d. Is there any other solutions / best and most politically correct version?
5) Am I going to have this issue when I deploy to Heroku?
From what i'm reading, this is an issue of my OS not containing the correct CA files. the ca_file part is due to my first attempts to add the correct ca_file to my requests. I'm guessing I don't need that. I am using a Proxy with heroku because this API requires a static IP.
Here is my generic code
cert = File.read(File.join(Rails.root, 'ssl', 'test_env', 'their_test_cert.der'))
ca_file = File.read(File.join(Rails.root, 'ssl', 'test_env', 'Class3PublicPrimaryCA.der'))
uri = URI("https://xml.theirtestenv.com/api/receive")
headers = {
'x-IK-Version' => 'IKR/V4.00',
}
proxy_host = "myproxyhose"
proxy_port = "1234"
proxy_user = "myproxyuser"
proxy_pass = "myproxypass"
proxy_request = Net::HTTP.new(uri.hostname, '443', proxy_host, proxy_port, proxy_user, proxy_pass)
# http.key = OpenSSL::PKey::RSA.new(rsa_key)
proxy_request.use_ssl = true
proxy_request.cert = OpenSSL::X509::Certificate.new(cert)
proxy_request.ca_file = ca_file
proxy_request.verify_mode = OpenSSL::SSL::VERIFY_PEER
# proxy_request.ssl_version = :SSLv3
# This doesn't seem to matter whether I put this or not...
# Tried variations of these...
# proxy_request.ssl_version = :TLSv1
# proxy_request.ciphers = ['DES-CBC3-SHA']
post_request = Net::HTTP::Post.new(uri, headers)
post_request.content_type = "multipart/related"
response = proxy_request.request(post_request)
puts response.inspect
Also, i've noticed no matter what proxy_requst.ssl_version I put, my error always specifies SSLv2/v3, does that mean on their end they are requiring that version?
Sorry for all the questions. Thanks in advance
It's been awhile, but I just wanted to post that this was a couple of issues, the certificates I was passing were not the correct ones they were for the wrong environment. Once the correct certificates were passed this started working, though I never got the SSL Version questions quite figured out.

OpenSSL::SSL::SSLError: hostname does not match the server certificate

All of sudden today morning my HTTP client (HTTParty) threw an error OpenSSL::SSL::SSLError: hostname does not match the server certificate
Firstly I'm not able to understand which so today we have been make that api call almost all day number times from past 2 years without any issue
Secondly I don't understand how do I solve it since it internal to HTTParty
The only thing I know of is that I cant set SSL_CERT_FILE in ENV but as said I already have ROOT CA listed in my /etc/ssl/certs (SSL_CERT_DIR)
Here my output
irb(main):001:0> require "openssl"
=> true
irb(main):002:0> puts OpenSSL::OPENSSL_VERSION
OpenSSL 1.0.1 14 Mar 2012
=> nil
irb(main):003:0> puts "SSL_CERT_FILE: %s" % OpenSSL::X509::DEFAULT_CERT_FILE
SSL_CERT_FILE: /usr/lib/ssl/cert.pem
=> nil
irb(main):004:0> puts "SSL_CERT_DIR: %s" % OpenSSL::X509::DEFAULT_CERT_DIR
SSL_CERT_DIR: /usr/lib/ssl/certs
Lastly as said nothing has change on Openssl and code wise only thing that has happen is the patch the openssl version citing HEARTBLEED vulnerability
Mind you we just patch the openssl version but didnt recompile the RUBY could that be a issue for this
Ruby in question is ruby 1.9.3p327
Net::HTTP library is version httparty-0.13.0
NOTE: - As a solution I didn't except to have VERIFY_NONE options in OPENSSL
It's hard to be sure without knowing host you are connecting too, but I guess that they simply changed the certificate at the servers end. The problem might be, that your script does not support SNI (server name indication, e.g. multiple host names and certificates behind the same IP), but the server providers now changed the default certificate for this site (the one which is used if client does not support SNI).
But like I said, it's hard to be sure with this lack of details in the question.

(Ruby) Getting Net::SMTP working with Gmail...?

Does anyone have any quality (and up-to-date) information regarding sending mail via Gmail using Ruby's Net::SMTP? I've seen several examples -- most dating from 2007 to mid-2008 and none of them work for me. I need more current examples that use the most recent 1.8.7 release. I'd also appreciate if the documentation didn't only cover simple examples that no one ever really uses.
Currently I'm receiving an error:
SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: unknown protocol
I'm not terribly familiar with SSL as regards the SMTP protocol, so this is all lost on me. Unfortunately the Net::SMTP documentation only covers the bases and doesn't provide a full list of the various potential OpenSSL::SSL contexts either so I can't try various ones.
Anyway, if anyone has any quality info on getting this to work with Gmail it would be most appreciated.
Best.
Actually the below works for gmail without a plugin or a gem, at least with Ruby 1.9.1p376, but good luck finding documentation that'll tell you so:
require 'net/smtp'
msg = "Subject: Hi There!\n\nThis works, and this part is in the body."
smtp = Net::SMTP.new 'smtp.gmail.com', 587
smtp.enable_starttls
smtp.start(YourDomain, YourAccountName, YourPassword, :login) do
smtp.send_message(msg, FromAddress, ToAddress)
end
YourAccountName looks like you#example.com & YourDomain can probably be anything you like, but I use the actual domain name.
I actually just got this working. Wrote a quick script to test it.
I was getting a different error than you were (requiring STARTTLS), I also found I had to use port 587 instead of 465.
I found the trick to get it working in a Rails plugin I found. (agilewebdevelopment.com/plugins/net_smtp_tls_support)
if you 'eval' this file (it adds tls support to the standard Net::SMTP library):
http://happiness-is-slavery.net/wp-content/rails-plugins/smtp_add_tls_support/lib/smtp_add_tls_support.rb
then run 'Net::SMTP.enable_tls()'
everything seems to work fine.
Here's my code:
require 'rubygems'
require 'net/smtp'
eval File.read("smtp_tls.rb")
Net::SMTP.enable_tls()
FROM_EMAIL = "REMOVED"
PASSWORD = "REMOVED"
TO_EMAIL = "REMOVED"
msgstr = <<END_OF_MESSAGE
From: Your Name <#{FROM_EMAIL}>
To: my phone <#{TO_EMAIL}>
Subject: text message
Date: Sat, 23 Jun 2001 16:26:43 +0900
Message-Id: <unique.message.id.string#example.com>
This is a test message.
END_OF_MESSAGE
Net::SMTP.start('smtp.gmail.com', 587, 'gmail.com',
FROM_EMAIL, PASSWORD, :plain) do |smtp|
smtp.send_message msgstr, FROM_EMAIL, TO_EMAIL
end
obviously, i downloaded the above mentioned file to the same directory and named it 'smtp_tls.rb'
Hope this helps!
Are you connecting to smtp.gmail.com port 465 I am assuming?
.
openssl s_client -connect smtp.gmail.com:587
CONNECTED(00000003)
8298:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:601:
The error looks very similar to yours. The following command does work:
openssl s_client -starttls smtp -connect smtp.gmail.com:587
So I think what is happening is that you do not have STARTTLS support enabled. I am not sure how to do it in ruby buy what I did find out is that the action_mailer_tls plugin allows this by patching Net::SMTP. As of Ruby 1.8.7, Net::SMTP has this support built-in.
If you are using Ruby < 1.8.7 here is the patch.
You also need to have two newline characters between the subject and the body of the message. Otherwise, only the subject will be transmitted, and the message body will be left blank.
msg = "Subject: My Subject Goes Here\n\nMy message goes here"

Resources