I am trying to convert this command to ruby code using OpenSSL:
openssl s_client -verify_hostname www.example.com -connect example.com:443
I've pieced together that openssl has a verify_hostname method but I dont think that I am using this properly. Contextually I have a problem verifying a domain's SAN field. For a given domain I get wrong results. I provide a domain that is clearly a name mismatch buy I get a verify_result code of 0 which is ok. The command above gives me the right information but I can't seem to get this to translate in ruby. Is there an online tool that can convert this for me?
-verify_hostname is a parameter and s_client is a method name in your command. Check the output of openssl s_client --help to get more information. OpenSSL gem has a verify_hostname method but its source code looks like it's only checking if provided strings are correct, it does not call the provided host.
All http clients will do the SSL verification for you by default (unless you disable it):
require "net/http"
begin
Net::HTTP.get(URI("https://expired.badssl.com"))
rescue OpenSSL::SSL::SSLError
puts "bad ssl cert"
end
If you'd like to to the exact thing as your command does you'd probably need to download host's certificate first and then instantiate it with OpenSSL::X509::Certificate.new verify it with the check_validity method.
Related
In trying to develop a tool for that would use IMAP to access Gmail, I'm running into difficulties with even this simple start-up code:
require 'net/imap'
imap = Net::IMAP.new('imap.gmail.com', ssl: true)
Running that, it fails as follows (Note: lightly edited for easier display):
Traceback (most recent call last):
5: from test-imap:2:in `<main>'
4: from test-imap:2:in `new'
3: from /.../.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/net/imap.rb:1092:in `initialize'
2: from /.../.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/net/imap.rb:1533:in `start_tls_session'
1: from /.../.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/net/protocol.rb:44:in `ssl_socket_connect'
/.../.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/net/protocol.rb:44:in
`connect_nonblock': SSL_connect returned=1 errno=0 state=error:
certificate verify failed (self signed certificate) (OpenSSL::SSL::SSLError)
Searching around, I found some somewhat-similar issues, e.g. imap-backup issue #57, ruby/openssl issue #238 (still open, as of this writing closed after sharing the answer below), and rbenv/ruby-build issue #380... but nothing on SO.
Piecing info from the above resources together, I came up with this command to try:
openssl s_client -connect imap.gmail.com:993 \
-CAfile $(ruby -ropenssl -e 'puts OpenSSL::X509::DEFAULT_CERT_FILE') \
< /dev/null > /dev/null
Which reports:
depth=2 /OU=GlobalSign Root CA - R2/O=GlobalSign/CN=GlobalSign
verify return:1
depth=1 /C=US/O=Google Trust Services/CN=Google Internet Authority G3
verify return:1
depth=0 /C=US/ST=California/L=Mountain View/O=Google LLC/CN=imap.gmail.com
verify return:1
DONE
So, it seems that the SSL certificate does actually verify as OK in that way (as one might expect).
I did find some instructions for using net/imap with SSL with host-checking disabled, which works... but I'd really prefer not to do that.
I also found a non-IMAP gmail interface, but my intention is a tool that could also be used with other IMAP providers, so I'm specifically aiming to stick to IMAP here. So:
How can I get net/imap to connect successfully using SSL, and still validate the certificate (given that it is, in fact, valid)?
Ideal: upgrade to Ruby 2.6.3 or later
Upgrading to Ruby version 2.6.3 or later should fix things, as ruby pull #2077, which provides a fix for related issue #15594, was incorporated into the 2.6.3 change list. With 2.6.3, the original test snippet in the question now works.
If you can't upgrade:
That said, if upgrading is impractical for some reason, the commit that fixed imap-backup issue #57 offers a workaround-type solution. Instead of the following:
imap = Net::IMAP.new('imap.gmail.com', ssl: true)
Try this:
imap = Net::IMAP.new('imap.gmail.com', ssl: {ssl_version: :TLSv1_2})
That seems to make things work, even in Ruby 2.6.0.
In Ruby 2.4, I had to use this patch:
require 'net/imap'
class Net::IMAP
module UseSNI
def start_tls_session(*)
#sock.instance_variable_set(:#hostname, #host)
super
end
end
prepend UseSNI
end
class OpenSSL::SSL::SSLSocket
module UseSNI
def connect(*)
self.hostname = io.instance_variable_get(:#hostname)
super
end
end
prepend UseSNI
end
Another of of these questions, I know this question has been asked (and answered) a lot on StackOverflow, but I can't get any of those to work for me and I also have a few questions I would like to learn.
Here is my error:
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server hello A
To start, here is my system settings.
I am on OSX El Capitan version 10.11.6
openssl version
OpenSSL 0.9.8zh 14 Jan 2016
which openssl
/usr/bin/openssl
ruby -v
ruby 2.1.6p336 (2015-04-13 revision 50298) [x86_64-darwin14.0]
rbenv -v
rbenv 0.4.0
My questions are these:
1) Does this error mean that a certificate was sent back to me, and then my OpenSSL version was unable to verify it? Did the other server have a chance to read mine, or even see it yet? Is there a way to dig into this request using Net::HTTP and inspect this other than opening up a program like Wireshark? Once I call net::HTTP.new.request(request) I seem to lose control and it just errors.
2) Did I even successfully talk to the other server, and it denied me?
3) At what point in the request am I in when I get this message?
and most of all
4) What are my options to get past this point
4a. So far i'm seeing a possible brew solution, but I haven't been able to get brew to link
4b. I could manually install Mozilla's CA (Or any other CA) into my Mac OSX Keychain
4c. Can I attach the file using the request.ca_file = "file" as I tried in my code? (see below)
4d. Is there any other solutions / best and most politically correct version?
5) Am I going to have this issue when I deploy to Heroku?
From what i'm reading, this is an issue of my OS not containing the correct CA files. the ca_file part is due to my first attempts to add the correct ca_file to my requests. I'm guessing I don't need that. I am using a Proxy with heroku because this API requires a static IP.
Here is my generic code
cert = File.read(File.join(Rails.root, 'ssl', 'test_env', 'their_test_cert.der'))
ca_file = File.read(File.join(Rails.root, 'ssl', 'test_env', 'Class3PublicPrimaryCA.der'))
uri = URI("https://xml.theirtestenv.com/api/receive")
headers = {
'x-IK-Version' => 'IKR/V4.00',
}
proxy_host = "myproxyhose"
proxy_port = "1234"
proxy_user = "myproxyuser"
proxy_pass = "myproxypass"
proxy_request = Net::HTTP.new(uri.hostname, '443', proxy_host, proxy_port, proxy_user, proxy_pass)
# http.key = OpenSSL::PKey::RSA.new(rsa_key)
proxy_request.use_ssl = true
proxy_request.cert = OpenSSL::X509::Certificate.new(cert)
proxy_request.ca_file = ca_file
proxy_request.verify_mode = OpenSSL::SSL::VERIFY_PEER
# proxy_request.ssl_version = :SSLv3
# This doesn't seem to matter whether I put this or not...
# Tried variations of these...
# proxy_request.ssl_version = :TLSv1
# proxy_request.ciphers = ['DES-CBC3-SHA']
post_request = Net::HTTP::Post.new(uri, headers)
post_request.content_type = "multipart/related"
response = proxy_request.request(post_request)
puts response.inspect
Also, i've noticed no matter what proxy_requst.ssl_version I put, my error always specifies SSLv2/v3, does that mean on their end they are requiring that version?
Sorry for all the questions. Thanks in advance
It's been awhile, but I just wanted to post that this was a couple of issues, the certificates I was passing were not the correct ones they were for the wrong environment. Once the correct certificates were passed this started working, though I never got the SSL Version questions quite figured out.
All of sudden today morning my HTTP client (HTTParty) threw an error OpenSSL::SSL::SSLError: hostname does not match the server certificate
Firstly I'm not able to understand which so today we have been make that api call almost all day number times from past 2 years without any issue
Secondly I don't understand how do I solve it since it internal to HTTParty
The only thing I know of is that I cant set SSL_CERT_FILE in ENV but as said I already have ROOT CA listed in my /etc/ssl/certs (SSL_CERT_DIR)
Here my output
irb(main):001:0> require "openssl"
=> true
irb(main):002:0> puts OpenSSL::OPENSSL_VERSION
OpenSSL 1.0.1 14 Mar 2012
=> nil
irb(main):003:0> puts "SSL_CERT_FILE: %s" % OpenSSL::X509::DEFAULT_CERT_FILE
SSL_CERT_FILE: /usr/lib/ssl/cert.pem
=> nil
irb(main):004:0> puts "SSL_CERT_DIR: %s" % OpenSSL::X509::DEFAULT_CERT_DIR
SSL_CERT_DIR: /usr/lib/ssl/certs
Lastly as said nothing has change on Openssl and code wise only thing that has happen is the patch the openssl version citing HEARTBLEED vulnerability
Mind you we just patch the openssl version but didnt recompile the RUBY could that be a issue for this
Ruby in question is ruby 1.9.3p327
Net::HTTP library is version httparty-0.13.0
NOTE: - As a solution I didn't except to have VERIFY_NONE options in OPENSSL
It's hard to be sure without knowing host you are connecting too, but I guess that they simply changed the certificate at the servers end. The problem might be, that your script does not support SNI (server name indication, e.g. multiple host names and certificates behind the same IP), but the server providers now changed the default certificate for this site (the one which is used if client does not support SNI).
But like I said, it's hard to be sure with this lack of details in the question.
I am just trying to get my head around SSL.
I have set up a Jetty server on my localhost, and generated my own certificate using Keytool.
Now when I go to https://localhost:8443/ I get the can't trust this certificate error.
I use
keytool -export -alias pongus -keystore keystore -file certfile.cer
To create the certificate which I think is what the client needs to authenticate with the server. (This is where I could be very wrong!)
I have the following ruby code :
require 'net/https'
require 'openssl'
require 'open-uri'
puts 'yay' if File.exists?('certfile.cer')
uri = URI.parse("https://localhost:8443/")
http_session = Net::HTTP.new(uri.host, uri.port)
http_session.use_ssl = true
http_session.verify_mode = OpenSSL::SSL::VERIFY_PEER
http_session.ca_file = 'certfile.cer'
res = http_session.start do |http|
# do some requests here
http.get('/')
end
This does print 'yay', so the certfile.cer file does exist.
But I get the errors
/Applications/NetBeans/NetBeans 6.8.app/Contents/Resources/NetBeans/ruby2/jruby-1.4.0/lib/ruby/1.8/net/http.rb:586 warning: can't set verify locations
/Applications/NetBeans/NetBeans 6.8.app/Contents/Resources/NetBeans/ruby2/jruby-1.4.0/lib/ruby/1.8/net/http.rb:586:in `connect': certificate verify failed (OpenSSL::SSL::SSLError)
Any ideas what I am doing wrong?
EDIT
I want to get it so I guarantee that I am connecting to the right server, and the server can guarantee that it is me connecting to it, without any tampering in between. I am developing both the server and the client.
Your client needs access to its
private key.
You dont need the private key for server certificate verification. All you need is the certificate itself which contains the public key. Only the server has the private key. Well described here http://www.helpbytes.co.uk/https.php and here http://www.verisign.com/ssl/ssl-information-center/how-ssl-security-works/
My recommendation is simple. Check your certificate is correct.
openssl x509 -text -in mycert.crt
Also if you have access to the server you can explicitely validate if the certificate and key (used in httpd configuration) are correct (matches): http://kb.wisc.edu/middleware/page.php?id=4064 Please note this is explicit check ran on server. Never give out the private key. This check can be done only by the administrator to verify if the httpd was not misconfigured.
(openssl x509 -noout -modulus -in server.pem | openssl md5 ;\
openssl rsa -noout -modulus -in server.key | openssl md5) | uniq
You can also debug the SSL certificate communication using standard openssl command. Issue this command then wait few seconds and then type QUIT and hit enter. You will see the certificate the server sends out.
openssl s_client -connect your.server.com:443
Also try to import the certificate to your browser and access the URL resource. Browser is able to validate it by clicking on https (Firefox and Chrome). Then you will see the certificate itself and validity information.
All the above was all about the server certificate. This is only one part of the problem. "I am connecting to the right server, and the server can guarantee that it is me connecting to it" Actully in your code above you only check for the server cert. Now. If you want a client certificate (the second part of your statement) than you need this in Ruby:
File.open( "client_certificate.pem", 'rb' ) { |f| cert = f.read }
File.open( "client_key.pem", 'rb' ) { |f| key = f.read }
http_session.cert = OpenSSL::X509::Certificate.new(cert)
http_session.key = OpenSSL::PKey::RSA.new(key, nil)
This is how client cert should be used in Ruby. If your private key is encrypted with a password just pass it instead nil in the second argument of RSA constructor.
I highly recommend to get server certificate working (your code) and then start with client certificate. Please note you keep your current code (ca_cert, validation constant) and add the above four lines to it.
Hope this helps.
Your client needs access to its private key. The private key is not in the certificate, the certificate only contains the public key. Sorry, I don't know ruby, but a common technique is to bundle the private key and certificate in a single PKCS#12, aka p12, file and supply this to the crypto library.
Change
http_session.verify_mode = OpenSSL::SSL::VERIFY_PEER
to
http_session.verify_mode = OpenSSL::SSL::VERIFY_NONE
Once you do that, the SSL will work properly. I have used this multiple times in my development environments, always works flawlessly.
Does anyone have any quality (and up-to-date) information regarding sending mail via Gmail using Ruby's Net::SMTP? I've seen several examples -- most dating from 2007 to mid-2008 and none of them work for me. I need more current examples that use the most recent 1.8.7 release. I'd also appreciate if the documentation didn't only cover simple examples that no one ever really uses.
Currently I'm receiving an error:
SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: unknown protocol
I'm not terribly familiar with SSL as regards the SMTP protocol, so this is all lost on me. Unfortunately the Net::SMTP documentation only covers the bases and doesn't provide a full list of the various potential OpenSSL::SSL contexts either so I can't try various ones.
Anyway, if anyone has any quality info on getting this to work with Gmail it would be most appreciated.
Best.
Actually the below works for gmail without a plugin or a gem, at least with Ruby 1.9.1p376, but good luck finding documentation that'll tell you so:
require 'net/smtp'
msg = "Subject: Hi There!\n\nThis works, and this part is in the body."
smtp = Net::SMTP.new 'smtp.gmail.com', 587
smtp.enable_starttls
smtp.start(YourDomain, YourAccountName, YourPassword, :login) do
smtp.send_message(msg, FromAddress, ToAddress)
end
YourAccountName looks like you#example.com & YourDomain can probably be anything you like, but I use the actual domain name.
I actually just got this working. Wrote a quick script to test it.
I was getting a different error than you were (requiring STARTTLS), I also found I had to use port 587 instead of 465.
I found the trick to get it working in a Rails plugin I found. (agilewebdevelopment.com/plugins/net_smtp_tls_support)
if you 'eval' this file (it adds tls support to the standard Net::SMTP library):
http://happiness-is-slavery.net/wp-content/rails-plugins/smtp_add_tls_support/lib/smtp_add_tls_support.rb
then run 'Net::SMTP.enable_tls()'
everything seems to work fine.
Here's my code:
require 'rubygems'
require 'net/smtp'
eval File.read("smtp_tls.rb")
Net::SMTP.enable_tls()
FROM_EMAIL = "REMOVED"
PASSWORD = "REMOVED"
TO_EMAIL = "REMOVED"
msgstr = <<END_OF_MESSAGE
From: Your Name <#{FROM_EMAIL}>
To: my phone <#{TO_EMAIL}>
Subject: text message
Date: Sat, 23 Jun 2001 16:26:43 +0900
Message-Id: <unique.message.id.string#example.com>
This is a test message.
END_OF_MESSAGE
Net::SMTP.start('smtp.gmail.com', 587, 'gmail.com',
FROM_EMAIL, PASSWORD, :plain) do |smtp|
smtp.send_message msgstr, FROM_EMAIL, TO_EMAIL
end
obviously, i downloaded the above mentioned file to the same directory and named it 'smtp_tls.rb'
Hope this helps!
Are you connecting to smtp.gmail.com port 465 I am assuming?
.
openssl s_client -connect smtp.gmail.com:587
CONNECTED(00000003)
8298:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:601:
The error looks very similar to yours. The following command does work:
openssl s_client -starttls smtp -connect smtp.gmail.com:587
So I think what is happening is that you do not have STARTTLS support enabled. I am not sure how to do it in ruby buy what I did find out is that the action_mailer_tls plugin allows this by patching Net::SMTP. As of Ruby 1.8.7, Net::SMTP has this support built-in.
If you are using Ruby < 1.8.7 here is the patch.
You also need to have two newline characters between the subject and the body of the message. Otherwise, only the subject will be transmitted, and the message body will be left blank.
msg = "Subject: My Subject Goes Here\n\nMy message goes here"