Self signed TLS cert not accepted only in ruby - ruby

I'm securing an internal web server using stunnel and easyRSA.
I've tried to use the default x509 types in easyRSA but my probes failed because the server cert does not have the clientAuth, I've added a new type of cert which adds the server and client auth
extendedKeyUsage = serverAuth,clientAuth
Stunnel is configured with verify=2 which should also check client certificates
This is my signing certs structure:
Root.CA
| Signer.CA
| | server.crt
| | clientA.crt
| \ clientB.crt
| WebClient.CA
| | server.crt
| | user.alice.crt
| \ user.bob.crt
| RPC.CA
| | server.crt
| | rpcClient1.crt
| | rpcClient2.crt
\ \ rpcClient3.crt
The idea is to use client certificates on webpages and have a central CA that will be trusted everywhere it is needed. This works fine with nginx client certs
But the rpc client in ruby can't verify the server cert. I'm using this code to debug the connection
require 'openssl'
require 'net/http'
host = 127.0.0.1 # Not really but for example
port = 443
ruby = File.join(RbConfig::CONFIG['bindir'], RbConfig::CONFIG['ruby_install_name'])
ruby_version = RUBY_VERSION
if patch = RbConfig::CONFIG['PATCHLEVEL']
ruby_version += "-p#{patch}"
end
puts "%s (%s)" % [ruby, ruby_version]
openssl_dir = OpenSSL::X509::DEFAULT_CERT_AREA
mac_openssl = '/System/Library/OpenSSL' == openssl_dir
puts "%s: %s" % [OpenSSL::OPENSSL_VERSION, openssl_dir]
[OpenSSL::X509::DEFAULT_CERT_DIR_ENV, OpenSSL::X509::DEFAULT_CERT_FILE_ENV].each do |key|
puts "%s=%s" % [key, ENV[key].to_s.inspect]
end
ca_file = ENV[OpenSSL::X509::DEFAULT_CERT_FILE_ENV] || OpenSSL::X509::DEFAULT_CERT_FILE
ca_path = (ENV[OpenSSL::X509::DEFAULT_CERT_DIR_ENV] || OpenSSL::X509::DEFAULT_CERT_DIR).chomp('/')
puts "\nHEAD https://#{host}:#{port}"
http = Net::HTTP.new(host, port)
http.use_ssl = true
# Explicitly setting cert_store like this is not needed in most cases but it
# seems necessary in edge cases such as when using `verify_callback` in some
# combination of Ruby + OpenSSL versions.
http.cert_store = OpenSSL::X509::Store.new
http.cert_store.set_default_paths
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
http.verify_depth = 5
failed_cert = failed_cert_reason = nil
# http.ca_file = File.read('ca.crt')
http.cert = OpenSSL::X509::Certificate.new(File.read('tls.crt'))
http.key = OpenSSL::PKey::RSA.new(File.read('tls.key'))
if mac_openssl
warn "warning: will not be able show failed certificate info on OS X's OpenSSL"
# This drives me absolutely nuts. It seems that on Rubies compiled against OS X's
# system OpenSSL, the mere fact of defining a `verify_callback` makes the
# cert verification fail for requests that would otherwise be successful.
else
http.verify_callback = lambda { |verify_ok, store_context|
if !verify_ok
failed_cert = store_context.current_cert
failed_cert_reason = "%d: %s" % [ store_context.error, store_context.error_string ]
end
verify_ok
}
end
user_agent = "net/http #{ruby_version}"
req = Net::HTTP::Head.new("/?#{rand()}", 'user-agent' => user_agent)
begin
res = http.start { http.request(req) }
abort res.inspect if res.code.to_i >= 500
puts "OK"
rescue Errno::ECONNREFUSED
puts "Error: connection refused"
exit 1
rescue OpenSSL::SSL::SSLError => e
puts "#{e.class}: #{e.message}"
if failed_cert
puts "\nThe server presented a certificate that could not be verified:"
puts " subject: #{failed_cert.subject}"
puts " issuer: #{failed_cert.issuer}"
puts " error code %s" % failed_cert_reason
end
ca_file_missing = !File.exist?(ca_file) && !mac_openssl
ca_path_empty = Dir["#{ca_path}/*"].empty?
if ca_file_missing || ca_path_empty
puts "\nPossible causes:"
puts " `%s' does not exist" % ca_file if ca_file_missing
puts " `%s/' is empty" % ca_path if ca_path_empty
end
exit 1
end
This is the output:
$ ruby client.rb
/Users/user/.rvm/rubies/ruby-2.7.0/bin/ruby (2.7.0-p0)
OpenSSL 1.1.1h 22 Sep 2020: /usr/local/etc/openssl#1.1
SSL_CERT_DIR=""
SSL_CERT_FILE=""
HEAD https://127.0.0.1:443
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)
The server presented a certificate that could not be verified:
subject: /C=US/ST=State/L=Location/O=My Certificates/OU=Security/CN=myCN/emailAddress=signer#cn.app
issuer: /C=US/ST=State/L=Location/O=My Certificates/OU=Security/CN=myCN/emailAddress=signer#cn.app
error code 19: self signed certificate in certificate chain
Possible causes:
`/usr/local/etc/openssl#1.1/certs/' is empty
If I create a cert bundle with cat root.crt rpc.crt > bundle.crt and set the SSL_CET_FILE to it I get:
$ SSL_CERT_FILE=bundle.crt ruby client.rb
/Users/user/.rvm/rubies/ruby-2.7.0/bin/ruby (2.7.0-p0)
OpenSSL 1.1.1h 22 Sep 2020: /usr/local/etc/openssl#1.1
SSL_CERT_DIR=""
SSL_CERT_FILE="/Users/user/easyrsa/certs/rpc/bundle.crt"
HEAD https://127.0.0.1:443
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: certificate verify failed (certificate rejected)
The server presented a certificate that could not be verified:
subject: /C=US/ST=State/L=Location/O=My Certificates/OU=Security/CN=server#rpc/emailAddress=signer#cn.app
issuer: /C=US/ST=State/L=Location/O=My Certificates/OU=Security/CN=rpc#ca/emailAddress=signer#cn.app
error code 0: ok
Possible causes:
`/usr/local/etc/openssl#1.1/certs/' is empty
Which tells me it can connect, but it is not being accepted anyway
I've tried to verify the connection with OpenSSL and I can connect and get a response:
openssl s_client -host 127.0.0.1 -port 443 -state -debug \
-cert tls.crt \
-key key.crt \
-CAfile <(kubectl get secret tls-explorer -n explorer --output="jsonpath={ .data.ca\.crt }" | base64 -d)
# If I then type "GET / HTTP/1.1" it returns me the contents
Also I'm using Paw as a postman replacement, and with it I can connect just fine with any of the rpc client cert
So what could be the problem?

Related

Send email with attachment using openssl s_client in bash script

How to send an email with attachment using openssl s_client command as:
openssl s_client -crlf -quiet -starttls smtp -connect $SMTPHostName:587
using bash script
You can pipe smtp commands into the openssl s_client with
$ openssl s_client -crlf -quiet -starttls smtp -connect smtp.xs4all.nl:587 <<< QUIT
Result:
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
verify return:1
depth=0 CN = smtp.xs4all.nl
verify return:1
250 8BITMIME
221 2.0.0 Bye
Available commands can be shown with 'help'.
The sequence of commands is usually something like this:
helo mail.example.com
data
To: dummy#dummy.nl
From: dummy#example.com
Date: Fri, 15 dec 2021 14:00:00 -0200
Message-ID: <xxxx.xxxx.xx#mail.example.com>
This is a test
.
NB. There may be an authentication needed.
To add an attachment something like the following needs to be added to the email.
Content-Type: image/png;
name="bullet.png"
Content-Transfer-Encoding: x-uuencode
begin 664 bullet.png
MB5!...
<snipped>
'
end
The file bullet.png was encoded with uuencode to ascii and added to the message. You can use other MIME formats.
Commands can vary from mailserver to mailserver depending on what for mailextensions and RFC's they support

How do you decrypt a file created with openssl in ruby?

I created a file using: echo "test" | openssl enc -aes256 -salt -a -k test
Decryption code:
def decrypt_string(b64_text, decryption_key)
encrypted_text = Base64.decode64(b64_text)
_header = encrypted_text[0, 8]
salt = encrypted_text[8, 8]
payload = encrypted_text[16..-1]
decipher = OpenSSL::Cipher.new('aes-256-cbc').decrypt
d_1 = OpenSSL::Digest::MD5.new(decryption_key + salt).digest
d_2 = OpenSSL::Digest::MD5.new(d_1 + decryption_key + salt).digest
decipher.key = (d_1 + d_2)
decipher.iv = OpenSSL::Digest::MD5.new(d_2 + decryption_key + salt).digest
decipher.update(payload) + decipher.final
end
But when I call: decrypt_string('U2FsdGVkX1+5Sar5DYmbDtze7yvHKdq/ZuZIVnkImDc=', 'test')
I get OpenSSL::Cipher::CipherError: bad decrypt
I can not use external gems and do not want to make an OS call. How do you use the built in openssl library?
If I run your code it does not return an OpenSSL::Cipher::CipherError (bad decrypt) error, so I think the b64_text variable is different to the one you posted.
Can you try running the following in your IRB:
str = `echo "test" | openssl enc -aes256 -salt -a -k test`
decrypt_string(str, 'test')
Does the error still persist?

AES 256 CBC encrypt with Ruby with OpenSSL not working

I'm trying to encrypt data with OpenSSL library in Ruby using passphrase.
Ruby code looks like this:
require('openssl')
require('base64')
cipher = OpenSSL::Cipher.new ('AES-256-CBC')
cipher.encrypt
cipher.iv = iv = cipher.random_iv
pwd = 'topsecret'
salt = OpenSSL::Random.random_bytes 8
iter = 10000
key_len = cipher.key_len
digest = OpenSSL::Digest::SHA256.new
key = OpenSSL::PKCS5.pbkdf2_hmac(pwd, salt, iter, key_len, digest)
cipher.key = key
puts "salt=#{salt.unpack('H*')[0]}"
puts "key=#{key.unpack('H*')[0]}"
puts "iv=#{iv.unpack('H*')[0]}"
encrypted = cipher.update 'my data to encrypt'
encrypted << cipher.final
puts "encrypted=#{Base64.strict_encode64(encrypted)}"
# it returns:
# salt=1332e5603cbc018a
# key=11a168cf01556a5ee3e22e049f0e65d3adcd75f39e32c7d19aec32a0ccb40d93
# iv=35a08f2d3e719abbee78a0f4fe47c938
# encrypted=E3Ag6cRL2R+xytgw01i6tKSFpV7s7bKoiiWvPA1FYxM=
Unfortunately, when I try to decrypt this, I get error bad magic number:
$ echo "E3Ag6cRL2R+xytgw01i6tKSFpV7s7bKoiiWvPA1FYxM=" | openssl enc -aes-256-cbc -base64 -d -p -pass pass:topsecret
bad magic number
However, when I try this in terminal by running openssl enc command, it works:
$ echo 'my data' | openssl enc -aes-256-cbc -base64 -p -pass pass:topsecret
salt=8135837A305553F2
key=8B4373ABD786BAC107F4112640E95E920C77C017FCEC18E1BD919CED42F0298E
iv =910637CE50FADF27D944B7A8DD239E6D
U2FsdGVkX1+BNYN6MFVT8oWa5P/oxZFwzMk1DRCSSGg=
$ echo "U2FsdGVkX1+BNYN6MFVT8oWa5P/oxZFwzMk1DRCSSGg=" | openssl enc - aes-256-cbc -d -p -base64 -pass pass:topsecret
salt=8135837A305553F2
key=8B4373ABD786BAC107F4112640E95E920C77C017FCEC18E1BD919CED42F0298E
iv =910637CE50FADF27D944B7A8DD239E6D
my data
I think I tried every possible combination of generating key/IV from passphrase but I get error every time. Is anyone able to spot where is the problem with this way? I've spent entire day on this.

Ruby 1.8.7 and Net::HTTP: Making an SSL GET request with client cert?

I'm trying to fetch a resource via SSL using Net::HTTP. Here is the relevant code fragment:
req = Net::HTTP::Get.new(ContentURI.path)
https = Net::HTTP.new(ContentURI.host, ContentURI.port)
https.use_ssl = true
https.cert = OpenSSL::X509::Certificate.new(#cert_raw)
https.key = OpenSSL::PKey::RSA.new(#cert_key_raw)
https.verify_mode = OpenSSL::SSL::VERIFY_PEER
https.ca_file = File.join(TestDataPath, 'cacert.pem')
resp = https.start { |cx| cx.request(req) }
or with the alternate last line:
resp = https.get(ContentURI.path)
I have verified that the various bits (cert, key, CA cert, etc.) are correct.
The problem is that the cx.request(req) throws an exception:
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server session ticket A
The Apache SSL error log on the server contains the following:
[Tue Jan 24 11:47:26 2012] [debug] ssl_engine_kernel.c(1876): OpenSSL: Loop: SSLv3 read finished A
[Tue Jan 24 11:47:26 2012] [debug] ssl_engine_kernel.c(1905): OpenSSL: Exit: error in SSLv3 write session ticket A
[Tue Jan 24 11:47:26 2012] [debug] ssl_engine_kernel.c(1905): OpenSSL: Exit: error in SSLv3 write session ticket A
[Tue Jan 24 11:47:26 2012] [info] [client 10.11.88.53] SSL handshake interrupted by system [Hint: Stop button pressed in browser?!]
[Tue Jan 24 11:47:26 2012] [info] [client 10.11.88.53] Connection closed to child 0 with abortive shutdown (server _SERVERNAME_:443
The cert, key, and CA cert file work with this SSL host through other tools; I'm just having trouble reproducing that success programatically using Net::HTTP[S].
Thanks to anyone who can identify what I'm doing wrong!
I'd say this is matter of your Apache setup rather than problem with Ruby itself.
Check this out:
$ curl https://palladium.wejn.cz/sslcerttest/
curl: (56) SSL read: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure, errno 0
$ curl https://palladium.wejn.cz/sslcerttest/ -E client.pem
<pre>Yop.</pre>
And then:
#!/usr/bin/ruby
require 'net/http'
require 'net/https'
require 'openssl'
require 'uri'
ContentURI = URI.parse("https://palladium.wejn.cz/sslcerttest/")
#cert_raw = File.read('client.pem')
#cert_key_raw = #cert_raw
TestDataPath = '.'
req = Net::HTTP::Get.new(ContentURI.path)
https = Net::HTTP.new(ContentURI.host, ContentURI.port)
https.use_ssl = true
https.cert = OpenSSL::X509::Certificate.new(#cert_raw)
https.key = OpenSSL::PKey::RSA.new(#cert_key_raw)
https.verify_mode = OpenSSL::SSL::VERIFY_PEER
https.ca_file = File.join(TestDataPath, 'cacert.pem')
resp = https.start { |cx| cx.request(req) }
p resp
p resp.body
results in:
$ ruby a.rb
#<Net::HTTPOK 200 OK readbody=true>
"<pre>Yop.</pre>\n"
$ dpkg -l ruby1.8 | tail -n1 | awk '{print $3}'
1.8.7.302-2squeeze1
$ ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [x86_64-linux]
Of course, the apache config in question is:
<Directory /var/www/sslcerttest/>
SSLVerifyClient require
SSLVerifyDepth 5
SSLCACertificateFile /var/www/sslcerttest/cacert.pem
SSLCACertificatePath /var/www/sslcerttest/
SSLOptions +FakeBasicAuth
SSLRequireSSL
SSLRequire %{SSL_CLIENT_S_DN_O} eq "Wejn s.r.o." and %{SSL_CLIENT_S_DN_OU} eq "Server Certificate"
</Directory>
Maybe posting additional details (apache config to test against) would be of some help when tracking this issue down...

OpenSSL Command Line and Ruby OpenSSL library differ when using same parameters to encrypt text

Note: Using OpenSSL for symmetric encryption of text.
I made a Ruby script to test OpenSSL and I found I'm getting different results. The key, iv, and ciphers are identical, so I would expect the results to be identical. But they are not. Here's my script:
require 'openssl'
require 'base64'
key = "00000000000000000000000000000000"
iv = "00000000000000000000000000000000"
### OPENSSL Command Line ###
puts "*** OpenSSL Command Line ***"
print "Encrypted via Command Line: "
string = `printf %s \"Hello\" | openssl enc -aes-128-cbc -K #{key} -iv #{iv} -base64`
puts string
puts string.inspect
print "Decrypted Data is: "
puts `printf %s \"BC2+AQJ6ZQx0al3GXba+EQ==\n\" | openssl enc -d -aes-128-cbc -K #{key} - iv #{iv} -base64`
puts "\n"
### Ruby OpenSSL Library ###
puts "*** OpenSSL Ruby Library ***"
cipher = OpenSSL::Cipher.new('aes-128-cbc').encrypt
cipher.padding = 1
cipher.key = key
cipher.iv = iv
encrypted_data = cipher.update("Hello")
encrypted_data << cipher.final
encrypted_data = Base64.encode64(encrypted_data)
puts "Encrypted via Ruby is: #{encrypted_data}"
puts encrypted_data.inspect
decipher = OpenSSL::Cipher.new('aes-128-cbc').decrypt
decipher.key = key
decipher.iv = iv
data = decipher.update(Base64.decode64(encrypted_data))
data << decipher.final
puts "Decrypted Data: #{data}"
The results are:
*** OpenSSL Command Line ***
Encrypted via Command Line: BC2+AQJ6ZQx0al3GXba+EQ==
"BC2+AQJ6ZQx0al3GXba+EQ==\n"
Decrypted Data is: Hello
*** OpenSSL Ruby Library ***
Encrypted via Ruby is: ZkeNEgsUXi1J7ps6kCQxdQ==
"ZkeNEgsUXi1J7ps6kCQxdQ==\n"
Decrypted Data: Hello
Just a curious result. Any idea what's causing the data to be different?
Just a guess, without knowing Ruby's OpenSSL interface:
You give the keys and initialization vector to the command line OpenSSL in hexadecimal encoding. E.g. your key and initialization vector are 0x000....
I suppose your Ruby library takes the key and initialization vector as binary data, e.g you are actually passing a key and initialization vector consisting of 0x30303030... (assuming ASCII or anything compatible to it) instead of 0x00000....
Pack them to a binary(Hex) sequence will fix it.
Test on my machine(Mac ox 10.11.1 ruby-2.2.3).
cipher.key = ["#{key}"].pack('H*')
cipher.iv = ["#{iv}"].pack('H*')
ruby Packs
decipher.key = ["#{key}"].pack('H*')
decipher.iv = ["#{iv}"].pack('H*')

Resources