We wrote a puppet function which needs to do a http get request over tls to a server. The http request is done directly from the puppet master. The remote server needs apparently more recents ciphers than provided by our function (we use the ruby Net/Http library). Consequently, the remote server immedialty close the connection (Client Hello, [ACK], [FIN, ACK] )
def http_client
uri = URI("https://#{#config['server']}")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE unless #config['vmssc_tls_verify']
http
end
We tried to CuRL the same API from the puppet master, with success, it appears the openssl version used by CuRL is different than the one used by puppet.
The difference we found was that in curl we had like 32 ciphers in SHA256 but with the Net/Http library used by puppet we only had 11 ciphers in SHA1.
Puppet server is running with jruby, so we tried to add java cryptography extension in openJDK but without success.
We also observe that the ciphers listed in the /etc/puppetlabs/puppetserver/conf.d/puppetserver.conf file are not honored (hence we got the list of the ciphers used by the client in our tcpdump) :
# settings related to HTTP client requests made by Puppet Server
http-client: {
# A list of acceptable protocols for making HTTP requests
ssl-protocols: [
TLSv1.2,
]
# A list of acceptable cipher suites for making HTTP requests
cipher-suites: [
TLS_RSA_WITH_AES_256_CBC_SHA256,
TLS_RSA_WITH_AES_256_CBC_SHA,
TLS_RSA_WITH_AES_128_CBC_SHA256,
TLS_RSA_WITH_AES_128_CBC_SHA,
]
We tried to upgrade the version of openJDK to an oracle JDK but without success.
We're running on a puppetserver version 2.7.0 with Jruby and openJDK 8 with a TLS_Version 1.2.
Also when the puppetserver is started in native ruby everything works fine, we've got the goods ciphers. We don't want to stay with that configuration for the performance issue.
Related
I have an old server that only accepts SSLv3 and SSLv3. I want to set the ssl version when I make the request.
http = Net::HTTP.new(#request.uri.host, #request.uri.port)
http.use_ssl = #request.uri.scheme == 'https'
http.ssl_version = :SSLv3
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
leads to
[2020-09-10T05:37:14.452819 #254200] FATAL -- :
OpenSSL::SSL::SSLError (SSL_CTX_set_min_proto_version):
I tried other combinations but failed.
http.ssl_version = :SSLv3
http.ssl_version = "SSLv3"
How can I set the specific ssl version? I am using Ruby 2.7.1 and Rails 6. Thanks.
In most current versions of OpenSSL, the support for SSLv3 is statically disabled during compile time since the protocol is widely considered as insecure. As such, there is often no way to still use SSLv3 on those OpenSSL versions.
If SSLv3 is a hard requirement, you will likely have to first compile your own OpenSSL version with suitable configuration options, followed by the compilation of your Ruby against your custom OpenSSL.
Preferably, you should either update the server to support newer more secure protocols (such as TLSv1.2 or TLSv1.3) or ask the operator of the server to do so.
After a long day I managed to get to the bottom of what I believe is a SSL/TLS cipher negotiation issue with a server that doesn't support the latest and greatest versions.
Stack:
Ubuntu 14.04 fully patched
OpenSSL 1.0.1f 6 Jan 2014
irb 0.9.6(09/06/30)
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux] (using rbenv)
After 60 seconds the snippet below gives me an error:
require 'net/http'
require 'openssl'
uri = URI.parse('https://some_old_server/my/path')
http = Net::HTTP.new('some_old_server', 443)
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
http.use_ssl = true
response = http.request(Net::HTTP::Get.new(uri.request_uri))
Errno::ECONNRESET: Connection reset by peer - SSL_connect
If I add this to the code, it works:
(...)
http.ciphers = ['AES128-SHA']
(...)
=> #<Net::HTTPOK 200 OK readbody=true>
This isn't a ruby-specific issue but ideally there's a ruby solution. I can't lock the ciphers to 'AES128-SHA' because the same code handles a number of sites that may or may not support this cipher.
Has anyone ever come across this and found a generic solution?
EDIT: this seems to be caused by the "TLS hang bug" and was fixed in openssl 1.0.1g.
New question: is there a work-around that can be implemented on the ruby side?
More information.
A Gentoo server running OpenSSL 1.0.1j 15 Oct 2014 doesn't have this issue. I tried installing 1.0.1j on the Ubuntu 14.04 server, recompiling ruby (rbenv install 2.2.2) and the error was still present.
I've tried to monkey patch ext/openssl but that didn't work.
Using the whole cipher list from the link above doesn't work. However, using a small subset does work:
require 'net/http'
require 'openssl'
uri = URI.parse('https://some_old_server/my/path')
http = Net::HTTP.new('some_old_server', 443)
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
http.use_ssl = true
http.ciphers = %w{
AES128-GCM-SHA256
AES256-GCM-SHA384
AES128-SHA256
AES256-SHA256
AES128-SHA
AES256-SHA
ECDHE-ECDSA-RC4-SHA
ECDHE-RSA-RC4-SHA
RC4-SHA
}.join(":")
response = http.request(Net::HTTP::Get.new(uri.request_uri))
Openssl agrees with ruby (as it should). Running these, on the same system, replicates the issue as I see them in ruby:
openssl s_client -connect some_old_server:443
CONNECTED(00000003)
(...)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 295 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---
Passing the cipher:
openssl s_client -cipher AES128-SHA -connect some_old_server:443
CONNECTED(00000003)
(...)
---
No client certificate CA names sent
---
SSL handshake has read 2721 bytes and written 425 bytes
---
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : AES128-SHA
Session-ID: removed
Session-ID-ctx:
Master-Key: removed
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1454394952
Timeout : 300 (sec)
Verify return code: 20 (unable to get local issuer certificate)
---
I read somewhere to use
http.ssl_options = OpenSSL::SSL::OP_ALL
but ssl_options isn't available in Net::HTTP on ruby 2.2.2.
After spending more time on this than I'd care to admit, my solution was to upgrade from Ubuntu 14.04 to 15.10 which comes with OpenSSL 1.0.2d 9 Jul 2015.
While the TLS negotiation still hangs using the openssl CLI, in Ruby it does not:
require 'net/http'
require 'openssl'
require 'pp'
uri = URI.parse('https://broken_server/my/path')
http = Net::HTTP.new('broken_server', 443)
http.instance_eval {
#ssl_context = OpenSSL::SSL::SSLContext.new
#ssl_context.set_params({:options=>OpenSSL::SSL::OP_ALL})
}
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
http.use_ssl = true
pp response = http.request(Net::HTTP::Get.new(uri.request_uri))
SSL context code above courtesy of #vinhboy.
The CLI equivalent of the above is turned-on with the -bugs option:
openssl s_client -bugs -connect broken_server:443
I've spent the last couple of days looking for a solution to this problem, and though there are quite a few semi-relevant questions, none of the answers show a successful use case for sending a successful HTTP request over Tor to a hidden service using Ruby.
One thing that seems unclear in all the answers is how the Tor browser bundle acts as a proxy for requests.
I've tried multiple avenues of attack, the most recent being an attempt to adapt the PHP code here for sending curl requests to hidden services using Ruby CURL libraries.
This is my code using the Curb gem, which implements some of the libcurl bindings for Ruby:
require 'curb'
url = 'http://am4wuhz3zifexz5u.onion' #a known, functioning hidden service
c = Curl::Easy.new(url) do |curl|
curl.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Firefox/24.0" #the user agent string from the most recent version of the tor browser
curl.verbose = true
end
c.perform
c.inspect
I then tried to run this through socksify, using the IP address and port specified by the Tor browser bundle's proxy settings:
$ socksify_ruby 127.0.0.1 9150 tor-test/tor-test.rb
This produced the following output:
* Adding handle: conn: 0x7fb1fe195e00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fb1fe195e00) send_pipe: 1, recv_pipe: 0
* Could not resolve host: jhiwjjlqpyawmpjx.onion
* Closing connection 0
Every approach I've tried has produced a similar result: the URI for the hidden service cannot be resolved
Can anyone help me out here or point me in the right direction? I feel that since the Tor browser can connect programmatically to hidden services, we all should be able to as well : )
Curl only uses a proxy if you set it up in your "curl"-block.
For example:
c = Curl::Easy.new() do |curl|
curl.proxy_tunnel = true
curl.proxy_type = Curl::CURLPROXY_SOCKS5 # also available and default Curl::CURLPROXY_HTTP
curl.proxy_url = '127.0.0.1:9050' # local tor client/proxy
curl.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Firefox/24.0" #the user agent string from the most recent version of the tor browser
curl.verbose = true
curl.url = url # your example url
curl.perform
curl.inspect
end
Unfortunately curl does not use the proxy for hostname resolution. In other words, I did't find a way to force curl to use the proxy for hostname resolution.
But you can try
#enable socksify debug
Socksify::debug = true
#own try via direct use of socksify and Net::HTTP
uri = URI.parse('http://am4wuhz3zifexz5u.onion/') #a known, functioning hidden service
# some debug stuff - just ignore ;-)
puts uri
puts uri.host
puts uri.port
puts uri.path
res1 = Net::HTTP.SOCKSProxy('127.0.0.1', 9050).start(uri.host, uri.port) do |http|
http.get(uri.path)
end
I have a test environment that uses Ruby to drive a server over an https connection. Since the latest versions of Ruby refuse to connect to an https server with an invalid certificate (see this earlier question of mine) and I would like to start using a newer version of Ruby, I am trying to set up a valid certificate.
I have created a CA certificate to use (there are multiple servers being tested so this seems the easier way), and have successfully used it to sign a new certificate which has been installed on a server and is being used. I have added the CA certificate to the browser store and it (the browser) will now connect to the server without complaint. So I am confident my certificates are valid and set up correctly.
I know that Ruby does not use the same store as the browser. I have used the CA file available here to test connecting to other (public) servers (set using the Net::HTTP#ca_file= method) and this also works.
What I cannot get to work is Ruby connecting to my server using my certificate. I have tried various ways of pointing it at my certificate (including adding my certificate to the file linked above) and it always gives the same error:
SSL_connect SYSCALL returned=5 errno=0 state=SSLv2/v3 read server hello A (OpenSSL::SSL::SSLError)
What do I have to do to convince Ruby to accept my certificate and connect to my server?
The code I am using is:
require 'net/https'
uri = URI.parse("https://hostname/index.html")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
http.ca_file = "My CA cert file"
request = Net::HTTP::Get.new(uri.path)
response = http.request(request)
I'm assuming this is wrong somehow. What I want to know is, what should I do to use my CA certificate?
I assume that your Tomcat doesn't like the protocol version that Ruby tries to negotiate. Ruby uses SSLv23 by default, but I've heard other cases where this was a problem for Java-based web servers. The error message you are getting indicates that the handshake fails while setting up the connection and trying to read the server's response. Try adding either
http.ssl_version = :TLSv1
or
http.ssl_version = :SSLv3
and see if that already helps.
If this does not fix the problem yet, it would be very interesting to see why your server rejects the connection attempt. Try running your Tomcat with -Djavax.net.debug=ssl and please post the relevant parts (connection information, exception stacktrace) as to why the attempt fails.
I am using ruby 1.9.3 and faced the same error while using nokogiri to parse some secure urls.
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: (null)
The above answer provided by emboss is correct but make sure the ssl error generated is this one that is mentioned above. I have followed the same and found a solution like this mentioned below.
uri = URI(url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.ssl_version = :SSLv3
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
response = http.get(url)
now the response is having the correct html parsed for the secured url that is passed to the codes in the url .
Make sure that the your certificate file is in PEM format, not CRT (so the documentation for Net::HTTP in Ruby 1.9.3 says).
Update it looks like the documentation is not up to date, Ruby 1.9.3 will accept any kind of certificate.
I want to create a http2 server using node-http2 module without TLS. My code is as follows:
http2 = require('http2');
const bunyan = require('bunyan');
var log = bunyan.createLogger({name: "HTTP2 server without TLS!"});
var options = {
log: log
}
var server = http2.raw.createServer(options, function(request, response) {
console.log("Receiving HTTP2 request!");
// response.writeHead(200);
response.end('Hello world from HTTP2!');
});
server.listen(8000);
However, it does not work. When connecting to this server from chrome, it shows downloading something. When I closed the server, the downloading is finished with blank file (26 bytes).
Does anyone know what is wrong here? Do I need to configure the browser? Thanks in advance!
Chrome and all other browsers only support HTTP/2 over TLS (h2) and not plain HTTP/2 (h2c). So your browser does not understand what is returned from the server and apparently node-http2 does not send a proper error response when it receives a non-http2 request.
The problem seems not just from the browser. Using [curl] curllink that supports http2 over an http:// URL does not working either. Following is the output from the curl:
$ curl -I --http2 http://54.208.83.136:8000/ -v -k
* Trying 54.208.83.136...
* Connected to 54.208.83.136 (54.208.83.136) port 8000 (#0)
> HEAD / HTTP/1.1
> Host: 54.208.83.136:8000
> User-Agent: curl/7.47.1
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQAAP__
>
As we see from the curl output. It sends http/1.1 Upgrade request with proper headers set as it supposed to do according to the [http2 rfc] rfclink.
On the server side, the logs were very long, so I present here only the content of msg in the relevant three logs.
New incoming HTTP/2 connection
Client connection header prelude does not match
PROTOCOL ERROR, Fatal error, closing connection
So basically the server closed the connection because the client connection header prelude does not match. By checking the code, I figured out the error was originated from the readPrelude function of [endpoint.js] endpointlink. It is a function to read the client header, but I don't know what is wrong in the client header :(.
Thus maybe I can say the node-http2 module does not support http2 over plaintext.
Update: it turns out that I was wrong. The node-http2 module do support http2 over plaintext with direct connecting, it does not support HTTP/2 server with Upgrade from HTTP/1.1. The problem resulted from the client side using Upgrade mechanism to connect to the server not supporting Upgrade. Using nghttp client to connect sever with prior knowledge works as follows.
$ nghttp http://127.0.0.1:8000/
Hello world from HTTP2!
nghttpd server also supports HTTP2 without TLS, even though it does not support HTTP Upgrade.
$ nghttpd -d /Documents/Proxy 8080 --no-tls -v
So I highly suggest to use nghttp when you want to test HTTP2 without TLS.