I'm trying to develop a XMPP "Proxy" which will be in the middle of a standard Jabber communication.
The schema will be something like this:
Pidgin ---> Proxy <--- eJabberD
|
v
Console
The purpose of this proxy is to log all the stanzas which go over the wire. IMHO, this is very convenient when you're developing XMPP based solutions.
I'm doing this with EventMachine and Ruby, and the main problem is to know how to decypher the traffic after the TLS/SASL handshake.
Before the starttls, all works perfectly, the server and client can talk between them, but when the tls handshake begins, although it works, it is impossible to dump the clear content as all of the traffic is encrypted.
I'm not an expert in TLS/SASL thing, so I don't know which is the best approach to do this. I think one way to achieve this, should be to grab the certificate in the handshake and use it to decypher the content as it goes throught the proxy.
Thanks!
If you could do what you say (grab the certificate on the wire and use it to decrypt), then TLS would be pretty worthless. This is one of the primary attacks TLS exists to prevent.
If the server will allow it, just don't send starttls. This is not required by spec. If starttls is required by your server, then you can configure it to use a null cipher, which will leave the traffic unencrypted. Not all servers will support that of course.
You can man-in-the-middle the starttls. Respond with your own tunnel to the client, and send a separate starttls negotiation to the server. This should generate certificate warnings on the client, but since you control the client you can tell it to accept the certificate anyway.
If you control the server, you can use the private key from it to decrypt the traffic. I'm not aware with any off-the-shelf code to do that easily, but it's writable.
Related
I am currently working on GO app and I wanted to know how can I detect someone using reverse proxy apps such as Charles, Wireshark and Fiddler.
I tried to read about SSL-Pinning but I couldn't find anything useful.
Charles and Fiddler are no reverse proxies but forward proxies. TLS interception in such proxies can be detected based on the certificate returned - it will not be the original server certificate but one created by the server. And certificate validation will fail unless the proxy is specifically trusted by importing the proxies CA certificate. SSL pinning helps too since it expects the server to use a specific certificate or a specific CA - which the SSL intercepting proxy cannot provide.
Wireshark is no kind of proxy but passive packet capturing, i.e. it makes no changes to the traffic. Because it is passive it cannot be detected from inside the client application and SSL pinning would not help either. Wireshark can not decrypt TLS traffic though by its own - it would need to have the connection specific secrets from either inside the client application or inside the server application.
I've read a lot of related topics in the net, but I still don't have an answer to my question.
Is it possible to implement flow described below?
Proxy receive request.
If request is encrypted and proxy cert is trusted then intercept.
If request is not encrypted, then intercept.
If request is encrypted and proxy cert is NOT trusted then pass it through without interception.
This behaviour should be default for all traffic going through the proxy.
It'd be also really nice to be able to get all possible info for passing encrypted requests (src and dst ip addresses etc.). Basically the same info which I can get with fiddler.
Not really. The main problem is that mitmproxy can not know if proxy cert is trusted by the client or not.
In the SSL/TLS protocol client starts with the CLIENT_HELLO and in response the server (in this case motmproxy) sends back the SERVER_HELLO message containing the generated server certificate.
The client now checks if the received server certificate is trusted. If not the connection is terminated. As far as I know the SSL/TLS spec does not define how to do so. Sems clients end back an SSL_ALERT message, other simply drop the connection, and a third group continues the SSL/TLS handshake but have certain internal values set in a way that always let the handshake fail.
There is a mitmproxy script that tries to identify connections that were not successful and then if the client asks for the same domain a second time bypasses interception.
Of course this requires that the client resends requests which is not always the case.
https://github.com/sociam/x-ray/blob/master/mitmproxy/examples/tls_passthrough.py
I have a doubt on HTTPS. One of my seniors told me that Https does not actually use SSL/TLS, but only their the encryption algorithms.
He says that the handshaking process with certificates are done in the transport layer, but the security key encryption for actual payload is done in the Application layer.
And he also said that Https can actually be considered as a presentation layer protocol.
Is he correct?
HTTPS is specified in RFC 2818: "HTTP Over TLS".
Although the specification is about TLS (because it's an IETF specification, and IETF only uses "TLS"), it's in fact about SSL or TLS, depending on the version of SSL/TLS used (see difference between SSL and TLS).
So yes, HTTPS does use SSL/TLS. As the RFC says:
Conceptually, HTTP/TLS is very simple. Simply use HTTP over TLS
precisely as you would use HTTP over TCP.
Essentially, the encryption keys are negotiated during the SSL/TLS handshake, and the HTTP exchange isn't aware of those keys.
If you're not convinced, look at some browser traffic using Wireshark. All you'll see is some SSL/TLS traffic (the HTTP exchanged being encrypted by it).
If you want to analyse some traffic, you can set up your own server and use its private key to look at the normal HTTP exchange on top of SSL/TLS using Wireshark too, as described in the Wireshark SSL page. (You'll need to disable EDH cipher suites, because they provide perfect forward secrecy, which prevent you from deciphering sniffed traffic even if you have the server private key.)
This page also has some example HTTPS data you can download and look at with Wireshark, without having to install your own server.
From the browser point of view, you can also look at the traffic as reported by the developer tools (Firebug and so on) when using HTTPS, you'll just see plain HTTP traffic, since the SSL/TLS layer is taken care of by the SSL/TLS library underneath.
I wouldn't stress too much about the OSI layers in general. They look good in theoretical networking classes, but are in fact difficult to apply to the TCP/IP stack (see comparison and "layering considered harmful"), especially when you throw SSL/TLS into it, which is precisely designed to be an invisible layer as far as the application layer is concerned (SSL/TLS usually sits on top of TCP, like any other application protocol on top of TCP, but it sits under any application protocol it protects).
Your senior should be your junior. He doesn't know what he is talking about. Avoid him in future. HTTPS is simply HTTP over SSL, with the minor addition of hostname checking to ensure that the hostname in the certificate agrees with the site you connected to.
Specifically:
One of my seniors told me that Https does not actually use SSL/TLS, but only their the encryption algorithms
Wrong.
He says that the handshaking process with certificates are done in the transport layer
Wrong for both SSL and HTTPS. It is done by the application, typically in a library such as OpenSSL or JSSE. The transport layer only does the TCP part, none of the SSL part.
but the security key encryption for actual payload is done in the Application layer.
Wrong again: payload encryption is done in the application layer for both SSL and HTTPS.
And he also said that Https can actually be considered as a presentation layer protocol.
It is an application layer protocol. There are very few examples of presentation layer protocols (XDR and TN3270 comes to mind). This isn't one of them. But I agree with Bruno. The OSI layers are not applicable to TCP/IP, which has its own layer model, and, since the unlamented demise of the OSI protocol effort, they aren't applicable to anything else either.
From Wikipedia:
HTTP operates at the highest layer of the OSI Model, the Application layer; but the security protocol operates at a lower sublayer, encrypting an HTTP message prior to transmission and decrypting a message upon arrival.
https://en.wikipedia.org/wiki/Https#Network_layers
I have learned a little about https, but was not clear why it is needed.
What if I encrypt the data using the most powerful algorithms like RSA instead of sending through a HTTPS zone? Can someone explain with a few reasons why we need https?
On the Wikipedia article it says
Technically, it is not a protocol in itself; rather, it is the result
of simply layering the Hypertext Transfer Protocol (HTTP) on top of
the SSL/TLS protocol
On the SSL/TLS article on Wikipeida,
TLS and SSL encrypt the segments of network connections at the
Application Layer for the Transport Layer, using asymmetric
cryptography for key exchange, symmetric encryption for
confidentiality, and message authentication codes for message
integrity.
So the key exchange does use asymmetric cryptography and RSA is an asymmetric cryptography algorithm.
After key exchange has been performed in a secure manner further communication can be done through symmetric cryptographic algorithms. The reasoning behind using both symmetric and asymmetric algorithms can be found here.
What if I encrypt the data using the most powerful algorithms like RSA
instead of sending through a HTTPS zone?
You will have to implement all by yourself, i.e. reinvent the wheel. HTTPS is by default supported in every browser.
Can someone explain with a few reasons why we need https?
Secure communication that is widely supported. If you have in the middle of communication someone with sniffer tool like Wireshark, he/she will be able to see all packets that you and your peer exchange. Try to catch the HTTPS communication, you won't be able to see anything meaningful in the body of the request.
You could indeed encrypt the data by yourself, but you will face a big problem: The encrypting code must be available on the server as well as on the client (normally the browser).
Implementing the encryption on the server can be done securely. On the client side you can either install a software (plugin), or you can send JavaScript to the client. The problem is: how do you get the encrypting code to the client? Everybody evesdropping, will get the javascript code as well, so he can do the same things as the client will be able to do.
Instead of forcing the user to install a plugin, you can use the built-in support for SSL, every browser understands this protocol already. You could think of it, as an already installed plugin for encryption.
I've seen several examples of writing an HTTP proxy in Ruby, e.g. this gist by Torsten Becker, but how would I extend it to handle HTTPS, aka for a "man in the middle" SSL proxy?
I'm looking for a simple source code framework which I can extend for my own logging and testing needs.
update
I already use Charles, a nifty HTTPS proxy app similar to Fiddler and it is essentially what I want except that it's packaged up in an app. I want to write my own because I have specific needs for filtering and presentation.
update II
Having poked around, I understand the terminology a little better. I'm NOT after a full "Man in the Middle" SSL proxy. Instead, it will run locally on my machine and so I can honor whatever SSL cert it offers. However, I need to see the decrypted contents of packets of my requests and the decrypted contents of the responses.
Just for background information, a normal HTTP proxy handles HTTPS requests via the CONNECT method: it reads the host name and port, establishes a TCP connection to this target server on this port, returns 200 OK and then merely tunnels that TCP connection to the initial client (the fact that SSL/TLS is exchanged on top of that TCP connection is barely relevant).
This is what the do_CONNECT method if WEBrick::HTTPProxyServer.
If you want a MITM proxy, i.e. if you want to be able to look inside the SSL/TLS traffic, you can certainly use WEBrick::HTTPProxyServer, but you'll need to change do_CONNECT completely:
Firstly, your proxy server will need to embed a mini CA, capable of generating certificates on the fly (failing that, you might be able to use self-signed certificates, if you're willing to bypass warning messages in the browser). You would then import that CA certificate into the browser.
When you get the CONNECT request, you'll need to generate a certificate valid for that host name (preferable with a Suject Alt. Name for that host name, or in the Subject DN's Common Name), and upgrade the socket into an SSL/TLS server socket (using that certificate). If the browser accepts to trust that certificate, what you get from thereon on this SSL/TLS socket is the plain text traffic.
You would then have to handle the requests (get the request line, headers and entity) and take it to use it via a normal HTTPS client library. You might be able to send that traffic to a second instance of WEBrick::HTTPProxyServer, but it would have to be tweaked to make outgoing HTTPS requests instead of plain HTTP requests.
Webrick can proxy ssl:
require 'webrick'
require 'webrick/httpproxy'
WEBrick::HTTPProxyServer.new(:Port => 8080).start
from my experience HTTPS is nowhere near "simple". Do you need a proxy that would catch traffic from your own machine? There are several applications, like Fiddler. Or google for alternatives. Comes with everything you need to debug the web traffic.
That blog is no way to write a proxy. It's very easy: you just accept a connection, read one line which tells you what to connect to, attempt the upstream connection, if it fails send the appropriate response and close the socket, otherwise just start copying bytes in both directions, simultaneously, until EOS has occurred in both directions. The only difference HTTPS makes is that you have to speak SSL instead of plaintext.