I'm trying to implement tls communication between a browser (using the forge js library) using socket.io as transport and a java application as the TLS server.
The tls traffic is base64 encoded so I cannot use a regular sniffer (like tcdump) to analyze the traffic; I can convert the traffic back to binary but it's still hard to interpret the tls records.
Is there a tool that can make sense of the messages but that does not expect any specific transport for the traffic?
You can certainly use Wireshark. If you're not using a port where SSL/TLS is normally used, you may have to right-click on a packet and choose Decode As... -> SSL.
(This being said, there's little point in implementing TLS within the browser using JavaScript: this is not going to be secure.)
Related
I have a doubt on HTTPS. One of my seniors told me that Https does not actually use SSL/TLS, but only their the encryption algorithms.
He says that the handshaking process with certificates are done in the transport layer, but the security key encryption for actual payload is done in the Application layer.
And he also said that Https can actually be considered as a presentation layer protocol.
Is he correct?
HTTPS is specified in RFC 2818: "HTTP Over TLS".
Although the specification is about TLS (because it's an IETF specification, and IETF only uses "TLS"), it's in fact about SSL or TLS, depending on the version of SSL/TLS used (see difference between SSL and TLS).
So yes, HTTPS does use SSL/TLS. As the RFC says:
Conceptually, HTTP/TLS is very simple. Simply use HTTP over TLS
precisely as you would use HTTP over TCP.
Essentially, the encryption keys are negotiated during the SSL/TLS handshake, and the HTTP exchange isn't aware of those keys.
If you're not convinced, look at some browser traffic using Wireshark. All you'll see is some SSL/TLS traffic (the HTTP exchanged being encrypted by it).
If you want to analyse some traffic, you can set up your own server and use its private key to look at the normal HTTP exchange on top of SSL/TLS using Wireshark too, as described in the Wireshark SSL page. (You'll need to disable EDH cipher suites, because they provide perfect forward secrecy, which prevent you from deciphering sniffed traffic even if you have the server private key.)
This page also has some example HTTPS data you can download and look at with Wireshark, without having to install your own server.
From the browser point of view, you can also look at the traffic as reported by the developer tools (Firebug and so on) when using HTTPS, you'll just see plain HTTP traffic, since the SSL/TLS layer is taken care of by the SSL/TLS library underneath.
I wouldn't stress too much about the OSI layers in general. They look good in theoretical networking classes, but are in fact difficult to apply to the TCP/IP stack (see comparison and "layering considered harmful"), especially when you throw SSL/TLS into it, which is precisely designed to be an invisible layer as far as the application layer is concerned (SSL/TLS usually sits on top of TCP, like any other application protocol on top of TCP, but it sits under any application protocol it protects).
Your senior should be your junior. He doesn't know what he is talking about. Avoid him in future. HTTPS is simply HTTP over SSL, with the minor addition of hostname checking to ensure that the hostname in the certificate agrees with the site you connected to.
Specifically:
One of my seniors told me that Https does not actually use SSL/TLS, but only their the encryption algorithms
Wrong.
He says that the handshaking process with certificates are done in the transport layer
Wrong for both SSL and HTTPS. It is done by the application, typically in a library such as OpenSSL or JSSE. The transport layer only does the TCP part, none of the SSL part.
but the security key encryption for actual payload is done in the Application layer.
Wrong again: payload encryption is done in the application layer for both SSL and HTTPS.
And he also said that Https can actually be considered as a presentation layer protocol.
It is an application layer protocol. There are very few examples of presentation layer protocols (XDR and TN3270 comes to mind). This isn't one of them. But I agree with Bruno. The OSI layers are not applicable to TCP/IP, which has its own layer model, and, since the unlamented demise of the OSI protocol effort, they aren't applicable to anything else either.
From Wikipedia:
HTTP operates at the highest layer of the OSI Model, the Application layer; but the security protocol operates at a lower sublayer, encrypting an HTTP message prior to transmission and decrypting a message upon arrival.
https://en.wikipedia.org/wiki/Https#Network_layers
I've seen several examples of writing an HTTP proxy in Ruby, e.g. this gist by Torsten Becker, but how would I extend it to handle HTTPS, aka for a "man in the middle" SSL proxy?
I'm looking for a simple source code framework which I can extend for my own logging and testing needs.
update
I already use Charles, a nifty HTTPS proxy app similar to Fiddler and it is essentially what I want except that it's packaged up in an app. I want to write my own because I have specific needs for filtering and presentation.
update II
Having poked around, I understand the terminology a little better. I'm NOT after a full "Man in the Middle" SSL proxy. Instead, it will run locally on my machine and so I can honor whatever SSL cert it offers. However, I need to see the decrypted contents of packets of my requests and the decrypted contents of the responses.
Just for background information, a normal HTTP proxy handles HTTPS requests via the CONNECT method: it reads the host name and port, establishes a TCP connection to this target server on this port, returns 200 OK and then merely tunnels that TCP connection to the initial client (the fact that SSL/TLS is exchanged on top of that TCP connection is barely relevant).
This is what the do_CONNECT method if WEBrick::HTTPProxyServer.
If you want a MITM proxy, i.e. if you want to be able to look inside the SSL/TLS traffic, you can certainly use WEBrick::HTTPProxyServer, but you'll need to change do_CONNECT completely:
Firstly, your proxy server will need to embed a mini CA, capable of generating certificates on the fly (failing that, you might be able to use self-signed certificates, if you're willing to bypass warning messages in the browser). You would then import that CA certificate into the browser.
When you get the CONNECT request, you'll need to generate a certificate valid for that host name (preferable with a Suject Alt. Name for that host name, or in the Subject DN's Common Name), and upgrade the socket into an SSL/TLS server socket (using that certificate). If the browser accepts to trust that certificate, what you get from thereon on this SSL/TLS socket is the plain text traffic.
You would then have to handle the requests (get the request line, headers and entity) and take it to use it via a normal HTTPS client library. You might be able to send that traffic to a second instance of WEBrick::HTTPProxyServer, but it would have to be tweaked to make outgoing HTTPS requests instead of plain HTTP requests.
Webrick can proxy ssl:
require 'webrick'
require 'webrick/httpproxy'
WEBrick::HTTPProxyServer.new(:Port => 8080).start
from my experience HTTPS is nowhere near "simple". Do you need a proxy that would catch traffic from your own machine? There are several applications, like Fiddler. Or google for alternatives. Comes with everything you need to debug the web traffic.
That blog is no way to write a proxy. It's very easy: you just accept a connection, read one line which tells you what to connect to, attempt the upstream connection, if it fails send the appropriate response and close the socket, otherwise just start copying bytes in both directions, simultaneously, until EOS has occurred in both directions. The only difference HTTPS makes is that you have to speak SSL instead of plaintext.
I've read about WebSockets but they don't seem to be pure "sockets", because there is an application layer protocol over them. "ws:"
Is there any way of doing a pure socket connection from a web browser, to enliven webpages?
Here are my random stabs in the dark
Applets sockets provided by Java (need java installed)
Flash sockets provided by Flash (need flash installed)
But about HTML5, Why are they called WebSockets if they aren't Sockets?
Is the websocket protocol so simple to implement that it is "almost"-sockets?
I've read about WebSockets but they don't seem to be pure "sockets", because there is an application layer protocol over them.
[Is the] websocket protocol so simple to implement that [it is] "almost"-sockets?
Allowing regular socket connections directly from the browser is never going to happen because it opens up a huge risk. WebSockets is about as close to raw sockets from the browser as you are going to get. The initial WebSockets handshake is similar to an HTTP handshake (allowing web servers to proxy/bridge it) and adds CORS type security. In addition, WebSockets is a message based transport (rather than streaming as raw TCP) and this is done using a two byte header on each message frame.
Even flash is not able to quite make raw TCP connections. Flash sockets also add CORS security, but instead of an in-band handshake, flash socket connections make a connection to port 843 on the target server to request a security policy file.
Is there any way of doing a pure socket connection from a web browser, to enliven webpages?
Yes, you can use my websockify bridge/proxy which allows a WebSockets enabled browser to connect directly to a TCP socket via websockify.
But about HTML5, Why are they called WebSockets if they aren't Sockets?
WebSockets are a transport built on TCP sockets. After the handshake there is very minimal overhead (typically just a two byte header).
I can't improve on Kanaka's answers to your secondary questions, and I know this question is a year old. But for the main question, Is there any way of doing a pure socket connection from a web browser, to enliven webpages? There is a project called the Java / JavaScript Socket Bridge that might be what you (or anyone coming across this page from a Google search) are looking for. The advantage of this method over what others have mentioned is that it does not require either a client-side or a server-side service to be run. So, for instance, if you wanted to implement an IRC client purely in JavaScript but your web host does not allow you sufficient rights to proxy the connection, this Java applet would be the way to go. The only concern is making sure the client has Java installed and allowed.
You can just send data between a client and a server with WebSockets. Simply speaking, the only difference that WebSockets introduces is that the client:
adds some header bytes, like the type of data and the length
adds masks and encodes the data using them
The server also has to add header bytes, but does not need to encode the data.
If you implement the protocol correctly (server side, that is, since the browser already has an implementation), you can use it with ease to send text and binary data. (Although browser support is narrow, especially for the latter.)
The benefit of WebSocket is that it is HTTP based. You can use it also in environments there http proxies are used. Thus Websocket has a higher infrastructure compatibility as plain tcp.
Additionally http/WebSocket is providing you some features which you otherwise have to specify on your own:
Redirect
NAT keepalive
Multiplexing via URI
Framing
If you are asking for some data to be pushed from server it is widely termed as COMET or Reverse Ajax.
Web sockets is still not very popular as there are inherent firewall issues and minimal support yet from popular browsers.
You can take a look at http://www.ape-project.org/ as this is one of the most popular implementations (but native to unix/linux only for now. For windows they suggest using a virtual box or vmware based implementation)
I have a local server outputting my real-time home sensor data, and I want to visualize it in my browser.
My question is, can I use a websocket to open the connection from my browser to the local server? How would I go about doing that?
The local server runs on a non-http designated port number, and I can't change that.
Yes and no.
No:
WebSockets are not raw TCP connections. They have an HTTP compatible handshake (for both security and compatibility with existing servers) and have some minimal framing for each packet to make WebSockets a message based protocol. Also, the current WebSocket API and protocol that exists in browsers as of today do not directly support binary data messages. They only UTF-8 encoded payloads.
Yes:
You can use websockify to proxy a WebSockets connection to a raw binary TCP server. websockify is a python proxy/bridge that has binary support and also includes a javascript library to make interacting with it easier. In addition, websockify includes the web-socket-js fallback/polyfill (implemented in Flash) for browser that do not have native WebSockets support. The downside is that you have to run websockify somewhere (either on the client system, the server system, or some other system). Also, websockify is Linux/UNIX only for now. On the plus side, websockify has a special mode that you can use to launch and wrap an existing service.
Disclaimer: I made websockify.
I'm trying to develop a XMPP "Proxy" which will be in the middle of a standard Jabber communication.
The schema will be something like this:
Pidgin ---> Proxy <--- eJabberD
|
v
Console
The purpose of this proxy is to log all the stanzas which go over the wire. IMHO, this is very convenient when you're developing XMPP based solutions.
I'm doing this with EventMachine and Ruby, and the main problem is to know how to decypher the traffic after the TLS/SASL handshake.
Before the starttls, all works perfectly, the server and client can talk between them, but when the tls handshake begins, although it works, it is impossible to dump the clear content as all of the traffic is encrypted.
I'm not an expert in TLS/SASL thing, so I don't know which is the best approach to do this. I think one way to achieve this, should be to grab the certificate in the handshake and use it to decypher the content as it goes throught the proxy.
Thanks!
If you could do what you say (grab the certificate on the wire and use it to decrypt), then TLS would be pretty worthless. This is one of the primary attacks TLS exists to prevent.
If the server will allow it, just don't send starttls. This is not required by spec. If starttls is required by your server, then you can configure it to use a null cipher, which will leave the traffic unencrypted. Not all servers will support that of course.
You can man-in-the-middle the starttls. Respond with your own tunnel to the client, and send a separate starttls negotiation to the server. This should generate certificate warnings on the client, but since you control the client you can tell it to accept the certificate anyway.
If you control the server, you can use the private key from it to decrypt the traffic. I'm not aware with any off-the-shelf code to do that easily, but it's writable.