How do I get the HTTPS event from raw data?
If you are asking how to decrypt captured HTTPS network packets after the fact, that is not normally possible. You need at least the HTTPS session keys, which can only be retrieved by modifying the browser - but if you have that kind of access to the browser, you can intercept the unencrypted data anyway.
Things are easier if you have the private key of the HTTPS server, although there are encryption algorithms that use Diffie-Hellman key exchange to offer perfect forward secrecy, thus making the decryption of captured data impossible.
See also this Wikipedia article, if you would like more information on the TLS/SSL protocol that is used in HTTPS.
If you are only interested in monitoring your own browser, e.g. for debugging, you might be able to use a plugin, such as LiveHTTP Headers for Firefox, that will tap into the browser internals to show you what is being transmitted and received via an encrypted connection.
Related
I'm trying to use wireshark to decode, view, and ultimately log my own https traffic--response bodies included. According to the wireshark docs, I need provide the file location of the private RSA key used to decode messages. My question is this:
Where on osx is the private rsa key used in https interactions, is this a single key? Many?
Wireshark docs seem to be telling me to make an RSA key. Given that I'm not experienced enough with this topic, messing with system keys because I read a thing on the internet seems like a pit of despair. What should I do?
What I'm really trying to do is log unencrypted https requests/responses with bodies, while listening to web traffic. If there's a better way I'm all ears.
What I'm really trying to do is log unencrypted https requests/responses with bodies, while listening to web traffic. If there's a better way I'm all ears.
Don't mess around with Wireshark for this. The documentation you're reading is outdated; modern TLS cipher suites do not use pure RSA for key exchange. This configuration was only supported by SSL 2.0, which was superseded by SSL 3.0 in 1996, and is no longer supported by any moern browser. Long story short -- it doesn't actually work in practice.
Instead, use a HTTPS proxy server. Several common tools for this purpose are:
mitmproxy
Charles Proxy (commercial)
fiddler
Many of these tools will also allow you to alter the contents of an HTTPS session, which is certainly not something that Wireshark will do.
Is the data secure if posted programmatically (not through a browser) to an https endpoint?
My understanding is browser encrypts data and sends it to https endpoint. How can a Ruby or Node.js or any other program do the same?
Yes. If you connect to an https endpoint with curl, wget, or whatever library, the transfer is secure from the source of the connection to the destination. That source could be a server (your webserver) or the client browser.
However, if it's done in client side JS or other browser scripting language, you have to make sure the initial request from client to your site is secure as well if first passing secure data to the client for it to pass to the destination https server.
I checked node.js request library as well as Ruby HTTParty libraries. Both these support SSL encryption based on proper options (port: 443 etc.). In general if we use any well supported library that enables HTTP gets and posts, we should be covered in terms of transmitting data securely to the https endpoint.
I think I understand what you mean and that question has been answered. However, I would just point out that HTTPS does not make your data secure, only the connection and even that is only encrypted from eavesdropping which is not really secure.
There is, of course, lots more to think about and do to make your data secure end-to-end.
This question tries to look into whether doing HTTPS log in is very important for any website.
Is it true that for many websites, if the login is done through HTTP but not HTTPS, then anybody can pretty much see the userID and password easily along the internet highway (or by looking between a router and the internet connection in an Internet Cafe)?
If so... do popular frameworks actually use HTTPS by default (or at least as an option), such as Rails 2.3.5 or Django, CakePHP, or .Net?
Yes, any machine on the pathway (that the packets pass through) can just examine the contents of the those packets. All it takes is a capturing proxy or a promiscuous mode network card with something like WireShark. Assuming that the passwords aren't encrypted in some other way (at a higher level), they will be visible.
I can't answer the second part of your question since I have no knowledge of those particular products but I would say that the inability to use secure sockets would pretty much make them useless.
Pax is right about passwords that aren't otherwise encrypted being visible.
Still, most sites don't use SSL still, and it does put the users at a certain degree of risk when accessing sites from public wifi.
HTTPS isn't a framework level option, it would be something you'd do when you set up the webserver. If you were to use an apache configuration for instance, you would open it up to a properly configured https, close http and install a certification. The framework wouldn't have a direct influence on that portion of the release.
If the user credentials are submitted via an HTML webform without HTTPS, then it is unsecure, the data is submitted in plain text. However, if the website uses HTTP authentication instead, then the server can send back a 401 reply (or 407 for proxies) to any request that does not provide valid credentials. 401/407 is the server's way to ask for credentials, and the reply provides a list of authentication schemes (Digest, NTLM, Negotiate, etc) that the server supports, which are usually more secure by themselves. The client/browser sends the same request again with the necessariy credentials in one of the schemes, then the server either sends the requested data, or sends another 401/407 reply if the credentials are rejected.
I am writing a little app similar to omegle. I have a http server written in Java and a client which is a html document. The main way of communication is by http requests (long polling).
I've implemented some sort of security by using the https protocol and I have a securityid for every client that connects to the server. When the client connects, the server gives it a securityid which the client must always send back when it wants a request.
I am afraid of the man in the middle attack here, do you have any suggestions how I could protect the app from such an attack.
Note that this app is build for theoretical purposes, it won't be ever used for practical reasons so your solutions don't have to be necessarily practical.
HTTPS does not only do encryption, but also authentication of the server. When a client connects, the server shows it has a valid and trustable certificate for its domain. This certificate can not simply be spoofed or replayed by a man-in-the-middle.
Simply enabling HTTPS is not good enough because the web brings too many complications.
For one thing, make sure you set the secure flag on the cookies, or else they can be stolen.
It's also a good idea to ensure users only access the site via typing https://<yourdomain> in the address bar, this is the only way to ensure an HTTPS session is made with a valid certificate. When you type https://<yourdomain>, the browser will refuse to let you on the site unless the server provides a valid certificate for <yourdomain>.
If you just type <yourdomain> without https:// in front, the browser wont care what happens. This has two implications I can think of off the top of my head:
The attacker redirects to some unicode domain with a similar name (ie: looks the same but has a different binary string and is thus a different domain) and then the attacker provides a valid certificate for that domain (since he owns it), the user probably wouldn't notice this...
The attacker could emulate the server but without HTTPS, he would make his own secured connection to the real server and become a cleartext proxy between you and the server, he can now capture all your traffic and do anything he wants because he owns your session.
Say I was trying to access https://www.secretplace.com/really/really/secret.php, what's actually sent in plain text before the SSL session is established?
Does the browser intervene, see that I want https, initiate a SSL session with secretplace.com (i.e. without passing the path in plain text) and only after the SSL session is set up pass the path?
Just curious.
HTTP Secure
The level of protection depends on the correctness of the implementation of the web browser and the server software and the actual cryptographic algorithms supported.
Also, HTTPS is vulnerable when applied to publicly-available static content. The entire site can be indexed using a web crawler, and the URI of the encrypted resource can be inferred by knowing only the intercepted request/response size. This allows an attacker to have access to the plaintext (the publicly-available static content), and the encrypted text (the encrypted version of the static content), permitting a cryptographic attack.
Because SSL operates below HTTP and has no knowledge of higher-level protocols, SSL servers can only strictly present one certificate for a particular IP/port combination. This means that, in most cases, it is not feasible to use name-based virtual hosting with HTTPS. A solution called Server Name Indication (SNI) exists which sends the hostname to the server before encrypting the connection, although many older browsers don't support this extension. Support for SNI is available since Firefox 2, Opera 8, and Internet Explorer 7 on Windows Vista.
In general, the name of the server you are talking to is leaked ("stackoverflow.com"). This was probably leaked via DNS before SSL/TLS could begin connecting, though.
The server's certificate is leaked. Any client certificate you sent (not a common configuration) may or may not have been sent in-the-clear. An active attacker (man-in-the-middle) could probably just ask your browser for it and receive it anyway.
The path portion of the URL ("/questions/2146863/how-much-data-is-leaked-from-ssl-connection") should NOT be leaked. It is passed encrypted and secure (assuming the client and server are set up correctly and you didn't click-through any certificate errors).
The other poster is correct, that there are possible traffic analysis attacks which may be able to deduce some things about static content. If the site is very large and dynamic (say stackoverflow.com) I suspect that it could be quite difficult to get much useful info out of it. However, if there are only a few files with distinctive sizes, which downloads could be obvious.
Form POST data should NOT be leaked. Although usual caveats apply if you are transmitting objects of known sizes.
Timing attacks may reveal some information. For example, an attacker ccould put stress on various parts of the application (e.g., a certain database table) or pre-load some static files from the disk and watch how your connection slows down or speeds up in response.
This is an information "leak" but probably not a big deal for most sites.
The request is made by your browser to https://url:443, and that's in the clear. Then the server and client will negotiate a ciphersuite to protect the data. Normally, this will include a symmetric encryption algorithm (eg; 3DES or RC4 or AES) and a message authentication code (such as HMAC-SHA1) to detect tampering. Note that technically, both of these are optional, it IS possible to have SSL with no encryption but unlikely today.
Once the client (your browser) and the web server have agreed on a ciphersuite and keys are determined, the rest of the conversation is protected.
To be honest, I would hook up a protocol analyzer and watch it all unfold before your eyes!!
Remember, that SSL is at the Transport layer of the TCP/IP stack, it's below the browser data, so it's all protected.
Hope that helps.