I'm trying to use wireshark to decode, view, and ultimately log my own https traffic--response bodies included. According to the wireshark docs, I need provide the file location of the private RSA key used to decode messages. My question is this:
Where on osx is the private rsa key used in https interactions, is this a single key? Many?
Wireshark docs seem to be telling me to make an RSA key. Given that I'm not experienced enough with this topic, messing with system keys because I read a thing on the internet seems like a pit of despair. What should I do?
What I'm really trying to do is log unencrypted https requests/responses with bodies, while listening to web traffic. If there's a better way I'm all ears.
What I'm really trying to do is log unencrypted https requests/responses with bodies, while listening to web traffic. If there's a better way I'm all ears.
Don't mess around with Wireshark for this. The documentation you're reading is outdated; modern TLS cipher suites do not use pure RSA for key exchange. This configuration was only supported by SSL 2.0, which was superseded by SSL 3.0 in 1996, and is no longer supported by any moern browser. Long story short -- it doesn't actually work in practice.
Instead, use a HTTPS proxy server. Several common tools for this purpose are:
mitmproxy
Charles Proxy (commercial)
fiddler
Many of these tools will also allow you to alter the contents of an HTTPS session, which is certainly not something that Wireshark will do.
Related
I am using a third party (OS X) tool to help me process OFX financial data. It works, but I am interested in knowing what exactly is going on behind the scenes to make it work (the structure of the HTTP requests).
I setup Charles as an SSL proxy for all traffic in hopes that I could observe the requests being made by this tool, but the program runs and Charles gets nothing. No requests show up whatsoever. How is that possible? Is there something I am not understanding about how Charles or other packet sniffing tools work? What are some ways that web requests could be made that wouldn't show up in a tool like Charles?
Charles is not a packet sniffer. It's a proxy. The app initiating the connection has to "voluntarily" use the proxy for the proxy to see anything. If an app uses a high-level networking API like NSURLConnection then it will, by virtue of the frameworks, automatically pick up the system-wide proxy settings and use the proxy. If, instead, the app wrote their networking using low-level socket API, then they will not end up going through the proxy unless they specifically re-implement that functionality.
If you want to see everything, you will need a real promiscuous-mode packet sniffer, which Charles is not. Unfortunately, using a "real" packet sniffer will just show you the gibberish going over the wire for SSL connections, so that's probably not what you want either. If an app has "in-housed" its SSL implementation and is not using a properly configured system-wide proxy, sniffing its traffic unencrypted will be considerably harder (you'll probably have to use a debugger or some other runtime hooking approach.)
I'm working on a study project and need to create a software which should encrypt LAN traffic between computers with Windows. So I need to capture, encrypt and resend all outbound traffic, and capture and decrypt all inbound traffic.
Currently I see two way to do it:
1) IP over UDP. I need encrypt IP packets and send them through UDP link, receive them and decrypt.
2) Encrypt payload of IP packets and decrypt it on another side.
I actually don't know how to do it better and where to start. All suggestions/examples will be helpful.
If you really only need to encrypt the traffic you can simply install a "manually keyed" IPSec SA. See instructions at MSDN
That being said, encryption is the easy part. Authenticating the peers and key agreements is the hard part.
Cryptography is hard to get right, so you definitely want do not want to invent a probably insecure wheel, but opt for a peer-reviewed standard solution, such as the Internet Key Exchange protocol. There is an (unfortunately discontinued) internet draft of a minimal IKE implementation.
Please note that it is perfectly OK to use IKEv2 as the key agreement / authentication protocol for any application - not just for ESP. But if you need to encapsulate ALL IP, Encapsulating Security Payload in tunnel mode is your friend, and the lucky thing here is that ANY OS that is IPv6 compliant MUST implement it, so using ESP is in practice just a matter of installing the key material to your OS kernel's IP stack.
In case you need code samples, I have made a minimal proof-of-concept level implementation of an initiating end of an IKEv2 peer in Python. A Perl implementation doing the same can be found from these IETF proceedings slides
I have learned a little about https, but was not clear why it is needed.
What if I encrypt the data using the most powerful algorithms like RSA instead of sending through a HTTPS zone? Can someone explain with a few reasons why we need https?
On the Wikipedia article it says
Technically, it is not a protocol in itself; rather, it is the result
of simply layering the Hypertext Transfer Protocol (HTTP) on top of
the SSL/TLS protocol
On the SSL/TLS article on Wikipeida,
TLS and SSL encrypt the segments of network connections at the
Application Layer for the Transport Layer, using asymmetric
cryptography for key exchange, symmetric encryption for
confidentiality, and message authentication codes for message
integrity.
So the key exchange does use asymmetric cryptography and RSA is an asymmetric cryptography algorithm.
After key exchange has been performed in a secure manner further communication can be done through symmetric cryptographic algorithms. The reasoning behind using both symmetric and asymmetric algorithms can be found here.
What if I encrypt the data using the most powerful algorithms like RSA
instead of sending through a HTTPS zone?
You will have to implement all by yourself, i.e. reinvent the wheel. HTTPS is by default supported in every browser.
Can someone explain with a few reasons why we need https?
Secure communication that is widely supported. If you have in the middle of communication someone with sniffer tool like Wireshark, he/she will be able to see all packets that you and your peer exchange. Try to catch the HTTPS communication, you won't be able to see anything meaningful in the body of the request.
You could indeed encrypt the data by yourself, but you will face a big problem: The encrypting code must be available on the server as well as on the client (normally the browser).
Implementing the encryption on the server can be done securely. On the client side you can either install a software (plugin), or you can send JavaScript to the client. The problem is: how do you get the encrypting code to the client? Everybody evesdropping, will get the javascript code as well, so he can do the same things as the client will be able to do.
Instead of forcing the user to install a plugin, you can use the built-in support for SSL, every browser understands this protocol already. You could think of it, as an already installed plugin for encryption.
I'm in the process of putting my whole website behind SSL because something requires it right now. It's the first time I do that. I normally debug my (php/asp.net) website via Wireshark, and I really worry about the fact this will be totally obscure/encrypted once running in SSL.
Should I say goodbye to fast, simple debugging with a packet sniffer, or there are well-known practices to debug it?
You can decrypt (some) SSL traffic with WireShark. Since you own the domain, you own the private key the site will use. Configure WireShark to use it to decrypt the traffic to/from your site.
http://wiki.wireshark.org/SSL/
https://wirewatcher.wordpress.com/2010/07/20/decrypting-ssl-traffic-with-wireshark-and-ways-to-prevent-it/
How do I get the HTTPS event from raw data?
If you are asking how to decrypt captured HTTPS network packets after the fact, that is not normally possible. You need at least the HTTPS session keys, which can only be retrieved by modifying the browser - but if you have that kind of access to the browser, you can intercept the unencrypted data anyway.
Things are easier if you have the private key of the HTTPS server, although there are encryption algorithms that use Diffie-Hellman key exchange to offer perfect forward secrecy, thus making the decryption of captured data impossible.
See also this Wikipedia article, if you would like more information on the TLS/SSL protocol that is used in HTTPS.
If you are only interested in monitoring your own browser, e.g. for debugging, you might be able to use a plugin, such as LiveHTTP Headers for Firefox, that will tap into the browser internals to show you what is being transmitted and received via an encrypted connection.