I'm searching for some examples on how to write a proxy in Ruby that supports HTTPS. I have a simple proxy implemented with Webricks HTTPProxyServer, but I noticed, that HTTPS traffic is just tunneling (as it should ;) ). But I want to record the content with VCR (regarding my question here VCRProxy: Record PhantomJS ajax calls with VCR inside Capybara) and as long the content is only tunnled through, VCR can't record it.
So I was thinking of writing the proxy as a man-in-the-middle, generate SSL certificates on the fly (I don't care about certificate errors, its just for testing), and then I would be able to record the content / playback it later.
So if somebody has a good ressource from how to start, or a tutorial or a gist, please let me know.
PS: I have already seen this questions, but they don't provide any further stuff (and it need to be in ruby):
Man in the Middle (MITM) proxy with HTTPS support
How do I write a simple HTTPS proxy server in Ruby?
Help with HTTP Intercepting Proxy in Ruby?
An old question, but for the sake of completeness here goes another answer.
I've implemented a HTTP/HTTPS interception proxy in Ruby, the project is hosted in github.
The project is new, so it's not (yet) as mature as Python's mitmproxy, but it supports HTTPS with certificates generation on-the-fly.
There's an excellent MITM proxy in Python aptly named mitmproxy. The netlib library by the author does the tricks and mitmproxy uses it.
The codebase isn't large and it shouldn't be hard to go through it given that you know Ruby.
Related
general questions about the using of *fail_if_no_peer_cert *and *fail_if_no_peer_cert *params in *rabbitmq_management *on windows
if a client calls management API through https, the requests are secured by the certificate which is installed on the server and trusted on the client. actually, it means, this certificate shouldn't be rabbitmq compatible, it can be just http-certificate... is it correct?
in case i want to validate clients as well, i must have these params verify and true? what is the best practice? because i see these params explained in AMPQ settings and never in management
actually my motivation for this questions is just to understand if i need to deal with this issue at all. because setting *fail_if_no_peer_cert *as true makes a lot of things much more complicated. foe example you just can simply open management api plugin in browser, you need deal with client certificate.
I'm trying to use wireshark to decode, view, and ultimately log my own https traffic--response bodies included. According to the wireshark docs, I need provide the file location of the private RSA key used to decode messages. My question is this:
Where on osx is the private rsa key used in https interactions, is this a single key? Many?
Wireshark docs seem to be telling me to make an RSA key. Given that I'm not experienced enough with this topic, messing with system keys because I read a thing on the internet seems like a pit of despair. What should I do?
What I'm really trying to do is log unencrypted https requests/responses with bodies, while listening to web traffic. If there's a better way I'm all ears.
What I'm really trying to do is log unencrypted https requests/responses with bodies, while listening to web traffic. If there's a better way I'm all ears.
Don't mess around with Wireshark for this. The documentation you're reading is outdated; modern TLS cipher suites do not use pure RSA for key exchange. This configuration was only supported by SSL 2.0, which was superseded by SSL 3.0 in 1996, and is no longer supported by any moern browser. Long story short -- it doesn't actually work in practice.
Instead, use a HTTPS proxy server. Several common tools for this purpose are:
mitmproxy
Charles Proxy (commercial)
fiddler
Many of these tools will also allow you to alter the contents of an HTTPS session, which is certainly not something that Wireshark will do.
Hello. I seek some guidance. I need a http web proxy with websocket support. The catch is, that I need to retrieve the context of HTTP request (headers, paramters, etc, used for retrieiving routing table) and in the same time I need websockets to work. So far I found onlt either HTTP-only proxies or TCP proxies that lack the context of HTTP request.
Is anyone aware of existence of a library or tool that will let me accomplish that?
I considered:
proxy.py, seems ugly and I couldn't make it work for my case in ~6 hours.
twisted, no experience nor success with this one either
nginx module in lua, looks promising and fast
I am using a third party (OS X) tool to help me process OFX financial data. It works, but I am interested in knowing what exactly is going on behind the scenes to make it work (the structure of the HTTP requests).
I setup Charles as an SSL proxy for all traffic in hopes that I could observe the requests being made by this tool, but the program runs and Charles gets nothing. No requests show up whatsoever. How is that possible? Is there something I am not understanding about how Charles or other packet sniffing tools work? What are some ways that web requests could be made that wouldn't show up in a tool like Charles?
Charles is not a packet sniffer. It's a proxy. The app initiating the connection has to "voluntarily" use the proxy for the proxy to see anything. If an app uses a high-level networking API like NSURLConnection then it will, by virtue of the frameworks, automatically pick up the system-wide proxy settings and use the proxy. If, instead, the app wrote their networking using low-level socket API, then they will not end up going through the proxy unless they specifically re-implement that functionality.
If you want to see everything, you will need a real promiscuous-mode packet sniffer, which Charles is not. Unfortunately, using a "real" packet sniffer will just show you the gibberish going over the wire for SSL connections, so that's probably not what you want either. If an app has "in-housed" its SSL implementation and is not using a properly configured system-wide proxy, sniffing its traffic unencrypted will be considerably harder (you'll probably have to use a debugger or some other runtime hooking approach.)
I am looking to get access to all HTTP traffic on my machine (my windows machine - not a server). From what I understand having a local proxy through which all traffic routes is the way to go. I have been Googling but failed to find any resources (in respect to Ruby) to help me out. Any tips or links are much appreciated.
There's an HTTP Proxy in WEBrick (part of Ruby stdlib) and here's an implementation example.
If you like living on the edge there's also em-proxy by Ilya Grigorik.
This post by Ilya implies that it does seem to need some tweaking to solve your problem.
Is having a proxy built in Ruby the important point here? Or just to "get access to all HTTP traffic on your machine"? If the latter, there's a free program called HTTP Sniffer and Analyzer that can supposedly do this. I have not used it but I have seen it get some positive reviews. There are several other such programs, though most seem to be paid. On OS X, Linux, etc, you can use the in-built tcpdump in clever ways to get a similar effect.