I have Charles Proxy set up to look at outgoing https requests, and I need to re-route traffic from one server to a local http server.
I have a MacOSX machine that is set up this way: I have an ethernet connection that I hardwire, and share the internet connection via the airport interface. On a second machine, I've installed the Charles cert, and when I connect via the shared interface. I can see the traffic (unencrypted) in Charles, so I know the communications and certs are all working properly.
I need to intercept all the https traffic going to one server (https://www.foo.com) to a local http server (localhost:8001). I've tried using Map Remote, but it doesn't seem to unencrypt the traffic before forwarding it (or possibly it re-encrypts it).
How can I configure Charles to do this? (or, please point me to any other software package, if Charles isn't capable of this)
I figured out what was happening, there were two issues.
I had misconfigured the Map Remote entry, and my two different clients (MyApp and curl) were hitting two different servers - the app was hitting the correct server (locally) but the request was malformed.
Curl from the macOSX box where the proxy was running was NOT looping through the proxy, since I hadn't included the -x localhost:8888 flag.
Related
I need to develop an app that intercepts HTTPS traffic on the machine and change it.
One approach is to use a local proxy, like Fiddler works.
I also saw HTTP Debugger (https://www.httpdebugger.com/) which works in a similar way to Fiddler but say they don't use a proxy.
Do you know what technology HTTP Debugger are using in order to intercept HTTPS traffic without proxy?
One way is using a pre-master secret key.
Here is an example with Wireshark: https://www.comparitech.com/net-admin/decrypt-ssl-with-wireshark/
Wireshark using WinPcap https://www.winpcap.org/ network capture library to intercept traffic, so there is no proxy involved.
EDIT: I don't know how exactly works HTTP Debugger, but if there is no proxy the only way is to decrypt traffic on network level.
I made a proxy server in python 3. It listens on the port 4444. It basically receives the request from clients and sends it to the server. I want to use it as a firewall to my Dvwa server. So added another functionality to the proxy. What it does is, before sending the request to the DVWA server, it validates the input.
But the problem is, the clients have to configure their proxy settings in the browser to use my proxy server. Is there any way to access the proxy without configuring the browser settings. Basically I want to host the proxy server instead of the original web server. So that all the traffic goes through the proxy before going to the webserver.
Thanks in advance...
You don't say whether your Python3 proxy is hosted on the same machine as the DVWA.
Assuming it is, the solution is simple: a reverse-proxy configuration. Your proxy transparently accepts and forwards requests to your server who then processes them and sends them back via the proxy to the client.
Have your proxy listen on port 80
Have the DVWA listen on a port other than 80 so it's not clashing (e.g. 8080)
Your proxy, which is now receiving requests for the IP/hostname which would otherwise go to the DVWA, then forwards them as usual.
The client/web browser is none the wiser that anything has changed. No settings need changing.
That's the best case scenario, given the information provided in your question. Unfortunately, I can't give any alternative solutions without knowing the network layout, where the machines reside, and the intent of the project. Some things to consider:
do you have a proper separation of concerns for this middleware you're building?
what is the purpose of the proxy?
is it for debugging/observing traffic?
are you actually trying to build a Web Application Firewall?
HTTP proxy with SSL and DNS support.
I must be lacking some key concepts about proxy-ing because I cannot grasp this. I am looking to run a simply http or https proxy without interfering with SSL. Simply, a fully transparent proxy that can passthrough all the traffic to the browser connected via HTTP or HTTPS proxy without modifying or intercepting any packets. Not able to find any code online or I'm not using the right keywords.
EX. On the browser adding server.someVPN.com:80 on the HTTP proxy field and as soon as you try to visit a website, it prompts for authentication. Then it works perfectly with any domain, any security, any ssl, no further steps needed. Most VPN providers have this.
How's this possible? it even resolves DNS itself. I thought on transparent proxy the dns relies on the client. Preferably looking for a nodeJS solution but any lang works.
Please don't propose any solutions such as SOCKS5 or sock forwarding or DNS overriding or CA based MITM. According to HTTP 1.1 which supports 'CONNECT' this should be easy.
Not looking to proxy specific domains, looking for an all inclusive solution just like most VPN Providers providers.
----Found the answer too quickly, feel free to delete this post/question admins.
The way it works is that the browser knows it is talking to a proxy server, so for example if the browser want to connect to htttp://www.example.com it sends a CONNECT www.example.com:443 HTTP/1.1 to the proxy server, the proxy server resolves wwww.example.com via DNS and then opens a TCP connection to wwww.example.com port 443 and proxies the TCP stream transparently to the client.
I don't know any solution for nodejs. Common proxy servers include Squid, Privoxy and Apache Traffic Server
See also: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT
Found the solution right after I asked...
This module works perfectly https://github.com/mpangrazzi/harrier
Does exactly what I was asking for.
I am trying to build a SOCKS solution for forward proxy. I am using dante SOCKS proxy as I have heard that big companies like google uses it as forward proxy solution.
on the SOCKS server, I am allowing based on FQDN's like google.com:443
Now the problem is, when the client constructs the packet, it tries to resolve google.com and gets X.X.X.X and sends connect request to SOCKS server. Now when the server receives the packets, it tries to reconstruct the packet to send out to internet, the server again does DNS resolution and if the server gets response as Y.Y.Y.Y, then it doesn't allow client's request as the destination IP in the client's request is different then the server's resolved IP address.
There was a solution in dante client which tells client to put a dummy destination address 0.0.0.1 and sends request to server and server processes it properly then. However that is creating a problem with internal domains as after using that dns resolution method, every requests goes through dante server :(
Please let me know
If there is any solution through which would help me in maintaining a DNS record expiry DC wide for e.g. google.com resolves to X.X.X.X and I should be able to resolve to this same IP address on 100's of DNS client and in case if the record changes, then it should immediately change/expire on client.
Any other proxy/socks solution which should be transparent to applications for forward proxy
I went ahead with this solution in case anyone is curious to see the solution.
I used PowerDNS Auth Server with Pipe backend. The requests would land to PowerDNS server for resolution, it will pass on all the data to Pipe backend script with ABI, the script analysis the requests, sees if it is present under cached variable/memory map, if it is cache hit, it will respond using cached DNS records else it will use a DNS resolver to resolve that query like a resolver resolves normally.
PowerDNS version lower than 4.1 supports Pipe backend + resolver. This way, the request would first land to pipe backend script, if the script doesn't have any entries cached, it will not respond or will respond blank and then PowerDNS would resolve it with the mentioned resolver server in the configuration. However with version 4.1 and above, the resolver part is removed from PowerDNS Auth server hence you need to handle that behaviour via Pipe backend script.
It depends on your client. Firefox, for example, sends hostname to SOCKS proxy without resolving it. You can confirm that by Wireshark.
PS. assume you are using a SOCKS5/4a proxy. SOCKS4 does not support hostname. Ref: https://en.wikipedia.org/wiki/SOCKS#SOCKS4a
My clients are using a proxy server on their corporation,and I want to set up dev environment for testing development related to proxy issues.
So, I want to set up a proxy server which blocks all port 80 requests, unless the request is requested by the proxy server.
This is what I tried:
Installing fiddler2 on port 8888.
Setting up two rules on windows firewall:
Block all port 80 requests.
Allow all requests from fiddler exe.
Then, I opened FF and changed the proxy server to be 127.0.0.1:8888.
Unfortunately, the requests from the fiddler are still blocked.
What am I doing wrong?
Is there other program which do that easier? (Tried also with CCProxy without success).
Found the answer this great post:
Block all the outbound connections on the firewall.
Allow request from fiddler.exe