I made a proxy server in python 3. It listens on the port 4444. It basically receives the request from clients and sends it to the server. I want to use it as a firewall to my Dvwa server. So added another functionality to the proxy. What it does is, before sending the request to the DVWA server, it validates the input.
But the problem is, the clients have to configure their proxy settings in the browser to use my proxy server. Is there any way to access the proxy without configuring the browser settings. Basically I want to host the proxy server instead of the original web server. So that all the traffic goes through the proxy before going to the webserver.
Thanks in advance...
You don't say whether your Python3 proxy is hosted on the same machine as the DVWA.
Assuming it is, the solution is simple: a reverse-proxy configuration. Your proxy transparently accepts and forwards requests to your server who then processes them and sends them back via the proxy to the client.
Have your proxy listen on port 80
Have the DVWA listen on a port other than 80 so it's not clashing (e.g. 8080)
Your proxy, which is now receiving requests for the IP/hostname which would otherwise go to the DVWA, then forwards them as usual.
The client/web browser is none the wiser that anything has changed. No settings need changing.
That's the best case scenario, given the information provided in your question. Unfortunately, I can't give any alternative solutions without knowing the network layout, where the machines reside, and the intent of the project. Some things to consider:
do you have a proper separation of concerns for this middleware you're building?
what is the purpose of the proxy?
is it for debugging/observing traffic?
are you actually trying to build a Web Application Firewall?
Related
HTTP proxy with SSL and DNS support.
I must be lacking some key concepts about proxy-ing because I cannot grasp this. I am looking to run a simply http or https proxy without interfering with SSL. Simply, a fully transparent proxy that can passthrough all the traffic to the browser connected via HTTP or HTTPS proxy without modifying or intercepting any packets. Not able to find any code online or I'm not using the right keywords.
EX. On the browser adding server.someVPN.com:80 on the HTTP proxy field and as soon as you try to visit a website, it prompts for authentication. Then it works perfectly with any domain, any security, any ssl, no further steps needed. Most VPN providers have this.
How's this possible? it even resolves DNS itself. I thought on transparent proxy the dns relies on the client. Preferably looking for a nodeJS solution but any lang works.
Please don't propose any solutions such as SOCKS5 or sock forwarding or DNS overriding or CA based MITM. According to HTTP 1.1 which supports 'CONNECT' this should be easy.
Not looking to proxy specific domains, looking for an all inclusive solution just like most VPN Providers providers.
----Found the answer too quickly, feel free to delete this post/question admins.
The way it works is that the browser knows it is talking to a proxy server, so for example if the browser want to connect to htttp://www.example.com it sends a CONNECT www.example.com:443 HTTP/1.1 to the proxy server, the proxy server resolves wwww.example.com via DNS and then opens a TCP connection to wwww.example.com port 443 and proxies the TCP stream transparently to the client.
I don't know any solution for nodejs. Common proxy servers include Squid, Privoxy and Apache Traffic Server
See also: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT
Found the solution right after I asked...
This module works perfectly https://github.com/mpangrazzi/harrier
Does exactly what I was asking for.
I'm running a local web server written in Go and I can debug traffic going to it from my browser; but, I can't see the http request that it makes to external services.
Do I have to run some particular configuration of the web server in order to get the traffic to appear in fiddler? It is running as a background process.
Short answer: you can't...
...unless your web application is written to open a connection to a Proxy server and route requests through that connection (e.g. connect to a remote proxy, and then send requests through it).
Typically what developers do is just dump the Web Request/Response to a debug file to inspect during development (or to debug on a live server, by enabling it with a flag at runtime).
Fiddler is a "proxy" service/server. When you are using it normally to debug browser requests, your Browser is configured to connect to a Proxy server. That is, it will send all web requests through your fiddler's local server (I think it's localhost:8888 if i remember from my Windows days of using Fiddler) which in turn makes a connection to your local web server that you are debugging.
You can read more about Proxies at Wikipedia.
In that picture above, your local web server would be Alice. Meaning, Alice would need to be configured to connect to a proxy server and then make web requests through it.
EDIT:
(for the "I really need this" crowd)
If you really want to modify your web server to send requests through a proxy, there are a few Go packages already written to help you. GoProxy is one such package.
This may not be the correct place for this question as it's part networking, but here goes.
I am wanting to put together a WebApi (using the ASP.NET MVC WebApi framework) to be consumed by client machines external to our network. However the client machines resolve web traffic through a proxy server for which our software does not have authentication. We have noticed that outgoing FTP connections are possible though.
So I am wondering whether we can host the webapi and have client machines connect out through Port 21? Does that even make sense? Sorry if it's a stupid question.
I managed to find some answers and thought I would share for anyone that might be interested.
Binding WebApi to ports other than 80
This is possible, but tricky. When you publish the Api project onto IIS (or wherever you are hosting it) you just bind it to an alternative port. You then also make sure you forward that port in your router. Then, clients of the API just specify the host using your custom port to access the endpoint through that port: http://myhostname.com:21/api/values or whatever.
Complications
Testing the endpoints can be tricky as Chrome blocks HTTP traffic being sent via some ports - port 21 is one such port. So to test it you need to write a client exe that can hit the endpoints to make sure they are working (like a console application).
Despite figuring all of this out, I still could not connect out through the firewall. I suspect that some configuration is blocking the traffic because even though it is going out through an open port (21), it is not FTP traffic: it's HTTP traffic.
A Solution
It occurred to me that SOAP operates through a range of protocols (FTP, SMTP, HTTP, to name a few) and formats its messages as XML. So in this scenario it would make more sense to use a SOAP service via Port 21 rather than REST which is strictly HTTP.
We often find columns like Address, Port in web browser proxy settings. I know when we use proxy to visit a page, the web browser request the web page from the proxy server, but what I want to know is how the whole mechanism works? I have observed that many ISP allow only access to a single IP(of their website) after we exhausted our free data usage. But when we enter the site which we wants to browse in proxy URL and then type in the allowed IP, the site get loaded. How this works?
In general, your browser simply connects to the proxy address & port instead of whatever IP address the DNS name resolved to. It then makes the web request as per normal.
The web proxy reads the headers, uses the "Host" header of HTTP/1.1 to determine where the request is supposed to go, and then makes that request itself relaying all remaining data in both directions.
Proxies will typically also do caching so if another person requests the same page from that proxy, it can just return the previous result. (This is simplified -- caching is a complex topic.)
Since the proxy is in complete control of the connection, it can choose to route the request elsewhere, scrape request and reply data, inject other things (like ads), or block you altogether. Use SSL to protect against this.
Some web proxies are "transparent". They reside on a gateway through which all IP traffic must pass and use the machine's networking stack to redirect outgoing connections to port 80 to a local port instead. It then behaves the same as though a proxy was defined in the browser.
Other proxies, like SOCKS, have a dedicated protocol that allows non-HTTP requests to be made as well.
There are 2 types of HTTP proxies, there are the ones that are reversed and the ones that
are forward.
The web browser uses a forward proxy, basically it is sending all http traffic through the proxy, the proxy will take this traffic out to the internet. Every http packet that comes out from your computer, will be send to the proxy before going to the target site.
The ISP blocking does not work when using a proxy because, every packet that comes out from your machine is pointing to the proxy and not to the targe site. The proxy could be getting internet through another ISP that has no blocks whatsoever.
I have a Play! Application which needs to use HTTP and HTTPS. The application is running behind a proxy server (using Apache) that forwards web requests to the play application.
The proxy is using one port for HTTP requests, and another port that is intended for HTTPs requests. Note that the ports on the proxy are not the same ports as the ones used by the Play application (this is due to provider restrictions!).
The Play application is using the "standard" ports 9000 for HTTP and 9443 for HTTPs. The proxy receives HTTP requests on Port 8080 and forwards them to Play's port 9000. The proxy receives HTTPs requests on Port 8090 and forwards them to Play's port 9443.
My problem is that when I use the secure() method of making pages appear in Play, Play's logic causes the app to attempt to use 9443 as the port for HTTPs. This causes the request to be lost because the proxy is using a different port.
The same appears to happen when I want to switch from HTTPs to HTTP. I cannot seem to make the system go to the port used by the proxy.
Somehow I need to make the system go to the ports known to the proxy server, without screwing up my routes. Is there some way to do this?
Thanks in advance for any help/insights.
I have found my own "answer", though it is somewhat of a cludge.
It turns out that, based on what I can ascertain from the documentation, there really is no way to tell Play to switch between ports when a Play application is behind a proxy. This is because while Play does recognize the port that a request comes in on, it has no way of telling what proxy port it should use when going between secure and unsecure ports. It knows, for example, that an HTTP request comes through a proxy port 8080, and it knows that subsequent requests to its port 9000 will come from that proxy port. What it does not know to do is to switch to another proxy port when someone attempts to use https to access its port 9443. Consequently, if you have a page like
http://toproxy:8080/links that has one or more links that use the secure() method to activate https, then Play will resolve the link to be https://toproxy:8080 -- despite the fact that the proxy server may want to use port 8090 for HTTPS requests. Because proxy port 8080 is redirected to Play's port 9000, use of that port for HTTPS requests always fails. This is true in Play 2.0 as well as Play 1.X.
I believe that Play needs some standard configuration parameter that can tell it to map proxy ports to its HTTP and HTTPS ports. That way, when it is behind a proxy server a developer can use the secure() method and Play will be able to resolve the URL to the correct proxy port. This capability should be available in 1.X as well as Version 2.
Until someone actually implements this (I might if I ever get the time, but with all that I am committed to do people shouldn't hold their breath), my cludge is to simply use the rediirect() method to switch between HTTP and HTTPS proxy ports. The redirect() method apparently allows us to enter full URLs, so I simply call the full URL of the page that I switch requests on.
For example: in the aforementioned page http://toproxy:8080/links, I may have a link to a login page that I want to protect using HTTPS. To do this I create two actions: one for the redirect to the proxy HTTPS port (for this example, call it gotoLogin()) and another for actually rendering the login page (for this example, call it loginPage() and give it a route of /loginpage).
In gotoLogin() I redirect to loginPage in the following manner:
redirect("https://toproxy:8090/loginpage");
This makes Play come in on proxy port 8090, which is redirected to Play's port 9443.
When the user properly logs in, my authentication action simply uses another redirect() call:
redirect("http://toproxy:8080/destination_page");
This causes Play to go back to the proper proxy port for unsecured requests.
This is not the best solution for this problem. There is a possibility that the proxy server can be somehow configured to make the proper switch between HTTP and HTTPS ports, but investigating this may take some time for someone not an expert in proxy server configuration (which describes me!). Of course, the best solution would be to implement in Play the kind of proxy port handling that I have previously described.
But this solution works and can be adapted to many different situations. Unless someone comes up with another better solution, this solution is the one I am using.
Play should recognise the port the request is coming from, and use that port. It is possible your apache config is not set up correctly.
Have you put the following line in your apache config?
ProxyPreserveHost on
You may also need XForwardedSupport=127.0.0.1 depending on your server configuration.