All the examples for node-http-proxy show a {target: <URL>} option, but I don't want to proxy to a single target. How do I set it up for outbound requests (to any URL)?
Since node-http-proxy seems to be a proxy for inbound connections to a server farm, and not for outbound connections to random websites, I found another tool instead:
http://newspaint.wordpress.com/2012/11/05/node-js-http-and-https-proxy/
This allows me to customize the requests and responses as needed.
Related
I made a proxy server in python 3. It listens on the port 4444. It basically receives the request from clients and sends it to the server. I want to use it as a firewall to my Dvwa server. So added another functionality to the proxy. What it does is, before sending the request to the DVWA server, it validates the input.
But the problem is, the clients have to configure their proxy settings in the browser to use my proxy server. Is there any way to access the proxy without configuring the browser settings. Basically I want to host the proxy server instead of the original web server. So that all the traffic goes through the proxy before going to the webserver.
Thanks in advance...
You don't say whether your Python3 proxy is hosted on the same machine as the DVWA.
Assuming it is, the solution is simple: a reverse-proxy configuration. Your proxy transparently accepts and forwards requests to your server who then processes them and sends them back via the proxy to the client.
Have your proxy listen on port 80
Have the DVWA listen on a port other than 80 so it's not clashing (e.g. 8080)
Your proxy, which is now receiving requests for the IP/hostname which would otherwise go to the DVWA, then forwards them as usual.
The client/web browser is none the wiser that anything has changed. No settings need changing.
That's the best case scenario, given the information provided in your question. Unfortunately, I can't give any alternative solutions without knowing the network layout, where the machines reside, and the intent of the project. Some things to consider:
do you have a proper separation of concerns for this middleware you're building?
what is the purpose of the proxy?
is it for debugging/observing traffic?
are you actually trying to build a Web Application Firewall?
I am looking for a specific kind of proxy that is meant to operate in a rendezvous mode, such that two nodes can make an outgoing connection to the same proxy, send a routing token, and have their packets relayed to each other from that point.
Proxy servers like HAProxy would be perfect but AFAIK they do not offer something like that: the goal of the proxy in this case is to make another outgoing connection and route the packets to that location. In this case, I want two nodes to connect to the proxy, and have their packets relayed between them through the proxy, after sending a routing token that can be used to associate the two nodes.
I could write my own server to perform such type of relaying, but I am wondering if something already exists to do something like this. I am looking for such a solution as a fallback for cases where NAT traversal protocols like ICE/STUN/TURN are not feasible due to a highly restricted network environment that does not allow UDP traffic. The base protocol for the proxy could be TCP, HTTP or WebSocket, which would be easier to allow in a firewall with a simple rule.
Any ideas or recommendations?
I believes SOCKSv5 has everything you are asking for.
two nodes can make an outgoing connection to the same proxy. send a routing token, and have their packets relayed to each other from that point.
The routing token in this case would be the endpoint address and/or the user credentials. I would first look at the supper simple implementation built into the 'ssh' utility, this guide goes over how to get everything set up. If you need something more granular then look into dante.
The only tricky part is when you try to use the user credential option with SOCKv5 as it is not as well supported in browsers, but is possible with addons.
When an HTTP proxy server is used, is the number of connections negotiated between the client and the proxy reduced as compared to the client connecting directly to various http sites directly (without proxy)?
For example, when connecting directly to two different domains, it is clear that at least two connections must be made. In the case of a proxy, does the client usually use a single connect to the proxy for both "connections"?
Similarly, are there cases where a client that connecting to a single domain but accessing several resources would see a reduced number of connections using a proxy? E.g., can the proxy present a HTTP/1.1-style persistent connect even when the ultimate destination doesn't support it? Are proxies able to use longer persistent connection timeout periods?
In the case of a proxy, does the client usually use a single connect to the proxy for both "connections"?
While it would possible to use the same connection to a HTTP proxy to include HTTP requests to different targets most clients don't do it from what I've seen. Also, it would only work with HTTP and not HTTPS since in the latter case the whole TLS connection to the target is tunneled through the proxy and the close of this tunneled connection is also the close of the underlying TCP connection to the proxy. And, HTTP requests to multiple targets would only be possible with a HTTP proxy but not a SOCKS proxy since SOCKS essentially builds a tunnel to a specific target and this target is set at the beginning of the connection and can never be changed.
That said, while I've not seen it for browser to proxy connections I've seen a patched squid used (long ago) to do this in order to optimize proxy to proxy connections.
E.g., can the proxy present a HTTP/1.1-style persistent connect even when the ultimate destination doesn't support it?
While this would be possible too it is also not common. Usually the proxy does not fully decouple client and server, i.e. a server-triggered close of the connection between server and proxy usually results in close of the connection between proxy and client too. The reason is probably that it would work for only for HTTP anyway and not HTTPS and that it makes the implementation of the proxy more complex since things like repeating a request on sudden close of a persistent connection by the server between requests would now need to be handled by the proxy instead of simply forwarding the close and let the client deal with it.
If UDP is blocked, i need to enable HTTP(s) fallback with aspera connect plugin thru HAProxy. Has anyone successfully done this ? I have read all the documentation but there is nothing that describes the HAProxy use case
I assume you talk about a client side forward proxy.
If you use the IBM Aspera Connect Client, it comes with HTTP (forward) proxy configuration.
http://download.asperasoft.com/download/docs/connect/3.9.6/user_osx/webhelp/index.html#dita/network_environment.html
This will work when the client falls back to HTTP.
Note that the client will fallback to HTTP only if this feature is enabled on the server side.
You can see that by examining the parameters provided to the client by the browser (put in dev mode). In that case, it has http parameters (port).
Typically, parameter: https_fallback_port
If not, then there is no possible http fallback, and it will not use the (forward) proxy.
If you are talking about reverse proxy in front of a server configured with fallback, this is part of the (public) documentation of the IBM Aspera Proxy.
We often find columns like Address, Port in web browser proxy settings. I know when we use proxy to visit a page, the web browser request the web page from the proxy server, but what I want to know is how the whole mechanism works? I have observed that many ISP allow only access to a single IP(of their website) after we exhausted our free data usage. But when we enter the site which we wants to browse in proxy URL and then type in the allowed IP, the site get loaded. How this works?
In general, your browser simply connects to the proxy address & port instead of whatever IP address the DNS name resolved to. It then makes the web request as per normal.
The web proxy reads the headers, uses the "Host" header of HTTP/1.1 to determine where the request is supposed to go, and then makes that request itself relaying all remaining data in both directions.
Proxies will typically also do caching so if another person requests the same page from that proxy, it can just return the previous result. (This is simplified -- caching is a complex topic.)
Since the proxy is in complete control of the connection, it can choose to route the request elsewhere, scrape request and reply data, inject other things (like ads), or block you altogether. Use SSL to protect against this.
Some web proxies are "transparent". They reside on a gateway through which all IP traffic must pass and use the machine's networking stack to redirect outgoing connections to port 80 to a local port instead. It then behaves the same as though a proxy was defined in the browser.
Other proxies, like SOCKS, have a dedicated protocol that allows non-HTTP requests to be made as well.
There are 2 types of HTTP proxies, there are the ones that are reversed and the ones that
are forward.
The web browser uses a forward proxy, basically it is sending all http traffic through the proxy, the proxy will take this traffic out to the internet. Every http packet that comes out from your computer, will be send to the proxy before going to the target site.
The ISP blocking does not work when using a proxy because, every packet that comes out from your machine is pointing to the proxy and not to the targe site. The proxy could be getting internet through another ISP that has no blocks whatsoever.