There are several different approaches to security for web proxying:
No security. Client connects to proxy with HTTP, makes GET request (proxy probably uses HTTP to reach destination).
HTTPS connection to proxy, then GET request. Encryption only between proxy and client.
HTTP to proxy, CONNECT tunnel to destination (host named in clear-text), letting client HTTPS to destination. Security-wise, seems equivalent to direct HTTPS.
HTTPS to proxy, CONNECT tunnel to destination, then HTTPS to destination. (E.g., so that not even the destination host name metadata can be eavesdropped on an untrusted local network like public wifi.)
My question is about the final case. Is there any standard or RFC that describes it?
Does/must the content get double-encrypted or not? That is, on the client side, should the proxy simply forward the raw data stream (without manipulation) after the CONNECT is established, or should it still continue applying its (now-redundant) secure wrapper?
Related
Our environment requires that all requests to the public internet go through a proxy that uses HTTP tunneling (HTTP Connect). I have a service from a vendor that is not able to use an HTTP Connect proxy.
My plan is to set up a transparent proxy that the service can send traffic to, which will redirect traffic to the HTTP Connect proxy.
service -> transparent proxy -> http connect proxy -> public internet
Is it possible to redirect this traffic without decrypting and re-encrypting the HTTPS request? I won't be able to install any custom certs on the initial service to allow this (and I don't want to view the contents of the HTTP messages anyway). Also, are there any proxies that are made specifically to do this, or will I need to write my own?
I am trying to implement a Socks5 server that could relay both HTTP and HTTPS traffic.
As the RFC1928 mentions, the following steps to establish a connection and forward the data must be taken :
Client sends a greeting message to the proxy.
Client & proxy authentication (assuming it is successful).
Client sends a request to the proxy to connect to the destination.
The proxy connects to the destination and sends back a response to the client to indicate a successful open tunnel.
The proxy reads the data from the client and forwards it to the destination.
The proxy reads the data from the destination and forwards it to the client.
So far, the proxy works as it should. It is able to relay HTTP traffic using its basic data forwarding mechanism. However, any request from the client to an HTTPS website will be aborted because of SSL/TLS encryption.
Is there another sequence/steps that should be followed to be able to handle SSL/TLS (HTTPS) traffic?
The sequence you have described is correct, even for HTTPS. When the client wants to send a request to an HTTPS server through a proxy, it will request the proxy to connect to the target server's HTTPS port, and then once the tunnel is established, the client will negotiate a TLS handshake with the target server, then send an (encrypted) HTTP request and receive an (encrypted) HTTP response. The tunnel is just a passthrough of raw bytes, the proxy has no concept of any encryption between the client and server. It doesn't care what the bytes represent, its job is just to pass them along as-is.
Is there any different between HTTP tunnel and HTTP proxy?
In some software I see two fields for proxy either HTTP and HTTP tunnel.
I see the traffic of some http proxy but I cant understand the different between http proxy and http tunnel.
The terms are often intermixed, tunnel providers are called proxies.
Originally, tunneling is the technique of using one protocol to transport data inside another protocol.
A proxy (as in proxy representative)
A proxy acts as an intermediary. It will hide your IP address from the destination (unless it adds it in a HTTP header field such as "Forward"). A proxy uses the same protocol throughout, it can alter the network flow, do caching or security scanning etc. So it's more of an extra hop on the way to the destination.
For example you can use a SOCKS proxy as a HTTP tunnel, i.e. you transport HTTP over it. This is due to the fact that SOCKS is a protocol that is designed to tunnel IP packets.
To add to the confusion, you can use a HTTP proxy to transport some other protocols such as FTP.
A good example for a tunnel is a VPN. Tunnels are often used to evade censorship or firewall rules blocking traffic.
I'm building a WebClient library. Now I'm implementing a proxy feature, so I am making some research and I saw some code using the CONNECT method to request a URL.
But checking it within my web browser, it doesn't use the CONNECT method but calls the GET method instead.
So I'm confused. When I should use both methods?
TL;DR a web client uses CONNECT only when it knows it talks to a proxy and the final URI begins with https://.
When a browser says:
CONNECT www.google.com:443 HTTP/1.1
it means:
Hi proxy, please open a raw TCP connection to google; any following
bytes I write, you just repeat over that connection without any
interpretation. Oh, and one more thing. Do that only if you talk to
Google directly, but if you use another proxy yourself, instead you
just tell them the same CONNECT.
Note how this says nothing about TLS (https). In fact CONNECT is orthogonal to TLS; you can have only one, you can have other, or you can have both of them.
That being said, the intent of CONNECT is to allow end-to-end encrypted TLS session, so the data is unreadable to a proxy (or a whole proxy chain). It works even if a proxy doesn't understand TLS at all, because CONNECT can be issued inside plain HTTP and requires from the proxy nothing more than copying raw bytes around.
But the connection to the first proxy can be TLS (https) although it means a double encryption of traffic between you and the first proxy.
Obviously, it makes no sense to CONNECT when talking directly to the final server. You just start talking TLS and then issue HTTP GET. The end servers normally disable CONNECT altogether.
To a proxy, CONNECT support adds security risks. Any data can be passed through CONNECT, even ssh hacking attempt to a server on 192.168.1.*, even SMTP sending spam. Outside world sees these attacks as regular TCP connections initiated by a proxy. They don't care what is the reason, they cannot check whether HTTP CONNECT is to blame. Hence it's up to proxies to secure themselves against misuse.
A CONNECT request urges your proxy to establish an HTTP tunnel to the remote end-point.
Usually is it used for SSL connections, though it can be used with HTTP as well (used for the purposes of proxy-chaining and tunneling)
CONNECT www.google.com:443
The above line opens a connection from your proxy to www.google.com on port 443.
After this, content that is sent by the client is forwarded by the proxy to www.google.com:443.
If a user tries to retrieve a page http://www.google.com, the proxy can send the exact same request and retrieve response for him, on his behalf.
With SSL(HTTPS), only the two remote end-points understand the requests, and the proxy cannot decipher them. Hence, all it does is open that tunnel using CONNECT, and lets the two end-points (webserver and client) talk to each other directly.
Proxy Chaining:
If you are chaining 2 proxy servers, this is the sequence of requests to be issued.
GET1 is the original GET request (HTTP URL)
CONNECT1 is the original CONNECT request (SSL/HTTPS URL or Another Proxy)
User Request ==CONNECT1==> (Your_Primary_Proxy ==CONNECT==> AnotherProxy-1 ... ==CONNECT==> AnotherProxy-n) ==GET1(IF is http)/CONNECT1(IF is https)==> Destination_URL
As a rule of thumb GET is used for plain HTTP and CONNECT for HTTPS
There are more details though so you probably want to read the relevant RFC-s
http://www.ietf.org/rfc/rfc2068.txt
http://www.ietf.org/rfc/rfc2817.txt
The CONNECT method converts the request connection to a transparent TCP/IP tunnel, usually to facilitate SSL-encrypted communication (HTTPS) through an unencrypted HTTP proxy.
My websocket server listens on port 8080 with no proxy.
Most of the time I'm getting requests with the Upgrade Websocket header and it works fine.
Sometimes I'm getting HTTP CONNECT requests.
Is this a valid request?
Does it means that there is a proxy server between the client and the server?
How my server is suppose to respond to the CONNECT request?
Thanks
You are getting CONNECT requests because you are likely to have configured your browser to use a proxy. If you directed your browser to use port 8080 on your local IP address, it will assume there is a proxy and that means when you ask for a secure connection, the browser leads with CONNECT.
You will need to add support for SSL/TLS tunnelling to your server to deal with this.