I need to put a web proxy in place to log user activity at work after a recent incident. My first thought was Squid proxy but after some research it seems that https requests are a total nightmare. These days more sites are https than http so I need to log both. Can anyone recommend a proxy server or otherwise to pass all http and https requests through to log?
Thanks
Squid can very well handle HTTP as well ass HTTPS traffic. How you should configure squid depends how you want the configure clients (I mean browser).
In general Squid proxy server can be configured to listen for both HTTP and HTTPS traffic on specific port (by default 3128) for squid and clients can be configured manually or using DHCP Option 252 + WPAD (Web Proxy Auto-Discovery Protocol).
Alternately Squid can be configured in transparent mode intercepting the traffic on your network, in this case Squid will listen on different ports for HTTP and HTTPS traffic.
Shahnawaz
Related
I'm trying to set up a proxy server on my local mac.
http - seems to work.
But Safari is not connecting via https.
Did I miss something?
No it doesn't. You need to specify a separate https port and a ssl certificate, as documented in the squid config:
The socket address where Squid will listen for client requests made
over TLS or SSL connections. Commonly referred to as HTTPS.
This is most useful for situations where you are running squid in
accelerator mode and you want to do the TLS work at the accelerator
level.
You may specify multiple socket addresses on multiple lines, each
with their own certificate and/or options.
The tls-cert= option is mandatory on HTTPS ports.
See http_port for a list of modes and options.
http://www.squid-cache.org/Doc/config/https_port/
By design, it is quite hard to intercept https traffic:
When a browser creates a direct secure connection with an origin
server, there are no HTTP CONNECT requests. The first HTTP request
sent on such a connection is already encrypted. In most cases, Squid
is out of the loop: Squid knows nothing about that connection and
cannot block or proxy that traffic.
You also need to load the proxy settings for the browser as a PAC file, otherwise the browsers won't connect or throw a certificate warning:
Chrome The Chrome browser is able to connect to proxies over SSL
connections if configured to use one in a PAC file or command line
switch. GUI configuration appears not to be possible (yet).
More details at
http://dev.chromium.org/developers/design-documents/secure-web-proxy
Firefox The Firefox 33.0 browser is able to connect to proxies over
TLS connections if configured to use one in a PAC file. GUI
configuration appears not to be possible (yet), though there is a
config hack for embedding PAC logic.
There is still an important bug open:
Using a client certificate authentication to a proxy:
https://bugzilla.mozilla.org/show_bug.cgi?id=209312
https://wiki.squid-cache.org/Features/HTTPS
HTTP proxy with SSL and DNS support.
I must be lacking some key concepts about proxy-ing because I cannot grasp this. I am looking to run a simply http or https proxy without interfering with SSL. Simply, a fully transparent proxy that can passthrough all the traffic to the browser connected via HTTP or HTTPS proxy without modifying or intercepting any packets. Not able to find any code online or I'm not using the right keywords.
EX. On the browser adding server.someVPN.com:80 on the HTTP proxy field and as soon as you try to visit a website, it prompts for authentication. Then it works perfectly with any domain, any security, any ssl, no further steps needed. Most VPN providers have this.
How's this possible? it even resolves DNS itself. I thought on transparent proxy the dns relies on the client. Preferably looking for a nodeJS solution but any lang works.
Please don't propose any solutions such as SOCKS5 or sock forwarding or DNS overriding or CA based MITM. According to HTTP 1.1 which supports 'CONNECT' this should be easy.
Not looking to proxy specific domains, looking for an all inclusive solution just like most VPN Providers providers.
----Found the answer too quickly, feel free to delete this post/question admins.
The way it works is that the browser knows it is talking to a proxy server, so for example if the browser want to connect to htttp://www.example.com it sends a CONNECT www.example.com:443 HTTP/1.1 to the proxy server, the proxy server resolves wwww.example.com via DNS and then opens a TCP connection to wwww.example.com port 443 and proxies the TCP stream transparently to the client.
I don't know any solution for nodejs. Common proxy servers include Squid, Privoxy and Apache Traffic Server
See also: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT
Found the solution right after I asked...
This module works perfectly https://github.com/mpangrazzi/harrier
Does exactly what I was asking for.
Is there any different between HTTP tunnel and HTTP proxy?
In some software I see two fields for proxy either HTTP and HTTP tunnel.
I see the traffic of some http proxy but I cant understand the different between http proxy and http tunnel.
The terms are often intermixed, tunnel providers are called proxies.
Originally, tunneling is the technique of using one protocol to transport data inside another protocol.
A proxy (as in proxy representative)
A proxy acts as an intermediary. It will hide your IP address from the destination (unless it adds it in a HTTP header field such as "Forward"). A proxy uses the same protocol throughout, it can alter the network flow, do caching or security scanning etc. So it's more of an extra hop on the way to the destination.
For example you can use a SOCKS proxy as a HTTP tunnel, i.e. you transport HTTP over it. This is due to the fact that SOCKS is a protocol that is designed to tunnel IP packets.
To add to the confusion, you can use a HTTP proxy to transport some other protocols such as FTP.
A good example for a tunnel is a VPN. Tunnels are often used to evade censorship or firewall rules blocking traffic.
If UDP is blocked, i need to enable HTTP(s) fallback with aspera connect plugin thru HAProxy. Has anyone successfully done this ? I have read all the documentation but there is nothing that describes the HAProxy use case
I assume you talk about a client side forward proxy.
If you use the IBM Aspera Connect Client, it comes with HTTP (forward) proxy configuration.
http://download.asperasoft.com/download/docs/connect/3.9.6/user_osx/webhelp/index.html#dita/network_environment.html
This will work when the client falls back to HTTP.
Note that the client will fallback to HTTP only if this feature is enabled on the server side.
You can see that by examining the parameters provided to the client by the browser (put in dev mode). In that case, it has http parameters (port).
Typically, parameter: https_fallback_port
If not, then there is no possible http fallback, and it will not use the (forward) proxy.
If you are talking about reverse proxy in front of a server configured with fallback, this is part of the (public) documentation of the IBM Aspera Proxy.
My clients are using a proxy server on their corporation,and I want to set up dev environment for testing development related to proxy issues.
So, I want to set up a proxy server which blocks all port 80 requests, unless the request is requested by the proxy server.
This is what I tried:
Installing fiddler2 on port 8888.
Setting up two rules on windows firewall:
Block all port 80 requests.
Allow all requests from fiddler exe.
Then, I opened FF and changed the proxy server to be 127.0.0.1:8888.
Unfortunately, the requests from the fiddler are still blocked.
What am I doing wrong?
Is there other program which do that easier? (Tried also with CCProxy without success).
Found the answer this great post:
Block all the outbound connections on the firewall.
Allow request from fiddler.exe