Can I use varnish cache with https or will this have little to no performance gain? What are the pros and cons? I've set up my vcl for http only. I want to try this with https now.
I've read this but it's from 2011:
https://www.varnish-cache.org/docs/trunk/phk/ssl.html
Varnish in itself does not support SSL and is very unlikely to do so in the overseeable future.
To use SSL and still be able to cache with varnish you have to terminate the SSL before the request is sent to varnish. This can be done efficiently by for instance HAProxy or Nginx.
To find out exactly how to configure this; a simple google search for ssl termination haproxy/nginx will yield more than enough results-
You set the X-Forwarded-For headers in HAProxy. If there is already set an X-Forwarded-For header other reverse proxies will always just add their own to it, the left-most or first address is the source address. You don't have to think about that, anything that reads and uses X-Forwarded-For headers will sort that out automagically.
You also want to set the X-Forwarded-Proto so you can do all sorts of magic in Varnish, like redirecting traffic not using TLS without hitting your backend servers and separate the caches, as Varnish doesn't talk TLS, which can lead to some interesting results, like images not being served up because they are requested over HTTP when the page is served over HTTPS.
Side question, are you using HAProxy to actually load balance between multiple backends? If not, why not just terminate the TLS connection in Apache, send that to Varnish and then back to Apache again?
Related
I have created a listener on the LB (load balancer) with rules so that requests to different subdomains. I have set a CNAME for each subdomain in Cloudflare.
The problem is when I try to use Proxy feature, when I turn it off, my page works without a problem, but when i turn it on, it results in a time out.
There is a way to use Proxy feature with an LB?
The very simple answer is yes, this is very possible to do.
The longer answer is that a timeout indicates that some change in the request by Cloudflare creates a timeout for your AWS servers. Just a couple of examples could be:
A firewall rule that times out Cloudflare's IPs
Servers are not respecting X-Forwarded-For header and all requests from a small group of IPs are messing with application logic
It would help to know if the requests are reaching your load balancer and furthermore if the servers behind the LB are receiving the requests.
How can I detect, server side, if the client supports SPDY?
I want my website to be as fast as possible. Here's my thinking: (Note: my website does not need to transmit sensitive data.) If a browser connects to my website with HTTPS but it doesn't support SPDY , it'll be a waste. Unnecessary overhead due to HTTPS right? On the other hand, if the browser connects via HTTP and does support SPDY, that will be a missed opportunity.
It looks like NPN is the technology that the client and server use to negotiate on SPDY or not. That happens in the web server, before it ever hits my application code, right? I suppose then what I'd really need is a modified version of NPN (not even sure if that's really its own thing outside of SPDY) or mod_spdy. Ideally such a version would have an option called use_spdy_if_available_otherwise_redirect_to_http. :-)
Oh, and if all this isn't complicated enough, I'm currently using Cloudflare's CDN service. I'm pretty sure I have no recourse to modify how they operate in this regard, and thus have no chance, right?
All data is sensitive: the sites you visit, the pages you've viewed, etc. By aggregating this data across many pages, you can infer a lot about the user: their intent, interests, and so on. Hence, we need HTTPS everywhere. For more, see our Google I/O talk [1] on the subject.
In terms of detecting SPDY support, yes you want to use NPN/ALPN (ALPN is a successor [2]). The client sends a ProtocolNameList in their handshake, which advertises which protocols they support. Most servers will use this to auto-negotiate SPDY, but if you want to control this decision yourself, you'd have to modify your server implementation to invoke some sort of callback when the secure handshake is being performed.
That said, given what I said earlier about HTTPS everywhere, I would advise you against this altogether. Use HTTPS everywhere and let the browser and server auto-negotiate SPDY if its supported.
[1] https://www.youtube.com/watch?v=cBhZ6S0PFCY
[2] http://chimera.labs.oreilly.com/books/1230000000545/ch04.html#ALPN
I agree with igrigorik's advice: do not redirect users from HTTPS to HTTP. That's just not cool. Regardless, I had this detection problem today and my answer's below.
In NGINX (I'm running 1.7.7), the $spdy variable will be set if the client connects with SPDY connection. Otherwise, $spdy will not have a value. For example, I'm passing a custom URL parameter to a php script:
server {
listen 443 ssl spdy;
...
...
# add SPDY rewrite param
if ($spdy) {
rewrite ^/detect-spdy.js /detect-spdy.js.php?spdy=$spdy last;
}
# fallback to non-SPDY rewrite
rewrite ^/detect-spdy.js /detect-spdy.js.php last;
# add response header if needed later
add_header x-spdy $spdy;
}
How we can enable squid to cache web content (let says from firefox) for SSL connection, i mean for https URLs?
Actually SQUID can be used to access HTTPS traffic - it is in essence a man-in-the-middle attack - and there are caveats:
See: http://wiki.squid-cache.org/Features/SslBump
I have not tried cacheing this data yet, so can't say that it will work with absolute certainty. If/when I do, I'll update this post.
SSL encrypts the traffic between server and client so it cannot be read by a middle man. When using Squid as a proxy it simply cannot see the actual content in the traffic and therefore it has no means of caching it. The SSL traffic is just random bits that look different each time even if the same content is transferred multiple times and that is how encryption should work. It simply cannot be cached.
I have no problems getting Firefox (version 23.0.1 on Windows) to route SSL traffic via Squid. In Firefox Network Connection settings I just point SSL Proxy and HTTP Proxy to the same Squid installation.
After that I can successfully access https URLs in Firefox and in Squid's access_log I see entries like these:
1379660084.878 115367 10.0.0.205 TCP_MISS/200 6581 CONNECT www.gravatar.com:443 - DIRECT/68.232.35.121 -
Do you have any details about how it doesn't work for you? Squid has quite complicated possibilities to deny and allow certain types of traffic, so it is possible there is a configuration issue in Squid. Do you get any error messages in Squid's logfiles?
I've configured my system with NginX listening on port 80, serving static content and proxying dynamic requests to a backend server.
I can configure NginX to cache content generated by the backend, but I want this cached content be served only when the Backend responds with an error http 5xx, or when it's totally down.
We tried the proxy_cache_use_stale option with max-age of 1 second, it worked but it has one negative side.. which is simply dozens of requests being served from cache during this 1 second cache-aged-content. These requests served from cache will miss further Backend processing (Stats for example).
We can only afford to live with this negativity IF the backend was down,
Thus, the cache will act as a backup or a failover solution. But as long as the backend is up and responding, no requests should be served from cache.
I would appreciate any hints
Take a look at proxy_cache_use_stale
proxy_intercept_errors might be what you're looking for.
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_intercept_errors
I have a reverse proxy that does HTTPS on the outside, but HTTP on the inside.
This means that by default in-app URLs will have HTTP as the scheme, as this is the way it's being contacted.
How can the proxy tell the backend that HTTPS should be used?
The proxy can add extra (or overwrite) headers to requests it receives and passes through to the back-end. These can be used to communicate information to the back-end.
So far I've seen a couple used for forcing the use of https in URL scheme:
X-Forwarded-Protocol: https
X-Forwarded-Ssl: on
X-Url-Scheme: https
And wikipedia also mentions:
# a de facto standard:
X-Forwarded-Proto: https
# Non-standard header used by Microsoft applications and load-balancers:
Front-End-Https: on
This what you should add to the VirtualHost on apache: other proxies should have similar functionality
RequestHeader set X-FORWARDED-PROTOCOL https
RequestHeader set X-Forwarded-Ssl on
# etc.
I think it's best to set them all, or set one that works and remove the other known ones. To prevent evil clients messing with them.
It took me several hours of googling to find the magic setting for my environment. I have a SSL httpd Apache reverse proxy in front of a jetty app server and an apache2 http server. This answer actually gave me the information that worked. For me, adding:
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
to the site conf file was enough for the destination to use https instead of http as the protocol when building links in the response. I tried the X-FORWARDED-PROTOCOL above, but that didn't work. Hopefully this will help in future Google searches!