Varnish.4.x not caching range requests - caching

We are trying to get Varnish to cache range requests. We are using Varnish 4.0.
We the following configurations
vcl 4.0;
import std;
# Default backend definition. Set this to point to your content server.
backend default {
.host = "194.142.x.x";
.port = "8008";
}
sub vcl_recv {
if (req.url ~ "(?i)\.(png|avi|mkv|mp4)(\?[a-z0-9]+)?$") {
unset req.http.Cookie;
}
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
set req.http.host = regsub(req.http.host, "v\.","\rms\.");
std.log("REWRITED TO"+req.http.host+" "+req.url);
}
sub vcl_backend_response {
# Happens after we have read the response headers from the backend.
#
# Here you clean the response headers, removing silly Set-Cookie headers
# and other mistakes your backend does.
}
sub vcl_deliver {
# Happens when we have all the pieces we need, and are about to send the
# response to the client.
#
vcl 4.0;
import std;
# Default backend definition. Set this to point to your content server.
backend default {
.host = "194.142.x.x";
.port = "8008";
}
sub vcl_recv {
if (req.url ~ "(?i)\.(png|avi|mkv|mp4)(\?[a-z0-9]+)?$") {
unset req.http.Cookie;
}
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
set req.http.host = regsub(req.http.host, "v\.","\rms\.");
std.log("REWRITED TO"+req.http.host+" "+req.url);
}
sub vcl_backend_response {
# Happens after we have read the response headers from the backend.
#
# Here you clean the response headers, removing silly Set-Cookie headers
# and other mistakes your backend does.
}
sub vcl_deliver {
# Happens when we have all the pieces we need, and are about to send the
# response to the client.
#
how ever range requests are taking too long to be served so we feel its not being cached as the original server is hit.

Can you solve this...?
I'm pretty sure the block config:
if (req.url ~ "(?i)\.(png|avi|mkv|mp4)(\?[a-z0-9]+)?$") {
unset req.http.Cookie;
}
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
set req.http.host = regsub(req.http.host, "v\.","\rms\.");
std.log("REWRITED TO"+req.http.host+" "+req.url);
Are in wrong position, needs to be in vcl_backend_response not in vcl_recv

Caching partial objects with varnish 4.0
sub vcl_recv {
if (req.http.Range ~ "bytes=") {
set req.http.x-range = req.http.Range;
}
}
sub vcl_hash {
if (req.http.x-range ~ "bytes=") {
hash_data(req.http.x-range);
unset req.http.Range;
}
}
sub vcl_backend_fetch {
if (bereq.http.x-range) {
set bereq.http.Range = bereq.http.x-range;
}
}
sub vcl_backend_response {
if (bereq.http.x-range ~ "bytes=" && beresp.status == 206) {
set beresp.ttl = 10m;
set beresp.http.CR = beresp.http.content-range;
}
}
sub vcl_deliver {
if (resp.http.CR) {
set resp.http.Content-Range = resp.http.CR;
unset resp.http.CR;
}
}

Related

Changing default_ttl on Varnish 4 vcl file

I am willing to change the default ttl value from the configuration file and not through the varnishd.
sub vcl_backend_response {
...
set beresp.ttl = 24h;
...
}

Leverage Browser Caching in PageSpeed Insights with Varnish & Nginx

Does anyone know how to remove the Leverage Browser Caching message from PageSpeed Insights using Varnish Cache and Nginx?
Google PageSpeed Image
I've tried adding the location ~*... block to the server block but that crashed the site. I think I'm missing a Varnish setting but can't find it.
Thanks in advance!
To override these headers and still put the element into cache for 2 minutes, the following configuration may be used:
sub vcl_fetch {
if (beresp.ttl < 120s) {
set beresp.ttl = 120s;
# Set the clients TTL on this object
set beresp.http.Cache-Control = "max-age=120";
}
}
Or, in Varnish 4.0 terms:
sub vcl_backend_response {
if (beresp.ttl < 120s) {
set beresp.ttl = 120s;
unset beresp.http.Cache-Control;
set beresp.http.Cache-Control = "max-age=120";
}
}

How to override stale cache while cache is not expired in varnish

After reading the varnish 4.1 document, I found that I can set 'req.hash_always_miss' to true to force a 'miss' while looking up cache and fetch the latest content in backend.
But I still can not figure out how to configure the vcl file.
below is the config file in my server.
backend default {
.host = "127.0.0.1";
.port = "8080";
}
acl purgers {
"localhost";
}
sub vcl_recv {
if (req.method == "PURGE") {
if (!client.ip ~ purgers) {
return(synth(405,"Not allowed."));
}
set req.hash_always_miss = true;
}
}
sub vcl_backend_response {
unset beresp.http.set-cookie;
set beresp.ttl = 1d;
}
while backend content change, I want to refresh the varnish cache before it get expired.
so I call curl -X PURGE http://localhost:6081/path/to/my/content
I do get the latest content as a response, but I found that varnish cache is still not get refreshed. Can someone tell me why? thanks

Varnish: cache only specific domain

I have been Googling aggressively, but without luck.
I'm using Varnish with great results, but I would like to host multiple websites on a single server (Apache), without Varnish caching all of them.
Can I specify what websites by URL to cache?
Thanks
(edited after comment) It's req.http.host, so in your vcl file (e.g. default.vcl) do:
sub vcl_recv {
# dont cache foo.com or bar.com - optional www
if (req.http.host ~ "(www\.)?(foo|bar)\.com") {
pass;
}
# cache foobar.com - optional www
if (req.http.host ~ "(www\.)?foobar\.com") {
lookup;
}
}
And in varnish3-vcl:
sub vcl_recv {
# dont cache foo.com or bar.com - optional www
if (req.http.host ~ "(www\.)?(foo|bar)\.com") {
return(pass);
}
# cache foobar.com - optional www
if (req.http.host ~ "(www\.)?foobar\.com") {
return(lookup);
}
}
Yes,
in vcl_recv you just match the hosts that you would like not to cache and pass them. Something like this (untested):
vcl_recv {
# dont cache foo.com or bar.com - optional www
if (req.host ~ "(www)?(foo|bar).com") {
return(pass);
}
}
For Varnish 4
replace lookup with hash
default.vcl:
sub vcl_recv {
# dont cache foo.com or bar.com - optional www
if (req.http.host ~ "(www\.)?(foo|bar)\.com") {
return(pass);
}
# cache foobar.com - optional www
if (req.http.host ~ "(www\.)?foobar\.com") {
return(hash);
}
}

Using a second backend with Varnish 1.0.3-2 in case of 404 from first backend

We used to have a caching proxy setup using a very early version of Varnish (0.5ish, I think) which used the 'restart' action to send requests to a second backend in the case of a 404 on the first.
The new version of Varnish doesn't seem to support this - the 'restart' action no longer seems to be supported, and the 'req.restarts' variable is no longer recognised. Is such behaviour possible?
The documentation seems to be out of date, as do many of the online examples. man 7 vcl seems to reflect current behaviour though.
If it's not possible with Varnish, can you suggest another solution?
Here are the relevant bits of our old Varnish config:
sub vcl_recv {
# remove cookies
remove req.http.Cookie;
if (req.restarts == 0) {
set req.backend = backend1;
} else if (req.restarts == 1) {
set req.backend = backend2;
}
# remove any query strings
set req.url = regsub(req.url, "\?.*", "");
# force lookup even when cookies are present
if (req.request == "GET" && req.http.cookie) {
lookup;
}
}
sub vcl_fetch {
# we might set a cookie from the Rails app
remove obj.http.Set-Cookie;
# force minimum ttl of 1 year
if (obj.ttl < 31536000s) {
set obj.ttl = 31536000s;
}
if (obj.status != 200 && obj.status != 302) {
restart;
}
}
It seems this behaviour has been reinstated in more recent versions of Varnish.

Resources