I have been Googling aggressively, but without luck.
I'm using Varnish with great results, but I would like to host multiple websites on a single server (Apache), without Varnish caching all of them.
Can I specify what websites by URL to cache?
Thanks
(edited after comment) It's req.http.host, so in your vcl file (e.g. default.vcl) do:
sub vcl_recv {
# dont cache foo.com or bar.com - optional www
if (req.http.host ~ "(www\.)?(foo|bar)\.com") {
pass;
}
# cache foobar.com - optional www
if (req.http.host ~ "(www\.)?foobar\.com") {
lookup;
}
}
And in varnish3-vcl:
sub vcl_recv {
# dont cache foo.com or bar.com - optional www
if (req.http.host ~ "(www\.)?(foo|bar)\.com") {
return(pass);
}
# cache foobar.com - optional www
if (req.http.host ~ "(www\.)?foobar\.com") {
return(lookup);
}
}
Yes,
in vcl_recv you just match the hosts that you would like not to cache and pass them. Something like this (untested):
vcl_recv {
# dont cache foo.com or bar.com - optional www
if (req.host ~ "(www)?(foo|bar).com") {
return(pass);
}
}
For Varnish 4
replace lookup with hash
default.vcl:
sub vcl_recv {
# dont cache foo.com or bar.com - optional www
if (req.http.host ~ "(www\.)?(foo|bar)\.com") {
return(pass);
}
# cache foobar.com - optional www
if (req.http.host ~ "(www\.)?foobar\.com") {
return(hash);
}
}
Related
After reading the varnish 4.1 document, I found that I can set 'req.hash_always_miss' to true to force a 'miss' while looking up cache and fetch the latest content in backend.
But I still can not figure out how to configure the vcl file.
below is the config file in my server.
backend default {
.host = "127.0.0.1";
.port = "8080";
}
acl purgers {
"localhost";
}
sub vcl_recv {
if (req.method == "PURGE") {
if (!client.ip ~ purgers) {
return(synth(405,"Not allowed."));
}
set req.hash_always_miss = true;
}
}
sub vcl_backend_response {
unset beresp.http.set-cookie;
set beresp.ttl = 1d;
}
while backend content change, I want to refresh the varnish cache before it get expired.
so I call curl -X PURGE http://localhost:6081/path/to/my/content
I do get the latest content as a response, but I found that varnish cache is still not get refreshed. Can someone tell me why? thanks
We are trying to get Varnish to cache range requests. We are using Varnish 4.0.
We the following configurations
vcl 4.0;
import std;
# Default backend definition. Set this to point to your content server.
backend default {
.host = "194.142.x.x";
.port = "8008";
}
sub vcl_recv {
if (req.url ~ "(?i)\.(png|avi|mkv|mp4)(\?[a-z0-9]+)?$") {
unset req.http.Cookie;
}
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
set req.http.host = regsub(req.http.host, "v\.","\rms\.");
std.log("REWRITED TO"+req.http.host+" "+req.url);
}
sub vcl_backend_response {
# Happens after we have read the response headers from the backend.
#
# Here you clean the response headers, removing silly Set-Cookie headers
# and other mistakes your backend does.
}
sub vcl_deliver {
# Happens when we have all the pieces we need, and are about to send the
# response to the client.
#
vcl 4.0;
import std;
# Default backend definition. Set this to point to your content server.
backend default {
.host = "194.142.x.x";
.port = "8008";
}
sub vcl_recv {
if (req.url ~ "(?i)\.(png|avi|mkv|mp4)(\?[a-z0-9]+)?$") {
unset req.http.Cookie;
}
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
set req.http.host = regsub(req.http.host, "v\.","\rms\.");
std.log("REWRITED TO"+req.http.host+" "+req.url);
}
sub vcl_backend_response {
# Happens after we have read the response headers from the backend.
#
# Here you clean the response headers, removing silly Set-Cookie headers
# and other mistakes your backend does.
}
sub vcl_deliver {
# Happens when we have all the pieces we need, and are about to send the
# response to the client.
#
how ever range requests are taking too long to be served so we feel its not being cached as the original server is hit.
Can you solve this...?
I'm pretty sure the block config:
if (req.url ~ "(?i)\.(png|avi|mkv|mp4)(\?[a-z0-9]+)?$") {
unset req.http.Cookie;
}
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
set req.http.host = regsub(req.http.host, "v\.","\rms\.");
std.log("REWRITED TO"+req.http.host+" "+req.url);
Are in wrong position, needs to be in vcl_backend_response not in vcl_recv
Caching partial objects with varnish 4.0
sub vcl_recv {
if (req.http.Range ~ "bytes=") {
set req.http.x-range = req.http.Range;
}
}
sub vcl_hash {
if (req.http.x-range ~ "bytes=") {
hash_data(req.http.x-range);
unset req.http.Range;
}
}
sub vcl_backend_fetch {
if (bereq.http.x-range) {
set bereq.http.Range = bereq.http.x-range;
}
}
sub vcl_backend_response {
if (bereq.http.x-range ~ "bytes=" && beresp.status == 206) {
set beresp.ttl = 10m;
set beresp.http.CR = beresp.http.content-range;
}
}
sub vcl_deliver {
if (resp.http.CR) {
set resp.http.Content-Range = resp.http.CR;
unset resp.http.CR;
}
}
I have a website www.mysite.com running behind a load balancer. There are two servers in the load balancer cluster. Each runs Varnish 3.0 and Apache/PHP (I know varnish could load balance for me - but we have a preference for a different LB tech).
Every now and again I need to purge an URL or two...
In my VCL I have 127.0.0.1 as a trusted URL for PURGEs. And a standard purge config:
vcl_recv:
....
if (req.request == "PURGE") {
# Allow requests from trusted IPs to purge the cache
if (!client.ip ~ trusted) {
error 405 "Not allowed.";
}
return(lookup); # #see vcl_hit;
}
...
sub vcl_hit {
if (req.request == "PURGE") {
purge;
error 200 "Purged (via vcl_hit)";
}
if (!(obj.ttl > 0s)) {
return (pass);
}
return (deliver);
}
sub vcl_miss {
if (req.request == "PURGE"){
purge;
error 404 "Not in Cache";
}
return (fetch);
}
Now from a shellscript I want to invalidate an URL.
curl -X PURGE http://127.0.0.1/product-47267.html
Doesnt work, but
curl -X PURGE http://www.mysite.com/product-47267.html
Does work. Problem here is - I need to invalidate on each local machine in cluster - not have the request go out and back in via the load balancer (because I dont know which machine will take the PURGE).
Hope this makes sense
LW
You need to connect to localhost but Varnish still need to know which Host you want to PURGE.
I'm not sure but try something like the following :
curl -X PURGE -H "Host: www.mysite.com" http://127.0.0.1/product-47267.html
I have a varnish 3.xx server which currently works.
Varnish is caching the login page of my site.
www.mysite.com/staff
but it may have different urls depending on the staff members link, for example
www.mysite.com/staff/index.php?/Tickets/Ticket/View/222200
My varnish config file is set as follows to exclude caching the staff page, but it is not working, as it is caching the login page and it is will not login untill i restart varnish to clear it's cache.
sub vcl_recv {
# Allow purge only from internal users
if (req.request == "PURGE") {
if (!client.ip ~ internal_net) {
error 405 "Not allowed.";
}
return (lookup);
# Exclude the following
if (req.url ~ "^/login\.php" ||
req.url ~ "^/search\.php" ||
req.url ~ "^/admin(.*)" ||
req.url ~ "^/admin(.*)" ||
req.url ~ "^/search(.*)" ||
req.url ~ "^/visitor(.*)" ||
req.url ~ "^/staff(.*)" ||
req.url ~ "^/staff\.php"
) {
return(pass);
}
if (req.http.cookie ~ "vb(.*)" ||
req.http.cookie ~ "bb(.*)" ||
req.http.cookie ~ "SWIFT_(.*)" ||
req.url ~ "\?(.*\&)?s=[a-fA-F0-9]{32}(\&|$)" ||
req.http.cookie ~ "bb_password") {
return(pass);
} else {
unset req.http.cookie;
}
}
Do you perhaps have another method to exclude and entire directory from being cached?
IE: everything from /staff no matter what the suffix is after that must not be cached
The exclusion should work perfectly the way you have implemented it. However if the code you pasted is your actual VCL you have an open if() statement in the PURGE section.
sub vcl_recv {
# Allow purge only from internal users
if (req.request == "PURGE") {
if (!client.ip ~ internal_net) {
error 405 "Not allowed.";
}
return (lookup);
# Exclude the following
should read
sub vcl_recv {
# Allow purge only from internal users
if (req.request == "PURGE") {
if (!client.ip ~ internal_net) {
error 405 "Not allowed.";
}
return (lookup);
}
# Exclude the following
Varnish should not accept invalid VCL though, so if the error does not exist in your actual VCL, please update the question with your entire VCL.
We used to have a caching proxy setup using a very early version of Varnish (0.5ish, I think) which used the 'restart' action to send requests to a second backend in the case of a 404 on the first.
The new version of Varnish doesn't seem to support this - the 'restart' action no longer seems to be supported, and the 'req.restarts' variable is no longer recognised. Is such behaviour possible?
The documentation seems to be out of date, as do many of the online examples. man 7 vcl seems to reflect current behaviour though.
If it's not possible with Varnish, can you suggest another solution?
Here are the relevant bits of our old Varnish config:
sub vcl_recv {
# remove cookies
remove req.http.Cookie;
if (req.restarts == 0) {
set req.backend = backend1;
} else if (req.restarts == 1) {
set req.backend = backend2;
}
# remove any query strings
set req.url = regsub(req.url, "\?.*", "");
# force lookup even when cookies are present
if (req.request == "GET" && req.http.cookie) {
lookup;
}
}
sub vcl_fetch {
# we might set a cookie from the Rails app
remove obj.http.Set-Cookie;
# force minimum ttl of 1 year
if (obj.ttl < 31536000s) {
set obj.ttl = 31536000s;
}
if (obj.status != 200 && obj.status != 302) {
restart;
}
}
It seems this behaviour has been reinstated in more recent versions of Varnish.