I am trying to get nginx running as a cache proxy. The server is up and running properly - except it seems its not caching properly.
According to a scan with pingdom:
The following cacheable resources have a short freshness lifetime. Specify
an expiration at least one week in the future for the following resources:
/layout/theme1/css/bootstrap-responsive.css
/layout/theme1/css/bootstrap.min.css
/layout/theme1/css/font-awesome.css
/layout/theme1/css/pages/dashboard.css
/layout/theme1/css/style.css
/layout/theme1/css/theme.css
/layout/theme1/img/logo.png
/layout/theme1/js/bootstrap.js
/layout/theme1/js/chart.min.js
/layout/theme1/js/excanvas.min.js
/layout/theme1/js/jquery.js
My nginx proxy_cache settings are as follows:
Proxy Cache Path Settings:
proxy_cache_path /var/cache/nginx/marketers.coop levels=1:2 keys_zone=marketers.coop:10m inactive=525600m max_size=5120m;
location settings:
location / {
proxy_pass http://192.227.210.138:8080;
proxy_cache cache;
proxy_cache_valid 30d;
proxy_cache_valid 404 30d;
proxy_no_cache $no_cache;
proxy_cache_bypass $no_cache;
proxy_cache_bypass $cookie_session $http_x_update;
location ~* ^.+\.(jpg|jpeg|gif|png|ico|svg|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|odt|ods|odp|odf|tar|wav|bmp|rtf|js|mp3|avi|mpeg|flv|html|htm)$ {
proxy_pass http://192.227.210.138:8080;
proxy_cache cache;
proxy_cache_valid 30d;
proxy_cache_valid 404 30d;
root /home/admin/web/marketers.coop/public_html;
access_log /var/log/apache2/domains/marketers.coop.log combined;
access_log /var/log/apache2/domains/marketers.coop.bytes bytes;
expires 365d;
try_files $uri #fallback;
}
}
As can bee seen both css and js are set to be cached, yet neither the expire of 365 days nor the proxy cache valid for 30 days seems to be having any effect. What am I doing wrong?
Related
I'm using "Filling the Cache Slice-by-Slice" from this article https://www.nginx.com/blog/nginx-caching-guide/
proxy_cache_path /tmp/mycache keys_zone=mycache:10m;
server {
listen 80;
proxy_cache mycache;
slice 1m;
proxy_cache_key $host$uri$is_args$args$slice_range;
proxy_set_header Range $slice_range;
proxy_http_version 1.1;
proxy_cache_valid 200 206 1h;
location / {
proxy_pass http://origin:80;
}
}
nginx is caching video from server, but browser is not caching video. Help me please.
I'm solved problem. I'm added ETag and Last-Modified on backend.
I'm trying to get my head around caching images for my open source image hosting serivce PictShare.
Pictshare has a smart query system where an uploaded image can be in a "virtual subdirectory" that changes the image. For example this is the link to the uploaded stackoverflow logo: https://www.pictshare.net/6cb55fe938.png
I can resize it to 300 width by adding /300/ to the URL before the image name: https://www.pictshare.net/300/6cb55fe938.png
Since I'm dealing with a lot of traffic lately I want my nginx proxy to be able to cache all images in from all virtual sub folders but it's not working. I've read many articles and many stackoverflow posts but no solution worked for me.
So far this my productive vhost file
proxy_cache_path /etc/nginx/cache/pictshare levels=1:2 keys_zone=pictshare:50m max_size=1000m inactive=30d;
proxy_temp_path /etc/nginx/tmp 1 2;
proxy_cache_key "$scheme$request_method$host$request_uri";
proxy_ignore_headers "Set-Cookie";
proxy_hide_header "Set-Cookie";
proxy_buffering on;
server {
...
location / {
proxy_pass http://otherserver/pictshare/;
include /etc/nginx/proxy_params;
location ~* \.(?:jpg|jpeg|gif|png|ico)$ {
expires max;
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
proxy_cache_valid 200 301 302 1y;
proxy_cache pictshare;
proxy_pass http://otherserver/pictshare$request_uri;
}
}
}
The problem is that no files are cached and I see every image request on the proxy destination.
The only way I got it to work was by adding a special location to the host file that has caching explicitly enabled:
location /cached {
proxy_cache_valid 200 1y;
proxy_cache pictshare;
expires 1y;
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
include /etc/nginx/proxy_params;
proxy_pass http://otherserver/pictshare/thumbs/;
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
}
The obvious problem with this solution is that the images are only cached when the request starts with /cached eg: https://www.pictshare.net/cached/6cb55fe938.png
Adding the caching commands to the root directory is no option for me since I don't want the forms and pages to be cached, just the images
Where is my mistake?
proxy_cache_path /etc/nginx/cache/pictshare levels=1:2 keys_zone=my_cache:50m max_size=3g inactive=180m;
proxy_temp_path /etc/nginx/tmp 1 2;
proxy_cache_key "$scheme$request_method$host$request_uri";
location ~* ^.+\.(jpe?g|gif|png|ico|pdf)$ {
access_log off;
include /etc/nginx/proxy.conf;
proxy_pass http://backend;
proxy_cache pictshare;
proxy_cache_valid any 12h;
add_header X-Proxy-Cache $upstream_cache_status;
root /var/www/public_html/cached; }
location / {
include /etc/nginx/proxy.conf;
proxy_pass http://backend;
root /var/www/public_html;
}
nginx first searches for the most specific prefix location given by literal strings regardless of the listed order. In the configuration above the only prefix location is “/” and since it matches any request it will be used as a last resort. Then nginx checks locations given by regular expression in the order listed in the configuration file. The first matching expression stops the search and nginx will use this location. If no regular expression matches a request, then nginx uses the most specific prefix location found earlier.
http://nginx.org/en/docs/http/request_processing.html
How to make a preemptive cache with nginx?
Currently, the cache becomes stale and unloads lots of images at once.
In my http section I have
proxy_cache_path /var/cache/nginx levels=1:1 keys_zone=zone:10m;
In my server configuration I have something like
server {
listen 80 default deferred;
server_name myservername
root /myapp/public;
client_max_body_size 2G;
proxy_cache_bypass $http_pragma;
proxy_cache_valid 200 301 302 304 1M;
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
proxy_cache zone;
gzip_static on;
try_files $uri #app;
location #app {
if ($request_uri ~* "\.(ico|css|js|gif|jpe?g|png)\?[0-9]+$") {
expires max;
break;
}
client_body_buffer_size 32k;
proxy_buffers 8 64k;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://myupstream;
}
}
Well, i have my site.conf file like this:
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cache_key "$scheme$request_method$host$request_uri";
proxy_cache_path /etc/nginx/cache/pag levels=1:2 keys_zone=APP:100m inactive=1m;
proxy_temp_path /etc/nginx/cache/tmp;
add_header X-Cache $upstream_cache_status;
server {
listen 80;
root /etc/nginx/html;
index index.html index.htm;
server_name www.example.com;
error_page 404 /404.html;
location /404.html {
internal;
}
location / {
proxy_pass http://127.0.0.1:8080/;
proxy_cache APP;
proxy_cache_valid 200 1m;
proxy_cache_methods POST;
expires 1m;
}
}
With this configuration, everything (including POST request methods) is cached for 1 min, OK.
What i need? I need that only this pages can be cached:
1) www.example.com
2) www.example.com/index.html
3) www.example.com/test/page.html
4) www.example.com/test/text.txt (this is a file requested by POST thru page.html, and i need it cached also)
5) www.example.com/test/page2.php?var1=val1&var2=val2 (val1 and val2 are dynamics)
My question is: What i have to put in location / to match the 1-5 items? Like this:
location (1-5 items match) {
proxy_pass http://127.0.0.1:8080/;
proxy_cache APP;
proxy_cache_valid 200 1m;
proxy_cache_methods POST;
expires 1m;
}
Other pages (not cached) will be automatically redirected to 127.0.0.1:8080. I know this can be do like this:
location / {
proxy_pass http://127.0.0.1:8080/;
}
NOTE 1: Other PHP pages receive POST|GET request methods, but i don't need it in cache, only aboves.
NOTE: 2 127.0.0.1:8080 is an apache server that runs PHP, so i can request PHP pages.
Since apache runs on the same host, simply serve the html files you do not want cached through nginx. As for the php pages, send the correct expiration headers in your application and everything will work correctly.
I want to fetch a file from the cache conditionally, based on a custom header in the request.
If the X-Proxy header is present in the request, return the file only if it's present in the cache. Otherwise fetch it from the internet if necessary.
Here's my .conf file:
worker_processes 1;
events {
worker_connections 1024;
}
http {
proxy_cache_path /home/nginx/proxy levels=1:2 keys_zone=one:15m inactive=7d max_size=1000m;
proxy_temp_path /home/nginx/temp;
proxy_buffering on;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_set_header Connection "";
proxy_http_version 1.1;
server {
listen 8000;
location / {
root /home/nginx/preload;
try_files /$uri #local #remote;
}
location #local {
internal;
add_header X-Local true;
add_header X-Cache $upstream_cache_status;
proxy_pass http://$http_host$uri$is_args$args;
proxy_cache one;
proxy_cache_key backend$request_uri;
proxy_cache_valid 200 1h;
proxy_cache_use_stale error timeout invalid_header;
}
location #remote {
resolver 8.8.8.8;
add_header X-Remote true;
add_header X-Cache $upstream_cache_status;
if ($http_x_proxy) {
return 404;
}
proxy_pass http://$http_host$uri$is_args$args;
proxy_cache one;
proxy_cache_key backend$request_uri;
proxy_cache_valid 200 1h;
proxy_cache_use_stale error timeout invalid_header;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
The problem is that the try_files directive always passes into my #remote location, even if the fetched file is cached. How do I tell it that the file was found when it returns from #local?
try_files directive only accepts one named location, so apparently it goes for the last one. This blog post proposes a workaround that works in your case. In case you don't won't read the whole post, you can add the following lines at the end of #local block:
proxy_intercept_errors on;
recursive_error_pages on;
error_page 404 = #remote;
and change your try_files to this:
try_files /$uri #local;