I'm using Nginx + uWsgi + web2py framework, and I want to make Nginx to cache the HTML responses generated by web2py.
The HTML headers generated by web2py are these:
Cache-Control:max-age=300, s-maxage=300, public
Connection:keep-alive
Content-Length:147
Content-Type:text/html; charset=utf-8
Date:Mon, 27 Mar 2017 16:27:54 GMT
Expires:lun, 27 mar 2017 16:32:54 GMT
Server:Rocket 1.2.6 Python/2.7.6
X-Powered-By:web2py
Those are the ones served directly with the web2py embedded server.
The same request served with nginx and uwsgi (without any cache configuration) produces these headers:
Cache-Control:max-age=300, s-maxage=300, public
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/html; charset=utf-8
Date:Mon, 27 Mar 2017 16:31:09 GMT
Expires:lun, 27 mar 2017 16:36:09 GMT
Server:nginx
Transfer-Encoding:chunked
Vary:Accept-Encoding
X-Powered-By:web2py
Now, I want to implement uwsgi_cache for nginx configuration, and I'm trying like this:
uwsgi_cache_path /tmp/nginx_cache/ levels=1:2 keys_zone=mycache:10m max_size=10g inactive=10m use_temp_path=off;
server {
listen 80;
server_name myapp.com;
root /home/user/myapp;
location / {
uwsgi_cache mycache;
uwsgi_cache_valid 200 15m;
uwsgi_cache_key $request_uri;
add_header X-uWSGI-Cache $upstream_cache_status;
expires 1h;
uwsgi_pass unix:///tmp/myapp.socket;
include uwsgi_params;
uwsgi_param UWSGI_SCHEME $scheme;
uwsgi_param SERVER_SOFTWARE nginx/$nginx_version;
}
}
However, every time I hit an URL, I get a MISS in the response headers, indicating that nginx didn't serve the request from cache:
Cache-Control:max-age=3600
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/html; charset=utf-8
Date:Mon, 27 Mar 2017 16:37:29 GMT
Expires:Mon, 27 Mar 2017 22:37:29 GMT
Server:nginx
Transfer-Encoding:chunked
Vary:Accept-Encoding
X-Powered-By:web2py
X-uWSGI-Cache:MISS
The nginx process is running as "www-data" user/group. I've checked the permissions of the folder /tmp/nginx_cache/ and they are ok: the user has permissions to read and write the folder. Also, inside the /tmp/nginx_cache/ a "temp" folder is created by nginx, but no cache files are written there.
I've tried also adding proxy_ignore_headers to location block in order to instruct nginx to ignore some headers like Set-Cookie and Cache-Control, like this:
location / {
uwsgi_cache mycache;
uwsgi_cache_valid 200 15m;
uwsgi_cache_key $scheme$proxy_host$uri$is_args$args;
add_header X-uWSGI-Cache $upstream_cache_status;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control Set-Cookie Vary;
uwsgi_pass unix:///tmp/myapp.socket;
include uwsgi_params;
uwsgi_param UWSGI_SCHEME $scheme;
uwsgi_param SERVER_SOFTWARE nginx/$nginx_version;
}
However, this makes no difference: the first request isn't cached, and all the subsequent requests are a MISS, that is, they aren't served from cache.
I've found this similar post, where the person who answers points out that it could be a problem of the response headers generated by (in this case) web2py:
https://serverfault.com/questions/690164/why-is-this-nginx-setup-not-caching-responses
Why nginx isn't caching the responses?
I've found the cause of the issue: I was mixing uwsgi_cache_* directives with proxy_cache_* directives, and they belong to different Nginx modules. I just needed to replace proxy_ignore_headers with uwsgi_ignore_headers.
Notice that proxy_cache module is different than uwsgi_cache, they have very similar directives, but they are two different modules.
Related
I'm trying to cache static files on a server, rather than going to the 'upstream' server each time. This upstream server happens to be Cloudfront,
Here is my nginx configuration:
nginx.conf http context:
proxy_cache_key "$scheme$host$request_uri";
proxy_cache_path /var/spool/nginx levels=1:1 keys_zone=oly_zone:1000m;
proxy_cache_use_stale updating;
proxy_cache_valid 200 301 302 10m;
proxy_cache_valid any 10s;
proxy_cache oly_zone;
website.conf:
location /gameimages/stock/ {
proxy_ignore_headers "Set-Cookie";
proxy_hide_header "Set-Cookie";
add_header X-Proxy-Cache $upstream_cache_status;
proxy_cache_valid 404 1s;
proxy_cache_valid any 15d;
proxy_cache oly_zone;
proxy_pass http://d34sdfsfsadfasdfmhbsdafirsdfsdffelaut.cloudfront.net/;
}
I thought this worked, but an example response header shows this:
Accept-Ranges:bytes
Age:11515
Connection:keep-alive
Content-Length:11577
Content-Type:image/jpeg
Date:Mon, 08 Aug 2016 19:25:16 GMT
ETag:"57a47349-2d39"
Last-Modified:Fri, 05 Aug 2016 11:06:49 GMT
Server:nginx/1.4.1
Via:1.1 3ba457b8dbcd4sadfsdfe93515e26caad.cloudfront.net (CloudFront)
X-Amz-Cf-Id:N0Dlk5c28sdfsf5Cvfskb3-T6PRBfSXfEPsdfasfuOLW7SHa1hjQ==
X-Cache:Hit from cloudfront
X-Proxy-Cache:HIT
It seems to be hitting both CloudFront and the cache on the server. Am I doing something wrong?
Thanks,
Michael
If issue is still actual - I have found an solution.
Generally CloudFront should be excluded from assets loading path and S3 bucket should be used directly as data source.
Solution: https://dpb587.me/blog/2015/06/20/using-nginx-to-reverse-proxy-and-cache-s3-objects.html
I my case only one line had to be added to get cache magic working:
# use google as dns
resolver 8.8.8.8;
Also sometimes SELinux requres some tuning to prevent nginx (13: Permission denied) errors:
sudo setsebool httpd_can_network_connect on -P
sudo semanage permissive -a httpd_t
I've got nginx sitting between ELBs. I've got a couple application pools behind an ELB that nginx passes traffic back to and I want to cache static content. My problem is nginx doesn't appear to be caching any responses. This is the cache configuration:
proxy_cache_path /usr/share/nginx/cache/app levels=1:2 keys_zone=cms-cache:8m max_size=1000m inactive=600m;
proxy_temp_path /usr/share/nginx/cache/;
location / {
proxy_pass http://cms-pool;
proxy_cache cms-cache;
proxy_cache_valid 200 302 60m;
proxy_cache_valid 404 1m;
}
After some reading I found there might be some headers causing the issue but after hiding the obvious ones I had no luck and ended up breaking the application since I was hiding all the backend cookies. These are the headers I tried removing:
proxy_ignore_headers Cache-Control Expires Set-Cookie;
proxy_hide_header Cache-Control;
proxy_hide_header Set-Cookie;
I'm at a loss right now as to why requests are not being cached, here's an output from curl of the headers that came through with the above header configuration (the cookies and such were set from nginx/elb in front of nginx):
Accept-Ranges: bytes
Connection: keep-alive
Content-length: 6821
Content-Type: text/html
Date: Sun, 16 Nov 2014 19:25:41 GMT
ETag: W/"6821-1415964130000"
Expires: Thu, 01 Jan 1970 00:00:00 GMT
HTTP/1.1 200 OK
Last-Modified: Fri, 14 Nov 2014 11:22:10 GMT
Pragma: no-cache
Server: nginx/1.7.6
Set-Cookie: AWSELB=4BB7AB49169E74EC05060FB9839BD30C2CB1D0E43D90837DC593EB2BA783FB372E90B6F6F575D13C6567102032557C76E00B1F5DB0B520CF929C3B81327C1D259A9EA5C73771C4EA3DB6390EB40484EDF56491135B;PATH=/
Set-Cookie: frontend=CgAAi1Ro+jUDNkZYAwMFAg==; path=/
Update I found that the above wasn't entirely accurate as there's a 302 that directs a user to login which hits another backend that doesn't have static resources, as such the headers above are coming from the login backend. I adjusted the URI to point to just the images but no caching is occurring. I'm using the following location block:
location /app/images {
proxy_pass http://cms-pool/app/images;
proxy_cache cms-cache;
proxy_cache_valid 200 302 60m;
proxy_cache_valid 404 1m;
proxy_ignore_headers Cache-Control Expires Set-Cookie;
proxy_hide_header Cache-Control;
proxy_hide_header Set-Cookie;
}
These are the headers which are coming through now:
Accept-Ranges: bytes
Connection: keep-alive
Content-length: 12700
Content-Type: image/png
Date: Mon, 17 Nov 2014 09:25:38 GMT
ETag: "0cd80ce9afecf1:0"
HTTP/1.1 200 OK
Last-Modified: Wed, 12 Nov 2014 17:05:06 GMT
Server: nginx/1.7.6
Set-Cookie: AWSELB=4BB7AB49169E74EC05060FB9839BD30C2CB1D0E43D638163025E92245C6C6E40197CA48C5B22F3E8FDA53365109BC1C16C808322881855C100D4AC54E5C0EC6CDE91B96151F66369C7B697B04D2C08439274033D81;PATH=/
Set-Cookie: tscl-frontend=CgAK1FRpvxI4b0bQAwMEAg==; path=/
X-Powered-By: ASP.NET
This is caused by a wobbly implementation of HTTP caching headers in what's behind ELB.
Per RFC 2616 (HTTP 1.1) :
Pragma directives MUST be passed through by a proxy or gateway
application, regardless of their significance to that application,
since the directives might be applicable to all recipients along the
request/response chain.
HTTP/1.1 caches SHOULD treat "Pragma: no-cache" as if the client had
sent "Cache-Control: no-cache". No new Pragma directives will be
defined in HTTP.
Note: because the meaning of "Pragma: no-cache as a response
header field is not actually specified, it does not provide a
reliable replacement for "Cache-Control: no-cache" in a response
The Pragma: no-cache header doesn't mean anything in an HTTP reply because it's behaviour is unspecified by RFCs.
But as nginx acts as a (reverse) proxy in your case, it will honor the header as if it was Cache-Control: no-cache header to maintain compatibility with HTTP 1.0 protocol's Pragma header defined in RFC 1945.
It will also pass it through client's response headers as it doesn't have to suppose anything about the actual meaning of it.
So either correct this bad implementation or append Pragma header to proxy_ignore_headers and proxy_hide_header directives.
Now I am trying to set cookie from my J2EE application for my jasper server on the localhost and getting the below response . I have passed domain name as .report.com but cookie is not created in browser . is there something wrong in below response to generate a cookie successfully in browser
Info : As localhost is not a valid domain name so I have changed localhost to jasper.report.com on system32/host file .
Accept-Ranges bytes
Cache-Control no-cache,no-store,must-revalidate
Connection Keep-Alive
Content-Encoding gzip
Content-Length 148
Content-Type text/html;charset=utf-8
Date Wed, 08 Oct 2014 14:36:43 GMT
Expires Thu, 01 Dec 1994 16:00:00 GMT
Keep-Alive timeout=5, max=100
Server Apache/2
Set-Cookie JSESSIONID=89E39BF51000E23270CE40EAE425EBF4; Domain=.report.com; Expires=Wed, 08-Oct-2014 19:36:44 GMT; Path=/jasperserver/; HttpOnly
Vary Accept-Encoding
x-frame-options SAMEORIGIN
I have recently figured out how to enable GZIP (or Deflate) on my WAMP server that I will be using to serve my intranet application.
However, when testing in Google Chrome I see that the PHP file is compressed but Javascript files and CSS are not. The Response header shows that it is not compressed and Google Pagespeed confirms this.
Firefox on the other hand recives all files with compression without a problem.
Here are the headers for the main CSS file as an example:
Google Chrome
Date: Wed, 18 Jul 2012 14:48:43 GMT
Content-Length: 6533
Last-Modified: Wed, 18 Jul 2012 00:42:33 GMT
Server: Apache/2.2.21 (Win64) PHP/5.3.10
ETag: "a00000001509b-1985-4c50ff04b26ef"
Vary: Accept-Encoding
Content-Type: text/css
Accept-Ranges: bytes
200 OK
Firefox
Date: Wed, 18 Jul 2012 14:33:14 GMT
Server: Apache/2.2.21 (Win64) PHP/5.3.10
Last-Modified: Wed, 18 Jul 2012 00:42:33 GMT
Etag: "a00000001509b-1985-4c50ff04b26ef"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 1273
Content-Type: text/css
200 OK
Is this a problem with my WAMP setup, code, or is it just Google Chrome?
Thank you.
Google chrome does support JS/CSS gzip.
Request URL:http://d1o7y22ifnbryp.cloudfront.net/static/7793/css/all.min.css
Request Method:GET
Status Code:200 OK
Request Headersview source
Accept:text/css,*/*;q=0.1
Accept-Charset:GBK,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:zh-CN,zh;q=0.8
Connection:keep-alive
Host:d1o7y22ifnbryp.cloudfront.net
User-Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5
Response Headers
Accept-Ranges:bytes
Cache-Control:max-age=604800
Connection:keep-alive
Content-Encoding:gzip
Content-Length:17933
Content-Type:text/css
Date:Wed, 18 Jul 2012 15:43:46 GMT
ETag:"805dc-19c87-4c4a305735340"
Expires:Wed, 25 Jul 2012 15:43:46 GMT
Last-Modified:Thu, 12 Jul 2012 14:45:57 GMT
Server:Apache/2.2.22 (Amazon)
Vary:Accept-Encoding
I think the problem should be in apache configuration.
I'm using Apache 2.2.16.
I've got the following HTML file in my htdocs folder:
<html>
<link media="screen" type="text/css" href="/css/test.css" rel="stylesheet" />
<body>XXXX</body>
</html>
I'm trying to make the CSS file cache in my browser so that it does not make another request for it, apart from on the first ever page load.
However, every time I press refresh, I can see that the CSS file is loaded again from the server. I see 200 response codes for the file in my server logs.
I've got the following in my httpd.conf file:
<LocationMatch "\.(css)$">
Header set Cache-Control "max-age=2592000"
Header set Expires "Thu, 15 Apr 2020 20:00:00 GMT"
Header unset Last-Modified
Header set Content-Type text/css
Header unset ETag
</LocationMatch>
Here are my response headers, as reported by Firebug:
Date Mon, 29 Nov 2010 10:48:49 GMT
Server Apache/2.2.16 (Win32)
Accept-Ranges bytes
Content-Length 18107
Cache-Control max-age=2592000
Expires Thu, 15 Apr 2020 20:00:00 GMT
Content-Type text/css
Keep-Alive timeout=5, max=97
Connection Keep-Alive
I've read a few articles about setting the caching headers, but I just can't see to get it working.
Any advice greatly appreciated.
I've seen similar problems with configurations that manually set expires & cache-control. Just letting mod_expires do that "heavy lifting" might already solve these issues?