Despite of doing it in each file type handler, is there any simple way to set default expiry date of all types of file, e.g. json, jpg, html, css, js, etc?
As Nagi said, the connection handler called /handlers/main.c can be used to generate custom expiration HTTP headers.
You can also use content-type handlers which name in the /handlers directory must match the file type (html.c, json.c, jpg.c, etc.).
By default, G-WAN generates HTTP headers which may help proxy servers to do their job:
Date: Thu, 29 Nov 2012 15:00:55 GMT
Last-Modified: Sun, 25 Nov 2012 13:54:46 GMT
ETag: "810c7fa9--50b22326-7ec3"
But a more fine-grained strategy can be used. That could be done in a future version by defining those content-related expirations in content-type handlers.
Proof that a server is a living (customer-driven) creature:
Related
Downloading a file directly (without any scripts) from the server, sometimes it fails for slow internet connections.
e,g for any file, each time users can download some percent of the file. And sometimes they can download it completely.
In the cases that download fails, browser doesn't show a "Failed" text. it seems the file has been downloaded completely.
I thought it might be with the keep-alive connection. I use these response headers:
Accept-Ranges:bytes
Cache-Control:max-age=172800
Connection:Keep-Alive
Content-Length:36195412
Content-Type:application/x-rar-compressed
Date:Tue, 28 Jul 2015 07:00:49 GMT
ETag:"a9825a8-2284c54-518bca9ddaaad"
Expires:Thu, 30 Jul 2015 07:00:49 GMT
Keep-Alive:timeout=5
Last-Modified:Wed, 17 Jun 2015 20:37:46 GMT
Server:Apache/2.4.12
Vary:User-Agent
And one more question.You can see in the headers that keep-alive is set like this: keep-alive:timeout=5 there is no max here. so is it default to 0 or something else?
Thanks.
Well, I figured out that it is not related to keep-alive. I and most of our clients were using proxies, it made some problems.
We are adding our custom website font via our CDN. Our CSS code is below. This is in our main.css file which is included in the header of our website.
This works in Safari and Chrome on OSX, but on Windows, it works only in Safari. And maybe IE10. On Firefox it simply doesn't work. And on IE9 etc it works sometimes. (What an irritating browser.)
Some websites suggest that Firefox wants a relative font URL. But we need to use a CDN for our font. How do Google fonts work in Firefox is there is a "same origin" issue?
Our main server is Nginx. The static font files are being served from there, so the Apache suggestion of Allow Origin "*" does not help us much. The CDN in this case is origin-pulled from our website and carries our own headers. So if we need to so send a header via Nginx, we can do that.
THE FONT CSS
#font-face{font-family:'Custom-Sans'
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-light-webfont.eot')
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-light-webfont.eot?#iefix') format('embedded-opentype'),url('http://cache.MYDOMAIN.com/Custom-Sans-light-webfont.woff') format('woff'),url('http://cache.MYDOMAIN.com/Custom-Sans-light-webfont.ttf') format('truetype');font-weight:200;font-style:normal}
#font-face{font-family:'Custom-Sans'
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-thin-webfont.eot')
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-thin-webfont.eot?#iefix') format('embedded-opentype'),url('http://cache.MYDOMAIN.com/Custom-Sans-thin-webfont.woff') format('woff'),url('http://cache.MYDOMAIN.com/Custom-Sans-thin-webfont.ttf') format('truetype');font-weight:100;font-style:normal}
#font-face{font-family:'Custom-Sans'
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-regular-webfont.eot')
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-regular-webfont.eot?#iefix') format('embedded-opentype'),url('http://cache.MYDOMAIN.com/Custom-Sans-regular-webfont.woff') format('woff'),url('http://cache.MYDOMAIN.com/Custom-Sans-regular-webfont.ttf') format('truetype');font-weight:normal;font-style:normal}
#font-face{font-family:'Custom-Sans'
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-semibold-webfont.eot')
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-semibold-webfont.eot?#iefix') format('embedded-opentype'),url('http://cache.MYDOMAIN.com/Custom-Sans-semibold-webfont.woff') format('woff'),url('http://cache.MYDOMAIN.com/Custom-Sans-semibold-webfont.ttf') format('truetype');font-weight:500;font-style:normal}
#font-face{font-family:'Custom-Sans'
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-bold-webfont.eot')
;src:url('http://cache.MYDOMAIN.com/Custom-Sans-bold-webfont.eot?#iefix') format('embedded-opentype'),url('http://cache.MYDOMAIN.com/Custom-Sans-bold-webfont.woff') format('woff'),url('http://cache.MYDOMAIN.com/Custom-Sans-bold-webfont.ttf') format('truetype');font-weight:700;font-style:normal}
And --
THE HEADERS OF A FONT FILE (Served by Nginx)
HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Origin: *.MYDOMAIN.com
Cache-Control: max-age=315360000
Content-Type: application/octet-stream
Date: Sun, 10 Aug 2014 15:10:29 GMT
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Last-Modified: Mon, 28 Jul 2014 14:52:44 GMT
Server: Hosting Inc
Vary: Accept-Encoding
Content-Length: 20077
The access control header needs a scheme in addition to the domain:
Access-Control-Allow-Origin: http://*.mydomain.com
It also needs a port if you're using something other than 80.
I'm trying to improve the caching policy for a web site.
I'm implementing an aggressive caching strategy first, for resources that won't change at all, and would like files like jquery-min for example to be downloaded only once, and then be served from browser cache.
I used Apache Mod-Expire module to accomplish this, and it's working pretty well.
However, I'm surprised that when I completely remove my cache-control configuration, Firefox browser does not re-download jquery file : it already gets it from the cache, without any instruction from me. How is Firefox making this kind of decision ? Is it related to the ETag hash ?
Here are the headers I get without my configuration for that particular file :
Accept-Ranges bytes
Connection Keep-Alive
Content-Encoding gzip
Content-Length 27073
Content-Type application/javascript
Date Fri, 23 Aug 2013 09:48:06 GMT
Etag "225f8-13309-4e385823c7b80"
Keep-Alive timeout=15, max=99
Last-Modified Fri, 09 Aug 2013 15:34:22 GMT
Server Apache
Vary Accept-Encoding
Can anyone explain to me what is the browser behavior when there's no Cache-Control policy defined ? is it browser dependant ?
Thank you,
Mathieu.
I am working on a chrome extension which modifies the http response headers.
https://chrome.google.com/webstore/detail/super-cache/fglobbnbihckpkodmeefhagijjcjnbeh/details
I am not able to cache main_frame requests. I am able to control the caching of the static requests though.
For example if I hit http://apple.com I receive the following headers for the main_frame.
Accept-Ranges:bytes
Cache-Control:max-age=276
Connection:keep-alive
Content-Encoding:gzip
Content-Length:3310
Content-Type:text/html; charset=UTF-8
Date:Tue, 12 Mar 2013 09:24:12 GMT
Expires:Tue, 12 Mar 2013 09:28:48 GMT
Server:Apache
Vary:Accept-Encoding
But every time I hit the url the browser tries to access the server and ultimately receive a 200 response. I have tried all the possible combinations that the headers can be set to enable caching on the main_frame.
I want that when the user hits the url from the navigation bar of chrome and no requests are made.
You're missing some sort of cache validation in your response headers. ETag header can be used to control that, by adding values to it that would identify a unique response. You can read a bit about it in the Apache ETag documentation, but I'd simply include ETag: [filename] in your response headers in your example:
Accept-Ranges:bytes
Cache-Control:max-age=276
Connection:keep-alive
Content-Encoding:gzip
Content-Length:3310
Content-Type:text/html; charset=UTF-8
Date:Tue, 12 Mar 2013 09:24:12 GMT
Expires:Tue, 12 Mar 2013 09:28:48 GMT
Server:Apache
ETag: File:"somefile.html"
Vary:Accept-Encoding
These ETag values can include pretty much anything, such as file name, file size, custom values,... that can be separated by a semicolon ;. If the values include spaces, then enclose them in double quotation marks ". For Example:
ETag: File:"YouTube_cd_Fdly3rX8.jpg"; Size:12169
Together with Cache-Control, Expires and some other header values that might change (when included and browser knows how to interpret them), will form a basis for Browser's cache validator.
Looking at your sample response headers, you might want to increase the max-age value in your Cache-Control to a lot higher value, as your example suggest they should be cached client-side for only 276 seconds. The Expires header value also seems a bit short.
More on how to set these values and how browsers are expected to validate cache control headers can be read in the RFC2616, Section 14.9.
EDIT: After further debugging, checking and re-checking the behavior of Chrome's cache validation, it turns out it indeed doesn't respect properly set Cache-Control response headers. On request of the OP, I've reported this issue to the Chrome support:
Chrome, Version 25.0.1364.172 m
Disrespecting Cache-Control on main document requests when serving
static files from a web server, while respecting same header response
on linked contents.
Test setup:
Requesting a static HTML document from a web server (MIME text/html),
that contains another static HTML document withing an IFRAME (also
MIME text/html). The IFRAME served document has same response headers
attached to it by a web server response as the main document:
Date: Thu, 21 Mar 2013 16:29:28 GMT
Expires: Thu, 21 Mar 2013 16:33:59 GMT
Cache-Control: max-age=301, max-stale=299, only-if-cached
Expected behavior:
Main document and the document served within an IFRAME will be cached
locally with initial request for the duration of at least 301
(max-age) seconds, and additional 299 (max-stale) seconds for normal
(non-forced) load requests. Any subsequent requests within this
time-frame of 301 seconds that aren't expected to invalidate local
cache (such as forced-refresh with CTRL+F5 or Reload context menu
command) and are initiated by a normal page load request (e.g.
re-entering relevant URL in the address bar) will be loaded from the
local cache with a status message 200 OK (from cache), if none of the
local cache controlling information indicates it otherwise (same URL,
requested within valid cache time-frame, document was tagged to be
cached in its response headers correctly).
Problem:
The main document is not loaded via its cached copy and an additional
request is made to the web server, resulting in a status code 304 Not
Modified. The document within an IFRAME however is loaded from the
local cache correctly and results in a status message 200 OK (from
cache).
Notes:
None of the cache-control tags or any combination of their values seem
to have any positive effect on the behavior of local cache for the
main document. Including a non-unique ETag value does not resolve the
problem of caching main document either. Other major vendor browsers
(tested in IE, Firefox, Opera) respect Cache-Control headers on main
document.
Plone 3.3.5, LinguaPlone and Products.CacheSetup installed.
Language cookie set for portal_css and portal_javascript files
HTTP/1.1 200 OK
Server: Zope/(unreleased version, python 2.4.5, linux2) ZServer/1.1 Plone/3.3.5
Expires: Tue, 26 Apr 2011 10:42:56 GMT
Last-Modified: Tue, 19 Apr 2011 10:42:56 GMT
Cache-Control: max-age=604800
Content-Type: application/x-javascript;charset=utf-8
Set-Cookie: I18N_LANGUAGE="en"; Path=/
Content-Length: 192404
Date: Tue, 19 Apr 2011 10:42:56 GMT
X-Varnish: 452768899
Age: 0
Via: 1.1 varnish
Connection: keep-alive
Pituus: 192404 (188K) [application/x-javascript]
Tallennetaan kohteeseen ”tiny_mce-cachekey1974.js”
This prevents caching. I found some old information related to this:
http://www.evax.fr/papers/nginx-varnish-and-multilingual-plone
1) Why it is set? I assume all resources are language neutral by default and only in special conditions contain language specific stuff.
2) How do I get rid off it? Maybe there is a smarter way than monkey-patching, but not documented anywhere.
3) Any changes related to this in Plone 4.x?
The cookie is set by the LanguageTool, and is only set when not present yet in the request, or different from what was set in the request.
The LanguageTool normally would only look at that very cookie for deciding what language to use. In normal Plone use therefor, you will never see that Set-Cookie header on CSS and JS resources. LinguaPlone however, configures the LanguageTool to look at a lot more information, including the URL you were trying to access, to determine a language to use for the response.
With LinguaPlone installed, what the LanguageTool does is to determine the correct language for every resource accessed, as that is the only way to ensure a consistent UI language through-out.
The cookie is normally already set on load of the main page (which subsequently loads JS and CSS with the cookie set). Thus, in the normal scenario your CSS and JS are perfectly cacheable. You are only seeing the Set-Cookie header because there is no corresponding Cookie: I18N_LANGUAGE="en" header in the request.
If you have to control this for your specific use-cases, you could monkeypatch Products.PloneLanguageTool.LanguageTool.LanguageTool.__call__, which is a before-publishing-traverse hook that will trigger the language determination on every request. I'd recommend using collective.monkeypatcher for this. Here is what I did in a recent project where the site was migrated with a mixed set of languages throughout the tree (something we'll have to untangle at some point in the future):
configure.zcml:
<?xml version="1.0"?>
<configure
xmlns="http://namespaces.zope.org/zope"
xmlns:monkey="http://namespaces.plone.org/monkey"
>
<!-- other directives -->
<include package="collective.monkeypatcher" />
<monkey:patch
description="Patch LanguageTool before traverse hook to prevent setting
the language cookie"
class="Products.PloneLanguageTool.LanguageTool.LanguageTool"
original="__call__"
replacement=".patches.LanguageTool__call__"
preserveOriginal="true"
/>
</configure>
and in the patches.py module:
from ZPublisher.HTTPRequest import HTTPRequest
def LanguageTool__call__(self, container, req):
"""The __before_publishing_traverse__ hook.
Patched to *not* set the language cookie, as this breaks the site model.
"""
self._old___call__(container, req)
if not isinstance(req, HTTPRequest):
return None
response = req.response
if 'I18N_LANGUAGE' in response.cookies:
if 'set_language' in req.form:
return None
del response.cookies['I18N_LANGUAGE']
Note that the patch deletes the cookie after the fact, unless set_language was used on the current request.
There are no changes pending for this behaviour for Plone 4.x.