I am working on a chrome extension which modifies the http response headers.
https://chrome.google.com/webstore/detail/super-cache/fglobbnbihckpkodmeefhagijjcjnbeh/details
I am not able to cache main_frame requests. I am able to control the caching of the static requests though.
For example if I hit http://apple.com I receive the following headers for the main_frame.
Accept-Ranges:bytes
Cache-Control:max-age=276
Connection:keep-alive
Content-Encoding:gzip
Content-Length:3310
Content-Type:text/html; charset=UTF-8
Date:Tue, 12 Mar 2013 09:24:12 GMT
Expires:Tue, 12 Mar 2013 09:28:48 GMT
Server:Apache
Vary:Accept-Encoding
But every time I hit the url the browser tries to access the server and ultimately receive a 200 response. I have tried all the possible combinations that the headers can be set to enable caching on the main_frame.
I want that when the user hits the url from the navigation bar of chrome and no requests are made.
You're missing some sort of cache validation in your response headers. ETag header can be used to control that, by adding values to it that would identify a unique response. You can read a bit about it in the Apache ETag documentation, but I'd simply include ETag: [filename] in your response headers in your example:
Accept-Ranges:bytes
Cache-Control:max-age=276
Connection:keep-alive
Content-Encoding:gzip
Content-Length:3310
Content-Type:text/html; charset=UTF-8
Date:Tue, 12 Mar 2013 09:24:12 GMT
Expires:Tue, 12 Mar 2013 09:28:48 GMT
Server:Apache
ETag: File:"somefile.html"
Vary:Accept-Encoding
These ETag values can include pretty much anything, such as file name, file size, custom values,... that can be separated by a semicolon ;. If the values include spaces, then enclose them in double quotation marks ". For Example:
ETag: File:"YouTube_cd_Fdly3rX8.jpg"; Size:12169
Together with Cache-Control, Expires and some other header values that might change (when included and browser knows how to interpret them), will form a basis for Browser's cache validator.
Looking at your sample response headers, you might want to increase the max-age value in your Cache-Control to a lot higher value, as your example suggest they should be cached client-side for only 276 seconds. The Expires header value also seems a bit short.
More on how to set these values and how browsers are expected to validate cache control headers can be read in the RFC2616, Section 14.9.
EDIT: After further debugging, checking and re-checking the behavior of Chrome's cache validation, it turns out it indeed doesn't respect properly set Cache-Control response headers. On request of the OP, I've reported this issue to the Chrome support:
Chrome, Version 25.0.1364.172 m
Disrespecting Cache-Control on main document requests when serving
static files from a web server, while respecting same header response
on linked contents.
Test setup:
Requesting a static HTML document from a web server (MIME text/html),
that contains another static HTML document withing an IFRAME (also
MIME text/html). The IFRAME served document has same response headers
attached to it by a web server response as the main document:
Date: Thu, 21 Mar 2013 16:29:28 GMT
Expires: Thu, 21 Mar 2013 16:33:59 GMT
Cache-Control: max-age=301, max-stale=299, only-if-cached
Expected behavior:
Main document and the document served within an IFRAME will be cached
locally with initial request for the duration of at least 301
(max-age) seconds, and additional 299 (max-stale) seconds for normal
(non-forced) load requests. Any subsequent requests within this
time-frame of 301 seconds that aren't expected to invalidate local
cache (such as forced-refresh with CTRL+F5 or Reload context menu
command) and are initiated by a normal page load request (e.g.
re-entering relevant URL in the address bar) will be loaded from the
local cache with a status message 200 OK (from cache), if none of the
local cache controlling information indicates it otherwise (same URL,
requested within valid cache time-frame, document was tagged to be
cached in its response headers correctly).
Problem:
The main document is not loaded via its cached copy and an additional
request is made to the web server, resulting in a status code 304 Not
Modified. The document within an IFRAME however is loaded from the
local cache correctly and results in a status message 200 OK (from
cache).
Notes:
None of the cache-control tags or any combination of their values seem
to have any positive effect on the behavior of local cache for the
main document. Including a non-unique ETag value does not resolve the
problem of caching main document either. Other major vendor browsers
(tested in IE, Firefox, Opera) respect Cache-Control headers on main
document.
Related
I followed this tutorial (https://learn.microsoft.com/en-us/aspnet/core/performance/caching/response?view=aspnetcore-2.1) to implement ResponseCache on my controller-action.
In short, I added
services.AddResponseCaching(); and app.UseResponseCaching(); in the startup and this tag [ResponseCache( Duration = 30)] on my controller.
Then I added a <h2>#DateTime.Now</h2> in my view and what I expected.... was the same datetime.now for 30 seconds.
But it doesn't, it just shows the new time on every reload (F5).
I made sure my devtools in chrome do not say 'disable cache'.
It's both with and without the chrome devtools open, on my local machine, now trying on a brandnew .net core mvc project.
One thing I noticed (with devtools open) is that the request has this header: Cache-Control: max-age=0. Does this influence the behaviour?
I thought it would mean something because it looks like the request says 'no cache' but that strikes me as weird because I didn't put the header in and I would say the default behaviour of chrome wouldn't be to ignore caches?
A header like Cache-Control: max-age=0 effectively disables all caching. Resources are basically expired as soon as they come off the wire, so they are always fetched. This header originates from the server. The client has nothing to do with it.
Assuming you haven't disabled response caching manually in some way by accident. Then, the most likeliest situation is that you're doing something where the response caching middleware will never cache. The documentation lists the following conditions that must be satisfied before responses will be cached, regardless of what you do:
The request must result in a server response with a 200 (OK) status code.
The request method must be GET or HEAD.
Terminal middleware, such as Static File Middleware, must not process the response prior to the Response Caching Middleware.
The Authorization header must not be present.
Cache-Control header parameters must be valid, and the response must be marked public and not marked private.
The Pragma: no-cache header must not be present if the Cache-Control header isn't present, as the Cache-Control header overrides the Pragma header when present.
The Set-Cookie header must not be present.
Vary header parameters must be valid and not equal to *.
The Content-Length header value (if set) must match the size of the response body.
The IHttpSendFileFeature isn't used.
The response must not be stale as specified by the Expires header and the max-age and s-maxage cache directives.
Response buffering must be successful, and the size of the response must be smaller than the configured or default SizeLimit.
The response must be cacheable according to the RFC 7234 specifications. For example, the no-store directive must not exist in request or response header fields. See Section 3: Storing Responses in Caches of RFC 7234 for details.
However, in such situations, the server should be sending Cache-Control: no-cache, not max-age=0. As a result, I'm leaning towards some misconfiguration somewhere, where you have set this max age value and either forgot or overlooked it.
This is working for me in a 3.1 app to not let F5/Ctrl+F5 or Developer Tools in Firefox or Chrome bypass server cache for a full response.
In startup add this little middleware before UseResponseCaching().
// Middleware that fixes server caching on F5/Reload
app.Use(async (context, next) =>
{
const string cc = "Cache-Control";
if (context.Request.Headers.ContainsKey(cc))
{
context.Request.Headers.Remove(cc);
}
const string pragma = "Pragma";
if (context.Request.Headers.ContainsKey(pragma))
{
context.Request.Headers.Remove(pragma);
}
await next();
});
app.UseResponseCaching();
Haven't noticed any problems...
Okay, so I noticed my Cloudfront isn't caching when I ran Google Page Tools and it told me that my images had no expiration set. I use Amazon S3 through Cloudfront. There's no headers set on S3 because I have hundreds of folders and thousands of image files with new ones uploaded every hour.
I went to my Cloudfront console, to Behaviours, edited the only one there and set:
Minimum TTL: 86400
Maximum TTL: 31536000
Default TTL: 86400
And I checked the 'Customize' option for 'Object Caching'. I then went to invalidate and invalidated all my objects (*). I waited until it was done, but my headers when requesting a file still shows:
Age:8
Connection:keep-alive
Date:Mon, 07 Dec 2015 00:44:39 GMT
ETag:"429d87a5fd35288d207635d2a853fa0b"
Server:AmazonS3
Via:1.1 (my-ID-here).cloudfront.net (CloudFront)
X-Amz-Cf-Id:RxHlfhhnrSk9YwIqpFySnPVrscndnknZ9RKlIryXCLwh4RCK9vK6Vw==
X-Cache:Hit from cloudfront
What am I doing wrong?
Was this under "Leverage browser caching" or a similar section of Page Tools?
If under 'Leverage Browser Caching', - this doesn't mean the files aren't being cached, but it means that the requested files aren't requesting end user browsers to cache them - for instance using the 'Cache-Control' or 'Expires' headers. CloudFront, unless configured otherwise, caches files from S3 -- so the absence of these headers doesn't affect Cloudfront's caching.
You can manually add these in S3 for the individual objects.
Some more info can be found in the CloudFront documentation
I'm running a Piranha CMS based page, using it as a content source in passive mode.
While working on optimizing pagespeed i saw that all image requests handled by piranha return the following response headers:
GET /media/4b3b3fa3-ff7b-4af7-81f2-168474edd23f/50/20
Cache-Control:no-cache
Content-Length:52826
Content-Type:image/jpeg
Date:Mon, 17 Nov 2014 11:53:28 GMT
Expires:-1
Pragma:no-cache
X-UA-Compatible:IE=Edge,chrome=1
Naturally, i want these images to be cached where possible.
I looked around the code on github and saw that this information appears to be set in /Piranha/Web/ClientCache.cs.
Is there a way to influence the caching headers set by piranha?
The configuration section isn't all that clear in the current version but the caching is controlled by two parameters that you can set in the manager interface from System > Parameters. Here you have the two parameters CACHE_PUBLIC_EXPIRES and CACHE_PUBLIC_MAXAGE which specifies the time, in minutes, of the client browser cache.
These are set to 0 by default since you don't want cache enabled during development which disables the cache and renders the no-cache headers. When you activate the cache E-tag, Last modified & Expires headers will be generated for your media files.
Best regards
Håkan
I'm trying to improve the caching policy for a web site.
I'm implementing an aggressive caching strategy first, for resources that won't change at all, and would like files like jquery-min for example to be downloaded only once, and then be served from browser cache.
I used Apache Mod-Expire module to accomplish this, and it's working pretty well.
However, I'm surprised that when I completely remove my cache-control configuration, Firefox browser does not re-download jquery file : it already gets it from the cache, without any instruction from me. How is Firefox making this kind of decision ? Is it related to the ETag hash ?
Here are the headers I get without my configuration for that particular file :
Accept-Ranges bytes
Connection Keep-Alive
Content-Encoding gzip
Content-Length 27073
Content-Type application/javascript
Date Fri, 23 Aug 2013 09:48:06 GMT
Etag "225f8-13309-4e385823c7b80"
Keep-Alive timeout=15, max=99
Last-Modified Fri, 09 Aug 2013 15:34:22 GMT
Server Apache
Vary Accept-Encoding
Can anyone explain to me what is the browser behavior when there's no Cache-Control policy defined ? is it browser dependant ?
Thank you,
Mathieu.
Plone 3.3.5, LinguaPlone and Products.CacheSetup installed.
Language cookie set for portal_css and portal_javascript files
HTTP/1.1 200 OK
Server: Zope/(unreleased version, python 2.4.5, linux2) ZServer/1.1 Plone/3.3.5
Expires: Tue, 26 Apr 2011 10:42:56 GMT
Last-Modified: Tue, 19 Apr 2011 10:42:56 GMT
Cache-Control: max-age=604800
Content-Type: application/x-javascript;charset=utf-8
Set-Cookie: I18N_LANGUAGE="en"; Path=/
Content-Length: 192404
Date: Tue, 19 Apr 2011 10:42:56 GMT
X-Varnish: 452768899
Age: 0
Via: 1.1 varnish
Connection: keep-alive
Pituus: 192404 (188K) [application/x-javascript]
Tallennetaan kohteeseen ”tiny_mce-cachekey1974.js”
This prevents caching. I found some old information related to this:
http://www.evax.fr/papers/nginx-varnish-and-multilingual-plone
1) Why it is set? I assume all resources are language neutral by default and only in special conditions contain language specific stuff.
2) How do I get rid off it? Maybe there is a smarter way than monkey-patching, but not documented anywhere.
3) Any changes related to this in Plone 4.x?
The cookie is set by the LanguageTool, and is only set when not present yet in the request, or different from what was set in the request.
The LanguageTool normally would only look at that very cookie for deciding what language to use. In normal Plone use therefor, you will never see that Set-Cookie header on CSS and JS resources. LinguaPlone however, configures the LanguageTool to look at a lot more information, including the URL you were trying to access, to determine a language to use for the response.
With LinguaPlone installed, what the LanguageTool does is to determine the correct language for every resource accessed, as that is the only way to ensure a consistent UI language through-out.
The cookie is normally already set on load of the main page (which subsequently loads JS and CSS with the cookie set). Thus, in the normal scenario your CSS and JS are perfectly cacheable. You are only seeing the Set-Cookie header because there is no corresponding Cookie: I18N_LANGUAGE="en" header in the request.
If you have to control this for your specific use-cases, you could monkeypatch Products.PloneLanguageTool.LanguageTool.LanguageTool.__call__, which is a before-publishing-traverse hook that will trigger the language determination on every request. I'd recommend using collective.monkeypatcher for this. Here is what I did in a recent project where the site was migrated with a mixed set of languages throughout the tree (something we'll have to untangle at some point in the future):
configure.zcml:
<?xml version="1.0"?>
<configure
xmlns="http://namespaces.zope.org/zope"
xmlns:monkey="http://namespaces.plone.org/monkey"
>
<!-- other directives -->
<include package="collective.monkeypatcher" />
<monkey:patch
description="Patch LanguageTool before traverse hook to prevent setting
the language cookie"
class="Products.PloneLanguageTool.LanguageTool.LanguageTool"
original="__call__"
replacement=".patches.LanguageTool__call__"
preserveOriginal="true"
/>
</configure>
and in the patches.py module:
from ZPublisher.HTTPRequest import HTTPRequest
def LanguageTool__call__(self, container, req):
"""The __before_publishing_traverse__ hook.
Patched to *not* set the language cookie, as this breaks the site model.
"""
self._old___call__(container, req)
if not isinstance(req, HTTPRequest):
return None
response = req.response
if 'I18N_LANGUAGE' in response.cookies:
if 'set_language' in req.form:
return None
del response.cookies['I18N_LANGUAGE']
Note that the patch deletes the cookie after the fact, unless set_language was used on the current request.
There are no changes pending for this behaviour for Plone 4.x.