Okay, so I noticed my Cloudfront isn't caching when I ran Google Page Tools and it told me that my images had no expiration set. I use Amazon S3 through Cloudfront. There's no headers set on S3 because I have hundreds of folders and thousands of image files with new ones uploaded every hour.
I went to my Cloudfront console, to Behaviours, edited the only one there and set:
Minimum TTL: 86400
Maximum TTL: 31536000
Default TTL: 86400
And I checked the 'Customize' option for 'Object Caching'. I then went to invalidate and invalidated all my objects (*). I waited until it was done, but my headers when requesting a file still shows:
Age:8
Connection:keep-alive
Date:Mon, 07 Dec 2015 00:44:39 GMT
ETag:"429d87a5fd35288d207635d2a853fa0b"
Server:AmazonS3
Via:1.1 (my-ID-here).cloudfront.net (CloudFront)
X-Amz-Cf-Id:RxHlfhhnrSk9YwIqpFySnPVrscndnknZ9RKlIryXCLwh4RCK9vK6Vw==
X-Cache:Hit from cloudfront
What am I doing wrong?
Was this under "Leverage browser caching" or a similar section of Page Tools?
If under 'Leverage Browser Caching', - this doesn't mean the files aren't being cached, but it means that the requested files aren't requesting end user browsers to cache them - for instance using the 'Cache-Control' or 'Expires' headers. CloudFront, unless configured otherwise, caches files from S3 -- so the absence of these headers doesn't affect Cloudfront's caching.
You can manually add these in S3 for the individual objects.
Some more info can be found in the CloudFront documentation
Related
Ok folks, here is my question
My struggle is with Cache-Control: public, max-age=259200 header that I have set up via middleware.
Here is how I did it:
Route::get('/', 'MainPageController#getMain')->middleware(['cache.headers:public;max_age=2592000', 'cacheable:2592000']);
The second middleware('cacheable:2592000') on the same route caches responses with Php Redis
Now the problem, I can see that the Cache-Control in the response header, but what I do not see is last-modified and expires headers on all of the text/Html type responses.
Also, I somehow need to set up the Last-Modified header to correspond with the date from the updated_at column. I have tried to add it manually to the main view, just to check if it works, but found another problem, when content is cached with Php Redis, the header won't show up.
Also, I do see that the content is served from disk cache yet the Status Code: 200 (from disk cache) shown as 200 instead of 304.
My app is built with Laravel 7 on Ubuntu 20.04 / NGINX with the Forge server.
If you need any additional information, please let me know.
Thanks
I've done lots of reading on this but none of the solutions I have seen seem to work for IIS websites - most seem to suggest some server-side solution but none of that works for me.
I'm optimising one of our sites, and PageSpeed, YSlow and Lighthouse all complain that images I'm serving from our CloudFront CDN don't have any cache headers. The CDN serves from an S3 bucket.
e.g. https://static.edie.net/webimages/new_new_new.png (expiration not specified)
Crops up as both 'There are static components without a far-future expiration date' and 'Leverage browser caching for the following cacheable resources'
I can't for the life of me work out how to make CloudFront serve a cache header for images like this.
I have set
Cache-Control: max-age=5500000
on the s3 bucket/file itself, and if you check the file via the bucket: https://devedienet.s3.amazonaws.com/webimages/new_new.png then it has the cache header present.
But that doesn't seem to affect the CloudFront image, which only has these headers:
Age: 12153
Connection: keep-alive
Date: Mon, 22 Oct 2018 11:18:49 GMT
ETag: "940fd4d68428cf3e4f88a45aab4d7157"
Server: AmazonS3
Via: 1.1 4f95eb10423b781564e79d7c85f85795.cloudfront.net (CloudFront)
X-Amz-Cf-Id: TZAWy8U12-ohhe-dwTkCLqXHbJKI7CJqQd21I-lvq-8rloZjTew6aw==
x-amz-meta-s3b-last-modified: 20181017T105350Z
X-Cache: Hit from cloudfront
I've tried adding custom behavours into AWS' Control Panel for the CloudFront distribution:
webimages/*.png
Minimum TTL: 5500000
But again this seems to have no effect.
Note that I invalidated all the images in the folder after adding the new rule above, this but no dice.
Am I missing something or misunderstanding what is required?
Since you are serving content from S3 through cloudfront, then you need to add the following headers to objects in S3 while uploading files to S3. Expires: {some future date}
Bonus: You do not need to specify this header for every object individually. You can upload a bunch of files together on S3, click next, and then on the screen that asks S3 storage class, scroll down and add these headers. And don't forget to click save!
I was facing a similar problem, so after doing some reading and experimenting what I figured out is that Cloudfront's Object Caching values of Minimum TTL, Max TTL, Default TTL doesn't add the Cache-Control header explicitly in the Response Headers for the resource if the resource doesn't have one at Server level. Secondlly even if the resource has aCache-control metadata added at S3, it should fall in between
MinTTL < s3 Cache < Max TTL .
The Object caching values state that for the provided value the resources will be cached for that much time at edge location and no Cache-Control will be added in Response headers for the resource.
What I did instead was to create a Lambda function and add it under Lambda associations by updating the Cache Behaviour settings for Viewer Response. Here is my Lambda fn. This added the Cache-Control header in the requested resource.
'use strict';
exports.handler = (event, context, callback) => {
const response = event.Records[0].cf.response;
const headers = response.headers;
const headerCache = 'Cache-Control';
headers[headerCache.toLowerCase()] = [{
key: headerCache,
value: 'max-age=1096000'
}];
callback(null, response);
};
Has anybody tried?
Here is the use case. In a first request-response cycle, this would happen:
Request 1:
GET / HTTP/1.1
...
Response 1
HTTP/1.0 200 OK
Etag: version1
Cache-control: max-age=1
... angly html here
....<link href="mycss.css" >
...
Request 2:
GET /mycss.css HTTP/1.1
...
Response 2 (probably pushed):
Etag: version1
Cache-control: max-age=<duration-of-the-universe>
...
... brackety css ...
...
and then, when the browsers goes a second time to the same page, it will of course fetch again the "/" resource because of the very short max-age:
GET / HTTP/1.1
...
If-not-modified: version1
But it won't fetch mycss.css if it has it in cache. However, the server can use the validator present in the "if-not-modified" header of the request for "/" to get an idea of the client's cache age, and may conclude that mycss.css version's of the browser is too old. In that case, before even answering the previous request, the server can "promise" a new version of mycss.css/
By the specs, should the browser accept and use it?
Overview:
I still don't know what the answer to my question is from a purely theoretical side, but at least today it doesn't seem possible in practice to do cache-busting this way :-(, with neither Google Chrome or Firefox. Both reject or ignore the pushed stream if they believe that the resource they have in cache is fresh.
I also got this from somebody who prefers to remain anonymous:
Browsers will typically put resources received through push in a
"demilitarized zone" and only once the client asks for that resource
it will be moved into the actual cache. So just pushing random
things will not make them end up in the browser cache even if the
browser accepts them at the push moment.
Update
As early 2016, it is still not possible, due mainly to lack of consensus on how this should be handled, and if it should be allowed at all or not.
As this page shows, even with HTTP/2, the way to solve the stale assets issue is to create a unique URL for each asset version, and then ensure that the user receives that new URL when they re-visit the page.
I'm running a Piranha CMS based page, using it as a content source in passive mode.
While working on optimizing pagespeed i saw that all image requests handled by piranha return the following response headers:
GET /media/4b3b3fa3-ff7b-4af7-81f2-168474edd23f/50/20
Cache-Control:no-cache
Content-Length:52826
Content-Type:image/jpeg
Date:Mon, 17 Nov 2014 11:53:28 GMT
Expires:-1
Pragma:no-cache
X-UA-Compatible:IE=Edge,chrome=1
Naturally, i want these images to be cached where possible.
I looked around the code on github and saw that this information appears to be set in /Piranha/Web/ClientCache.cs.
Is there a way to influence the caching headers set by piranha?
The configuration section isn't all that clear in the current version but the caching is controlled by two parameters that you can set in the manager interface from System > Parameters. Here you have the two parameters CACHE_PUBLIC_EXPIRES and CACHE_PUBLIC_MAXAGE which specifies the time, in minutes, of the client browser cache.
These are set to 0 by default since you don't want cache enabled during development which disables the cache and renders the no-cache headers. When you activate the cache E-tag, Last modified & Expires headers will be generated for your media files.
Best regards
Håkan
I am working on a chrome extension which modifies the http response headers.
https://chrome.google.com/webstore/detail/super-cache/fglobbnbihckpkodmeefhagijjcjnbeh/details
I am not able to cache main_frame requests. I am able to control the caching of the static requests though.
For example if I hit http://apple.com I receive the following headers for the main_frame.
Accept-Ranges:bytes
Cache-Control:max-age=276
Connection:keep-alive
Content-Encoding:gzip
Content-Length:3310
Content-Type:text/html; charset=UTF-8
Date:Tue, 12 Mar 2013 09:24:12 GMT
Expires:Tue, 12 Mar 2013 09:28:48 GMT
Server:Apache
Vary:Accept-Encoding
But every time I hit the url the browser tries to access the server and ultimately receive a 200 response. I have tried all the possible combinations that the headers can be set to enable caching on the main_frame.
I want that when the user hits the url from the navigation bar of chrome and no requests are made.
You're missing some sort of cache validation in your response headers. ETag header can be used to control that, by adding values to it that would identify a unique response. You can read a bit about it in the Apache ETag documentation, but I'd simply include ETag: [filename] in your response headers in your example:
Accept-Ranges:bytes
Cache-Control:max-age=276
Connection:keep-alive
Content-Encoding:gzip
Content-Length:3310
Content-Type:text/html; charset=UTF-8
Date:Tue, 12 Mar 2013 09:24:12 GMT
Expires:Tue, 12 Mar 2013 09:28:48 GMT
Server:Apache
ETag: File:"somefile.html"
Vary:Accept-Encoding
These ETag values can include pretty much anything, such as file name, file size, custom values,... that can be separated by a semicolon ;. If the values include spaces, then enclose them in double quotation marks ". For Example:
ETag: File:"YouTube_cd_Fdly3rX8.jpg"; Size:12169
Together with Cache-Control, Expires and some other header values that might change (when included and browser knows how to interpret them), will form a basis for Browser's cache validator.
Looking at your sample response headers, you might want to increase the max-age value in your Cache-Control to a lot higher value, as your example suggest they should be cached client-side for only 276 seconds. The Expires header value also seems a bit short.
More on how to set these values and how browsers are expected to validate cache control headers can be read in the RFC2616, Section 14.9.
EDIT: After further debugging, checking and re-checking the behavior of Chrome's cache validation, it turns out it indeed doesn't respect properly set Cache-Control response headers. On request of the OP, I've reported this issue to the Chrome support:
Chrome, Version 25.0.1364.172 m
Disrespecting Cache-Control on main document requests when serving
static files from a web server, while respecting same header response
on linked contents.
Test setup:
Requesting a static HTML document from a web server (MIME text/html),
that contains another static HTML document withing an IFRAME (also
MIME text/html). The IFRAME served document has same response headers
attached to it by a web server response as the main document:
Date: Thu, 21 Mar 2013 16:29:28 GMT
Expires: Thu, 21 Mar 2013 16:33:59 GMT
Cache-Control: max-age=301, max-stale=299, only-if-cached
Expected behavior:
Main document and the document served within an IFRAME will be cached
locally with initial request for the duration of at least 301
(max-age) seconds, and additional 299 (max-stale) seconds for normal
(non-forced) load requests. Any subsequent requests within this
time-frame of 301 seconds that aren't expected to invalidate local
cache (such as forced-refresh with CTRL+F5 or Reload context menu
command) and are initiated by a normal page load request (e.g.
re-entering relevant URL in the address bar) will be loaded from the
local cache with a status message 200 OK (from cache), if none of the
local cache controlling information indicates it otherwise (same URL,
requested within valid cache time-frame, document was tagged to be
cached in its response headers correctly).
Problem:
The main document is not loaded via its cached copy and an additional
request is made to the web server, resulting in a status code 304 Not
Modified. The document within an IFRAME however is loaded from the
local cache correctly and results in a status message 200 OK (from
cache).
Notes:
None of the cache-control tags or any combination of their values seem
to have any positive effect on the behavior of local cache for the
main document. Including a non-unique ETag value does not resolve the
problem of caching main document either. Other major vendor browsers
(tested in IE, Firefox, Opera) respect Cache-Control headers on main
document.