So lets say I have a CloudFront distribution and I call /path1 then /path1 again soon after. The flow will be
1st request is a CloudFront miss and goes to server
2nd request is a CloudFront hit
But what if theres 2 parallel hits to CloudFront. At the same time such that when the 2nd request reaches CloudFront before the 1st request finishes such that CloudFront does not have the cached response yet. Will it wait for the 1st request to finish and return that? Or both requests will hit the server?
When looking at my NGINX logs, it looks like when parallel calls reaches server, they all reach server. Is there any way to avoid that? Like if a rogue client makes too many request for same path, I was hopping that CloudFront can only make 1 request to server and return that same response. Is that possible?
If the first request is still serving to the client, CloudFront hasn't cached it yet , one the first request is complete which means CloudFront received complete data from Origin and served it to the client, it keeps the response in the cache and served further requests.
If you're making second request before the first request is completed, it'll a MISS from cloudfront and you'll see both requests reached to CloudFront.
However, if you're talking about burst of parallel requests , CloudFront has a way to handle them:
Simultaneous Requests for the Same Object
"When a CloudFront edge location receives a request for an object and either the object isn't currently in the cache or the object has expired, CloudFront immediately sends the request to your origin. If there's a traffic spike—if additional requests for the same object arrive at the edge location before your origin responds to the first request—CloudFront pauses briefly before forwarding additional requests for the object to your origin. Typically, the response to the first request will arrive at the CloudFront edge location before the response to subsequent requests. This brief pause helps to reduce unnecessary load on your origin server. If additional requests are not identical because, for example, you configured CloudFront to cache based on request headers or cookies, CloudFront forwards all of the unique requests to your origin."
Related
I have a question in regard to how CloudFront will use an S3 object's ETag to determine if it needs to send a refreshed object or not.
I know that the ETag will be part of the Request to the CloudFront distribution, in my case I'm seeing the "weak" (shortened) version:
if-none-match: W/"eabcdef4036c3b4f8fbf1e8aa81502542"
If this ETag being sent does not match the S3 Object's current ETag value, then the CloudFront will send the latest version.
I'm seeing this work as expected, but only after the CloudFront's cache policy has been reached. In my case it's been set to 20 mins.
CloudFront with a Cache Policy:
Minimum TTL: 1
Maximum TTL: 1200 <-- (20 mins)
Default TTL: 900
Origin Request Policy is not set
S3 Bucket:
Set to only allow access via its corresponding CloudFront
distribution above.
Bucket and objects not public
The test object (index.html) in this case has only one header set:
Content-Type = text/html
While I am using the CloudFront's Cache Policy, I've also tested
using the S3 Object header of Cache-Control = max-age=6000
This had no affect on the refresh of the "index.html" object in
regard to the ETag check I'm asking about.
The Scenario:
Upon first "putObject" to that S3 bucket, the "index.html" file has an ETag of:
eabcdef4036c3b4f8fbf1e8aa81502542
When I hit the URL (GET) for that "index.html" file, the cache of 20 mins is effectively started.
Subsequent hits to the "index.html" URL (GET) has the Request with the value
if-none-match: W/"eabcdef4036c3b4f8fbf1e8aa81502542"
I also see "x-cache: Hit from cloudfront" in the Response coming back.
Before the 20 mins is up, I'll make a change to the "index.html" file and re-upload via a "putObject" command in my code.
That will then change the ETag to:
exyzcde4099c3b4f8fuy1e8aa81501122
I would expect then that the next Request to CloudFront, before the 20-minute TTL and with the old "if-none-match" value, would then prompt the CloudFront to see the ETag is different and send the latest version.
But in all cases/tests it doesn't. CloudFront will seem to ignore the ETag difference and continue to send the older "index.html" version.
It's only after the 20 mins (cache TTL) is up that the CloudFront sends the latest version.
At that time the ETag in the Request changes/updates as well:
if-none-match: W/"exyzcde4099c3b4f8fuy1e8aa81501122"
Question (finally, huh?):
Is there a way to configure CloudFront to listen to the incoming ETag, and if needed, send the latest Object without having to wait for the Cache Policy TTL to expire?
UPDATE:
Kevin Henry's response explains it well:
"CloudFront doesn't know that you updated S3. You told it not to check with the origin until the TTL has expired. So it's just serving the old file until the TTL has expired and it sees the new one that you uploaded to S3. (Note that this doesn't have anything to do with ETags)."
So I decided to test how the ETag would be used if I turned the CloudFront Caching Policy to a TTL of 0 for all three CloudFront settings. I know that this defeats the purpose, and one of the strengths, of CloudFront, but I'm still wrapping my head around certain key aspects of CDN caching.
After setting the cache to 0, I'm seeing a continual "Miss from CloudFront" in the Response coming back.
I expected this, and in the first response I see a HTTP status of 200. Note the file size being returned is 128KB for this test.
Subsequent calls to this same file return a HTTP status of 304, with a file size being returned around 400B.
As soon as I update the "index.html" file in the S3 bucket, and call that same URL, the status code is 200 with a file size of 128KB.
Subsequent calls return a status of 304, again with an average of 400B in file size.
Looking again at the definition of an HTTP status of 304:
https://httpstatuses.com/304
"A conditional GET or HEAD request has been received and would have resulted in a 200 OK response if it were not for the fact that the condition evaluated to false.
In other words, there is no need for the server to transfer a representation of the target resource because the request indicates that the client, which made the request conditional, already has a valid representation; the server is therefore redirecting the client to make use of that stored representation as if it were the payload of a 200 OK response."
So am I correct in thinking that I'm using the Browser's cache at this point?
The calls to the CloudFront will now pass the requests to the Origin, where the ETag is used to verify if the resource has changed.
As it hasn't, then a 304 is returned and the Browser kicks in and returns its stored version of "index.html".
Would this be a correct assumption?
In case you're wondering, I can't use the invalidation method for clearing cache, as my site could expect several thousand invalidations a day. I'm hosting a writing journal site, where the authors could update their files daily, therefore producing new versions of their work on S3.
I would also rather not use the versioning method, with a timestamp or other string added as a query to the page URL. SEO reasons for this one mainly.
My ideal scenario would be to serve the same version of the author's work until they've updated it, at which time the next call to that same page would show its latest version.
This research/exercise is helping me to learn and weigh my options.
Thanks again for the help/input.
Jon
"I would expect then that the next Request to CloudFront, before the 20-minute TTL and with the old if-none-match value, would then prompt the CloudFront to see the ETag is different and send the latest version."
That is a mistaken assumption. CloudFront doesn't know that you updated S3. You told it not to check with the origin until the TTL has expired. So it's just serving the old file until the TTL has expired and it sees the new one that you uploaded to S3. (Note that this doesn't have anything to do with ETags).
CloudFront does offer ways to invalidate the cache, and you can read more about how to combine that with S3 updates in these answers.
We can enable bucket versioning and object with new etag is picked up by the cloudfront
I have a web application that requests a lot of media assets in parallel using AJAX. All assets are coming from the same Cloudfront Origin, which is itself directly plugged into an S3 bucket.
I'm seeing requests from Cloudfront with TTFB of the order of seconds. Even more odd, it seems that those requests are basically queued until a previous request has been served:
Those two requests are initiated in parallel, and you can see that it's not Chrome queueing them, but Cloudfront not answering anything to the second (2KB) request until the first request has completed download. This is slowing down my application by a huge margin, and I cannot figure out what is going wrong... I see the same behavior when I check with Safari too.
Here are the two requests details
As you can see, they are also both Hit from cloudfront.
Finally, as it might be relevant, I'm using a lambda function in my Origin's behavior to add the proper Vary headers, to prevent Chrome from using cached requests without the CORS headers that will make subseqeuent CORS request fail (see details here).
Here is my complete Origin's behavior settings:
Any help is appreciated, and please feel free to ask more details if needed! Thanks a lot in advance.
I am facing issue where i am sending one GET request using Jmeter. Response headers shows every time it miss Varnish cache on Server and response is returned from Application Sever. Please find below header
X-Cache: MISS
X-Cache-Hits: 0
If i send exact same request using Postman, first time it miss Varnish cache but if i send same request again, it hits Varnish cache and cache hits counts increased.
X-Cache → HIT
X-Cache-Hits → 1
I have tried Jmeter versions 2.6,2.9,2.11 and 2.13, but observed same behavior. Even when request is sent from Fiddler, i can see from header response is returned from Varnish Cache itself.
It just simple get request. I have compared JMeter and Postman request, both requests are exact same. Please let me know how i can resolve this issue.
Based on when you wrote above, I can guess that:
All 1st requests are processed in the same way, doesn't matter how they were send.
As a part of response to your first request, server returns you a command to set up new header, in the same way as it process cookies (SET-COOKIE logic). So, server expects that your next request will contain this required X-Cache header.
But Jmeter is not a browser and doesn't correlate next request with previously received data (by default at least). So, all is OK if you replay this scenario with browser (and its extensions). And your Jmeter sends the same request every time.
If you compare 1st and 2nd request sent by your browser, you'll find that your 2nd request contains required hearer.
So, if I'm right, to resolve the issue:
Identify the way how your server tells the client to add new header to next request (Javascript?)
Implement this logic in your Jmeter scenario.
Or just add X-Cache header to your request.
My expectation is that Postman (whatever it is) respects ETag header while JMeter doesn't.
I believe adding HTTP Cache Manager should resolve your issue.
I have added unique key in header every time request goes top server and it started returning response from Varnish cache. Unique is random number.
i also checked Postman also send one unique parameter in each and every request. Though I am still not sure why unique is making difference here.
We have setup Cloudfront with our own server as its origin and have a json call that is cached for 60 seconds (max-age), and say we successfully cache the response in Cloudfront. Now what happens when Cloudfront tries to update the json response after 60 seconds by calling our server and our server responds with an error (or no response at all when it is down). Does it keep on serving the old response, or return an error?
According to the api docs:
If your origin server is unavailable and CloudFront gets a request for an object that is in the edge cache
but that has expired (for example, because the period of time specified in the Cache-Control max-age
directive has passed), CloudFront continues to serve the expired version of the object. For more information
about object expiration
For JPEG image delivery in my web app, I am considering using Amazon S3 (or Amazon Cloudfront
if it turns out to be the better option) but have two, possibly opposing,
requirements:
The images are private content; I want to use signed URLs with short expiration times.
The images are large; I want them cached long-term by the users' browser.
The approach I'm thinking is:
User requests www.myserver.com/the_image
Logic on my server determines the user is allowed to view the image. If they are allowed...
Redirect the browser (is HTTP 307 best ?) to a signed Cloudfront URL
Signed Cloudfront URL expires in 60 seconds but its response includes "Cache-Control max-age=31536000, private"
The problem I forsee is that the next time the page loads, the browser will be looking for
www.myserver.com/the_image but its cache will be for the signed Cloudfront URL. My server
will return a different signed Cloudfront URL the second time, due to very short
expiration times, so the browser won't know it can use its cache.
Is there a way round this without having my webserver proxy the image from Cloudfront (which obviously negates all the
benefits of using Cloudfront)?
Wondering if there may be something I could do with etag and HTTP 304 but can't quite join the dots...
To summarize, you have private images you'd like to serve through Amazon Cloudfront via signed urls with a very short expiration. However, while access by a particular url may be time limited, it is desirable that the client serve the image from cache on subsequent requests even after the url expiration.
Regardless of how the client arrives at the cloudfront url (directly or via some server redirect), the client cache of the image will only be associated with the particular url that was used to request the image (and not any other url).
For example, suppose your signed url is the following (expiry timestamp shortened for example purposes):
http://[domain].cloudfront.net/image.jpg?Expires=1000&Signature=[Signature]
If you'd like the client to benefit from caching, you have to send it to the same url. You cannot, for example, direct the client to the following url and expect the client to use a cached response from the first url:
http://[domain].cloudfront.net/image.jpg?Expires=5000&Signature=[Signature]
There are currently no cache control mechanisms to get around this, including ETag, Vary, etc. The nature of client caching on the web is that a resource in cache is associated with a url, and the purpose of the other mechanisms is to help the client determine when its cached version of a resource identified by a particular url is still fresh.
You're therefore stuck in a situation where, to benefit from a cached response, you have to send the client to the same url as the first request. There are potential ways to accomplish this (cookies, local storage, server scripting, etc.), and let's suppose that you have implemented one.
You next have to consider that caching is only just a suggestion and even then it isn't a guarantee. If you expect the client to have the image cached and serve it the original url to benefit from that caching, you run the risk of a cache miss. In the case of a cache miss after the url expiry time, the original url is no longer valid. The client is then left unable to display the image (from the cache or from the provided url).
The behavior you're looking for simply cannot be provided by conventional caching when the expiry time is in the url.
Since the desired behavior cannot be achieved, you might consider your next best options, each of which will require giving up on one aspect of your requirement. In the order I would consider them:
If you give up short expiry times, you could use longer expiry times and rotate urls. For example, you might set the url expiry to midnight and then serve that same url for all requests that day. Your client will benefit from caching for the day, which is likely better than none at all. Obvious disadvantage is that your urls are valid longer.
If you give up content delivery, you could serve the images from a server which checks for access with each request. Clients will be able to cache the resource for as long as you want, which may be better than content delivery depending on the frequency of cache hits. A variation of this is to trade Amazon CloudFront for another provider, since there may be other content delivery networks which support this behavior (although I don't know of any). The loss of the content delivery network may be a disadvantage or may not matter much depending on your specific visitors.
If you give up the simplicity of a single static HTTP request, you could use client side scripting to determine the request(s) that should be made. For example, in javascript you could attempt to retrieve the resource using the original url (to benefit from caching), and if it fails (due to a cache miss and lapsed expiry) request a new url to use for the resource. A variation of this is to use some caching mechanism other than the browser cache, such as local storage. The disadvantage here is increased complexity and compromised ability for the browser to prefetch.
Save a list of user+image+expiration time -> cloudfront links. If a user has an non-expired cloudfront link use it for an image and don't generate a new one.
It seems you already solved the issue. You said that your server is issuing a redirect http 307 to the cloudfront URL (signed URL) so the browser caches only the cloudfront URL not your URL(www.myserver.com/the_image). So the scenario is as follows :
Client 1 checks www.myserver.com/the_image -> is redirect to CloudFront URL -> content is cached
The CloudFront url now expires.
Client 1 checks again www.myserver.com/the_image -> is redirected to the same CloudFront URL-> retrieves the content from cache without to fetch again the cloudfront content.
Client 2 checks www.myserver.com/the_image -> is redirected to CloudFront URL which denies its accesss because the signature expired.