I have a backend API service that requires bearer-token authorization, and it is fronted by Azure API Management. One particular endpoint returns a large dataset (at least 3MB in size) but the endpoint has no parameters so any given response is valid for all users.
I have an external Azure Redis Cache configured for this APIM instance, and I want to add caching to this one endpoint, with the response cached for ALL users (not per-user).
To start off, I'm trying to get this working on a different parameter-less endpoint that only returns a 3Kb response - when I get it working, I'll apply the policy to the 3MB+ endpoint.
I tried adding a simple cache-store/cache-lookup policy:
<cache-lookup vary-by-developer="false" vary-by-developer-groups="false"
downstream-caching-type="none" caching-type="external" />
But that didn't work, and the following cache-lookup trace message was displayed:
"Request contains Authorization header. Cache Lookup policy was not applied."
So, I had to add another parameter to the cache-lookup policy:
allow-private-response-caching="true"
When I did this, the cache-lookup ran, but the trace message showed it was a miss:
{
"message": "Cache lookup resulted in a miss. Cache headers listed below were removed from the request to prompt the backend service to send back a complete response.",
"cacheKey": "**************************************************************",
"cacheHeaders": [
{
"name": "Cache-Control",
"value": "no-cache, no-store"
}
]
}
However, the cache-store trace message said that the response would be cached:
"Response will be buffered during streaming and stored in the cache after it is received in full."
So I ran the trace again, but it resulted in another "cache miss". No matter how many times I run the trace, it always says the cache is a miss, but the cache-key is identical every time. The cache duration is 3600, so the response should be cached for an hour.
If the key isn't changing, why does each lookup always result in a miss?
Related
I have a question in regard to how CloudFront will use an S3 object's ETag to determine if it needs to send a refreshed object or not.
I know that the ETag will be part of the Request to the CloudFront distribution, in my case I'm seeing the "weak" (shortened) version:
if-none-match: W/"eabcdef4036c3b4f8fbf1e8aa81502542"
If this ETag being sent does not match the S3 Object's current ETag value, then the CloudFront will send the latest version.
I'm seeing this work as expected, but only after the CloudFront's cache policy has been reached. In my case it's been set to 20 mins.
CloudFront with a Cache Policy:
Minimum TTL: 1
Maximum TTL: 1200 <-- (20 mins)
Default TTL: 900
Origin Request Policy is not set
S3 Bucket:
Set to only allow access via its corresponding CloudFront
distribution above.
Bucket and objects not public
The test object (index.html) in this case has only one header set:
Content-Type = text/html
While I am using the CloudFront's Cache Policy, I've also tested
using the S3 Object header of Cache-Control = max-age=6000
This had no affect on the refresh of the "index.html" object in
regard to the ETag check I'm asking about.
The Scenario:
Upon first "putObject" to that S3 bucket, the "index.html" file has an ETag of:
eabcdef4036c3b4f8fbf1e8aa81502542
When I hit the URL (GET) for that "index.html" file, the cache of 20 mins is effectively started.
Subsequent hits to the "index.html" URL (GET) has the Request with the value
if-none-match: W/"eabcdef4036c3b4f8fbf1e8aa81502542"
I also see "x-cache: Hit from cloudfront" in the Response coming back.
Before the 20 mins is up, I'll make a change to the "index.html" file and re-upload via a "putObject" command in my code.
That will then change the ETag to:
exyzcde4099c3b4f8fuy1e8aa81501122
I would expect then that the next Request to CloudFront, before the 20-minute TTL and with the old "if-none-match" value, would then prompt the CloudFront to see the ETag is different and send the latest version.
But in all cases/tests it doesn't. CloudFront will seem to ignore the ETag difference and continue to send the older "index.html" version.
It's only after the 20 mins (cache TTL) is up that the CloudFront sends the latest version.
At that time the ETag in the Request changes/updates as well:
if-none-match: W/"exyzcde4099c3b4f8fuy1e8aa81501122"
Question (finally, huh?):
Is there a way to configure CloudFront to listen to the incoming ETag, and if needed, send the latest Object without having to wait for the Cache Policy TTL to expire?
UPDATE:
Kevin Henry's response explains it well:
"CloudFront doesn't know that you updated S3. You told it not to check with the origin until the TTL has expired. So it's just serving the old file until the TTL has expired and it sees the new one that you uploaded to S3. (Note that this doesn't have anything to do with ETags)."
So I decided to test how the ETag would be used if I turned the CloudFront Caching Policy to a TTL of 0 for all three CloudFront settings. I know that this defeats the purpose, and one of the strengths, of CloudFront, but I'm still wrapping my head around certain key aspects of CDN caching.
After setting the cache to 0, I'm seeing a continual "Miss from CloudFront" in the Response coming back.
I expected this, and in the first response I see a HTTP status of 200. Note the file size being returned is 128KB for this test.
Subsequent calls to this same file return a HTTP status of 304, with a file size being returned around 400B.
As soon as I update the "index.html" file in the S3 bucket, and call that same URL, the status code is 200 with a file size of 128KB.
Subsequent calls return a status of 304, again with an average of 400B in file size.
Looking again at the definition of an HTTP status of 304:
https://httpstatuses.com/304
"A conditional GET or HEAD request has been received and would have resulted in a 200 OK response if it were not for the fact that the condition evaluated to false.
In other words, there is no need for the server to transfer a representation of the target resource because the request indicates that the client, which made the request conditional, already has a valid representation; the server is therefore redirecting the client to make use of that stored representation as if it were the payload of a 200 OK response."
So am I correct in thinking that I'm using the Browser's cache at this point?
The calls to the CloudFront will now pass the requests to the Origin, where the ETag is used to verify if the resource has changed.
As it hasn't, then a 304 is returned and the Browser kicks in and returns its stored version of "index.html".
Would this be a correct assumption?
In case you're wondering, I can't use the invalidation method for clearing cache, as my site could expect several thousand invalidations a day. I'm hosting a writing journal site, where the authors could update their files daily, therefore producing new versions of their work on S3.
I would also rather not use the versioning method, with a timestamp or other string added as a query to the page URL. SEO reasons for this one mainly.
My ideal scenario would be to serve the same version of the author's work until they've updated it, at which time the next call to that same page would show its latest version.
This research/exercise is helping me to learn and weigh my options.
Thanks again for the help/input.
Jon
"I would expect then that the next Request to CloudFront, before the 20-minute TTL and with the old if-none-match value, would then prompt the CloudFront to see the ETag is different and send the latest version."
That is a mistaken assumption. CloudFront doesn't know that you updated S3. You told it not to check with the origin until the TTL has expired. So it's just serving the old file until the TTL has expired and it sees the new one that you uploaded to S3. (Note that this doesn't have anything to do with ETags).
CloudFront does offer ways to invalidate the cache, and you can read more about how to combine that with S3 updates in these answers.
We can enable bucket versioning and object with new etag is picked up by the cloudfront
I am using a cache mediator inside the proxy, that proxy I am calling the inside sequence(I need to use cache into several API that's why I am calling that proxy inside sequence). then I am calling into the rest API.
Issue: when I am hitting the rest API services through postman, cache is not working.
the same services I am hitting from SoapUI, cache is working fine(second time onwards the response is coming from cache storage).
it's chrome also it's working.
I believe Postman sends a random token in every request. Postman-Token: <Token>. Cache mediator works by checking the headers and payload of the request. When a random header value is sent every time, cache mediator will diagnose it as a different request. Hence the response won't be served from the Cache.
To overcome the issue add the "Postman-Token" header in the cache mediator configuration under <headersToExcludeInHash/>
https://docs.wso2.com/display/EI660/Cache+Mediator
I read Http spec. but I have a doubt and I hope someone can help me.
When a cache receives a request and has a stored response that must be validated (before being served to the received request), does the cache send the received request (adding the conditional header fields it needs for validation) to the next server OR does the cache generate a new request (with conditional header fields it needs for validation) and send the generated request to the next server?
Thank you very much! :)
I think the idea is that the client would issue the request with the key headers, and the server would either respond with the content or a 304 to use whatever was in the local cache.
This behavior should be the same for upstream caches along the network path all the way to the source of truth.
"When a cache receives a request..."
Cache doesn't receive HTTP request. It is user-agent (browser) that check cache to see whether there is any cache entry matched for an HTTP request. Cache itself is just a bunch of data stored in disk/memory.
"Does the cache send the received request...OR does the cache generate a new request..."
Cache doesn't send HTTP request. It is user-agent (browser)'s job to send the request.
In summary, cache is just bytes of data, it doesn't know when and where HTTP request is sent. All cache validation logic (cache related HTTP headers) is implemented by user-agent.
I am facing issue where i am sending one GET request using Jmeter. Response headers shows every time it miss Varnish cache on Server and response is returned from Application Sever. Please find below header
X-Cache: MISS
X-Cache-Hits: 0
If i send exact same request using Postman, first time it miss Varnish cache but if i send same request again, it hits Varnish cache and cache hits counts increased.
X-Cache → HIT
X-Cache-Hits → 1
I have tried Jmeter versions 2.6,2.9,2.11 and 2.13, but observed same behavior. Even when request is sent from Fiddler, i can see from header response is returned from Varnish Cache itself.
It just simple get request. I have compared JMeter and Postman request, both requests are exact same. Please let me know how i can resolve this issue.
Based on when you wrote above, I can guess that:
All 1st requests are processed in the same way, doesn't matter how they were send.
As a part of response to your first request, server returns you a command to set up new header, in the same way as it process cookies (SET-COOKIE logic). So, server expects that your next request will contain this required X-Cache header.
But Jmeter is not a browser and doesn't correlate next request with previously received data (by default at least). So, all is OK if you replay this scenario with browser (and its extensions). And your Jmeter sends the same request every time.
If you compare 1st and 2nd request sent by your browser, you'll find that your 2nd request contains required hearer.
So, if I'm right, to resolve the issue:
Identify the way how your server tells the client to add new header to next request (Javascript?)
Implement this logic in your Jmeter scenario.
Or just add X-Cache header to your request.
My expectation is that Postman (whatever it is) respects ETag header while JMeter doesn't.
I believe adding HTTP Cache Manager should resolve your issue.
I have added unique key in header every time request goes top server and it started returning response from Varnish cache. Unique is random number.
i also checked Postman also send one unique parameter in each and every request. Though I am still not sure why unique is making difference here.
I'm getting the following error intermittently when invoking the custom search api from a server side setup:
HttpError 403 when requesting https://www.googleapis.com/customsearch/v1?q=John+Doe+john%40simpler.com&alt=json&cx=&key= returned "There is a per-IP or per-Referer restriction configured on your API key and the request does not match these restrictions. Please use the Google Developers Console to update your API key configuration if request from this IP or referer should be allowed.
I'm using a server api key, and have confirmed that the configured server ip address is correct. And about 50% of the time my request come back fine, too. I'm issuing the request from the server like this:
service = build("customsearch", "v1",
developerKey=api_key)
custom_search_context = <my_context>
res = service.cse().list(
q=search_query_string,
cx=custom_search_context,
).execute()
My requests per sec are well with in the configured limit of 10/sec and daily purchased limit of 5000 requests.
One more thing I noticed is that Google counts a forbidden request towards the daily limit, too.
Any pointers on why I'm being presented with the error only intermittently would be very helpful
The error can be raised when you're exceeding a request/second limit. Can you confirm that your request rate is below your configured user rate limit? It might be worth noting that the limit is enforced even if you don't explicitly provide a user value in your requests.