We're using Azure CDN, but we've stumbled upon a problem. Before, content could not be updated. But we added the option for our users to crop their picture, which changes the thumbnails. See, the image is not being created as new, instead we just update the stream of the blob.
There doesn't seem to be any method to clear the cache, update any headers or anything else.
Is the only answer here to make a new blob and delete the old?
Thanks.
CDN would still cache the content, unless the cache-expiry passes, or the file name changes.
CDN is best for static content with a high cache hit ratio.
Using CDN for dynamic content is not recommended, because it causes the user to wait for a double hop from storage to cdn and from cdn to user.
You also pay twice the bandwidth on the initial load.
I guess the only workaround right now is to pass a dummy parameter in the request from the client to force download the file every time.
http://resourceurl?dummy=dummyval
Related
I have a website forum where users exchange photos and text with one another on the home page. The home page shows 20 latest objects - be they photos or text. The 21st object is pushed out out of view. A new photo is uploaded every 5 seconds. A new text string is posted every second. In around 20 seconds, a photo that appeared at the top has disappeared at the bottom.
My question is: would I get a performance improvement if I introduced a CDN in the mix?
Since the content is changing, it seems I shouldn't be doing it. However, when I think about it logically, it does seem I'll get a performance improvement from introducing a CDN for my photos. Here's how. Imagine a photo is posted, appearing on the page at t=1 and remaining there till t=20. The first person to access the page (closer to t=1) will enable to photo to be pulled to an edge server. Thereafter, anyone accessing the photo will be receiving it from the CDN; this will last till t=20, after which the photo disappears. This is a veritable performance boost.
Can anyone comment on what are the flaws in my reasoning, and/or what am I failing to consider? Also would be good to know what alternative performance optimizations I can make for a website like mine. Thanks in advance.
You've got it right. As long as someone accesses the photo within the 20 seconds that the image is within view it will be pulled to an edge server. Then upon subsequent requests, other visitors will receive a cached response from the nearest edge server.
As long as you're using the CDN for delivering just your static assets, there should be no issues with your setup.
Additionally, you may want to look into a CDN which supports HTTP/2. This will provide you with improved performance. Check out cdncomparison.com for a comparison between popular CDN providers.
You need to consider all requests hitting your server, which includes the primary dynamically generated HTML document, but also all static assets like CSS files, Javascript files and, yes, image files (both static and user uploaded content). An HTML document will reference several other assets, each of which needs to be downloaded separately and thus incurs a server hit. Assuming for the sake of argument that each visitor has an empty local cache, a single page load may incur, say, ~50 resource hits for your server.
Probably the only request which actually needs to be handled by your server is the dynamically generated HTML document, if it's specific to the user (because they're logged in). All other 49 resource requests are identical for all visitors and can easily be shunted off to a CDN. Those will just hit your server once [per region], and then be cached by the CDN and rarely bother your server again. You can even have the CDN cache public HTML documents, e.g. for non-logged in users, you can let the CDN cache HTML documents for ~5 seconds, depending on how up-to-date you want your site to appear; so the CDN can handle an entire browsing session without hitting your server at all.
If you have roughly one new upload per second, that means there is likely a magnitude more passive visitors per second. If you can let a CDN handle ~99% of requests, that's a dramatic reduction in actual hits to your server. If you are clever with what you cache and for how long and depending on your particular mix of anonymous and authenticated users, you can easily reduce server loads by a magnitude or two. On the other side, you're speeding up page load times accordingly for your visitors.
For every single HTML document and other asset, really think whether this can be cached and for how long:
For HTML documents, is the user logged in? If no, and there's no other specific cookie tracking or similar things going on, then the asset is static and public for all intents and purposes and can be cached. Decide on a maximum age for the document and let the CDN cache it. Even caching it for just a second makes a giant difference when you get 1000 hits per second.
If the user is logged in, set the cache pragma to private, but still let the visitor's browser cache it for a few seconds. These headers must be decided upon by your forum software while it's generating the document.
For all other assets which aren't access restricted: let the CDN cache it for a long time and you can practically forget about ever having to serve those particular files ever again. These headers can be statically configured for entire directories in the web server.
I have problem with caching in CDN for Azure Storage. I have set up storage, add CDN on it, and put custom domain.
And all works fine, until recently, when one image I uploaded, get stuck on CDN. When I'm getting to it without CDN all works fine, but over CDN it always shown old image. I tried everything, I put custom cache expiration, I deleted, I moved it... But nothing works. And I waited for one day, maybe will fix by Auzura automatically, or some caching will expiry, but nothing.
Does anybody had similar problem before? How to fix it?
All other images (blobs) in same container works fine.
You have to be really careful when planning the CDN usage. When you enable for CDN, you have to be in full control of all and any Blob files that will be served through the CDB.
This is by being explicit about setting x-ms-blob-cache-control property on a Put Blob, Put Block List, or Set Blob Properties request, or use the Azure Managed Library to set the BlobProperties.CacheControl property.
In case you forgot to set this property before the file was accessed by the CDN, the CDN assumes 7 days as TTL (Time-to-live for that file). Any consequent change of the settings (cache-control property of the blob) will not take effect until after the 7 days TTL elapses. I believe you have accidentally entered into this default 7 days TTL (hoping it is not the worst - wrongly set cache-expire header with a longer period)
You can read more on best practices for controlling CDN content here. And I warmly ask you to give your 3 votes at this feature request.
I have some image urls which I want to cache locally and save so that I don't need to make a web request again and again as needed.
Now, I am confused whether there is any significant benefit of using webclient's openreadasync method over bitmap for fetching the image for first time for saving it to IsolatedStorage.
For me, I think bitmap would be a better option as I would be able to get a event for progress.
This post gives good info on various image caching options.
http://blogs.msdn.com/b/swick/archive/2011/04/07/image-tips-for-windows-phone-7.aspx
Matt mentioned the fact that default image caching only works per session. So if you are implementing your own Image caching, then you will have to implement a image downloader for which the WebClient OpenReadAsync provides a way to store file locally
If you were't considering a local cache, UriSource would have been the choice.
If you want to cache images beyond the current application instance lifetime, have a look at http://blogs.msdn.com/b/delay/archive/2010/10/04/there-s-no-substitute-for-customer-feedback-improving-windows-phone-7-application-performance-now-a-bit-easier-with-lowprofileimageloader-and-deferredloadlistbox-updates.aspx which will show a way of saving the images to IsolatedStorage and then display it from there. This means you won't have to get it over the network each time the app is run.
If you're using this for lots of images be sure to manage the images you save as well so you don't fill up the disk with lots of old images you'll never need again.
I'm trying to wrap my head around what approach I should use to force CDN refreshes of user profile photos on a website where CloudFront is the CDN serving the profile photos, and S3 is the underlying file store.
I need to ensure that user profile photos are up to date as soon as a user updates their profile photos. I see three options that I can do to update profile photos and ensure that website users get the latest image as soon as profile photos are revised. Of these approaches, is one better than the other in terms of ensuring fresh content and maximum long term cost effectiveness? Are there better approaches to ensuring fresh content and maximum long term cost effectiveness?
Issue one S3 put object request to save the file with its original file name, and issue one Amazon CloudFront invalidation request. Amazon CloudFront allows up to 1000 free invalidation requests per month which seems a bit on the low side
Issue one S3delete object request to delete the original photos, then one S3 put object request to save the new photo with a unique, new photo file name. This would be two S3 requests per file update, and would not require a CloudFront CDN invalidation request. CloudFront would then serve the latest files as soon as they were updated, providing image URLs were automatically set to the new file names
Issue one S3 put object request to save the file with its original file name, and then client side append a version code to the CDN URLs (i.e. /img/profilepic.jpg?x=timestamp) or something along that line. I'm not sure how effective this strategy is in terms of invalidating cached CloudFront objects
Thanks
CloudFront invalidation can take a while to kick in and is recomendded as a last resort to remove content that must be removed (like a copyright infringement).
The best approach is you versioned URLs. For profile images I would use an unique ID (such as a GUID). Whenever a user uploads a new photo replace that URL (and delete the old photo if you wish).
When you update your DB with the new ID of the user profile photo CloudFront will pull the new image and the change will be immediate.
I need a way for cache images and html files in PhoneGap from my site. I'm planning that users will see site without internet connection like it will be with it. But I see information only about sql data storing, but how can I store images (and use later).
To cache images check out this library -of which I'm the creator-:
imgcache.js
. It's designed for the very purpose of caching images using the local filesystem. If you check out the examples you will see that it can also detect when an image fails to be loaded (because you're offline or you have a very bad connection) and then replaces it automatically with the cached image. The user of the webapp doesn't even notice it's offline.
As for html pages.. if they're html static files, they could be stored locally in the web app (file:// in phonegap).
If they're dynamically generated pages, check the localStorage API if you have a small amount of data, otherwise the filesystem API.
For my web app I retrieve only json data from my server (and process/render it using Backbone+Underscore). The json payload is stored into the localStorage. If the application gets offline, it will fetch json data from the localStorage instead of the server (home-baked fork of Backbone.dualStorage)
You then get the full offline experience: pages+images.
Caching like you might need for simple offline operation is not exactly that easy.
Your first option is the cache manifest. It has some limitations (like the size of the cache) but might work for you since it was designed to do what you want.
Another options is that you can store content on the disk of the device using the file system APIs. This has some drawbacks like security and the fact that you have to load the file from a path / url that is different than you might normally load it from on the web. Check out the hydra plugin for an example of this.
One final option might be to store stuff in localStorage (which has the benefit of being private on all platforms) and then pull it out of there when needed ... that means base64'ing all your images tho so that is a pretty big departure from just standard caching.
Caching is very much possible on Android OS. but on Apple as stated above there are limitations with the size of the images and cache size etc.
If you are willing to integrate and allow the caching on iOS you can use "cache manifest" to do so. but keep the draw backs and limitations in mind.
Also
if you want to save the file to Documents folder under my App, Apple will reject your App. The reason is the system backup all data under Documents folder to iCould after iOS6, so Apple does not allow big data like images or JSON file which could sync from your server again to keep in this folder.
So there is another work around which is good So one can use LocalFileSystem.TEMPORARY instead. It does not save the data to Library/Cache, but it save data to temp folder of App, which does not been auto backup to iCloud and not auto deleted either.
Regards
Rajeev