GWT like perfect caching for non-GWT resources - caching

Perfect Caching
Perfect Caching allows the browser to cache resources forever and guarantees that changed resources are propagated immediately to the browser. In GWT this is accomplished by naming a file like the hash code of its content. As this leads to a new file name each time the content of a file changes, the browser can cache these files forever without having to ask the server whether or not there is a new version available. More about perfect caching can be found here.
As this concept works very well I would like to apply it to all of my non-GWT resources (some JS, CSS and image files).
As my build process is Maven based it would be great if there was a plugin which could automatically rename the files according to their hash code and which could adjust all of the references to this file.
Here's an example
Renaming
foo.css -> foo_39757cec04498955db62043f7ecfefc2.cache.css
logo.png -> logo_35bcdbbabe1944afc75eeeb16f06d1ad.cache.png
Update references in files
Replace all occurances of "foo.css" by "foo_39757cec04498955db62043f7ecfefc2.cache.css"
Replace all occurances of "logo.png" by "logo_35bcdbbabe1944afc75eeeb16f06d1ad.cache.png"
Does anybody know whether something like that already exists?
Many thanks in advance,
Michael

If you're using Apache (httpd) you can setup mod_pagespeed to do that. Since GWT does some of that stuff already, be careful to properly configure mod_pagespeed so that performance doesn't degrade.

mod_pagespeed indeed disables HTML caching by default, as typically HTML is dynamic. However, you can override that behavior.
There is a new option in the most recent binary release (0.10.21.*) that provides the behavior you want: explicitly control of your HTML caching headers:
ModPagespeedModifyCachingHeaders off
http://code.google.com/speed/page-speed/docs/install.html#ModifyCachingHeaders

Related

Jenkins UI customization

I want to change the look and feel(ui customization) of Jenkins. Also I would like to add new views(say like new html pages or web pages) with navigation to the required jenkins pages etc.
Please let me know if any single plugins will help me to do so.Any relevant information(how ever generic) will be very helpful.
Any suggestions or links or tutorials is also appreciated.
PS:- Pretty new to jenkins.The inputs from here will help me to add more details to the questions.
I am looking for documents or tutorials that specify Skinning Jenkins using plugins like :-
https://wiki.jenkins-ci.org/display/JENKINS/Simple+Theme+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/jQuery+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/jQuery+UI+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/JSWidgets+Plugin
The plugin page is providing very little information on how to use these and the benefits and the extend to which the UI can be changed.
Any doc or link is appreciated.
Assuming you don't want to write a Jenkins plugin, for adding pages, the best suggestion I can make is to use an HTTP proxy such as NginX, and configure it so that the pages you want to add are plain html files, and Jenkins is proxied for the rest of them. To a visitor, they will look like they are all part of the same site; you could copy code from the head and body sections of Jenkins-served pages to include some of the navigation.
The Simple Theme Plugin, which you found, will let you do basic customization of the look and feel of Jenkins. I do that for my build server and proxy it using this configuration fragment for NginX. The relevant CSS is in this CSS file - toward the end, look for the // JENKINS CUSTOMIZATION comment.
We use the Simple theme plugin - pointed at a css file for the simple styling, and a JS file to fix a couple of DOM oddities (some of the tables in the new look and feel have mismatching column counts).
Those two files need only be hosted either a handy http server, or you can place them in usercontent.
You need only refresh the page in the browser to see the changes. Both files can then happily reference other files served up too.
Handy things to note:
Jenkins has jquery, parts of YUI loaded and prototype loaded - so you can use them in your scripts.
If while debugging, the refresh gets in the way then use the console to enter the following to temporarily stop it without pausing JS: refreshPart = function() {}
When making DOM tree changes to content that is refreshed - attach it to the layout updates with:
layoutUpdatecallback.add(my_function) - that way your changes are applied to new incoming content.

How to use bootstrap size option in joomla 3.0?

I m new to joomla world. pls can any1 tell how to use bootstrap size option in joomla 3.0?
and i have 1 more question, what is the use of index.html in every modules folder which has no content in it ?
Second question answer
Web servers list all its directory-content in the browser if there's not present an index.html, making it easy for attackers to click on any of the links and view the contents; worse, if it's a PHP file, which will invariably execute upon clicking. That brings three risks:
Direct access to a PHP file exposes sensitive information (e.g. the
server's path structure) to directly alter codes.
It makes easier uploading hacking scripts to a site through any of
its vulnerable component. This allows for direct web access which
compromises the site.
It reveals the names and size of the site's files and helps
identifying any vulnerable extension, making it an easy target
The index.html files prevent the file listings from such exposures.
The "bootstrap size" option in the module parameters has to be supported by the used module chrome. From the default system chromes, only the html5 one does support it. Depending on your template, there may be other chromes as well which do support it. But since it's a rather new parameter, most templates probably don't support it yet.

How do I set caching headers for my CSS/JS but ensure visitors always have the latest versions?

I'd like to speed up my site's loading time in part by ensuring all CSS/JS is being cached by the browser, as recommend by Google's PageSpeed tool. But I'd like to ensure that visitors have the latest CSS/JS files, if they are updated and the cache now contains old code.
From my research so far, appending something like "?459454" to the end of the CSS/JS url is popular. But wouldn't that force the visitor's browser to re-download the CSS/JS file every time?
Is there a way to set the files to be cached by the browser, but ensure the browser knows about updated versions of the cached files?
If you're using Apache, you can use mod_pagespeed (mentioned earlier by symcbean) to do this automatically.
It would work best if you also use the ModPagespeedLoadFromFile directive since that will create a new URL as soon as it detects that the resource has changed on disk, however it will work fine without that (it will use the cache expiry time returned when it fetches the resource to rewrite it).
If you're using nginx, you could use ngx_pagespeed.
If you're using IIS, you could use IISpeed, which is not a Google product and I don't know it's full feature set.
Version numbers will work, but you can also append a hash of the file to the filename with your web framework or asset build script:
<script src="script-5054a101c8b164cbfa570d97fe23cc0d.js"></script>
That way, once your HTML changes to reflect this new version, browsers will just download and cache the updated version of your script.
As you say, append a query string to the URL of the asset, but only change it if the content is different, or change it when you deploy a new version.
appending something like "?459454" to the end of the CSS/JS url is popular. But wouldn't that force the visitor's browser to re-download the CSS/JS file every time?
No it won't force them to download each time, however there are a lot of intermediate proxies out there which ignore query strings on cacheable content - hence many tools (including mod_pagespeed which does automatic url rewriting based on file conents, and content merging on the fly along with lots of other cool tricks) move the version information into the path / filename.
If you've only got .htaccess type access then you can strip the version information out to map direct to a file, or use a scripted 404 redirector (but this is probably only a good idea if you're behind a caching reverse proxy).

Versioning and caching static files: CSS, JS, images -- What to consider

For an (enterprise) web project i want to keep previous versions of the static files so that projects can decide for themselves when they are ready to implement design changes. My initial plan is to provide folders for static content like so:
company.com/static/1.0.0/
company.com/static/1.0.0/css/
company.com/static/1.0.0/js/
company.com/static/1.0.0/images/
company.com/static/2.0.0/
company.com/static/2.0.0/css/
company.com/static/2.0.0/js/
company.com/static/2.0.0/images/
Each file in these folders should then have a cache-policy to cache "forever" -- one year at least. I also plan to concatenate css files and js files into one, in order to minimize number of requests.
Then i would also provide a current folder (which symlinks to the latest released version)
company.com/static/current/
company.com/static/current/css/
company.com/static/current/js/
company.com/static/current/images/
This will solve my first problem (that projects and sub websites can lock their code to a certain version and can upgrade whenever they are ready).
But then I can see some caching issue. Now i cannot "just" cache current folder, since it will change for each release. What should my caching policies be on that folder.
Also, for each release, most of the static files will never change anyway. Is it relevant to cache them forever, and rename if there are changes?
I am looking for advice here, since i want to know about your best trade-off between caching and changing the files.
Beware of HTTP caching. I looked into this some time ago.
my blog article on the HTTP caching
There are three approaches you can select from:
Use resource's path as a cache key, i.e. when it changed - the browser will have to download new version of your resources. In this case you don't need /current folder at all, you just need to avoid .html page caching and put appropriate path to your resources in it.
You can point browser to /current folders only and add ETag to your resources, in this case another server request will be made from the client, but it will be conditional request (i.e. with If-None-Match header), so you can return 304 response (with no resource body) until your customer decide to migrate to another version. Another drawback of such solution (if you have several customers who use different versions) is that /current folder will contain only some single version of the design.
As you're going to concatenate resources into single files, you can specify resource version as part of url: /current/js/combined.js?version=1.0.0.0 But this is not much different from first approach.
Hope this helps.
It might be worth your while looking at how Google, Microsoft etc. have implemented the caching policies for their jQuery CDNs
Your policy of caching forever is OK for the versioned URLs.
For the current URLs you're obviously going to need a shorter expiry time.
Couple of things to consider:
How are the applications going to be able to test against /current/ i.e. if they use it how do you know a change isn't going to break an existing application?
Caching forever is only really about reducing requests during the 'current session' as most browser caches aren't big enough to hold files for a long time (they get removed as people browse others sites)

client-side file caching

If I understand correctly, a broswer caches images, JS files, etc. based on the file name. So there's a danger that if one such file is updated (on the server), the browser will use the cached copy instead.
A workaround for this problem is to rename all files (as part of the build), such that the file name includes an MD5 hash of it's contents, e.g.
foo.js -> foo_AS577688BC87654.js
me.png -> me_32126A88BC3456BB.png
However, in addition to renaming the files themselves, all references to these files must be changed. For exmaple a tag such as <img src="me.png"/> should be changed to <img src="me_32126A88BC3456BB.png"/>.
Obviously this can get pretty complicated, particularly when you consider that references to these files may be dynamically created within server-side code.
Of course, one solution is to completely disable caching on the browser (and any caches between the server and the browser) using HTTP headers. However, having no caching will create it's own set of problems.
Is there a better solution?
Thanks,
Don
The best solution seems to be to version filenames by appending the last-modified time.
You can do it this way: add a rewrite rule to your Apache configuration, like so:
RewriteRule ^(.+)\.(.+)\.(js|css|jpg|png|gif)$ $1.$3
This will redirect any "versioned" URL to the "normal" one. The idea is to keep your filenames the same, but to benefit from cache. The solution to append a parameter to the URL will not be optimal with some proxies that don't cache URLs with parameters.
Then, instead of writing:
<img src="image.png" />
Just call a PHP function:
<img src="<?php versionFile('image.png'); ?>" />
With versionFile() looking like this:
function versionFile($file){
$path = pathinfo($file);
$ver = '.'.filemtime($_SERVER['DOCUMENT_ROOT'].$file).'.';
echo $path['dirname'].'/'.str_replace('.', $ver, $path['basename']);
}
And that's it! The browser will ask for image.123456789.png, Apache will redirect this to image.png, so you will benefit from cache in all cases and won't have any out-of-date issue, while not having to bother with filename versioning.
You can see a detailed explanation of this technique here: http://particletree.com/notebook/automatically-version-your-css-and-javascript-files/
Why not just add a querystring "version" number and update the version each time?
foo.js -> foo.js?version=5
There still is a bit of work during the build to update the version numbers but filenames don't need to change.
Renaming your resources is the way to go, although we use a build number and embed that in to the file name instead of an MD5 hash
foo.js -> foo.123.js
as it means that all your resources can be renamed in a deterministic fashion and resolved at runtime.
We then use custom controls to generate links to resources at on page load based upon the build number which is stored in an app setting.
We followed a similar pattern to PJP, using Rails and Nginx.
We wanted user avatar images to be browser cached, but on an avatar's change we needed the cache to be invalidated ASAP.
We added a method to the avatar model to append a timestamp to the file name:
return "/images/#{sourcedir}/#{user.login}-#{self.updated_at.to_s(:flat_string)}.png"
In all places in the code where avatars were used, we referenced this method rather than an URL. In the Nginx configuration, we added this rewrite:
rewrite "^/images/avatars/(.+)-[\d]{12}.png" /images/avatars/$1.png;
rewrite "^/images/small-avatars/(.+)-[\d]{12}.png" /images/small-avatars/$1.png;
This meant if a file changed, its URL in the HTML changed, so the user's browser made a new request for the file. When the request reached Nginx, it got rewritten to the simple name of the file.
I would suggest using caching by ETags in this situation, see http://en.wikipedia.org/wiki/HTTP_ETag. You can then use the hash as the etag. A request will still be submitted for each resource, but the browser will only download items that have changed since last download.
Read up on your web server / platform docs on how to use etags properly, most decent platforms have built-in support.
Most modern browsers check the if-modified-since header whenever a cacheable resource is in a HTTP request. However, not all browsers support the if-modified-since header.
There are three ways to "force" the browser to load a cached resource.
Option 1 Create a query string with a version#. src="script.js?ver=21". The downside is many proxy servers wont cache a resource with query strings. It also requires site-wide updating for changes.
Option 2 Create a naming system for your files src="script083010.js". However the downside to option 1 is that this as well requires site-wide updates whenever a file changes.
Option 3 Perhaps the most elegant solution, simply set up the caching headers: last-modified and expires in your server. The main downside to this is users may have to recache resources because they expired yet never changed. Additionally, the last-modified header does not work well when content is being served from multiple servers.
Here a few resources to check out: Yahoo Google AskApache.com
This is really only an issue if your web server sets a far-future "Expires" header (setting something like ExpiresDefault "access plus 10 years" in your Apache config). Otherwise, a browser will make a conditional GET, based on the modified time and/or the Etag. You can verify what is happening on your site by using a web proxy or an extension like Firebug (on the Net panel). Your question doesn't mention how your web server is configured, and what headers it is sending with static files.
If you're not setting a far-future Expires header, there's nothing special you need to do. Your web server will usually handle conditional GETs for static files based on last modified time just fine. If you are setting a far-future Expires header then yes, you need to add some sort of version to the file name like your question and the other answers have mentioned already.
I have also been thinking about this for a site I support where it would be a big job to change all references. I have two ideas:
1.
Set distant cache expiry headers and apply the changes you suggest for the most commonly downloaded files. For other files set the headers so they expire after a very short time - eg. 10 minutes. Then if you have a 10 minute downtime when updating the application, caches will be refreshed by the time users go to the site. General site navigation should be improved as the files will only need downloading every 10 minutes not every click.
2.
Each time a new version of the application is deployed to a different context that contains the version number. eg. www.site.com/app_2_6_0/ I'm not really sure about this as users bookmarks would be broken on each update.
I believe that a combination of solutions works best:
Setting cache expiry dates for each type of resource (image, page, etc) appropreatly for that resource, for example:
Your static "About", "Contact" etc pages probably arn't going to change more than a few time a year, so you could easily put a cache time of a month on these pages.
Images used in these pages could have eternal cache times, as you are more likey to replace an image then to change one.
Avatar images might have an expiry time of a day.
Some resources need modified dates in their names. For example avatars, generated images, and the like.
Some things should never be caches, new pages, user content etc. In these cases you should cache on the server, but never on the client side.
In the end you need to carfully consider each type of resource to determine what cache time to instruct the browser to use, and always be conservitive if you are unsure. You can increase the time later, but it's much more pain to uncache something.
You might want to check out the approach taken by the grails "uiperformance" plugin, which you can find here. It does a lot of the things you mention, but automates them (set expiry time to a long time, then increments version numbers when files change).
So if you're using grails, you get this stuff for free. If you are not - maybe you can borrow the techniques employed.
Also - borrowed form the ui-performance page, - read the following 14 rules.
ETags seemingly provide a solution for this...
As per http://httpd.apache.org/docs/2.0/mod/core.html#fileetag, we can set the browser to generate ETags on file-size (instead of time/inode/etc). This generation should be constant across multiple server deployments.
Just enable it in (/etc/apache2/apache2.conf)
FileETag Size
& you should be good!
That way, you can simply reference your images as <img src='/path/to/foo.png' /> and still use all the goodness of HTTP caching.

Resources