Apparently "Time to Interact" is the new metric to use when measuring the perceived speed of a webpage. I'm interested in understanding a bit more about what this actually is.
The term was apparently coined by Radware, and is being pushed as the most meaningful performance measurement (compared to things such as Time to First/Last Byte, Time to Render etc.).
It is described as:
the point which a page displays its primary interactive (think
clickable) content, rather than full page load.
This seems pretty subjective to me; what is the "primary interactive content" of a webpage for example?
There have been reports citing results for the measurement, so some how this is being measured, and further, it must be automated as the result sets are pretty big (~500 sites were tested).
Other than the above quote, I cannot find any more information on how to measure this.
As Google are placing more emphasis on above the fold content (or visible content), I am wondering whether this metric is actually more like "Time to First Meaningful Render", i.e. it is contextual to the current page goal. So for example, on an eCommerce site's product page, this could be the main image, and an add to basket link.
I am keen to understand this metric, as to me it does seem like the most useful one. My question is therefore whether anyone is measuring this, and if so how are they doing so?
You kind of answered your own question, it is subjective, and contextual to you current project.
What if I'm testing a site with only HTML without any complex resources? There is no point measuring TTI there. On the other hand, let's see this demo site.
Bigger picture here.
Blue line is marking the "COMContentLoaded" event (main document is loaded and markup parsed), red line indicates the load event, where all page resources are loaded. The TTI line would go in-between the two lines, that is defined differently for each project, based on some essential to interact resources loaded event.
For example, let's say that the pictures on the demo site are not essential to the core features of the site. While the main site loaded in 0.8 seconds, the 3 big pictures took 36 extra seconds to load, so in this case using the overall response time as a KPI would yield ~36second response time, while if you define TTI excluding those big, non essential resources, you end up with < 1s response time.
I am keen to understand this metric, as to me it does seem like the most useful one.
Definitely useful, but as you said it in your question, it's specific to the project. You wouldn't measure TTI on a simple, relatively static web app, you would probably measure overall response time. I always define KPIs "tailored" for the current project, instead of trying to use common metrics, and "force them" on a project.
My question is therefore whether anyone is measuring this, and if so how are they doing so?
Definitely used it before, you should identify the essential resources for your site, and when the last of those resources are loaded, that is your TTI. This could be a javascript file, a css, etc...
Websites are getting more complex. Whereas they might not always contain more content they still have more resources to load as the user interaction/user experience is more complex from a technical point of view. Ajax helps us to load different parts separately. So rather than one page load we have the loading of several small things. And for each of these parts we can measure the loading performance. But there might some parts on the site that might be more important than others. The "primary interactive content" is that part of your view that enables the user to do what he intends to do, for example buy a train ticket. If some advertisement or a special animation on the left side of the screen hasn't loaded this does not prevent the user to buy start buying a ticket. But of course "primary interactive content" as a term is quite vague and you have to define it for your specific application. It is the point an average user can and will start to interact with the website while some parts are sill loading.
This is how I understand the concept and I see the difference to "Time to First Meaningful Render" here: you might have a basket rendered on your eCommerce page but the GUI is not yet responsive. So you see something meaningful but the interactivity is not yet there. Therefore TTI >= TtFMR.
Measuring TTI requires you to define what elements are required for interactivity which not only depends on what the site does but also HOW it does it. So it highly depends on your implementation/technology.
Related
The built-in Sitecore rendering stats http://<sitename>/sitecore/admin/stats.aspx is really helpful for identifying inefficient and slow-loading XSLT renders. Recently I've started switching to .ascx sub layouts to take advantage of the Sitecore C# API which can help improve performance when used correctly.
However, I've noticed that sub layouts (as opposed to XSLT renders) are not reported correctly on the stats page. See the screenshot below....
I know for a fact that this sub layout takes about 1.8 seconds to generate (I calculated this in the code-behind). Caching is turned off. I've refreshed the page 20 times to ensure I get an average. You will see that the "Avg. items" is always 0 - I can live with this - but the "Avg. time (ms)" is less than 1ms which is just clearly wrong.
Does anyone have any insights into this? Has anyone found a way to get it to work correctly?
Judging whether a statistic is right/wrong is going to rely on understanding exactly what it is measuring.
Digging around in Sitecore.Diagnostics.Statistics using Reflector I note the following:
Sitecore.Web.UI.Webcontrol contains a field m_timer
This is 'started' in the BeforeRender() method and 'stopped' in the AfterRender() method
Data from that timer is sent to Statistics.AddRenderingData() and is logged against the control
This means it is measuring the time taken to render the control, which for an XSLT includes the processing time for preparing all the data in it, but as much of the work of a normal ASCX is done prior to the Render-stage the statistic is much less useful. Incorporating the Load stage in the time would inadvertently include the processing time for all child components, since the Load sequence is chained and called recursively, so that probably doesn't help much either.
I suspect there is no good way of measuring the processing time for a specific ASCX control (excluding children) without first acquiring cumulative data then post-processing the call chain and splitting the time apart. This is the sort of thing RedGate ANTS does really well, but might not be so good if it was being executed on a live production system, given the overheads.
I am trying to learn how to use BDD for our development process and I sometimes end-up writing things that implies a UI design, so for brand new development or new features, the UI does not always exists.
For example, if I say this in a scenario "When a column header is clicked" it implies that this feature is based on some sort of table or grid, but at this point we are still just writing user-stories so there is no UI yet.
That gets me confused to know at what point in the process do we come up with a UI design ?
Keep in mind, I only have read articles about BDD and I think it would help our team a lot but still very new at this! Thx!
If you write your scenarios with a focus on the capabilities of the system, you'll be able to refactor the underlying steps within those scenarios more easily. It keeps them flexible. So I'd ask - what does clicking the column get for you? Are you selecting something? What are you going to do with the selection? Are you searching for something and sorting by a value?
I like to see scenarios which say things like:
When I look for the entry
When I go to the diary for January
When I look at the newest entries
When I look at the same T-shirt in black
These could all involve clicking on a column header, but the implementation detail doesn't matter. It's the capability of the system.
Beneath these high-level scenarios and steps I like to create a screen or page with the smaller steps like clicking buttons in it. This makes it easy to refactor.
I wrote this in a DSL rather than English, but it works with the same idea - you can't tell from the steps whether it's a GUI or a web page, and some of the steps involve multiple UI actions:
http://code.google.com/p/wipflash/source/browse/Example.PetShop.Scenarios/PetRegistrationAndPurchase.cs
Hope you find it interesting and maybe it helps. Good luck!
I guess you can write around that by saying "when I sort the information by X, then..." But then you would have to adjust your scenario to remove any mention of the data being displayed in a grid format, which could lead to some rather obtuse writing.
I think it's a good idea to start with UI design as soon as you possibly can. In the case you mentioned above, I think it would be perfectly valid to augment the user story with sketch of the relevant UI as you would imagine it, and then refine it as you go along. A pencil sketch on a piece of paper should be fine. Or you could use a tablet and SketchBook Pro if you want something all digital.
My point is that I don't see a real reason for the UI design to be left out of user stories. You probably already know that you're going to build a Windows, WPF, or Web application. And it's safe to assume that when you want to display tabular data, you'll be using a grid. Keeping these assumptions out of the requirements obfuscates them without adding any real value.
User stories benefit from the fact, that you describe concrete interactions and once you know concrete data and behaviour of the system for it, you might as well add more information about the way you interact. This allows you to use some tools like Cucumber, which with Selenium enables you to translate a story to a test. You might go even further and e.g. for web apps capture all pages you start concrete story at and collect all interactions with that page resulting in some sort of information architecture you might use for documentation or prototyping and later UI testing.
On the other hand, this makes your stories somewhat brittle when it comes to UI changes. I think the agile way of thinking about this is same as when it comes to design changes - do not design for the future, do the simplest possible thing, in the future you might need to change it anyway.
If you stripped your user stories of all concrete things (even inputs) you will end up with use cases(at least in their simplest format, depends on how you write your stories). Use cases are in this respect not brittle at all, they specify only goals. This makes them resistant to change, but its harder to transfer information automatically using tools.
As for the process, RUP/UP derives UI from use cases, but I think agile is in its nature incremental (I will not say iterative, this would exclude agile methods like FDD and Kanban). This means, as you implement new story, you add to your UI what is necessary. This only makes adding UI specifics in stories more reasonable. The problem is, that this is not a very good way to create UI or more generally UX(user experience). This is exactly what one might call a weakpoint of agile. The Agile manifesto concentrates on functional software, but that is it. There are as far as I know no agile techniques for designing UI or UX.
I think you just need to step back a bit.
BAD: When I click the column header, the rows get sorted by the column I clicked.
GOOD: Then I sort the rows by name, or sometimes by ZIP code if the name is very common, like "Smith".
A user story / workflow is a sequence of what the user wants to achieve, not a sequence of actions how he achieves that. You are collecting the What's so you can determine the best How's for all users and use cases.
Looking at a singular aspect of your post:
if I say this in a scenario "When a column header is clicked" it implies that this feature is based on some sort of table or grid, but at this point we are still just writing user-stories so there is no UI yet.
If this came from a user, not from you, it would show a hidden expectation that there actually is a table or grid with column headers. Even coming from you it's not entirely without value, as you might be a user, too. It might be short-sighted, thinking of a grid just because it comes from an SQL query, or it might be spot-on because it's the presentation you expect the data in. A creative UI isnÄt a bad thing as such, but ignoring user expectations is.
I would like to know as a newbie programmer what the benefits are of using for example google search API or newest buzz API for data content gathering instead of screen scraping; obviously apart from the legal aspects.
API's are less likely to change than a screen layout.
One big downside of screen scraping is that the screen can change and break your scraper. So you end up having to continually adjust your code to match theirs, and since you don't know about changes ahead of time, you suffer downtime/outages as a result.
Also, you may be violating their TOS, and they won't like it. If you have paying customers for your service, you can find yourself between a rock and a hard place pretty quickly.
Also, if you're simulating many users, you'll produce an unanticipated drag on the servers. So using a published/permitted API would be much more efficient for you, and for the web site serving up the source material.
I would really appreciate your help with finding out how long it takes a 1-3 year experenced programmer to convert a few HTML pages into joomla 1.5 dynamic pages. I know that some of it depends on how complex the pages are but i'm talking about average pages. That's my first question, my other question is how long will it take a 1-3 year experenced programmer to install all of these componants: Video module, photo gallery module, vertuemart shopping cart. I pay programmers to do this work but i have to make as sure as i can that i'm not over paying them. Thanks in advance for answering these two questions...George
Depends on complexity and quality of html/css design. Usually 1+ hours, if you want additional modules styled( K2, etc..) you need to add extra time, if style is different for every page, than it take more, plus configuration. Basically conversion is not that difficult, just replace main text with content and add some blocks/regions. I would say average about 8 hours
As already mentioned, it depends... there's not really anywhere near enough info to give even a rough estimate.
Is Joomla already installed? If it is installed and the desired template is in place, then cutting-and-pasting some page content can take a few minutes if it's just text. If not, allow 2-3 hours for basic installation, including debugging and testing of the standard components like sending email. Then another 1-2 hours for basic installation of components. Testing, debugging and setup of Virtuemart can take a lot longer, depending on what options for shipping and payments you want.
If you're using a good pre-built template that you're 100% happy with then there should be little to do there, but just positioning and adding discovering what menus work best in which module positions can take a lot of time. Often there is no support in a template for particular components so further styling for the additions is required. Purchased templates vary wildly in quality, some are just not worth the effort, and sometimes template developers take quick-and-dirty shortcuts to get components to look good in their demo, but can take hours to sort out to be useful for anything else.
If you want the Joomla site to look like your old site, or have a custom template built, or radically convert an existing template it can take the length of a piece of string. (One of my clients will easily spend 20 hours endlessly asking me to slightly change spacing, fonts, and colors after signing off on a design and promising that he wouldn't make any more changes. I guess because he can't visualize how things look until he sees the completed site.)
There are plenty of good photo galleries that are bug-free. That shouldn't take long, especially if it comes with a template you already like.
So you may be wondering why all the estimates above vary so much. It just depends on what you've got and what you are really looking for, and what experience the "programmer" has, if it is even in Joomla, or some other CMS, or PHP, or whatever.
Step one in a project like this is to find a programmer you trust not to rip you off; get reliable references if it's someone new. Then get as good an estimate from him or her as he can give, bearing in mind he might have no experience of how you work or how detailed you've been in laying out the plan. Getting an estimate from someone else is not worth much. Then get progress reports as you go along to see where the hours are going so you can judge how to proceed cost-effectively.
is hard to answer at your Q's, the info providet by you is insuficient to make an estimation.
You didn't mentioned anything about customization, complexity of functionalities, integration of those component. Also the time frame depend on the programmer experience and knowledge in Joomla! not just Html knowledge.
Usually, to install some component is easy, lets say for the components you mentioned i need something like 6-8 h, but this is just the installation process. From here to a good joomla! website is a long way. The more time consuming is the customization and integration of all the functionalities, and this depend on clients requests.
You mentioned Virtuemart, this also can be a bottle neck, Virtuemart installation depend on the shop categories and products no., shipping integration, payment method, images processing,
Other Issue can be the template integration, for a good website is better to have the same look on all the components, VM, Photo gallery etc Your template is acquired one or is a custom development?
But to answer at your Q 1,
1) 5 to 10 pages should take 2-4h (text edit, custom typography)
2) 40 to 80h for Video module, photo gallery module, Virtuemart shopping chart
Keep in mind that is just a rough estimation.
Roland
In my opinion, it should take no more than 2-3 hours with minimal configuration included. The only thing that would need more time than this is the video component. Configuration of these components should make the time vary.
I've started to add the time taken to render a page to the footer of our internal web applications. Currently it appears like this
Rendered in 0.062 seconds
Occasionally I get rendered times like this
Rendered in 0.000 seconds
Currently it's only meant to be a guide for users to judge whether a page is quick to load or not, allowing them to quickly inform us if a page is taking 17 seconds rather than the usual 0.5. My question is what format should the time be in? At which point should I switch to a statement such as
Rendered in less than a second
I like seeing the tenths of a second but the second example above is of no use to anyone, in fact it just highlights the limits of the calculation I use to find the render time. I'd rather not let the users see that at all! Any answers welcome, including whether anything should be included on the page.
"Rendered instantly" sounds way better than "Rendered in less than a second".
I'm not sure there's any value in telling users how long it took for the server to render the page. It could well be worth you logging that sort of information, but they don't care.
If it takes the server 0.001 of a second to draw the page but it takes 17 seconds for them to load it (due to network, javascript, page size, their rubbish PC, etc) their perception will be the latter.
Then again adding the render time might help you fend off the enquiries about any percieved slowness with a "talk to your local network admin" response.
Given that you know the accuracy of your measurements you could have the 0.000 text be "Rendered in less than a thousandth of a second"
Rather than relying on your users to look at the page footer and to let you know if the value exceeds some patience threshold, it might be a better idea to log the page render times in a log file on the server. Once you have all that raw data, you can look for particular pages that tend to take longer than normal to render.
With more detailed logging, you could also measure the elapsed times in database queries or whatever if your web app relies on external systems.
I think I over-emphasized it was for the users.
I know by using in trace in the web.config I can get accurate information on page render times along with times for accessing the database.
We have in the past had problems with applications running too slowly over the network although it's now fixed I'm adding the label to new applications so that users are aware it is something we are taking seriously and it's a very simple indicator for the developers.
Taking all that into account I like "Rendered Instantly" and write a lot of sense so I'll accept both your answer and kokos'.
Thanks