I use JMeter for checking load testing.
I note a time with stopwatch when i check load time personally it was
8.5 seconds
when i run same case with JMeter it gave load time of 2 seconds
There is huge difference between them, How can i verify the actual time?
e.g : if one user taking 9 seconds to load the form while in JMeter it is given load time 2 seconds
Client time is a complex item, as you can see from the clip from the Chrome Developer tools, performance tab, above. There's lots going on at the client which does lead to a difference between the time you see with an HTTP protocol test tool, such as JMETER (and most of the other performance test tools on the planet) and the actual client render.
You can address this Delta in a number of ways:
Run a single GUI Virtual user. Name your timing records such as "Login" and "login_GUI." The delta between the two is your client weight. Make sure to run the GUI virtual user on a dedicated host to avoid resource contention
Run a test with all browsers. This was state of the art in 1995. Because of the resource cost and the skew imposed on trying to figure out the cost of the server response the entire industry shifted to protocol level virtual users. Some are trying to bring back this model as "state of the art." It is not
Ask a performance question earlier, also known as "shift left..." Every developer has these developer tools at their disposal, as does every functional tester. If you find that a client is slow for one user, be curious and use the developer tools to identify, "why?" If you are waiting to multi user performance testing to answer questions related to client weight, then you have waited too long and often will not have the time or resources to change the page architecture in meaningful ways to reduce the client page cost. This is where understanding earlier has tremendous advantages for making changes.
I picked the graphic above deliberately to illustrate the precise challenge you have. Notice, the loading of the components takes less than a tenth of a second. These are the requests that JMETER would be making. But the page takes almost five seconds to "render." Jmeter is not broken, it is working as designed. It is your understanding that needs to change on which tools can be used to pull particular stats for analysis.
You can't compare JMeter load time to browser as is, also because your browser will load JavaScript files and can call JavaScript functions on page load while JMeter doesn't execute JavaScript.
JMeter is not a browser, it works at protocol level. As far as
web-services and remote services are concerned, JMeter looks like a
browser (or rather, multiple browsers); however JMeter does not
perform all the actions supported by browsers. In particular, JMeter
does not execute the Javascript found in HTML pages. Nor does it
render the HTML pages as a browser does (it's possible to view the
response as HTML etc., but the timings are not included in any
samples, and only one sample in one thread is ever displayed at a
time).
Just a side note - you can use plugin to check exact load time in chrome.
Well-behaved JMeter test timing should be equal or similar to real user timing, if there is a 4x times difference - most probably your JMeter configuration is not correct.
Probably the most important. Make sure your HTTP Request samplers are configured to retrieve so called "embedded resources" (images, scripts, styles) which are referenced in the web page
If your application is using AJAX technology make sure you execute AJAX-driven requests as well and add their elapsed time to main sampler using i.e. Transaction Controller.
Make sure you mimic browser's:
Cookies via HTTP Cookie Manager
Headers via HTTP Header Manager
Cache via HTTP Cache Manager
Assuming all above you should be receiving similar to real user experience page load time. See How to make JMeter behave more like a real browser article for more detailed information on the above tips.
In addition to the answers provided by James and user7294900, please find these images to help you understand the reason behind the difference in time given by your stop watch and JMeter.
Below image gives the ideology behind how JMeter provides the time.
Below image gives the ideology behind how you have measured the time with
your stop watch.
Notice that there are additional actions performed by the browser when you are taking the time using your stop watch. This is the reason behind the huge difference in time between JMeter and your stop watch.
In addition to this, ensure that you are using the same test environmental conditions for both the tests (like same network conditions, same LG etc.)
Hope this helps!
Across all browsers/devices, I find random different pages, at random times, are very slow to load/don't load. The browser is stuck on 'Waiting for website.com'. I will wait 20 seconds and nothing will happen until I manually refresh the page. As I realise this is very vague, can you suggest a) most likely issues to look for first or b) some diagnostic tools that I could use to try and de-bug the issue as a starting point, so that my hosts/developers can solve the issue. Here are some results of recent speed tests.
One thing to also add is that, it seems it more often gets stuck on particular pages. Namely the pages where users take practice tests. After each time the user clicks 'Next', their selected answer is inserted into the database. My speculation is that potentially it's an issue with the DB itself, or the process which inserts into the database. It's when clicking 'Next', that the whole website sometimes just dies as described above.
Results from Google Speed Test
Waterfall image
A wait time of 20secs at random times and random pages could possibly be due to stop-the-world garbage collection. So GC logs are probably a good starting point.
A thread sampler such as Djigger a colleague of mine wrote might probably also help you figuring out what the machine is doing during the 20 seconds.
If that doesn't help I suggest to use a Profiler or better an APM tool to monitor whats going on on your system. Those tools give a you a broader insight of the internals.
You need to run a few page speed tests and look at the waterfall images.
It is very common on shared servers for the server to be too busy to get to your request. 20 seconds would indicate a serious issue with the server.
Another common reason is the page has a link to a third party resource and that resource is too often unavailable.
In your case the culprit is website.com and I assume that is your site.
Use something like webpagetest.org to run the tests.
In the waterfall image below
Dark Green is DNS lookup time.
Orange is the time for Browser to connect to server.
Green is the wait time for server to put image in output buffer.
Blue is the time for the server to transmit to the Browser.
The problem with the sample waterfall page is the index page took 4 seconds to be generated or retrieved. Most likely this is a Word Press site with plugins.
I suspect yours may be 20 seconds. But due to the randomness, is is also a good possibility it is a page resource that is stalled.
If it is the index page, then you likely have a poor ISP and or one of the other users of the server is hogging the CPU.
Keep running the tests until you see the problem occur.
It will be very obvious where the problem is located.
You can post the waterfall image and send me a message if you have any questions.
Waterfall from webpagetest.org
I would like to measure loading time of a document in my testing web app. I have used JMeter for this, but I am getting different values for each run. I am measuring average time in the summary report.
I am not sure, that the value is proper or not.Is this approach is correct or Is there any plugin JMeter available?
I have used HTTP watch to get rendering time, but I can't use that tool for more than 1 user (Load Testing). I am using JMeter 2.13. Could you please help me in this?
With the help of aggregate report or csv / xml results you get required information regarding response times BUT
In Jmeter, Response time = Processing time + Latency(time taken by network while transferring data)
In Browser, Response time = Processing time + Latency + Rendering time
Hence you will found a difference between http watch response times and jmeter response times.
If you need to include rendering times also in your response times, then use tools, like loadrunner (commercial), selenium (open source) and so on. Personally in my opinion client side rendering is not a measurable value, unless all of the users accessing the application are having same configuration of hardware, software and network access. However, while JMeter test running with peak load to the system, manually browse the site using various browsers and with the help of developer tools you can find rendering times.
I am getting different values for each run - this will depends upon test data you are using, server health status, network delays and so on.
I doubt you'll be able to get 2 fully identical test run results, there always will be some form of fluctuation caused by underlying hardware and software implementations. You should be receiving similar results with some statistical noise.
If this is not the case, your JMeter test might be misconfigured. From "realness" perspective mind the following configuration:
Make sure you have Retrieve All Embedded Resources from HTML Files box checked and you Use concurrent pool. The easiest way to configure it for all the samplers is using HTTP Request Defaults
Add HTTP Cache Manager to your test plan. Previous setting "tells" JMeter to fetch embedded resources like scripts, styles, etc. from the pages. Real browsers do it as well but they do it only once, on subsequent requests these resources are being returned from browser's cache.
Add HTTP Cookie Manager to your test plan. It represents browser cookies, enables cookie-based authentication and maintains sessions.
Add HTTP Header Manager to represent browser headers like User-Agent, Content-Type, encoding, etc.
When you use a straight HTTP Protocol layer virtual user, independent of the tool (Jmeter, LoadRunner, SOASTA, Grinder, ...) then what you will be timing will be the request/response information coming from the server with very low coloration from the local processing on the client for JavaScript and the final "drawing on the screen" which is rendering.
Up until the point where the server is degraded due to number of requests or network limitations the only area where you can tune is in the page architecture, which if you are waiting to the last 100 yards before deployment to address then you are likely in trouble.
Steve Souders has written quite a bit on the subject of page architecture in his books "High Performance Websites" and related works. In short, the rule of thumb comes down to making fewer requests, smaller responses and serving the data from the closest possible location to the client. These have the effect of minimizing the most expensive finite resource to a web client, the network. For instance, a browser sprite reduces the number of calls for images, minification and compression reduce the size of the transmission and a CDN changes the number of hops to the requested item to a location closer to the end client.
In order to affect changes to page architecture you need to move upstream into your development cycle and your functional testing cycle. You will need to work with development to implement hard gates where code/pages cannot be submitted to the project without first passing performance gates related to design. Your development team and functional testing members will need to respect those gates. As to what the gates should be, I refer you back to the works of Mr Souders as a great source of data for construction of your gate rules.
This gets you to the level of "works for one: Performant for one." Then you can use that as a known good to answer the questions related to server scalability and at which point the service to the client from requests begins to degrade. If you have a CDN in your organization, be sure to take that into account in your test model, for if you do not then you will overload your server vs production.
As far as actual speeding of the "rendering" or drawing on the screen? You need to purchase a faster video card barring changes from the browser manufacturer. Speeding up JavaScript? Make sure that all of your JavaScript is as small and as lean as possible. Have your functional test team test on very dirty browsers with lots of add-ins as well as lower powered hardware for a view of maximum out of spec response. If you need a view of what your standard hardware model looks like from your clients (Browser/OS/some hardware into) then you can process the data in your HTTP request logs and specifically the user agent related to client configuration information.
To what extent does Google Analytics impact performance?
I'm looking for the following:
Benchmarks (including response times/pageload times et al)
Links or results to similar benchmarks
One (possible) method of testing Google Analytics (GA) on your site:
Serve ga.js (the Google Analytics JavaScript file) from your own server.
Update from Google Daily (test 1) and Weekly (test 2).
I would be interested to see how this reduces the communication between the client webserver and the GA server.
Has anyone conducted any of these tests? If so, can you provide your results? If not, does anyone have a better method for testing the performance hit (or lack thereof) for using GA?
2018 update: Where and how you mount Analytics has changed over and over and over again. The current gtag.js code does a few things:
Load the gtag script but async (non-blocking). This means it doesn't slow your page down in any other way than bandwidth and processing.
Create an array on the page called window.datalayer
Define a little gtag() function that just pushes whatever you throw at it into that array.
Calls that with a pageload event.
Once the main gtag script loads, it syncs this array with Google and monitors it for changes. It's a good system and unlike the previous systems (eg stuffing code in just before </body>) it means you can call events before the DOM has rendered, and script order doesn't really matter, as long as you define gtag() first.
That's not to say there isn't a performance overhead here. We're still using bandwidth on loading up the script (it's cached locally for 15 minutes), and it's not a small pile of scripts that they throw at you, so there's some CPU time processing it.
But it's all negligible compared to (eg) modern frontend frameworks.
If you're going for the absolute, most cut-down website possible, avoid it completely. If you're trying to protect the privacy of your users, don't use any third party scripts... But if we're talking about an average modern website, there is much lower hanging fruit than gtag.js if you're hitting performance issues.
There are some great slides by Steve Souders (client-side performance expert) about:
Different techniques to load external JavaScript files in parallel
their effect on loading time and page rendering
what kind of "in progress" indicators the browser displays (e.g. 'loading' in the status bar, hourglass mouse cursor).
I haven't done any fancy automated testing or programmatic number crunching, but using good old Firefox with the Firebug plugin and a pair of JS variables to tell the time difference before and after all GA code is executed, here is what I found.
Two things are downloaded:
ga.js is the JavaScript file containing the code. This is 9kb, so the initial download is negligible and the filename isn't dynamic so it's cached after the first request.
a 35 byte gif file with a dynamic url (via query string args), so this is requested every time. 35 bytes is a negligible download as well (firebug says it took me 70ms to dl it).
As far as execution time, my first request with a clean browser cache was an average of about 330ms each time and subsequent requests were between 35 and 130 ms.
From my own experience it has adding Google-Analytics has not changed the load times.
According to FireBug it loads in less then a second (648MS avg), and according so some of my other test ~60% - 80% of that time was transferring the data from the server, which of course will vary from user to user.I don't preticularly think that caching the analytics code locally will change the load times much, for the above reasons.
I use Google-Analytics on more then 40 websites without it ever being the cause of any, even small, slowdown, the most amount of time is spent getting the images which, due to their typical sizes, is understandable.
You can host the ga.js on your servers with no problems whatsoever, but the idea is that your users will have the ga.js cached from some other site they may have visited. So downloading ga.js, because it's so popular, adds very little overhead in many cases (i.e., it's already been cached).
Plus, DNS lookups do not cost the same in different places due to network topology. Caching behavior would change depending on whether users use other sites that include ga.js or not.
Once the JavaScript has been loaded, the ga.js does communicate with Google servers, but that is an asynchronous process.
There's no/minimal site overhead on the server side.
The HTML for Google Analytics is three lines of javascript that you place at the bottom of your webpage. It's nothing really, and doesn't consume any more server resource than a copyright notice.
On the client side, the page can take a little bit (up to a couple of seconds) of time to finish displaying a page. However - In my experience, the only bit of the page not loaded is the Google stuff, so users can see your page perfectly fine. You just get the throbber at the top of the page throbbing for a little longer.
(Note: You need to place your google analytics code block at the bottom of any served pages for this to be the case. I don't know what happens if the code block is placed at the top of your HTML)
The traditional instructions from Google on how to include ga.js use document.write(). So, even if a browser would somehow asynchronously load external JavaScript libraries until some code is actually to be executed, the document.write() would still block the page loading. The later asynchronous instructions do not use document.write() directly, but maybe insertBefore also blocks page loading?
However, Google sets the cache's max-age to 86,400 seconds (being 1 day, and even set to be public, so also applicable to proxies). So, as many sites load the very same Google script, the JavaScript will often be fetched from the cache. Still, even when ga.js is cached, simply clicking the reload button will often make a browser ask Google about any changes. And then, just like when ga.js was not cached yet, the browser has to await the response before continuing:
GET /ga.js HTTP/1.1
Host: www.google-analytics.com
...
If-Modified-Since: Mon, 22 Jun 2009 20:00:33 GMT
Cache-Control: max-age=0
HTTP/1.x 304 Not Modified
Last-Modified: Mon, 22 Jun 2009 20:00:33 GMT
Date: Sun, 26 Jul 2009 12:08:27 GMT
Cache-Control: max-age=604800, public
Server: Golfe
Note that many users click reload for news sites, forums and blogs they already have open in a browser window, making many browsers block until a response from Google is received. How often do you reload the SO home page? When Google Analytics response is slow, then such users will notice right away. (There are many solutions published on the net to asynchronously load the ga.js script, especially useful for these kind of sites, but maybe no longer better than Google's updated instructions.)
Once the JavaScript has loaded and executed, the actual loading of the web bug (the tracking image) should be asynchronous. So, the loading of the tracking image should not block anything else, unless the page uses body.onload(). In this case, if the web bug fails to load promptly then clicking reload actually makes things worse because clicking reload will also make the browser request the script again, with the If-Modified-Since described above. Before the reload the browser was only awaiting the web bug, while after clicking reload it also needs the response for the ga.js script.
So, sites using Google Analytics should not use body.onload(). Instead, one should use something like jQuery's $(document).ready() or MooTools' domready event.
See also Google's Functional Overview, explaining How Does Google Analytics Collect Data?, including How the Tracking Code Works. (This also makes it official that Google collects the contents of first-party cookies. That is: the cookies from the site you're visiting.)
Update: in December 2009, Google has released an asynchronous version. The above should tell everyone to upgrade just to be sure, though upgrading does not solve everything.
It really depends on the day. I'm just adding this to a blog. I'm in california, very close to their main data centers, on a fast low latency business DSL, on a overclocked i5 with plenty of RAM running a recent linux kernel and stable firefox.
here's a sample page load:
google-analytics alone added 5 seconds just of network download time... to get 15Kb!
You can see blogger.com served 34Kb in 300 mili seconds. That's 32x faster!
Also, look how the Red Line (which represents the onLoad event, meaning, there's no more script executing on the page and the so the browser can finally stops the loading indicators/spinings/etc) ... look how far to the right it is. that's probably 3seconds of garbage javascript processing that happened there. It's very uncommon for that line to be very far away from the end of the resources download bars. I'm done debugging this and it's 1/3 analytics fault, 2/3 blogger fault. ...one would think google stuff was fast.
Edit:
Some more data. Here's a request with everything cached. the above one was first visit.
I've removed the googleplus crap from above for two reasons, I was trying to see if they were playing some part on the slow onLoad event (they aren't) and because It is mostly useless.
So, With this we can see that the network time is the least of your worries. Even on a fast computer with modern software, the toll google analytics + blogger take on processing time will still dump your page load past 7s. Without the blogger, just check this very site, i'm seeing 0.5s of delay after resources are loaded and the red line kicks in.
Loading any extra javascript to your page is going to increase the download time from the client's perspective. You can ameliorate this by loading it at the bottom of your page so that your page is rendered even if GA is not loaded. I would avoid caching because you would lose the advantage of the client cache for your page. If the client has it cached from some other page, your page's request will be filled from the client itself. If you change it to load from your site, it will require a download even if the client already has the code (which is likely). Adding a task to your software processes to avoid loading the file from Google seems unwarranted for what may be an unnecessary optimization. It would be hard to test this since it would always serve up faster locally, but what really matters is how fast it works for your customers. If you decide to evaluate keeping it locally, make sure you test it from your home internet connection --- not the machine sitting next to the server in your rack.
Use FireBug and YSlow to check for yourself. What you will discover however is that GA is about 9KB in size (which is actually quite substantially for what it does) and that it also sometimes does NOT load very fast (for what reasons I don't know, I think it might be the servers "choking" sometimes)
We removed it due to performance issues on our Ajax Samples, but then again for us being ultra fast and responsive was priority 1, 2 and 3
Nothing noticeable.
The call to Google (including DNS lookup, loading the Javascript if not already cached and the actual tracer calls themselves) should be done by the client's browser in a separate thread to actually loading your page. Certainly the DNS lookup will be done by the underlying system and will not, to my knowledge, count as a lookup within the browser (browsers have a limit on the number of request threads they will use per site).
Beyond that, the browser will load the Google script in parallel along with all other embedded resources, so you will potentially get an extremely slight increase in the time it takes to download everything, in the worst case (we're talking in the order of milliseconds, unnoticable. If the Google script is loaded last by the browser, or you don't have many external resources on your page, or if your page's external resources are cached by the browser, or if Google's script is cached by the browser (extremely likely) then you won't see any difference. It's just absolutely trivial overall, the same effect as sticking an extra tiny picture on your page, roughly speaking.
About the only time it might make a concrete difference is if you have some behaviour that fire on the onLoad event (which waits for external resources to load), and the Google servers are down/slow. The latter is unlikely to happen often, but if this were the case then the onLoad even won't fire until the script is downloaded. You can work around this anyway by using various "when DOM loaded" events, which are generally more responsive as you don't have to wait for your own scripts/images to load this way either.
If you're really that worried about the effects on page load time, then have a look a the "Net speed" section of Firebug, which will quantify this and draw you a pretty graph. I would encourage you to do this for yourself anyway as even if other people give you the figures and benchmarks you request, it will be completely different for your own site.
Well, I have have searched, researched and expored extensively on net. But I have not found any statistical data that claims either in favour or against of the premise.
However, this excerpt from http://www.ga-experts.com claims that its a Myth that GA slows down your website.
Err, well okay, maybe slightly, but
we’re talking about milliseconds. GA
works by page tagging, and any time
you add more content to a web page, it
will increase loading times. However
if you follow best practice (adding
the tag before the </body> tag) then
your page will load first. Also, bear
in mind that any page tag based web
analytics package (which is the
majority) will work the same way
From the answers above and all other sources, what I feel is that whatever slowdown it causes in not percieved by the user as the Script is included at the bottom of the page. But if we talk of complete page-loads we might say that it slows down the page-load time.
Please post in more info if you have and DATA if you have any.
I don't think this is what your looking for but what are you worried about performance for?
If its your server... then there's obviously no impact as it resides on Google servers.
If its your users that your worried about then there is no impact either. As long as you place it just above the body tag then your users will not receive anything slower than they would before... the script is loaded last and has no affect on the appearance to the user. So there essentially not waiting on anything and even continue to browse through the page without noticing that its still loading.
The question was will Google Analytics cause your site to slow down and the answer is yes. Right now at the time of writing this Google-Analytics.com is not working so sites that have that in their pages won't load the pages so yes, it can slow down and cause your site to not even load. It's uncommon for google-analytics.com to be down this long which right now has been over 10 minutes, but it just shows that it is possible.
There are two aspects to it.
Analytics script's' (and a gif) download
Downloaded scripts execution
Download time is almost always less than 100ms, which is acceptable.
Here comes the twist.
analytics.js execution 250ms
re-marketing (if enabled) 300ms
demographic (if enabled) 200ms
So analytics with re-marketing takes 750ms on average. I feel that this is a huge number when it comes to performance overhead.
I noticed frequent I/o and CPU overload in cPanel resulting with:
Site unreachable error
And that stopped after I disabled WP Analytics plugin. So I reckon it does have some impact.