There seems to be a lot of stress/load testing tool that support AJAX.
I am wondering how well does these tool implemented.
Do they only record http request and replay it?
Is it the right way to test AJAX app?
How does google test their ajax apps?
Most of the load testing tools out there do AJAX load testing the same way: they execute the raw HTTP traffic that is seen during a "recording" phase (which can be page requests, image requests, or even AJAX requests). The main difference among them is how good their recorder/IDE tool is and how easily it helps you parameterize the HTTP requests such that they reflect real world traffic based on dynamic/realtime results.
Warning, blatant plug: The only real exception to this is my company, BrowserMob. Instead of simulating the traffic observed, it actually uses real web browsers to drive back load. As such, the AJAX stuff is handled by the browser.
Useful link: Separate from the blatant plug above (though I do hope you check it out - we're up front with the pricing and provide a free trial), I recently wrote an article for Ajaxian about AJAX load testing. It goes in to more detail about the technical implications of using real browser users (RBUs) vs. virtual users (VUs).
Take a look at LoadBooster(https://www.loadbooster.com). It utilizes headless scriptable browser PhantomJS/CasperJs to test web sites. Phantomjs will parse and render every page, execute the client-side script. The headless browser approach is easier to write test scenarios to support complex AJAX heavy Web 2.0 app，browser navigation, mouse click and keystrokes into the browser or wait until an element exists in DOM. LoadBooster support selenium HTML script too.
Disclaimer: I work for LoadBooster.
If you are worried about functionality only, something like Watin, Watir, Selenium or any functional tool for that matter would work. As long as you put sufficient timing in your functional tests to allow for ajax callbacks then that should do the trick.
To add to my response,
If you are talking about unit testing your javascript you could use something like qunit as described by a fellow LosTechian in this posting. This is a pretty inventive use of an nunit addin and qunit for integrating js unit tests.
By pure coincidence, the same person that posted that qunit testing blog post just posted one yesterday about this very topic
Google apps are written in GWT, which comes with its own extension to JUnit. Article on unit testing ajax applications with GWT.
If you don't want to setup your own load testing server there are a couple of free online load testing services that can run load tests directly over the Internet. For example http://loadimpact.com or http://loadstorm.com
I've used Virtual User Generator, which is a part of the Loadrunner software from HP, to test AJAX applications. The software has several application protocols that can be used to record web applications, e.g. AJAX, and Click and Script.
For the majority of web application load testing it is sufficient to record and replay http requests. This will give the result of how the servers are handling the load. If your web application does a lot of asynchronous loading, and rendering on the client side, e.g. parsing large datasets of xml or json, or many DOM modifications, it can be relevant to include the browser tier to measure the end user experience.
All load testers would support AJAX- they're just additional http connections.
There are a few free ones out there- Jmeter, BadBoy, Grinder that all do it well.
All of them have some sort of support for recording/playback, but that's not always what you're looking for.
Easiest way I've done it is to record a sample session, replace a few params with variables and loop it off of a csv or excel file.
Great starting point: video of a google presentation on open source testing.
Edit: updated video link.
I've used SilkPerformer at a previous job. According to the link, they have some AJAX enhancements. Unfortunately, Silk is far from free.
Check out Jiffy. It's an end-to-end measurement suite, and is subsequently kind of complex. However, the statistics are quite impressive.
Siege? it can do HTTP testing and pass whatever you want.
You can also have a look at fwptt it is open source. If you are a .net developer you can make use of the parameter automatic handling and the possibility to use your own .net object for doing the tests.
I've successfully used JMeter to load test our Ajax (JSF/RichFaces) application. I didn't bother with JMeter's recording tool - rather I used the HttpFox plugin for Firefox to monitor what the browser is POST-ing to the server and I recreated this in JMeter.
It did get a bit complex, but the load test is now fairly robust. JMeter has all kinds of useful 'elements' to extract ID's from a web page, perform conditional logic, increment counters etc.
Better write isolated test method or API for load testing ajax application. There are some reasons:
It's not so easy to write functional tests to Ajax applications, for example for GWT.
You can use Jmeter WebDriver plugin, but for each run it starts browser which will use most of RAM and CPU.
You will load backend not frontend, so you can avoid ajax.
You can devide your testing like that: for Ajax application use Selenium or PhantomJS/CasperJS. For load testing use JMeter, Gatling via API not via Ajax.
My choice is firebug(browser addon). its very lightweight and easy to handle
Related
I have to do a performance test for application(It has UI) ,It has API requests ,how to find the browser rendering time for APIs , can we do it with retrieve embedded resource. I am quite new to this ,Can someone please guide me.
Not really, as per JMeter project main page:
JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).
The only way to measure browser rendering time is using a real browser, if you want to do this along with the existing JMeter performance tests - go for WebDriver Sampler, it provides JMeter integration with Selenium browser automation framework so you will be able to kick off a real browser and measure its rendering time (response time of the WebDriver Sampler will be the time from opening the page till the rendering finishes.
If you need the breakdown - consider using Navigation Timing API
While checking performance of public website why we exclude java scripts, style sheets and include only basic request which returns HTML file(host "www.test.com" and "/" in path.
You need to exclude these scripts, styles and images only during recording of your test scenario with the HTTP(S) Test Script Recorder as if you record these calls they will be captured, but when you replay them it will have nothing in common with that happens in the reality.
Real browsers act as follows:
They download these scripts, images and styles when requesting the main page, but do it in parallel using several concurrent threads to speed up the process. So the main request forks into several sub-requests which are downloading all this stuff. In JMeter you can mimic this behaviour using "Retrieve All Embedded Resources" switch either in HTTP Request or in HTTP Request Defaults
As you already know, browsers download scripts, images and styles and do it using several concurrent threads. The next point is that well-behaved browsers do it only once, on subsequent requests this stuff is being returned from the browser's cache. So you need to add a HTTP Cache Manager to your Test Plan to simulate this functionality representing browser cache and not overload the server
Regarding 3-rd party content. If your web site uses scripts, images, videos, etc. from other domains - it is recommended to exclude these requests from your load testing scope as most likely you don't have any control of these 3-rd party content providers and won't be able to do anything even if the bottleneck is due to external server, your load test should focus solely on your application.
See How To Make JMeter Behave More Like A Real Browser guide for more information on above hints and few more recommendations.
Because JMeter is only retrieving page elements, and does not execute javascript or render these components like real browsers do. Personally i like to retrieve all embedded resources to give myself a better understanding of the total load time, even if javascript is not executed/rendered.
You can also do a selective retrieval, or a full retrieval if you wish.
Use the checkbox named "retrieve embedded resources", or specify which elements should be retrieved. Getting full retrieval is also handy to see whether there are broken links to components in the page.
I know this is an open ended question, but hopefully it will get some good answers before the thread is locked...
I'm wondering what methods there are to programmatically check (language agnostic) if a website is online from a client perspective (assume you can't make changes to the site/server, but you can rely on certain behaviours of the site.)
The result of each method could stack to provide a measure of certainty that the site is up/down - that is, a method does not have to provide a definite indication if the site is up/down on its own.
Some common tests just to check 'upness' may be:
Ping the site (which in the case of shared hosting isn't very
indicative)
Send a http head/get request and check the status
Others I can think of to check that the site is up and functioning:
Check you received a well formed html response i.e. html to html
tags, if the site is experiencing trouble it may spit an error and
exit without writing the rest of the page (not all that reliable
though because the site may handle most errors in a better way)
Check certain content is or is not on the page, i.e. perhaps there is some content that is always present on your pages, or always present in the case of an error
Can anybody think of any other methods that could be used to help determine if a site is in fact up/down and functioning/not functioning correctly from within a program?
If your get request on a page that displays info from database comes back with status 200 and matching keywords are found, you can be pretty certain that your site is up and running.
And you don't really need to write your own script to do that. There are free services such as GotSiteMonitor, Pingdom, UptimeRobot etc. allows you to monitor your site.
Based your set of test on the unit tests priciple. It is normally used in programming to test classes, modules or other artefacts after changes have been made. You can use any of the available frameworks, so don't have to reinvent the wheel. You must describe (implement) tests to be run, in your case a typical test should request a url inside the page and then do some evaluations like:
call result (for example return code of curl execution)
http return code
http headers
response mime type
response size
response content (test against a regular expression)
This way you can add, remove and modify single tests without having to care about the framework, once you are up. You can also chain tests, so perform a login in one test and virtually click a button in subsequent test.
There are also tools to handle such test runs automatically including visualization of results, statistics and the like.
OK, it sounds like you want to test and monitor your website from a customer experience perspective rather than purely establishing if a server is up (using ping for example). An effective way to replicate the customer experience is to simulate tests against the site using one of the headless browser testing tools (phantomJS is great a great choice) as they will render the page fully (including images, CSS, JS etc.) giving you a real page load time. These tools also allow you to make assertions on all aspects of the HTML content and HTTP response.
pingdom recently started offering a (paid for) service to perform these exact types of checks for alongside their existing monitoring solution. The demo is worth looking at, their interface for writing the actual tests is very nice.
is there any way I can scrape web pages that uses AJAX?
by using something like ruby + mechanize on linux server that doesn't have monitor attached (linode.com for example)
http://watir.com/ would be a solution but I guess not applicable to linode.
Check out TestPlan. It can do testing without a monitor -- by using the HTMLUnit backend. It handles quite a lot of JavaScript, including AJAX. I use it to scrape several pages and have built several tests of AJAX with it.
You can also run TestPlan with a browser if you want. This gives you the best of both worlds: develop tests and visually see what is happening, and then switch to the display-less mode.
When I make HTTP Request in JMeter I get Response data like "This page uses JavaScript and requires a JavaScript enabled browser." How is it possible to fix this problem.
JMeter is not a browser, and does not interpret the JavaScript in downloaded pages.
From the JMeter wiki:
JMeter does not process Javascript or applets embedded in HTML pages.
JMeter can download the relevant resources (some embedded resources
are downloaded automatically if the correct options are set), but it
does not process the HTML and execute any Javascript functions.
If the page uses Javascript to build up a URL or submit a form, you
can use the Proxy Recording facility to create the necessary sampler.
If this is not possible, then manual inspection of the code may be
needed to determine what the Javascript is doing.
Depending on what you are doing, you could create an execution test using Selenium IDE for Firefox. The test will run in your browser so the JavaScript will also run. Note though that I never used Selenium as a substitute for JMeter and don't know about common features to both the tools.
I suppose you can use the WebDriver plugin to run real browser tests (IE/Firefox/Chrome/Selenium).
There is good documentation here
You can add WebDriver to JMeter test to fully evaluate the page rendering.
Web Driver Sampler automates the execution and collection of
Performance metrics on the Browser (client-side). A large part of
performance testing, up to this point, has been on the server side of
things. However, with the advancement of technology, HTML5, JS and CSS
improvements, more and more logic and behaviour have been pushed down
to the client. This adds to the overall perceived performance of
website/webapp, but this metric is not available in JMeter. Things
that add to the overall browser execution time may include:
Client-side Javascript execution - eg. AJAX, JS templates
CSS transforms - eg. 3D matrix transforms, animations
3rd party plugins - eg. Facebook like, Double click ads, site analytics, etc
All these things add to the overall browser execution time, and this
project aims to measure the time it takes to complete rendering all
this content.
Official guide: https://jmeter-plugins.org/wiki/WebDriverTutorial/
You need to add HTTP Cookie/Cache Manager to your thread in order to solve this.