I´ve recently started using JMeter to create load tests for my web applications. I really like the tool, and after watching some videos it was really easy to get started with creating tests.
There´s however one thing that I´m not clear about.
Reading on the JMeter homepage, there´s a "Best practice" section. Among other things, it says:
The most important thing to do is filter out all requests you aren't interested in. For instance, there's no point in recording image requests (JMeter can be instructed to download all images on a page - see HTTP Request ). These will just clutter your test plan.
I´ve seen this on other pages aswell, saying that you shouldn´t include requests for images or any other static resources in your tests. However I´ve still not been able to find a single page which gives a good explanation as to WHY you shoudn´t include static resources.
Sure, JMeter isn´t a browser, but requests for static resources would no doubt affect performance of your application? Can someone please give me a good explanation :-)
It all depends on what you are trying to test.
In general, there are two types of performance test I do with JMeter: specific tests, where I look at things that I'm worried about, and "safety net" tests, where I measure the entire application to make sure it does indeed work the way I expect it to.
Specific tests nearly always deal with the dynamic aspects of the web application - the server-side code (.aspx, .php, .jsp etc.). This is where most applications have their bottlenecks - the effort to run a server-side script is many, many times higher than the effort to retrieve a CSS file from disk and serve it up to the browser without any additional processing. If I'm testing the server-side scripts, I don't want to also load the assets - because they clutter up the tests, and consume bandwidth at the test client end. I don't want my tests to fail because my JMeter server is downloading a 5MB video file on each thread and consuming all the bandwidth, when what I'm actually trying to do is see how many logins per second the server can support.
There's very little point in testing your webserver's ability to serve static files - Microsoft, the Apache team, whoever, have already done a brilliant job at that; unless you have a very specific concern, there are better ways to spend your testing budget.
Safety net tests put the whole thing together to prove that it all really does work the way I expect it. Usually, I run these on a production(like) environment, so I have a CDN, production-grade hardware, and the "live" application config. I usually employ a cloud-based testing service for this, so I can see performance from different locations, and generate enough load to stress production-grade kit. You could use JMeter for this (and there are a couple of JMeter Cloud services I've used in the past). It's expensive, it may require downtime, and you should only do it as a safety net.
When you want to do a proper performance test (especially a stress test), where you need to produce your application's response time as a function of number of users/threads in time, you need to include all static resources, just as jMeter Proxy saved them when you recorded your test.
To take browser cache into account you can either use HTTP Cache Manager or Once Only controller, so that each thread only downloads static stuff once, with their first request.
HTTP Cache Manager is the recommended way to go and much easier to set up, simply include it in your test, as the first child of a thread group.
Once Only controller is regularly used when you need to log-in users only on their first request.
BTW parametrization of non-static HTTP Requests is recommended, you won't e.g. search for the same product name or e.g. buy the same book every time, that's usually the starting point which can give you a general idea of performance efficiency of you app.
Hope this helps...
Unless your app is used by casual visitors who only look at one page and then go, there is a good chance that the static resources are being downloaded once, and then served from the browser cache.
Moreover, although static resources affect the bandwidth and the overall response time for the user, they should have a small impact on the server load, and they might not be the kind of thing that you want to measure.
I guess you need to try mimicking what actual, real users would do with the application.
Related
I am not able to find out anywhere that how can we do performance test manually.
Please help me out for this query.
Thanks!
Maybe you are looking for JMeter or a similar tool.
What browser? Most of the current browsers support the W3C Navigation Timing spec and expose performance data directly on the DOM. You can access it from the console, from javascript on your pages or from browser extensions that display the information.
If you want more detail like a resource load waterfall then you can usually access that directly from the dev tools provided by the various browsers.
One thing you will want to be really careful of is to make sure you do your testing in a configuration that is similar to the users. If you are running a server locally and testing from a browser on the same machine or even the same network then your performance data will be pretty worthless (unless it's an intranet app).
you can perform manual testing (Performance testing) for any webpage by optimizing your css, Javascript and images ( size).
I think JMeter is a best tool for same to check webpage testing if you want add some scripting you can also add.
Also you can check Yslow addons of firefox.This addons give you filter data to optimized your page perfromes.
Also there are some online link available.
How can we run performance testing manually for any webpage?
You can simple use GTMatrix tool which will response of your site Performaces overall in detail.
The best way to go for Performance Testing without any tool is to provide a Standard loading time for each page as per one's experience knowledge. Else request the client to provide an ideal time for each page. Against which the loading time can be verified. But in case of multiple user simultaneously JMeter is the best hands on Approach available. Its Open source. Easy to understand. And you get reports too.
But of course there are multiple factors that would hinder the Performance. They are :
Your network speed
The Server speed on which your application is hosted
The number of Simultaneous users using
The Heavy images in pages
Last but not the least unnecessary links, codes, in short memory consumption in Code, could be loops not required. All the gifts from Developer Teams !!
Is there an advantage of some sort (speed or performance wise) to embed your CSS and JS into your web page, as opposed to keeping the code in sparate files? I was raised to believe that keeping code separate in separate files makes things easier to maintain. However, on high profile websites like amazon or google even facebook, I see a lot of embed code. Is there a performance reason they choose to do so or is it just an old/new way of doing things. I suppose my question is similar to this one: Should I inline CSS & JS in mobile sites to save bandwidth?
But I would like to hear form experts, most notably from people who worked on high profile web sties and have done so, if any.
P.S.
Bonus Question: Last html comment on amazon web pages is <!-- MEOW --> does it mean anything or is it just a funny prank?
There are good reasons to inline resources, but as with most things, it also has its tradeoffs. The simplest case for inlining is cases where the cost of an HTTP connection is much more than the resource itself, ex: if you have a 10x10 icon you need to show, a dedicated request for that may not be worth it vs. inlining the data via a data URI.
This is especially true when and if you have many small resources that need to be fetchd. Most browsers limit themselves to a max of 6 connections per host, so if you have 60 resources which need to be fetched, then you'll be blocked for a significant chunk of time.
To work around these case we've invented other workarounds: domain sharding to go over the 6 connection limit, and "spriting" to fetch one resource vs multiple.
If you take a look at mod_pagespeed (Apache module), which does many of these optimizations on the fly for you, then the recommended setting we provide is to inline any resource that's below 2kb. That's a pretty good rule of thumb for today's stack.
Once SPDY is more widely deployed, many of these workarounds can be eliminated: no need to do domain sharding, cost of extra requests is much less, etc.
Stoyan did an experiment that you might find interesting http://www.phpied.com/style-tag-to-inline-style-attrrib/
CSS/JS external files typically get cached on the user's hard drive under that users browser's profile. So unless you change the code frequently, you won't really be doing yourself a favor by putting it inline.
Definitely saves you time from maintenance, but you can easily call in a javascript/css file and embed the code on the page you're populating on the server side, but that also means you're making your server do additional work.
As for the MEOW - yeah, them trying to be funny, or it's code... for... cat...
I want to test the performance of my website. I have hosted it on godaddy and I want to see how it performance when 100s of users are trying to access it.
Is their a way to do the above? Is their a script that can be developed to send multiple page request?
Thanks
Consider trying Jmeter or siege.
Apache Bench is commonly used for doing load testing (which is pretty much what you are describing). There are also a bunch of services that will do it for you (some free, most with varying costs).
You could simply script curl or whet to beat on it in parallel but just throwing load at it isn't terribly useful if you don't also track how the site performs under the load (which is where the other tools come in).
One thing to watch out for is if you test just the base page/application or if you use a real browser engine to test the full page (including images and static resources).
I have been tasked with looking for a performance testing solution for one of our Java applications running on a Weblogic server. The requirement is to record production requests (both GET and POST including POST data) and then run these requests in a performance test environment with a copy of the production database.
The reasons for using production requests instead of a test script are:
It is a large application with no existing test scripts so it would be a a large amount of work to write scripts to cover the entire application.
Some performance issues only appear when users do a number of actions in a particular order.
To test using actual user interaction with the system not an estimation at how the users may interact with the system. We all know that users will do things we have not thought of.
I want to be able to fix performance issues and rerun the requests against the fixed code before releasing to production.
I have looked at using JMeters Access Log Sampler with server access logs however the access logs do not contain POST data and the access log sampler only looks at the request URL so it cannot simulate users submitting form data.
I have also looked at using the JMeter HTTP Proxy Server however this can record the actions of only one user and requires the user to configure their browser to use the proxy. This same limitation exist with Tsung and The Grinder.
I have looked at using Wireshark and TCReplay but recording at the packet level is excessive and will not give any useful reports at a request level.
Is there a better way to analyze production performance considering I need to be able to test fixes before releasing to production?
That is going to be a hard ask. I work with Visual Studio Test Edition to load test my applications and we are only able to "estimate" the users activity on the site.
It is possible to look at the logs and gather information on the likelyhood of certain paths through your app. You can then look at the production database to look at the likely values entered in any post requests. From that you will have to make load tests that approach the useage patterns of your production site.
With any current tools I don't think it is possible to record and playback actual user interation.
It is possible to alter your web app so that is records and logs every request and post against session and datetime. This custom logging could be then used to generate load test requests against a test website. This would be some serious code change to your existing site and would likely have performance impacts.
That said, I have worked with web apps that do this level of logging and the ability to analyse the exact series of page posts/requests that caused an error is quite valuable to a developer.
So in summary: It is possible, but I have not heard of any off the shelf tools that do it.
Please check out this Whitepaper by Impetus Technologies on this page.. http://www.impetus.com/plabs/sandstorm.html
Honestly, I'm not sure the task you're being asked to do is even possible, let alone a good idea. Depending on how complex the application's backend is, and how perfect you can recreate the state (ie: all the way down to external SOA services or the time/clock), it may not be possible to make those GET and POST requests reproduce the same behavior.
That said, performance testing against production data is always great, but it usually requires application-specific knowledge that will stress said data. Simply repeating HTTP GETs and POSTs will almost certainly not yield useful results.
Good luck!
I would suggest the following to get the production requests and simulate the accurate workload:
1) Use coremetrics: CoreMetrics provides such solutions using which you can know the application usage patterns. This would help in coming up with an accurate workload model. This model can then be converted into test scripts and executed against a masked copy of production database. This will provide you accurate results about the application performance in realtime.
2) Another option would be creating a small utility using AOP (Aspect oriented apporach) so that it can trace all the requests and corresponding method traces. This would help in identifying the production usage pattern and in turn accurate simulation of workload. AOP frameworks such as AspectJ can be used. This would not require any changes in code. The instrumentation can be done on the fly. The other benefit would be that thi cna only be enabled for a specific time window and then it can be turned off.
Regards,
batterywalam
What can be the various performance testing scenarios to be considered for a website with huge traffic? Is there any way to identify the elements of the code which are adversely affecting the site performance?
Please provide something similar to checklist of generalised scenarios to be tested to ensure proper performance testing.
It would be good to start with some load testing tools like JMeter or PushToTest and start running it against your web application. JMeter simulates HTTP traffic and loads the server that way. You can do that as well as load test AJAX parts of your application with PushToTest because it can use Selenium Scripts.
If you don't have the resources (computers to run load tests) you can always use a service like BrowserMob to run the scripts against a web accessible server.
It sounds like you need more of a test plan than a suggestion of tools to use. In performance testing, it is best to look at the users of the application -
How many will use the application on a light day? How many will use the app on a heavy day?
What type of users make up your user population?
What transactions will each of these user types perform?
Using this information, you can identify the major transactions and come up with different user levels (e.g. 10, 25, 50, 100) and percentages of user types (30% user A, 50% user B, ...) to test these transactions with. Time each of these transactions for each test you execute and examine how the transaction times change as compared to your user levels.
After gathering some metrics, since you should be able to narrow transactions to individual pieces of code, you will be able to know where to focus your code improvements. If you still need to narrow things down further, finer tests within each transaction can be created to provide more granular results.
Concurrency will kill you here, as you need to test your maximum projected concurrent users + wiggling room hitting the database, website, and any other web service simultaneously. It really depends on the technologies you're using, but if you have a large interaction of different web technologies, you may want to check out Neoload. I've had nothing but success with this web stress tool, and the support is top notch if you need to emulate specific, complicated behavior (such as mocking AMF traffic, or using responses from web pages to dictate request behavior.)
If you have a DB layer then this should be the initial focus of your attention, once the system is stable (i.e. no memory leaks or other resource issues). If the DB is not the bottle neck (or not relevant) then you need to correlate CPU/Memory/Disk IO and Network traffic with the increasing load and increasing response times. This gives you an idea of capacity and correlation (but not cause) to resource usage.
To find the cause of a given issue with resources you need to establish a Six Sigma style project where you define the problem and perform root case analysis in order to pin point the piece of code (or resource configuration) that is the bottleneck. Once you have done this a couple of times in your environment, you will notice patterns of workload, resource usage and counter measures (solutions) that will guide you in your future performance testing 'projects'.
To choose correct performance scenarios you need to go through the next basic checklist:
High priority scenarios from the business logic perspective. For example: login/order transactions, etc.
Mostly used scenarios by end users. Here you may need information from monitoring tools like NewRelic, etc.
Search / filtering functionality (if applicable) - Scenarios which involve different user roles/permissions
Performance test is a comparison test either with the previous release of the same application or with the existing players in the market.
Case 1- Existing application
1)Carry out the test for the same scenarios as covered before to get a clear picture on the response of the application before and after the upgrade.
2)If you need to dig deeper you can get back to the database team to understand which functionalities are getting more requests. Also ask them on the total number of requests on an average on any particular day so that you can take a call on what user load and time duration to be given for the test.
Case 2- New Application
1) Look for existing market players and design your test as per the critical functions of the rival product (for e.g. Gmail might support many functions what what is being used often is launch ->login ->compose mail -> inbox ->outbox).
2) Any time you can get back to your clients on what they suppose to be business critical scenarios or scenarios that will be used more often..