I'm told that we need to do some performance testing on one of our web-applications, so I'm trying to get some JMeter stuff to work, which as far as I know would simulate the HTTP GETS and POSTS. However, one of my colleagues is telling me that if I use it, it'd only accomplish FE testing. But, if I do this, it still is able to create items in the database and interact with the logic, so I figured it should be sufficient for performance testing of the back-end. Her reasoning is that "if it goes through http pages, we can’t tell which affects the performance".
So am I totally wrong? I'm confused.
The whole idea of the load testing thing is to simulate real-life users actions and behavior as close to reality as possible.
In JMeter terms that assumes presence and appropriate configuration of the following test elements:
HTTP Cookie Manager - to represent browser cookies and deal with cookie-based authentication
HTTP Header Manager - to represent browser headers like User-Agent, Accept-Encoding, Accept-Language, Content-Type, etc.
HTTP Cache Manager - browsers download embedded resources like scripts, styles, images, etc. but do it only once, on subsequent requests aforementioned entities are being returned from cache. To simulate this behavior you need to have HTTP Cache Manager
HTTP Request Samplers need to be configured to fetch embedded resources from the web pages and use a separate thread pool for this. See How to make JMeter behave more like a real browser guide for more details on how to configure realistic behavior.
So given JMeter test is good designed and implemented it is quite enough to test backend as well. If during load test you figure out that bottleneck is i.e. database, you may need to load-test the database separately, JMeter is capable of doing this as well, however I'm a strong believer that load testing should be done against environment as close to production as possible and should target the whole system rather than individual components.
Related
Most of the tools out there do not seem to be interpreting jQuery / Javascript code that is loaded on the page that I want to test. This is however important because that code will, in turn initiate a number of calls to other endpoints in my webapp which adds up to load in a real life scenario.
I've looked at JMeter already and am a bit reluctant to give it a try as the landing page mentions explicitly that it does not do Javascript interpreting at all.
What would be some recommendation of tools that can provide a more accurate measurement of load testing by including Javascript code?
None of the load testing tools really executes JavaScript, they all act on protocol level and JavaScript is being executed by browser.
There are 3 options how you could proceed if you want 100% realistic testing:
HP LoadRunner has TrueClient protocol which is basically headless browser with JavaScript capabilities.
JMeter with WebDriver Sampler plugin - the way to kick off real browsers from JMeter test
Selenium Grid (or other way to kick off several Selenium instances at once)
All 3 above options are very resource intensive, you will required at least 1 CPU core and a couple of gigabytes or RAM for a single browser instance so I would recommend a little bit different approach. JMeter cannot execute JavaScript, but it can execute JavaScript-driven HTTP Requests so create main load using JMeter (or equivalent) and while the load test is running use Selenium to automatically check real browsing experience or YSLow to do the same manually.
Load testing tools doesn't execute JavaScript.
You know which endpoints your JavaScript code is using so just add these endpoints to your JMeter scenario.
The thick client problem has been around since the early 1990s with traditional thick client-server applications. In fact, the earliest commercial tools were all driving full clients on X Windows before adding API level virtual user capabilities. On the commercial front this ability to drive full, thick clients is still expressed as GUI virtual users (Mercury/HP/Microfocus LoadRunner, Segue/Borland/Microfocus Silk Performer, Rational/IBM Performance Tester) allowing you to measure the weight between API and user level if needed.
Here is the thing, for a given business process and data set the end client is predictable in its behavior, and in the requests that it makes to the servers that are feeding it data. There may be a few odd conditions where you need to reproduce an algorithm from JavaScript to C (or even run it directly in some tools) for branching on a type of API/HTTP request, but these are not numerous.
I don't know if this question makes much sense but i would like to know if load testing can be done on different IE versions. For instance my product supports IE 8, 9 , 10.... Please clarify more on this.
Thanks.
When we do load testing, we are more interested in knowing the server performance. Not the client's browser performance.
JMeter itself acts like a browser. It sends the Http request & once it gets the response, it can display it like a browser(it does not execute javascript files though. It makes sense because javascript is going to be executed in client's machine.)
So, for your question, JMeter is not a tool for your requirement. You may have a look at the httpwatch which shows the page load time for the browser.
This only makes sense if
You build pages dynamically based upon browser type and version
If your load for your exceptional condition on page generation, which is typically early versions of IE, is significant
Simulating different browser versions is not always a requirement is performance testing, but could be needed in specific case. Do it if it really makes sense to do.
There are commercial tools which can easily simulate the different browser types/versions like LoadRunner, Microsoft Visual Studio etc. If you are going to use open source, trying mimicing the scenarios by providing browser details/properties in user agents header etc., just like the actual browser sends. You can capture this by checking the request headers from the browser (press F12 on chrome and see network)
The only thing you can do with JMeter is adding a HTTP Header Manager to send different User-Agent header value for each Internet Explorer versions. You can refer Understanding user-agent strings guide to see user agent values for different IE versions, however I don't think it will provide the full picture as JMeter doesn't actually "render" page.
So I would go the following way:
Use JMeter to put your server under expected load
Use one Selenium instance per Internet Explorer version to perform real browser testing of the system when it's under JMeter's load to measure performance differences.
I have a site that is being polled rather hard for the JSON representation of the same resources from multiple clients (browsers, other applications, unix shell scripts, python scripts, etc).
I would like to add some caching such that some of the resources are cached within the server for a configurable amount of time, to avoid the CPU hit of handling the request and serialising the resource to JSON. I could of course cache them myself within the handlers, but would then take the serialisation hit on every request, and would have to modify loads of handlers too.
I have looked at the openrasta-caching module but think this is only for controlling the browser cache?
So any suggestions for how I can get openrasta to cache the rendered representation of the resource, after the codec has generated it?
Thanks
openrasta-caching does have preliminary support for server-side caching, with an API you can map to the asp.net server-side cache, using the ServerCaching attribute. That said it's not complete, neither is openrasta-caching for that matter. It's a 0.2 that would need a couple of days of work to make it to a good v1 that completely support all the scenarios I wanted to support that the asp.net caching infrastructure doesn't currently support (mainly, make the caching in OpenRasta work exactly like an http intermediary rather than an object and .net centric one as exists in asp.net land, including client control of the server cache for those times you want the client to be allowed to force the server to redo the query). As I currently have no client project working on caching it's difficult to justify any further work on that plugin, so I'm not coding anything for it right now. I do have 4 days free coming up though, so DM me if you want openrasta-caching to be bumped to 0.3 with whatever requirements you have in that that would fit 4 days of work.
You can implement something simpler using an IOperationInterceptor and plugging in the asp.net pipeline using that, or be more web-friendly and implement your caching out of process using squid and rely on openrasta-caching for generating the correct http caching instructions.
That said, for your problem, you may not even need server caching if the cost is in json. If you map a last modified or an Etag to what the handler returns, it'll generate correctly a 304 where appropriate, and bypass json rendering alltogether, provided your client does conditional requests (and it should).
There's also a proposed API to allow you to further optimize your API by doing a first query on last modified/etag to return the 304 without retrieving any data.
I´ve recently started using JMeter to create load tests for my web applications. I really like the tool, and after watching some videos it was really easy to get started with creating tests.
There´s however one thing that I´m not clear about.
Reading on the JMeter homepage, there´s a "Best practice" section. Among other things, it says:
The most important thing to do is filter out all requests you aren't interested in. For instance, there's no point in recording image requests (JMeter can be instructed to download all images on a page - see HTTP Request ). These will just clutter your test plan.
I´ve seen this on other pages aswell, saying that you shouldn´t include requests for images or any other static resources in your tests. However I´ve still not been able to find a single page which gives a good explanation as to WHY you shoudn´t include static resources.
Sure, JMeter isn´t a browser, but requests for static resources would no doubt affect performance of your application? Can someone please give me a good explanation :-)
It all depends on what you are trying to test.
In general, there are two types of performance test I do with JMeter: specific tests, where I look at things that I'm worried about, and "safety net" tests, where I measure the entire application to make sure it does indeed work the way I expect it to.
Specific tests nearly always deal with the dynamic aspects of the web application - the server-side code (.aspx, .php, .jsp etc.). This is where most applications have their bottlenecks - the effort to run a server-side script is many, many times higher than the effort to retrieve a CSS file from disk and serve it up to the browser without any additional processing. If I'm testing the server-side scripts, I don't want to also load the assets - because they clutter up the tests, and consume bandwidth at the test client end. I don't want my tests to fail because my JMeter server is downloading a 5MB video file on each thread and consuming all the bandwidth, when what I'm actually trying to do is see how many logins per second the server can support.
There's very little point in testing your webserver's ability to serve static files - Microsoft, the Apache team, whoever, have already done a brilliant job at that; unless you have a very specific concern, there are better ways to spend your testing budget.
Safety net tests put the whole thing together to prove that it all really does work the way I expect it. Usually, I run these on a production(like) environment, so I have a CDN, production-grade hardware, and the "live" application config. I usually employ a cloud-based testing service for this, so I can see performance from different locations, and generate enough load to stress production-grade kit. You could use JMeter for this (and there are a couple of JMeter Cloud services I've used in the past). It's expensive, it may require downtime, and you should only do it as a safety net.
When you want to do a proper performance test (especially a stress test), where you need to produce your application's response time as a function of number of users/threads in time, you need to include all static resources, just as jMeter Proxy saved them when you recorded your test.
To take browser cache into account you can either use HTTP Cache Manager or Once Only controller, so that each thread only downloads static stuff once, with their first request.
HTTP Cache Manager is the recommended way to go and much easier to set up, simply include it in your test, as the first child of a thread group.
Once Only controller is regularly used when you need to log-in users only on their first request.
BTW parametrization of non-static HTTP Requests is recommended, you won't e.g. search for the same product name or e.g. buy the same book every time, that's usually the starting point which can give you a general idea of performance efficiency of you app.
Hope this helps...
Unless your app is used by casual visitors who only look at one page and then go, there is a good chance that the static resources are being downloaded once, and then served from the browser cache.
Moreover, although static resources affect the bandwidth and the overall response time for the user, they should have a small impact on the server load, and they might not be the kind of thing that you want to measure.
I guess you need to try mimicking what actual, real users would do with the application.
I have been tasked with looking for a performance testing solution for one of our Java applications running on a Weblogic server. The requirement is to record production requests (both GET and POST including POST data) and then run these requests in a performance test environment with a copy of the production database.
The reasons for using production requests instead of a test script are:
It is a large application with no existing test scripts so it would be a a large amount of work to write scripts to cover the entire application.
Some performance issues only appear when users do a number of actions in a particular order.
To test using actual user interaction with the system not an estimation at how the users may interact with the system. We all know that users will do things we have not thought of.
I want to be able to fix performance issues and rerun the requests against the fixed code before releasing to production.
I have looked at using JMeters Access Log Sampler with server access logs however the access logs do not contain POST data and the access log sampler only looks at the request URL so it cannot simulate users submitting form data.
I have also looked at using the JMeter HTTP Proxy Server however this can record the actions of only one user and requires the user to configure their browser to use the proxy. This same limitation exist with Tsung and The Grinder.
I have looked at using Wireshark and TCReplay but recording at the packet level is excessive and will not give any useful reports at a request level.
Is there a better way to analyze production performance considering I need to be able to test fixes before releasing to production?
That is going to be a hard ask. I work with Visual Studio Test Edition to load test my applications and we are only able to "estimate" the users activity on the site.
It is possible to look at the logs and gather information on the likelyhood of certain paths through your app. You can then look at the production database to look at the likely values entered in any post requests. From that you will have to make load tests that approach the useage patterns of your production site.
With any current tools I don't think it is possible to record and playback actual user interation.
It is possible to alter your web app so that is records and logs every request and post against session and datetime. This custom logging could be then used to generate load test requests against a test website. This would be some serious code change to your existing site and would likely have performance impacts.
That said, I have worked with web apps that do this level of logging and the ability to analyse the exact series of page posts/requests that caused an error is quite valuable to a developer.
So in summary: It is possible, but I have not heard of any off the shelf tools that do it.
Please check out this Whitepaper by Impetus Technologies on this page.. http://www.impetus.com/plabs/sandstorm.html
Honestly, I'm not sure the task you're being asked to do is even possible, let alone a good idea. Depending on how complex the application's backend is, and how perfect you can recreate the state (ie: all the way down to external SOA services or the time/clock), it may not be possible to make those GET and POST requests reproduce the same behavior.
That said, performance testing against production data is always great, but it usually requires application-specific knowledge that will stress said data. Simply repeating HTTP GETs and POSTs will almost certainly not yield useful results.
Good luck!
I would suggest the following to get the production requests and simulate the accurate workload:
1) Use coremetrics: CoreMetrics provides such solutions using which you can know the application usage patterns. This would help in coming up with an accurate workload model. This model can then be converted into test scripts and executed against a masked copy of production database. This will provide you accurate results about the application performance in realtime.
2) Another option would be creating a small utility using AOP (Aspect oriented apporach) so that it can trace all the requests and corresponding method traces. This would help in identifying the production usage pattern and in turn accurate simulation of workload. AOP frameworks such as AspectJ can be used. This would not require any changes in code. The instrumentation can be done on the fly. The other benefit would be that thi cna only be enabled for a specific time window and then it can be turned off.
Regards,
batterywalam