I have developped a HTTP web service which is queried by smartphone. I want to test the performances of all this service containing :
A java server (java 6, java + play framework)
A database (Mysql 5.1.41)
A Linux (ubuntu) server (kernell 2.6.32)
I have tried leading test campaigns using python scripts with many threads or sequential tests. But it's hard to have conclusions...
I want to be able to have the maximum number of request per second for my service, the average time for each request... complete dashboards displaying a lot of information
I can do many scripts to test that but I am shure that well-known softwares permits to conduct these tests. Ideally these softwares could also display information about where I loose time ...
Do you have hints ?
Thanks for your help
Some tools I've used for HTTP benchmarking
Apache Bench
Siege
JMeter
Of these, JMeter is probably best for the situation you describe. All of these display a lot of information, but won't explain where you lose time.
For that, I'd suggest a profiler such as JVisualVM (comes with the JDK) or YourKit. From a profile you can observe where you spend the most time and focus on optimizing that.
Related
I need to undergo performance testing for my project and I have learned how to Handle the Jmeter for Performance testing through online, but still, i was unable to find the solution how to analyze the result? from the report.I do know how to analyze the result so that I can't able to find the Performance Issue I n m application, where the error had been occurring, so from that how I can improve that performance.Is there is any article or video tutorial to learn how to analyze the result?
There are 2 possible test outcomes:
Positive: the performance of your application matches SLA and/or NFR. In this case you need to provide the report as a proof, it might be i.e. HTML Reporting Dashboard
Negative: the performance of your application is beyond the expectations. In this case you need to perform some investigation on the reasons, which could be in:
Simply lack of resources, i.e. your application doesn't have enough headroom to operate due to not enough CPU, RAM, Network or Disk bandwidth. So make sure you are monitoring these resources on the application under test side, you can do it using i.e. JMeter PerfMon Plugin.
The same but on JMeter load generator(s) side. If JMeter cannot send requests fast enough the application won't be able to serve more requests so if JMeter machine doesn't have enough resources - the situation will be the same so make sure to monitor the same metrics on JMeter host(s) as well
You have some software configuration problem (i.e. application server thread pool, db connection pool, memory settings, caching settings, etc. are not optimal). In the majority of cases web, application and database servers default configuration is not suitable for high loads and it needs to be tuned so try playing with different settings and see the impact.
Your application code is not optimal. In case when there is a plenty of free hardware resources and you are sure that infrastructure is properly set up (i.e. other applications behave fine) it might be a problem with your application under test code. In this case you will need to re-run your test with profiler tool telemetry to see what are the most time and resources consuming methods and how they can be optimised.
It might also be a networking related problem, i.e. faulty router or bad cable or whatever.
There are too many possible reasons however the approach should be the same: the whole system acts at the speed of its slowest component so you need to identify this component and determine why it is slow. See Understanding Your Reports posts series to learn how to read JMeter load test results and identify the bottlenecks from them.
I have to test my rest API such that 100k API calls are made simultaneously(within 500ms).Any Idea how to simulate it?Utility to use?
I would suggest to use JMeter too, it enable your test to create multiple concurrent jmeter server. Just to be clear you can control multiple remote JMeter engines from a single JMeter client and replicate a test across many computers and thus simulate a larger load on the target server.
To be honest, your target is quite high (100k API calls simultaneously within 500ms), i.e. you'll need a lot of jmeter servers. When you create stress tests, there are not magical recipes, guides or manuals. Trial and error is a fundamental method of solving this kind of problems.
In my experience, I first try with few concurrent users and see how the server react. Then increase the number of concurrent users till to reach an intolerable performance decrease or, worst, a bottleneck .
http://jmeter.apache.org/usermanual/remote-test.html
You will obviously need a load testing tool which can be run in a distributed mode, i.e. 1 controller and X load generators executing the same test.
Grinder - scripts are written in some Python dialect
Apache JMeter - this guy doesn't require any specific knowledge, you can create tests using simple GUI
Tsung - is written in Erlang, known for capability to produce high loads even on low-end hardware.
See Open Source Load Testing Tools: Which One Should You Use? article for more information on above tools.
JMeter
The Apache JMeter™ application is open source software, a 100% pure
Java application designed to load test functional behavior and measure
performance. It was originally designed for testing Web Applications
but has since expanded to other test functions.
My company has over a hundred users of a specific CRM web application, which is provided as a service by another company to us.
The users of this application are very dissatisfied with its average response time, and I need to find a way to gather metrics during a certain period of time (let's say .. a week) to prove the service provider that they are really providing a bad service.
If the application were mine, I would get the metrics from New Relic or some other equivalent monitoring service, but since it is not, I'm looking for something that could do some sort of client side monitoring.
I already checked Page Speed from Google and YSlow from Yahoo, but both are only useful when you want to test the application during a few seconds. They are not meant for the long term monitoring I need.
Would anybody know a way to get this kind of monitoring from a client side perspective?
LoadRunner is no charge for 50 users, but what you really need is not a test tool but a synthetic user monitor which runs every n number of minutes and pulls the stats. You can build it yourself using LoadRunner 12, Jmeter, or any other http sampling technology. You could also use a service like Gomez for sampling or mpulse from SOASTA for tracking every page component across all users.
Keep in mind that your developer tools will time all of the components of the request to give you some page times. As will Dynatrace for the web client.
If you have access to the web server then consider configuring the web server logs to capture the w3c time-taken field, which will track every request. Depending upon the server the level of granularity can be to the millionth of a second on each and every request.
You could also look at a service like LiteSquare which can process those web logs and provide ammunition for changes to the server to improve performance on a no-gain, no-charge model.
One (expensive) solution would be using LoadRunner endurance test feature. Check here for a demonstration.
Another tool is Oracle OATS.
JMeter is a free tool, though I'm not sure if it's reliable enough to run for a whole week.
These are load generator tools, so if you are testing as a single client, you should carefully chose your load amount (e.g. one user).
Last but not least, you could create your own webservice client, and create a cron job to run it on your specified time of day and log the access time.
If what you want is to get data from their server, this is impossible ... without hacking into it. All you can do is monitor the website as a client, using some of the above tools, make a report and present that to them. But even so they could challenge your bandwidth, your test method etc.
I recommend that you negotiate with them to give you their logs and to prove that their system can support a certain amount of load. If you are a customer to them, you can file a complain or test additional offers.
Dynatrace was already mentioned in combination with Load Testing. As you said that you want to monitor your live system I want to bring Dynatrace up again. Most of the time it is used to do live system monitoring to understand what end users are actually doing. It is also available as a 30 day trial - so - no need to buy it - but - use it for your sanity check: http://bit.ly/dttrial
I've got a PHP site up and running, and the db is mysql. before launching the site, I would like to test the traffic handling. Now am assuming that there are soe softwares that would simulate the traffic and log the processes running on my site. Any recommendation of software I should use? the traffic doesn't have to be real, but nonetheless, I would like to generate a high traffic to investigate the threshold of the site.
Appreciate the help
You can use Gatling https://github.com/excilys/gatling.
It's a stress tool written in Scala which aims at being more efficient and lighter than Jmeter.
Basically you record a scenario on your website and then run it 'n' times in parallel.
Here is the wiki for more infos https://github.com/excilys/gatling/wiki/Basic-Usage
You can use Jmeter:
It's free.
it's easy to Start with lot of documentation on its Website and on internet
it has a proxy feature to easily create test plan from browser navigation
It is easy to start up processes on other machines. It remote testing, can be done from GUI or console.
The scripts can be written in beanshell, java, or any jsr223 language ( groovy, Javascript, scala, jexl ...)
it has a lot of built- in samplers and thanks to its plugin architecture it's very Easy to add new ones or use any scripting engine to do what's missing
it has great user mailing list
it has very reactive support
it's now a top Level Apache
it can run thousands of users
professional solutions exist to run it from cloud
...
See:
Performing a Stress Test on Web Application?
Best way to stress test a website
How do you test the performance of a website?
what is the best tool (open or commercial) currently available, that lets me send customized requests to a web server and get back a response to check the performance?
i will be sending it a load of more than 20K per second, but i need to get numbers for each call made. also, the numbers might be in some microseconds or nanoseconds. How in this small measurement unit, can i work out a baseline and a benchmark?
If you're using Apache, Apache AB is a benchmarking to test how many requests your serve can serve per second and how well it handles load and concurrency. It's an open-source project - check it out here.
In addition, wikipedia has a nice list of benchmarking software for testing servers.
You can use the Web Application Stress Tool of Microsoft
The Microsoft WAS web stress tool is designed to realistically simulate multiple browsers requesting pages from a web site. You can use this tool to gather performance and stability information about your web application. This tool simulates a large number of requests with a relatively small number of client machines. The goal is to create an environment that is as close to production as possible so that you can find and eliminate problems in the web application prior to deployment.
You can find a list of Open Source software for performance (most of them are for web that send custom request to webserver).
Don't know if either of these have granularity better than milliseconds but check out JMeter (open source) and LoadRunner (Commercial). LoadRunner is not cheap but it allows you to span load generation across multiple machines with aggregated results.