what is the best tool (open or commercial) currently available, that lets me send customized requests to a web server and get back a response to check the performance?
i will be sending it a load of more than 20K per second, but i need to get numbers for each call made. also, the numbers might be in some microseconds or nanoseconds. How in this small measurement unit, can i work out a baseline and a benchmark?
If you're using Apache, Apache AB is a benchmarking to test how many requests your serve can serve per second and how well it handles load and concurrency. It's an open-source project - check it out here.
In addition, wikipedia has a nice list of benchmarking software for testing servers.
You can use the Web Application Stress Tool of Microsoft
The Microsoft WAS web stress tool is designed to realistically simulate multiple browsers requesting pages from a web site. You can use this tool to gather performance and stability information about your web application. This tool simulates a large number of requests with a relatively small number of client machines. The goal is to create an environment that is as close to production as possible so that you can find and eliminate problems in the web application prior to deployment.
You can find a list of Open Source software for performance (most of them are for web that send custom request to webserver).
Don't know if either of these have granularity better than milliseconds but check out JMeter (open source) and LoadRunner (Commercial). LoadRunner is not cheap but it allows you to span load generation across multiple machines with aggregated results.
Related
Most of the tools out there do not seem to be interpreting jQuery / Javascript code that is loaded on the page that I want to test. This is however important because that code will, in turn initiate a number of calls to other endpoints in my webapp which adds up to load in a real life scenario.
I've looked at JMeter already and am a bit reluctant to give it a try as the landing page mentions explicitly that it does not do Javascript interpreting at all.
What would be some recommendation of tools that can provide a more accurate measurement of load testing by including Javascript code?
None of the load testing tools really executes JavaScript, they all act on protocol level and JavaScript is being executed by browser.
There are 3 options how you could proceed if you want 100% realistic testing:
HP LoadRunner has TrueClient protocol which is basically headless browser with JavaScript capabilities.
JMeter with WebDriver Sampler plugin - the way to kick off real browsers from JMeter test
Selenium Grid (or other way to kick off several Selenium instances at once)
All 3 above options are very resource intensive, you will required at least 1 CPU core and a couple of gigabytes or RAM for a single browser instance so I would recommend a little bit different approach. JMeter cannot execute JavaScript, but it can execute JavaScript-driven HTTP Requests so create main load using JMeter (or equivalent) and while the load test is running use Selenium to automatically check real browsing experience or YSLow to do the same manually.
Load testing tools doesn't execute JavaScript.
You know which endpoints your JavaScript code is using so just add these endpoints to your JMeter scenario.
The thick client problem has been around since the early 1990s with traditional thick client-server applications. In fact, the earliest commercial tools were all driving full clients on X Windows before adding API level virtual user capabilities. On the commercial front this ability to drive full, thick clients is still expressed as GUI virtual users (Mercury/HP/Microfocus LoadRunner, Segue/Borland/Microfocus Silk Performer, Rational/IBM Performance Tester) allowing you to measure the weight between API and user level if needed.
Here is the thing, for a given business process and data set the end client is predictable in its behavior, and in the requests that it makes to the servers that are feeding it data. There may be a few odd conditions where you need to reproduce an algorithm from JavaScript to C (or even run it directly in some tools) for branching on a type of API/HTTP request, but these are not numerous.
I have to test my rest API such that 100k API calls are made simultaneously(within 500ms).Any Idea how to simulate it?Utility to use?
I would suggest to use JMeter too, it enable your test to create multiple concurrent jmeter server. Just to be clear you can control multiple remote JMeter engines from a single JMeter client and replicate a test across many computers and thus simulate a larger load on the target server.
To be honest, your target is quite high (100k API calls simultaneously within 500ms), i.e. you'll need a lot of jmeter servers. When you create stress tests, there are not magical recipes, guides or manuals. Trial and error is a fundamental method of solving this kind of problems.
In my experience, I first try with few concurrent users and see how the server react. Then increase the number of concurrent users till to reach an intolerable performance decrease or, worst, a bottleneck .
http://jmeter.apache.org/usermanual/remote-test.html
You will obviously need a load testing tool which can be run in a distributed mode, i.e. 1 controller and X load generators executing the same test.
Grinder - scripts are written in some Python dialect
Apache JMeter - this guy doesn't require any specific knowledge, you can create tests using simple GUI
Tsung - is written in Erlang, known for capability to produce high loads even on low-end hardware.
See Open Source Load Testing Tools: Which One Should You Use? article for more information on above tools.
JMeter
The Apache JMeter™ application is open source software, a 100% pure
Java application designed to load test functional behavior and measure
performance. It was originally designed for testing Web Applications
but has since expanded to other test functions.
I have Two Question related to Native App Performance Testing?
1)I have a Payment App, and it comes with bank security which is installed at the time of app installation. It sends an token number and rest of the data in encrypted format. Is it possible to handle such kind of request using Jmeter or any other performance testing tool, do i need to change some setting in app server or jmeter to get this done ?
2)Mobile App uses Device ID, so if i simulated load on cloud server it will use same Device ID which i used while creating script? is it possible to simulate different mobile ID to make it real-time?
any Help or references will be appreciated ..:)
(1) Yes. This is why performance testing tools are built around general purpose programming languages, to allow you (as the tester) to leverage your foundation skills in programming to leverage the appropriate algorithms and libraries to represent the same behavior as the client
(2) This is why performance testing tools allow for parameterization of the sending datastream to the server/application under test
I'm not an expert in JMeter. But work a lot with Loadrunner (LR) (Performance Testing Tool from HP). Though JMeter and LR are different tools, they work under same principle and objective and so objective of performance testing.
As James Pulley mentioned, the performance testing tool may have the capability. But the question is,
Have your tried recording your app with JMeter? Since your app is a native kind, please do the recording from simulator/emulator and check the feasibility. JMeter might not be the right candidate for mobile app load testing.
Alternatively there are lot of other tools available (both commercial and opensource) in market for your objective.
Best Regards
With the raise of several mobile network technologies, load testing a mobile application has become a different ball game in comparison with normal web app load testing. This is because of the differences in the response times that occur in different mobile networks such as 2G, 3G, 4G, etc. Additionally the client being a mobile device has plenty of physical constraints such as limited CPU, RAM, internal storage etc. All of these need to be considered while conducting performance testing of a mobile application if one wants to simulate a scenario close to a real time condition.
Coming to your 2 questions,
1) Yes it is possible but the amount of manual effort that needs to be invested to make the script execution ready might vary (since you are mentioning there is data in encrypted format - some are easy to understand and some are just crude and difficult to handle using JMeter). But there might not be any app server setting that would be required to change (unless of course you are unable to handle the encryption with JMeter in which case, the encryption might have to be disabled for QA phase)
2) As rightly said by James Pulley, these values can be parameterized. However, I fear that these values will be validated by the app server and hence the values need to be appropriately fed in the requests.
You can refer to this link for reference on how to do Mobile Performance Testing for Native application http://www.neotys.com/documents/doc/neoload/latest/en/html/#4234.htm#o4237
.The same could be extrapolated to JMeter to an extent.
I have developped a HTTP web service which is queried by smartphone. I want to test the performances of all this service containing :
A java server (java 6, java + play framework)
A database (Mysql 5.1.41)
A Linux (ubuntu) server (kernell 2.6.32)
I have tried leading test campaigns using python scripts with many threads or sequential tests. But it's hard to have conclusions...
I want to be able to have the maximum number of request per second for my service, the average time for each request... complete dashboards displaying a lot of information
I can do many scripts to test that but I am shure that well-known softwares permits to conduct these tests. Ideally these softwares could also display information about where I loose time ...
Do you have hints ?
Thanks for your help
Some tools I've used for HTTP benchmarking
Apache Bench
Siege
JMeter
Of these, JMeter is probably best for the situation you describe. All of these display a lot of information, but won't explain where you lose time.
For that, I'd suggest a profiler such as JVisualVM (comes with the JDK) or YourKit. From a profile you can observe where you spend the most time and focus on optimizing that.
I want to benchmark a new server using historical HTTP-request data. I have a textfile that contains one day's worth of real historical requests to a production server. What is the best tool for sending that list of requests on the server I'm testing? The tool I use should be able to configure the following:
Number of threads making the requests
Number of requests/second sent
A list of request URLs to use when making the requests.
Apache Bench seems like a close fit. However, Bench does not seem to be able to take in a list of request URLs as a parameter. What would you recommend?
I have been using http_load to pretty good success.
http://acme.com/software/http_load/
Http_load is a Unix command line tool that allows you to specify the number of requests/second and the number of threads to use when running. It pulls urls from a text file that you specify in the command. The tool is very similar to Apache Bench, with the big difference being that http_load allows you to use a list of URLs to be used when making requests. Apache Bench makes request to a single URL only.
If all your requests are GET requests (no POST) then you can use the JMeter Access Log Sampler. Here are some straight forward step-by-step instructions on how to set it up. It will run through your requests either in order or using a number of concurrent threads and you can specify how many requests should run. Then you can use the other features of JMeter like reports to analyze the results.
I would reccomend Visual Studio Test Edition. It would be a relativley simple matter to create a coded webtest that loads your url's for testing.
This advice predicates a knowledge of C# or VB for coding and the ability to install and licence Visual Studio. Visual Studio does have a trial edition available so you can have a taste of what you are getting first.
Visual Studio does not require the target site to be running any particular hardware or software, but it does provide more information on the load of the server due to the use of Perfmon counters and any ASP.Net application will provide more detail on the running app.
The caveat to this is that I have not actually used any other web testing software.