I have to test my rest API such that 100k API calls are made simultaneously(within 500ms).Any Idea how to simulate it?Utility to use?
I would suggest to use JMeter too, it enable your test to create multiple concurrent jmeter server. Just to be clear you can control multiple remote JMeter engines from a single JMeter client and replicate a test across many computers and thus simulate a larger load on the target server.
To be honest, your target is quite high (100k API calls simultaneously within 500ms), i.e. you'll need a lot of jmeter servers. When you create stress tests, there are not magical recipes, guides or manuals. Trial and error is a fundamental method of solving this kind of problems.
In my experience, I first try with few concurrent users and see how the server react. Then increase the number of concurrent users till to reach an intolerable performance decrease or, worst, a bottleneck .
http://jmeter.apache.org/usermanual/remote-test.html
You will obviously need a load testing tool which can be run in a distributed mode, i.e. 1 controller and X load generators executing the same test.
Grinder - scripts are written in some Python dialect
Apache JMeter - this guy doesn't require any specific knowledge, you can create tests using simple GUI
Tsung - is written in Erlang, known for capability to produce high loads even on low-end hardware.
See Open Source Load Testing Tools: Which One Should You Use? article for more information on above tools.
JMeter
The Apache JMeter™ application is open source software, a 100% pure
Java application designed to load test functional behavior and measure
performance. It was originally designed for testing Web Applications
but has since expanded to other test functions.
Related
I need some input from loadrunner/jmeter experts.
I'm having automation scripts which is written in selenium+ java .
Can i use same scripts to do performance testing using loadrunner/jemeter.?
why i want do like this : to maintain single code base. if any changes happen so i will able to correct at one place and use it for UI Automation and performance testing.
Any help appreciate.
Selenium automates browsers that it!
Jmeter is specifically to use for Performance test. to be more specific either stress testing or load testing.
JMeter and Selenium can be used together for load testing of a website. Selenium is used to perform load testing and JMeter is used to measure the performance of the same. One can measure how fast HTML pages take to load, and also check javascript and CSS performances. This is done using a combination of the graphical JMeter interface and some scripting to invoke Selenium code.
Please read more about it here
JMeter and Selenium are both used for web application testing. But the main difference is that JMeter is mostly used to test the performance of web applications under load and stress. On the other hand, Selenium is best suited for automation and cross browser testing of a website. Based on what type of testing is necessary for the project, one can choose JMeter or Selenium or integrate both if that serves their purpose.
Theoretically yes, it is possible, but it is not recommended even by Selenium developers, be informed of the following additional constraints:
Browsers are quite resource intensive, for example Firefox 94 needs 1 CPU core and 2 GB of RAM per instance so you can calculate how much RAM you will need to simulate, say, 100 users.
You won't get metrics and KPIs normally available as the result of a performance test run
Trying to Performance test an application developed in OJET technology. Which tool/protocol should I use for scripting? I tried HTTP/Web protocol with Jmeter and Load Runner. But that doesn't capture all the requests and responses at the javascript/browser level. Hence I am facing issues in correlating the dynamic values during test design. Hence, scripts fail during the replay. Currently trying to do it with Truclient Web protocol as an alternative. But I need to know which tool/protocol should I use for scripting?
According to OJET looks like this is a web app generator.
If you choose to start with JMeter use post-processor such as regex to catch and save every value that is needed for as arg in the next request.
Don't be afraid of these dynamic values. Try to follow next articles to get the idea.
None of tools will provide you automatic correlation without issue. Nor LoadRunner, nor Jmeter. It is always tricky.
Ask more specific questions when you start facing issue.
Jmeter catch correlations
You need to implement real user using your application with 100% accuracy in terms of network footprint
Both JMeter and/or LoadRunner are not capable of executing client-side JavaScript, the options are in:
Implement these JavaScript-driven network calls using scripting (in JMeter it will be JSR223 Test Elements)
Use a real browser, LoadRunner's Truclient protocol is basically a headless web browser, in JMeter can be integrated with Selenium browser automation framework via WebDriver Sampler
With regards to "which protocol/tool" to use:
Implementing JavaScript calls manually will take extra effort, however your test will consume less resources (CPU, RAM, etc.)
Using real browsers will take less efforts, but the test will consume much more resources (something like 1 CPU core and 2 GB of RAM per user/ browser instance) and you won't have metrics like Connect Time, Latency, etc.
LoadRunner TruClient. This will handle all of the Javascript executions and dynamic elements related to session, state, date/time, object identifiers, ... You will still need to appropriately handle user input items.
Most of the tools out there do not seem to be interpreting jQuery / Javascript code that is loaded on the page that I want to test. This is however important because that code will, in turn initiate a number of calls to other endpoints in my webapp which adds up to load in a real life scenario.
I've looked at JMeter already and am a bit reluctant to give it a try as the landing page mentions explicitly that it does not do Javascript interpreting at all.
What would be some recommendation of tools that can provide a more accurate measurement of load testing by including Javascript code?
None of the load testing tools really executes JavaScript, they all act on protocol level and JavaScript is being executed by browser.
There are 3 options how you could proceed if you want 100% realistic testing:
HP LoadRunner has TrueClient protocol which is basically headless browser with JavaScript capabilities.
JMeter with WebDriver Sampler plugin - the way to kick off real browsers from JMeter test
Selenium Grid (or other way to kick off several Selenium instances at once)
All 3 above options are very resource intensive, you will required at least 1 CPU core and a couple of gigabytes or RAM for a single browser instance so I would recommend a little bit different approach. JMeter cannot execute JavaScript, but it can execute JavaScript-driven HTTP Requests so create main load using JMeter (or equivalent) and while the load test is running use Selenium to automatically check real browsing experience or YSLow to do the same manually.
Load testing tools doesn't execute JavaScript.
You know which endpoints your JavaScript code is using so just add these endpoints to your JMeter scenario.
The thick client problem has been around since the early 1990s with traditional thick client-server applications. In fact, the earliest commercial tools were all driving full clients on X Windows before adding API level virtual user capabilities. On the commercial front this ability to drive full, thick clients is still expressed as GUI virtual users (Mercury/HP/Microfocus LoadRunner, Segue/Borland/Microfocus Silk Performer, Rational/IBM Performance Tester) allowing you to measure the weight between API and user level if needed.
Here is the thing, for a given business process and data set the end client is predictable in its behavior, and in the requests that it makes to the servers that are feeding it data. There may be a few odd conditions where you need to reproduce an algorithm from JavaScript to C (or even run it directly in some tools) for branching on a type of API/HTTP request, but these are not numerous.
I've got a PHP site up and running, and the db is mysql. before launching the site, I would like to test the traffic handling. Now am assuming that there are soe softwares that would simulate the traffic and log the processes running on my site. Any recommendation of software I should use? the traffic doesn't have to be real, but nonetheless, I would like to generate a high traffic to investigate the threshold of the site.
Appreciate the help
You can use Gatling https://github.com/excilys/gatling.
It's a stress tool written in Scala which aims at being more efficient and lighter than Jmeter.
Basically you record a scenario on your website and then run it 'n' times in parallel.
Here is the wiki for more infos https://github.com/excilys/gatling/wiki/Basic-Usage
You can use Jmeter:
It's free.
it's easy to Start with lot of documentation on its Website and on internet
it has a proxy feature to easily create test plan from browser navigation
It is easy to start up processes on other machines. It remote testing, can be done from GUI or console.
The scripts can be written in beanshell, java, or any jsr223 language ( groovy, Javascript, scala, jexl ...)
it has a lot of built- in samplers and thanks to its plugin architecture it's very Easy to add new ones or use any scripting engine to do what's missing
it has great user mailing list
it has very reactive support
it's now a top Level Apache
it can run thousands of users
professional solutions exist to run it from cloud
...
See:
Performing a Stress Test on Web Application?
Best way to stress test a website
How do you test the performance of a website?
what is the best tool (open or commercial) currently available, that lets me send customized requests to a web server and get back a response to check the performance?
i will be sending it a load of more than 20K per second, but i need to get numbers for each call made. also, the numbers might be in some microseconds or nanoseconds. How in this small measurement unit, can i work out a baseline and a benchmark?
If you're using Apache, Apache AB is a benchmarking to test how many requests your serve can serve per second and how well it handles load and concurrency. It's an open-source project - check it out here.
In addition, wikipedia has a nice list of benchmarking software for testing servers.
You can use the Web Application Stress Tool of Microsoft
The Microsoft WAS web stress tool is designed to realistically simulate multiple browsers requesting pages from a web site. You can use this tool to gather performance and stability information about your web application. This tool simulates a large number of requests with a relatively small number of client machines. The goal is to create an environment that is as close to production as possible so that you can find and eliminate problems in the web application prior to deployment.
You can find a list of Open Source software for performance (most of them are for web that send custom request to webserver).
Don't know if either of these have granularity better than milliseconds but check out JMeter (open source) and LoadRunner (Commercial). LoadRunner is not cheap but it allows you to span load generation across multiple machines with aggregated results.