I am planning to load the backend apis with 1000 concurrent users using jmeter and at the same time using chrome lighthouse or pagespeed insight tool to understand how the client side is performing.
Is this the right approach or is there any better way to do that ?
Tried to load the backend api with 1000 concurrent load and at the same time used lighthouse tool to check the client side performance.
Client side performance and backend performance are two different beat which are not very related.
If you're interested in images and style loading time - JMeter can give you that information, just configure it to download embedded resources using HTTP Request Defaults
What JMeter cannot provide is i.e. JavaScript execution time, page rendering time, web vitals and so on, if you want to collect this information in an automated manner - JMeter can be integrated with Selenium browser automation framework using WebDriver Sampler
Related
Trying to Performance test an application developed in OJET technology. Which tool/protocol should I use for scripting? I tried HTTP/Web protocol with Jmeter and Load Runner. But that doesn't capture all the requests and responses at the javascript/browser level. Hence I am facing issues in correlating the dynamic values during test design. Hence, scripts fail during the replay. Currently trying to do it with Truclient Web protocol as an alternative. But I need to know which tool/protocol should I use for scripting?
According to OJET looks like this is a web app generator.
If you choose to start with JMeter use post-processor such as regex to catch and save every value that is needed for as arg in the next request.
Don't be afraid of these dynamic values. Try to follow next articles to get the idea.
None of tools will provide you automatic correlation without issue. Nor LoadRunner, nor Jmeter. It is always tricky.
Ask more specific questions when you start facing issue.
Jmeter catch correlations
You need to implement real user using your application with 100% accuracy in terms of network footprint
Both JMeter and/or LoadRunner are not capable of executing client-side JavaScript, the options are in:
Implement these JavaScript-driven network calls using scripting (in JMeter it will be JSR223 Test Elements)
Use a real browser, LoadRunner's Truclient protocol is basically a headless web browser, in JMeter can be integrated with Selenium browser automation framework via WebDriver Sampler
With regards to "which protocol/tool" to use:
Implementing JavaScript calls manually will take extra effort, however your test will consume less resources (CPU, RAM, etc.)
Using real browsers will take less efforts, but the test will consume much more resources (something like 1 CPU core and 2 GB of RAM per user/ browser instance) and you won't have metrics like Connect Time, Latency, etc.
LoadRunner TruClient. This will handle all of the Javascript executions and dynamic elements related to session, state, date/time, object identifiers, ... You will still need to appropriately handle user input items.
JMeter does not support the client side performance testing (verifying the content rendering time / page loading time) out of the box.
In single page applications requests are sent to the server as API calls. I am interested in verifying the client side page loading time for concurrent users with JMeter.
What is the best way to test the SPA with JMeter ?
Do you recommend Selenium WebDriver plugin?
Is there any other tools that can be used to verify the client side
load testing for concurrent users?
As per JMeter project main page:
JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).
So I doubt whether it makes sense even to consider JMeter for pure frontend operation, most probably you're looking for Performance tab of your browser developer tools or more sophisticated solutions like Lighthouse or WebPageTest
With regards to WebDriver Sampler plugin, it just provides JMeter integration with Selenium browser automation framework so you will be able to launch a real browser per JMeter thread (virtual user) and the rest depends on your implementation, most probably you will want to query window.performance APIs values to get the metrics you're looking for via executeScript() function
In your opinion which will be better the record or the creation of the scripts by Exemple with selenium and junit?
Does the record method allow test visibility and maintainability of the code?
JMeter(Open Source)/LoadRunner(Licensed) are the most popular tools for performance testing(Load testing). Selenium is not the right choice for this job, it is the automation tool for web applications.
If you want to start in performance testing, I would suggest you to use JMeter for this job.
It is not a good way to use selenium for load testing, But Jmeter's main purpose is to do performance testing on web applications. So Jmeter is the right tool to to do performance related testing.
Selenium is a browser automation framework, it means 1 virtual user will kick off a real browser instance and modern web browsers are very CPU and memory intensive, for example Firefox 72.0.2 requires 1 CPU core and 2 GB or RAM per browser instance.
JMeter in its turn operates on HTTP protocol level which has much less footprint.
So if you need to simulate large number of users - it's better to consider JMeter.
If you're doing client-side performance testing (measure JavaScript execution time, page rendering speed, etc.) - go for Selenium.
If you need both - create main load with JMeter and use 1-2 virtual users running Selenium
Check out How to Convert Selenium Scripts into the JMX Converter article to learn how Selenium tests can be transformed into JMeter using Proxy2JMX service of the Taurus tool
Most of the tools out there do not seem to be interpreting jQuery / Javascript code that is loaded on the page that I want to test. This is however important because that code will, in turn initiate a number of calls to other endpoints in my webapp which adds up to load in a real life scenario.
I've looked at JMeter already and am a bit reluctant to give it a try as the landing page mentions explicitly that it does not do Javascript interpreting at all.
What would be some recommendation of tools that can provide a more accurate measurement of load testing by including Javascript code?
None of the load testing tools really executes JavaScript, they all act on protocol level and JavaScript is being executed by browser.
There are 3 options how you could proceed if you want 100% realistic testing:
HP LoadRunner has TrueClient protocol which is basically headless browser with JavaScript capabilities.
JMeter with WebDriver Sampler plugin - the way to kick off real browsers from JMeter test
Selenium Grid (or other way to kick off several Selenium instances at once)
All 3 above options are very resource intensive, you will required at least 1 CPU core and a couple of gigabytes or RAM for a single browser instance so I would recommend a little bit different approach. JMeter cannot execute JavaScript, but it can execute JavaScript-driven HTTP Requests so create main load using JMeter (or equivalent) and while the load test is running use Selenium to automatically check real browsing experience or YSLow to do the same manually.
Load testing tools doesn't execute JavaScript.
You know which endpoints your JavaScript code is using so just add these endpoints to your JMeter scenario.
The thick client problem has been around since the early 1990s with traditional thick client-server applications. In fact, the earliest commercial tools were all driving full clients on X Windows before adding API level virtual user capabilities. On the commercial front this ability to drive full, thick clients is still expressed as GUI virtual users (Mercury/HP/Microfocus LoadRunner, Segue/Borland/Microfocus Silk Performer, Rational/IBM Performance Tester) allowing you to measure the weight between API and user level if needed.
Here is the thing, for a given business process and data set the end client is predictable in its behavior, and in the requests that it makes to the servers that are feeding it data. There may be a few odd conditions where you need to reproduce an algorithm from JavaScript to C (or even run it directly in some tools) for branching on a type of API/HTTP request, but these are not numerous.
what is the best tool (open or commercial) currently available, that lets me send customized requests to a web server and get back a response to check the performance?
i will be sending it a load of more than 20K per second, but i need to get numbers for each call made. also, the numbers might be in some microseconds or nanoseconds. How in this small measurement unit, can i work out a baseline and a benchmark?
If you're using Apache, Apache AB is a benchmarking to test how many requests your serve can serve per second and how well it handles load and concurrency. It's an open-source project - check it out here.
In addition, wikipedia has a nice list of benchmarking software for testing servers.
You can use the Web Application Stress Tool of Microsoft
The Microsoft WAS web stress tool is designed to realistically simulate multiple browsers requesting pages from a web site. You can use this tool to gather performance and stability information about your web application. This tool simulates a large number of requests with a relatively small number of client machines. The goal is to create an environment that is as close to production as possible so that you can find and eliminate problems in the web application prior to deployment.
You can find a list of Open Source software for performance (most of them are for web that send custom request to webserver).
Don't know if either of these have granularity better than milliseconds but check out JMeter (open source) and LoadRunner (Commercial). LoadRunner is not cheap but it allows you to span load generation across multiple machines with aggregated results.