When I check the calls' waterfalls in dev tools, there are 3 calls that run in parallel (roughly) to each other. I understand resources can be loaded in parallel with "concurrent", but I couldn't find a way to run these HTML calls in parallel. Is there a way I can run these requests in parallel? My response time for the transaction is much higher than the manual time due to this.
Running on Version 12.60
Related
Can someone please explain the "Limit max thread number" property for the bzm parallel controller plugin? When monitoring requests from the browser we can see there are some requests that happen in parallel. Is this property to be set to the number of requests observed occurring in parallel?
What would be the issue of not setting a limit, if the scenario is run as part of an Azure pipeline? Would it automatically determine the max number of requests that are to be executed in parallel?
Thank you.
Parallel Controller executes all it's children in parallel.
The limit is introduced to support browser-like behaviour, the majority of modern browsers open up to 6 connections to a server in order to download embedded resources and execute AJAX calls.
So if you want your JMeter test to behave more like a real browser and you're using Parallel Controller to mimic calls to embedded resources or AJAX requests you should set the limit to match the browser you're trying to simulate
I ran a few tests using k6(OSS) by load impact and found it great in terms of usability compared to JMeter
I am doing a feasibility study to choose a load testing tool that should help me do API testing. I am inclined towards using K6 because I believe it is developer friendly but could not find resources that advise regarding maximum load I can simulate using K6.
Would it be possible to simulate 1 million rps(requests per second) using K6? If yes, how should I go about achieving this?
In theory, yes, if you use multiple k6 instances, you can achieve however many requests per second you want. A single k6 instance can produce anywhere from thousands to tens of thousands requests per second, depending on a lot of different factors - machine specs, script complexity, VUs, sleep times, network conditions, etc.
Right now k6 doesn't have a native distributed execution mode though, so you'd have to schedule the different instances yourself. There's a REST API (https://docs.k6.io/docs/rest-api) and you can output metrics to a centralized collector like InfluxDB (https://docs.k6.io/docs/results-output), but it'd take some work to execute a single test on multiple machines. A native k6 distributed execution mode is planned, but work on it hasn't started yet.
You can run k6 on the Load Impact (https://loadimpact.com) cloud (Cloud Execution mode) to access multiple k6 instances executing in parallel. Then, as noted, you can generate a large number of requests per second with the specific RPS being highly dependent on your script and other factors.
I wonder if some one solved the issue of browser multi thread with a request response script for load test
If you are going to use real Chrome and FF browsers for load test you can consider the following options:
Selenium Grid
Selenium-Grid allows you run your tests on different machines against different browsers in parallel. That is, running multiple tests at the same time against different machines running different browsers and operating systems. Essentially, Selenium-Grid support distributed test execution. It allows for running your tests in a distributed test execution environment.
Apache JMeter with the WebDriver Sampler plugin. This way you will be able to control concurrency and get performance metrics in form of HTML Reporting Dashboard.
Simulating web browser concurrency is something most load testing tools do very badly, if at all. We have tried to do this in k6 by letting each VU (virtual user) use multiple, concurrent TCP connections to fetch things in parallel. There is a special API function - http.batch() - that enables this functionality. http.batch() accepts multiple URLs as input parameters, and will fetch as many as possible in parallel.
Like Dmitri writes, Jmeter has a plugin that provides concurrency - sort of. However, what it actually does (unless I'm misinformed) is to spread out requests over multple VUs. Each VU will still only use one concurrent connection, which means that if you e.g. want to simulate 100 real browser users, you may need to fire up 1,000 VUs to do so realistically. This is not very different from what you would get with any other load testing tool, in my opinion: all of them allow you to start more VUs to create more concurrency and more traffic.
I'd say that apart from k6 and maybe one or perhaps two other tools (and Jmeter is not one of them), your only option if you want to really simulate the way browsers behave, is to use Selenium Grid or something similar to fire up a large number of real, actual browsers to do the load testing. The drawback is that real browsers are very expensive to run: they want lots of CPU and memory. But they provide the by far best browser "simulation".
What's the differences between JMeter and Tsung? I read that Tsung may generate more load than JMeter if test with same hardware, but how close it to reality?
Tsung is written in Erlang and said to be capable of running an extreme number of simultaneous users (10000+)
Jmeter is written in Java and quite capable of generating large amounts of load, assuming your test plan is good.
Here are some limitations in regards to performance in JMeter
Every user in Jmeter is an OS thread. This increases overhead when using a lot of concurrent users (jmeter best practices recommends using a low number of threads, http://jmeter.apache.org/usermanual/best-practices.html - in my experience you may run into problems when using more than 1000 threads per jmeter instance, but that may vary a lot depending on your test plan). You will also need to tweak JVM settings when running larger tests.
You can easily destroy performance & run out of PermGen memory if you have dynamic scripts (scripts that have to be recompiled every time because of jmeter variable expansion). Put your scripts in separate files or use a compilation cache key to avoid recompilation.
Using some test components (such as the Tree View, which keeps every request & response in memory) can wreak havoc on your load generator
I've tested some VERY large sites with JMeter, and as long as you are able to reduce the number of threads (reducing user-waits to keep throughput at your desired level) Jmeter is fine. There is quite a large community around JMeter, plugins for load testing using a lot of protocols & monitoring various systems. JMeter has also good scripting support - java, javascript, whatever that can be loaded in to a jvm basically (including for example groovy), so it is very extensible.
At one time (using jmeter 2.6 I think) I ran some 30,000 database requests per second (Oracle JDBC) from a single load generator, and there have been some optimizations since, so as long as you dont have extreme requirements, Jmeter is fine. Pick the one that suits your needs and your experience.
Note: I have very little experience using Tsung.
Edit: Nowadays I use Locust (https://github.com/locustio/locust/). Tsung has not been updated since 2017, and Locust's user/threading model (greenlets) supports more concurrent users than Jmeter. But most importantly it has a more flexible workflow (actual Python code instead of configuration/clicking in a GUI)
It's always depends on your scenario and amount off datas used for variablility, the ratio is closed to ten, when you'll be able to run 100 user/second with JMeter, Tsung will be easy with 1000 user/sec.
Our client asked us to run a stress test on their Web application simulating 62,000 users (threads), the test consists of 13-15 HTTP requests, with 1 second delay between every HTTP request, the test should run for 10.5 hours continuously.
I had previous experience with JMeter running up to 10,000 users, but have not tried for larger number.
Is there a limit for the number of threads that JMeter can handle, or is this limited by the hardware of test server?
agreed with Ray when you run jmeter for this much amount of thread the distributed testing is the best option, and the hardware and network shall be capable to handle this kind of load, and in case you want to do that in short go for http://blazemeter.com, they are Scalable from 1,000 to 100,000 concurrent users.
In principle it is limited by several factors including the configuration (CPU/Mem) of your JMeter machine. That said, it is a VERY large number of threads for just one JMeter. To be honest: I wouldn't run even 10000 threads on one machine. You might want to look into using JMeter distributed, see the manual (http://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf).
Except running script remotely, you'll need to configure your script with minimal usage of CPU and memory.
Use JMeter best practices and JMeter tuning tips to reach it