I wonder if some one solved the issue of browser multi thread with a request response script for load test
If you are going to use real Chrome and FF browsers for load test you can consider the following options:
Selenium Grid
Selenium-Grid allows you run your tests on different machines against different browsers in parallel. That is, running multiple tests at the same time against different machines running different browsers and operating systems. Essentially, Selenium-Grid support distributed test execution. It allows for running your tests in a distributed test execution environment.
Apache JMeter with the WebDriver Sampler plugin. This way you will be able to control concurrency and get performance metrics in form of HTML Reporting Dashboard.
Simulating web browser concurrency is something most load testing tools do very badly, if at all. We have tried to do this in k6 by letting each VU (virtual user) use multiple, concurrent TCP connections to fetch things in parallel. There is a special API function - http.batch() - that enables this functionality. http.batch() accepts multiple URLs as input parameters, and will fetch as many as possible in parallel.
Like Dmitri writes, Jmeter has a plugin that provides concurrency - sort of. However, what it actually does (unless I'm misinformed) is to spread out requests over multple VUs. Each VU will still only use one concurrent connection, which means that if you e.g. want to simulate 100 real browser users, you may need to fire up 1,000 VUs to do so realistically. This is not very different from what you would get with any other load testing tool, in my opinion: all of them allow you to start more VUs to create more concurrency and more traffic.
I'd say that apart from k6 and maybe one or perhaps two other tools (and Jmeter is not one of them), your only option if you want to really simulate the way browsers behave, is to use Selenium Grid or something similar to fire up a large number of real, actual browsers to do the load testing. The drawback is that real browsers are very expensive to run: they want lots of CPU and memory. But they provide the by far best browser "simulation".
Related
I'd few ques on technical details of JMeter mostly pertaining to distributed setup vs independent JMeter engines (since JMeter controller can become a bottleneck in case of several JMeter load generators). Would be great if anybody can help with the understanding here -
How is JMeter distributed setup orchestrated by JMeter controller (i.e. called master or client)? Can we use the same logic to synchronize test among independent JMeter engines (independent mode)?
Is there a way to pool connections across vUsers?
Function of ASYNC_QUEUE in backend listener and it's expected side-effects in independent mode (mentioned above), what happens when queue is full?
Does/Is there a way for JMeter to execute javascript/act as headless browser?
How does DNS resolution happen for JMeter? Does it resolve for each vuser?
Your "question" looks like a compilation of interview questions rather than something connected with your single current concern and I don't think it's a proper place/way to ask it, I believe it should be: one post - one question.
Whatever
How is JMeter distributed setup orchestrated by JMeter controller - JMeter master sends .jmx script to slaves and collects results from them. Theoretically you can implement your own mechanism for delivering the test plan and eventual dependencies to the individual JMeter engines and running the test at the same time. Then you will need to collect the .jtl results files from the engines and combine it into a single one.
Is there a way to pool connections across vUsers? - JMeter does it internally
When the queue is full no more new sample results will be taken for processing by the backend listener so the results won't be "realtime" anymore, you will see the new results as free slots will be appearing in the queue
For JMeter per-se - no, AJAX calls can be simulated using Parallel Controller, for client-side performance testing, JavaScript execution profiling and rendering speed measurement you will need to use a read browser, no matter normal or headless, there is WebDriver Sampler plugin providing JMeter integration with Selenium
DNS resolution is dependent on underlying OS and/or JVM DNS resolution implementation, there is DNS Cache Manager which enables overriding hosts entries and using custom DNS resolver so each thread looks up the IP address on its own
We are planning to do load testing in browser so would like to know if you suggest any tools to achieve the 10k+ load requests.
Also would like to know if it can be achieved through jmeter for browser test.
Thanks In Advance
JMeter per se works on HTTP protocol level, however given you properly configure JMeter to behave like a real browser from the application under test perspective it will be no difference at all, it wouldn't recognise whether it's being hit by JMeter with 10k of virtual users or 10k real browsers.
The most problematic point of using real browsers is an immense amount of hardware you will need for the test, if you look at Firefox 71 browser requirements you will see that for normal operation of browser you will need a CPU core and 2 GB of RAM so for 10k real browsers you will need 10K processors and 20 terabytes of RAM.
If you still want to proceed with real browsers be aware that JMeter can be integrated with Selenium browser automation framework using WebDriver Sampler but most probably you will have to go for Distributed Testing
We are using JMeter to perform load testing of apis. If want to perform load or stress testing on web application which has JavaScript rendering, would JMeter with Selenium only option or any other options can be utilized like Selenium functional tests integrate with any performance tool.
Please suggest.
Have gone through/referred to this questions:
How to approach "end-client" performance testing on single-page (web) applications?
You can choose to use Selenium but it's hard to generate high load using selenium based test. You can choose to run two performance tests in parallel
Back-end Performance test: Use this to inject load on the back-end. Record Network traffic for the journey steps using JMeter proxy. Take out third-party and static urls, which are not relevant to load test.
Front-end performance test: Use this to measure end-user impact. Use between 1 - 5 virtual users in this test. Generating load is not important in this case, so you can use low amount of virtual users in this case. You can either use selenium based load test, webpagetest.org or similar tools.
This approach is more efficient than the selenium based performance test because selenium based load test will require very high amount of compute resources to scale.
If you use taurus to run your perf test then it should make it easier to run two JMeter tests in parallel using bzt scenario1.jmx scenario2.jmx
Hope this helps.
Depending on what you're trying to achieve and how many machine power you have you can consider:
Running Selenium tests in parallel using Selenium Grid but in this case:
you will need to ensure to have enough CPU, RAM, etc., i.e Firefox 72 needs 1 CPU and 2 GB of RAM per single browser instance
you will not have HTTP protocol specific metrics (Connect time, time to first byte, time to last byte, etc.) and load test specific metrics (active threads over time, transactions per second, etc)
Conducting the main load using JMeter HTTP Request samplers (just make sure to properly configure JMeter to behave like a real browser) and have 1-2 threads running WebDriver Sampler to collect client-side performance metrics i.e from Performance object
I ran a few tests using k6(OSS) by load impact and found it great in terms of usability compared to JMeter
I am doing a feasibility study to choose a load testing tool that should help me do API testing. I am inclined towards using K6 because I believe it is developer friendly but could not find resources that advise regarding maximum load I can simulate using K6.
Would it be possible to simulate 1 million rps(requests per second) using K6? If yes, how should I go about achieving this?
In theory, yes, if you use multiple k6 instances, you can achieve however many requests per second you want. A single k6 instance can produce anywhere from thousands to tens of thousands requests per second, depending on a lot of different factors - machine specs, script complexity, VUs, sleep times, network conditions, etc.
Right now k6 doesn't have a native distributed execution mode though, so you'd have to schedule the different instances yourself. There's a REST API (https://docs.k6.io/docs/rest-api) and you can output metrics to a centralized collector like InfluxDB (https://docs.k6.io/docs/results-output), but it'd take some work to execute a single test on multiple machines. A native k6 distributed execution mode is planned, but work on it hasn't started yet.
You can run k6 on the Load Impact (https://loadimpact.com) cloud (Cloud Execution mode) to access multiple k6 instances executing in parallel. Then, as noted, you can generate a large number of requests per second with the specific RPS being highly dependent on your script and other factors.
I'm planning to do a load test of our ASP/.NET web application and need to simulate about 600 concurrent users on our system.
Initially we'll just be running the load test tools (probabaly JMETER or WCAT/WAST) from our personal workstations which are Windows 7/32 Bit Dells (Dual Core processors). I was wondering about how many users I can expect to be able to simulate from one client.
If I can easily do 200 users per client, I'll need to identify 2-3 more clients for the test.
I wanted to ask the community based on their experience how many users I should expect per client on a standard windows box.
Any help is appreciated!
This highly depends on the test plan itself and cannot be answered that easily.
If you for example have 500 users that just do one request and then have a waiting timer for five minutes, this should work. If all users constantly do requests without waiting, this will put much more load on your machine.
It depends on the samplers in use. HTTP requests are less costly than SOAP requests for example.
It also depends on the listeners you have active.
For a normal load test I usually have around 100-300 threads active. I would suggest to start with such a number and to monitor the load (CPU, network) on your client to see how much potential there is.
Without more details about the test scenarios and the hardware, it is hard to give specific answers. But our Load Tester product can (usually) handle this level of users pretty easily on a single machine (assuming relatively modern hardware). The testing tool should scale linearly up to a point, so you should be able to get a good estimate by running 50 users through a scenario that is similar to what you expect to test.