I run test in Jmeter with Selenium WebDriver Sampler
on Linux X86 and java SDK 11
The test run with 50 users.
I run it from the command line with non Gui mode and with Chrome headless mode.
but after 5 minutes the CPU going up to 100% and the memory almost full (8G).
What can I do to improve it?, I need run the test with 200 users and up.
Thanks,
Izik
You're expecting too much from your machine.
Although there are no specific RAM requirements defined at the Chrome System Requirements page, on my machine a single Chrome instance with a single tab in "porno" mode consumes almost 1GB of RAM. And this is given http://example.com page open, not modern web application with tons of JavaScript
I bet if you run the following command on your machine - you will get at least 3GB
(Get-Process chrome | Measure-Object WorkingSet -sum).sum
As per WebDriver Sampler tutorial
Note: It is NOT the intention of this project to replace the HTTP Samplers included in JMeter. Rather it is meant to compliment them by measuring the end user load time.
So my expectation is that you should be conducting the load using HTTP Request samplers and forget about using real browsers (or use 1 instance to collect client-side performance metrics). Just consider following recommendations from How to make JMeter behave more like a real browser article to ensure that protocol-based JMeter test has the same network footprint as real browser produces.
If you have to use real browsers for performance testing you won't be able to launch 200 browser instances on a machine with 8GB of RAM, you will have to find another 30-40 machines of this specifications and go for Distributed Testing.
This is expected, Since you are using Selenium it will use the JVM and browser which will consume a lot of memory. I would suggest you to distribute the test in multiple machine if you are going the Selenium route for load testing. This way you will be able to load test for more more number of users.
The best would be to stick to HHTP sampler as suggested above. You could also record and make necessary changes.
Related
We are planning to do load testing in browser so would like to know if you suggest any tools to achieve the 10k+ load requests.
Also would like to know if it can be achieved through jmeter for browser test.
Thanks In Advance
JMeter per se works on HTTP protocol level, however given you properly configure JMeter to behave like a real browser from the application under test perspective it will be no difference at all, it wouldn't recognise whether it's being hit by JMeter with 10k of virtual users or 10k real browsers.
The most problematic point of using real browsers is an immense amount of hardware you will need for the test, if you look at Firefox 71 browser requirements you will see that for normal operation of browser you will need a CPU core and 2 GB of RAM so for 10k real browsers you will need 10K processors and 20 terabytes of RAM.
If you still want to proceed with real browsers be aware that JMeter can be integrated with Selenium browser automation framework using WebDriver Sampler but most probably you will have to go for Distributed Testing
I'm doing a jmeter test the test is on localhost . My testplan is to hit login and then another http request . The test is running and getting response . But when I run the same test again I'm getting different results. I'm doing it in GUI mode . Please help me I'm new to jmeter.
You're violating at least 2 main best practices:
JMeter's GUI mode is for tests development and debugging only, when it comes to test execution you must run JMeter in command-line non-GUI mode
Having load generator (JMeter) and the system under test on the same machine is not the best idea, both tools may be very resource intensive when it comes to high loads and at some point they will start fighting for the OS resources like CPU, RAM, Network and Disk IO, etc. so when it comes to the bottleneck identification you will not be able to tell for sure why response times are high or throughput is low as the reasons could be in:
JMeter doesn't have enough headroom to operate and cannot send requests fast enough
Application cannot respond fast enough due to lack of RAM or CPU overload
CPU is constantly switching the context from JMeter to the application under test and vice versa
We are using JMeter to perform load testing of apis. If want to perform load or stress testing on web application which has JavaScript rendering, would JMeter with Selenium only option or any other options can be utilized like Selenium functional tests integrate with any performance tool.
Please suggest.
Have gone through/referred to this questions:
How to approach "end-client" performance testing on single-page (web) applications?
You can choose to use Selenium but it's hard to generate high load using selenium based test. You can choose to run two performance tests in parallel
Back-end Performance test: Use this to inject load on the back-end. Record Network traffic for the journey steps using JMeter proxy. Take out third-party and static urls, which are not relevant to load test.
Front-end performance test: Use this to measure end-user impact. Use between 1 - 5 virtual users in this test. Generating load is not important in this case, so you can use low amount of virtual users in this case. You can either use selenium based load test, webpagetest.org or similar tools.
This approach is more efficient than the selenium based performance test because selenium based load test will require very high amount of compute resources to scale.
If you use taurus to run your perf test then it should make it easier to run two JMeter tests in parallel using bzt scenario1.jmx scenario2.jmx
Hope this helps.
Depending on what you're trying to achieve and how many machine power you have you can consider:
Running Selenium tests in parallel using Selenium Grid but in this case:
you will need to ensure to have enough CPU, RAM, etc., i.e Firefox 72 needs 1 CPU and 2 GB of RAM per single browser instance
you will not have HTTP protocol specific metrics (Connect time, time to first byte, time to last byte, etc.) and load test specific metrics (active threads over time, transactions per second, etc)
Conducting the main load using JMeter HTTP Request samplers (just make sure to properly configure JMeter to behave like a real browser) and have 1-2 threads running WebDriver Sampler to collect client-side performance metrics i.e from Performance object
I wonder if some one solved the issue of browser multi thread with a request response script for load test
If you are going to use real Chrome and FF browsers for load test you can consider the following options:
Selenium Grid
Selenium-Grid allows you run your tests on different machines against different browsers in parallel. That is, running multiple tests at the same time against different machines running different browsers and operating systems. Essentially, Selenium-Grid support distributed test execution. It allows for running your tests in a distributed test execution environment.
Apache JMeter with the WebDriver Sampler plugin. This way you will be able to control concurrency and get performance metrics in form of HTML Reporting Dashboard.
Simulating web browser concurrency is something most load testing tools do very badly, if at all. We have tried to do this in k6 by letting each VU (virtual user) use multiple, concurrent TCP connections to fetch things in parallel. There is a special API function - http.batch() - that enables this functionality. http.batch() accepts multiple URLs as input parameters, and will fetch as many as possible in parallel.
Like Dmitri writes, Jmeter has a plugin that provides concurrency - sort of. However, what it actually does (unless I'm misinformed) is to spread out requests over multple VUs. Each VU will still only use one concurrent connection, which means that if you e.g. want to simulate 100 real browser users, you may need to fire up 1,000 VUs to do so realistically. This is not very different from what you would get with any other load testing tool, in my opinion: all of them allow you to start more VUs to create more concurrency and more traffic.
I'd say that apart from k6 and maybe one or perhaps two other tools (and Jmeter is not one of them), your only option if you want to really simulate the way browsers behave, is to use Selenium Grid or something similar to fire up a large number of real, actual browsers to do the load testing. The drawback is that real browsers are very expensive to run: they want lots of CPU and memory. But they provide the by far best browser "simulation".
We are using Apache JMeter for the performance testing of web application. Apparently response time is too high in comparison to loading page in browser during load. When we open the page during load it opens in 2 seconds however JMeter reports 70 seconds. I understand browser memory cache and disk cache are used in browser however isn't JMeter cache manager does same. How to assert it, comparing response header is one option. Any thoughts on this will be appreciated.
It may be wrong configuration in the script. There won't be much difference between web browser and Jmeter response times (browser rendering time is ignored in Jmeter, not a big factor but must be considered)
If you are using single Http Sampler for a web page and retrieving all resources in tha page, then select "Parallel Downloads" option to '6' in Http sampler advanced section. So, you are simulating the browser behaviour of parallel downloading of the resources like .js, .css, images etc.
If you recorded the script using Test Script Recorder, then there will be an Http Sampler for each resource requet for that page which will be sent sequentially, hence increase in response time. You might be in this case, as of now, there is no feature/option to send the http samplers in parallel. so I suggest using the approach of adding one sampler, use parallel download of resources option in Http Sampler advanced section.
Also, caching is an important factor which decides the response time. Adding Http Cache manager can solve the issue in jmeter. This simulates browser behaviour of caching. I don't think there will be huge difference b/w browser and jmeter in implementing caching althought may not be perfect.
The reason could be the lack of resources on JMeter machine or JMeter not being properly configured for producing high loads. JMeter default configuration is good for load tests development and debugging, you can run load tests up to certain amount of virtual users, but if you need to conduct a really high loads - you need to perform some configuration changes.
First of all, double check my guess using jvisualvm and your operating system monitoring tools (there is PerfMon JMeter Plugin which can be used for monitoring different metrics on application under test and JMeter load generators sides). If it is the case - take the next steps to get the most performance of your JMeter installation:
Increase JVM Heap Size, NewSize, switch to ConcurrentMarkSweep Garbage Collector.
Run your test in non-GUI mode.
Disable all the listeners during the test.
Follow other recommendations from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article.