I want to run basic load testing scenario on a mendix application.
The initial requirement is at least 10 users should be able to perform some activities like login , file upload concurrently.
I have used the webdriver sampler in Jmeter with selenium and written the JS script for the actions.
The issue is it launches 10 browser on the same machine.
Please suggest the proper way to handle this. Currently it seems that the errors which I am getting are because of I am running the test on my laptop instead of distributed setup.
Also the total script takes 60 seconds to finish for single user. What should be the ramp up time?
If not Jmeter, please suggest any other suitable tool for this scenario.
As per The WebDriver Sampler: Your Top 10 Questions Answered article
Q. How do I Allocate the WebDriver Sampler?
A. First of all, don’t use the WebDriver Sampler to create the load! You’ll need around 1 CPU core per virtual user to keep the JMeter resource consumption within an acceptable range. The WebDriver Sampler should be used in addition to HTTP Request Samplers. Here’s an example of how it should be used:
1,000 of users are simulated by HTTP Request Samplers
1 user is the WebDriver Sampler
You should use the WebDriver Sampler to evaluate a real-life user experience. It’s most commonly used to measure the load times of pages when the system is experiencing high loads. The HTTP Request doesn’t really “render” the page and it can’t execute JavaScript. That’s why the WebDriver Sampler is so valuable - JMeter isn’t a browser and it overcomes limitations posed by this fact.
So first of all I would recommend re-considering your approach and migrating to HTTP Request samplers, you might be able to execute 10 browsers but if you will need to scale your test to 100 or 1000 users you will definitely won't be able to use this "real browser" approach while HTTP Request samplers act on HTTP protocol level, given you properly configure JMeter it will be no difference for the application under test whether the requests are originating from JMeter or from the real browser and the resources footprint will be much less.
With regards to the ramp-up time: the overall idea is to increase the load gradually so you could correlate the changing number of virtual users with changing metrics like response time, throughput, etc. According to JMeter Documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
Related
I have a HTTP Request in my Thread Grpup that takes around 20 to 30 seconds to complete with a single user, so when I added 50 users I get a 500/Internal Server Error or 503/Server has been shutdown sometimes.
I want to add a Constant Timer with 40 seconds (in miliseconds) under the HTTP Request so maybe the application will have some time to process it. I am going to the rigth way?
If I add the Constant Timer will it be calculate as well in the Summary Report?
I need that the Jmeter give the time to the API (My aplication) complete the process (need at least 30 seconds) and it may be or not affetct my Summary Report
PreProcessors, Post-Processors and Timers are not counted in the Elapsed time. so response time will not be impacted.
However Throughout (the number of requests for the test duration) will be lower.
See JMeter Glossary for more information on the above metrics.
With regards to "right way" - real users don't "hammer" application non-stop, they need some time to "think" between operations so if you're simulating a real user you should have non-zero think time, however 40 seconds it kind of too much for me. Take a look at How to make JMeter behave more like a real browser article for more tips on properly configuring your JMeter test.
Which controller or timer I should use for controlling rps ?
Can you please let me know?
Thanks,
You can achieve 10000 RPS with 10000 threads (virtual users) only if your application response time is 1 second
if your application response time is less than one second - you will get more RPS
if your application response time is more than one second - you will get less RPS
There are 2 main per-requisities:
Your application has to be able to respond fast enough because if it is not capable of supporting 10k requests per second - no matter what you do you won't be able to reach the target throughput as JMeter waits for the response prior to starting the next request
JMeter has to be able to send requests fast enough so make sure to follow JMeter Best Practices and it might be the case you will have to go for distributed testing mode
The easiest way of conducting X requests per second load is using Concurrency Thread Group and Throughput Shaping Timer combination, both can be installed using JMeter Plugins Manager
While understanding concept of Concurrent thread and ultimate thread group, i am confused to understand result of summary/aggregate report while running concurrent thread or ultimate thread group .For example if i have 200 user and ramp up time 60 sec then i didn't see all sampler request as 200 sample after completing execution successfully but only few sampler request have 200 sample. when i use normal thread group then i always got thread count same for each sampler request after completing execution .
for realistic load testing with more user , could you please suggest me which thread group should prefer.
Could you please provide valuable guidance with some valuable link/book and also share me standard performance bench mark criteria or key parameter detail while doing load testing.(if any bench load parameter value does not meet standard then we can say that there is a performance issue)
Thanks for giving valuable time in advance.
Thanks
amit
This is due to the fact that:
Your application response time is too high
Your test duration is too low
For example I can see response times > 80 seconds:
it means that if a single virtual user has cumulative response time for 2 samplers > 160 seconds and the test duration is 120 seconds it will not be able to execute all the requests. Just increase your test duration to be i.e. 10 minutes and you should see more virtual users capable of executing all the Samplers you defined in the test plan.
Also given first user is capable of executing all the requests successfully and in time it looks like that your application gets overloaded hence cannot respond fast enough when the number of concurrent users reaches some "critical threshold", you can add listeners like Active Threads Over Time and Response Times vs Threads, this way you will be able to correlate increasing load with the increasing response time.
If also makes sense to collect:
Baseline health metrics of your application (CPU, RAM, Network, Disk usage, etc.), it can be done using JMeter PerfMon Plugin.
Lower level details like slowest methods, largest objects, heaviest database queries, etc. This form of information can be obtained using profiling tools specific to your application programming language(s).
Within my JMeter test plan I have transaction controller which contains multiple requests inside. There are no any timers between HTTP samplers and controller is configured to generate parent sample.
When I run that test most of the samples are OK but there are couple outliers for which response times for parent sample are enormous though HTTP response times inside controller are quite low. After checking I found that there are couple minute gaps between HTTP requests even no timers are configured.
E.g. first HTTP sampler started 04:08:34 and load time was 358 ms. Second sampler started 04:11:41 - so it took more than 3 minutes to start it. Then there were couple more similar requests and overall parent sample time is more than 6 minutes though sum of all HTTP sampler load times is less than 1 second.
Does anyone has an idea why it occasionally takes very long to start next HTTP request? Can it be caused by low resources (like memory) on the machines from which test is executed (it's distributed testing)?
If you don't have Timers or Flow Control Action sampler or Inter-Thread Communication Plugin JMeter executes samplers as fast as it can immediately one after another.
The only reasons I can think of is are:
lack of resources (CPU, RAM, Network, Disk) on JMeter side, I would recommend ensuring that JMeter has enough headroom to operate using i.e. JMeter PerfMon Plugin
the delay can be caused by PreProcessors, PostProcessors and Assertions so you could take a thread dump - it will provide information where exactly JMeter "hangs"
I am new to jmeter and scripted for login authentication. Project requirement is to see the load for 10K concurrent users.
Script is working fine but to enhance I need suggestions on how to do the following thigs:
How can I see that how much time/average time the server takes to load a page.
which thread grp (studied Ultimate thread group but it is not very clear to me), should be used to see the maximum load the server can sustain in a particular time, for that rampup time need to be adjusted (correct me if I am wrong).
Please tell how to adjust the rampup time with respect to users/waiting time etc., in short how to do incremental/proportional observation to see the server performance(there is no Gateway error etc)
If you're looking for your server capacity boundaries I would rather stick to "requests per second" rather than to "concurrent users" as users may work with different applications in a different way.
For instance, if it is image gallery - the majority of users will be browsing images and do this rather frequently, for instance request next image i.e. each 2 seconds. Given image load time 1 second it will be an image per 3 seconds - 20 images per minute. In this case 10000 users will create the load of 3333 requests per second.
If your site is articles collection, users will need some more time to read an article, i.e. 2 minutes. In that case 10000 users will create 83 requests per second load.
JMeter provides Constant Throughput Timer out of the box, you can set desired target throughput in requests per minute using it. And once you're already aware of JMeter Plugins project, it offers Throughput Shaping Timer - more advanced test element with extended functionality.
If you go "throughput" way, no matter which Thread Group you choose as the load will be orchestrated by aforementioned timers.
See What is the Relationship Between Users and Hits Per Second? article for more detailed explanation.
Once you design your test scenario run it in non-GUI mode (as JMeter's GUI is very resource intensive) as:
jmeter -n -t /path/to/your/testplan.jmx -l /path/to/resultsfile.jtl.
When the test finishes, open JMeter GUI, add Aggregate Report listener and inspect min/max/average response times per requests.