I have a test plan where I have to pick 100 users from a CSV file. If I give threads count as 500 and loop count as 1, then I can see 490 are failed and only 10 are passed. But at the same time, if I give threads as 5 and loop count as 100, all the tests are passed. Aren't these same where the total number of requests are 500?
Is this because 500 threads and 5 threads?
If you start 500 threads, By default, the file is only opened once, and each thread will use a different line from the file. However the order in which lines are passed to threads depends on the order in which you execute, which may vary between iterations.
Change Your Jmeter CSV-DataSet-Config property "Recycle on EOF - True"
; Your Issue will solve
Hope This Helps!
You get different results because you apply different load pattern. JMeter acts as follows:
Each Thread Group kicks off threads (virtual users) within the ramp-up period
Each Thread starts executing Samplers upside down (or according to the Logic Controllers)
When the thread doesn't have any more samplers to execute or loops to interate it is being shut down
So reasons could be in:
Your application isn't capable of handling 500 users. Checks its logs for error details. If there are no specific errors it might be lack of hardware resources, re-run your test with JMeter PerfMon Plugin Telemetry to check the impact of the increasing load onto hardware resources consumption
Your application underlying components is not suitable for high loads. Some application and database servers come with connection limitations, low memory allocation, etc. in other words configuration suitable for development and debugging. Production deployment assumes totally different configuration therefore it needs to be inspected and amended
Your JMeter instance cannot create the required load. Like in point 2 JMeter default configuration is good for tests development, however when it comes to running load tests you need to mind some important points:
Increase JVM Heap size for JMeter
Run your test in non-GUI mode
Disable all the listeners during test run
See Reducing resource requirements chapter of JMeter's User Manual for more information
Related
I need to run a load test of 25 threads, what will be the most efficient configuration to use? (ramp-up period....). I ran the load test with the below configuration and some of the threads failed but pass if I just ran the script individually.
Your configuration will mean that:
JMeter start with 1 thread and add another thread each 4 seconds
Once started each thread will begin to execute Samplers upside down (or according to the logic controllers)
When the thread executes the last sampler it will be shut down
When the last thread executes the last sampler the test ends
Depending on the number of samplers and application response time you may or may not achieve 25 users concurrency, you might want to check the actual number of concurrent users using Active Threads Over Time listener
If you want to make sure to have 25 online users set "Loop Count" to Infinite and "Specify Thread Lifetime" duration to be more than your ramp-up period. See JMeter Test Results: Why the Actual Users Number is Lower than Expected article for more details.
With regards to the failures - we cannot state anything meaningful without seeing request and response details, make sure to save them using i.e. View Results Tree listener and inspect response body for the failed requests
This is regarding JMeter
I have a thread group where number of threads is 100. I get response for all the 100 requests in say 3 minutes. However when I generate html report from the .jtl file, it still shows around 50 threads as active even after 6 to 7 minutes.
I was expecting that the active threads would also die out once the response is received. Should we kill/release threads or do we have a similar option in JMeter which I am missing. Can someone advise please.
If you have only 1 loop in Thread Group - each thread (virtual user) should be terminated once it finishes execution of all the Samplers, assuming you don't have i.e. Loop Controller or While Controller or Runtime Controller which may (and will) "hold" the threads alive.
Also make sure to follow recommendations from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article, in particular:
Run your test in non-GUI mode
Remove all the Listeners from the Test plan
If the issue persists - check jmeter.log file and take the thread dump to identify what exactly JMeter threads are doing.
While understanding concept of Concurrent thread and ultimate thread group, i am confused to understand result of summary/aggregate report while running concurrent thread or ultimate thread group .For example if i have 200 user and ramp up time 60 sec then i didn't see all sampler request as 200 sample after completing execution successfully but only few sampler request have 200 sample. when i use normal thread group then i always got thread count same for each sampler request after completing execution .
for realistic load testing with more user , could you please suggest me which thread group should prefer.
Could you please provide valuable guidance with some valuable link/book and also share me standard performance bench mark criteria or key parameter detail while doing load testing.(if any bench load parameter value does not meet standard then we can say that there is a performance issue)
Thanks for giving valuable time in advance.
Thanks
amit
This is due to the fact that:
Your application response time is too high
Your test duration is too low
For example I can see response times > 80 seconds:
it means that if a single virtual user has cumulative response time for 2 samplers > 160 seconds and the test duration is 120 seconds it will not be able to execute all the requests. Just increase your test duration to be i.e. 10 minutes and you should see more virtual users capable of executing all the Samplers you defined in the test plan.
Also given first user is capable of executing all the requests successfully and in time it looks like that your application gets overloaded hence cannot respond fast enough when the number of concurrent users reaches some "critical threshold", you can add listeners like Active Threads Over Time and Response Times vs Threads, this way you will be able to correlate increasing load with the increasing response time.
If also makes sense to collect:
Baseline health metrics of your application (CPU, RAM, Network, Disk usage, etc.), it can be done using JMeter PerfMon Plugin.
Lower level details like slowest methods, largest objects, heaviest database queries, etc. This form of information can be obtained using profiling tools specific to your application programming language(s).
We are creating a new hosted server for one of our APIs on managed containers (Kubernetes) and we're trying to validate that it can handle at least the same amount of traffic load requests.
We've started with one of the APIs, where we would need to handle at least 140k requests per minute, all endpoints combined.
To verify this, I created a simple JMeter test as follows:
-Test Plan
---Thread Group Endpoint1
-----HTTP Request -> a GET request with query params for /path1
---Thread Group Endpoint2
-----HTTP Request -> a GET request with query params for /path2
For a local test, I used the following setup:
Thread Groups Endpoint1 and Endpoint2 are set to 200 threads (users), ramp-up period of 1s, loop count = forever and duration 60s.
Using a Summary Report listener when running the test gets me a total of ~9300 # Samples.
Using this approach, is it safe to just increase the number of threads (users) for the Thread Groups until I reach the desired 140k requests per minute?
Note: I only used JMeter a little before, so I'm aware that the entire approach may be wrong, therefore any suggestions and steering to the right path are more than welcomed.
Your approach is viable as long as it represents real-life application usage. If it has 2 endpoints with equally/evenly distributed load - your setup is just fine. If there are more endpoints and some of them are used more than the others - consider defining the workload correspondingly either using different Thread Groups or other distribution mechanism such as Throughput Controller
Increasing the number of threads is also fine, however consider increasing the load gradually, to wit increase ramp-up time so your test could have:
Arrivals phase
Time to hold the load
Ramp-down phase
This way you will be able to correlate various metrics like increasing response time, throughput, number of errors, etc. with the increasing load. Also you will be able to state what was the number of threads/requests per second when the system reached saturation point/breaking point and does it recover when the load gets back.
Also make sure you're following JMeter Best Practices as 2300/2500 requests per second is not something JMeter can support out of the box and you will need to do some tuning, at least increase JVM Heap size allocated to JMeter.
You may not be able to achieve the desired 140k requests per minute using a single Jmeter Machine, in that case you'll need Distributed Load Testing approach here.
refer: http://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.html
Also keeping the ramp-up period of 1 second will lead to spike and unrealistic load in the system which will not give proper result unless you've pre-warmed your server, you should gradually increase the load as per real/estimated traffic pattern.
I would like to make Recovery testing to server, I can simulate connection pool full with high load and thus failed in timeout to get connection.
I need to start my test with high load, reach a failure and decrease load gradually,
JMeter can't change it dynamically so I search for plugins and Concurrency Thread Group is doing the opposite,
Increase it gradually, is there a way to make it decrease the value?
Is my only option is to start a new execution with decreased load?
Recovery testing is done in order to check how fast and better the application can recover after it has gone through any type of crash or hardware failure etc.
It can be done using vanilla JMeter without any plugins, JMeter acts as follows:
All threads defined in the Thread Group are started within specified ramp-up period (here you can gradually increase the load)
All threads start executing samplers upside down (or according to the Logic Controllers)
When thread doesn't have any samplers to execute and/or loops to iterate - it's being shut down.
So given you gradually start your requests and thread group is not configured to run forever - the ramp-down should be pretty much the same as ramp-up. You can "help" JMeter a little bit using Gaussian Random Timer, check out How to Ramp Down in JMeter guide for more details.
I found a jmeter plugin jp#gc - Ultimate Thread Group which I can start with high load by adding first row with high thread count and then decrease thread count by adding second (or more) row .