I'm having trouble achieving the desired 'requests per second' using a combination of 'bzm - Concurrency Thread Group' and 'Constant Throughput Timer' in JMeter.
Structure
My test is structured as follows:
Thread Group 1
Constant Throughput Timer
HTTP Request
Thread Group 2
Constant Throughput Timer
HTTP Request
Thread Group 3
Constant Throughput Timer
HTTP Request
Thread Group 4
Constant Throughput Timer
HTTP Request
Numbers
I am wanting to achieve 160 concurrent users for each of the Thread Groups.
Thread Group 1 I am aiming to make 350 requests per second, the others are aiming for 160 requests per second.
The Thread Groups have 160 as the 'Target Concurrency'.
Constant Throughput Timer has 21000 for Thread Group 1 (350 rps*60 = 21000 per minute).
Constant Throughput Timer has 9600 for the others (160 rps*60= 9600 per minute).
All of them have 'Calculate Throughput based on: this thread only'
I'm currently only getting a throughput of around 80 rps for Thread Groups 2-4 and less than 10 for Thread Group 1.
I've allowed JMeter to use 4GB of RAM and I'm not using all of this during the test. When I remove all Throughput Timers, I can achieve up to 14k rps.
Am I missing something? Is there a better way to achieve my desired throughput?
The reasons of not achieving desired number of requests per second could be in:
You simply don't have enough threads in thread group
JMeter cannot send requests fast enough due to lack of resources on its side
Application under test cannot respond fast enough
With regards to point 1 be aware that Constant Throughput Timer can only pause threads in order to limit JMeter's throughput to the desired number of requests per minute.
Given you use Concurrency Thread Group it makes more sense to switch to Throughput Shaping Timer which can be connected to the Concurrency Thread Group via feedback loop so JMeter will be able to kick off extra threads to achieve the desired RPS rate.
You can install Throughput Shaping Timer using JMeter Plugins Manager
With regards to point 2: make sure you're following JMeter Best Practices and monitoring baseline OS health metrics on machine where JMeter is running, if there is lack of CPU or RAM you might need to consider Distributed Testing
With regards to point 3: the same approach as for point 2 + use profiling tools to see where application spends the most of time
Related
I have a JMeter test plan with a concurrency thread group and a throughput shaping timer.
below is a screenshot of the test plan.
The post requests are two requests which are sent one after another for a device. there should be 200 transactions/requests needs to sent per sec which I am controlling by target concurrency in concurrency thread group, start RPS and end RPS in the throughput shaping timer.
the total TPS should be 200 ( sum of the first and second requests in the test plan) and the device would be different in each thread. can this be achieved with the above test plan. I see that post-event 2( second post-event in the test plan) are not sent for some of the devices towards the end of the test.
You should be able to achieve the target throughput (RPS) with the Throughput Shaping Timer with the feedback function in the Concurrency Thread Group.
RPS can be set on the Throughput Shaping Timer
This short video demonstrates the use of various options available in JMeter for Throughput Controlling.
Yes, it's possible, but be aware that "Concurrency" in the Concurrency Thread Group is the number of threads (virtual users) and the number of requests per second will be equal to the number of threads only if your application response time is 1 second.
If it's higher - you will need to proportionally increase the "Concurrency" in the Concurrency Thread Group.
More information: What is the Relationship Between Users and Hits Per Second?
You can connect Concurrency Thread Group with the Throughput Shaping Timer via Feedback Function, this way JMeter will be able to kick off extra threads if the current amount is not sufficient in order to reach/maintain the desired amount of requests per second.
Also re-consider using listeners in your test plan, they don't add any value but just consume valuable resources, you should be running JMeter tests in non-GUI mode with all listeners enabled/deleted and once test finishes open the .jtl results file with the listener of your choice or generate HTML Reporting Dashboard
I have tried with concurrent users thread group pairing with throughput timer , but the transactions were low , Is there any other way to achieve the above scenario
#dmitri
I happened to use this function ${__tstFeedback(throughput-timer,25,2500,100)} in my concurrent thread group still I what could see is not more than 200 transaction/sec
Concurency thread group setup
Throughput shaping timer
Transactions per sec
I always see this additional request added to my summary and aggregate reports results Concurrency Thread Group-ThreadStarter:
Concurrency Thread Group-ThreadStarter: 15034 27152 314 67905 17924.575009281336 0.5485566050286018 27.972210128343956 101.83185588876516 3.662355576998932 3727.836304376746
It might be a valid scenario depending on your application response time.
In order to be able to achieve 500 requests/second with 25 users each user needs to be able to execute 20 requests per second which means that your application response time needs to be 50 milliseconds or less.
If response time is above 50 milliseconds the number of requests per second will reduce proportionally.
If you have more than 1 request in your transaction - the response time constraint changes correspondingly like 2 requests - 25 milliseconds max, etc.
JMeter's Constant Throughput Timer can only capable of pausing the threads to limit the number of transactions per minute to the given value, if the current amount is not enough - it won't kick off extra threads. So it's your job to provide sufficient number of threads in the Thread Group
As the alternative you can consider going for Throughput Shaping Timer and Concurrency Thread Group combination, they can be connected via Feedback Function so if current number of threads is not enough to reach/maintain the desired throughput - more threads will be launched
I am so much confused with Jmeter test configuration. I found lots of answers but non those clear my doubt. My thread group contain 10 threads and all are in active state. I want to hit 20 requests per second to server. Load need to last for 60 seconds.
I put
thread count : 20
Ramp Up Period : 1
Loop Count: 60
Is it correct method to hit 20 requests/second last for 60 seconds?
For Request Per Second we should not calculate manually, instead leave the headache to JMeter. Just make sure, to have enough Threads to reach that TPS and JMeter will control the pacing. If the Threads are less and Request Per Second is more, than JMeter cant to anything and RPS wont be met.
Use Throughput Shaping Timer to achieve Request Per Second Throughput Shaping Timer
How Many Threads I Need To Produce Desired RPS?
Threads pool size can be calculated like RPS * <max response time> / 1000. The more rate desired the more threads you will need. The more response time service have the more threads you will need.
For example, if your service response time may be 2.5sec and target rps is 1230, you have to have 1230 * 2500 / 1000 = 3075 threads.
It is better to have some threads overhead to handle unexpected response time spikes. But beware of too much overhead, it will lead to "final spike".
Use Following settings in Thread Group and Throughput Shaping Timer:
Increase Number of Threads if 20 TPS is not met.
You can use stepping thread group or concurrency thread group instead of basic thread group to achieve your desired results.
https://jmeter-plugins.org/wiki/ConcurrencyThreadGroup/
Your configuration will be correct only if:
You have only one Sampler
It's response time is equal to 1 second
Correct configuration would be something like:
The best approach would be going for Concurrency Thread Group and Throughput Shaping Timer combination. They can be connected together via Feedback Function allowing the Concurrency Thread Group to kick off more threads if current amount is not enough to conduct the required load.
Our requirement is to execute
1 user/thread, 10 requests/sec, Total requests = 10,000
With current Thread Properties not able to achieve.
Only one user/thread send 10 request/sec and total should be 10000 requests. Any other way to achieve this in Jmeter ?
Is following approach is correct ? We used Loop controller. So each request will repeat 10 times
You will be able to achieve 10 requests/second with 1 virtual user only if your response time will be 100 milliseconds. If response time will be more than 100 milliseconds - you will not be able to reach the desired load.
If your application fails to respond in 100 milliseconds time frame - most probably you've found a performance bottleneck and you can report this to your application developers.
If you have some time to investigate the issue you can try to provide more information, i.e. what is the average response time, what are minimum, maximum, quantiles, the actual number of requests per second, etc, all this information can be retrieved using Aggregate Report listener.
Normally when people are looking for the answer if the application under test can support X requests per second they use > 1 virtual user as load test should represent real life usage of the system under test and 1 thread (virtual user) isn't normally associated with performance testing and derivatives.
So probably you should reconsider your approach to testing and try increasing the number of threads (virtual users). The throughput can be controlled using Precise Throughput Timer or Constant Throughput Timer. However be aware that the above timers can only pause JMeter to slow it down to the desired throughput. Another approach is using Concurrency Thread Group and Throughput Shaping Timer combination, they can be connected using Feedback Function so JMeter will kick off extra threads in order to reach/maintain the desired number of requests per second.
You can install Concurrency Thread Group and Throughput Shaping Timer using JMeter Plugins Manager
I have recorder a simple flow using jmeter. I have multiple http requests under one thread group. I am trying to run this thread group for 50 concurrent users in blazemeter. But when ever i try to run the jmx file i can see that htis/s is around 3 sec. Though my rampup time is 0 & thread is 50. My understanding is if the users are concurrent & each user (thread) have 3 requests to make, then minimum hits/s should be 50.
I have added a constant throughput timer at the bottom of the thread group & a throughput controller. In the timer target throughput is set to 300 & in the throughput controller throughput is set to 100 with percent execution.
How do i achieve maximum hits/s in blazemeter? Or am i missing something here. Thanks in advance.
I think that you're misusing Constant Throughput Timer.
Throughput of 300 requests per minute gives 300/60 = 5 requests per second
Constant Throughput Timer works on minute level so make sure that your tests lasts longer than 1 minute
Each JMeter thread representing virtual user executes nested samplers upside down, when thread has nothing more to do it is being shut down. Make sure that you have enough job for threads and if not - make sure that you provide enough loops (see point 2 as well)
Constant Throughput Timer cannot kick off any new threads, it is capable of throttling (pausing) current active threads to limit load to throughput defined. So try increasing threads count.
For more information refer to How to use JMeter's Throughput Constant Timer guide
You may also like to use Throughput Shaping Timer instead, it provides a little bit more flexibility than JMeter's CTT.