What's the difference between Number of Threads and the Constant Throughput Timer?
Before I heard of the Constant Throughput Timer, I thought that if I wanted to simulate 21 requests per minute , the following configuration was enough :
but now that I found this post, I'm not so sure.
This is my new configuration now that I make use of the Constant Throughput Timer :
Does the Constant Throughput Timer override the parameters specify in Thread Group (Number of threads,Ramp-up period, etc) ?
Why even making use of the Constant Throughput Timer I sometimes get a number of requests higher than the value specify in the Target Throughput (21 in the above image)
If a indicate a value of 21 in the Target Throughput, isn't this constant supposed to guarantee a maximum value of 21 requests? If so why do I sometimes get more requests than indicated.
All your comments and ideas will be appreciated.
P.S. I'm also using a Runtime controller.
No, it regulates the number of requests going to & response received from the server per minute.
As stated in the Jmeter user guide "the throughput value should not be changed too often during a test - it will take a while for the new value to take effect."
http://jmeter.apache.org/usermanual/component_reference.html#Constant_Throughput_Timer
The issue you have shown above will always happen in the first minute of the test run.
hope this will help.
Related
I have used the normal thread group of 40 threads,180000 of throughput, and expected TPS is 3000.., But , Resulted only 2750 tps., How can I improve the TPS?
I have tried the concurrency thread group along with the shaping timer combination..
Can u pls guide me..,Thanks in Advance.
TPS or the throughput is controlled by sending the required number of requests to the target server in a unit time (seconds, minutes, hours, etc)
It is not easy or good practice to control the throughput by changing the number of threads. When the server gets slow the throughput gets low and vice-versa.
You need to introduce delays when the server responds faster and introduce more threads when the server responds slower. It is not easy to achieve the desired throughput by setting the thread count.
Please note that
throughput will be lower if the server is not capable of handling it, or if other timers or time-consuming test elements prevent it.
JMeter has a number of ways to achieve the desired throughput by introducing delays to the requests.
Using Timers
Constant Throughput Timer
Precise Throughput Timer
Througput Shaping Timer
In the above cases, you need to ensure a sufficient number of threads are available to generate the required throughputs. This solution is not efficient if a large number of threads are created than the required number of threads.
The best approach would be the test should create additional threads on-demand and introduce delays between requests as required.
In JMeter, you can achieve this with Concurrency Thread Group along with the Throughput Shaping Timer.
This video demonstrate the use of the controllers.
First of all your application must be able to handle such a load. I don't know what your 180000 of throughput, stanza means,if you put it to the Constant Throughput Timer be aware that:
it is precise enough on "minute" level
it can only "pause" JMeter to "limit" its throughput to the given value
In general you can reach out 3000 TPS with 40 users only if 1 user will make 75 requests per second which means that your application response time should be below 75 milliseconds.
If response time will be higher - you won't be able to reach 3000 requests per second with 40 users.
Also JMeter should be able to send requests fast enough, so make sure to follow JMeter Best Practices and if this is not sufficient consider using Distributed Testing
More information: What is the Relationship Between Users and Hits Per Second?
As per the Documentation - "Of course, if the server is not able to handle such a load, the throughput will be lower." How is this behavior handled and throughput calculated?
Suppose I set the target throughput as 1500/min but my system could handle only 1000/min, what kind of feedback behavior is expected by JMeter and how is the total throughput affected?
Will the JMeter still try to achieve 1500/min with higher error rate (for failed requests) or JMeter shoots the throughput to 1500/min but gradually comes back near to 1000/min based on certain factors like error rate? If yes, what are the factors determining this step-down, any insight/reference to understand this step-down behavior?
Either you do not have enough threads in the test plan to achieve your desired TPS or your application response time is higher.
The total number of requests you will be able to make through JMeter depends on your application response time.
So you will need at least 25 threads to achieve 1500/min if your average response time is 1 second (assuming there is only 1 request).
If throughput isn't increasing even after providing with enough threads but the response time increasing and some of your requests got failed, then this indicates the performance bottleneck!
JMeter cannot send requests faster than your application is capable of responding. If your application can only serve 1000 requests per minute - you will have 1000 requests per minute no matter how many threads you have in the Thread Group or how many requests per minute you specify in the Constant Throughput Timer.
Also be aware that:
Constant Throughput Timer can only pause the Threads, it will not do anything if the current amount of threads is not enough to reach/maintain the desired throughput
Constant Throughput Timer is accurate enough on minute level so consider choosing proper ramp-up settings otherwise you can stress your application during first minute of your test
How can I find out maximum throughput my application can achieve within sla of 1 sec as response time.i.e response time should be within 1 sec
Currently I am using throughput controller and setting up the required throughput per minute, but I need to know max throughput my application can deliver with average response time less then 1 sec.
Using Thread Group, set Number of Threads (users) to some big number and set rump-up period to e.g.: 600s (10min)
Jmeter will slowly increase throughput and you can monitor Max response time in Summary report (column Max) - when it starts reaching 1s, you can read your desired Throughput (in Throughput column) Then I recommend set this number to Number of threads, decrease rump-up and run test for couple of hours.
Avoid using sleeps in your loop in JMeter. Beware of cache at application side. Monitor Faults in Error columns and check system health on both sides (CPU,mem,IO,Netw,...)
Please note that Throughput Controller does not control the throughput.
Following is from the JMeter official site.
This controller is badly named, as it does not control throughput.
Please refer to the Constant Throughput Timer for an element that can
be used to adjust the throughput.
You may use plugin Throughput Shaping Timer if you want to have controlled throughput changing with the time.
Change the threads (virtual users) and throughput to check the maximum through that can be handled by the server.
Use Duration Assertion to check the response time of each desired sampler.
Just don't limit JMeter's throughput.
Start with 1 thread (virtual user) and gradually increase the load until you see response times exceeding 1 second (you can use i.e. AutoStop Listener plugin for automatically stopping the test when average response becomes higher than 1 second)
Once done you can look into Transactions per Second and Server Hits per Second, it will give you the number you're looking for.
how does the delay between requests is computed. I have simple Jmeter script with following 4 components :
thread group - number of threads set to 10, reset other fields have default values.
Http Sampler - named as Thread - ${__threadNum} Req -${__counter(TRUE)}
hitting URL- google.co.in
Constant throughput Timer : Target throughput- 60, Calculate Throughput on -All Active threads.
View result in table listener.
Summarized test results are below :
Please explain how the delay between the requests and number of requests to be created are computed . Please Explain other field values of " Calculate Throughput on ".
Also If i have ultimate Thread group instead of thread group, Have hold for load set to some value, so in that case how the number of requests to be sent during hold load time period are calculated?
Constant Throughput Timer pauses active threads to reach throughput level you define.
Remember 2 things:
Throughput Timer isn't able to create threads, it can only slow them down so make sure that you provide enough threads to reach desired target throughput
Throughput Timer is quite accurate on "minute" level, you need to "wait" for it to start working as expected for 60 seconds.
See How to use JMeter's Throughput Constant Timer for more details.
In some cases for some people it's more convenient to use Throughput Shaping Timer which comes with JMeter Plugins, you can configure target throughput providing multiple steps and seeing the impact on a graph.
From JMeter Cookbook by Bayo Erinle, The goal of the Constant Throughput Timer component is to get your test plan samples as
close as possible to a specified desired throughput. It achieves this by introducing variable
pauses to the test plan in such a manner that will keep numbers as close as possible to
the desired throughput. That said, throughput will be lowered if the server resources of the
system under test can't handle the load. Also, other elements (for example, other timers,
the number of specified threads, and so on) within the test plan can affect attaining the
desired throughput.
I need to create a load test in which there's a maximum of 4 concurrent users per second. Then I need to repeat this for an hour.
Is there any way to achieve this in JMeter?
I have tried using this configuration:
Number of Threads: 4
Ramp-up period: 1
Loop count: Forever
Duration:3600
And to make sure that this run exactly for an hour I have also used a Runtime Controller with the same "3600" runtime value.
But these creates more concurrent users per second than what I need and besides it's also unlikely to have that many concurrent users as user usually take so time to think before they do something and this wasn't reflected in my configuration so also tried using Constant Throughput timer , but this only confused me more.
I'm sure I need to use the Constant Throughput Timer, but I don't have the faintest idea of how to do it in my case.
If a use the Constant Throughput Timer, how should I configure the Thread Properties (Number of threads, ramp-up period,loop count, schedule) ?
Do I really need to use a Runtime Controller, to ensure that the test plan runs exactly for an hour or is enough to specify a duration value of 3600?
Constant throughput timer allows you maintain throughput of your sever. (req/sec)
here requests are samplers. Threads are users/clients which are requesting server using samplers.
I hope this have cleared distinction between request and user i.e. sampler and thread.
Now Constant Throughput Timer works on sampler per minute basis. so for your requirement of 4 req per sec. it becomes 240 requests per minute.
and you want to maintain this throughput among all threads (if you have multiple thread groups) or among a single thread group if you have single thread group.
So JMeter engine will start all threads with given ramp-up but will allow only 4 samplers to executes at a given seconds i.e. only 4 request per second.
Thus throughput is achieved among all threads and you can continue this for a specified time using either ways,
Test duration
Runtime controller
I hope it cleared your confusion and solved the issue.