I have a Thread-Group that runs a set amount of time and control the throughput through a Constant Throughput Timer. So in theory, about the same number of Samples should be processed, regardless of the number of Threads deployed.
E.g. ignoring warmup/down the test runs for 10 min with a throughput of 5 Threads/s, therefore roughly 3000 Samples should be processed.
How does changing the number of Threads effect the test?
Ignoring the startup/shutdown period, will the results of the test differ if I have 1000 or 2000 Threads, if only 100 can be completed in a minute?
How can I work out the number of Threads required, given the throughput?
Constant Throughput Timer
Can only pause threads to match the given throughput value. It won't kick off more threads if you don't have enough defined at Thread Group level.
Is accurate enough on minute level. If your test lasts less than a minute - consider other options like playing with ramp-up.
For more information check out How to use JMeter's Throughput Constant Timer
Also it might be better and easier to use Throughput Shaping Timer available via JMeter Plugins project which extends CTT capabilities and provides easy understandable visual view of expected throughput
The number of threads will impact results if your server faces slowdowns.
If so and you don't have enough threads then you will not be injecting the expected load and end up with less transactions performed.
If you have more, then the target load will be injected potentially leading to server crash.
Finally be aware that if you don't make your test last enough, jmeter will potentially inject more than expected because it needs some time to reach the expected req/s , it will inject more at begining.
Related
I have used the normal thread group of 40 threads,180000 of throughput, and expected TPS is 3000.., But , Resulted only 2750 tps., How can I improve the TPS?
I have tried the concurrency thread group along with the shaping timer combination..
Can u pls guide me..,Thanks in Advance.
TPS or the throughput is controlled by sending the required number of requests to the target server in a unit time (seconds, minutes, hours, etc)
It is not easy or good practice to control the throughput by changing the number of threads. When the server gets slow the throughput gets low and vice-versa.
You need to introduce delays when the server responds faster and introduce more threads when the server responds slower. It is not easy to achieve the desired throughput by setting the thread count.
Please note that
throughput will be lower if the server is not capable of handling it, or if other timers or time-consuming test elements prevent it.
JMeter has a number of ways to achieve the desired throughput by introducing delays to the requests.
Using Timers
Constant Throughput Timer
Precise Throughput Timer
Througput Shaping Timer
In the above cases, you need to ensure a sufficient number of threads are available to generate the required throughputs. This solution is not efficient if a large number of threads are created than the required number of threads.
The best approach would be the test should create additional threads on-demand and introduce delays between requests as required.
In JMeter, you can achieve this with Concurrency Thread Group along with the Throughput Shaping Timer.
This video demonstrate the use of the controllers.
First of all your application must be able to handle such a load. I don't know what your 180000 of throughput, stanza means,if you put it to the Constant Throughput Timer be aware that:
it is precise enough on "minute" level
it can only "pause" JMeter to "limit" its throughput to the given value
In general you can reach out 3000 TPS with 40 users only if 1 user will make 75 requests per second which means that your application response time should be below 75 milliseconds.
If response time will be higher - you won't be able to reach 3000 requests per second with 40 users.
Also JMeter should be able to send requests fast enough, so make sure to follow JMeter Best Practices and if this is not sufficient consider using Distributed Testing
More information: What is the Relationship Between Users and Hits Per Second?
As per the Documentation - "Of course, if the server is not able to handle such a load, the throughput will be lower." How is this behavior handled and throughput calculated?
Suppose I set the target throughput as 1500/min but my system could handle only 1000/min, what kind of feedback behavior is expected by JMeter and how is the total throughput affected?
Will the JMeter still try to achieve 1500/min with higher error rate (for failed requests) or JMeter shoots the throughput to 1500/min but gradually comes back near to 1000/min based on certain factors like error rate? If yes, what are the factors determining this step-down, any insight/reference to understand this step-down behavior?
Either you do not have enough threads in the test plan to achieve your desired TPS or your application response time is higher.
The total number of requests you will be able to make through JMeter depends on your application response time.
So you will need at least 25 threads to achieve 1500/min if your average response time is 1 second (assuming there is only 1 request).
If throughput isn't increasing even after providing with enough threads but the response time increasing and some of your requests got failed, then this indicates the performance bottleneck!
JMeter cannot send requests faster than your application is capable of responding. If your application can only serve 1000 requests per minute - you will have 1000 requests per minute no matter how many threads you have in the Thread Group or how many requests per minute you specify in the Constant Throughput Timer.
Also be aware that:
Constant Throughput Timer can only pause the Threads, it will not do anything if the current amount of threads is not enough to reach/maintain the desired throughput
Constant Throughput Timer is accurate enough on minute level so consider choosing proper ramp-up settings otherwise you can stress your application during first minute of your test
I am trying to stress test my server.
To do so I am using Jmeter and here is my set up:
I use
my Setup
Thread: 1000
schedule for 3 mins
So as you see I keep going with 1000 thread for a period of 3 mins.
But when I look at the throughput I only get around 230 per second
results
So what should I do to increase the through put to for example 1000000 per second? How come increasing the thread which I assume means more load does not increase throughput?
According to JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
Throughput explicitly relies on the application response time. Looking into your results, the average response time is 3.5 seconds therefore you will not get more than 1000 / 3.5 = 285 requests per second
Theoretically you could use Throughput Shaping Timer and Concurrency Thread Group combination, this way JMeter will kick off extra threads if the current amount is not enough to reach/maintain the desired throughput, however looking into 8.5% error rate and maximum response time for your application > 2 minutes my expectation is that you will not be able to get more throughput because most probably your application is overloaded and cannot respond faster.
Throughput measures the number of transactions or requests that can be made in a given period of time. basically, it lists the number of requests server managed to serve in a given time period. Throughput value depends on lot of factors and maybe your application under test not able to cater the expected load.
So with 1000 threads, you can't expect a 1000 throughput.
It's up to you to find out how much throughput your application can handle. For that maybe you need to do different optimizations on your side like optimize your script, distribute load via JMeter execution, increase theard count,...etc
how does the delay between requests is computed. I have simple Jmeter script with following 4 components :
thread group - number of threads set to 10, reset other fields have default values.
Http Sampler - named as Thread - ${__threadNum} Req -${__counter(TRUE)}
hitting URL- google.co.in
Constant throughput Timer : Target throughput- 60, Calculate Throughput on -All Active threads.
View result in table listener.
Summarized test results are below :
Please explain how the delay between the requests and number of requests to be created are computed . Please Explain other field values of " Calculate Throughput on ".
Also If i have ultimate Thread group instead of thread group, Have hold for load set to some value, so in that case how the number of requests to be sent during hold load time period are calculated?
Constant Throughput Timer pauses active threads to reach throughput level you define.
Remember 2 things:
Throughput Timer isn't able to create threads, it can only slow them down so make sure that you provide enough threads to reach desired target throughput
Throughput Timer is quite accurate on "minute" level, you need to "wait" for it to start working as expected for 60 seconds.
See How to use JMeter's Throughput Constant Timer for more details.
In some cases for some people it's more convenient to use Throughput Shaping Timer which comes with JMeter Plugins, you can configure target throughput providing multiple steps and seeing the impact on a graph.
From JMeter Cookbook by Bayo Erinle, The goal of the Constant Throughput Timer component is to get your test plan samples as
close as possible to a specified desired throughput. It achieves this by introducing variable
pauses to the test plan in such a manner that will keep numbers as close as possible to
the desired throughput. That said, throughput will be lowered if the server resources of the
system under test can't handle the load. Also, other elements (for example, other timers,
the number of specified threads, and so on) within the test plan can affect attaining the
desired throughput.
I need to create a load test in which there's a maximum of 4 concurrent users per second. Then I need to repeat this for an hour.
Is there any way to achieve this in JMeter?
I have tried using this configuration:
Number of Threads: 4
Ramp-up period: 1
Loop count: Forever
Duration:3600
And to make sure that this run exactly for an hour I have also used a Runtime Controller with the same "3600" runtime value.
But these creates more concurrent users per second than what I need and besides it's also unlikely to have that many concurrent users as user usually take so time to think before they do something and this wasn't reflected in my configuration so also tried using Constant Throughput timer , but this only confused me more.
I'm sure I need to use the Constant Throughput Timer, but I don't have the faintest idea of how to do it in my case.
If a use the Constant Throughput Timer, how should I configure the Thread Properties (Number of threads, ramp-up period,loop count, schedule) ?
Do I really need to use a Runtime Controller, to ensure that the test plan runs exactly for an hour or is enough to specify a duration value of 3600?
Constant throughput timer allows you maintain throughput of your sever. (req/sec)
here requests are samplers. Threads are users/clients which are requesting server using samplers.
I hope this have cleared distinction between request and user i.e. sampler and thread.
Now Constant Throughput Timer works on sampler per minute basis. so for your requirement of 4 req per sec. it becomes 240 requests per minute.
and you want to maintain this throughput among all threads (if you have multiple thread groups) or among a single thread group if you have single thread group.
So JMeter engine will start all threads with given ramp-up but will allow only 4 samplers to executes at a given seconds i.e. only 4 request per second.
Thus throughput is achieved among all threads and you can continue this for a specified time using either ways,
Test duration
Runtime controller
I hope it cleared your confusion and solved the issue.