I am conducting a performance test (TPS) using jmeter.
I am requesting about 10,000 TPS, but the following two results are different.
(Position that 10,000 TPS responds normally)
1000 thread x 600 target throughput(in samples per minute)
100 thread x 6000 target throughput(in samples per minute)
I think the two results should be the same, but why is the response time delayed as the thread increases?
I think the two results should be the same - why they would be the same?
Let's imagine your system has fixed response time of 1 second, in that case:
With 1000 threads you will get 1000 requests per second and you can limit the throughput to 10 requests per second using the Constant Throughput Timer
With 100 threads you will get 100 requests per second, no limiting is required
And what if response time is 2 seconds?
With 1000 threads you will get 500 requests per second
With 100 threads you will get 50 requests per second
Constant Throughput Timer:
acts precise enough on "minute" scale, if your test lasts less than minute it might not apply the throughput
can only pause the threads to limit the throughput (requests per minute) to the desired value. If current number of threads is not enough in order to conduct the required load - the time won't have any effect.
If you want to send requests at the rate of 10000 TPS it worth considering going for the Throughput Shaping Timer and Concurrency Thread Group combination connected via the Feedback Function in this case JMeter will be able to kick off extra threads if current number is not sufficient.
But also be informed that:
JMeter should be able to start as many threads as needed to send 10000 TPS so make sure to follow JMeter Best Practices or even consider going for Distributed Testing Mode
Application needs to be able to handle the load and respond fast enough, JMeter waits for the previous response before starting the new request so if application is able to serve i.e. 5000 requests per second only you won't be able to reach 10000 by any means
Related
In my test plan , I only have one Thread Group.
Thread Group Screenshot
I have CSV file with 10 users and 15 API requests in Loop Controller.
Loop Controller Screenshot
All of my requests are passing, An average of 55 requests/second were being sent but I want to know the maximum capacity of the server.I need to sent 150 requests per second, So I added Constant Throughput Timer with Target Throughput 9000 Constant Throughput Timer Screenshot .But it is still sending an average of 55 requests/ second. Here is the screenshot of command line run output. Output
Can anyone help me understand what I am doing wrong here ? Which values should I change in order to acheive 150 requests per second ?
JMeter sends request, WAIT for answer, send another request
If you add timer, it basically add some sleep to this -> you can't increase throughput, only decrease. Check manual https://jmeter.apache.org/usermanual/component_reference.html#Constant_Throughput_Timer
You should check your server - Load, CPU, Memory, IOs - disks, network ...
Check your JMeter machine as well
Are the answers from server alright? Do you check e.g. HTTP 200 OK?
You can try add more ramp up time, inscrease/decrease user count, but first check your server!
You're trying to reach 150 requests per second with each thread, you need to amend at least this bit:
You're trying to reach 150 requests per second with 10 users which means that each user needs to execute 15 requests per second. It's only possible if response time of your application is 66 milliseconds or less and the average response time of your application is around 200 milliseconds so it explains the actual result
JMeter's Constant Throughput Timer can only pause the threads in order to limit requests execution rate to the desired value, it won't kick off extra threads if current amount is not sufficient.
So the options are in:
Increase the number of threads in the Thread Group to i.e. 50 (you may need even more)
Or consider switching to Throughput Shaping Timer and Concurrency Thread Group combination, the Throughput Shaping Timer is more "precise" and it can be connected to the Concurrency Thread Group via Feedback Function so you will be able to kick off more threads to reach and maintain the desired throughput
I am given a task in which I have to test my app for an hour with 300 TPS.
I have added my HTTP req in Jmeter and I need to configure below parameters to achieve my target
target concurrency
Ramp up time
Ramp-up steps
Hold target
thread iteration limit
I could guess by name few of these but still I am not quiet sure how to work around it to achieve my target. I am new to Jmeter can any one please help.
I assume you're talking about Concurrency Thread Group so answers are:
target concurrency - we don't know, it depends on your application response time, i.e. if response time is 1 second - go for 300, if response time is 2 seconds - go for 600, etc. If you don't know the response time you can put "sufficient" number of concurrent threads, for example 1000 and use Throughput Shaping Timer for limiting the load to 300 TPS
Ramp up time - again, we don't know your requirements. I can only recommend increasing the load gradually so you could correlate increasing load with increasing response time or number of errors so for example start with 1 TPS and increase the load to 300 TPS in i.e. 15 minutes and leave it running for 1 hour
Ramp-up steps - again, we don't know your requirements, start with above recommendation and adjust ramp-up period up or down as needed
Hold target - at least 60 minutes or 3600 seconds plus any ramp-up time
thread iteration limit - leave blank otherwise your test may end earlier than you expect
Example Concurrency Thread Group combination:
the values given in Schedule Feedback Function are for reference only, the example means that the Concurrency Thread Group will start 1000 threads initially with maximum of 1500 threads and have 300 spare threads in pool, hopefully it will be sufficient to produce 300 TPS load which is set by the Throughput Shaping Timer
I am trying to stress test my server.
To do so I am using Jmeter and here is my set up:
I use
my Setup
Thread: 1000
schedule for 3 mins
So as you see I keep going with 1000 thread for a period of 3 mins.
But when I look at the throughput I only get around 230 per second
results
So what should I do to increase the through put to for example 1000000 per second? How come increasing the thread which I assume means more load does not increase throughput?
According to JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
Throughput explicitly relies on the application response time. Looking into your results, the average response time is 3.5 seconds therefore you will not get more than 1000 / 3.5 = 285 requests per second
Theoretically you could use Throughput Shaping Timer and Concurrency Thread Group combination, this way JMeter will kick off extra threads if the current amount is not enough to reach/maintain the desired throughput, however looking into 8.5% error rate and maximum response time for your application > 2 minutes my expectation is that you will not be able to get more throughput because most probably your application is overloaded and cannot respond faster.
Throughput measures the number of transactions or requests that can be made in a given period of time. basically, it lists the number of requests server managed to serve in a given time period. Throughput value depends on lot of factors and maybe your application under test not able to cater the expected load.
So with 1000 threads, you can't expect a 1000 throughput.
It's up to you to find out how much throughput your application can handle. For that maybe you need to do different optimizations on your side like optimize your script, distribute load via JMeter execution, increase theard count,...etc
I am so much confused with Jmeter test configuration. I found lots of answers but non those clear my doubt. My thread group contain 10 threads and all are in active state. I want to hit 20 requests per second to server. Load need to last for 60 seconds.
I put
thread count : 20
Ramp Up Period : 1
Loop Count: 60
Is it correct method to hit 20 requests/second last for 60 seconds?
For Request Per Second we should not calculate manually, instead leave the headache to JMeter. Just make sure, to have enough Threads to reach that TPS and JMeter will control the pacing. If the Threads are less and Request Per Second is more, than JMeter cant to anything and RPS wont be met.
Use Throughput Shaping Timer to achieve Request Per Second Throughput Shaping Timer
How Many Threads I Need To Produce Desired RPS?
Threads pool size can be calculated like RPS * <max response time> / 1000. The more rate desired the more threads you will need. The more response time service have the more threads you will need.
For example, if your service response time may be 2.5sec and target rps is 1230, you have to have 1230 * 2500 / 1000 = 3075 threads.
It is better to have some threads overhead to handle unexpected response time spikes. But beware of too much overhead, it will lead to "final spike".
Use Following settings in Thread Group and Throughput Shaping Timer:
Increase Number of Threads if 20 TPS is not met.
You can use stepping thread group or concurrency thread group instead of basic thread group to achieve your desired results.
https://jmeter-plugins.org/wiki/ConcurrencyThreadGroup/
Your configuration will be correct only if:
You have only one Sampler
It's response time is equal to 1 second
Correct configuration would be something like:
The best approach would be going for Concurrency Thread Group and Throughput Shaping Timer combination. They can be connected together via Feedback Function allowing the Concurrency Thread Group to kick off more threads if current amount is not enough to conduct the required load.
I am trying to find out throughput of an endpoint using Apache Jmeter as testing tool.
I need to hit the endpoint at n threads(users) per second, and every thread should fire exactly one requests(sample).
For the above i configured bzm - Concurrency Thread Group.
Configuration One:
Target Concurrency: 100
Ramp Up Time(sec): 1
Ramp-up Steps Count: 1
Hold Target Rate Time(sec): 10
Configuration Two:
Target Concurrency: 1000
Ramp Up Time(sec): 5
Ramp-up Steps Count: 5
Hold Target Rate Time(sec): 60
What i expect is
a. Configuration One: only 1000 request must be fired in 10 sec.
b. Configuration Two: 60000 requests must be fired in 60 sec i.e 1000/sec.
As per your test case and test duration settings, I think, you are making confusion with the Threads and Requests.
Threads are virtual users. Threads and Requests aren't the same.
Ramp up is the time in which all the users arrive on your tested application server.
Requests are simulated by samplers but threads are the simulation of users.
Please note- The total number of requests are related to throughput, Whereas the number of active threads performing the same activity is related to concurrency.
You can achieve your requirements by using Constant Throughput Timer at your test plan level.
Constant Throughput timer allows you to maintain the throughput of your server (requests/sec). Constant Throughput Timer is only capable of pausing JMeter threads in order to slow them down to reach the target throughput. Also, it works only on a minute level so you need to properly calculate the ramp-up period and let your test run long enough.
Here are the workarounds:
Calculate the number of threads you need to achieve your target throughputs. Formula is:
RPS * max response time in second
Use the Ramp-up periods for the threads to active accordingly.
Make the loop count value to Forever
Set the test duration accordingly.
As Constant Throughput Timer works on a minute level, to achieve X RPS you have to configure your "Target Throughput" value to X*60/min and "Calculate Throughput based on" value as "All active threads".
For example, if you need 1000 RPS, then "Target Throughput" value should be 60000/min in Constant Throughput Timer.
Check out my answers on these threads to understand your scenario more briefly.
How should I calculate Ramp-up time in Jmeter
Struggling to maintain Requests per Second (RPS)?
Hope this helps!