I have set the number of threads and ramp-up time to 1/1 and I am iterating my 1000 records from data.csv for 1800 seconds.
Now given the numbers, I have set the CTT, constant time throughput to 2000 every minute and I expected the average throughput to be 2000/60 = 33.3/sec, but I get 18.7/sec, when I increased the throughput to 4000/60, I still get 18 or 19/sec.
Constant Throughput Timer cannot force threads to execute faster, it can only pause threads to limit JMeter's throughput to the defined value.
Each JMeter thread executes samplers as fast as it can, however next iteration won't start until previous is finished so given you use 1 thread - the throughput won't be higher than application response time.
Also be aware that Constant Throughput Timer is accurate enough on minute level so you can rather manipulate "requests per minute" rather than "requests per second", if your test is shorter than 1 minute - consider using Throughput Shaping Timer
So I would recommend increasing the number of virtual users to i.e. 50.
See How to use JMeter's Constant Throughput Timer for more details.
I guess your application average response time is around 50ms. Which means a single thread can only perform about 20 hits/sec (1 sec / 0.05 sec per hit = 20 hits/sec).
You have 2 solutions:
increase the number of threads to parallelize requests sent,
or make you app response faster (obviously harder).
At some point, when you application can't handle more load, you should see the hits/sec dropping and the average response time increase.
The graph below shows you an example of application which has a steady response time with up to 20 concurrent threads.
Related
i have to loadtest apis with 10k rpm,what should be the ramp up time and execution time.
10k rpm = 10k/60=166 req/sec i.e 166 concurrent users
but don't know to calculate exact ramptime time and execution time.
currently standard set is 60min execution and 15 min ramp up in the org
10k rpm = 10k/60=166 req/sec i.e 166 concurrent users
166 concurrent users don't necessarily mean 166 requests per second, moreover it will happen only if response time for each Sampler will be 1 second sharp.
If response time will be 2 seconds - you will have 83 requests per second
If response time will be 0.5 seconds - you will have 332 requests per second
So you need to supply sufficient amount of threads and add a relevant Timer like Constant Throughput Timer or Throughput Shaping Timer to slow down JMeter throughput to the desired value. The latter one can be connected with the Concurrency Thread Group so JMeter would be able to kick off extra threads if the current amount is not sufficient to conduct the required load.
And the system under test need to be able to respond fast enough because JMeter waits for the response from the previous request before starting a new one.
With regards to ramp-up, there are no exact numbers recommendations, the only approach you can follow is increasing the load gradually so you would be able to correlate the increasing load with increasing throughput, response time, number of errors and so on.
If you don't have better ideas you can go for the quote from the JMeter user manual:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
It completely depends on what exactly needs to be achieved
Things to consider:
Test execution duration and ramp up/ramp down of users, needs to evaluated based on the AUT(Application under Test) Real user utilization in Production.(Capture the number of users active at particular point of time in system)
Watch out for pattern of users entering/exiting the system.
Calculate User load/ RPM using Little's Law.
Some of the common test Types includes.
Baseline Test (Initial testing conducted with very minimal load to test the stability of the application)
Load Test (To evaluate applications performance under gradually increasing User load with expected concurrent number of users
Stress Test (Testing beyond normal conditions, increasing the user load until the Response time exceeds acceptable criteria. Usually done to find the breakpoint of the application, responsiveness of the application for higher volume traffic
Endurance Test ( To test for a longer durations, by applying varying loads.)
I am conducting a performance test (TPS) using jmeter.
I am requesting about 10,000 TPS, but the following two results are different.
(Position that 10,000 TPS responds normally)
1000 thread x 600 target throughput(in samples per minute)
100 thread x 6000 target throughput(in samples per minute)
I think the two results should be the same, but why is the response time delayed as the thread increases?
I think the two results should be the same - why they would be the same?
Let's imagine your system has fixed response time of 1 second, in that case:
With 1000 threads you will get 1000 requests per second and you can limit the throughput to 10 requests per second using the Constant Throughput Timer
With 100 threads you will get 100 requests per second, no limiting is required
And what if response time is 2 seconds?
With 1000 threads you will get 500 requests per second
With 100 threads you will get 50 requests per second
Constant Throughput Timer:
acts precise enough on "minute" scale, if your test lasts less than minute it might not apply the throughput
can only pause the threads to limit the throughput (requests per minute) to the desired value. If current number of threads is not enough in order to conduct the required load - the time won't have any effect.
If you want to send requests at the rate of 10000 TPS it worth considering going for the Throughput Shaping Timer and Concurrency Thread Group combination connected via the Feedback Function in this case JMeter will be able to kick off extra threads if current number is not sufficient.
But also be informed that:
JMeter should be able to start as many threads as needed to send 10000 TPS so make sure to follow JMeter Best Practices or even consider going for Distributed Testing Mode
Application needs to be able to handle the load and respond fast enough, JMeter waits for the previous response before starting the new request so if application is able to serve i.e. 5000 requests per second only you won't be able to reach 10000 by any means
I am given a task in which I have to test my app for an hour with 300 TPS.
I have added my HTTP req in Jmeter and I need to configure below parameters to achieve my target
target concurrency
Ramp up time
Ramp-up steps
Hold target
thread iteration limit
I could guess by name few of these but still I am not quiet sure how to work around it to achieve my target. I am new to Jmeter can any one please help.
I assume you're talking about Concurrency Thread Group so answers are:
target concurrency - we don't know, it depends on your application response time, i.e. if response time is 1 second - go for 300, if response time is 2 seconds - go for 600, etc. If you don't know the response time you can put "sufficient" number of concurrent threads, for example 1000 and use Throughput Shaping Timer for limiting the load to 300 TPS
Ramp up time - again, we don't know your requirements. I can only recommend increasing the load gradually so you could correlate increasing load with increasing response time or number of errors so for example start with 1 TPS and increase the load to 300 TPS in i.e. 15 minutes and leave it running for 1 hour
Ramp-up steps - again, we don't know your requirements, start with above recommendation and adjust ramp-up period up or down as needed
Hold target - at least 60 minutes or 3600 seconds plus any ramp-up time
thread iteration limit - leave blank otherwise your test may end earlier than you expect
Example Concurrency Thread Group combination:
the values given in Schedule Feedback Function are for reference only, the example means that the Concurrency Thread Group will start 1000 threads initially with maximum of 1500 threads and have 300 spare threads in pool, hopefully it will be sufficient to produce 300 TPS load which is set by the Throughput Shaping Timer
I am trying to stress test my server.
To do so I am using Jmeter and here is my set up:
I use
my Setup
Thread: 1000
schedule for 3 mins
So as you see I keep going with 1000 thread for a period of 3 mins.
But when I look at the throughput I only get around 230 per second
results
So what should I do to increase the through put to for example 1000000 per second? How come increasing the thread which I assume means more load does not increase throughput?
According to JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
Throughput explicitly relies on the application response time. Looking into your results, the average response time is 3.5 seconds therefore you will not get more than 1000 / 3.5 = 285 requests per second
Theoretically you could use Throughput Shaping Timer and Concurrency Thread Group combination, this way JMeter will kick off extra threads if the current amount is not enough to reach/maintain the desired throughput, however looking into 8.5% error rate and maximum response time for your application > 2 minutes my expectation is that you will not be able to get more throughput because most probably your application is overloaded and cannot respond faster.
Throughput measures the number of transactions or requests that can be made in a given period of time. basically, it lists the number of requests server managed to serve in a given time period. Throughput value depends on lot of factors and maybe your application under test not able to cater the expected load.
So with 1000 threads, you can't expect a 1000 throughput.
It's up to you to find out how much throughput your application can handle. For that maybe you need to do different optimizations on your side like optimize your script, distribute load via JMeter execution, increase theard count,...etc
I am trying to find out throughput of an endpoint using Apache Jmeter as testing tool.
I need to hit the endpoint at n threads(users) per second, and every thread should fire exactly one requests(sample).
For the above i configured bzm - Concurrency Thread Group.
Configuration One:
Target Concurrency: 100
Ramp Up Time(sec): 1
Ramp-up Steps Count: 1
Hold Target Rate Time(sec): 10
Configuration Two:
Target Concurrency: 1000
Ramp Up Time(sec): 5
Ramp-up Steps Count: 5
Hold Target Rate Time(sec): 60
What i expect is
a. Configuration One: only 1000 request must be fired in 10 sec.
b. Configuration Two: 60000 requests must be fired in 60 sec i.e 1000/sec.
As per your test case and test duration settings, I think, you are making confusion with the Threads and Requests.
Threads are virtual users. Threads and Requests aren't the same.
Ramp up is the time in which all the users arrive on your tested application server.
Requests are simulated by samplers but threads are the simulation of users.
Please note- The total number of requests are related to throughput, Whereas the number of active threads performing the same activity is related to concurrency.
You can achieve your requirements by using Constant Throughput Timer at your test plan level.
Constant Throughput timer allows you to maintain the throughput of your server (requests/sec). Constant Throughput Timer is only capable of pausing JMeter threads in order to slow them down to reach the target throughput. Also, it works only on a minute level so you need to properly calculate the ramp-up period and let your test run long enough.
Here are the workarounds:
Calculate the number of threads you need to achieve your target throughputs. Formula is:
RPS * max response time in second
Use the Ramp-up periods for the threads to active accordingly.
Make the loop count value to Forever
Set the test duration accordingly.
As Constant Throughput Timer works on a minute level, to achieve X RPS you have to configure your "Target Throughput" value to X*60/min and "Calculate Throughput based on" value as "All active threads".
For example, if you need 1000 RPS, then "Target Throughput" value should be 60000/min in Constant Throughput Timer.
Check out my answers on these threads to understand your scenario more briefly.
How should I calculate Ramp-up time in Jmeter
Struggling to maintain Requests per Second (RPS)?
Hope this helps!