I am trying to find out throughput of an endpoint using Apache Jmeter as testing tool.
I need to hit the endpoint at n threads(users) per second, and every thread should fire exactly one requests(sample).
For the above i configured bzm - Concurrency Thread Group.
Configuration One:
Target Concurrency: 100
Ramp Up Time(sec): 1
Ramp-up Steps Count: 1
Hold Target Rate Time(sec): 10
Configuration Two:
Target Concurrency: 1000
Ramp Up Time(sec): 5
Ramp-up Steps Count: 5
Hold Target Rate Time(sec): 60
What i expect is
a. Configuration One: only 1000 request must be fired in 10 sec.
b. Configuration Two: 60000 requests must be fired in 60 sec i.e 1000/sec.
As per your test case and test duration settings, I think, you are making confusion with the Threads and Requests.
Threads are virtual users. Threads and Requests aren't the same.
Ramp up is the time in which all the users arrive on your tested application server.
Requests are simulated by samplers but threads are the simulation of users.
Please note- The total number of requests are related to throughput, Whereas the number of active threads performing the same activity is related to concurrency.
You can achieve your requirements by using Constant Throughput Timer at your test plan level.
Constant Throughput timer allows you to maintain the throughput of your server (requests/sec). Constant Throughput Timer is only capable of pausing JMeter threads in order to slow them down to reach the target throughput. Also, it works only on a minute level so you need to properly calculate the ramp-up period and let your test run long enough.
Here are the workarounds:
Calculate the number of threads you need to achieve your target throughputs. Formula is:
RPS * max response time in second
Use the Ramp-up periods for the threads to active accordingly.
Make the loop count value to Forever
Set the test duration accordingly.
As Constant Throughput Timer works on a minute level, to achieve X RPS you have to configure your "Target Throughput" value to X*60/min and "Calculate Throughput based on" value as "All active threads".
For example, if you need 1000 RPS, then "Target Throughput" value should be 60000/min in Constant Throughput Timer.
Check out my answers on these threads to understand your scenario more briefly.
How should I calculate Ramp-up time in Jmeter
Struggling to maintain Requests per Second (RPS)?
Hope this helps!
Related
I am conducting a performance test (TPS) using jmeter.
I am requesting about 10,000 TPS, but the following two results are different.
(Position that 10,000 TPS responds normally)
1000 thread x 600 target throughput(in samples per minute)
100 thread x 6000 target throughput(in samples per minute)
I think the two results should be the same, but why is the response time delayed as the thread increases?
I think the two results should be the same - why they would be the same?
Let's imagine your system has fixed response time of 1 second, in that case:
With 1000 threads you will get 1000 requests per second and you can limit the throughput to 10 requests per second using the Constant Throughput Timer
With 100 threads you will get 100 requests per second, no limiting is required
And what if response time is 2 seconds?
With 1000 threads you will get 500 requests per second
With 100 threads you will get 50 requests per second
Constant Throughput Timer:
acts precise enough on "minute" scale, if your test lasts less than minute it might not apply the throughput
can only pause the threads to limit the throughput (requests per minute) to the desired value. If current number of threads is not enough in order to conduct the required load - the time won't have any effect.
If you want to send requests at the rate of 10000 TPS it worth considering going for the Throughput Shaping Timer and Concurrency Thread Group combination connected via the Feedback Function in this case JMeter will be able to kick off extra threads if current number is not sufficient.
But also be informed that:
JMeter should be able to start as many threads as needed to send 10000 TPS so make sure to follow JMeter Best Practices or even consider going for Distributed Testing Mode
Application needs to be able to handle the load and respond fast enough, JMeter waits for the previous response before starting the new request so if application is able to serve i.e. 5000 requests per second only you won't be able to reach 10000 by any means
I am given a task in which I have to test my app for an hour with 300 TPS.
I have added my HTTP req in Jmeter and I need to configure below parameters to achieve my target
target concurrency
Ramp up time
Ramp-up steps
Hold target
thread iteration limit
I could guess by name few of these but still I am not quiet sure how to work around it to achieve my target. I am new to Jmeter can any one please help.
I assume you're talking about Concurrency Thread Group so answers are:
target concurrency - we don't know, it depends on your application response time, i.e. if response time is 1 second - go for 300, if response time is 2 seconds - go for 600, etc. If you don't know the response time you can put "sufficient" number of concurrent threads, for example 1000 and use Throughput Shaping Timer for limiting the load to 300 TPS
Ramp up time - again, we don't know your requirements. I can only recommend increasing the load gradually so you could correlate increasing load with increasing response time or number of errors so for example start with 1 TPS and increase the load to 300 TPS in i.e. 15 minutes and leave it running for 1 hour
Ramp-up steps - again, we don't know your requirements, start with above recommendation and adjust ramp-up period up or down as needed
Hold target - at least 60 minutes or 3600 seconds plus any ramp-up time
thread iteration limit - leave blank otherwise your test may end earlier than you expect
Example Concurrency Thread Group combination:
the values given in Schedule Feedback Function are for reference only, the example means that the Concurrency Thread Group will start 1000 threads initially with maximum of 1500 threads and have 300 spare threads in pool, hopefully it will be sufficient to produce 300 TPS load which is set by the Throughput Shaping Timer
I am so much confused with Jmeter test configuration. I found lots of answers but non those clear my doubt. My thread group contain 10 threads and all are in active state. I want to hit 20 requests per second to server. Load need to last for 60 seconds.
I put
thread count : 20
Ramp Up Period : 1
Loop Count: 60
Is it correct method to hit 20 requests/second last for 60 seconds?
For Request Per Second we should not calculate manually, instead leave the headache to JMeter. Just make sure, to have enough Threads to reach that TPS and JMeter will control the pacing. If the Threads are less and Request Per Second is more, than JMeter cant to anything and RPS wont be met.
Use Throughput Shaping Timer to achieve Request Per Second Throughput Shaping Timer
How Many Threads I Need To Produce Desired RPS?
Threads pool size can be calculated like RPS * <max response time> / 1000. The more rate desired the more threads you will need. The more response time service have the more threads you will need.
For example, if your service response time may be 2.5sec and target rps is 1230, you have to have 1230 * 2500 / 1000 = 3075 threads.
It is better to have some threads overhead to handle unexpected response time spikes. But beware of too much overhead, it will lead to "final spike".
Use Following settings in Thread Group and Throughput Shaping Timer:
Increase Number of Threads if 20 TPS is not met.
You can use stepping thread group or concurrency thread group instead of basic thread group to achieve your desired results.
https://jmeter-plugins.org/wiki/ConcurrencyThreadGroup/
Your configuration will be correct only if:
You have only one Sampler
It's response time is equal to 1 second
Correct configuration would be something like:
The best approach would be going for Concurrency Thread Group and Throughput Shaping Timer combination. They can be connected together via Feedback Function allowing the Concurrency Thread Group to kick off more threads if current amount is not enough to conduct the required load.
I have tried plethora of configurations with varying values for Number of Threads and Ramp-Up period and also tried to use uniform random timer stuff but in vain. Any help appreciated !
You can use Ultimate Thread Group for varying your concurrency. You can download it via JMeter Plugins Manager.
Here is an example for 60 seconds test duration with different concurrency:
To generate 50 requests every 5 second means 10RPS. To generate this scenario, you can use Constant Throughput Timer.
Constant Throughput Timer allows you to maintain the throughput of your server (requests/sec). As Constant Throughput Timer works on minute level, At the end of your test, you will get your desired Throughput which you mentioned in Constant Throughput Timer i.e. if you have mentioned 600/min in Constant Throughput Timer with "Calculate Throughput based on" value as "All active threads", you will get 10RPS.
You can also monitor these by using listeners like Active Threads Over Time and Hits per Second.
There are many questions/answers available here to understand Ramp up time but I want to get something in detail for my test case.
Test Case : Expecting 1200 users per 5 minutes on the home page. So it will be like 5 users/second.
I have set following thread properties :
No. of Threads : 1200
Ramp-up Time - ? [I am not sure what to set]
Loop count - Forever
Scheduler - 300 Seconds[5 Minutes]
Can anyone please help me to set ramp up time for my test case? I am running the test on my local machine.
I want to check that how many users server can handle in 5 minutes. Our expectation is 1200 users.
As per you test case and duration settings, I think, you are making confusion with the Threads and Requests.
In short: You must use Ramp-up as 240 seconds for this scenario.
Let me describe you in details:
Ramp up is the time in which all the users arrive on your tested application server.
Requests are simulated by samplers but threads are the simulation of users.
Please note- The total number of requests are related to throughput, Whereas the number of active threads performing the same activity is related to concurrency.
From your requirements, I assume you want to measure the throughput which is related to the requests/second not the users per second. To achieve this, you can use a Constant Throughput Timer at your test plan level.
Constant Throughput timer allows you maintain throughput of your server (requests/sec). Here requests are samplers. Threads are users/clients which are requesting server using samplers.
You can achieve this by using both "Thread Group" or "Ultimate Thread Group".
As far as I know, it doesn't matter which type of Thread group you are using with Constant Throughput timer, at the end of the test in the results, you will get your desired Throughput which you mentioned in Constant Throughput Timer i.e. if you have mentioned 1200/min in Constant Throughput Timer with "Calculate Throughput based on" value as "All active threads" and there are 3 requests (samplers) in your thread group then JMeter will manage the requests in a way that it will generate only 6.6 requests/sec for each sampler i.e. 1200/min is divided among 3 requests, it doesn't matter whether you are using Ultimate Thread Group or the Thread Group; Throughput timer works in same way for all types of thread groups.
Now as per your requirement: (Using Thread Group):
No. of Threads : 1200
Ramp-up Time - 240 (Since you want 5 users per second, 1200/5= 240)
Loop count - Forever
Scheduler (Duration) - 780 Seconds [13 Minutes]
IMPORTANT:
Thread Group configurations will be the same which you have already mentioned. Number of threads will be the client load which you want to put(in you case it's 1200), duration : (240+300+240), loop : forever, ramp-up : 240 yes when you ramp up i.e. in the first minute generally you get more requests as threads are starting so it is normal behavior. To test your scenario run for exactly 5 minutes, you should test for 13 minutes. Exclude first 4 min and last 4 min of the test as that is warm up time for your test as well as server and last 4 min are warm down period.
Now as per your requirement: (Using Ultimate Thread Group):
Start Thread Count: 1200
Initial delay, sec: 0
Startup, sec: 240 (I assume you want 5 users come to your application for requesting expected samples per second)
Hold Load for: 300 sec (Since you required 5 minutes to test for 1200 users)
Shutdown Time, sec: 240
You can set just like this:
Don't get confuse the Thread number with the number of requests, each thread will create multiple requests for seconds in field Hold Load for.
Rampup is the speed at which users arrive on your application.
The number you put means that once it is reached all threads have been started.
The tiniest it is , the faster users arrive.
So nobody can answer that for you, you need to analyse your traffic.
Regarding the number of threads, 1200 threads might not be the good number, as 1200 users can navigate on application slowly or rapidly.
What you need is add a Constant Throughput Timer to control how much transactions per second you'll be reaching.
You should use the Ultimate Thread Group plugins available in Standard set.
To maintain concurrency the configuration should be like this:
Start Thread Count: 1200
Initial delay, sec: 0
Startup, sec: 60
Hold Load: 180
Shutdown Time, sec: 60
Ultimate Thread Group
Here all the 1200 threads will be generated and up within 60 sec of Startup time(Ramp up time) .Those 1200 threads will be active for next 180 sec, and after that, those threads will shut down within next 60 sec.
So overall duration of the test is 300 sec as you desire. Throughout the Hold Load time(180sec) the requests will be looped.
This ultimate thread group will give you more control over Threads.
Now if you want to control the rate of transactions per second or the number of samples to execute you can use Constant Throughput Timer. Constant Throughput Timer will let you control the flow of requests to be sent. The default "Thread Group" only control the way the users or threads will be generated.It doesn't control the rate of requests to be hit your application.So for this purpose, you could use "Constant Throughput Timer" to control your requests per second. For further detail, you could go through JMeter documentation.