I have a task to run the exact number of requests within an hour, with a given maximum number of concurrency, how do i design that with Jmeter?
For example:
Total number of request: 1000
Run time: 60 mins
Max Concurrency: 15
Meaning my script needs to reach the maximum of 15 requests at the same time at least once, and the rest can be distributed unevenly throughout an hour, and it needs to finish 1000 requests within an hour.
Thank you
You can set the hard limit on the number of requests which JMeter will execute by using Throughput Controller in "Total Executions" mode
You can slow down JMeter's number of requests per minute by using Constant Throughput Timer. If you need to execute 1000 requests in 60 minutes you need to run 16.6 requests per minute, in this case they will be evenly distributed.
If your workload patterns are more complex consider using Throughput Shaping Timer
Related
i have to loadtest apis with 10k rpm,what should be the ramp up time and execution time.
10k rpm = 10k/60=166 req/sec i.e 166 concurrent users
but don't know to calculate exact ramptime time and execution time.
currently standard set is 60min execution and 15 min ramp up in the org
10k rpm = 10k/60=166 req/sec i.e 166 concurrent users
166 concurrent users don't necessarily mean 166 requests per second, moreover it will happen only if response time for each Sampler will be 1 second sharp.
If response time will be 2 seconds - you will have 83 requests per second
If response time will be 0.5 seconds - you will have 332 requests per second
So you need to supply sufficient amount of threads and add a relevant Timer like Constant Throughput Timer or Throughput Shaping Timer to slow down JMeter throughput to the desired value. The latter one can be connected with the Concurrency Thread Group so JMeter would be able to kick off extra threads if the current amount is not sufficient to conduct the required load.
And the system under test need to be able to respond fast enough because JMeter waits for the response from the previous request before starting a new one.
With regards to ramp-up, there are no exact numbers recommendations, the only approach you can follow is increasing the load gradually so you would be able to correlate the increasing load with increasing throughput, response time, number of errors and so on.
If you don't have better ideas you can go for the quote from the JMeter user manual:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
It completely depends on what exactly needs to be achieved
Things to consider:
Test execution duration and ramp up/ramp down of users, needs to evaluated based on the AUT(Application under Test) Real user utilization in Production.(Capture the number of users active at particular point of time in system)
Watch out for pattern of users entering/exiting the system.
Calculate User load/ RPM using Little's Law.
Some of the common test Types includes.
Baseline Test (Initial testing conducted with very minimal load to test the stability of the application)
Load Test (To evaluate applications performance under gradually increasing User load with expected concurrent number of users
Stress Test (Testing beyond normal conditions, increasing the user load until the Response time exceeds acceptable criteria. Usually done to find the breakpoint of the application, responsiveness of the application for higher volume traffic
Endurance Test ( To test for a longer durations, by applying varying loads.)
I am conducting a performance test (TPS) using jmeter.
I am requesting about 10,000 TPS, but the following two results are different.
(Position that 10,000 TPS responds normally)
1000 thread x 600 target throughput(in samples per minute)
100 thread x 6000 target throughput(in samples per minute)
I think the two results should be the same, but why is the response time delayed as the thread increases?
I think the two results should be the same - why they would be the same?
Let's imagine your system has fixed response time of 1 second, in that case:
With 1000 threads you will get 1000 requests per second and you can limit the throughput to 10 requests per second using the Constant Throughput Timer
With 100 threads you will get 100 requests per second, no limiting is required
And what if response time is 2 seconds?
With 1000 threads you will get 500 requests per second
With 100 threads you will get 50 requests per second
Constant Throughput Timer:
acts precise enough on "minute" scale, if your test lasts less than minute it might not apply the throughput
can only pause the threads to limit the throughput (requests per minute) to the desired value. If current number of threads is not enough in order to conduct the required load - the time won't have any effect.
If you want to send requests at the rate of 10000 TPS it worth considering going for the Throughput Shaping Timer and Concurrency Thread Group combination connected via the Feedback Function in this case JMeter will be able to kick off extra threads if current number is not sufficient.
But also be informed that:
JMeter should be able to start as many threads as needed to send 10000 TPS so make sure to follow JMeter Best Practices or even consider going for Distributed Testing Mode
Application needs to be able to handle the load and respond fast enough, JMeter waits for the previous response before starting the new request so if application is able to serve i.e. 5000 requests per second only you won't be able to reach 10000 by any means
I am trying to stress test my server.
To do so I am using Jmeter and here is my set up:
I use
my Setup
Thread: 1000
schedule for 3 mins
So as you see I keep going with 1000 thread for a period of 3 mins.
But when I look at the throughput I only get around 230 per second
results
So what should I do to increase the through put to for example 1000000 per second? How come increasing the thread which I assume means more load does not increase throughput?
According to JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
Throughput explicitly relies on the application response time. Looking into your results, the average response time is 3.5 seconds therefore you will not get more than 1000 / 3.5 = 285 requests per second
Theoretically you could use Throughput Shaping Timer and Concurrency Thread Group combination, this way JMeter will kick off extra threads if the current amount is not enough to reach/maintain the desired throughput, however looking into 8.5% error rate and maximum response time for your application > 2 minutes my expectation is that you will not be able to get more throughput because most probably your application is overloaded and cannot respond faster.
Throughput measures the number of transactions or requests that can be made in a given period of time. basically, it lists the number of requests server managed to serve in a given time period. Throughput value depends on lot of factors and maybe your application under test not able to cater the expected load.
So with 1000 threads, you can't expect a 1000 throughput.
It's up to you to find out how much throughput your application can handle. For that maybe you need to do different optimizations on your side like optimize your script, distribute load via JMeter execution, increase theard count,...etc
I have about 300 users (configured in the thread group) who would perform an activity (e.g.: run an e-learning course) twice. That would mean I need to expect about 600 iterations i.e 300 users performing an activity twice.
My thread group contains the following transaction controllers:
Login
Dashboard
Launch Course
Complete Course
Logout
As I need 600 iterations per 5400 seconds i.e 3600 + 900 + 900 seconds (1 hour steady state + 15 mins ramp-up and 15 mins ramp-down), and the sum of sampler requests within the total thread group are 18, would I be correct to say I need about 2 RPS?
Total number of iterations * number of requests per iteration = Total number of requests
600 * 18 = 10800
Total number of requests / Total test duration in seconds = Requests per second
10800 / 5400 = 2
Are my calculations correct?
In addition, what is the best approach to achieve the expected throughput?
Your calculation looks more or less correct. If you need to limit your test throughput to 2 RPS you can do it using Constant Throughput Timer or Throughput Shaping Timer.
However 2 RPS is nothing more than statistical noise, my expectation is that you need much higher load to really test your application performance, i.e.
Simulate the anticipated number of users for a short period. Don't care about iterations, just let your test to run i.e. for an hour with the number of users you expect. This is called load testing
Do the same but for longer period of time (i.e. overnight or weekend). This is called soak testing.
Gradually increase the number of users until you will see errors or response time will start exceeding acceptable thresholds. This is called stress testing.
I have set the number of threads and ramp-up time to 1/1 and I am iterating my 1000 records from data.csv for 1800 seconds.
Now given the numbers, I have set the CTT, constant time throughput to 2000 every minute and I expected the average throughput to be 2000/60 = 33.3/sec, but I get 18.7/sec, when I increased the throughput to 4000/60, I still get 18 or 19/sec.
Constant Throughput Timer cannot force threads to execute faster, it can only pause threads to limit JMeter's throughput to the defined value.
Each JMeter thread executes samplers as fast as it can, however next iteration won't start until previous is finished so given you use 1 thread - the throughput won't be higher than application response time.
Also be aware that Constant Throughput Timer is accurate enough on minute level so you can rather manipulate "requests per minute" rather than "requests per second", if your test is shorter than 1 minute - consider using Throughput Shaping Timer
So I would recommend increasing the number of virtual users to i.e. 50.
See How to use JMeter's Constant Throughput Timer for more details.
I guess your application average response time is around 50ms. Which means a single thread can only perform about 20 hits/sec (1 sec / 0.05 sec per hit = 20 hits/sec).
You have 2 solutions:
increase the number of threads to parallelize requests sent,
or make you app response faster (obviously harder).
At some point, when you application can't handle more load, you should see the hits/sec dropping and the average response time increase.
The graph below shows you an example of application which has a steady response time with up to 20 concurrent threads.