I have started 300 requests per second by providing following values:
Number of Threads : 300
Tamp-up : 0
But I am getting following results:
summary + 55 in 21s
summary + 225 in 31.1s
summary = 280 in 31.1s
what different configurations would be needed to start all requests in one go?
IMHO, you have to take in account the time it takes to run the transaction itself.
I tend NOT to use synchronization, but to take measure on longer period of time, e.g. 15 minutes.
E.g., if your system is able to deliver one page in 2 seconds, you need to run AT LEAST 600 threads to deliver the throughput you want (probably more).
Also, remember that the time of a single page increase with the load, so a single measurement is not enough, AND take care of errors: you have to define an acceptable threshold for errors (e.g. 0.01%), and stop measurement when you go above that.
If you need to start all 300 requests at the same moment you need to use Synchronizing Timer
If you need provide constant load at 300 requests per second rate - you'll need Constant Throughput Timer
Two above would cover the majority of use cases, however if you want more control on your load pattern look into Throughput Shaping Timer (available through JMeter plugin)
Related
i have to loadtest apis with 10k rpm,what should be the ramp up time and execution time.
10k rpm = 10k/60=166 req/sec i.e 166 concurrent users
but don't know to calculate exact ramptime time and execution time.
currently standard set is 60min execution and 15 min ramp up in the org
10k rpm = 10k/60=166 req/sec i.e 166 concurrent users
166 concurrent users don't necessarily mean 166 requests per second, moreover it will happen only if response time for each Sampler will be 1 second sharp.
If response time will be 2 seconds - you will have 83 requests per second
If response time will be 0.5 seconds - you will have 332 requests per second
So you need to supply sufficient amount of threads and add a relevant Timer like Constant Throughput Timer or Throughput Shaping Timer to slow down JMeter throughput to the desired value. The latter one can be connected with the Concurrency Thread Group so JMeter would be able to kick off extra threads if the current amount is not sufficient to conduct the required load.
And the system under test need to be able to respond fast enough because JMeter waits for the response from the previous request before starting a new one.
With regards to ramp-up, there are no exact numbers recommendations, the only approach you can follow is increasing the load gradually so you would be able to correlate the increasing load with increasing throughput, response time, number of errors and so on.
If you don't have better ideas you can go for the quote from the JMeter user manual:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
It completely depends on what exactly needs to be achieved
Things to consider:
Test execution duration and ramp up/ramp down of users, needs to evaluated based on the AUT(Application under Test) Real user utilization in Production.(Capture the number of users active at particular point of time in system)
Watch out for pattern of users entering/exiting the system.
Calculate User load/ RPM using Little's Law.
Some of the common test Types includes.
Baseline Test (Initial testing conducted with very minimal load to test the stability of the application)
Load Test (To evaluate applications performance under gradually increasing User load with expected concurrent number of users
Stress Test (Testing beyond normal conditions, increasing the user load until the Response time exceeds acceptable criteria. Usually done to find the breakpoint of the application, responsiveness of the application for higher volume traffic
Endurance Test ( To test for a longer durations, by applying varying loads.)
I am using Jmeter for load testing and I'm new to this. I have an API where I want to send around 36000 requests in a given time, which is- 5 minutes. What should be the configuration of threads, ramp-up time, loop-count, and constant throughput timer for this scenario?
I am using the following configurations, but I am unable to reach the decided RPS-
Thread- 1000
Ramp-up- 5 Minute
loop-count 36
constant throughput timer- 7200
Where is my configuration wrong?
You can try to reduce the ramp-up period to be close to zero and increase the number of loops to "infinite", the total number of requests can be limited using Throughput Controller
In general there could be 2 main reasons of not being able to conduct the required load:
JMeter cannot produce the desired number of hits per second. Things to try:
Make sure to follow JMeter Best Practices
Increase number of threads in Thread Group
Consider switching to Distributed Testing mode
Application cannot handle that many requests per second. Things to try:
Inspect configuration and make sure it's suitable for high loads
Inspect CPU, RAM, Disk, etc. usage during the load test, it might be simply lack of resources, it can be done using JMeter PerfMon Plugin
Re-run your test with profiler tool telemetry enabled
Raise a ticket as it is a performance bottleneck
I am trying to stress test my server.
To do so I am using Jmeter and here is my set up:
I use
my Setup
Thread: 1000
schedule for 3 mins
So as you see I keep going with 1000 thread for a period of 3 mins.
But when I look at the throughput I only get around 230 per second
results
So what should I do to increase the through put to for example 1000000 per second? How come increasing the thread which I assume means more load does not increase throughput?
According to JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
Throughput explicitly relies on the application response time. Looking into your results, the average response time is 3.5 seconds therefore you will not get more than 1000 / 3.5 = 285 requests per second
Theoretically you could use Throughput Shaping Timer and Concurrency Thread Group combination, this way JMeter will kick off extra threads if the current amount is not enough to reach/maintain the desired throughput, however looking into 8.5% error rate and maximum response time for your application > 2 minutes my expectation is that you will not be able to get more throughput because most probably your application is overloaded and cannot respond faster.
Throughput measures the number of transactions or requests that can be made in a given period of time. basically, it lists the number of requests server managed to serve in a given time period. Throughput value depends on lot of factors and maybe your application under test not able to cater the expected load.
So with 1000 threads, you can't expect a 1000 throughput.
It's up to you to find out how much throughput your application can handle. For that maybe you need to do different optimizations on your side like optimize your script, distribute load via JMeter execution, increase theard count,...etc
I'm getting the following results, where the throughput does not have a change, even when I increase the number of threads.
Scenario#1:
Number of threads: 10
Ramp-up period: 60
Throughput: 5.8/s
Avg: 4025
Scenario#2:
Number of threads: 20
Ramp-up period: 60
Throughput: 7.8/s
Avg: 5098
Scenario#3:
Number of threads: 40
Ramp-up period: 60
Throughput: 6.8/s
Avg: 4098
The my JMeter file consists of a single ThreadGroup that contains a single GET.
When I perform the request for an endpoit where the response time faster (less than 300 ms) I can achieve throughput greater than 50 requests per seconds.
Can you see the bottleneck of this?
Is there a relationship between response time and throughput?
It's simple as JMeter user manual states:
Throughput = (number of requests) / (total time)
Now assuming your test contains only a single GET then Throughput will be correlate average response time of your requests.
Notice Ramp-up period: 60 will start to create threads over 1 minute, so it will add to total time of execution, you can try to reduce it to 10 or equal to Number of threads.
But you may have other sampler/controllers/component that may effect total time.
Also in your case especially in Scenario 3, maybe some requests failed then you are not calculating Throughput of successful transactions.
In ideal world if you increase number of threads by factor of 2x - throughput should increase by the same factor.
In reality the "ideal" scenario is hardly achievable so it looks like a bottleneck in your application. The process of identifying the bottleneck normally looks as follows:
Amend your test configuration to increase the load gradually so i.e. start with 1 virtual user and increase the load to i.e. 100 virtual users in 5 minutes
Run your test and look into Active Threads Over Time, Response Times Over Time and Server Hits Per Second listeners. This way you will be able to correlate increasing load with increasing response time and identify the point where performance starts degrading. See What is the Relationship Between Users and Hits Per Second? for more information
Once you figure out what is the saturation point you need to know what prevents your application from from serving more requests, the reasons could be in:
Application simply lacks resources (CPU, RAM, Network, Disk, etc.), make sure to monitor the aforementioned resources, this could be done using i.e JMeter PerfMon Plugin
The infrastructure configuration is not suitable for high loads (i.e. application or database thread pool settings incorrect)
The problem is in your application code (inefficient algorithm, large objects, slow DB queries). These items can be fetched using a profiler tool
Also make sure you're following JMeter Best Practices as it might be the case JMeter is not capable of sending requests fast enough due to either lack of resources on JMeter load generator side or incorrect JMeter configuration (too low heap, running test in GUI mode, using listeners, etc)
I have a task to load test on application which needs to respond 2350 users per second. For that I have set us something like this in Jmeter:
I have added a Thread group. In that I have set:
Number of threads(users): 2350
Ramp-up period: 1 Second
Loop Count: 1
Will it solve my purpose to load test of application with 2350 users??
It will. But only if response time for each virtual user will be equal 1 second.
There are 2 common load patterns, for implementing both you will need Timers
Actually it might be the case you don't need as much as 2350 thread to simulate 2350 users as real life users don't hammer the server non-stop, they need some time to think between requests. Besides page loading time also needs to be considered.
Let's imagine you have 2350 users. Each user performs an action each 15 seconds. Page loading time is 5 seconds. So each user will be able to hit the server 3 times per minute. So 2350 users will produce 7050 requests in minute which stands for 117.5 requests per second only. If this is what you're looking for consider adding Constant Timer or Uniform Random Timer
If you need to simulate 2350 requests per second, not users, you need to handle it a little bit differently. There are 2 Timers which are designed to set exact "throughput" - a number of requests per time unit. They are:
Constant Throughput Timer
Throughput Shaping Timer - an advanced version of the Constant Throughput Timer, available via JMeter Plugins project.
Remember that above timers can only pause the threads, they won't kick off new virtual users if you don't provide enough on Thread Group level so make sure you have at least as much as you try to simulate, or it's better to have 2x more in your virtual pocket just in case. Also check out JMeter tuning tips from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure as JMeter default configuration isn't something you can use to create such a load.
The setup you described just creates 2350 parallel users (2350 separate connections), but doesn't guarantee that every thread will be completed exactly in 1 second. Ramp-up period - how fast all threads will start to send requests. So, in the case you described Jmeter will do the following:
Create 2350 separate threads.
The difference between 1st and last thread start will be 1 second (approximately).
Every thread will be implemented only once (loop count).
For real scenarios when you need specific throughput for continuous period of time it's better to use Constant Throughput Timer. It controls the amount of requests sent by thread(s) and changes delay between requests when it's necessary to meet the value you defined. So, your real throughput doesn't really depend on the total amount of threads (users). Sometimes less users can send requests faster (depends on your application).
To control your throughput while running, just add Summary Report to your test plan.
Moreover, for this specific scenario (2350 users) it can be potentially difficult to generate so many requests in 1 second. In this case, you need to use distributed load with some Jmeter slaves and 1 master.