How can I find out maximum throughput my application can achieve within sla of 1 sec as response time.i.e response time should be within 1 sec
Currently I am using throughput controller and setting up the required throughput per minute, but I need to know max throughput my application can deliver with average response time less then 1 sec.
Using Thread Group, set Number of Threads (users) to some big number and set rump-up period to e.g.: 600s (10min)
Jmeter will slowly increase throughput and you can monitor Max response time in Summary report (column Max) - when it starts reaching 1s, you can read your desired Throughput (in Throughput column) Then I recommend set this number to Number of threads, decrease rump-up and run test for couple of hours.
Avoid using sleeps in your loop in JMeter. Beware of cache at application side. Monitor Faults in Error columns and check system health on both sides (CPU,mem,IO,Netw,...)
Please note that Throughput Controller does not control the throughput.
Following is from the JMeter official site.
This controller is badly named, as it does not control throughput.
Please refer to the Constant Throughput Timer for an element that can
be used to adjust the throughput.
You may use plugin Throughput Shaping Timer if you want to have controlled throughput changing with the time.
Change the threads (virtual users) and throughput to check the maximum through that can be handled by the server.
Use Duration Assertion to check the response time of each desired sampler.
Just don't limit JMeter's throughput.
Start with 1 thread (virtual user) and gradually increase the load until you see response times exceeding 1 second (you can use i.e. AutoStop Listener plugin for automatically stopping the test when average response becomes higher than 1 second)
Once done you can look into Transactions per Second and Server Hits per Second, it will give you the number you're looking for.
Related
In my test plan , I only have one Thread Group.
Thread Group Screenshot
I have CSV file with 10 users and 15 API requests in Loop Controller.
Loop Controller Screenshot
All of my requests are passing, An average of 55 requests/second were being sent but I want to know the maximum capacity of the server.I need to sent 150 requests per second, So I added Constant Throughput Timer with Target Throughput 9000 Constant Throughput Timer Screenshot .But it is still sending an average of 55 requests/ second. Here is the screenshot of command line run output. Output
Can anyone help me understand what I am doing wrong here ? Which values should I change in order to acheive 150 requests per second ?
JMeter sends request, WAIT for answer, send another request
If you add timer, it basically add some sleep to this -> you can't increase throughput, only decrease. Check manual https://jmeter.apache.org/usermanual/component_reference.html#Constant_Throughput_Timer
You should check your server - Load, CPU, Memory, IOs - disks, network ...
Check your JMeter machine as well
Are the answers from server alright? Do you check e.g. HTTP 200 OK?
You can try add more ramp up time, inscrease/decrease user count, but first check your server!
You're trying to reach 150 requests per second with each thread, you need to amend at least this bit:
You're trying to reach 150 requests per second with 10 users which means that each user needs to execute 15 requests per second. It's only possible if response time of your application is 66 milliseconds or less and the average response time of your application is around 200 milliseconds so it explains the actual result
JMeter's Constant Throughput Timer can only pause the threads in order to limit requests execution rate to the desired value, it won't kick off extra threads if current amount is not sufficient.
So the options are in:
Increase the number of threads in the Thread Group to i.e. 50 (you may need even more)
Or consider switching to Throughput Shaping Timer and Concurrency Thread Group combination, the Throughput Shaping Timer is more "precise" and it can be connected to the Concurrency Thread Group via Feedback Function so you will be able to kick off more threads to reach and maintain the desired throughput
As per the Documentation - "Of course, if the server is not able to handle such a load, the throughput will be lower." How is this behavior handled and throughput calculated?
Suppose I set the target throughput as 1500/min but my system could handle only 1000/min, what kind of feedback behavior is expected by JMeter and how is the total throughput affected?
Will the JMeter still try to achieve 1500/min with higher error rate (for failed requests) or JMeter shoots the throughput to 1500/min but gradually comes back near to 1000/min based on certain factors like error rate? If yes, what are the factors determining this step-down, any insight/reference to understand this step-down behavior?
Either you do not have enough threads in the test plan to achieve your desired TPS or your application response time is higher.
The total number of requests you will be able to make through JMeter depends on your application response time.
So you will need at least 25 threads to achieve 1500/min if your average response time is 1 second (assuming there is only 1 request).
If throughput isn't increasing even after providing with enough threads but the response time increasing and some of your requests got failed, then this indicates the performance bottleneck!
JMeter cannot send requests faster than your application is capable of responding. If your application can only serve 1000 requests per minute - you will have 1000 requests per minute no matter how many threads you have in the Thread Group or how many requests per minute you specify in the Constant Throughput Timer.
Also be aware that:
Constant Throughput Timer can only pause the Threads, it will not do anything if the current amount of threads is not enough to reach/maintain the desired throughput
Constant Throughput Timer is accurate enough on minute level so consider choosing proper ramp-up settings otherwise you can stress your application during first minute of your test
I am trying to stress test my server.
To do so I am using Jmeter and here is my set up:
I use
my Setup
Thread: 1000
schedule for 3 mins
So as you see I keep going with 1000 thread for a period of 3 mins.
But when I look at the throughput I only get around 230 per second
results
So what should I do to increase the through put to for example 1000000 per second? How come increasing the thread which I assume means more load does not increase throughput?
According to JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
Throughput explicitly relies on the application response time. Looking into your results, the average response time is 3.5 seconds therefore you will not get more than 1000 / 3.5 = 285 requests per second
Theoretically you could use Throughput Shaping Timer and Concurrency Thread Group combination, this way JMeter will kick off extra threads if the current amount is not enough to reach/maintain the desired throughput, however looking into 8.5% error rate and maximum response time for your application > 2 minutes my expectation is that you will not be able to get more throughput because most probably your application is overloaded and cannot respond faster.
Throughput measures the number of transactions or requests that can be made in a given period of time. basically, it lists the number of requests server managed to serve in a given time period. Throughput value depends on lot of factors and maybe your application under test not able to cater the expected load.
So with 1000 threads, you can't expect a 1000 throughput.
It's up to you to find out how much throughput your application can handle. For that maybe you need to do different optimizations on your side like optimize your script, distribute load via JMeter execution, increase theard count,...etc
Our requirement is to execute
1 user/thread, 10 requests/sec, Total requests = 10,000
With current Thread Properties not able to achieve.
Only one user/thread send 10 request/sec and total should be 10000 requests. Any other way to achieve this in Jmeter ?
Is following approach is correct ? We used Loop controller. So each request will repeat 10 times
You will be able to achieve 10 requests/second with 1 virtual user only if your response time will be 100 milliseconds. If response time will be more than 100 milliseconds - you will not be able to reach the desired load.
If your application fails to respond in 100 milliseconds time frame - most probably you've found a performance bottleneck and you can report this to your application developers.
If you have some time to investigate the issue you can try to provide more information, i.e. what is the average response time, what are minimum, maximum, quantiles, the actual number of requests per second, etc, all this information can be retrieved using Aggregate Report listener.
Normally when people are looking for the answer if the application under test can support X requests per second they use > 1 virtual user as load test should represent real life usage of the system under test and 1 thread (virtual user) isn't normally associated with performance testing and derivatives.
So probably you should reconsider your approach to testing and try increasing the number of threads (virtual users). The throughput can be controlled using Precise Throughput Timer or Constant Throughput Timer. However be aware that the above timers can only pause JMeter to slow it down to the desired throughput. Another approach is using Concurrency Thread Group and Throughput Shaping Timer combination, they can be connected using Feedback Function so JMeter will kick off extra threads in order to reach/maintain the desired number of requests per second.
You can install Concurrency Thread Group and Throughput Shaping Timer using JMeter Plugins Manager
I am novice to Jmeter, and I have certain queries which I am not able to get from the Jmeter home site.
What does throughput value exactly mean - does it mean it is no of requests per second for each thread or it is no of requests cumulatively across threads ?
I want to run a test for 5 minutes evenly spread across 125k requests, how do I specify throughput and delay values assuming no of threads is 60?
The values starttime , endtime, duration, delay, rampup period, etc given in Jmeter file are of seconds/milliseconds ? How can I configure them ?
How can I find the transactions per second value during or after the test ?
As per JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
You can set test execution time either using "Scheduler" section of Thread Group where you can set Duration (seconds) for the test. By default JMeter executes samplers as fast as it can however you can use Timers to simulate user think time or add pauses between requests for any other reason. Desired execution rate in "requests per minute" can be set via Constant Throughput Timer
The values are in seconds. Configure them according to your load scenario. As per thread group documentation:
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
The easiest option is using Transactions per Second listener available through JMeter Plugins