I have hard time to understand the throughput for multiple requests total throughput.
for example, I send 10 request to app server like request 1, request 2, so on, and I get results for them and for total, example, my request 1 has 17/ sec, request 2 has 18/sec and Total throughput is 115/sec.
so the application throughput is 17/sec or 115/sec ?
I don't know how to explain or understand the results.
I think you are confusing between response time and Throughput
Throughput is measured in requests per second
Throughput shows how many requests sent in one second
i.e. 30.0 requests/minute is saved as 0.5.
Related
I am conducting a performance test (TPS) using jmeter.
I am requesting about 10,000 TPS, but the following two results are different.
(Position that 10,000 TPS responds normally)
1000 thread x 600 target throughput(in samples per minute)
100 thread x 6000 target throughput(in samples per minute)
I think the two results should be the same, but why is the response time delayed as the thread increases?
I think the two results should be the same - why they would be the same?
Let's imagine your system has fixed response time of 1 second, in that case:
With 1000 threads you will get 1000 requests per second and you can limit the throughput to 10 requests per second using the Constant Throughput Timer
With 100 threads you will get 100 requests per second, no limiting is required
And what if response time is 2 seconds?
With 1000 threads you will get 500 requests per second
With 100 threads you will get 50 requests per second
Constant Throughput Timer:
acts precise enough on "minute" scale, if your test lasts less than minute it might not apply the throughput
can only pause the threads to limit the throughput (requests per minute) to the desired value. If current number of threads is not enough in order to conduct the required load - the time won't have any effect.
If you want to send requests at the rate of 10000 TPS it worth considering going for the Throughput Shaping Timer and Concurrency Thread Group combination connected via the Feedback Function in this case JMeter will be able to kick off extra threads if current number is not sufficient.
But also be informed that:
JMeter should be able to start as many threads as needed to send 10000 TPS so make sure to follow JMeter Best Practices or even consider going for Distributed Testing Mode
Application needs to be able to handle the load and respond fast enough, JMeter waits for the previous response before starting the new request so if application is able to serve i.e. 5000 requests per second only you won't be able to reach 10000 by any means
In my test plan , I only have one Thread Group.
Thread Group Screenshot
I have CSV file with 10 users and 15 API requests in Loop Controller.
Loop Controller Screenshot
All of my requests are passing, An average of 55 requests/second were being sent but I want to know the maximum capacity of the server.I need to sent 150 requests per second, So I added Constant Throughput Timer with Target Throughput 9000 Constant Throughput Timer Screenshot .But it is still sending an average of 55 requests/ second. Here is the screenshot of command line run output. Output
Can anyone help me understand what I am doing wrong here ? Which values should I change in order to acheive 150 requests per second ?
JMeter sends request, WAIT for answer, send another request
If you add timer, it basically add some sleep to this -> you can't increase throughput, only decrease. Check manual https://jmeter.apache.org/usermanual/component_reference.html#Constant_Throughput_Timer
You should check your server - Load, CPU, Memory, IOs - disks, network ...
Check your JMeter machine as well
Are the answers from server alright? Do you check e.g. HTTP 200 OK?
You can try add more ramp up time, inscrease/decrease user count, but first check your server!
You're trying to reach 150 requests per second with each thread, you need to amend at least this bit:
You're trying to reach 150 requests per second with 10 users which means that each user needs to execute 15 requests per second. It's only possible if response time of your application is 66 milliseconds or less and the average response time of your application is around 200 milliseconds so it explains the actual result
JMeter's Constant Throughput Timer can only pause the threads in order to limit requests execution rate to the desired value, it won't kick off extra threads if current amount is not sufficient.
So the options are in:
Increase the number of threads in the Thread Group to i.e. 50 (you may need even more)
Or consider switching to Throughput Shaping Timer and Concurrency Thread Group combination, the Throughput Shaping Timer is more "precise" and it can be connected to the Concurrency Thread Group via Feedback Function so you will be able to kick off more threads to reach and maintain the desired throughput
I write a JMeter test and use 1000 threads, and get a throughput of 330 requests per second. What was the average response time?
same test in number 2 and I use 100 threads and get a throughput of 330 requests per second. What was the average response time?
I think it has to do with little law, but I have no idea how to solve it? Any help, thanks.
We don't know, in order to determine the average response time we need to know your test duration
JMeter calculates average response time as arithmetic mean or the all response times for individual samplers, it can be observed in i.e. Aggregate Report listener.
Also the fact you have the same throughput for 100 and 1000 users looks utterly suspicious, for well-behaved application you should get 10x times more throughput for 1000 users than for 100 users.
The reasons could be in:
Your application cannot handle more than 330 requests per second which indicates a performance bottleneck
JMeter cannot produce more than 330 requests per second, make sure to follow JMeter Best Practices or consider Distributed Testing if your load generator hardware specifications are too low to produce the required load.
Can there be systems where it has high throughput but also high response times. I believe it can be possible. Consider the system which gets 100 hits
per second(HPS) and response time is 2 sec. It generates the throughput of 100 QPS. So we can say it is high throughput(100 QPS) system but low response time(2 sec).
Is that correct ?
But at the same time if throughput(say 80) is lesser than HPS(say 100) then it means sometime down the line system will choke as outlet rate is lesser than inlet rate . Right ?
Now other way round, can there be system with low response time and also low throughput ? I believe that is also possible but not able to think a
release scenario?
So bottom line is we have to look both stats together i.e. throughput and response time as it is not mandatory if one is good another is also good at same
time?
In ideal world it would be
Response time is always the same
Throughput increased by the same factor as load increases, for example:
with 100 virtual users you have 100 requests per second
with 200 virtual users you have 200 requests per second
etc.
In real world at some point when you increase the load throughput will not increase, instead of it response time will increase hence the system will not be able to serve 200 users as fast as 100 users. This is known as performance bottleneck
More information:
What is the Relationship Between Users and Hits Per Second?
JMeter Glossary
When i carryout load testing in jmeter i have list of samples. Each sample returns its own throughput. However in the aggregated graph or summary result it has a total row at the bottom and adds all the throughput. What does this signify?
Can i just use the total throughput as the total throughput of the entire test run. Why does summary report adds up all the sample throughput rather than showing the average throughput?
In the following picture i ran load test with 2 user and 2 ramp up time.
As shown above the total actually sums up the throughput rather than aggregating it.
However, when i carry out test with 1 user and 1 ramp up time then it aggregates the throughput and displays the average throughput of the samplers.
In the below figure i carried out test with 1 thread and 1 ramp up time.
IS this a bug?
No, It's not a bug!!!
The Throughput is the number of requests per unit of time (seconds, minutes, hours) that are sent to your server during the test.
The Throughput is the real load processed by your server (Application under test) during a run but it does not tell you anything about the performance of your server during this same run.
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So, In your case For 2 User : The application handled 10.7 requests/second.
And For the Single user : The Application handled 22.9 requests/second.
Its not sums up here, If you see in your screenshot the sum is coming around 14.4/sec, Sot its not sum of all throughput. Its calculated value based on the load you had given and your application would support mentioned throughput.
In your case if a user accesses the application it supports 22.9 requests per sec. But when two users access the application then it supports 10.7 requests per sec.
Please look at here for more info about Jmeter throughput
Jmeter aggregate report total throughput - how is calculated
In case of performance testing, average is something we all avoid.
Going back to actual question. Consider you have 5 requests in one workflow. You are running this test for 50 iterations. So making 250 requests during the load test.
Now you want to analyze individual request performance as well as overall system performance. In this case, when you want to drill down and look at individual request in order to find bottlenecks, you look at the throughput and response time of request.
If you want to find the overall load your system can handle, look at the total throughput.