Can there be systems where it has high throughput but also high response times. I believe it can be possible. Consider the system which gets 100 hits
per second(HPS) and response time is 2 sec. It generates the throughput of 100 QPS. So we can say it is high throughput(100 QPS) system but low response time(2 sec).
Is that correct ?
But at the same time if throughput(say 80) is lesser than HPS(say 100) then it means sometime down the line system will choke as outlet rate is lesser than inlet rate . Right ?
Now other way round, can there be system with low response time and also low throughput ? I believe that is also possible but not able to think a
release scenario?
So bottom line is we have to look both stats together i.e. throughput and response time as it is not mandatory if one is good another is also good at same
time?
In ideal world it would be
Response time is always the same
Throughput increased by the same factor as load increases, for example:
with 100 virtual users you have 100 requests per second
with 200 virtual users you have 200 requests per second
etc.
In real world at some point when you increase the load throughput will not increase, instead of it response time will increase hence the system will not be able to serve 200 users as fast as 100 users. This is known as performance bottleneck
More information:
What is the Relationship Between Users and Hits Per Second?
JMeter Glossary
Related
I write a JMeter test and use 1000 threads, and get a throughput of 330 requests per second. What was the average response time?
same test in number 2 and I use 100 threads and get a throughput of 330 requests per second. What was the average response time?
I think it has to do with little law, but I have no idea how to solve it? Any help, thanks.
We don't know, in order to determine the average response time we need to know your test duration
JMeter calculates average response time as arithmetic mean or the all response times for individual samplers, it can be observed in i.e. Aggregate Report listener.
Also the fact you have the same throughput for 100 and 1000 users looks utterly suspicious, for well-behaved application you should get 10x times more throughput for 1000 users than for 100 users.
The reasons could be in:
Your application cannot handle more than 330 requests per second which indicates a performance bottleneck
JMeter cannot produce more than 330 requests per second, make sure to follow JMeter Best Practices or consider Distributed Testing if your load generator hardware specifications are too low to produce the required load.
Please help me about below summary report.what a server performance if throughput is in hrs.
Throughput comes in hrs what is mean that server performance is good or not?
Throughput - the Throughput is measured in requests per second/minute/hour.
The time unit is chosen so that the displayed rate is at least 1.0.
When you have throughput in hours means the throughput is low.
If you have not controlled the throughput explicitly , it means that server+network performance is low with respect to your JMeter client.
Note
When the throughput is saved to a CSV file, it is expressed in requests/second, i.e. 30.0 requests/minute is saved as 0.5.
I am trying to stress test my server.
To do so I am using Jmeter and here is my set up:
I use
my Setup
Thread: 1000
schedule for 3 mins
So as you see I keep going with 1000 thread for a period of 3 mins.
But when I look at the throughput I only get around 230 per second
results
So what should I do to increase the through put to for example 1000000 per second? How come increasing the thread which I assume means more load does not increase throughput?
According to JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
Throughput explicitly relies on the application response time. Looking into your results, the average response time is 3.5 seconds therefore you will not get more than 1000 / 3.5 = 285 requests per second
Theoretically you could use Throughput Shaping Timer and Concurrency Thread Group combination, this way JMeter will kick off extra threads if the current amount is not enough to reach/maintain the desired throughput, however looking into 8.5% error rate and maximum response time for your application > 2 minutes my expectation is that you will not be able to get more throughput because most probably your application is overloaded and cannot respond faster.
Throughput measures the number of transactions or requests that can be made in a given period of time. basically, it lists the number of requests server managed to serve in a given time period. Throughput value depends on lot of factors and maybe your application under test not able to cater the expected load.
So with 1000 threads, you can't expect a 1000 throughput.
It's up to you to find out how much throughput your application can handle. For that maybe you need to do different optimizations on your side like optimize your script, distribute load via JMeter execution, increase theard count,...etc
Do we need to adjust Throughput given by jmeter, to find out the actual tps of the system
For eg : I am getting 100 tps for concurrent 250 users. This ran for 10 hrs. Can I come to a conclusion like my software can handle 100 transactions per second. Or else do I need to do some adjustment and need to get a value. Why i am asking this because when load started, system will take sometime to perform in adequate level (warm up time). If so how to do this. Please help me to understand this.
By default JMeter sends requests as fast as it can, the main factor which are affecting TPS rate are:
number of threads (virtual users) - this you can define in Thread Group
your application response time - this is not something you can control
Ideally when you increase number of threads the number of TPS should increase by the same factor, i.e. if you have 250 users and getting 100 tps you should get 200 tps for 500 users. If this is not the case - these 500 users are beyond saturation point and your application bottleneck is somewhere between 250 and 500 users (if not earlier).
With regards to "warm up" time - the recommended approach of conducting the load is doing it gradually, this way you will allow your application to get prepared to increasing load, warm up caches, let JIT compiler/optimizer to go their work, etc. Moreover this way you will be able to correlate the increasing load with increasing/decreasing throughput, response time, number of errors, etc. while having 250 users released at once doesn't tell the full story. See
The system warmup period varies from one system to the other. Warm up period is where configurations are cached, different libraries are initialized (eg. Builder.init()) and other initial functions that usually don't happen for subsequent calls. If you study results of the load test, there is a slow period at the very beginning. For most systems, it could be as small as 5 to 10 minutes. These values could be even negligible if the test is as long as 10 hours. But then again, average calculation can be effected if the results give extremely low values at the start (it always depend on the jump from initial warming up period to normal operations).
As per jmeter configurations this thread may explain the configuration. How to exclude warmup time from JMeter summary?
When i carryout load testing in jmeter i have list of samples. Each sample returns its own throughput. However in the aggregated graph or summary result it has a total row at the bottom and adds all the throughput. What does this signify?
Can i just use the total throughput as the total throughput of the entire test run. Why does summary report adds up all the sample throughput rather than showing the average throughput?
In the following picture i ran load test with 2 user and 2 ramp up time.
As shown above the total actually sums up the throughput rather than aggregating it.
However, when i carry out test with 1 user and 1 ramp up time then it aggregates the throughput and displays the average throughput of the samplers.
In the below figure i carried out test with 1 thread and 1 ramp up time.
IS this a bug?
No, It's not a bug!!!
The Throughput is the number of requests per unit of time (seconds, minutes, hours) that are sent to your server during the test.
The Throughput is the real load processed by your server (Application under test) during a run but it does not tell you anything about the performance of your server during this same run.
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So, In your case For 2 User : The application handled 10.7 requests/second.
And For the Single user : The Application handled 22.9 requests/second.
Its not sums up here, If you see in your screenshot the sum is coming around 14.4/sec, Sot its not sum of all throughput. Its calculated value based on the load you had given and your application would support mentioned throughput.
In your case if a user accesses the application it supports 22.9 requests per sec. But when two users access the application then it supports 10.7 requests per sec.
Please look at here for more info about Jmeter throughput
Jmeter aggregate report total throughput - how is calculated
In case of performance testing, average is something we all avoid.
Going back to actual question. Consider you have 5 requests in one workflow. You are running this test for 50 iterations. So making 250 requests during the load test.
Now you want to analyze individual request performance as well as overall system performance. In this case, when you want to drill down and look at individual request in order to find bottlenecks, you look at the throughput and response time of request.
If you want to find the overall load your system can handle, look at the total throughput.