One of my clients is asking about the expected TPS before the execution. He has given the below requirements to initiate the load test,
The expected hourly active user count: 14250 (concurrent)
The total think time: 625 Seconds (4 thread groups with different user flows. each flow action controller has 5 Seconds delay)
No Pace time: 0
Total Number of endpoints: 212 (4 thread groups with different user flows)
Can anyone help me to calculate the approximate TPS (Transactions per second) value that we reach for the above configuration?
Is this data set will sufficient to do this calculation? Appreciate it if anyone can help with this calculation.
It's not sufficient because you don't know the response time for all 212 endpoints, you can assume what could be the TPS if response time would be 1 second or 2 seconds, but it's not possible to predict the actual TPS without knowing what is the response time.
According to JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
More information: How do I Correlate the Number of (Concurrent) Users with Hits Per Second
Related
I am in confusion here for what is the right parameter to find how many requests my service can handle in a sec..
Eg: According to docs & this post TPS(transactions/sec) is calculated based on elapsed time of the request which seems to be fair when you have one service instance. Eg: My elapsed time is 1 second so my tps is 1 which makes sense, but the calculations fail when i have 3 service instance(H-Scaled) though the elapsed time remains the same but now i can process 3 concurrent requests in that same second which should ideally read back as 3 tps but it doesnt
Q:Then what is the right parameter in jmeter report to check for this ? or is my theory wrong?
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
And request is something produced by JMeter's Sampler
If you're doing some scalability testing you can measure it as follows:
Run a stress test with 1 service instance, i.e. start with 1 user and gradually increase the load at the same time looking at TPS. At some point you will reach the stage where increasing the number of users won't result in increased TPS due to some bottleneck. Measure the number of users and the TPS just before the bottleneck hits you.
Re-run your test with 3 service instances, you should see that the number of users and TPS before the bottleneck is higher now.
I discovered, that in Aggregate Report TOTAL THROUGHPUT value depends on thread count. And if we run tests with only one thread, total throughput is calculated as 1 / Total Average (and multiplied by 1000 to convert milliseconds to seconds, see the screenshot below).
But when we set thread count to 2 or more, total throughput is calculated the unknown way, so what I want to know is which formula is used when calculating total throughput in this case (thread count > 1), because it does not seem to be an average of all requests throughput, it's also not calculated as 1 / Total Average like described in the first case. So how exactly does this work? (Screenshot for 2 threads attached below)
Thanks.
Screenshot for 1 thread used:
aggregate_1_thread.png
Screenshot for 2 threads used:
aggregate_2_threads.png
As per doc:
http://jmeter.apache.org/usermanual/component_reference.html#Aggregate_Report
Throughput - the Throughput is measured in requests per second/minute/hour. The time unit is chosen so that the displayed rate is at least 1.0. When the throughput is saved to a CSV file, it is expressed in requests/second, i.e. 30.0 requests/minute is saved as 0.5.
So result depends both on Response time and Number of threads which influences those response times.
The total number of requests is divided by the time taken to run them, see:
https://github.com/apache/jmeter/blob/trunk/src/core/org/apache/jmeter/visualizers/SamplingStatCalculator.java#L198
I have about 300 users (configured in the thread group) who would perform an activity (e.g.: run an e-learning course) twice. That would mean I need to expect about 600 iterations i.e 300 users performing an activity twice.
My thread group contains the following transaction controllers:
Login
Dashboard
Launch Course
Complete Course
Logout
As I need 600 iterations per 5400 seconds i.e 3600 + 900 + 900 seconds (1 hour steady state + 15 mins ramp-up and 15 mins ramp-down), and the sum of sampler requests within the total thread group are 18, would I be correct to say I need about 2 RPS?
Total number of iterations * number of requests per iteration = Total number of requests
600 * 18 = 10800
Total number of requests / Total test duration in seconds = Requests per second
10800 / 5400 = 2
Are my calculations correct?
In addition, what is the best approach to achieve the expected throughput?
Your calculation looks more or less correct. If you need to limit your test throughput to 2 RPS you can do it using Constant Throughput Timer or Throughput Shaping Timer.
However 2 RPS is nothing more than statistical noise, my expectation is that you need much higher load to really test your application performance, i.e.
Simulate the anticipated number of users for a short period. Don't care about iterations, just let your test to run i.e. for an hour with the number of users you expect. This is called load testing
Do the same but for longer period of time (i.e. overnight or weekend). This is called soak testing.
Gradually increase the number of users until you will see errors or response time will start exceeding acceptable thresholds. This is called stress testing.
I have one observation and want to get knowledge on Throughput calculation,Some time Throughput is displaying in seconds,some times in minutes and some times in Hours,please any one provide exact answer to calculate throughput and when it will display in Seconds,Minutes and Hours in Jmeter Summary Report
From JMeter Docs:
Throughput is calculated as requests/unit of time. The time is
calculated from the start of the first sample to the end of the last
sample. This includes any intervals between samples, as it is supposed
to represent the load on the server. The formula is: Throughput =
(number of requests) / (total time).
unit time varies based on the throughput values.
examples:
In 10 seconds, 10 requests are sent, then throughput is 10/10 = 1/sec
In 10 seconds, 1 requests are sent, then throughput is 1/10 = 0.1/sec = 6/min (showing 0.1/sec in decimals will be automatically shown in next higher unit time)
If you understand, it is to avoid small values (like, 0.1, 0.001 etc). In such cases, higher unit time is more friendly in understanding, while all unit times are correct. It is a matter of usability.
so,
1/sec = 60/min = 3600/hour = SAME
When i carryout load testing in jmeter i have list of samples. Each sample returns its own throughput. However in the aggregated graph or summary result it has a total row at the bottom and adds all the throughput. What does this signify?
Can i just use the total throughput as the total throughput of the entire test run. Why does summary report adds up all the sample throughput rather than showing the average throughput?
In the following picture i ran load test with 2 user and 2 ramp up time.
As shown above the total actually sums up the throughput rather than aggregating it.
However, when i carry out test with 1 user and 1 ramp up time then it aggregates the throughput and displays the average throughput of the samplers.
In the below figure i carried out test with 1 thread and 1 ramp up time.
IS this a bug?
No, It's not a bug!!!
The Throughput is the number of requests per unit of time (seconds, minutes, hours) that are sent to your server during the test.
The Throughput is the real load processed by your server (Application under test) during a run but it does not tell you anything about the performance of your server during this same run.
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So, In your case For 2 User : The application handled 10.7 requests/second.
And For the Single user : The Application handled 22.9 requests/second.
Its not sums up here, If you see in your screenshot the sum is coming around 14.4/sec, Sot its not sum of all throughput. Its calculated value based on the load you had given and your application would support mentioned throughput.
In your case if a user accesses the application it supports 22.9 requests per sec. But when two users access the application then it supports 10.7 requests per sec.
Please look at here for more info about Jmeter throughput
Jmeter aggregate report total throughput - how is calculated
In case of performance testing, average is something we all avoid.
Going back to actual question. Consider you have 5 requests in one workflow. You are running this test for 50 iterations. So making 250 requests during the load test.
Now you want to analyze individual request performance as well as overall system performance. In this case, when you want to drill down and look at individual request in order to find bottlenecks, you look at the throughput and response time of request.
If you want to find the overall load your system can handle, look at the total throughput.