I received a JMeter script from someone, a script that contains 3 Throughput Controllers. The strange part is that they have "Percent Executions" selected and summing the Throughput from each of the controllers I only reach 70 ( the First one has a throughput of 40, second one 20 and third one 10). The questions are:
1. Shouldn't their sum be 100 (100%) ?
2. If the sum is 70 and the first one has a throughput of 40, does it mean that the first throughput controller will run 40% of the sum of 70?
3. Could someone give me more details related to "Percent executions vs Total executions"?
-JMeter help doesn't help much.
The Throughput Controller can operate in 2 modes:
Total Executions :- defines how many times the child elements will be executed (absolute number).
(A). If “Per User” is unchecked - the underlying sampler(s) will be executed as many times as defined in the Throughput field. For instance, if you have 100 users, the throughput of 100 and the box is unchecked, the underlying test elements will be executed 100 times.
(B). If “Per User” is checked - the child sampler(s) will be executed as many times as defined in the Throughput field, multiplied by the “Number of Threads” in the current Thread Group. For instance, if you have 100 users, the throughput of 100 and the box is checked, underlying test elements will be executed 100,000 times.
Percent Executions:- the child elements will be executed according to the percentage of iterations (threads * loops) as defined in the Throughput” field.
All these information is quoted from BlazeMeter Blog about Using Throughput Controller with Different Execution Percentages. This blog will help you understand your test executions scenario as per your configuration of Throughput Controller inside the Test Script.
Related
In my test plan I want to send N requests per second (RPS), where:
20% from N I send to localhost:8888/api/journal (example address)
80% to localhost:8888/api/simulation
I use Throughput Controller to implement percentage and Constant Throughput Timer to send N RPS.
When my requests in single Thread Group - it works well. Test plan screen below:
But if I place my requests in separate Thread Groups - Throughput Controller doesn't work. Like this:
Throughput Controller config: Based on is Percent Execution and Throughput is 20.0.
Constant Throughput Timer config: Target Throughput is 600.0 and Calculate Throughput based on is all active threads.
2 similar Thread Groups: 4 threads, 1 rampup, infinity loop count, 5 seconds duration.
At first screen I get ~10 RPS and 80/20 percentage, at second the same ~10 RPS but 50/50
What's wrong with second configuration? Thanks.
jmx-file link - you can copy it (changed to the second test plan)
Summary of discussion in comments section.
This behavior seems unexpected and it's worth opening a bug with https://bz.apache.org/bugzilla/.
There are 2 workarounds:
Add additional request under Throughput Controller;
Set Loop Count to a finite number of iterations.
I am in confusion here for what is the right parameter to find how many requests my service can handle in a sec..
Eg: According to docs & this post TPS(transactions/sec) is calculated based on elapsed time of the request which seems to be fair when you have one service instance. Eg: My elapsed time is 1 second so my tps is 1 which makes sense, but the calculations fail when i have 3 service instance(H-Scaled) though the elapsed time remains the same but now i can process 3 concurrent requests in that same second which should ideally read back as 3 tps but it doesnt
Q:Then what is the right parameter in jmeter report to check for this ? or is my theory wrong?
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
And request is something produced by JMeter's Sampler
If you're doing some scalability testing you can measure it as follows:
Run a stress test with 1 service instance, i.e. start with 1 user and gradually increase the load at the same time looking at TPS. At some point you will reach the stage where increasing the number of users won't result in increased TPS due to some bottleneck. Measure the number of users and the TPS just before the bottleneck hits you.
Re-run your test with 3 service instances, you should see that the number of users and TPS before the bottleneck is higher now.
I need to divide the load on my application with some percentages i.e. Login Module - 60%, Accounts - 10%, Other Modules - 30%. After few research i find a option under Throughput Controller section in jmeter using which i can control these percentages. I find one checkbox there named "per user". Now i am not getting this check box.
As per blazemeter blog here i tried one scenario as below with "Per user" checkbox checked.
Select "Total Execution" from dropdown.
Marked Throughput as 40.
Threads used - 10, Loop count 1
Now, as per the blog specific transactions should execute for 400 times. but there was Zero execution for that transaction.
I tried another scenario with "Per user" checkbox checked.
Select "Total Execution" from dropdown.
Marked Throughput as 60.
Threads used - 10, Loop count 1
Now, as per the blog specific transactions should execute for 600 times. but it executed 10 times.
Can any experts out there share what am i doing wrong here? Or there is some more clarity needed how this checkbox works.
To understand the Throughput Controller (TC), just add one TC and one sampler (inside TC) and Aggregate Report in combination. then, play with all the parameters in the Throughput Controller.
From Official Documentation:
Total executions:
causes the controller to stop executing after a certain number of executions have occurred.
and
Per User: If checked, per user will cause the controller to calculate
whether it should execute on a per user (per thread) basis. If
unchecked, then the calculation will be global for all users. For
example, if using total execution mode, and uncheck "per user", then
the number given for throughput will be the total number of executions
made. If "per user" is checked, then the total number of executions
would be the number of users times the number given for throughput.
Read both the statements carefully multiple times.
In both the scenarios that you specified, you have maximum executions of 10. (Thread count * Loop Count). Though you specified, Total Executions as 40 or 60, First, you should provide more than 60, in order to see all those 40/60 iterations get executed. So, always specify more iterations (using Thread count & Loop Count) than the Total Executions.
You have to consider Percentage Executions instead of Total Executions to match your requirements. Again, I suggest to simulate for one sample and understand the behaviour by varying the percentages.
Following are some scenarios and expected behaviour (EB).
Scenario:1
Thread Group - 10, Loop Count - 1, Throughput - 40 (Total Executions), Per User - Checked.
EB: Sampler will run only 10 times.
Scenario:2
Thread Group - 40, Loop Count - 1, Throughput - 40 (Total Executions), Per User - Checked.
EB: Sampler will run only 40 times.
Scenario:3
Thread Group - 40, Loop Count - 1, Throughput - 40 (Total Executions), Per User - Unchecked.
EB: Sampler will run only 40 times.
Scenario:4
Thread Group - 100, Loop Count - 1, Throughput - 40 (Total Executions), Per User - Checked.
EB: Sampler will run only 100 times. calculated whether each user executed 40 times. As the limit is not reached, it is executes all 100 iterations.
Scenario:5
Thread Group - 100, Loop Count - 1, Throughput - 40 (Total Executions), Per User - Unchecked.
EB: Sampler will run only 40 times. calculated at the global level. As the sampler reached 40 times for all the threads, stops executing it.
Scenario:6
Thread Group - 100, Loop Count - 40, Throughput - 40 (Total Executions), Per User - Checked.
EB: Sampler will run 400 times (Each user -> 40 times, 100*40). calculated whether each user executed 40 times. Here, even each user limit is also reached and no more executions after 40.
Scenario:7
Thread Group - 100, Loop Count - 1, Throughput - 40 (Total Executions), Per User - Unchecked.
EB: Sampler will run only 40 times. calculated at the global level. As the sampler reached 40 times for all the threads, stops executing it.
I have a JMeter test plan using which I am running 10 - 500 threads. Each thread submits a job. I am basically collecting results like 10 jobs and measure the latency of each job. I know summary report gives a nice report on throughput, but that report is not suitable for my test because my test plan has 1 POST plus 11 GET calls in it and the summary report gives throughout of each those calls. But I need to measure throughput for each 10 threads, 50 and 100 threads respectively. Could someone let me know how should I go this in JMeter or do I have to calculate manually? Note: I'm allowing 10sec ramp up time for 10 threads.
You are right: summary report gives you throughput for each call. To measure multiple calls at once, add them under Transaction controller. For example, say you want to measure overall throughput of all GETs at once, put them all under Transaction controller, but not the POST request, so summary graph will contain measurement for each of the GET requests, plus separate line which includes all of them at once.
Another (non-interactive) option is to save results as CSV file, including label and latency, and calculate throughput using Excel (or awk) from the file.
To measure for different number of users, you need to run test multiple times with that number of concurrent threads/users.
When i carryout load testing in jmeter i have list of samples. Each sample returns its own throughput. However in the aggregated graph or summary result it has a total row at the bottom and adds all the throughput. What does this signify?
Can i just use the total throughput as the total throughput of the entire test run. Why does summary report adds up all the sample throughput rather than showing the average throughput?
In the following picture i ran load test with 2 user and 2 ramp up time.
As shown above the total actually sums up the throughput rather than aggregating it.
However, when i carry out test with 1 user and 1 ramp up time then it aggregates the throughput and displays the average throughput of the samplers.
In the below figure i carried out test with 1 thread and 1 ramp up time.
IS this a bug?
No, It's not a bug!!!
The Throughput is the number of requests per unit of time (seconds, minutes, hours) that are sent to your server during the test.
The Throughput is the real load processed by your server (Application under test) during a run but it does not tell you anything about the performance of your server during this same run.
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So, In your case For 2 User : The application handled 10.7 requests/second.
And For the Single user : The Application handled 22.9 requests/second.
Its not sums up here, If you see in your screenshot the sum is coming around 14.4/sec, Sot its not sum of all throughput. Its calculated value based on the load you had given and your application would support mentioned throughput.
In your case if a user accesses the application it supports 22.9 requests per sec. But when two users access the application then it supports 10.7 requests per sec.
Please look at here for more info about Jmeter throughput
Jmeter aggregate report total throughput - how is calculated
In case of performance testing, average is something we all avoid.
Going back to actual question. Consider you have 5 requests in one workflow. You are running this test for 50 iterations. So making 250 requests during the load test.
Now you want to analyze individual request performance as well as overall system performance. In this case, when you want to drill down and look at individual request in order to find bottlenecks, you look at the throughput and response time of request.
If you want to find the overall load your system can handle, look at the total throughput.