I have a JMeter test plan using which I am running 10 - 500 threads. Each thread submits a job. I am basically collecting results like 10 jobs and measure the latency of each job. I know summary report gives a nice report on throughput, but that report is not suitable for my test because my test plan has 1 POST plus 11 GET calls in it and the summary report gives throughout of each those calls. But I need to measure throughput for each 10 threads, 50 and 100 threads respectively. Could someone let me know how should I go this in JMeter or do I have to calculate manually? Note: I'm allowing 10sec ramp up time for 10 threads.
You are right: summary report gives you throughput for each call. To measure multiple calls at once, add them under Transaction controller. For example, say you want to measure overall throughput of all GETs at once, put them all under Transaction controller, but not the POST request, so summary graph will contain measurement for each of the GET requests, plus separate line which includes all of them at once.
Another (non-interactive) option is to save results as CSV file, including label and latency, and calculate throughput using Excel (or awk) from the file.
To measure for different number of users, you need to run test multiple times with that number of concurrent threads/users.
Related
I have created a performance test script as below. I am running 4 Thread Groups in parallel (Even though there are 14 Thread Groups, only 4 are enabled and I am running only those enabled 4 Thread Groups). I have used default Thread Groups.
I have used Flow Control Action to simulate the user think time and set it as 3 seconds.
My requirement is to achieve Throughput as 6.6/seconds. What is the best way to achieve it? Also, does user think time creates any impact on Throughput?
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So basically throughput is the number of requests which JMeter was able to within test duration time frame. If you introduce artificial delay of 3 seconds - it will make the overall throughput lower
Other (and the main factor) is your application response time because JMeter waits for previous request to finish before executing new one so there could be 2 options:
The amount of threads in your Thread Groups is not sufficient to create the desired throughput. If this is the case - just add more threads (virtual users) in Thread Group(s)
The amount of threads in your Thread Groups is too high and you're getting higher throughput that you expect. If this is the case you can pause JMeter even more by adding Constant Throughput Timer and specifying the desired number of the requests per minute you want to have. If you need 6.6 requests per second it will mean 396 requests per minute
Also remember that JMeter itself must be able to send requests fast enough so make sure to follow JMeter Best Practices
My suggestion is to consider using the Arrivals Thread Group. This TG will allow you to configure the desire average throughput (ATP); the TG will instantiate the required threads needed to achieve the ATP goal.
As for the Think Time (TT), it should be a business requirement that emulates the pauses that users take when using the application in the real world.
I am trying to stress test my server.
To do so I am using Jmeter and here is my set up:
I use
my Setup
Thread: 1000
schedule for 3 mins
So as you see I keep going with 1000 thread for a period of 3 mins.
But when I look at the throughput I only get around 230 per second
results
So what should I do to increase the through put to for example 1000000 per second? How come increasing the thread which I assume means more load does not increase throughput?
According to JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
Throughput explicitly relies on the application response time. Looking into your results, the average response time is 3.5 seconds therefore you will not get more than 1000 / 3.5 = 285 requests per second
Theoretically you could use Throughput Shaping Timer and Concurrency Thread Group combination, this way JMeter will kick off extra threads if the current amount is not enough to reach/maintain the desired throughput, however looking into 8.5% error rate and maximum response time for your application > 2 minutes my expectation is that you will not be able to get more throughput because most probably your application is overloaded and cannot respond faster.
Throughput measures the number of transactions or requests that can be made in a given period of time. basically, it lists the number of requests server managed to serve in a given time period. Throughput value depends on lot of factors and maybe your application under test not able to cater the expected load.
So with 1000 threads, you can't expect a 1000 throughput.
It's up to you to find out how much throughput your application can handle. For that maybe you need to do different optimizations on your side like optimize your script, distribute load via JMeter execution, increase theard count,...etc
When i carryout load testing in jmeter i have list of samples. Each sample returns its own throughput. However in the aggregated graph or summary result it has a total row at the bottom and adds all the throughput. What does this signify?
Can i just use the total throughput as the total throughput of the entire test run. Why does summary report adds up all the sample throughput rather than showing the average throughput?
In the following picture i ran load test with 2 user and 2 ramp up time.
As shown above the total actually sums up the throughput rather than aggregating it.
However, when i carry out test with 1 user and 1 ramp up time then it aggregates the throughput and displays the average throughput of the samplers.
In the below figure i carried out test with 1 thread and 1 ramp up time.
IS this a bug?
No, It's not a bug!!!
The Throughput is the number of requests per unit of time (seconds, minutes, hours) that are sent to your server during the test.
The Throughput is the real load processed by your server (Application under test) during a run but it does not tell you anything about the performance of your server during this same run.
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So, In your case For 2 User : The application handled 10.7 requests/second.
And For the Single user : The Application handled 22.9 requests/second.
Its not sums up here, If you see in your screenshot the sum is coming around 14.4/sec, Sot its not sum of all throughput. Its calculated value based on the load you had given and your application would support mentioned throughput.
In your case if a user accesses the application it supports 22.9 requests per sec. But when two users access the application then it supports 10.7 requests per sec.
Please look at here for more info about Jmeter throughput
Jmeter aggregate report total throughput - how is calculated
In case of performance testing, average is something we all avoid.
Going back to actual question. Consider you have 5 requests in one workflow. You are running this test for 50 iterations. So making 250 requests during the load test.
Now you want to analyze individual request performance as well as overall system performance. In this case, when you want to drill down and look at individual request in order to find bottlenecks, you look at the throughput and response time of request.
If you want to find the overall load your system can handle, look at the total throughput.
I want to test the capacity of the web app that can handle without break.how to get the average requests per second from the aggregate report.
Is throughput is equal to average requests per second ?
I don't really need Apache definition.please make it simple.
Number of threads : 25
ramp-up :1
Loop:10
I have 3 slaves.
samples:250
Avg:1594
Throughput:10.4
Your question contains the answer: Throughput:10.4
According to JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So my assumption is that you generate the load of 10 requests per second (it may be per minute however, I need to know how long did your 250 samples execute to tell for sure)
If you're interested in "requests per second over time" you can utilize Server Hits Per Second listener which is available via JMeter Plugins project.
Alternative option of visualizing your load test results is using Loadosophia.org cloud where you can upload your test result and see graphs, distributions, export load test report as PDF, etc.
I have started 300 requests per second by providing following values:
Number of Threads : 300
Tamp-up : 0
But I am getting following results:
summary + 55 in 21s
summary + 225 in 31.1s
summary = 280 in 31.1s
what different configurations would be needed to start all requests in one go?
IMHO, you have to take in account the time it takes to run the transaction itself.
I tend NOT to use synchronization, but to take measure on longer period of time, e.g. 15 minutes.
E.g., if your system is able to deliver one page in 2 seconds, you need to run AT LEAST 600 threads to deliver the throughput you want (probably more).
Also, remember that the time of a single page increase with the load, so a single measurement is not enough, AND take care of errors: you have to define an acceptable threshold for errors (e.g. 0.01%), and stop measurement when you go above that.
If you need to start all 300 requests at the same moment you need to use Synchronizing Timer
If you need provide constant load at 300 requests per second rate - you'll need Constant Throughput Timer
Two above would cover the majority of use cases, however if you want more control on your load pattern look into Throughput Shaping Timer (available through JMeter plugin)