Can anyone please explain like how to analyze jmeter's Summary Report?
Example:
Label : Login Action(sampler)
Sample# : 1
average: 104 // What does this mean actually?
min : 104 // What does this mean actually?
max : 104
stddev : 0 // What does this mean actually?
error% : 0
Throughput : 9.615384615 // What does this mean actually?
Kb/Sec : 91.74053486 // What does this mean actually?
Average Bytes : 9770 // What does this mean actually?
It is pretty straightforward:
Average, min and max is the response times for the request in milliseconds. The response time os from the request is sent to the response is received. Since you have only one request they are of course all equal.
stddev is a measure of the variation of the response times: http://en.wikipedia.org/wiki/Standard_deviation.
Throughput is number of requests per second. With a average response time a little over 100ms the throughput is a little below 10.
Kb/Sec is the number of kilobytes transferred per second. It is Average Bytes (per request) * Throughput. I am not sure if the average bytes is for only the response or both the request and the response. My guess is that it is the latter. It is the latter: response headers and body.
I have just found the very nice and simple explanation here:
http://jmeterresults.blogspot.jp/2012/07/jmeterunderstanding-summary-report.html
Related
can someone explain what does the MAX statistic refers to in the below response. I don't see it documented anywhere.
localhost:8081/actuator/metrics/http.server.requests?tag=uri:/myControllerMethod
Response:
{
"name":"http.server.requests",
"description":null,
"baseUnit":"milliseconds",
"measurements":[
{
"statistic":"COUNT",
"value":13
},
{
"statistic":"TOTAL_TIME",
"value":57.430899
},
{
"statistic":"MAX",
"value":0
}
],
"availableTags":[
{
"tag":"exception",
"values":[
"None"
]
},
{
"tag":"method",
"values":[
"GET"
]
},
{
"tag":"outcome",
"values":[
"SUCCESS"
]
},
{
"tag":"status",
"values":[
"200"
]
},
{
"tag":"commonTag",
"values":[
"somePrefix"
]
}
]
}
You can see the individual metrics by using ?tag=url:{endpoint_tag} as defined in the response of the root /actuator/metrics/http.server.requests call. The details of the measurements values are;
COUNT: Rate per second for calls.
TOTAL_TIME: The sum of the times recorded. Reported in the monitoring system's base unit of time
MAX: The maximum amount recorded. When this represents a time, it is reported in the monitoring system's base unit of time.
As given here, also here.
The discrepancies you are seeing is due to the presence of a timer. Meaning after some time currently defined MAX value for any tagged metric can be reset back to 0. Can you add some new calls to /myControllerMethod then immediately do a call to /actuator/metrics/http.server.requests to see a non-zero MAX value for given tag?
This is due to the idea behind getting MAX metric for each smaller period. When you are seeing these metrics, you will be able to get an array of MAX values rather than a single value for a long period of time.
You can get to see this in action within Micrometer source code. There is a rotate() method focused on resetting the MAX value to create above described behaviour.
You can see this is called for every poll() call, which is triggered every some period for metric gathering.
What does MAX represent
MAX represents the maximum time taken to execute endpoint.
Analysis for /user/asset/getAllAssets
COUNT TOTAL_TIME MAX
5 115 17
6 122 17 (Execution Time = 122 - 115 = 17)
7 131 17 (Execution Time = 131 - 122 = 17)
8 187 56 (Execution Time = 187 - 131 = 56)
9 204 56 From Now MAX will be 56 (Execution Time = 204 - 187 = 17)
Will MAX be 0 if we have less number of request (or 1 request) to the particular endpoint?
No number of request for particular endPoint does not affect the MAX (see Image from Spring Boot Admin)
When MAX will be 0
There is Timer which set the value 0. When the endpoint is not being called or executed for sometime Timer sets MAX to 0. Here approximate timer value is 2 minutes (120 seconds)
DistributionStatisticConfig has .expiry(Duration.ofMinutes(2)).
which sets some measurements to 0 if there is no request has been made in between expiry time or rotate time.
How I have determined the timer value?
For that, I have taken 6 samples (executed the same endpoint for 6 times). For that, I have determined the time difference between the time of calling the endpoint - time for when MAX set back to zero
More Details
UPDATE
Document has been updated.
NOTE:
Max for basic DistributionSummary implementations such as CumulativeDistributionSummary, StepDistributionSummary is a time
window max (TimeWindowMax).
It means that its value is the maximum value during a time window.
If the time window ends, it'll be reset to 0 and a new time window starts again.
Time window size will be the step size of the meter registry unless expiry in DistributionStatisticConfig is set to other value
explicitly.
How to create thread delay in seconds based on the calculation(duration/throughput) using JSR223 Timer. what should I write inside the script section ?We are having the Duration and throughput as Jmeter parameters in our Test Plan
The easiest way is to use Precise Throughput Timer or Throughput Shaping Timer - both can be configured with combination of the desired throughput and test duration. The timers are smart enough in order to pause JMeter threads to reach the needed throughput.
If for some reason the above timers are not suitable you can consider implementing so called Pacing - a dynamic delay between Thread Group iterations.
The example code would be something like:
//Sets the pacing length based on the last requests response time. 4500 is the time in ms
Long pacing = 4500 - prev.getTime();
//If the response time is less than 4500 ms, set the delay value to myDelay
if ( pacing > 0 )
{
//iPacing is equal to the int value of pacing if pacing is not equal to null, otherwise iPacing is null
Integer iPacing = pacing != null ? pacing.intValue() : null;
log.info(String.valueOf(iPacing));
vars.put("myDelay", String.valueOf(iPacing));
return iPacing;
}
//The response time is greater than or equal to 4500 ms, set myDelay to 0
else
{
vars.put("myDelay", "0");
return 0;
}
While I'm doing load testing of the golang api there is a report is generated but I don't know what is it and how to read it:-
I run the command in terminal
echo "GET http://localhost:8080/api" | vegeta attack -rate=100/m | vegeta report
then it will produce a below report:-
Requests [total, rate] 138, 1.68
Duration [total, attack, wait] 1m22.20931745s, 1m22.200130205s, 9.187245ms
Latencies [mean, 50, 95, 99, max] 8.956174ms, 9.06458ms, 10.682252ms, 16.007578ms, 46.439935ms
Bytes In [total, mean] 19596, 142.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:138
Error Set:
or when I run the echo "GET http://localhost:8080/api" | vegeta attack -rate=100/m | vegeta report -type=json
then the report generated in json format like below:-
{"latencies:
{"total":103506418,
"mean":9409674,
"50th":9484403,
"95th":11918898,
"99th":12008257,
"max":12008257},
"bytes_in":{"total":1562,"mean":142},
"bytes_out":
{"total":0,"mean":0},
"earliest":"2018-10-16T14:15:13.251091124+05:30",
"latest":"2018-10-16T14:15:19.251141502+05:30",
"end":"2018-10-16T14:15:19.260119671+05:30",
"duration":6000050378,
"wait":8978169,
"requests":11,
"rate":1.8333179401848014,
"success":1,
"status_codes":{"200":11},
"errors":[]}
How to understand this report. Is there any document for this or anybody knows about it?
Let's understand it line by line
Requests [total, rate] 138, 1.68
This line prints the total number of requests fired in the session (138) along with the rate per second (1.8 requests per second)
Duration [total, attack, wait] 1m22.20931745s, 1m22.200130205s, 9.187245ms
Total Duration of the attack which should be sum of time spent in requests and time spent in waiting for response
Latencies [mean, 50, 95, 99, max] 8.956174ms, 9.06458ms, 10.682252ms, 16.007578ms, 46.439935ms
This is simple and most useful : mean latency in milliseconds, 50th percentile, 95th percentile and 99th percile latency along with the request which took max latency
99th percentile latency would mean 99% of responses were served within this time
Depending on your product, you should consider 95th or 99th as the real number to improve
Bytes In [total, mean] 19596, 142.00
Total bytes received for all response as well as the mean bytes per response
Bytes Out [total, mean] 0, 0.00
Total bytes sent for all requests as well as the mean bytes per request. Since you are using GET which does not contain any payload it is 0 for you
Success [ratio] 100.00%
Success percentage : 100% of your requests were successful
Status Codes [code:count] 200:138
Status code division by response code: In your case all 138 requests responded with 200 response
Error Set:
Error code division: If there were any errors 400/500s , it would be reported here. This is empty as you have 100% success rate
is the total throughput shown in last row in Summary Report correct ? I m using Jmeter 2.11
I find it difficult to match the displayed figure by manipulation.
I followed the formula (x/sec) : Number of request / Total response time required (in sec)
Or 1/Avg total response time (sec).
for example : 50 request taking avg response time as 2000 ms each then throughput = 50/(50*2) = 0.5/sec
But Jmeter shows different value than 0.5/sec or 30/min
Can someone help me here?
I was also having similar assumption. But this is the formula for calculating throughput.
endTime = lastSampleStartTime + lastSampleLoadTime
startTime = firstSampleStartTime
converstion = unit time conversion value
Throughput = Numrequests / ((endTime - startTime)*conversion)
(I got this few months back from the below answer)
Calculating throughput from Jmeter jtl log file
Can anyone please explain me how the throughput,Kb/sec and average bytes are measured in the Summary Report?
I got following Summary Report for login action.
Label : Login Action(sampler)
Sample# : 1
average: 104
min : 104
max : 104
stddev : 0
error% : 0
Throughput : 9.615384615
Kb/Sec : 91.74053486
Average Bytes : 9770
Could you please give me the detail explanation? I cant clear when gone through google.
Thanks in advance
Througput in KB/s = (Transaction Throughput* Average Bytes) / 1024
For example:
91.74053486 = (9.615384615 X 9770) / 1024
These values are detailed here. They're fairly self explanatory.