I use JMeter to test my webapp application, I have aggregate graph with some score values, but I actually don't know what they mean...
Aggregate graph shows for example:
average
median
min
max
I don't know about what refer that values.
For what refer 90% line?
I also don't know what's the unit of throughput per second (bytes?).
Anybody knows?
JMeter documentation shows only general information about reports and listeners.
This link contains a helpful explanation for Jmeter Usuage, Results, Tips, Considerations and Deployment.
Good Luck!
Throughput - means number of requests per one second. So if two users open your website at the same time throughput will be 2/s - 2 requests in one second.
How it can be useful: check your website analytics and you see number of hosts and hits per one day. Throughput stands for hits per day. If analytics shows 200 000 hits per day this means: 200 000 / 86400 (seconds in one day) = 2,31 hits/s.
Average - the average of response time. I think you know what is response time - it's time between sending request and getting response from server. To get the average response time you should sum all samplers response time and devide to number of samplers. Sampler means user, request, hit, the meaning is the same.
Min - the minimal response time in all samplers. Differently we may say the fastest response.
Max - opposite of Min, the slowest response.
The throughput is generally measured in requests/second in things Jmeter.
As far as knowing what requests are within the 90% line, not really a way to do it with this listener. This is representing aggregate information, so it only reports on information about all the tests, not specific results.
For some different methods and ideas on getting useful information out of the responses, take a look at this jmeter wiki on log analysis.
If you don't already have it, jmeter plugins has a lot of useful controllers and listeners that can make understanding the results easier as well.
Related
I am testing a scenario with 400 threads. Although I am almost getting no errors, I have very high average response. What can bring about this problem? Seems like server gives no time-out but gives response so late. I've addded the summary report. It is as follows:
This table doesn't tell the full story, if response time seems "so high" to you - this is definitely the bottleneck and you can report it already.
What you can do to localize the problem is:
Consider using a longer ramp-up period, i.e. start with 1 user and add 1 more user every 5 seconds (adjust these numbers according to your scenario) so you would have arrival phase, the "plateau" and the load decrease phase. This approach will allow you to correlate increasing load and increasing response time by looking at Active Threads Over Time and Response Times Over Time charts. This way you will be able to state that:
response time remains the same up to X concurrent users
after X concurrent users it starts growing so throughput is going down
after Z concurrent users response time exceeds acceptable threshold
It would also be good to see CPU, RAM, etc. usage on the server side as increased response time might be due to lack of resources, you can use JMeter PerfMon Plugin for this
Inspect your server configuration as you might need to tune it for high loads (same applies to JMeter, make sure to follow JMeter Best Practices)
Use a profiler tool on server side during the next test execution, it will show you the slowest places in your application code
I am running my jmeter script for almost a week and observed an interesting thing today. Below is the scenario:
Overview: I am gradually increasing the load on the application. In my last test I gave load of 100 users on the app and today I increased the load to 150 users.
Result of 150 users test:
Response time of the requests decreased compared to the last test. (Which is a good sign)
Throughput decreased drastically to half of what I got in the previous test with less load.
Received 225 errors while executing the test.
My questions are:
What could be the possible reason for such strange behavior of throughput? Why did throughput decrease instead of increasing with the increasing load?
Did I get good response time as many of my requests failed?
NOTE: Till 100 users test throughput was increasing with the increasing load of users.
Can anyone please help me with this question. I am a new bee in performance testing. Thanks in Advance!!
Also, would like to request if anyone can suggest good articles/site etc on finding performance bottleneck and learning crucial things in performance.
Most probably these 225 requests which failed returned failure immediately therefore average response time decreased, that's why you should be looking into i.e. Response Times Over Time chart and pay more attention to percentiles as mean response time can mask the real problem.
With regards to the bottleneck discovery, make sure to collect as much information from the server side as you can, i.e.
CPU, RAM, Network, Disk usage from JMeter PerfMon Plugin
Slow queries log from the database
"heaviest" functions and largest objects from the profiling tool for your application
I write a JMeter test and use 1000 threads, and get a throughput of 330 requests per second. What was the average response time?
same test in number 2 and I use 100 threads and get a throughput of 330 requests per second. What was the average response time?
I think it has to do with little law, but I have no idea how to solve it? Any help, thanks.
We don't know, in order to determine the average response time we need to know your test duration
JMeter calculates average response time as arithmetic mean or the all response times for individual samplers, it can be observed in i.e. Aggregate Report listener.
Also the fact you have the same throughput for 100 and 1000 users looks utterly suspicious, for well-behaved application you should get 10x times more throughput for 1000 users than for 100 users.
The reasons could be in:
Your application cannot handle more than 330 requests per second which indicates a performance bottleneck
JMeter cannot produce more than 330 requests per second, make sure to follow JMeter Best Practices or consider Distributed Testing if your load generator hardware specifications are too low to produce the required load.
I ran a JMeter test for 193 samples
where I could see my average response time as 5915ms and Throghput as 1.19832.
I just want to know how are they exactly related
All the answers are in JMeter Glossary
Elapsed time. JMeter measures the elapsed time from just before sending the request to just after the last response has been received.
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
The relationship is: higher response time - lower throughput and vice versa.
You can use charts like Transactions per Second for throughput and Response Times Over Time for response times to get them plotted on your test timeline and Composite Graph to put them together. This way you will be able to track the trends.
All 3 charts can be installed using JMeter Plugins Manager
TL;DR
No, but yes.
Both aren't related directly, but when increasing Throughput, it will probably affect server response time due to load/stress on server.
If there are timeout errors response time will probably increase.
But for validation or firewall errors - response time will probably decrease.
There's a long explanation in JMeter archive, last is using Disney to demonstrate:
Think of your last trip to disney or your favorite amusement park. Lets define capacity of the ride to be the number of people that can sit on the ride per turn (think roller coaster). Throughput will be the number of people that exit the ride per unit of time. Lets define service time the the amount of time you get to sit on the ride. Lets define response time or latency to be your time queuing for the ride (dead time) plus service time.
In terms of load/Performance testing. Throughput and Response times are inversely proportional. i.e
With increase in response time throughput should decrease.
With increase in Throughput response time should decrease.
You can get more detailed definitions in this blog:
https://nirajrules.wordpress.com/2009/09/17/measuring-performance-response-vs-latency-vs-throughput-vs-load-vs-scalability-vs-stress-vs-robustness/
Throughout increases to some extent and remains stable when all the resources becomes busy. Now, if user requests increases further at this point response time would increase. But if response time increase is only because of internal queuing then due to the fact that system is taking more requests in at the same time response time is also increasing, throughout doesn't change. When queues are full more requests should fail. If response increase is due to some delay in processing or serving the request, for example running a query on database then due to the fact that system is not accepting more request and at the same time response time is also increasing, consequently throughout would drop.
Just a general explaination.
Respose Time : It is the time calculated when user send the request till request gets finnished.
Throughput : It is server property that number of transaction or request can be made during certain amount of time. here 1.19832 /minute means server cand hadle 1.19832 sample per minute.
As Respose Time increses Throughput increases.
I received an requirement were I need to display the response time, number of threads running, latency and throughput in one report. I used below code in Beanshell post processor to display throughput, response time and number of threads:
long repons=prev.getTime();
vars.put("responseTime",String.valueOf(recons));
//print("res" +responseTime);
log.info("Response time" + repons);
long thread=prev.getAllThreads();
vars.put("threads", Integer.toString(prev.getAllThreads()));
log.info("Thread number is"+thread);
float throughput=thread/repons;
log.info("Through put"+throughput);
I guess it is wrong. Can anyone help on this?
You have syntax error in your script, you have repons in the first line and recons in the second, they should be the same
It is better to use JSR223 Elements and Groovy language for scripting.
And finally, your approach is wrong, according to JMeter glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So you need to divide total number of requests by total time taken to
execute these requests, your "code" most likely will be returning zero throughput
You can consider the following workarounds:
Use Backend Listener and a 3rd-party visualisation tool, see Real-time results article for details.
Run your JMeter test via Taurus framework which has Interactive Reporting feature