after running my performance script , the statistic report generated is quite confusing.
Response Time :6s minimum
90th pct:4.53s
95th pct:4.9s
How come minimum response time is greater then 90th and 95th percentile , am I doing something wrong here ?
It should not be the case, the relevant code lives under StatCalculator.java class
If you do believe that this is a JMeter bug - you should raise an issue in JMeter Bugzilla
The first thing they will ask you is your .jtl results file so if it doesn't contain top-secret information you can update your question with the .jtl file contents so we would be able to look into the issue as well.
P.S. Any chance you're using a Transaction Controller with non-default naming policy? If yes, it might have an impact on results
P.P.S. Any change you're using JMeter Plugins or amending sample results on the fly via Groovy scripting? If yes, it also may influence the test metrics
Related
Just starting with jmeter and making some experiments I found something that looks kind of odd to me. I connected jmeter with influxdb and measured the avg. time response of one single request in a infinite loop. When I stopped the test I realized that the last time in the results csv created by jmeter is not the same as the one taken by influxdb. Specifically jmeter last measure is 13s higher than the one registered by influxdb. Any ideas on what could be happening?
I've tried to google it but haven't found any documentation or problem related
JMeter sends aggregated metrics, to wit it doesn't send each and every SampleResult but collects the results within some "window", default value is 5 seconds, controllable via backend_influxdb.send_interval JMeter Property
And metrics which are being sent are described here
You can try decreasing the 5 seconds window by amending the aforementioned backend_influxdb.send_interval JMeter property and setting it i.e. to 1000 ms so JMeter would send the data more often but it will create extra overhead so make sure that JMeter has enough headroom to operate and increasing metrics sending rate doesn't affect the overall throughput.
I'm about a week into learning JMeter and I've run a few test scripts which generate a summary.csv which contains your standard ; Samples, Average, Median etc...
[My Question]
I was wondering if there was a way to add a threshold for the summary.csv so if Average time is higher than x amount of milliseconds, then the user will be informed that the specific result was slower than expected. (Maybe this can be displayed on the summary.csv, I'm not sure what my options are tbh on how to output this)
I am aware that we can use assertions (specifically duration assertion) through the test script but the issue I have with assertions is that it stops the test once an assertion fails, stopping it from generating a summary.csv
Thank you for any input/opinions you guys have :) It is much appreciated!
Have a great day and stay safe everyone!
They are there already and they're controllable by the following JMeter Properties:
jmeter.reportgenerator.apdex_satisfied_threshold
jmeter.reportgenerator.apdex_tolerated_threshold
there is also a property which can apply thresholds to specific samplers or Transaction Controllers: jmeter.reportgenerator.apdex_per_transaction
Just declare the properties with the values of your choice in the user.properties file and next time you generate the dashboard its APPDEX section will reflect the thresholds.
More information: JMeter HTML Reporting Dashboard - General Settings
I have created a test plan in Jmeter and ran it for 10 users, it has run successfully without any error, as in the below screenshot of the listeners which I have added in my test plan.
In the above listeners, how may I come to know that the values of these fields Standard Deviation, Throughput, Median, Error% calculated as expected Or is there any ideal/expected/benchmark values of the above fields through which I compare and found that my test plan work as standard. Moreover how may I able to explain that the performance of my test plan is fine/good/better or best
Please suggest me thanks
It sounds like you don't really understand what you're doing so I would recommend starting with i.e. Performance Testing Guidance for Web Applications e-book.
With regards to the "values" - we have no any idea whether the "values" match your expectations. There are no any reference "values", normally your project should have non-functional requirements or SLAs which should define maximum response time or minimum number of hits per unit of time.
Check out JMeter Glossary to learn what the "values" mean.
If you don't have NFRs or SLAs defined you still can perform a stress test like:
Make sure that your JMeter test behaves like a real browser, at least I fail to see:
HTTP Cookie Manager
HTTP Cache Manager
HTTP Header Manager
You should be running your test in command-line non-GUI mode
Start with 1 virtual user and gradually increase the load until
you see the saturation point
you start seeing performance degradation
This way you will be able to state what is the maximum number of users you system can support without issues
I just want to get the aggregate report include a column for latency.
I have already tried uncommenting jmeter.save.saveservice.latency=true in bin/jmeter.properties.
I am currently using JMeter 3.3
Any help would be greatly appreciated. Thanks!
UPDATE:
A little hack I tried that kind of worked was adding a JSR223 PostProcessor with the following 3 lines of code:
long startTime = prev.getStartTime();
long latency = prev.getLatency();
prev.setEndTime(startTime + latency);
I say kind of worked because it is a hack and even though it was able to correctly reset the average column from load time to latency, it started throwing off the throughput values.
The end goal in my case is just to get the latency along with the other values by running the CMDRunner.jar on the JTL file to output the summary results CSV file.
You can't .
Alternatively why not use the Web Report that exists since 3.0 and that provides a lot of rich information much better than all existing listeners.
See:
https://jmeter.apache.org/usermanual/generating-dashboard.html
We are benchmarking our server and therefor we are using multiple benchmark tools. We already used ApacheBench giving us some great results and were also easy to plot in a graph.
Example plot AB results
Now we are using JMeter but having a hard time to get a good plot. We would love to have almost (completely) the same plo as the one from AB. So response time vs. requests. We tried already every listener but haven't found any satisfactory result.
We are using JMeter 2.10.
You can use:
Core Response time graph
Jmeter plugins : http://jmeter-plugins.org which has a lot of nice and very useful graphs which will meet your needs ( response time, percentiles, distribution, response codes per second ...)
Option 1: Isn't Response Time Graph what you're looking for?
Option 2: It's possible to build any graph from JMeter .jtl CSV or XML results file using Excel or equivalent, Google Charts API, JFreeChart, Javascript, etc.
Option 3: There is a JMeter Listener which automatically builds pretty and professional looking graphs for your performance test and have results comparison killer-feature