I am trying to find out if it is possible to get Jmeter to generate standard deviations in its report WITHOUT THE GUI! (<<< Important!). Script based ONLY. I have perused the Internets and most Jmeter information is long since out of date, going back to 2007-2017.
It seems Jmeter has come along way since then incorporating most of the "plugins". I am running JMeter 5.3.
I basically generate tests with
jmeter -t test.jmx -l results.csv -jmeter.save.saveservice.output_format=csv
Then, as per the documentation I run,
jmeter -g results.csv -o report
But it seems options to give this command are not plentiful.
This command invocation generates an html site, and it generates areport/statistics.json file. However, this file merely contains elements of the "AggregateReport", which are
{
"transaction" : "Total",
"sampleCount" : 1000,
"errorCount" : 0,
"errorPct" : 0.0,
"meanResTime" : 42.56400000000011,
"medianResTime" : 37.0,
"minResTime" : 31.0,
"maxResTime" : 1327.0,
"pct1ResTime" : 49.0,
"pct2ResTime" : 64.0,
"pct3ResTime" : 95.93000000000006,
"throughput" : 20.10777768840988,
"receivedKBytesPerSec" : 17.002539863669266,
"sentKBytesPerSec" : 6.8610525856088636
}
which does NOT include the "Standard Deviation". I am at a loss of "Why not?", but that is another question.
I know people will tell me to add a "SummaryReport" Listener, but that only seems to be useful in the GUI, and this approach MUST be scripted based. I need to be able to read the results out of a file sometime long after the run has completed.
I cannot seem to find the any options to give Jmeter to generate a report that lists the Standard Deviation, or even to generate a file containing a "SummaryReport" in addition to the "AggregateReport".
It appears the only report that is generated is this deficient "AggregateReport".
Any insight?
One of those JMeter plugins you mention provides a command line tool to write the "AggregateReport" to a CSV file:
JMeterPluginsCMD.sh --plugin-type AggregateReport --input-jtl results.csv --generate-csv aggregate_report.csv
This requires the following three plugins to be installed:
https://jmeter-plugins.org/?search=jpgc-cmd
https://jmeter-plugins.org/?search=jpgc-synthesis
https://jmeter-plugins.org/?search=jpgc-filterresults
Command line options are listed here: https://jmeter-plugins.org/wiki/JMeterPluginsCMD/
Both the AggregateReport and the SynthesisReport include standard deviation.
Related
I am new to JMeter so bear with me...
I have a setUp Thread Group where I am grabbing a token and then re-using that in the HTTP Header Manager within the main Thread Group. Within that Thread Group I have the following parameters set...
I run this command to execute the tests:
jmeter -n -t PSC_Token.jmx -l testPsc.jtl -f
When I open the testPsc.jtl file though in Summary Report, I would expect that each request would show 600 for # Samples (200 threads * 3 loop count) but it is showing 1200 for each.
I tried deleting the file entirely and re-running it, just in case it was appending or something strange. That doesn't resolve the issue though.
Any ideas?
You're writing the same data into the same file 2 times, the options are in:
Disable (or better delete) the Summary Report listener, in general Listeners don't add any value, they only consume resources
Or remove -l command line argument and run your test just like:
jmeter -n -t PSC_Token.jmx
Also be aware that according to JMeter Best Practices you should always be using the latest version of JMeter so consider upgrading to JMeter 5.5 (or whatever is the latest stable version available at JMeter Downloads page)
Created two dummy sample projects as (dummy1.jmx and dummy2.jmx) and executed below commands with default settings (JMeter 5.3 default installation with all required plugins installed).
#> jmeter.bat -n -t dummy1.jmx -l dummy1.csv -j dummy1-jmeter.log to execute load
Generated report and timestamps look perfect both in dashboard and graphs
**#> jmeter.bat -g dummy1.csv -o dummy1 -j dummy1-report-jmeter.
#> jmeter.bat -n -t dummy2.jmx -l dummy2.csv -j dummy2-jmeter.log to execute load
Generated report and timestamps look perfect both in dashboard and graphs
#> jmeter.bat -g dummy2.csv -o dummy2 -j dummy2-report-jmeter.log
Used MergeResults plugin to merge the above CSV files to a single file and generated HTML report
#> JMeterPluginsCMD.bat --generate-csv dummy1-dummy2.csv --input-jtl merge.properties --plugin-type MergeResults
Found merged timeStamp label is not valid and also generated report shows invalid DateTime.
#> jmeter.bat -g dummy1-dummy2.csv -o merged -j merged-report-jmeter.log
Is this a bug or am I missing configuration? Even adding jmeter.save.saveservice.timestamp_format=yyyy/MM/dd HH:mm:ss.SSS to user.properties didn't help
merge.properties
inputJtl1=dummy1.csv
prefixLabel1=TEST1:
includeLabels1=.*
excludeLabelsl=
includeLabelRegex1=true
excludeLabelRegex1=
startOffset1=
endOffset1=
inputJtl2=dummy2.csv
prefixLabel2=TEST2:
includeLabels2=.*
excludeLabels2=
includeLabelRegex2=true
excludeLabelRegex2=
startOffset2=
endOffset2=
Unfortunately we cannot help without:
Seeing your merge.properties file contents
Knowing what do you expect
In the meantime I can only tell you where did this 2000-01-01 date came from:
It's declared here:
private static final long REF_START_TIME = 946681200000L;
And being added to the original SampleResult timestamp here:
res.setTimeStamp(res.getTimeStamp() - startTimeRef + REF_START_TIME);
I don't know whether it is a bug or it's designed to work like this (however the crazy logic of substraction of sampler start time from its timestamp is beyond my limited understanding), it's better to check at JMeter Plugins support forum
In the meantime you can use services like BM.Sense for comparing different test runs resutls
I am using Jmeter Version 4. For example I am running test for four hours, And during the test run, I want the result file for the test ran from 2nd to 3rd hour.Is it possible to get the result file like that?
I know that we can get the result file from starting to 3rd hour.But I want from 2nd to 3rd hour.
Can I get that.Please suggest?
The easiest option is going for Filter Results Tool which has --start-offset and --end-offset parameters specifying how to "cut" the original .jtl file (in seconds) so you could do something like:
FilterResults --output-file from2ndto3rd_hour.jtl --input-file /path/to/large/result.jtl --start-offset 7200 --end-offset 10800
Filter Results Tool can be installed using JMeter Plugins Manager:
Ideally, you should use this solution that allows you to have live results:
https://jmeter.apache.org/usermanual/realtime-results.html
But if you want to work with CSV, your best bet would be to modify the timestamp format by adding to user.properties :
jmeter.save.saveservice.timestamp_format=yyyyMMddHHmmss
And ensure JMeter flushes on every write to avoid having partial lines:
jmeter.save.saveservice.autoflush=true
And then use grep, for example to take results between 15 and 16h on 26 january 2019
grep "2019012615" results.csv > filter.csv
If you don't want to rely on grep, then you can take the whole file and generate the HTML report using :
jmeter -Jjmeter.reportgenerator.start_date=20190126150000 -Jjmeter.reportgenerator.end_date=20190126160000 -g results.csv -o reportfolder
I am trying to print 95 Percentile and 99 Percentile response times in the jmeter aggregate report from the command line
For this, I have tried the solution mentioned in here: Jmeter: Generating aggregate report through commnd line is not including 95% and 99% line in generated report
but that didn't work. I have tried editing the jmeter.properties file too.
Also, the other solution that I came across is to generate the ResponseTimesPercentiles graph using CMDRunner.jar in csv format but that gives all the values from 0-99.9 which I don't need. So is there a way I can customize the csv file to just have ReponseTimesPercentiles for desired values?
Also tried this command:
java -jarCMDRunner.jar --tool Reporter --generate-csv test.csv --input-jtl results.jtl --include-labels aggregate_report_99%_line --plugin-type AggregateReport
which didn't work.
So, is there any way that I can generate the 95% and 99 Percentiles in the aggregate report
The functionality is available by default starting from JMeter version 2.13
The behavior is controllable via the following properties:
aggregate_rpt_pct1=90
aggregate_rpt_pct2=95
aggregate_rpt_pct3=99
If you don't see percentiles like at the above image in your Aggregate Report - add the above lines to user.properties file.
NB:
The approach won't work on JMeter versions earlier than 2.13
JMeter restart will be required to pick the properties up, dynamic changes won't work.
You can do it from CMD, First make the plan using GUI, then in the bin folder of Jmeter, open CMD and run the following command.
jmeter -n -t "[PATH-FOR-PLAN]\myplan.jmx" -l "[PATH-OF-RESULT]\results.csv" -e -o "[PATH-OF-OUTPUT]\output"
The output folder will have a webpage with the aggregate report
I am using Jmeter version : 2.10. I test has couple of Http samplers. When i run the test from command line
jmeter.bat -n -t "C:\Jmeter Projects\TestSuite.jmx" -l "C:\Jmeter Projects\testResult.xml"
the result of the test creates "testResult.xml" but when i open it the data is in cvs format
1384844043490,1527,HTTP Request-1,200,OK,Thread Group 1-1,text,true,2776,1527
1384844045046,350,HTTP Request-2,200,OK,Thread Group 1-1,text,true,395,350
1384844045411,4890,HTTP Request-3,200,OK,Thread Group 1-1,text,true,977530,4888
1384844050304,286,HTTP Request-5 ,200,OK,Thread Group 1-1,text,true,4091,28
There is no header. I do not mind csv format if i can get the table header or the .jtl format.
I think it was since JMeter 2.8 that the default JTL format is CSV as opposed to XML. I didn't try this myself, but this should work for you:
-Jjmeter.save.saveservice.output_format=xml
Just make the changes as suggested by Alon and execute your test plan. Saving report as .jtl file.
jmeter.bat -n -t "C:\Jmeter Projects\TestSuite.jmx" -l "C:\Jmeter Projects\testResult.jtl"
Then using JMeter plugins generate the kind of report you wish for.