image of summary report
I need to create script to use this total error % then calculate something like
If error% is between 10 and 20 % console log message
If error% is between 21 and 40% console log message
If error% is between 40 and 60% console log message
If error% is majors than 61% console log message
Any suggestions ??
a lot of different none working approaches
I tried using the performance plugin in Jenkins but didn't worked well for this case
A "script" in which language? And are you sure you really need this because Summariser already includes this functionality
If you run JMeter test in command-line non-GUI mode like:
jmeter -n -t test.jmx -l result.jtl
The result.jtl is a CSV file which contains test metrics like sampler label, elapsed time, whether it was successful or not, etc.
So you could process the .jtl file somewhere in tearDown Thread Group and calculate the failed requests percentage with JSR223 Sampler
Code as text just in case:
SampleResult.setIgnore()
def jtl = new File('result.jtl').readLines()
def results = jtl.takeRight(jtl.size() - 1).collect { result -> result.split(',')[7] }
def failures = results.count { it == 'false' }
def failuresPercentage = (failures * 100.0f) / results.size()
println('Percentage of failures: ' + failuresPercentage + '%')
and in action:
Related
I am new to JMeter so bear with me...
I have a setUp Thread Group where I am grabbing a token and then re-using that in the HTTP Header Manager within the main Thread Group. Within that Thread Group I have the following parameters set...
I run this command to execute the tests:
jmeter -n -t PSC_Token.jmx -l testPsc.jtl -f
When I open the testPsc.jtl file though in Summary Report, I would expect that each request would show 600 for # Samples (200 threads * 3 loop count) but it is showing 1200 for each.
I tried deleting the file entirely and re-running it, just in case it was appending or something strange. That doesn't resolve the issue though.
Any ideas?
You're writing the same data into the same file 2 times, the options are in:
Disable (or better delete) the Summary Report listener, in general Listeners don't add any value, they only consume resources
Or remove -l command line argument and run your test just like:
jmeter -n -t PSC_Token.jmx
Also be aware that according to JMeter Best Practices you should always be using the latest version of JMeter so consider upgrading to JMeter 5.5 (or whatever is the latest stable version available at JMeter Downloads page)
I'm executing Jmeter from command line using following command
!JMeter -Jjmeter.save.saveservice.samplerData=true -Jjmeter.save.saveservice.response_data=true -Jjmeter.save.saveservice.output_format=xml -Jjmeter.save.saveservice.responseHeaders=true -Jjmeter.save.saveservice.requestHeaders=true -Jsummariser.out=false -n -t .jmx -l JmeterReports\TestReport.xml -j JmeterReports\jmeter.log
At the end of the run, I get a tidying up message and it takes 50 mins. Any hint on how to avoid this. it impacts my testing time.
00:08:42.083 login ticket value is :: LT-1418054-HsUfB5qYlXKKhrnJGGcoGeCeQtTf5
00:59:51.971 Tidying up ... # Tue Mar 22 12:30:26 CET 2022 (1647948626380)
00:59:51.971 ... end of run
First of all consider disabling all these Results File Configuration overrides as storing all requests and responses data into the .jtl file for one hour test causes massive disk IO, moreover writing XML is more resource intensive process than default CSV output
If the problem is still there take a thread dump and inspect what the threads are doing and which are stuck or waiting
Monitor the JVM metrics using JVisualVM or equivalent, it might be the case JVM does excessive garbage collection due to low heap space or something like this
More information:
Reducing resource requirements
9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure
I'm trying to use Jenkins to automate performance testing with JMeter,
each build is a single JMeter test and I want to increase the number of users(threads) for each Jenkins build if the previous was successful.
I have configured most of the build, with SSH plugin I can restart Tomcat, copy catalina.out, with publishing performance I can open the .jtl file and determine if the build was successful.
What I want is to execute a different batch command for the next build(to increase the number of users(threads) and user id's)
For example:
jmeter -Jthreads=10 -n -t C:\TestScripts\script.jmx -l C:\TestScripts\Jenkins.jtl
jmeter -Jthreads=20 -n -t C:\TestScripts\script.jmx -l C:\TestScripts\Jenkins.jtl
jmeter -Jthreads=30 -n -t C:\TestScripts\script.jmx -l C:\TestScripts\Jenkins.jtl...
Is there some good jmeter plugin some counter that i can use to increase some variable by 10 each time:
jmeter -Jthreads=%variable1%...
I have tried by setting environmental variables and then incrementing that variable by:
"SET /A thread+=10"
but it doesn't change that variable because jenkins opens its own CMD, a new process :
("cmd /c call C:\WINDOWS\TEMP\jenkins556482303577128680.bat")
Use the following SET command to increase threads variable by 10:
SET /A threads=threads+10
Or inside double quotes:
SET /A "threads+=10"
Not knowing your Jenkins configuration, and which plugins you have installed and how do you run the test it is quite hard to come up with the best solution.
The only "universal" workaround I can think of is writing the current number of threads into a file in Jenkins workspace and reading the value of threads from the file on next execution.
Add setUp Thread Group to your Test Plan
Add JSR223 Sampler to your Thread Group
Put the following Groovy code into "Script" area:
import org.apache.jmeter.threads.ThreadGroup
import org.apache.jorphan.collections.SearchByClass
import org.apache.commons.io.FileUtils
SampleResult.setIgnore()
def file = new File(System.getenv('WORKSPACE') + System.getProperty('file.separator') + 'threads.number')
if (file.exists()) {
def newThreadNum = (FileUtils.readFileToString(file, 'UTF-8') as int) + 10
FileUtils.writeStringToFile(file, newThreadNum as String)
def engine = ctx.getEngine()
def test = org.apache.commons.lang.reflect.FieldUtils.getField(engine.getClass(), 'test', true)
def testPlanTree = test.get(engine)
SearchByClass<ThreadGroup> threadGroupSearch = new SearchByClass<>(ThreadGroup.class)
testPlanTree.traverse(threadGroupSearch)
def threadGroups = threadGroupSearch.getSearchResults()
threadGroups.each {
it.setNumThreads(newThreadNum)
}
} else {
FileUtils.writeStringToFile(file, props.get('threads'))
}
The code will write down the current number of threads in all Thread Groups into a file called threads.number in Jenkins Workspace and on subsequent runs it reads the value from it, adds 10 and writes it back.
For now i am creating 20 .jmx files (1.jmx, 2.jmx , 3.jmx ...) each whith a different number of users. and calling them whit this command :
jmeter -n -t C:\TestScripts\%BUILD_NUMBER%.jmx -l C:\TestScripts\%BUILD_NUMBER%.jtl
the first billd will call 1.jmx the second 2.jmx ...
it isn't the best method but it works for now. I will try your advice over the weekend when i have more time.
i have found the a solution that works for me, it inst pretty. I created a python script which changes a .CVS fil from which JMeter reads the number of threads and the starting user id. This python script incremets the starting user id by the number of threads in the previous bild and the number of threads by 10
file = open('C:\\Users\\mp\\AppData\\Local\\Programs\\Python\\Python37-32\\eggs.csv', 'r')
a,b=file.readlines()[0].split(",")
print(a,b)
b=int(b)
a=int(a)
b=a+b
a=a+10
print(a,b)
f = open("C:\\Users\\mp\\AppData\\Local\\Programs\\Python\\Python37-32\\eggs2.csv", "a")
f.write(str(a)+","+str(b))
f.close()
I have python on my pc and a i am calling the script in Jenkins as a windows Bach command
C:\Users\mp\AppData\Local\Programs\Python\Python37-32\python.exe C:\Users\mp\AppData\Local\Programs\Python\Python37-32\rename_write_file.py
I am much better in python than Java so I implemented this in Python.
So for each new test,the CSV file from which jmeter reads values is changed.
I have the following scenario:
Thread group1 - this sends requests to a server (lots of threads and iterations)
HTTP Request
Thread group2 - 1 thread 1 iterations
JSR223 sampler (has a while loop which periodically check if a number is 0) and collects some times
The scenario ends when threadgroup 2 finishes
I run the above scenario using the non GUI mode and i am interested in the RPS. (as shown below 222.0/s), but only for the first threadGroup.
summary = 50002 in 00:03:29 = 222.0/s Avg: 4151 Min: 38 Max: 797601 Err: 0 (0.00%)
Now, when the last threadGroup ends i will have:
summary = 50003 in 00:09:12 = 90.7/s Avg: 4136 Min: 38 Max: 797601 Err: 0 (0.00%)
The RPS is now low only because the last threadGroup takes very long to finish.
Is there any way the time from the last threadGroup can not be taken into consideration of the total time ? Or maybe another approach to this problem.
You can use jmeter.reportgenerator.sample_filter property in order to exclude your JSR223 Sampler from the report.
Run your test in command-line non-GUI mode like:
jmeter -n -t test.jmx -l result.jtl
The command to generate FULL results would look like:
jmeter -g result.jtl -o ALL-RESULTS
It will create ALL-RESULTS folder holding information on all the samplers
The command to generate results WITHOUT JSR223 Sampler would be:
jmeter -Jjmeter.reportgenerator.sample_filter="^((?!JSR223 Sampler).)*$" -g result.jtl -o FILTERED-RESULTS
It will create FILTERED-RESULTS folder with HTTP Request sampler only (or whatever else samplers, only JSR223 Sampler(s) will be excluded.
More information:
Generating Report Dashboard
Configuring JMeter
Apache JMeter Properties Customization Guide
I have an aggregate report:
Do I have to add another listener or add anything else to the testplan?
For ex: consider the above snapshot:
I'm running 9002 samples, I just need to know the total time required for all the 9002 samples?
If your 9002 Samples are played in a sequence by only 1 user, just use a TransactionController that will contain your HTTP Request and you will have this.
If 9002 samples are played by different users then you can export the CSV file that contains the test results and use Excel to sum up the elapsed column.
To generate a CSV file, run test in NON GUI Mode using:
jmeter -n -l results.csv -t test.jmx