I'm stuck on finding solution on one problem with Jmeter. I need to put some logic into my Test Plan that can give simple report PASS/FAIL calculated on test cases execution results and put in generated JTL report afterwards. For instance
All tests passed - Test Plan result=PASS
One or more tests failed - Test Plan result=FAIL
The majority of suitable options assume using third-party tools, to wit:
you can run JMeter test in Jenkins and use Performance plugin, it allows to conditionally fail the build if the amount of failed requests exceeds specified threshold
you can run JMeter test using Taurus tool as a wrapper, it has flexible and powerful Pass/Fail Criteria Subsystem allowing to set different criteria definitions to mark the test as passed or failed. If build is failed Taurus process returns non-zero exit code.
If above approaches are not suitable for any reason please elaborate your question and explain how and where you would like to see this "FAIL" or "PASS" result.
Add one BeanShell Listener and one BeanShell Sampler at the end of your Thread Group and put this in Listener:
if(sampleEvent.getResult() instanceof org.apache.jmeter.protocol.http.sampler.HTTPSampleResult)
if (!sampleEvent.getResult().isResponseCodeOK())
vars.put("res", -1);
And in BS Sampler put:
if you wanna store result as property:
props.put("testPlanResult", vars.get("res") != -1 ? "PASS" : "FAIL");
if you wanna store result in a file:
f = new FileOutputStream("/path/to/file.txt", false);
p = new PrintStream(f);
p.println("Result: " + (vars.get("res") != -1 ? "PASS" : "FAIL"));
p.close();
f.close();
From here you can do what ever you need with created property or file containing result...
Hope this helps you!
EDIT:
You will need to add this import if writing result to file:
import org.apache.jmeter.services.FileServer;
Related
Just wanna know if this scenario is possible in jmeter:
I will create a website using docker image then i will have to check the exact accessibility time(from creating up to launching/checking the site). Once the requirement is met, the timer will stop and it will give me the exact time on when it is actually up. Also, in the view result tree, i only want to get the last successful status. The View Result Tree will only show the last successful status and will disregard the failed status. This is the sample of my thread that i am using: enter image description here
If you really want to discard non-successful results you can do this using JSR223 Assertion and the following Groovy code:
if (!prev.isSuccessful()) {
prev.setIgnore()
}
where prev is a shorthand for parent SampleResult and setIgnore() function tells the listener to ignore the result in case of failure.
More information: Scripting JMeter Assertions in Groovy - A Tutorial
I have a situation where I check the response data and if there is a specific variable exist then I like to simulate a test failure even though that HTTP req code was 200. For example in Bean Post Processor I have:
if ( (prev.getResponseDataAsString().indexOf(Z2) >= 0) || (matches > 1) ){
System.out.println(ctx.getCurrentSampler().getName() +" --> Failed ....")
}
I know how to do it when I want to set the result to success (prev.setResponseOK();) how do I do it if I want to set it to fail? so the GUI shows red and not green?
Thank you
See in JSR223 Sampler
Unlike the BeanShell Sampler, the JSR223 Sampler does not set the ResponseCode, ResponseMessage and sample status via script variables. Currently the only way to changes these is via the SampleResult methods:
SampleResult.setSuccessful(true/false)
prev is a SampleResult object so you can mark it as failed:
prev.setSuccessful(false)
prev - (SampleResult) - gives access to the previous SampleResult
Sounds like a use case for a Response Assertion which you can use in order to conditionally set pass/fail criteria for a Sample basing on presence/absence of certain patterns in the response data.
Here is an example of failed HTTP Request sampler with 200 status code due to absence of the anticipated data in the response:
See How to Use JMeter Assertions in Three Easy Steps for more details.
With regards to JMeter Best Practices which you're violating by the way by using Beanshell PostProcessor:
You should be using built-in JMeter test elements where possible
If you have to go for scripting make sure to choose the most performing option which is unfortunately not Beanshell
Good afternoon !
I will try to explain you clearly my problem.
The context
I have a JMeter TestPlan which send HTTP requests to a server. I have a Beanshell script to assert each different case of error returned.
302 response code -> OK
200 response code -> ?
In each error 200, I check the response data string to see if it is an error or a correct case. (User error like User don't have correct rights is OK, but Server is unavailable is ERROR and both have 200 as response code.)
Here is my test plan :
The goal
As I have several errors returned by only one assertion script, I am not able to differenciate each error, except by uncollaspe the assertion in a ViewResultTree. But I disable it when launching my test, and I will launch my TestPlan remotely.
I had the idea to manually count each error. All my samples goes in my Assertion script, and goes to the correct if block according to their content. I increment some variables (JMeter.properties in fact) in each block.
int test = Integer.parseInt(props.getProperty("302"));
test++;
props.setProperty("302", ""+test);
I want to display all those variables in a JFrame at the end of my testplan like this :
The problem
My problem is that I don't know how to launch a Beanshell script before and after the TestPlan.
I want a first script to be started before any sample is send, just to initialize all my properties variables to 0 (else, they keep the value of the last TestPlan).
And, I want a second one to display my Frame with all the variables after the test plan is finished. (Currently it is a JFrame but it will not stay like this.)
Tested solutions
1) For my first script, I set a Counter (JMeter > Config Element > Counter) in the beginning of my test plan to 0.
I use it to check if my test already started of not with an If Controller :
I have a Pre-Processor Beanshell with props.set("302","0"); where "302" is my property to count all 302 response code.
It correctly works but I want to know if there is a proper way to do this.
2) Then, for my second script, I tried to use ${JMeterThread.last_sample_ok} in an If Controller aswell but it doesn't work like I expected. If I put it after my sample, it start after all OK assertion, and if I put it at the end of the test plan, it is never called.
How can I run my beanshell script once, after all my threads are stopped (i.e. all sample finished) ?
Thank you in advance, I hope you understood everything !
JMeter SetUp thread group and TearDown thread group are meant for exactly this.
Add your beanshell component to the setUp thread group to do some setup activities before your actual test starts. Similarly the tearDown thread group runs after your test execution is complete.
I have 17 tests in a .jmx that I call from Jenkins. Out of this, 15 runs correctly and return meaningful results. However, the 2 last resturns with a run time of 0ms. I looked at the logs at there is no exception. My question is where (which log file) can I look in this case?
There are different options to detect that your test is doing what it should do.
Assertions
Result File
Log file
The easiest one is using Assertions to check sampler response data. The most commonly used is Response Assertion
Next one is configuring JMeter to save request and response fields you're interested in. See properties, starting from jmeter.save.saveservice in jmeter.properties file. Uncomment and set to true those, you feel may help you to get to the bottom of the issue
The next one is the most flexible and informative. I would suggest to use some logging to get things sorted out. JMeter writes a file called jmeter.log. You can add Beanshell Pre Processor and Beanshell Post Processor to the sampler which run time is 0ms with something like:
In Post Processor:
log.info("Starting test " + sampler.getName() + " at " + new Date());
In Post Processor:
log.info("Response code " + prev.getResponseCode());
log.info("Response message " + prev.getResponseMessage());
log.info("Execution time " + prev.getTime());
etc. You can even see full response data in the log as
log.info(new String(data));
See How to use BeanShell guide for more details on advanced Beanshell scripting.
All stdout and stderr output from the Jenkins build will go to the console output at [Jenkins URL]/job/[job name]/[build number]/console
If you don't see anything useful there, the best approach is to run the JMeter tests outside Jenkins, but on the same machine and under the same user environment that Jenkins would use. You may need to enable some extra debugging info; see this link for some tips.
I am new into Apache Jmeter. Basically I want to load test our couple of thrift APIs but have no clue where to start with. It is in java where api takes 2 parameter and then send java object as response.
Any pointer would be very helpful.
JMeter isn't especially for it but it's flexible enough to support your use case.
There is an extensibility mechanism which uses BeanShell. JMeter provides BeanShell Sampler which is capable of invoking Java code, including using external jars.
Simple usage:
Start with empty JMeter project
Create a Thread Group with all defaults (you can play with number of threads, ramp-up, etc)
Add a BeanShell Sampler with following code:
Thread.sleep(2000L);
Add View Results Tree listener
Save and run
You should see a green triangle (or triangles) basing on your number of threads and loops) with output like following:
Thread Name: Thread Group 1-1
Sample Start: 2013-11-02 14:48:11 GMT+03:00
Load time: 5030
Latency: 0
Size in bytes: 0
Headers size in bytes: 0
Body size in bytes: 0
Sample Count: 1
Error Count: 0
Response code: 200
Response message: OK
If you use any of techniques to analyze results, i.e.
JMeter embedded listeners like Aggregate Report, Summary Report, Graph Resuls, etc.
Storing results to CSV file and opening them with Excel or equivalent (see jmeter.properties file under /bin directory of your JMeter installation. Properties prefix is "jmeter.save.saveservice."
JMeter Ant Task (see Test.jmx and build.xml in /extras folder under your JMeter installation)
JMeter Results Analysis Plugin
You'll see your request(s) success rate, min/max/average times (something like 2 seconds I guess) and some more information (depending on your configuration).
Particular your use case assumes
IMPORTANT Placing thrift (or whatever) jars under lib/ext folder (or you won't be able to access your APIs
importing classes you need to test somewhere in BeanShell Sampler
import yourpackage.YourClass;
Invoking methods you want to test from BeanShell Sampler
(optional) do some assertions on responses. i.e.
if (yourresponse != yourexpectedresponse){
IsSuccess=false;
ResponseMessage= "Test Failed";
}
Hope this helps
You can use JSR223 Sampler + Groovy (add groovy-all.jar in jmeter/lib) and look at this client example, see NonblockingClient code for an example:
http://www.javacodegeeks.com/2012/03/apache-thrift-quickstart-tutorial.html
Make your groovy code call a least the following at end:
SampleResult.setSuccessful(true/false)
SampleResult.setResponseCode("code")
SampleResult.setResponseMessage("message")
See:
http://jmeter.apache.org/usermanual/component_reference.html#JSR223_Sampler
And of course, ensure you add the required dependencies in jmeter/lib.
I have writtena CustomThriftSampler for JMeter to load test HBase through thrift service. You can get the details about it at my blog - http://1-st.blogspot.in/2013/12/load-testing-thrift-services-custom.html . Couldn't create a generalized code. Anyway its simple and starightforward java code. Anyone could try it. If time permit I shall write a generalised code and commit to github!!