I was asked to enhance my assertions to provide some better log messaging within my JMeter test plan that tests APIs using basic CRUD methodology. The test plan is being checked into Jenkins and being run automatically. When something goes wrong, the level of messaging is not adequate for the support team.
Within the first thread group, I have an HTTP Request to create a new record within the database based on the payload being passed in. Under this request, I have a BeanShell Assertion as follows:
if (ResponseCode.equals("200") == true)
{
SampleResult.setResponseOK();
}
I'm now trying to enhance this to account for 409, and 500 responses.
I've attempted the, but it does not seem to work:
if (ResponseCode.equals("200") == true)
{
SampleResult.setResponseOK();
}
else if (ResponseCode.equals("409") == true)
{
FailureMessage = "Creation of a new CAE record failed: Attempting to create a duplicate record.";
}
else (ResponseCode.equals("500") == true)
{
FailureMessage = "Creation of a new CAE record failed: Unable to connect to server";
}
Additionally, if the ResponseCode is not 200, then I need to drop out of the entire thread group and go to the next thread group.
I've read several questions on this site, as well as How to Use BeanShell: JMeter's Favorite Built-in Component and How to Use JMeter Assertions in Three Easy Steps, but I'm still confused. Not being a developer and still new to JMeter, I'm in need of guidance.
Any and all help is much appreciated.
Selecting 'Stop Thread' in the Thread Group would help you to stop the thread group in case of any error - assuming you have other thread groups to execute consecutively. if not, the test will stop.
In the beanshell assetion include
else if (ResponseCode.equals("409") == true)
{
Failure = true;
FailureMessage = "Creation of a new CAE record failed: Attempting to create a duplicate record.";
}
Related
I'm executing JMeter task for a few hours on a server,
I want to be able to pause execution for a few seconds/minutes and resume when server finish restarted
Is there a way to signal JMeter to pause and resume its execution?
I saw similar question, but it doesn't fit my issue
As of current JMeter version 5.3 there is no way to accomplish your "issue" with built-in JMeter components.
The easiest solution I can think if is: given you're restarting your server it should be not available for some time and when it becomes available - it should respond with a HTML page containing some text.
So you can "wait" for the server to be up and running as follows:
Add JSR223 Sampler to the appropriate place in the Test Plan where you need to "wait' for the server to be up and running
Put the following code into "Script" area:
import org.apache.http.client.config.RequestConfig
import org.apache.http.client.methods.HttpGet
import org.apache.http.impl.client.HttpClientBuilder
import org.apache.http.util.EntityUtils
SampleResult.setIgnore()
def retry = true
def requestConfig = RequestConfig.custom().setConnectTimeout(1000).setSocketTimeout(1000).build()
def httpClient = HttpClientBuilder.create().setDefaultRequestConfig(requestConfig).build()
while (retry) {
def httpGet = new HttpGet('http://jmeter.apache.org')
try {
def entity = httpClient.execute(httpGet).getEntity()
if (EntityUtils.toString(entity).contains('Apache JMeter')) {
log.info('Application is up, proceeding')
retry = false
} else {
log.info('Application is still down, waiting for 5 seconds before retry')
sleep(5000)
}
}
catch (Throwable ex) {
sleep(5000)
ex.printStackTrace()
}
}
That's it, the code will try to open the web page and look for some text in it, if the page doesn't open and/or text is not present - it will wait for 5 seconds and retry
More information:
HttpClient Quick Start
Apache Groovy - Why and How You Should Use It
I have a test plan, where i have two thread groups, first thread group will run and will log the result into a file, the logged file will be the input to the second thread group.
Now, for doing this in thread group one i added BeanShell PostProcessor script as follows, but it works as follows
open jmeter and run the test plan (log file "testing.csv" will be created as fresh)
For first time "isHeaderWritten" property is false.
No without closing and reopening the jmeter, run the script again. i expect the log file "testing.csv" must be truncated and add the values freshly. But what acually happening is "testing.csv" file is appended with the new values.
Now, close the jmeter and open again, this time log file "testing.csv" will be truncated and fresh new values are added.
How to clear the file and re-write the records for each run (without closing the jemeter)
import org.apache.jmeter.util.JMeterUtils;
threadName=ctx.getThread().getThreadName();
log.info("threadName: "+threadName);
isHeaderWritten = JMeterUtils.getPropDefault("isHeaderWritten","false");
fileName="C:\\response\\testing.csv";
log.info("isHeaderWritten: "+isHeaderWritten);
if(isHeaderWritten == "true"){
f = new FileOutputStream(fileName, true);
fileHeader = "requestStatus,cNumber,pId,id,token";
}else{
log.error("isHeaderWritten:"+isHeaderWritten);
f = new FileOutputStream(fileName, false);
fileHeader = "requestStatus,cNumber,pId,id,token";
}
p = new PrintStream(f);
this.interpreter.setOut(p);
if(isHeaderWritten == "false"){
print(fileHeader);
JMeterUtils.setProperty("isHeaderWritten", "true");
}
log.info("Test results will be written to file "+fileName);
cNumber=vars.get("cNumber");
log.info("cNumber"+cNumber);
pId = vars.get("pId");
log.info("pId"+pId);
pmId = vars.get("pmId");
log.info("pmId"+pmId);
tTxt = vars.get("tTxt");
log.info("tTxt"+tTxt);
responseCode=prev.getResponseCode();
log.info("responseCode of "+requestString+ " is "+responseCode);
requestStatus = "Success";
if(!"201".equals(responseCode)){
requestStatus = "Failure"+","+cNumber+","+pId+","+pmId+","+tTxt;
}
result = requestStatus;
if("Success".equals(requestStatus)){
responseData=new String(data);
log.info("responseData of "+requestString+ " is "+responseData);
requestString=requestStatus+","+cNumber+","+pId+","+pmId+","+tTxt;
result = requestString;
log.info("result"+result);
}
log.debug(result);
print(result);
f.close();
According to JMeter Documentation:
Properties are global to JMeter, so can be used to communicate between threads and thread groups
Properties are not the same as variables. Variables are local to a thread; properties are common to all threads
So once you execute this line:
JMeterUtils.setProperty("isHeaderWritten", "true");
the property isHeaderWritten will become true and will remain true until you restart JMeter.
The easiest solution would be adding a tearDown Thread Group to your test plan (this Thread Group is being executed after any other thread group(s)) and add a JSR223 Sampler with the following code:
SampleResult.setIgnore();
org.apache.jmeter.util.JMeterUtils.setProperty("isHeaderWritten", "false");
Since JMeter 3.1 users are encouraged to switch to JSR223 Test Elements and Groovy language for any form of scripting mainly because Groovy performance is much better than Beanshell so consider migrating to JSR223 PostProcessor and Groovy language on next available opportunity
How can I get an overall PASS/FAIL result for a JMeter thread group without using a post processor on every sampler?
I've tried using a beanshell listener, but it doesn't work for instances where there are multiple samplers inside a transaction controller with "Generate Parent Sample" enabled. In that case, the listener only gets called once per transaction controller and I'm only able to access the result of the last sampler inside the transaction controller.
Edit:
I would like to be able to save a pass/fail value as Jmeter variable or property for the thread group. If one or more components of the thread group fail or return an error, then that would be an overall fail. This variable will then be used for reporting purposes.
My current beanshell listener code:
SampleResult sr = ctx.getPreviousResult();
log.info(Boolean.toString(sr.isSuccessful()));
if (!sr.isSuccessful()){
props.put("testPlanResult", "FAIL");
testPlanResultComment = props.get("testPlanResultComment");
if(testPlanResultComment == ""){
testPlanResultComment = sr.getSampleLabel();
}else {
testPlanResultComment = testPlanResultComment + ", " + sr.getSampleLabel();
}
props.put("testPlanResultComment", testPlanResultComment);
log.info(testPlanResultComment);
}
If you call prev.getParent() you will be able to fetch individual sub-samples via getSubResults() function, something like:
prev.getParent().getSubResults().each {result ->
log.info('Sampler: ' + result.getSampleLabel() + ' Elapsed time: ' + result.getTime() )
}
log.info('Total: ' + prev.getParent().getTime())
Demo:
More information: Apache Groovy - Why and How You Should Use It
My test plan looks like,
I am running some Http requests with assertion .
I need to check assertion is passed or failed (any built-in variable is available?) .
Need to call/extract assertion result and use it for my next API .
Any one help me on this.
I solved this case using BeanShell Listener by adding the following code and you can call assertion result on jmeter like ${results}
try
{
int results;
if (sampleResult.isSuccessful()) {
log.info("Response Assertion PASSED");
results = 1;
vars.put("results","1");}
else {
log.info("Response Assertion FAILED" );
results = 2;
vars.put("results","2");}
}
catch (Throwable ex) {
log.error("Error in Beanshell", ex);
throw ex;
}
I would suggest adding JSR223 Assertion where you can get parent sampler AssertionResult instance(s) and extract the required information from it like:
def assertionResult = SampleResult.getAssertionResults()[0]
if (assertionResult.isFailure()) {
log.info("Assertion " + assertionResult.getName() + " failed")
log.info(assertionResult.getFailureMessage())
}
Demo:
See How to Use JMeter Assertions in Three Easy Steps article for more information on using assertions. Also avoid using Beanshell as it may cause performance problems, if you need to go for scripting - stick to JSR223 Test Elements
1.Use "Regular Expression Extractor" to extract assertion result.
2.Use BeanShell Sampler to get assertion result,then,use it for your next API.
In my JMeter test plan, I'm trying to write all errors out to a log. I'm using a BeanShell Post-Processor configured as follows
import org.apache.jmeter.services.FileServer;
if (ResponseCode != null && ResponseCode.equals("200") == false) {
Failure = true;
// displays in Results Tree
FailureMessage ="Creation of a new CAE record failed. Response code " + ResponseCode + "." ;
// Static elements
part1 = "Creation of a new record failed. Response code: ";
part2 = ". Sorry!";
// Open File(s)
FileOutputStream f = new FileOutputStream("d:\\error.csv", true);
PrintStream p = new PrintStream(f);
// Write data to file
p.println( part1 + ResponseCode + part2 );
// Close File(s)
p.close();
f.close();
}
I'm trying to do a simple test where the HTTP request is doing a POST that is passing in a json file from c:jmeter/tests/payloads where the directory no longer exists. (let's say someone accidentally deletes it...)
The issue is the test is stopping (see below) and never getting to the BeanShell to write the error out to a log file. I need to capture all error responses, and only error responses.
I'm not sure how to handle this. I've read Jmeter. BeanShell PostProcessor and others, but they doesn't address the issue of what happens when it doesn't get to the BeanShell.
Any help is appreciated!
org.apache.jorphan.util.JMeterStopThreadException: End of sequence
at org.apache.jmeter.functions.FileToString.execute(FileToString.java:105)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:142)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:118)
at org.apache.jmeter.testelement.property.FunctionProperty.getStringValue(FunctionProperty.java:101)
at org.apache.jmeter.testelement.AbstractTestElement.getPropertyAsString(AbstractTestElement.java:274)
at org.apache.jmeter.config.Argument.getValue(Argument.java:146)
at org.apache.jmeter.protocol.http.util.HTTPArgument.getEncodedValue(HTTPArgument.java:236)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sendPostData(HTTPHC4Impl.java:1111)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.handleMethod(HTTPHC4Impl.java:453)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:329)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:74)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1146)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1135)
at org.apache.jmeter.threads.JMeterThread.process_sampler(JMeterThread.java:434)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:261)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.FileNotFoundException: File 'C:\JMeter\test\payloads\Request_1.json' does not exist
at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299)
at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1711)
at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1734)
at org.apache.jmeter.functions.FileToString.execute(FileToString.java:102)
SOLUTION
Based on Dmitri's feedback, I've switched from a Beanshell PostProcessor to Beanshell Assertion. After some tweaking, I got it to work where it now writes only errors (response != 200) to an errors.csv file. Instead of appending the file from a previous run, it overwrites with each run so only the last run's errors are captured.
If anyone thinks my solution could be improved, I'd be happy to receive the feedback. Thanks again to Kiril and Dmitri.
import org.apache.jmeter.services.FileServer;
if (ResponseCode != null && ResponseCode.equals("200") == true) {
SampleResult.setResponseOK();
}
else if (!ResponseCode.equals ("200") == true ) {
Failure = true;
FailureMessage ="Creation of a new record failed. Response code " + ResponseCode + "." ; // displays in Results Tree
print ("Creation of a new record failed: Response code " + ResponseCode + "."); // goes to stdout
log.warn("Creation of a new record failed: Response code " + ResponseCode); // this goes to the JMeter log file
// Static elements or calculations
part1 = "Unable to generate a new record via POST. The response code is: \"";
part2 = "\". \n\n For response code = \'Non-HTTP ressponse\', verify the payload file still exists. \n For response code = 409, check the recordTypeId and recordGrpId combination for validity. \n For response code = 500, verify the database and its host server are reachable. ";
// Open File(s)
FileOutputStream f = new FileOutputStream(FileServer.getFileServer().getBaseDir() + "\\error.csv");
PrintStream p = new PrintStream(f);
// Write data to file
p.println( part1 + ResponseCode + part2 );
// Close File(s)
p.close();
f.close();
}
There are no ResponseCode, Failure and FailureMessage in the Beanshell PostProcessor, switch to Beanshell Assertion instead.
Your ResponseCode.equals("200") clause assumes successful response, error responses usually have response codes > 400
See How to Use BeanShell: JMeter's Favorite Built-in Component guide for comprehensive information on Beanshell scripting in JMeter.
Jmeter overwrites your error.csv file instead of appending to it because you reopen it on every assertion call. Try to open it beforeheand, e.g. in separate Beanshell Sampler in setup thread group:
file = new FileOutputStream("error.csv", true);
bsh.shared.custom_log = new PrintStream(file)
And then use it in your beanshell assertion in a way like:
if (ResponseCode.equals("200")==false) {
bsh.shared.custom_log.println( part1 + ResponseCode + part2 );
}
Btw, AFAIK, you didn't need this part at all, because http responses with code 200 are OK by default:
if (ResponseCode != null && ResponseCode.equals("200") == true) {
SampleResult.setResponseOK();
}
I did't tested the code so there might be typos, but very similar one works for me.
Beanshell shared values are accessed under lock, so beware of possible performance issues if you writes to it heavily. With script like this and fairly short strings (50-100 chars), i'v got ~1k writes per second without significant impact on jmeter perfomance.