I created a script in jmeter, few positive cases and few are negative cases.
For Positive Cases - Response Code will come as 200
For Negative Cases - Response Code will come as 412.
As per Jmeter if Response Code 4xx or 5xx will be considered as Fail but in my case i am expecting result as 412 in negative cases and i want to consider that as Pass.
I tried with BeanShell Assertion but i didn't get the expected.
Code is as below:
String ErrorValue = "${ExpectedError}";
if((ErrorValue.equals("ERROR")) && (ResponseCode.equals("412")))
{
Failure = false;
}
else if(ErrorValue.equals("NO ERROR") && ResponseCode.equals("200"))
{
Failure = false;
}
else
{
Failure=true;
}
with about code i am able to check the expected error and response is same but if that is same how to change the status to pass i didn't get.
Please anyone help me.
Thanks
Sarada
Your Failure = false bit sets only Beanshell Assertion success. As far as I understand you need to change status of the parent sampler. In order to do so you need to invoke SampleResult.setSuccessful() method and set it to "true" as follows:
SampleResult.setSuccessful(true);
Full code:
String ErrorValue = "${ExpectedError}";
if((ErrorValue.equals("ERROR")) && (ResponseCode.equals("412")))
{
Failure = false;
SampleResult.setSuccessful(true);
}
else if(ErrorValue.equals("NO ERROR") && ResponseCode.equals("200"))
{
Failure = false;
SampleResult.setSuccessful(true);
}
else
{
Failure=true;
}
References:
SampleResult class JavaDoc
How to Use JMeter Assertions in 3 Easy Steps
If you are expecting a HTTP Response Code "failure" in JMeter but wish to flag the sample as successful this can be accomplished by a response assertion:
For example:
When validating a DELETE call works, we might want to re-try a GET and validate 404 as expected. Normally JMeter would consider this a failure, but in the context of our test it is not.
Add A Response Assertion to the after-delete GET call.
Apply To: Main Sample
Response Field to Test: Response Code
Check off "Ignore Status"
Pattern Matching Rules: Equals
Pattern to Test: 404
The status of failed or not is always ignored. However, only if the assertion of 404 matches will the request be a success.
For example, if the call returned a 500 jmeter would still ignore the "failed" status, but mark the sample as a failure because 500 != 404.
-Addled
Related
In my test plan I want to mark all the response with, http status code 200, 400, 422, 202 set as success.
Is there any way I can achieve this in a single assertion ?
You should also include what you have tried so far, to resolve this it is very simple,
Need to include a JSR223 assertion with the following script,
if("400".equals(SampleResult.getResponseCode()) || "200".equals(SampleResult.getResponseCode()) || "202".equals(SampleResult.getResponseCode()) || "422".equals(SampleResult.getResponseCode())) {
SampleResult.setSuccessful(true);
AssertionResult.setFailure(false);
}
else {
AssertionResult.setFailure(true);
}
Another solution could be with a Response Assertion
Add the Response Assertion into the Test Plan. This will ensure the Assertion is applied to all the Samplers (Responses)
Then configure the Response Assertion
Field to Test as Response Code
Pattern matching rules to Equals and Or
Click the Add button and add the response codes
In addition to the above configuration, you may check the Ignore Status if you want to Instructs JMeter to set the status to success initially.
Here is another solution based on the previous answer.
//Expected Response codes
def lstExpectedResponseCodes= ["400", "200", "202","422"]
//Actual Response code
String actualResponseCode=SampleResult.getResponseCode()
boolean hasResponseCode = lstExpectedResponseCodes.contains(actualResponseCode)
AssertionResult.setFailure(!hasResponseCode)
SampleResult.setSuccessful(hasResponseCode);
How to handle negative cases in JMETER, for example my expected output response is 400("There are no records") for an GET API?
In JMETER response is coming as failure or warning.
Is JMeter only handle positive scenarios like for all GET API response code should be 200 ?
Add Response Assertion as a child of the HTTP Request sampler which returns HTTP Status Code 400
Configure it as follows:
Tick Ignore status box
Set "Field to test" to Response code
Set "Pattern matching rules" to Equals
Add 400 as a "Pattern to test"
This way JMeter will pass only if the parent HTTP Request sampler returns 400 status code, otherwise it will fail.
You can add to HTTP Request Response Assertion with Ignore status checked
HTTP Responses with statuses in the 4xx and 5xx ranges are normally regarded as unsuccessful. The "Ignore status" checkbox can be used to set the status successful before performing further checks. Note that this will have the effect of clearing any previous assertion failures, so make sure that this is only set on the first assertion.
I tried with this, by adding a BeanShell Assertion with following code.
import org.apache.jmeter.assertions.AssertionResult;
String failureMessage = "";
String ResCode = SampleResult.getResponseCode();
if (!ResCode.equals("400")) {
failureMessage = "Got Response Code" + ResCode;
AssertionResult result = new AssertionResult("Expected Response 400");
result.setFailure(true);
result.setFailureMessage(failureMessage);
prev.addAssertionResult(result);
prev.setSuccessful(false);
SampleResult.setStartNextThreadLoop(true);
} else {
//failure criteria
}
I am trying to test my API response using JSON assertion in JMeter, but couldn't find out on how to achieve it. The API returns 2 values, and I need to check if the difference between these two value are consistent
API response:
{
"start": "12759898",
"end": "12759907"
}
I've tried like the above, but it seems to be wrong, as its a JSONPath variable.
Could anyone guide on how to evaluate these values? is it possible to achieve this?
It looks like a job for JSR223 Assertion
Add JSR223 Assertion as a child of the request which returns the above JSON
Put the following code into "Script" area:
def response = new groovy.json.JsonSlurper().parse(prev.getResponseData())
def start = response.start as long
def end = response.end as long
def delta = end - start
if (delta != 10) {
AssertionResult.setFailure(true)
AssertionResult.setFailureMessage('Expected: 10, got: ' + delta)
}
If the difference between start and end will not be equal to 10 - the request will be marked as failed.
More information:
Groovy: Parsing and producing JSON
Scripting JMeter Assertions in Groovy - A Tutorial
My test plan looks like,
I am running some Http requests with assertion .
I need to check assertion is passed or failed (any built-in variable is available?) .
Need to call/extract assertion result and use it for my next API .
Any one help me on this.
I solved this case using BeanShell Listener by adding the following code and you can call assertion result on jmeter like ${results}
try
{
int results;
if (sampleResult.isSuccessful()) {
log.info("Response Assertion PASSED");
results = 1;
vars.put("results","1");}
else {
log.info("Response Assertion FAILED" );
results = 2;
vars.put("results","2");}
}
catch (Throwable ex) {
log.error("Error in Beanshell", ex);
throw ex;
}
I would suggest adding JSR223 Assertion where you can get parent sampler AssertionResult instance(s) and extract the required information from it like:
def assertionResult = SampleResult.getAssertionResults()[0]
if (assertionResult.isFailure()) {
log.info("Assertion " + assertionResult.getName() + " failed")
log.info(assertionResult.getFailureMessage())
}
Demo:
See How to Use JMeter Assertions in Three Easy Steps article for more information on using assertions. Also avoid using Beanshell as it may cause performance problems, if you need to go for scripting - stick to JSR223 Test Elements
1.Use "Regular Expression Extractor" to extract assertion result.
2.Use BeanShell Sampler to get assertion result,then,use it for your next API.
In my JMeter test plan, I'm trying to write all errors out to a log. I'm using a BeanShell Post-Processor configured as follows
import org.apache.jmeter.services.FileServer;
if (ResponseCode != null && ResponseCode.equals("200") == false) {
Failure = true;
// displays in Results Tree
FailureMessage ="Creation of a new CAE record failed. Response code " + ResponseCode + "." ;
// Static elements
part1 = "Creation of a new record failed. Response code: ";
part2 = ". Sorry!";
// Open File(s)
FileOutputStream f = new FileOutputStream("d:\\error.csv", true);
PrintStream p = new PrintStream(f);
// Write data to file
p.println( part1 + ResponseCode + part2 );
// Close File(s)
p.close();
f.close();
}
I'm trying to do a simple test where the HTTP request is doing a POST that is passing in a json file from c:jmeter/tests/payloads where the directory no longer exists. (let's say someone accidentally deletes it...)
The issue is the test is stopping (see below) and never getting to the BeanShell to write the error out to a log file. I need to capture all error responses, and only error responses.
I'm not sure how to handle this. I've read Jmeter. BeanShell PostProcessor and others, but they doesn't address the issue of what happens when it doesn't get to the BeanShell.
Any help is appreciated!
org.apache.jorphan.util.JMeterStopThreadException: End of sequence
at org.apache.jmeter.functions.FileToString.execute(FileToString.java:105)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:142)
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:118)
at org.apache.jmeter.testelement.property.FunctionProperty.getStringValue(FunctionProperty.java:101)
at org.apache.jmeter.testelement.AbstractTestElement.getPropertyAsString(AbstractTestElement.java:274)
at org.apache.jmeter.config.Argument.getValue(Argument.java:146)
at org.apache.jmeter.protocol.http.util.HTTPArgument.getEncodedValue(HTTPArgument.java:236)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sendPostData(HTTPHC4Impl.java:1111)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.handleMethod(HTTPHC4Impl.java:453)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:329)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:74)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1146)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1135)
at org.apache.jmeter.threads.JMeterThread.process_sampler(JMeterThread.java:434)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:261)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.FileNotFoundException: File 'C:\JMeter\test\payloads\Request_1.json' does not exist
at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299)
at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1711)
at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1734)
at org.apache.jmeter.functions.FileToString.execute(FileToString.java:102)
SOLUTION
Based on Dmitri's feedback, I've switched from a Beanshell PostProcessor to Beanshell Assertion. After some tweaking, I got it to work where it now writes only errors (response != 200) to an errors.csv file. Instead of appending the file from a previous run, it overwrites with each run so only the last run's errors are captured.
If anyone thinks my solution could be improved, I'd be happy to receive the feedback. Thanks again to Kiril and Dmitri.
import org.apache.jmeter.services.FileServer;
if (ResponseCode != null && ResponseCode.equals("200") == true) {
SampleResult.setResponseOK();
}
else if (!ResponseCode.equals ("200") == true ) {
Failure = true;
FailureMessage ="Creation of a new record failed. Response code " + ResponseCode + "." ; // displays in Results Tree
print ("Creation of a new record failed: Response code " + ResponseCode + "."); // goes to stdout
log.warn("Creation of a new record failed: Response code " + ResponseCode); // this goes to the JMeter log file
// Static elements or calculations
part1 = "Unable to generate a new record via POST. The response code is: \"";
part2 = "\". \n\n For response code = \'Non-HTTP ressponse\', verify the payload file still exists. \n For response code = 409, check the recordTypeId and recordGrpId combination for validity. \n For response code = 500, verify the database and its host server are reachable. ";
// Open File(s)
FileOutputStream f = new FileOutputStream(FileServer.getFileServer().getBaseDir() + "\\error.csv");
PrintStream p = new PrintStream(f);
// Write data to file
p.println( part1 + ResponseCode + part2 );
// Close File(s)
p.close();
f.close();
}
There are no ResponseCode, Failure and FailureMessage in the Beanshell PostProcessor, switch to Beanshell Assertion instead.
Your ResponseCode.equals("200") clause assumes successful response, error responses usually have response codes > 400
See How to Use BeanShell: JMeter's Favorite Built-in Component guide for comprehensive information on Beanshell scripting in JMeter.
Jmeter overwrites your error.csv file instead of appending to it because you reopen it on every assertion call. Try to open it beforeheand, e.g. in separate Beanshell Sampler in setup thread group:
file = new FileOutputStream("error.csv", true);
bsh.shared.custom_log = new PrintStream(file)
And then use it in your beanshell assertion in a way like:
if (ResponseCode.equals("200")==false) {
bsh.shared.custom_log.println( part1 + ResponseCode + part2 );
}
Btw, AFAIK, you didn't need this part at all, because http responses with code 200 are OK by default:
if (ResponseCode != null && ResponseCode.equals("200") == true) {
SampleResult.setResponseOK();
}
I did't tested the code so there might be typos, but very similar one works for me.
Beanshell shared values are accessed under lock, so beware of possible performance issues if you writes to it heavily. With script like this and fairly short strings (50-100 chars), i'v got ~1k writes per second without significant impact on jmeter perfomance.