The postman collection passes when we execute the run manually but it fails often in the monitoring section of postman - postman-collection-runner

The issue is with the monitors. The postman collection passes when we execute the run manually but it fails often in the monitoring section of postman. I would like to determine the cause.

Related

Creating JMeter tests using BlazeMeter plugin, same day creation works perfectly in JMeter, the next day auth succeeds but other calls fail w/401

I'm using BlazeMeter plugin to create JMeter scripts. I've been able to create multiple scenarios and merge them into one JMeter test. All of the scenarios have an initial Auth with then a series of GET's and PUT's. The JMeter tests import successfully and I'm able to run them and get results but when I try on day two (the next day) to run them, the auth works fine but then I get a series of 401 unauthorized on the GET's and PUT's so I'm trying to figure out what setting is causing it to run the same day I create but not the day after. I've messed with the cookie, cache and auth manager settings to no avail (i.e. tried clearing/non clearing), but it doesn't work. Trying to understand why it would work on the day I created it but not the next. Any help would be appreciated.
Most probably your application uses some form of security token like Bearer Token which has some limited life time, i.e. one hour or one day so when you record your test scenario you're able to replay it successfully for limited amount of time.
But when the token expires you're not able to replay it successfully because you're not authorized anymore.
The solution would be performing the correlation: the process of extracting a dynamic value using a suitable JMeter Post-Processor and save it into a JMeter Variable. Once done replace the recorded hard-coded value with the dynamic variable from the Post-Processor. This time when the virtual user logs in he will get a "fresh" token so you will be able to replay your test successfully.
You can try out BlazeMeter Proxy Recorder, it's capable of exporting recorded scripts in "SmartJMX" mode with automatic detection and correlation of dynamic parameters. See How to Cut Your JMeter Scripting Time by 80% article for more details.

Why the application is throwing error when I execute a search transaction via Jmeter proxy, but not when I manually execute the transaction?

I am facing this problem, and I suspect it is security related.
I am using Jmeter corelation recorder to record my scripts for testing,
I tried recording a search transaction in my application which is a document management system and as soon as I search(after Login) , it throws an error, while when I run the functionality manually it runs well.
Should I try forcing Jmeter to use my local proxy for recording or even playback(I have developed the script with Blazemeter extension as the Jmeter proxy recording was not working) ? The script however fails to playback the search transaction, server returns a 500 error.
I also see that Blazemeter extension has no problem recording the search transaction, which makes me suspect, that probably a security certificate is missing when I try recording via Jmeter proxy.
Correlation recorder doesn't do any magic when it comes to identifying dynamic elements and coming up with proper regular expressions to handle them, you still need to do it manually prior to starting recording.
With regards to JMeter recording failing: yes, you need to import the JMeter's certificate into the browser, this way JMeter will be able to intercept and decrypt HTTPS traffic, see HTTPS recording and certificates chapter of JMeter's documentation on HTTP(S) Test Script Recorder for more details.
If you add View Results Tree listener and see the responses, most probably your login fails somewhere somehow hence search fails due to your virtual user fails to login.
So make sure that
JMeter sends the same network requests as the browser does
all dynamic values are correlated
HTTP Cookie Manager is added to your test plan
given all above criteria are implemented you shouldn't experience any problems.
There is a 3rd-party solution: BlazeMeter Proxy Recorder which seems to be capable of automatically detecting and correlating dynamic parameters, if you're new to this concept - even if it won't resolve your problem fully at least it will give you some clue.

Blazemeter Script Runninf on Jmeter doesnt save data in DB

I am creating a script through Blazemeter (which storing some data in DB) and running through JMeter. Script is running fine but data is not stored in DB. What I am missing?
The chance of replaying recorded script is minimal as modern web applications widely use dynamic parameters for security reasons or client-side state tracking
Make sure to add HTTP Cookie Manager to your Test Plan
Make sure to perform correlation of any dynamic parameters
With regards to "Script is running fine" - JMeter automatically treats responses with HTTP Status Codes below 400 as successful, so "green" result doesn't necessarily mean that your script is doing what it is supposed to be doing. Try inspecting responses using View Results Tree listener, most probably you're getting stuck at login page. Going forward you can consider adding Response Assertions in order to perform extra checks against the responses.

Perform actions before all WebTests in Visual Studio

I want to run some code (obtain a oauth valid token) before sending the SOAP requests of my webtests.
I am using Visual Studio.
I want to run the code that obtains the OAuth code before ALL test, not on each one.
¿Is that possible?
Thnaks!!
Assuming the web tests are being run as part of a load test then you can create a load test plugin and run some code from the load test starting event.
Your question reads as if you need to run one web test (possibly it contains just one request) to get the token and then run all the other tests. This can be achieved by using two scenarios in the load test. The first scenario contains the get-token web test, it is set to have one user and one iteration. The second scenario runs all the other web tests, the only change to it is to set its Delay Start Time property to a value that allows the get-token web test to complete.

my jmeter script is running inspite of the application server being shut down

I created a jmeter script for an MSTR application. The server on which this application is hosted was shut down by the Development team but my script is still running successfully.
Why is the script not giving errors??
In case of HTTP Requests JMeter automatically treats all HTTP Status Codes which are less than 400 as successful.
You can consider adding i.e. Response Assertion to ensure that the test is doing what it needs to do and, expected information is present at the page, not expected information is not present, etc. You can also set maximum response time via Duration Assertion, check response for being HTML/XHTML/XML-compliant via HTML Assertion, etc.
See How to Use JMeter Assertions in Three Easy Steps guide for comprehensive information on conditionally failing JMeter samplers using assertions
You are getting an impression that the script is running successfully based on the response code you are receiving. For correctness of the tests, it is advised that you add response assertions to your scripts and add certain text as a pattern which is expected as result of successful response for respective request.
In this case you also need to make sure that you don't add response assertion for each and every request as it can make the JMeter script heavy to execute and JMeter may run out of memory if appropriate memory is not allocated.
Add a response assertion and re-run the test and make practice to use it to validate the correctness of your script.

Resources