Sample Timeout is a useful tool for preventing "stuck" request that pauses the test, but it seems that it isn't fully supported:
For this to work, the sampler must implement Interruptible. The following samplers are known to do so:
AJP, BeanShell, FTP, HTTP, Soap, AccessLog, MailReader, JMS Subscriber, TCPSampler, TestAction, JavaSampler
For example SmtpSampler can't be interrupted with Sample Timeout.
Is there a specific reason some samplers can't be interrupted or should an enhancement be open for Smtp sampler or all samplers?
Is Runtime Controller is a better way to achieve same requirement?
The reason for some samplers not being interruptible is that they don't implement :
org.apache.jmeter.samplers.Interruptible
The reason behind this, is just lack of time, so yes you can open an enhancement and contribute a PR if you like.
Runtime Controller doesn't interrupt running samplers, it will just not call some of it's children if some time has been exceeded. So I think it does not answer the requirement.
Related
how can i reduce the retry from 30 .
when I get the below error
java.lang.IllegalStateException: More than 30 retries - stopping with exception
at kg.apc.jmeter.timers.VariableThroughputTimer.stopTest(VariableThroughputTimer.java:299) ~[jmeter-plugins-tst-2.5.jar:?]
also, how can I reduce the session time out for the HTTP requests made. ?
This 30 tries number is hard-coded therefore the only way to "reduce" this is to amend the number of tries, recompiling the plugin and replacing the .jar in the "lib/ext" folder of your JMeter installation.
The error you're getting means that the Throughput Shaping Timer tries to stop the test gracefully and fails to do it so it terminates it forcefully. You could take a thread dump and see what exactly your threads are doing and where they're stuck.
Make sure to follow JMeter Best Practices as one of the common reasons for this behaviour is improper JMeter tuning for high loads or lack of resources like CPU or RAM.
If you want to specify the connect/response timeout for JMeter's HTTP requests it can be done at "Advanced" tab of the HTTP Request sampler (or even better HTTP Request Defaults)
I am new to performance testing. I have a task on measuring the web application performance. I need to find out which modules/calls are causing deadlock, timeout and memory issues.
Q1. How can I use JMeter to find out deadlock, memory and timeout issues? If I do the following steps, it is the right way to trace those issues?
create a test plan in JMeter, which contains multiple Thread Group.
In each thread group, it contains multiple HTTP requests and 200 or
more users plus infinite loop.
Monitor JMeter results and SQL
profiler for deadlock.
Q2. JMeter is the right tool for tracking those issues? Or, should I use browser based performance testing tool such as LoadNinja, LoadView?
Thanks
Bonnie
Q1 JMeter per se doesn't provide any toolchain to detect deadlock and memory issues, the HTTP Request sampler (or even better HTTP Request Defaults) provides possibility to set the timeouts, if the value is blank - it will default to operating system timeout or web server timeout, whatever comes the first
If you conduct some form of stress test, i.e. start with 1 virtual user and gradually increase the load at some point you will see that response time starts growing and number of requests per second starts decreasing. So it's the point of maximum system performance and after that the performance will be degrading.
To monitor application under test memory you can use JMeter PerfMon Plugin, it will allow you to state whether the lack of RAM is the cause of the performance issue
With regards to deadlocks, it should result in HTTP Request sampler failure (or timeout), JMeter won't give you the underlying reason, but it will give you the timestamp and you should be able to check what happened with your application/database at that moment.
Q2 well-behaved JMeter test must produce the same network footprint as a real browser, if your test plan is good enough the system under test shouldn't be able to distinguish whether it's being hit by JMeter or by a real user using the real browser. JMeter will not give you client-side performance metrics like page rendering time or JavaScript execution time as:
JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).
I'm trying to setup a performance test for a websocket application, using JMeter.
The request is {"type":"subscribe_rq","id":1,"ts":"2018-10-16T00:00:00","data":{"sinceSeq":0}}.
Response is multipart and sequential; initial response and an update every second, as long as the connection is open. (I checked this with "WebSocket Test Client", a chrome extension).
Currently, I only get the first main response, but not the updates. Rather not sure how to get these updates. How to achieve this in JMeter? That is, how to keep the connection open for a specified period (say 5 secs) and receive the multiple responses during that period and assert it?
To keep the connection open I have a Constant Timer with 5 secs in Thread Delay. Not sure if this will work...
Going forward please remember to include essential parts of your query into the question itself, i.e. output from the "Network" tab of the browser developer tools or screenshot of this WebSocket Test Client (whatever it is) could tell the full story.
In the mean time, there is a project: JMeter WebSocket Samplers by Peter Doornbosch which has many useful sample JMeter Test Plans including the one which you can use as the basis: Single read sample.jmx which queries the data in the loop over the single WebSocket Connection.
Check out JMeter WebSocket Samplers - A Practical Guide article to get started with the WebSocket Samplers.
Was wondering if anybody has tried to use jmeter to test gRPC application.
I was hoping that
I could write a gRPC client class with a non-blocking/asynchronous stub that makes non-blocking calls to the server,
Create a Jar of the above client
Import the Jar to JMeter
Use the Java method in Jmeter BeanShell sampler
before investing time in trying the above I wanted to see if any body has tried something similar and
if above workaround work?
will each thread create a separate TCP connection?
We have tried the load test with python client and locust.io but python gRPC is not gevent compatible and even with async call e.g. stub.GetFeature.future, we are hitting a limit on the request per second per process (async call doesn't seem to be async, GIL bottleneck, once TCP stream)
if above workaround work?
Your solution will work. But if you need it long term, I would recommend, rather than having client class and using BeanShell sampler, implementing custom Java Sampler. It's very practical, since work-wise it will be similar/same as implementing custom client + BeanShell sampler script, but Java sampler is typically more efficient than BeanShell sampler, and maintainability of such solution will be better (you won't have 2 co-dependent components to maintain).
A more fancy option is to create your own JMeter Plug-in (the link I provide here is old, and not very accurate, but it's a good starting point). This is quite an investment, but might be worth it eventually if you find that a simpler solution generally works, but has some major limitations, or you need higher level of configurability and control.
will each thread create a separate TCP connection?
Each thread runs independently, but whether each thread will have its own connection will depend on how you implemented them. In straight forward implementation (where sampler creates and destroys connection), each thread will have a separate TCP connection. But JMeter has properties shared among threads, which, among the rest, can contain objects. So you could share a connection between threads that way. Or you can implement configuration element, which could hold a connection pool, shared by all threads.
Please see grpc-client java project (maven) that creates a gRPC client with JMeter samplers to enable stress testing with JMeter
Build the project with maven and copy the generated jar to JMeter lib/ext folder (e.g. /usr/local/apache-jmeter-3.1/lib/ext/) so that the samplers are in the class path
After that when you launch JMeter, you should be able to see the classes in the "java request" samplers.
The mobile app that I am testing with Jmeter makes 4 asynchronous API calls when logging in. Is there any way to simulate this with JMeter ?
Currently I can only get JMeter to make the calls synchronously, so when wrapping them all in the same transaction controller, the response time is the total of the 4 calls (instead of the highest one)
Many thanks
Currently JMeter doesn't offer a relevant test element, the easiest way of implementing your scenario is using JSR223 Sampler to perform nested asynchronous calls, something like:
See How to Load Test AJAX/XHR Enabled Sites With JMeter guide for more detailed explanation, code snippets,etc.
You can also consider developing a custom sampler with similar functionality and make it a part of your JMeter as a plugin or even share it with the community.
Take a look at Parallel Controller in JMeter.
All elements inside the Parallel Controller will be executed parallel to each other. This way, you will have one main flow and other flows with asynchronous requests, which will be executed in a parallel way.
https://www.blazemeter.com/blog/how-to-load-test-async-requests-with-jmeter