Was wondering if anybody has tried to use jmeter to test gRPC application.
I was hoping that
I could write a gRPC client class with a non-blocking/asynchronous stub that makes non-blocking calls to the server,
Create a Jar of the above client
Import the Jar to JMeter
Use the Java method in Jmeter BeanShell sampler
before investing time in trying the above I wanted to see if any body has tried something similar and
if above workaround work?
will each thread create a separate TCP connection?
We have tried the load test with python client and locust.io but python gRPC is not gevent compatible and even with async call e.g. stub.GetFeature.future, we are hitting a limit on the request per second per process (async call doesn't seem to be async, GIL bottleneck, once TCP stream)
if above workaround work?
Your solution will work. But if you need it long term, I would recommend, rather than having client class and using BeanShell sampler, implementing custom Java Sampler. It's very practical, since work-wise it will be similar/same as implementing custom client + BeanShell sampler script, but Java sampler is typically more efficient than BeanShell sampler, and maintainability of such solution will be better (you won't have 2 co-dependent components to maintain).
A more fancy option is to create your own JMeter Plug-in (the link I provide here is old, and not very accurate, but it's a good starting point). This is quite an investment, but might be worth it eventually if you find that a simpler solution generally works, but has some major limitations, or you need higher level of configurability and control.
will each thread create a separate TCP connection?
Each thread runs independently, but whether each thread will have its own connection will depend on how you implemented them. In straight forward implementation (where sampler creates and destroys connection), each thread will have a separate TCP connection. But JMeter has properties shared among threads, which, among the rest, can contain objects. So you could share a connection between threads that way. Or you can implement configuration element, which could hold a connection pool, shared by all threads.
Please see grpc-client java project (maven) that creates a gRPC client with JMeter samplers to enable stress testing with JMeter
Build the project with maven and copy the generated jar to JMeter lib/ext folder (e.g. /usr/local/apache-jmeter-3.1/lib/ext/) so that the samplers are in the class path
After that when you launch JMeter, you should be able to see the classes in the "java request" samplers.
Related
I'd few ques on technical details of JMeter mostly pertaining to distributed setup vs independent JMeter engines (since JMeter controller can become a bottleneck in case of several JMeter load generators). Would be great if anybody can help with the understanding here -
How is JMeter distributed setup orchestrated by JMeter controller (i.e. called master or client)? Can we use the same logic to synchronize test among independent JMeter engines (independent mode)?
Is there a way to pool connections across vUsers?
Function of ASYNC_QUEUE in backend listener and it's expected side-effects in independent mode (mentioned above), what happens when queue is full?
Does/Is there a way for JMeter to execute javascript/act as headless browser?
How does DNS resolution happen for JMeter? Does it resolve for each vuser?
Your "question" looks like a compilation of interview questions rather than something connected with your single current concern and I don't think it's a proper place/way to ask it, I believe it should be: one post - one question.
Whatever
How is JMeter distributed setup orchestrated by JMeter controller - JMeter master sends .jmx script to slaves and collects results from them. Theoretically you can implement your own mechanism for delivering the test plan and eventual dependencies to the individual JMeter engines and running the test at the same time. Then you will need to collect the .jtl results files from the engines and combine it into a single one.
Is there a way to pool connections across vUsers? - JMeter does it internally
When the queue is full no more new sample results will be taken for processing by the backend listener so the results won't be "realtime" anymore, you will see the new results as free slots will be appearing in the queue
For JMeter per-se - no, AJAX calls can be simulated using Parallel Controller, for client-side performance testing, JavaScript execution profiling and rendering speed measurement you will need to use a read browser, no matter normal or headless, there is WebDriver Sampler plugin providing JMeter integration with Selenium
DNS resolution is dependent on underlying OS and/or JVM DNS resolution implementation, there is DNS Cache Manager which enables overriding hosts entries and using custom DNS resolver so each thread looks up the IP address on its own
From what I have found and tested, you can use the WebSocket Sampler for Apache JMeter, however you cannot record your test requests because the WebSocket request will not be passed appropriately through and back from the Java Proxy Server.
So, what are my options? I can build every request by hand instead of recording - and keep using JMeter -very tedious.
Build a new WebSocket Sampler?
Build onto the Java Proxy Server?
Or use something besides JMeter?
Before I dive into another deep pit, a little good advice would be much appreciated! Thank you so much!
The mobile app that I am testing with Jmeter makes 4 asynchronous API calls when logging in. Is there any way to simulate this with JMeter ?
Currently I can only get JMeter to make the calls synchronously, so when wrapping them all in the same transaction controller, the response time is the total of the 4 calls (instead of the highest one)
Many thanks
Currently JMeter doesn't offer a relevant test element, the easiest way of implementing your scenario is using JSR223 Sampler to perform nested asynchronous calls, something like:
See How to Load Test AJAX/XHR Enabled Sites With JMeter guide for more detailed explanation, code snippets,etc.
You can also consider developing a custom sampler with similar functionality and make it a part of your JMeter as a plugin or even share it with the community.
Take a look at Parallel Controller in JMeter.
All elements inside the Parallel Controller will be executed parallel to each other. This way, you will have one main flow and other flows with asynchronous requests, which will be executed in a parallel way.
https://www.blazemeter.com/blog/how-to-load-test-async-requests-with-jmeter
can we have any control over the threads?
Consider i have 10 threads and i have provided my test data in .csv file. so can I control on threads like which thread should pick which data and may be some delay for few of the threads?
Also, can someone suggest me some book or online content wherein i can have information on internals of JMeter. Like when we run test plan, what all things are happening on memory side, reading of different properties files, receiving response, how threads internally works, etc.
Thanks,
Abhishek
JMeter is a very flexible and powerful tool. In theory, anything is possible it all depends on what your testing goals are. Even things not supported by JMeter can be coded in Java and easily integrated with a Java Sampler. Your question indicates you have not spent a lot of time experimenting with the tool, but hopefully my answer jump-starts that process for you.
JMeter has a lot of control features that can be used in conjunction with CSV data to control the flow of a thread. For example, use the CSV data to correctly enter the right block of a Switch Controller, validate an If Controller, or control the number of loops in a Loop Controller. Be sure you read the entire Getting Started Guide and familiarize yourself with the Component Reference Guide
In terms of how things work internally, your best bet is to build the JMeter project from source in an IDE like Eclipse. You can then step through the entire program in as much detail as you want.
Tutorial: Build JMeter from Source
Also, the /bin/jmeter file has a decent number of comments about how to properly configure JVM memory for a JMeter process.
You probably want to install at least the most basic JMeter Plugin Package.
Lastly, if you need one thread to control the behavior of another thread you can use FIFO Queues or set JVM properties via Beanshell which are global and not unique to a thread like runtime variables.
props.put("key","value");
Looked at documentation at jmeter saying java implementation of http request.
The API is best suited to single-threaded usage - various settings (e.g. proxy) are defined via system properties, and therefore apply to all connections.
This statement is confusing. Does this mean if you put 100 threads in ThreadGroup, you only have actually one java thread? Just one thread to multiple connections? Or it means something else.
I thought Jmeter is to creating multiple java threads for stress testing. We want to take advantage of multiple processors with multiple java threads for stress testing.
Does this mean if you put 100 threads in ThreadGroup, you only have actually one java thread?
No, but I agree it's confusing. 100 threads means up to 100 concurrent connections.
The documentation says that since the Java HTTP API is configured through system properties, it works best with single-threaded applications. In multi-threaded environment it's hard to apply settings to only a single thread only because system properties are global.
HTTPClient sampler (which you should prefer) doesn't have such limitations since everything is configured explicitly and on a connection basis.