Can we perform load tests on EPP applications? The application is a Windows batch file that interacts with the server. Which performance testing tool would be suitable for use? I need some help with this.
The key point is "interacts". If your goal is to simulate hunderds/thousands of Windows machines concurrencty running the batch file the easiest way would be replicating the network traffic generated by these .batch files so you could replay it at scale.
So you need to analyze the network protocol(s) which is(are) under the hood of the batch file used for communication with the server and check if the network footprint can be simulated using JMeter's built-in Samplers or JMeter Plugins, if the protocol communication can be replayed - you can just increase the number of threads (virtual users) in JMeter's Thread Group to simulate multiple simultaneous batches execution.
If batch file uses HTTP protocol for interacting with the server - you will be even able to record the traffic using JMeter's HTTP(S) Test Script Recorder, see How to Run Performance Tests of Desktop Applications Using JMeter article for more details.
It is also possible to kick off multiple batches instance using OS Process Sampler but in this case you won't get reasonable metrics and KPIs
Related
Can I use only Dynatrace for Load testing/Soak Testing/Capacity Testing etc. in place pf Apache JMeter (or other Testing tools)?
I can see load testing reports in Dynatrace but is Dynatrace alternative to Apache JMeter (or other JMeter alike testing tools)?
Dynatrace is an APM tool, it will not create any load but it can be used for collecting various metrics from the system under test like CPU, RAM, Network, Disk, Swap usage, HTTP calls, database calls, application-specific metrics, etc.
JMeter is the tool which generates the load by simulating behaviour of real user, but it doesn't collect any metrics from the system under test (unless you use a special plugin like JMeter PerfMon Plugin), it just sends requests, waits for response and measures time in-between as well as other metrics like connect time and latency, after that it calculates average response times and percentiles so you can correlate increasing load (number of active threads - virtual users) with the changing response time or errors per second or transactions per second
So:
JMeter (or other load testing tool) is used for generating the load
Dynatrace (or other APM tool) is used for monitoring the application while it is under the load to figure out the root cause of the perf
I'd few ques on technical details of JMeter mostly pertaining to distributed setup vs independent JMeter engines (since JMeter controller can become a bottleneck in case of several JMeter load generators). Would be great if anybody can help with the understanding here -
How is JMeter distributed setup orchestrated by JMeter controller (i.e. called master or client)? Can we use the same logic to synchronize test among independent JMeter engines (independent mode)?
Is there a way to pool connections across vUsers?
Function of ASYNC_QUEUE in backend listener and it's expected side-effects in independent mode (mentioned above), what happens when queue is full?
Does/Is there a way for JMeter to execute javascript/act as headless browser?
How does DNS resolution happen for JMeter? Does it resolve for each vuser?
Your "question" looks like a compilation of interview questions rather than something connected with your single current concern and I don't think it's a proper place/way to ask it, I believe it should be: one post - one question.
Whatever
How is JMeter distributed setup orchestrated by JMeter controller - JMeter master sends .jmx script to slaves and collects results from them. Theoretically you can implement your own mechanism for delivering the test plan and eventual dependencies to the individual JMeter engines and running the test at the same time. Then you will need to collect the .jtl results files from the engines and combine it into a single one.
Is there a way to pool connections across vUsers? - JMeter does it internally
When the queue is full no more new sample results will be taken for processing by the backend listener so the results won't be "realtime" anymore, you will see the new results as free slots will be appearing in the queue
For JMeter per-se - no, AJAX calls can be simulated using Parallel Controller, for client-side performance testing, JavaScript execution profiling and rendering speed measurement you will need to use a read browser, no matter normal or headless, there is WebDriver Sampler plugin providing JMeter integration with Selenium
DNS resolution is dependent on underlying OS and/or JVM DNS resolution implementation, there is DNS Cache Manager which enables overriding hosts entries and using custom DNS resolver so each thread looks up the IP address on its own
I'm monitoring the connect time and latency to connect from jmeter machine while running in GUI mode and that is in within acceptable limit.
Should we strictly follow non GUI mode even though I can able to perform load test with GUI mode?
I'm targeting 250 TPS and able to achieve that ..I have increased my memory and monitoring CPU and memory of load generator is below 60%.
Should I go for non GUI mode ?
The main limitation is that each event in the queue is being handled by a single event dispatch thread which will act as the bottleneck on your JMeter side.
My expectation is that your "250 TPS" look like:
while it should look like:
So check how does your load pattern look like using i.e. Transactions per Second listener (installable via JMeter Plugins Manager)
Also check how does your JVM look like especially when it comes to garbage collection, it can be done via i.e. JVisualVM, most probably you will see the same "chainsaw" pattern
You don't need to follow JMeter best practices, but
you may encounter issues to achieve specifc goals (as TPS)
your machine can't execute GUI or have low resources
you execute JMeter using a script or build tool as Jenkins
Also it's better to be familiar with JMeter CLI (non GUI) and its report capabilities
JMeter supports dashboard report generation to get graphs and statistics from a test plan.
Also it will be needed for using distributed testing
consider running multiple CLI JMeter instances on multiple machines using distributed mode (or not)
CLI also useful for Parameterising tests
The "loops" property can then be defined on the JMeter command-line:
jmeter … -Jloops=12
My application will take jmx file as test plan from user and execute it. I've JMeter distributed setup at the backend of my application (1 JMeter client and multiple JMeter servers).
The challenge I'm facing is with the #threads required to execute the test plan.
If I take #instances (JMeter servers executing the test) and #threads explicitly from the user, how do I configure user-provided test plan with this Thread-group before execution? This will allow me to verify these values with my cluster and machine capacity beforehand.
I had read about Include Controller, but that necessarily requires user test plan to have Test Fragment.
If not explicit, how do I verify the Thread group configuration in user test plan? Suppose, user has put large #threads in the test plan that could overload my JMeter server machine, how do I verify this value and probably warn the user to limit the #threads?
There is no such a thing as "#threads_per_instance" in JMeter world, if you run JMeter in distributed mode each server (or slave or load generator) executes .jmx test plan provided by the client as it is so if user uploads file with 100 threads in a Thread Group and you have 1 slave - you will have 100 users. If you have 2 slaves - you will have 200 users, etc.
Another thing is that you cannot tell how many threads you will be able to simulate from a single JMeter slave beforehand as it mainly depends on the nature of the test: request/response size, number of pre/post processors, assertions, listeners, etc. I would recommend implementing some form of JMeter slaves health monitoring and warn the user if CPU or RAM or Network usage starts exceeding 80%. It can be easily achieved using JMeter PerfMon Plugin
We don't know what technologies are being used in "your application" so this question is too broad. JMeter .jmx scripts are just XML files so you should be able to use XML parser to extract the number of threads in all Thread Groups and "validate" it somehow.
We have a Rest Server talking to 128 devices. I need to send 10 API calls to each device and to all 128 devices at the same time. So the Rest server needs to handle 1280 at a time every 15 mins. I am using jmeter and can create 1280 threads to make each API call unique to go to all 128 devices manually. Is there a better way to do it other than manually typing and populating the threads?
Basically, I am looking to automate the process of creating the threads through a script and load the jmx to GUI, or better/faster way using GUI itself
Normally the following test elements are being used:
Recording: using HTTP(S) Test Script Recorder for automation of the HTTP Request samplers creation. The idea is that you execute your test scenario on mobile device and JMeter captures the traffic and creates the relevant requests.
Parameterization: if your want to use different data for different virtual users you can configure JMeter to read it from external sources (files, databases, etc.), the most commonly used test element is CSV Data Set Config
Synchronizing Timer: this guy ensures that samplers it its scope are executed at exactly the same moment. Normally JMeter executes requests as fast as it can, but it doesn't guarantee that requests will happen at exactly the same moment unless you use the Syncrhonizing Timer.