How accurate are the results of Perfmon (Server health Performance) when compared to Windows Performance Monitor. Are they reliable?
I believe they are
Moreover, you can configure Windows Performance Monitor to store "interesting" counters in CSV format and use JMeter PerfMon TAIL command to read and plot the values
See How to Monitor Your Server Health & Performance During a JMeter Load Test
for more information on using JMeter PerfMon plugin
They are exactly the same and comparable with each other. Type perfmon from the run command, it will open Windows Performance Monitor.
The jmeter agent that collect metrics is right. But I want to say that the network IO that jmeter agent collect includes loopback IO, whose result is same as ifstat -l shows rather than ifstat.
Related
Can we perform load tests on EPP applications? The application is a Windows batch file that interacts with the server. Which performance testing tool would be suitable for use? I need some help with this.
The key point is "interacts". If your goal is to simulate hunderds/thousands of Windows machines concurrencty running the batch file the easiest way would be replicating the network traffic generated by these .batch files so you could replay it at scale.
So you need to analyze the network protocol(s) which is(are) under the hood of the batch file used for communication with the server and check if the network footprint can be simulated using JMeter's built-in Samplers or JMeter Plugins, if the protocol communication can be replayed - you can just increase the number of threads (virtual users) in JMeter's Thread Group to simulate multiple simultaneous batches execution.
If batch file uses HTTP protocol for interacting with the server - you will be even able to record the traffic using JMeter's HTTP(S) Test Script Recorder, see How to Run Performance Tests of Desktop Applications Using JMeter article for more details.
It is also possible to kick off multiple batches instance using OS Process Sampler but in this case you won't get reasonable metrics and KPIs
Can I use only Dynatrace for Load testing/Soak Testing/Capacity Testing etc. in place pf Apache JMeter (or other Testing tools)?
I can see load testing reports in Dynatrace but is Dynatrace alternative to Apache JMeter (or other JMeter alike testing tools)?
Dynatrace is an APM tool, it will not create any load but it can be used for collecting various metrics from the system under test like CPU, RAM, Network, Disk, Swap usage, HTTP calls, database calls, application-specific metrics, etc.
JMeter is the tool which generates the load by simulating behaviour of real user, but it doesn't collect any metrics from the system under test (unless you use a special plugin like JMeter PerfMon Plugin), it just sends requests, waits for response and measures time in-between as well as other metrics like connect time and latency, after that it calculates average response times and percentiles so you can correlate increasing load (number of active threads - virtual users) with the changing response time or errors per second or transactions per second
So:
JMeter (or other load testing tool) is used for generating the load
Dynatrace (or other APM tool) is used for monitoring the application while it is under the load to figure out the root cause of the perf
I'm monitoring the connect time and latency to connect from jmeter machine while running in GUI mode and that is in within acceptable limit.
Should we strictly follow non GUI mode even though I can able to perform load test with GUI mode?
I'm targeting 250 TPS and able to achieve that ..I have increased my memory and monitoring CPU and memory of load generator is below 60%.
Should I go for non GUI mode ?
The main limitation is that each event in the queue is being handled by a single event dispatch thread which will act as the bottleneck on your JMeter side.
My expectation is that your "250 TPS" look like:
while it should look like:
So check how does your load pattern look like using i.e. Transactions per Second listener (installable via JMeter Plugins Manager)
Also check how does your JVM look like especially when it comes to garbage collection, it can be done via i.e. JVisualVM, most probably you will see the same "chainsaw" pattern
You don't need to follow JMeter best practices, but
you may encounter issues to achieve specifc goals (as TPS)
your machine can't execute GUI or have low resources
you execute JMeter using a script or build tool as Jenkins
Also it's better to be familiar with JMeter CLI (non GUI) and its report capabilities
JMeter supports dashboard report generation to get graphs and statistics from a test plan.
Also it will be needed for using distributed testing
consider running multiple CLI JMeter instances on multiple machines using distributed mode (or not)
CLI also useful for Parameterising tests
The "loops" property can then be defined on the JMeter command-line:
jmeter … -Jloops=12
JMeter - Perfmon Metric Collector is used to monitor server resources but it's a gui functionality. So can I run test in non gui mode and then just use JMeter Perfmon Metric Collector for just monitoring remote server?
Yes. You can. Refer to Non-GUI Mode. You can set the required properties and once the test completes you can inspect the collected data by loading the file in the GUI application.
There are following options:
Either configure PerfMon Metrics Collector listener to save results into a file
and run your test in non-GUI mode. Once your test finishes you will be able to open it in GUI mode and see the charts (or parse the output CSV file with other software)
Or run your main test in command-line non-GUI mode and launch a separate JMeter instance to observe the server-side metrics in the realtime as the test goes, you will need just a Sampler which does nothing like Dummy Sampler with response time matching your metrics collection interval
Just need to know that perfmon plugin which is used in jmeter tool, does it analyse cpu/memory, disk utilization of local machine or the server where application is hosted?
Because as a user when give IP and port, we give these details of the remote machines when we perform load test.
Please let me know .
As per JMeter Scientist,
The PerfMon listener was implemented in following way: The
Host collects PerfMon, Remote nodes don't collect PerfMon.
So, Master will collect metrics from the Slaves.
This might help
PerfMon Metrics Collector fetches performance metrics from ServerAgent(s) via TCP or UDP. And it's up to you where you want to install the ServerAgent(s), on JMeter or Application Under Test side or both.
Normally ServerAgent(s) is/are installed on Application Under Test side, i.e. web servers, database servers, load balancers, etc. to measure the load on that end, however if you want to collect performance stats from load generators - feel free to install Server Agent(s) on JMeter machine(s).
See How to Monitor Your Server Health & Performance During a JMeter Load Test article for comprehensive information on PerfMon installation and usage