I was using non-GUI mode to perform a remote load testing with Jmeter from master server (Linux) to 5 slave servers (Linux). 5x "n" users have been run, "n" users on each server.
The results have been written to master server.
There are samples from all servers in results file but they relate to the number of active users from particular servers ("n") and not from all servers (5x "n").
There are no information in the result file about the real number of active users on all the servers.
As a result, a maximum number of active users is "n" on generated graphs which does not reflect the real load (5x "n" users).
Has anyone got a similar problem?
Is there anything I can do to correct the results already gathered?
Should I change any JMeter parameter to get the correct results in the next run?
Short Answer:
This is normal and no, there's nothing in JMeter you can do to fix it.
Long Answer:
Each Load Generator creates a number of threads n: the threads will be numbered 1-n. When the Controller collects all of the information, it sees 5 results for Thread 1, 5 results for Thread 2, ... Thread n. The Controller has no way of knowing that each of them are 5 separate concurrent threads and not just the same thread 5 sequential times.
Fixing it:
It depends on what you mean by a maximum number of active users is "n" on generated graphs. If this is something inside JMeter, then no, you can't fix it.
If it's a report-generator that you have created yourself, then yes, you can fix it by passing in the number of load generators.
Related
I'm doing a performance test with some instances of JMeter and something came up when I was analyzing the JMeter report.
To explain the context, I'm using 3 JMeter servers and each one launches 200 threads, the test runs for 1 minute on non-GUI mode. During the execution, I can clearly see that I have 600 active threads (200 per server).
At the end, my CSV file shows all samples from the 3 servers (I can see this by enabling "hostname" on the jmeter.properties file).
My problem:
On the report I have the Dashboard, that shows the data of my 3 servers (the samples number is the same that the CSV lines, which has all 3 servers).
At the same time, when I see my Active threads over time graph, it only shows 200 (which for me is just 1 server).
JMeter doc says that on the Times vs Threads graph:
In distributed mode, this graph shows a horizontal axis the number of threads for 1 server. It's a current limitation
And this only for this graph, which means that the other should display the information from all 3 servers...
Am I missing something? Why does the Active threads over time graph shows only 200 threads, even if I saw 600 on my console screen?
Follow Active Threads Over Time Listener notice you need to change thread group names:
If you are doing distributed tests, you should name the thread groups like this in your test plan:
${__machineName()}_My Threadgroup name
I have a test plan where I have to pick 100 users from a CSV file. If I give threads count as 500 and loop count as 1, then I can see 490 are failed and only 10 are passed. But at the same time, if I give threads as 5 and loop count as 100, all the tests are passed. Aren't these same where the total number of requests are 500?
Is this because 500 threads and 5 threads?
If you start 500 threads, By default, the file is only opened once, and each thread will use a different line from the file. However the order in which lines are passed to threads depends on the order in which you execute, which may vary between iterations.
Change Your Jmeter CSV-DataSet-Config property "Recycle on EOF - True"
; Your Issue will solve
Hope This Helps!
You get different results because you apply different load pattern. JMeter acts as follows:
Each Thread Group kicks off threads (virtual users) within the ramp-up period
Each Thread starts executing Samplers upside down (or according to the Logic Controllers)
When the thread doesn't have any more samplers to execute or loops to interate it is being shut down
So reasons could be in:
Your application isn't capable of handling 500 users. Checks its logs for error details. If there are no specific errors it might be lack of hardware resources, re-run your test with JMeter PerfMon Plugin Telemetry to check the impact of the increasing load onto hardware resources consumption
Your application underlying components is not suitable for high loads. Some application and database servers come with connection limitations, low memory allocation, etc. in other words configuration suitable for development and debugging. Production deployment assumes totally different configuration therefore it needs to be inspected and amended
Your JMeter instance cannot create the required load. Like in point 2 JMeter default configuration is good for tests development, however when it comes to running load tests you need to mind some important points:
Increase JVM Heap size for JMeter
Run your test in non-GUI mode
Disable all the listeners during test run
See Reducing resource requirements chapter of JMeter's User Manual for more information
I have recorded a script against the application that I want to test. Now, I am having a hard time arriving at the decision that what is that number that the application will run without any issue and to find out the max number of users. Here is what I have done-
I have run the Jmeter script for 10, 50, 100, 150 users
Until 50 users, it runs like a charm. After about 80 users the throughput starts to come down and some samples do not show up in Aggregate Report.
I see heap memory problems in my console for about 150 users over period of time. Is it the application problem or my machine problem?
Do you have an article where I could read about how to come to a conclusion about THE number?
UPDATE- after increasing the heap size, it is running smoothly for 100 users. I am even more confused now
Thank you
The problem can be anywhere!
Server Performance Metrics collector:
First you need an agent running in the application server to monitor the server performance while you are running the test.
This link will give you an idea about the set up.
JMeter Best Practices:
I think that you are running your test in GUI mode with listeners. Most likely the problem is with your machine/your test. Ensure that you follow this.
Samples not showing in aggregate Report:
You already asked a question on this in SO. Do not select 'Successes' in the Log/display only section of the listener while writing the results in the jtl. It will not write the failed requests details. You might need all the results. Once the jtl is created, you can always filter 'Success' only results as and when you want.
I'm running a distributed test using 5 JMeter clients (slaves). Each client is set to run 50 users. At the conclusion of the test I generate a series of graphs from the resulting JTL along with a SynthesisReport. The SynthesisReport details 250 samples for each request, as you'd expect, however the TimeVsThreads and the ThreadsStateOverTime peak at 50 users, as if they were showing the results from just one of the clients.
I've confirmed that the jmeter.properties files for each client are the same as I suspected that it was possibly an issue with the each clients results file configuration and settings for saveservice.
I can't imagine this is by design, has anyone experienced something similar and if so how did was it solved?
As per documentation:
http://jmeter-plugins.org/wiki/ActiveThreadsOverTime/
Just name your thread groups using a unique id for each generator ( hostname or a property you pass to injector and use with function __P if you have more than 1 injector per host) and it will work fine.
This is normal for Jmeter Distributed Testing.
The reason this occurs is that each load generator separately starts User Threads 1-50, so when cmdrunner runs, it sees 1 responses from each User Thread 1 (5 total), but can't differentiate between them.
If you're using a custom reporter tool (that wraps cmdrunner), you can multiply your peak users by your load generators to display a more accurate number at the top of your Report. But as long as you're calling cmdrunner, you won't be able to see the actual number of users on your graphs.
This is normal behavior of JMeter. 5 clients will run each 50 threads.
Open JMeter and run on all remote hosts, and check Active threads each will run 50 threads.
The default behavior of JMeter definitely seems to just duplicate your test plan across servers. So, if the test plan has 10 "threads", running it against X servers will yield 10x threads.
Is there any way to make this more intelligent? For example, maybe I only want one copy of some HTTP thread running even though I have 5 servers to distribute a more intense load.
Another example...I want to ensure that my sampler uses unique IDs for each thread, but my service requires that the usernames be pre-provisioned so they can't be preprovisioned...I haven't been able to find a straightforward way to coordinate this (statelessly) across my distributed servers.
A "simple" implementation might be if JMeter had distributed testing aware variables built in so the client sent the server something like ServerID and ServerCount so that the test plan could use the numeric serverId as a prefix or mod by the server count. Alternatively, JMeter could have an option to shard thread_num so that if you say 10,000 threads and have 10 servers, it will run 1,000 threads on each server with thread_num never being duplicated across the distributed test for a given sampler (Example, skip thread_num if thread_num % serverCount != serverId).
Any thoughts on the best way to accomplish this?
One approach to have distributed test-aware variable is to start each jmeter-server with different variable value:
bin/jmeter-server -Jvariable=valuehost1
And then in your test script just use:
${__P(variable)}