Below is the details of my test plan.
Number of Threads : 700
Loop Count : Forever Scheduler
Configuration : Duration : 250 seconds
At 250th second, threads begin to stop as the scheduled duration is reached.
Even after 30 mins, the last 23 threads are not stopped and the test is not completed.
Tried to run the same test through non GUI mode as well. But the response is same.
Even after 30 mins, the test doesn't stop its execution.
Can you please help if i have missed any configuration..
Most likely you are suffering from insufficient Java Heap Space allocation. By default JMeter is started with 512 Mb heap only which is fine for tests development and debugging but will be not enough when it comes to real load test (like in your scenario with 700 users).
To overcome this:
Open JMeter startup script (jmeter.bat for Windows OS family, jmeter for other operating systems)
Locate line like HEAP="-Xms512m -Xmx512m"
Change both values to be something like 80% of your total available physical RAM
Restart JMeter to pick the changes up
And yes, never run load tests in GUI mode
See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure for more recommendations on JMeter tuning and tweaking in order to get the maximum throughput from the single instance
Related
I started to monitor JMeter with script running with e.g. Jconsole and noted number of classes goes up gradually then quickly down, then gradually up again.
Is it typical behaviour of JMeter? Maybe even JVM in general? Or particular plug-in code should be involved? I'm concerned that to decrease number of classes GC is run and that can affect continuity of generated load. I was not able to find the answer via web search now.
ADDED: Test plan includes making HTTP request samplers, JSON Assertions, using concurrency thread group to increase load in steps + randomization (Random Controller, Random Variable Config Element).
ADDED 2:
Following advice by Dmitri, I run test JVM_ARGS="-Xlog:class+unload -Xlog:class+load"; jmeter ... for about 60 min (3600 sec test) I got around 116 000 classes loaded and 68 000 classes unloaded, below shows all unloaded classes with times unloaded (jdk.nashorn.internal.scripts.Script take most of occurrences and mean time confirm it happened sometime during the test - being technically correct not only at start or only at finish) (from Jupiter notebook):
time
size mean
classname
java.lang.invoke.LambdaForm 978.0 29.757680
jdk.nashorn.internal.runtime.Context 11.0 17.486000
jdk.nashorn.internal.scripts.ModuleGraphManipulator 305.0 17.489377
jdk.nashorn.internal.scripts.Script 66845.0 2308.991561
Any additional advice? What to look for further?
I would say this is due to some randomization as other mentioned test elements are unlikely to cause increase in the number of loaded classes and trigger unloading.
This is normal behavior of the JVM which can unload the classes if/when they are no longer referenced by the program.
You can add the following JVM options to your JMeter startup command:
-XX:+TraceClassLoading -XX:+TraceClassUnloading
so you will be able to see which exact classes are being loaded/unloaded at the given moment of time.
Unfortunately it is not possible to provide more information without seeing your test plan and JVM arguments, just ensure that you're following recommendations from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article to get confidence that your test will not crash due to a memory leak as JMeter gives enough freedom of shooting your own leg.
I have a test plan where I have to pick 100 users from a CSV file. If I give threads count as 500 and loop count as 1, then I can see 490 are failed and only 10 are passed. But at the same time, if I give threads as 5 and loop count as 100, all the tests are passed. Aren't these same where the total number of requests are 500?
Is this because 500 threads and 5 threads?
If you start 500 threads, By default, the file is only opened once, and each thread will use a different line from the file. However the order in which lines are passed to threads depends on the order in which you execute, which may vary between iterations.
Change Your Jmeter CSV-DataSet-Config property "Recycle on EOF - True"
; Your Issue will solve
Hope This Helps!
You get different results because you apply different load pattern. JMeter acts as follows:
Each Thread Group kicks off threads (virtual users) within the ramp-up period
Each Thread starts executing Samplers upside down (or according to the Logic Controllers)
When the thread doesn't have any more samplers to execute or loops to interate it is being shut down
So reasons could be in:
Your application isn't capable of handling 500 users. Checks its logs for error details. If there are no specific errors it might be lack of hardware resources, re-run your test with JMeter PerfMon Plugin Telemetry to check the impact of the increasing load onto hardware resources consumption
Your application underlying components is not suitable for high loads. Some application and database servers come with connection limitations, low memory allocation, etc. in other words configuration suitable for development and debugging. Production deployment assumes totally different configuration therefore it needs to be inspected and amended
Your JMeter instance cannot create the required load. Like in point 2 JMeter default configuration is good for tests development, however when it comes to running load tests you need to mind some important points:
Increase JVM Heap size for JMeter
Run your test in non-GUI mode
Disable all the listeners during test run
See Reducing resource requirements chapter of JMeter's User Manual for more information
I am doing some kind of endurance testing with a thread group of 5 threads, and add test action at the end to add delay of 15min after each loop of test execution (please refer to the screenshot), and run overall 32 loops (takes overall around 8 hours).
I let the test run at end of working day using jmeter GUI, and it should take around 8 hours to finish.
Strange thing happens, after jmeter run 4 or 5 loops of 15min, it does not run anymore loops for a couple of hours, then may run 1 or 2 loops in midnight, then continue running in the next morning.
I tried to shorten the delay from 15 min to 1 min or 5 min so that it finishes quicker, then all 32 loops finish with no problem.
So my question comes, how can this happen??? If it's because computer is dormant that jmeter halt, then how come it still runs about an hour after screenlock and also in midnight? If it's because script setting incorrect, then how come if the delay is 1 or 5min it runs ok?
Any suggestions on how to check for this issue? I checked the script settings carefully, nothing suspicious.
Thanks,
Strange thing happens, after jmeter run 4 or 5 loops of 15min, it does
not run anymore loops for a couple of hours, then may run 1 or 2 loops
in midnight, then continue running in the next morning.
Are you sure that your computer will not sleep / hibernate after about 1 - 1.5 hours of inactivity? I usually got this issue for overnight jobs. The time it runs at midnight may well be the time your antivirus software run (therefore awake your computer).
For these problems, just use a computer software out there and you should be fine. Do notice that some company also set up technical measures to prevent employees have their computer on overnight, better check them out.
I don't think this is due to Test Action sampler.
By the way, according to the documentation, you're pausing only current thread for 15 minutes while others are running as in "Pause" mode "Current Thread/All Threads" combobox value is ignored. If your target is to wait for 15 minutes before next iteration without delivering any load - consider using Synchronizing Timer or i.e. switch to Ultimate Thread Group
My expectation is that this is due to non-optimal JMeter configuration. Try the following steps
Increase JVM Heap size allocated to JMeter
Run your test in command-line non-GUI mode
Disable all the listeners, especially View Results Tree
Consider upgrade to JMeter 3.0
See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article for more information on above steps and few other JMeter tuning tips.
JMeter machines with versions: 2.13 r13365067, 2.11.20140918 |
Java: OpenJDK 1.7.0_79 |
OS: Debian 8.1
I'm having a problem where some HTTP requests seem to be processed far too long on a load injector that isn't really under load.
Examples from result files from tests with 20 vUs (with caching, on weaker load injector, JMeter v2.11) and 40vUs (without caching, on much higher spec'd load injector, JMeter v2.13):
<time_stamp>,3257,<request_name>,200,<thread_name>,true,28537,20,20,437
<time_stamp>,5158,<request_name>,304,<thread_name>,true,138,40,40,0
Memory is at 75% in the first case, and below 50% in the second. CPU doesn't seem to spike (measured in 1 sec intervals) and goes up to 20% max in both examples.
Checked the JVM's garbage collection, and it doesn't seem like the GC is at its limits at the time of the requests either (actually at no point during the test).
I noticed this in the case where I had caching (via Cache Manager with "Use Cache-Control/Expires headers..." checked) enabled, and, like in the second example above, get the unrealistic response time of 5158 ms.
This only happens at some steps during an iteration and to more than one thread, but not all.
It seems like JMeter is somehow processing the result too long, but I can't really see that my load injectors are under heavy load, to cause processing times of seconds.
Clearly this is messing up the performance statistics so I would like to know how this is happening.
Hope someone can help.
EDIT:
#First example: Case where ResponseTime >> Latecy > 0, happens on both JMeter machines (JMeter v2.11, JMeter v2.13).
#Second example: Case where ResponseTime >> Latecy = 0 happens only on the machine with JMeter v2.13.
2nd EDIT:
Turns out it doesn't matter what JMeter version I run (or on which node).
Regex'd my result file:
Of the same requested resources, cached (latency=0), with header check, about 10% took 1 second or multiple seconds. Without header check it is 6%.
You should run same JMeter version on all nodes. If this won't solve the problem, monitor your JMeter instance resource utilisation with jconsole.
I've recently started working in Jmeter and I have run a test with 40 virtual users. I've also used loadosophia plugin that uploads results to loadosophia. But I've few doubts about numbers I've got in the summary report in loadosophia.
I've run with 40 virtual users with 3 seconds as ramp up period but in loadosophia report it shows Max virtual users=17 and Average virtual users = 6.5 but Transaction count is 40 which actually should be virtual users. Does this indicate there were maximum of 17 virtual users were concurrent ?
Test run duration it shows 3 seconds, but I can see Maximum response time is 5078 milliseconds which is(5.7 seconds) how can it be test can be completed in 3 seconds while it shows some requests took more than 5 seconds to send response? (I am assuming it is not considering ramp up period which I gave 3 seconds, Am I right?)
Thanks in advance.
Currently your scenario looks as:
Start with 1 user (which will immediately start test actions)
Each second add 13 users (each of them will start test actions as soon as the thread is started)
When thread finished it's job it exits.
So there is no way to say exactly what will be the load at the "peak" where all threads which are active at the moment are hitting the application under test.
Not too sure regarding loadosophia, but if you expect 40 concurrent virtual users the only way to reach this load is providing enough loops and adding a Constant Throughput Timer to orchestrate the load and limit (or ramp-up) it to the value, you set in timer.
Another option is using a Synchronizing Timer which allows to "release" threads when specified amount is reached.