I did the same jmeter test in GUI and non-GUI mode on the same windows VM. All test configurations were the same, but the results were completely different. Connection time with non-GUI has become x10 higher. In addition,
javax.net.ssl.SSLException, javax.net.ssl.SSLHandshakeException, java.netSocketExceptions
errors appeared in a non-GUI mode.
You failed to submit all requests due to sockets limit in your machine,
This was caused probably due to ignoring the first best practice regarding resource requirement
suggestions on reducing resource usage.
Use CLI mode
CLI (non GUI) mode always will consume less resources
High resources GUI reduce resources available to produce significant load on SUT
Related
I'm monitoring the connect time and latency to connect from jmeter machine while running in GUI mode and that is in within acceptable limit.
Should we strictly follow non GUI mode even though I can able to perform load test with GUI mode?
I'm targeting 250 TPS and able to achieve that ..I have increased my memory and monitoring CPU and memory of load generator is below 60%.
Should I go for non GUI mode ?
The main limitation is that each event in the queue is being handled by a single event dispatch thread which will act as the bottleneck on your JMeter side.
My expectation is that your "250 TPS" look like:
while it should look like:
So check how does your load pattern look like using i.e. Transactions per Second listener (installable via JMeter Plugins Manager)
Also check how does your JVM look like especially when it comes to garbage collection, it can be done via i.e. JVisualVM, most probably you will see the same "chainsaw" pattern
You don't need to follow JMeter best practices, but
you may encounter issues to achieve specifc goals (as TPS)
your machine can't execute GUI or have low resources
you execute JMeter using a script or build tool as Jenkins
Also it's better to be familiar with JMeter CLI (non GUI) and its report capabilities
JMeter supports dashboard report generation to get graphs and statistics from a test plan.
Also it will be needed for using distributed testing
consider running multiple CLI JMeter instances on multiple machines using distributed mode (or not)
CLI also useful for Parameterising tests
The "loops" property can then be defined on the JMeter command-line:
jmeter … -Jloops=12
I'm doing a jmeter test the test is on localhost . My testplan is to hit login and then another http request . The test is running and getting response . But when I run the same test again I'm getting different results. I'm doing it in GUI mode . Please help me I'm new to jmeter.
You're violating at least 2 main best practices:
JMeter's GUI mode is for tests development and debugging only, when it comes to test execution you must run JMeter in command-line non-GUI mode
Having load generator (JMeter) and the system under test on the same machine is not the best idea, both tools may be very resource intensive when it comes to high loads and at some point they will start fighting for the OS resources like CPU, RAM, Network and Disk IO, etc. so when it comes to the bottleneck identification you will not be able to tell for sure why response times are high or throughput is low as the reasons could be in:
JMeter doesn't have enough headroom to operate and cannot send requests fast enough
Application cannot respond fast enough due to lack of RAM or CPU overload
CPU is constantly switching the context from JMeter to the application under test and vice versa
I want to perform load testing using JMeter to check where we are getting an application crash.
When I apply 1500 users and run the test using non-GUI mode I got heap space error but cannot crash my application.
As of current JMeter version (JMeter 5.0) default maximum heap space allocated to JVM is 1 gigabyte which is fine for tests development and/or debugging but might be not enough for stress test. Haven't you seen this message in the terminal window when you launch JMeter:
================================================================================
Don't use GUI mode for load testing !, only for Test creation and Test debugging.
For load testing, use NON GUI Mode:
jmeter -n -t [jmx file] -l [results file] -e -o [Path to web report folder]
& increase Java Heap to meet your test requirements:
Modify current env variable HEAP="-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m" in the jmeter batch file
Check : https://jmeter.apache.org/usermanual/best-practices.html
================================================================================
you can increase heap space allocated to JMeter to the required value. See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article for more recommendations on how to perform JMeter fine tuning for high loads
Ensure that JMeter has enough headroom to operate in terms of CPU, RAM, Network, Disk, etc. as if JMeter will lack resources - it will not be able to execute requests fast enough resulting in lesser load than you expect. If you don't have a monitoring system in place you can consider using JMeter PerfMon Plugin
It might be the case one machine will not be enough to put your application on knees, if you reach hardware limit on JMeter side but your application is still not loaded/overloaded - you will have to go for distributed testing
And last (however it should be first) recommendation: your test plan should represent real user activity as hammering login page doesn't make sense. Your test must represent real-life application usage with 100% accuracy, otherwise you will not be able to state that your application can support 1500 users.
Application is deployed in 16GBVM. While running JMeter from dedicated system throwing Network socket exception for 300 users
When i changed application to 8GB dedicated system its is working for 500 users trowing some other java exception.
Will JMeter support VM? Will get similar results(response time, throughput) like dedicated system.
Thanks in Advance.
JMeter doesn't know anything about underlying software or hardware, it is pure Java application so its run inside Java Virtual Machine which is an abstraction layer on top of OS which in its turn the abstraction layer on top of hardware.
So given VM and real device have similar hardware specifications you should get the similar results. Just make sure you following JMeter Best Practices, to wit:
Run your test in non-GUI mode
Disable all listeners during test run
Use only those pre/post processors and assertions which are absolutely required
Tune Java Heap and Garbage Collection settings according to total amount of available physical RAM and your load pattern.
See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article for comprehensive information and some JMeter tuning tips.
I am running a test for testing if my application is able to handle 250 concurrent users or not.first time when I ran the test,results were fine and number of samples generated in aggregate report is also fine but when I am running the same test again,i am getting drastic changes in aggregate report.This time number of samples got reduced and also the response time got higher.Whereas cpu usage and memory usage is fine and database server performance is also good.For this I am using stepping thread group.
please help me to get rid out of it.
What about CPU and RAM usage on the host, you're running JMeter on? Make sure that:
You running JMeter in non-GUI mode
You have all the listeners disabled
You have only absolutely minimum of pre/post processors and assertions added/enabled
JMeter has enough JVM heap space (70-80% of your total physical RAM)
See JMeter Performance and Tuning Tips for detailed explanations and more JMeter configuration tricks
Depending on the logic your application has you might not be able to handle 250 threads on single machine (not enough computing resources RAM, NIC bandwidth etc) You haven't provided details about your machine utilization during the run test and Jmeter logs for any warnings or errors. Check that.
We had the same kind of issues when we were testing heavy application (with sessions and long user flows). Master-slave config can fully resolve the issue.