I want to perform load testing using JMeter to check where we are getting an application crash.
When I apply 1500 users and run the test using non-GUI mode I got heap space error but cannot crash my application.
As of current JMeter version (JMeter 5.0) default maximum heap space allocated to JVM is 1 gigabyte which is fine for tests development and/or debugging but might be not enough for stress test. Haven't you seen this message in the terminal window when you launch JMeter:
================================================================================
Don't use GUI mode for load testing !, only for Test creation and Test debugging.
For load testing, use NON GUI Mode:
jmeter -n -t [jmx file] -l [results file] -e -o [Path to web report folder]
& increase Java Heap to meet your test requirements:
Modify current env variable HEAP="-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m" in the jmeter batch file
Check : https://jmeter.apache.org/usermanual/best-practices.html
================================================================================
you can increase heap space allocated to JMeter to the required value. See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article for more recommendations on how to perform JMeter fine tuning for high loads
Ensure that JMeter has enough headroom to operate in terms of CPU, RAM, Network, Disk, etc. as if JMeter will lack resources - it will not be able to execute requests fast enough resulting in lesser load than you expect. If you don't have a monitoring system in place you can consider using JMeter PerfMon Plugin
It might be the case one machine will not be enough to put your application on knees, if you reach hardware limit on JMeter side but your application is still not loaded/overloaded - you will have to go for distributed testing
And last (however it should be first) recommendation: your test plan should represent real user activity as hammering login page doesn't make sense. Your test must represent real-life application usage with 100% accuracy, otherwise you will not be able to state that your application can support 1500 users.
Related
I'm monitoring the connect time and latency to connect from jmeter machine while running in GUI mode and that is in within acceptable limit.
Should we strictly follow non GUI mode even though I can able to perform load test with GUI mode?
I'm targeting 250 TPS and able to achieve that ..I have increased my memory and monitoring CPU and memory of load generator is below 60%.
Should I go for non GUI mode ?
The main limitation is that each event in the queue is being handled by a single event dispatch thread which will act as the bottleneck on your JMeter side.
My expectation is that your "250 TPS" look like:
while it should look like:
So check how does your load pattern look like using i.e. Transactions per Second listener (installable via JMeter Plugins Manager)
Also check how does your JVM look like especially when it comes to garbage collection, it can be done via i.e. JVisualVM, most probably you will see the same "chainsaw" pattern
You don't need to follow JMeter best practices, but
you may encounter issues to achieve specifc goals (as TPS)
your machine can't execute GUI or have low resources
you execute JMeter using a script or build tool as Jenkins
Also it's better to be familiar with JMeter CLI (non GUI) and its report capabilities
JMeter supports dashboard report generation to get graphs and statistics from a test plan.
Also it will be needed for using distributed testing
consider running multiple CLI JMeter instances on multiple machines using distributed mode (or not)
CLI also useful for Parameterising tests
The "loops" property can then be defined on the JMeter command-line:
jmeter … -Jloops=12
I'm doing a jmeter test the test is on localhost . My testplan is to hit login and then another http request . The test is running and getting response . But when I run the same test again I'm getting different results. I'm doing it in GUI mode . Please help me I'm new to jmeter.
You're violating at least 2 main best practices:
JMeter's GUI mode is for tests development and debugging only, when it comes to test execution you must run JMeter in command-line non-GUI mode
Having load generator (JMeter) and the system under test on the same machine is not the best idea, both tools may be very resource intensive when it comes to high loads and at some point they will start fighting for the OS resources like CPU, RAM, Network and Disk IO, etc. so when it comes to the bottleneck identification you will not be able to tell for sure why response times are high or throughput is low as the reasons could be in:
JMeter doesn't have enough headroom to operate and cannot send requests fast enough
Application cannot respond fast enough due to lack of RAM or CPU overload
CPU is constantly switching the context from JMeter to the application under test and vice versa
Could someone help me to understand the Difference of running the load testing using local System and setting up master-slave system ? How it differs ? what is the best practice to do load testing on server.
If we are setting up master-slave do both should be in same sub-network ?
And we can generate the HTML report in Master system after running the script?
The number of virtual users you can simulate on one machine varies from several hundreds to several thousands (check out What’s the Max Number of Users You Can Test on JMeter? article for more details) but in any case it is limited.
Each thread (virtual user) has some "footprint" in terms of CPU, RAM, Network and Disk usage. So you need to ensure that the machine you're running JMeter on has enough capacity and it's not overloaded because if JMeter is not capable of sending requests fast enough - you will get false negative results because throughput will be low not due to the application under test problem, but due to the problem with JMeter.
So make sure to monitor essential OS health metrics like CPU, RAM, Network, Disk, Swap usage when you're running a load test. You can do it using i.e. JMeter PerfMon Plugin
If you're able to conduct the required load using only single JMeter machine - running the test in distributed mode doesn't make a lot of sense as you will not get any new results.
However if one machine cannot produce the required load - you will have to conduct distributed testing. The main idea is having multiple JMeter instances executing the same Test Plan.
For example if you identified that you can run only 1000 virtual users on one machine and you need to simulate 3000 ones you will need 4 machines for this.
Master machine to orchestrate the slaves and collect the results
3 slave machines each running 1000 virtual users.
Once you start JMeter Serve ron each slave machine you will be able to run your test in command-line non-GUI mode as follows:
jmeter -n -t your_test_plan.jmx -R IP.of.1st.slave, IP.of.2nd.slave, IP.of.3rd.slave -l result.jtl
if you want to generate HTML Reporting Dashboard after the test run you can do it as follows:
jmeter -n -t your_test_plan.jmx -R IP.of.1st.slave, IP.of.2nd.slave, IP.of.3rd.slave -l result.jtl -e -o /path/to/output/folder
I'm getting the following error:
out of memory error : java heap space error while running jmeter script for long run say 20 hrs.
I have set 10 GB heap size to run 10 threads only.
Could it be my application's bug? How can I be sure that this is application's bug not a jmeter issue?
JMeter is a Java application with a GUI. It also supports a non-GUI mode which is less resource intensive(CPU/RAM). If we run JMeter in non-GUI mode, it will consume less resource and we can run more threads.
Disable ALL Listeners during the Test Run. They are only for debugging and use them to design the desired script.
Listeners should be disabled during load tests. Enabling them causes additional overheads, which consume valuable resources that are needed by more important elements of your test.
Use Up-to-Date Software:
Java and JMeter should be kept updated.
Decide Which Metrics You Need to Store:
When it comes to storing requests and response headers, assertion results and response data can consume a lot of memory! So it is wise try not to store these values on JMeter unless it’s absolutely necessary.
JMeter SaveService badly configured: Check if you are saving requests and responses in your JTL.
For better and more elaborated understanding, this blog about 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure is helpful.
I have created a test plan for creating userprofile.
I want to run my test plan for 100 users but when i run it for 10 users then it is running successfully with rump up time of 2 sec; but when i try it for 100 users & more than that it is getting failed I am giving rump uptime of 40 sec for 100 users.
I am not able to understand what may be the problem with it.
In my test plan the thread user are differentiated with id
Thanks in Advance.
It's a wide question, this behavior can be caused by
Your application under test can't handle load of 100 threads. Check logs for errors and make sure that application/web server and/or database configuration allow 100+ concurrent connections. Also you can check "Latency" metric to see if there is a problem with infrastructure or application itself.
Your load generator machine can't create 100 concurrent threads. If so - you'll need to consider JMeter Distributed Testing
Your script isn't optimized. I.e. using memory-consuming listeners like "View Results Tree", any graph listeners, regular expression extractors. Try following JMeter Performance and Tuning Tips guide and see whether it resolves your issue.
Agree with Dmitri, reason could be one of the above three.
One more thing you can try.
You can run your jmeter in ui mode for validation of your script and after validation you can run it in non-ui mode which will save lot of memory and cpu processing (basically UI is heaviest part in jmeter).
you can run your jmeter script in non-ui mode like this,
Jmeter -n -t -H proxy -P port
generally on a single dual core machine with 2 GB ram (Load Generator in your case) 100 user test can be carried out successfully.
some more things you can look at to find out the actual bottleneck
1.check application server logs (server on which your application is hosted)
if there are any failures in that then see performance counters on server (CPU, Memory, network etc) to see anything is overloaded.
(if server is windows then check using perfmon if linux then try sar)
if something is overloaded then reason is your app server cant take load of 100 users
probably try tuning it more.
2.check load generator system performance counters (JVM heap usage,CPU,Memory etc)
if JVM heap size is small enough try increasing it but if other counters are overloaded then try distributed load testing.
3.remove unwanted/heavy listeners, assertion from script.
maybe this will help :)