I'm have a simple JMeter script with 300 threads that checks a list of 30 static URLs with a 2 second delay between requests.
I'm hitting CPU usage limit on my machine (EC2 Large Server).
Any idea on which switch and levers have the greatest impact on server resources?
For example:
Number of threads
Delay between requests
Number of steps (URLs) in script
Versions of JMeter (2.9 vs 2.10)
Thanks
Ophir
Just make sure that you've
Disabled all listeners
Running JMeter in non-GUI mode, i.e. via command-line, Ant, Maven or whatever
Your Post Processors (if any) aren't resource-consuming, i.e. you're trying to extract something with Regular Expression Extractor from immense output.
You're using the latest version of JMeter and Java (JMeter 2.10 is better than 2.9)
You're using 64-bit Java with enough heap space provided
You're following JMeter Best Practices
If you problem still persists, you can consider looking into following options:
Profile Java process running JMeter to detect what classes are most CPU-consuming
Use several "lesser" EC2 instances with 1 JMeter "master" and 2-3 slaves
If you have static URLs, just:
Use a CSV DataSet which will contain these URLs
One HTTP Request Sampler that will use it
With a large Server, 300 threads is really nothing.
You may read :
http://jmeter.apache.org/usermanual/best-practices.html
http://www.dzone.com/links/see_how_to_make_jmeter_run_thousands_of_threads_w.html
Related
I'd few ques on technical details of JMeter mostly pertaining to distributed setup vs independent JMeter engines (since JMeter controller can become a bottleneck in case of several JMeter load generators). Would be great if anybody can help with the understanding here -
How is JMeter distributed setup orchestrated by JMeter controller (i.e. called master or client)? Can we use the same logic to synchronize test among independent JMeter engines (independent mode)?
Is there a way to pool connections across vUsers?
Function of ASYNC_QUEUE in backend listener and it's expected side-effects in independent mode (mentioned above), what happens when queue is full?
Does/Is there a way for JMeter to execute javascript/act as headless browser?
How does DNS resolution happen for JMeter? Does it resolve for each vuser?
Your "question" looks like a compilation of interview questions rather than something connected with your single current concern and I don't think it's a proper place/way to ask it, I believe it should be: one post - one question.
Whatever
How is JMeter distributed setup orchestrated by JMeter controller - JMeter master sends .jmx script to slaves and collects results from them. Theoretically you can implement your own mechanism for delivering the test plan and eventual dependencies to the individual JMeter engines and running the test at the same time. Then you will need to collect the .jtl results files from the engines and combine it into a single one.
Is there a way to pool connections across vUsers? - JMeter does it internally
When the queue is full no more new sample results will be taken for processing by the backend listener so the results won't be "realtime" anymore, you will see the new results as free slots will be appearing in the queue
For JMeter per-se - no, AJAX calls can be simulated using Parallel Controller, for client-side performance testing, JavaScript execution profiling and rendering speed measurement you will need to use a read browser, no matter normal or headless, there is WebDriver Sampler plugin providing JMeter integration with Selenium
DNS resolution is dependent on underlying OS and/or JVM DNS resolution implementation, there is DNS Cache Manager which enables overriding hosts entries and using custom DNS resolver so each thread looks up the IP address on its own
I'm monitoring the connect time and latency to connect from jmeter machine while running in GUI mode and that is in within acceptable limit.
Should we strictly follow non GUI mode even though I can able to perform load test with GUI mode?
I'm targeting 250 TPS and able to achieve that ..I have increased my memory and monitoring CPU and memory of load generator is below 60%.
Should I go for non GUI mode ?
The main limitation is that each event in the queue is being handled by a single event dispatch thread which will act as the bottleneck on your JMeter side.
My expectation is that your "250 TPS" look like:
while it should look like:
So check how does your load pattern look like using i.e. Transactions per Second listener (installable via JMeter Plugins Manager)
Also check how does your JVM look like especially when it comes to garbage collection, it can be done via i.e. JVisualVM, most probably you will see the same "chainsaw" pattern
You don't need to follow JMeter best practices, but
you may encounter issues to achieve specifc goals (as TPS)
your machine can't execute GUI or have low resources
you execute JMeter using a script or build tool as Jenkins
Also it's better to be familiar with JMeter CLI (non GUI) and its report capabilities
JMeter supports dashboard report generation to get graphs and statistics from a test plan.
Also it will be needed for using distributed testing
consider running multiple CLI JMeter instances on multiple machines using distributed mode (or not)
CLI also useful for Parameterising tests
The "loops" property can then be defined on the JMeter command-line:
jmeter … -Jloops=12
I recorded a jmeter script, it has all the resources urls in the script. so if I put Parallel Controller with 6 connection and run the script, the script reports higher response time THAN if I put only main html URL and check download embedded resouces with 6 parallel connection. ..so why it reports higher response time with Parallel Controller?
Given both approaches execute the same requests you should be getting the same or at least similar results. If you see differences - capture the traffic from both runs using a 3rd-party tool like Fiddler or Wireshark and compare the network footprint for both cases.
I can only think of one possible reason for difference: cache. If you have HTTP Cache Manager in your test plan then JMeter respects Cache-Control headers and doesn't re-request the content on subsequent calls/iterations.
What's the differences between JMeter and Tsung? I read that Tsung may generate more load than JMeter if test with same hardware, but how close it to reality?
Tsung is written in Erlang and said to be capable of running an extreme number of simultaneous users (10000+)
Jmeter is written in Java and quite capable of generating large amounts of load, assuming your test plan is good.
Here are some limitations in regards to performance in JMeter
Every user in Jmeter is an OS thread. This increases overhead when using a lot of concurrent users (jmeter best practices recommends using a low number of threads, http://jmeter.apache.org/usermanual/best-practices.html - in my experience you may run into problems when using more than 1000 threads per jmeter instance, but that may vary a lot depending on your test plan). You will also need to tweak JVM settings when running larger tests.
You can easily destroy performance & run out of PermGen memory if you have dynamic scripts (scripts that have to be recompiled every time because of jmeter variable expansion). Put your scripts in separate files or use a compilation cache key to avoid recompilation.
Using some test components (such as the Tree View, which keeps every request & response in memory) can wreak havoc on your load generator
I've tested some VERY large sites with JMeter, and as long as you are able to reduce the number of threads (reducing user-waits to keep throughput at your desired level) Jmeter is fine. There is quite a large community around JMeter, plugins for load testing using a lot of protocols & monitoring various systems. JMeter has also good scripting support - java, javascript, whatever that can be loaded in to a jvm basically (including for example groovy), so it is very extensible.
At one time (using jmeter 2.6 I think) I ran some 30,000 database requests per second (Oracle JDBC) from a single load generator, and there have been some optimizations since, so as long as you dont have extreme requirements, Jmeter is fine. Pick the one that suits your needs and your experience.
Note: I have very little experience using Tsung.
Edit: Nowadays I use Locust (https://github.com/locustio/locust/). Tsung has not been updated since 2017, and Locust's user/threading model (greenlets) supports more concurrent users than Jmeter. But most importantly it has a more flexible workflow (actual Python code instead of configuration/clicking in a GUI)
It's always depends on your scenario and amount off datas used for variablility, the ratio is closed to ten, when you'll be able to run 100 user/second with JMeter, Tsung will be easy with 1000 user/sec.
Our client asked us to run a stress test on their Web application simulating 62,000 users (threads), the test consists of 13-15 HTTP requests, with 1 second delay between every HTTP request, the test should run for 10.5 hours continuously.
I had previous experience with JMeter running up to 10,000 users, but have not tried for larger number.
Is there a limit for the number of threads that JMeter can handle, or is this limited by the hardware of test server?
agreed with Ray when you run jmeter for this much amount of thread the distributed testing is the best option, and the hardware and network shall be capable to handle this kind of load, and in case you want to do that in short go for http://blazemeter.com, they are Scalable from 1,000 to 100,000 concurrent users.
In principle it is limited by several factors including the configuration (CPU/Mem) of your JMeter machine. That said, it is a VERY large number of threads for just one JMeter. To be honest: I wouldn't run even 10000 threads on one machine. You might want to look into using JMeter distributed, see the manual (http://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf).
Except running script remotely, you'll need to configure your script with minimal usage of CPU and memory.
Use JMeter best practices and JMeter tuning tips to reach it