I have a performance test in JMeter and would like to test maximum system performance/throughput with it. So number of active threads should be increased for example while Error rate is under 2 %. I found Constant Throughput Timer, put it into Thread Group but it only pause or slow down threads. I tried define it as follows, with a property: ${__P(throughput,)}, but not sure what should be correct value for this property. I can't see how JMeter could measure system maximal performance.
There is no Out of the box solution as of JMeter 3.3, see this enhancement request:
https://bz.apache.org/bugzilla/show_bug.cgi?id=57640
Still it is possible to dynamically add threads since JMeter 3.2 (see https://bz.apache.org/bugzilla/show_bug.cgi?id=60530) in a JSR223 Test Element using JMeterContext:
ctx.getThreadGroup().addNewThread(delay, ctx.getEngine());
So based on this, you could in a JSR223 Test Element (Pre/Post Processor or Sampler) check the presence of a file in a folder of your choice named :
NUMBER_OF_THREAD_TO_ADD.txt
If present, use its name to compute number of threads and call this method.
There is no such functionality in the "vanilla" JMeter howere it is possible with plugins, check out:
Concurrency Thread Group
Throughput Shaping Timer
They can be connected together via feedback loop so Concurrency Thread Group will add more threads if needed to reach the desired number of requests per second.
You can install both the plugins (and keep them up-to-date) using JMeter Plugins Manager
Related
I've got a distributed JMeter setup and I want to create a Java service wrapper around it.
The test plan (JMX file) is provided by the user. Hence, I don't know the properties and config used in there. The user test plan can have multiple thread groups and potentially custom thread group plugins like Ultimate Thread Group, Arrival Thread Group, etc.
I want to get the overall maximum #threads or maximum concurrency (active threads) that can be achieved by the user test plan. Is it possible to get this value before executing the test plan? This will help me to provision servers for this test run accordingly.
Is it possible to cap the overall max concurrency achieved by the test plan? (similar to what we can do with RPS using timers) (Also, please mention options for custom thread groups like Ultimate thread group)
This is not something you can efficiently control because:
It is possible to configure JMeter to kick off extra threads if current amount is not sufficient to reach/maintain the target throughput by using i.e. Throughput Shaping Timer and Concurrency Thread Group with feedback function
Users can use properties or functions/variables to parameterize/calculate the concurrency
Users can always use JSR223 Test Elements and do something like:
ctx.getThreadGroup().addNewThread(0, ctx.getEngine())
and this is not something you can detect because there are numerous ways of invoking this function including reflection
So the maximum you could do is to rely on JVM limits, by default you have 1 megabyte of stack size per thread which gives your 1024 threads for each gigabyte of memory allocated to Java so you can either increase stack size or decrease the memory allocated to the JVM according to your "caps"
I have a requirement to create simple Burst test in Jmeter:
10 requests-> 20 requests -> 30 requests
On single worker mode.
Throughput 20ms
Anticipated response time : <=200ms
How can we achieve this with Jmeter without using parallel threads?
I am looking for a simple solution.
Thanks a lot
Add plugin
Ultimate Thread Group to JMeter
Add Ultimate Thread Group component with settings of 3 rows
Set Start Threads Count 10,20 and 30
Use different start times so requests batch will be executed in different times
"Ultimate" means there will be no need in further Thread Group plugins. The features that everyone needed in JMeter and they finally available:
infinite number of schedule record
separate ramp-up time, shutdown time, flight time for each schedule record
and, of course, trustworthy load preview graph
Consider using Throughput Shaping Timer and Concurrency Thread Group combination which provides flexible way of defining load patterns:
They can be connected together using Feedback Function so JMeter will be able to kick off more threads in order to reach the desired throughput (number of requests per second) if current amount is not enough.
Both can be installed using JMeter Plugins Manager:
I am trying to configure a test for signup-login with invalid credentials imitating bruttforce attack. However, I want to make requests as a single user.
So the scenario is the next:
Request to sign up with valid params;
Attempt to log in with invalid params in a loop.
I am adding throughput timer but then I need to set up a high number of threads to generate for example 1 request per second.
How can I set exactly one user and run only one request per second in a loop?
UPDATE - SOLUTION
Everything was rather simple but still strange for me.
Maybe someone will face the same problem.
To get a number of requests per one user throughput timer should be placed not inside the controller but inside the thread itself.
To achieve Target Request Per Second use Throughout Shaping Timer : How to use Throughput Shaping Timer
JMeter Plugin link : Download from here
Few Important Notes:
JMeter threads of Thread Groups in scope of the Element will be
stopped when RPS schedule finishes.
Provide enough working threads for your RPS, JMeter timers can only delay threads (and limit RPS). You may pair this plugin with Concurrency Thread Group using Schedule Feedback Function to dynamically maintain thread count required to achieve target RPS.
If you're using versions of JMeter lower than 3.3 and if you have RPS that lower at the end of test, make threads to lower also. Оtherwise you'll have a spike in last second.
Avoid using zero RPS value as start of test, this produce spike also
Avoid zero RPS during the test, this may lead to nasty effects
Now, practical example showing 1 RPS for 1 User:
You might want to reconsider the location of the timer, it obeys JMeter Scoping Rules so it gets applied to all Samplers which are in your Thread Group.
If you want to apply it to only one - make it a child of the relevant sampler.
Also be aware that Constant Throughput Timer is precise enough on minute level so you might want to switch to the Precise Throughput Timer
My Test Plan have 3 Scenarios,
Submit Quote
Check Status
If Status Complete get Quote Details.
If I keep all these 3 scenarios in one thread then CTT is not putting constant throughput, when I remove create 2 thread 1 for Submit and other for Status and Get Details I am able to maintain CTT, but my 2nd and 3rd Scenario volume drops almost 50%. As I have dependency on 1st step I am using Thread communication and not sure how I can either get CTT in one thread or get same volume from 2 different threads.
Constant Throughput Timer is useful to shape the load where there is > 1 test iteration. If your thread group has only one iteration consider using Throughput Shaping Timer instead or play with ramp-up and thread number values. It is recommended to have only one instance of the CTT in your Test Plan as it's rather resource consuming by nature.
If your scenario assumes different throughput for different thread groups you can add a CTT per Thread Group (make sure that you select All active threads in current thread group (shared) in "Calculate Throughput based on" dropdown
If ITC test elements are throttling your test below desired throughput value consider using
__setProperty()
__property()
__threadNum()
functions to implement inter-thread communication instead. JMeter Properties are global, can be accessed from different thread groups and persist all the time while JMeter is running.
See How to use JMeter's Throughput Constant Timer for details.
If above information is not enough try to elaborate your query and include i.e. thread numbers, target throughput and what kind of information needs to be shared across thread groups.
Is it possible to automate the load tests in Jmeter and increase the number of threads until the first error is observed?
For example I start with testing 16 threads for every seconds and increase the number until i receive an error. But instead of doing this manually can I let this run automatically?
Looking into Pre-defined Properties section of JMeter's User Manual on Functions there is a JMeterThread.last_sample_ok variable holding result of the last sampler execution.
So if you build your test plan as follows:
Sampler which does test action
If Controller checking whether previous sampler was successful
If not - relevant actions (stop test, send email, stop ramping up virtual users, etc.)
The value you need to put in "Condition" input of If Controller should look like
"${JMeterThread.last_sample_ok}"=="false"
See How to use JMeter's 'IF' Controller and get Pie for more information on JMeter's If Controller.
Regarding threads in jmeter You may find those 2 links interesting:
What is the highest number of threads that is reasonable to simultaneously run in Jmeter?
JMeter max. thread limit
Regarding your methodology, why not use slow rampup and see the limit using what Dmitri T has provided ?