We've a distributed JMeter setup as described here - How to Change JMeter´s Load During Runtime
The test plan (JMX file) is provided by the user. Hence, we don't know the property names used in there. During runtime, the user can provide property names and values that we will directly pass to JMeter setup through beanshell script.
In this setup, can we put a limit/cap on values of certain JMeter properties (which can potentially affect our provided resources) that can be changed by the user at runtime?
For eg. we don't want the total RPS of the system to cross say, 300 RPS at anytime. Or if the user has provided runtime change capability of #threads, we don't want them to exceed say, 100 on any machine at anytime.
We want to refrain from storing any user-defined property names in our system to provide such validation.
You can inject a Constant Throughput Timer or a Precise Throughput Timer or a Throughput Shaping Timer into the user-provided .jmx script and put your maximum allowed concurrency there.
Even if there will be multiple timers in the test plan JMeter will apply the throughput of the slowest one so you won't make the test too fast in case if original RPS is lower than your maximum and vice versa, no matter what RPS user will want it will never get higher than 300 RPS which you will define.
The same approach applies to the number of threads in thread group.
Related
I have a few question to clarify on my understanding of how JMeter works.
a. Thread Group determine the number of users but it does not determine how many HTML requests are generated per sec ? By default, I notice that every user will send a HTML request at a rate of 2 RPS.
b. If I want to change the RPS per user, then I need to use the Through Put Timer. But the Timer can only lower the request rate from 2 RPS to a lower number. It does not increase the RPS.
c. In order to increase the RPS, I need to add more Threads.
d. Does this mean we are limited to 2 RPS per user ? I see some website have links to many other websites so a webpage refresh would make many requests.
Is this the way JMeter works ?
I have a load test which has 8 transaction (eg CRUD,...). I intend to create a overall Test Plan and I want to use INCLUDE to add all the 8 txn. Do I just record the website and INCLUDE ? What should I include, only the HTML requests ?
I'm also thinking of adding Think Time and Add Variables in the 8 scripts before I INCLUDE.
Do I add the Config Element (eg CSV Dataset Config) in the 8 scripts or the overall Test Plan ?
Thanks.
By default each JMeter thread (virtual user) executes requests as fast as it can. If you want to slow JMeter down to mimic a real user which doesn't hammer the server non-stop and needs some time to "think" between operations - use Timers. More information: How do I Correlate the Number of (Concurrent) Users with Hits Per Second
If you want more RPS - add more threads (assuming that the system under test can give you more RPS)
You should INCLUDE everything which is related to your website (images, scripts, styles, fonts, sounds, etc.) but in the same manner as your browser does, i.e. don't record these requests and instead configure JMeter to download embedded resources and use HTTP Cache Manager so JMeter would request these resources just like browser does. Any requests to "external" websites should be excluded (unless they're also developed and supported and in scope for testing)
That's a good approach, if you use a value more than once it makes sense to declare it via User Defined Variables so you would be able to amend the value only in one place
You add it according to your scenarios, be informed about JMeter Scoping Rules
In my application I want hit 20000 request in 10 hrs but I want distribute load in different time with different number of request means ex in 1 hrs 2000 request second hrs 3000 request third 1000 request like that how achieve this means how to separate load in diff time with diff no. of request
The easiest option is going for Throughput Shaping Timer, configuration implementing your described setup:
It's a good idea to use Concurrency Thread Group in combination with the Throughput Shaping Timer, they can be connected via Feedback Function so JMeter would be able to start extra threads if the current amount is not sufficient in order to reach/maintain the desired number of requests per second.
Both are JMeter Plugins and can be installed using JMeter Plugins Manager
Another solution could be using Constant Throughput Timer.
N.B. although the Timer is called the Constant Throughput timer, the throughput value does not need to be constant. It can be defined in terms of a variable or function call, and the value can be changed during a test. The value can be changed in various ways:
You could set the throughput using a property or variable.
Calculate the throughput values you need at different time intervals and set the property when the time is reached.
props.put("currentTPM", 120)
You will have some work in checking the duration since the test is started.
You may create a separate thread group to control the throughput. Rename the thread group name to TG-TM. Set the number of threads to 1 and loop count to infinite. Set the duration of the thread group.
def lstThrouputInOneHour= [2000,3000,1000,5000,4000,5000]
def currentIndex=vars.get("__jm__TG-TM__idx").toInteger()
if (currentIndex <lstThrouputInOneHour.size() ) {
def currentTPH=lstThrouputInOneHour[currentIndex]
def currentTPM=currentTPH.intdiv(60)
props.put("currentTPM",currentTPM.toString())
Thread.sleep(60*60*1000)
}
Note: Please introduce a startup delay to other thread groups to ensure they have access to the throughput value when they start.
This solution can be extended to work with Bean Shell Server where you could change the throughput values (JMeter properties) remotely
I've got a distributed JMeter setup and I want to create a Java service wrapper around it.
The test plan (JMX file) is provided by the user. Hence, I don't know the properties and config used in there. The user test plan can have multiple thread groups and potentially custom thread group plugins like Ultimate Thread Group, Arrival Thread Group, etc.
I want to get the overall maximum #threads or maximum concurrency (active threads) that can be achieved by the user test plan. Is it possible to get this value before executing the test plan? This will help me to provision servers for this test run accordingly.
Is it possible to cap the overall max concurrency achieved by the test plan? (similar to what we can do with RPS using timers) (Also, please mention options for custom thread groups like Ultimate thread group)
This is not something you can efficiently control because:
It is possible to configure JMeter to kick off extra threads if current amount is not sufficient to reach/maintain the target throughput by using i.e. Throughput Shaping Timer and Concurrency Thread Group with feedback function
Users can use properties or functions/variables to parameterize/calculate the concurrency
Users can always use JSR223 Test Elements and do something like:
ctx.getThreadGroup().addNewThread(0, ctx.getEngine())
and this is not something you can detect because there are numerous ways of invoking this function including reflection
So the maximum you could do is to rely on JVM limits, by default you have 1 megabyte of stack size per thread which gives your 1024 threads for each gigabyte of memory allocated to Java so you can either increase stack size or decrease the memory allocated to the JVM according to your "caps"
I have a performance test in JMeter and would like to test maximum system performance/throughput with it. So number of active threads should be increased for example while Error rate is under 2 %. I found Constant Throughput Timer, put it into Thread Group but it only pause or slow down threads. I tried define it as follows, with a property: ${__P(throughput,)}, but not sure what should be correct value for this property. I can't see how JMeter could measure system maximal performance.
There is no Out of the box solution as of JMeter 3.3, see this enhancement request:
https://bz.apache.org/bugzilla/show_bug.cgi?id=57640
Still it is possible to dynamically add threads since JMeter 3.2 (see https://bz.apache.org/bugzilla/show_bug.cgi?id=60530) in a JSR223 Test Element using JMeterContext:
ctx.getThreadGroup().addNewThread(delay, ctx.getEngine());
So based on this, you could in a JSR223 Test Element (Pre/Post Processor or Sampler) check the presence of a file in a folder of your choice named :
NUMBER_OF_THREAD_TO_ADD.txt
If present, use its name to compute number of threads and call this method.
There is no such functionality in the "vanilla" JMeter howere it is possible with plugins, check out:
Concurrency Thread Group
Throughput Shaping Timer
They can be connected together via feedback loop so Concurrency Thread Group will add more threads if needed to reach the desired number of requests per second.
You can install both the plugins (and keep them up-to-date) using JMeter Plugins Manager
I have 500 users in my csv file. I am doing load testing using jmeter. I want to run the script for first 100 hundred users. Once the execution for 100 concurrent user/Threads is done then I want to automatically increase the size of concurrent users to 200 and so on.
How I can achieve this ??
You can use Constant Throughput Timer to set throughput according to your test scenario. Despite its name it doesn't have to be "constant", you can put a variable into "Target Throughput" input so you'll be able to modify concurrency on the fly.
Another option which could be easier / more flexible is Throughput Shaping Timer available via JMeter Plugins