I have a task to load test on application which needs to respond 2350 users per second. For that I have set us something like this in Jmeter:
I have added a Thread group. In that I have set:
Number of threads(users): 2350
Ramp-up period: 1 Second
Loop Count: 1
Will it solve my purpose to load test of application with 2350 users??
It will. But only if response time for each virtual user will be equal 1 second.
There are 2 common load patterns, for implementing both you will need Timers
Actually it might be the case you don't need as much as 2350 thread to simulate 2350 users as real life users don't hammer the server non-stop, they need some time to think between requests. Besides page loading time also needs to be considered.
Let's imagine you have 2350 users. Each user performs an action each 15 seconds. Page loading time is 5 seconds. So each user will be able to hit the server 3 times per minute. So 2350 users will produce 7050 requests in minute which stands for 117.5 requests per second only. If this is what you're looking for consider adding Constant Timer or Uniform Random Timer
If you need to simulate 2350 requests per second, not users, you need to handle it a little bit differently. There are 2 Timers which are designed to set exact "throughput" - a number of requests per time unit. They are:
Constant Throughput Timer
Throughput Shaping Timer - an advanced version of the Constant Throughput Timer, available via JMeter Plugins project.
Remember that above timers can only pause the threads, they won't kick off new virtual users if you don't provide enough on Thread Group level so make sure you have at least as much as you try to simulate, or it's better to have 2x more in your virtual pocket just in case. Also check out JMeter tuning tips from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure as JMeter default configuration isn't something you can use to create such a load.
The setup you described just creates 2350 parallel users (2350 separate connections), but doesn't guarantee that every thread will be completed exactly in 1 second. Ramp-up period - how fast all threads will start to send requests. So, in the case you described Jmeter will do the following:
Create 2350 separate threads.
The difference between 1st and last thread start will be 1 second (approximately).
Every thread will be implemented only once (loop count).
For real scenarios when you need specific throughput for continuous period of time it's better to use Constant Throughput Timer. It controls the amount of requests sent by thread(s) and changes delay between requests when it's necessary to meet the value you defined. So, your real throughput doesn't really depend on the total amount of threads (users). Sometimes less users can send requests faster (depends on your application).
To control your throughput while running, just add Summary Report to your test plan.
Moreover, for this specific scenario (2350 users) it can be potentially difficult to generate so many requests in 1 second. In this case, you need to use distributed load with some Jmeter slaves and 1 master.
Related
I have used the normal thread group of 40 threads,180000 of throughput, and expected TPS is 3000.., But , Resulted only 2750 tps., How can I improve the TPS?
I have tried the concurrency thread group along with the shaping timer combination..
Can u pls guide me..,Thanks in Advance.
TPS or the throughput is controlled by sending the required number of requests to the target server in a unit time (seconds, minutes, hours, etc)
It is not easy or good practice to control the throughput by changing the number of threads. When the server gets slow the throughput gets low and vice-versa.
You need to introduce delays when the server responds faster and introduce more threads when the server responds slower. It is not easy to achieve the desired throughput by setting the thread count.
Please note that
throughput will be lower if the server is not capable of handling it, or if other timers or time-consuming test elements prevent it.
JMeter has a number of ways to achieve the desired throughput by introducing delays to the requests.
Using Timers
Constant Throughput Timer
Precise Throughput Timer
Througput Shaping Timer
In the above cases, you need to ensure a sufficient number of threads are available to generate the required throughputs. This solution is not efficient if a large number of threads are created than the required number of threads.
The best approach would be the test should create additional threads on-demand and introduce delays between requests as required.
In JMeter, you can achieve this with Concurrency Thread Group along with the Throughput Shaping Timer.
This video demonstrate the use of the controllers.
First of all your application must be able to handle such a load. I don't know what your 180000 of throughput, stanza means,if you put it to the Constant Throughput Timer be aware that:
it is precise enough on "minute" level
it can only "pause" JMeter to "limit" its throughput to the given value
In general you can reach out 3000 TPS with 40 users only if 1 user will make 75 requests per second which means that your application response time should be below 75 milliseconds.
If response time will be higher - you won't be able to reach 3000 requests per second with 40 users.
Also JMeter should be able to send requests fast enough, so make sure to follow JMeter Best Practices and if this is not sufficient consider using Distributed Testing
More information: What is the Relationship Between Users and Hits Per Second?
I have created a performance test script as below. I am running 4 Thread Groups in parallel (Even though there are 14 Thread Groups, only 4 are enabled and I am running only those enabled 4 Thread Groups). I have used default Thread Groups.
I have used Flow Control Action to simulate the user think time and set it as 3 seconds.
My requirement is to achieve Throughput as 6.6/seconds. What is the best way to achieve it? Also, does user think time creates any impact on Throughput?
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So basically throughput is the number of requests which JMeter was able to within test duration time frame. If you introduce artificial delay of 3 seconds - it will make the overall throughput lower
Other (and the main factor) is your application response time because JMeter waits for previous request to finish before executing new one so there could be 2 options:
The amount of threads in your Thread Groups is not sufficient to create the desired throughput. If this is the case - just add more threads (virtual users) in Thread Group(s)
The amount of threads in your Thread Groups is too high and you're getting higher throughput that you expect. If this is the case you can pause JMeter even more by adding Constant Throughput Timer and specifying the desired number of the requests per minute you want to have. If you need 6.6 requests per second it will mean 396 requests per minute
Also remember that JMeter itself must be able to send requests fast enough so make sure to follow JMeter Best Practices
My suggestion is to consider using the Arrivals Thread Group. This TG will allow you to configure the desire average throughput (ATP); the TG will instantiate the required threads needed to achieve the ATP goal.
As for the Think Time (TT), it should be a business requirement that emulates the pauses that users take when using the application in the real world.
Our requirement is to execute
1 user/thread, 10 requests/sec, Total requests = 10,000
With current Thread Properties not able to achieve.
Only one user/thread send 10 request/sec and total should be 10000 requests. Any other way to achieve this in Jmeter ?
Is following approach is correct ? We used Loop controller. So each request will repeat 10 times
You will be able to achieve 10 requests/second with 1 virtual user only if your response time will be 100 milliseconds. If response time will be more than 100 milliseconds - you will not be able to reach the desired load.
If your application fails to respond in 100 milliseconds time frame - most probably you've found a performance bottleneck and you can report this to your application developers.
If you have some time to investigate the issue you can try to provide more information, i.e. what is the average response time, what are minimum, maximum, quantiles, the actual number of requests per second, etc, all this information can be retrieved using Aggregate Report listener.
Normally when people are looking for the answer if the application under test can support X requests per second they use > 1 virtual user as load test should represent real life usage of the system under test and 1 thread (virtual user) isn't normally associated with performance testing and derivatives.
So probably you should reconsider your approach to testing and try increasing the number of threads (virtual users). The throughput can be controlled using Precise Throughput Timer or Constant Throughput Timer. However be aware that the above timers can only pause JMeter to slow it down to the desired throughput. Another approach is using Concurrency Thread Group and Throughput Shaping Timer combination, they can be connected using Feedback Function so JMeter will kick off extra threads in order to reach/maintain the desired number of requests per second.
You can install Concurrency Thread Group and Throughput Shaping Timer using JMeter Plugins Manager
I have a Thread-Group that runs a set amount of time and control the throughput through a Constant Throughput Timer. So in theory, about the same number of Samples should be processed, regardless of the number of Threads deployed.
E.g. ignoring warmup/down the test runs for 10 min with a throughput of 5 Threads/s, therefore roughly 3000 Samples should be processed.
How does changing the number of Threads effect the test?
Ignoring the startup/shutdown period, will the results of the test differ if I have 1000 or 2000 Threads, if only 100 can be completed in a minute?
How can I work out the number of Threads required, given the throughput?
Constant Throughput Timer
Can only pause threads to match the given throughput value. It won't kick off more threads if you don't have enough defined at Thread Group level.
Is accurate enough on minute level. If your test lasts less than a minute - consider other options like playing with ramp-up.
For more information check out How to use JMeter's Throughput Constant Timer
Also it might be better and easier to use Throughput Shaping Timer available via JMeter Plugins project which extends CTT capabilities and provides easy understandable visual view of expected throughput
The number of threads will impact results if your server faces slowdowns.
If so and you don't have enough threads then you will not be injecting the expected load and end up with less transactions performed.
If you have more, then the target load will be injected potentially leading to server crash.
Finally be aware that if you don't make your test last enough, jmeter will potentially inject more than expected because it needs some time to reach the expected req/s , it will inject more at begining.
I need to create a load test in which there's a maximum of 4 concurrent users per second. Then I need to repeat this for an hour.
Is there any way to achieve this in JMeter?
I have tried using this configuration:
Number of Threads: 4
Ramp-up period: 1
Loop count: Forever
Duration:3600
And to make sure that this run exactly for an hour I have also used a Runtime Controller with the same "3600" runtime value.
But these creates more concurrent users per second than what I need and besides it's also unlikely to have that many concurrent users as user usually take so time to think before they do something and this wasn't reflected in my configuration so also tried using Constant Throughput timer , but this only confused me more.
I'm sure I need to use the Constant Throughput Timer, but I don't have the faintest idea of how to do it in my case.
If a use the Constant Throughput Timer, how should I configure the Thread Properties (Number of threads, ramp-up period,loop count, schedule) ?
Do I really need to use a Runtime Controller, to ensure that the test plan runs exactly for an hour or is enough to specify a duration value of 3600?
Constant throughput timer allows you maintain throughput of your sever. (req/sec)
here requests are samplers. Threads are users/clients which are requesting server using samplers.
I hope this have cleared distinction between request and user i.e. sampler and thread.
Now Constant Throughput Timer works on sampler per minute basis. so for your requirement of 4 req per sec. it becomes 240 requests per minute.
and you want to maintain this throughput among all threads (if you have multiple thread groups) or among a single thread group if you have single thread group.
So JMeter engine will start all threads with given ramp-up but will allow only 4 samplers to executes at a given seconds i.e. only 4 request per second.
Thus throughput is achieved among all threads and you can continue this for a specified time using either ways,
Test duration
Runtime controller
I hope it cleared your confusion and solved the issue.