I am trying to understand the exact relationship between the number of users and the spawn rate in locust which I could not find anywhere in the documentation.
So assume that I set the number of users to be 100, the spawn rate to be 50 users per second, and the test duration to be 10 seconds.
Is this a correct interpretation that in 2 seconds (100/50) all the required 100 users are ready and for the next 8 seconds we have 100 constant users running the tasks?
Furthermore, if I use the LoadTestShape class where I can change the number of users and the spawn rate at every tick so that for the next tick I increase the number of users to 150, does it take 1 second to spawn another 50 users to reach 150, or it spawns a whole new batch of 150 users in 3 seconds?
Is this a correct interpretation that in 2 seconds (100/50) all the required 100 users are ready and for the next 8 seconds we have 100 constant users running the tasks?
Yes.
Furthermore, if I use the LoadTestShape class where I can change the number of users and the spawn rate at every tick so that for the next tick I increase the number of users to 150, does it take 1 second to spawn another 50 users to reach 150, or it spawns a whole new batch of 150 users in 3 seconds?
With LoadTestShape you set the target user count (total) and the rate at which you want to approach that number, so option #1.
Note that the exact rate of the ramp up is not 100% guaranteed. If you set the new rate to occur after X seconds, you cant be sure you’ll have all 150 users running at X+1 seconds.
Related
Example:
`1. No of Threads = 100
Ramp-up time = 20, means every 1 second = 5 requests will be processed.
Loop Count =1
In the same time:
If i will do like below
No of Threads = 100
Ramp-up time = 1,
Loop Count =1`
And put Synchronization Timer : No of Simulated user to group = 5.
In this case as well, J meter process 5 requests at one time.
So what is the different logic between the above 2 concepts.
Considering case 1 where the
Number of threads = 100, Ramp-up time = 20 and loop count =1.
This means that for every one sec 5 threads will be induced.
Depending on the time taken for the response the test will run for at least 20 sec or more.
Considering case 2 where the
Number of threads = 100, Ramp-up=1, loop count=1 and
Adding Synchronization Timer with No of Simulated user to group = 5
A batch of 5 users will be added at once.
Both the tests don't take the same amount of time to complete.
In case 1, 5 users are distributed over a period of one sec. i.e. per 0.2 sec one user is added.
In case 2, 5 users are added to the active thread in one go.
To know more about Synchronizing Timer
To best understand the difference between the 2 use cases, increase the number of threads to 200 while keeping the remaining parameters as is. Then the difference can be noticed easily.
means every 1 second = 5 requests will be processed. Loop Count =1 - this is wrong, JMeter will start 5 threads each second, the processing time depends on application response time. The concurrency (number of requests at the same moment of time) is not guaranteed
A quote from JMeter documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
See JMeter Ramp-Up - The Ultimate Guide article for more comprehensive explanation.
Synchronizing Timer guarantees that 5 requests will be executed at the same moment so in your 2nd scenario it would be sending requests in "batches" of 5. Normally people go for Synchronizing Timer to do some form of Spike Testing
This is with respect to Jmeter's loop count behavior.
Number of threads 4000
ramp up period 800
Thread count 2
Action to be taken after a sample error Continue
Same user on each iteration Yes
Delay thread creation until needed Yes
This is resulting in 8000 requests being made in 800 seconds. However, my use case is, 4000 requests in 800 seconds (count=1), then another 4000 in next 800 seconds (count=2).
What changes can I make for this?
Ramp-up period doesn't mean 8000 requests in 800 seconds, it results in:
JMeter starts with 1 virtual user and adds 5 virtual users each second for 800 seconds
Each virtual user executes Samplers upside down for the specified number of iterations
When there are no more Samplers to execute and loops to iterate the thread is being shut down
My expectation is that you only have 1 Sampler and its response time is relatively low (less than 1 second), you can check the actual amount of virtual users and produced load using Active Threads Over Time and Transactions Per Second listeners
If you need to implement 4000 requests for 800 seconds twice the easiest option would be going for the Throughput Shaping Timer and configuring it to reach/maintain 5 requests per second throughput for 800 seconds twice with 10 seconds time frame of doing nothing between loops.
We are running 25k users for load test.
Dev team asks to wait atleast 2 mins on every 1000 users boarded.. Is it possible by IfController by taking count of modules by 1000 and make it pause for 2 mins..?
You can use Ultimate Thread Group and set Delay of 2 minutes by separating every 10 K to a row with Initial delay
separate ramp-up time, shutdown time, flight time for each shedule record
Example:
I have about 300 users (configured in the thread group) who would perform an activity (e.g.: run an e-learning course) twice. That would mean I need to expect about 600 iterations i.e 300 users performing an activity twice.
My thread group contains the following transaction controllers:
Login
Dashboard
Launch Course
Complete Course
Logout
As I need 600 iterations per 5400 seconds i.e 3600 + 900 + 900 seconds (1 hour steady state + 15 mins ramp-up and 15 mins ramp-down), and the sum of sampler requests within the total thread group are 18, would I be correct to say I need about 2 RPS?
Total number of iterations * number of requests per iteration = Total number of requests
600 * 18 = 10800
Total number of requests / Total test duration in seconds = Requests per second
10800 / 5400 = 2
Are my calculations correct?
In addition, what is the best approach to achieve the expected throughput?
Your calculation looks more or less correct. If you need to limit your test throughput to 2 RPS you can do it using Constant Throughput Timer or Throughput Shaping Timer.
However 2 RPS is nothing more than statistical noise, my expectation is that you need much higher load to really test your application performance, i.e.
Simulate the anticipated number of users for a short period. Don't care about iterations, just let your test to run i.e. for an hour with the number of users you expect. This is called load testing
Do the same but for longer period of time (i.e. overnight or weekend). This is called soak testing.
Gradually increase the number of users until you will see errors or response time will start exceeding acceptable thresholds. This is called stress testing.
How to test 1 million users in Jmeter concurrently ? What should I use Number of threads & Ramp up period ?
I tried but I given Number of threads 1000000 and I have to give Ramp up period which don't allow to test in concurrent .
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the
start of a test, and short enough that the last threads start running
before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as
needed.
if you want all your threads starts simultaneously you can to set a ramp-up = 1 this means that jmeter will take 1 second to get all threads up and running! or 0 to starts them asap. but i hope that you have a good hardware to supports this.
http://jmeter.apache.org/usermanual/test_plan.html
Actually I would recommend choosing ramp-up so the load will increase gradually. For instance, your application is able to serve 500k users without any lag, from 500k to 700k response time grows, at 750k concurrent users it dies.
If you release all 1M users at once you won't get above information, all you will know is that your application doesn't support 1M users.
So I would recommend increasing loops count so existing users could continue work while new users are arriving so you will have gradually increasing load up to 1M of concurrent virtual users
Define desired test duration using "Scheduler" option in Thread Group (or using Runtime Controller)
Set "Loop Count" to Forever or put -1 there.
Set your ramp-up phase to be something between 20% and 50% of the overall test duration
Optionally it would be also good to implement "ramp-down" to see the system under test behavior when the load gets back to normal
Also keep in mind the following:
JMeter default settings are good enough for tests development and debugging, you will need to tweak them to achieve the maximum performance. Check out 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure guide for JMeter performance tuning recommendations
It is highly unlikely that you won't be able to simulate 1M of users using single machine so consider setting up Distributed JMeter architecture
1,000,000 threads would get you 1,000,000 Transactions Per Second only if the Response Times were under 1 second. You will need enough threads to include the system response time. 10,000,000 threads would allow for up to 10 second Response Times.
Others are correct, ramp up slowly to the full rate in order to have the time to view where issues lie.
I would use these settings to begin sizing the lab and slave count:
Number of Threads: 10,000,000
Ramp-Up Period: 1800 sec (30 mins)
Loop: Forever
Throughput Shaping Timer:
- Start RPS Row 1: 1
- End RPS Row 1: 1,000,000
- Duration Row 1: 1800 (same as rampup)
Start RPS Row 2: 1,000,000
End RPS Row 2: 1,000,000
Duration Row 2: 600 (10 minutes at full rate)
Start RPS Row 3: 1,000,000
End RPS Row 3: 1
Duration Row 3: 600 (gradually backing threads off to check for resource recovery)
Note what resource runs out first, adjust and repeat.