Let's say that I have the current configuration :
Number of Threads (users): 150
Ramp-up : 30
Loop Count : None
If I add up a duration of 2 mins so :
Number of Threads (users): 150
Ramp-up : 30
Loop Count : None
Duration (minutes) : 2
How does Jmeter going to react? If each Threads take about 10 seconds to complete
Thanks in advance
Both Loop Count and Duration (if both present) are taken into account, whichever comes first. So in first configuration, you are not limiting loop count or duration, so the script will run "forever". In second case, loop count is still not limited, but duration is. So the test will stop 2 minutes after startup of the very first user, and the time includes ramp-up time. Stopping includes not running new samplers, and hard stop for all running samplers.
In your case, 150 users will finish starting after 30 sec. That means the first thread to run will complete 3 iterations (x10 sec) by the time the last thread just started its first.
Within the remaining 90 sec, all threads will complete roughly 8-9 iterations.
So for the first thread you should expect 11-12 iterations, for the very last thread to start, 8-9 iterations. Remaining threads anywhere between those numbers. Assuming ~30 threads executed same number of iterations, between 8 and 12, you get roughly 1500 iterations in total (could be a little over or under). Last iteration of each thread may be incomplete (e.g. some samplers did not get to run before test ran out of time).
Generally, since duration may leave unfinished iterations, I think it's only good as a fall back or threshold in pipeline automation. For example: run is configured to complete 1000 iterations (should take about 16 min if iteration takes 10 sec). So duration is set to 24 min (gives about 50% slack). It won't be needed if performance is decent, but if execution takes extremely long, we may hard stop it at 24 min, since there's no point to continue: we already know something is wrong.
Related
How to calculate inputs (virtual users, ram up period) for performance and load testing using Jmeter.
If you're doing Load testing - you are supposed to be checking your system behaviour against the anticipated load, it means that you should know the expected number of users from NFR or SLA
If you don't have any requirements - go for the stress testing, i.e. start with 1 user and gradually increase the load until response time starts increasing or errors start occurring (whatever comes the first)
With regards to the ramp-up period duration - there are no strict requirements, as per JMeter Documentation
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
so if you don't have any better ideas - just go for the above recommendation.
Running JMeter version 5.4.1
I have the following setup for a JMeter thread group:
Number of Threads: 300
Ramp-up Period: 300 seconds
Loop count: infinite
Specify Thread Life time: Duration 600 seconds, startup delay: 0
Thread group is set to continue after a sample error.
So with the above setup, I'm expecting the test to spin up 1 thread per second, when it reaches 300(5min) seconds, it will have a full 300 threads running. Each thread should be running for 600 seconds(10min) before stopping. I'm expecting the total run time to be 900 seconds (Last thread starts at minute 5 + 10 min run time = 15min).
But the actual behavior is different and I can't figure out why.
Actual behavior:
--300 threads spin up after 300 seconds, 1 thread per second. Cool this is what I'm expecting.
--at 600 second mark, the first thread start to die. Cool so far so good
--at 12min which is 720 seconds, ALL threads have died off, and the test is complete... umm.. what happened here? I was expecting a full 300 seconds wind down time, not 120 seconds?
As a result, the full test finished at 720 seconds (12min), instead of the full 900 seconds (15min).
Can anyone explain to me what is going on here?
You have to configure your test duration accordingly.
In your case, the test finishes at 600 seconds. The last 120 seconds was the ramp-down time of those active threads. You can observe it through Active Threads Over Time Listener.
Your total test duration would be like warm-up period + hold load activity + warm-down period. i.e (300+600+300)= 1200 seconds or 20 mins.
300 threads will be active in 300 seconds, then all the 300 threads will hit requests to the server for 600 seconds, and then those 300 threads will be down in the next 300 seconds.
You can also use Ultimate Thread Group like this as well for a better understanding of your test, where the warm-down period is used as 120 seconds, so the total duration of the test is (5+10+2)=17 mins or 1020 seconds.
It isn't exact and specify duration of the test plus finishing iteartion
This denotes the total duration of the test. One disadvantage to run the test by specifying the duration value is that the test stops immediately when the test duration is completed; without completing the loop (iteration) due to which you could see some differences in transaction count.
You specified test duration of 600 seconds, why do you expect your test to last longer?
Once 600 seconds are passed JMeter will ask threads to stop and the time required for all of them to stop mainly depends on the nature of the test, request/response size, etc. In ideal conditions the test should be stopped immediately.
So if you want your test to run 15 minutes - set duration to 900 seconds.
If you want the gradual ramp-down - consider adding a Timer to your Test plan or switch to a custom thread group which allows precise control over threads, see How to Ramp Down in JMeter article for more details.
Coming back to threads termination taking 2 minutes, it should not be the case, make sure to run your JMeter test in command-line non-GUI mode and follow other JMeter Best Practices
Example:
`1. No of Threads = 100
Ramp-up time = 20, means every 1 second = 5 requests will be processed.
Loop Count =1
In the same time:
If i will do like below
No of Threads = 100
Ramp-up time = 1,
Loop Count =1`
And put Synchronization Timer : No of Simulated user to group = 5.
In this case as well, J meter process 5 requests at one time.
So what is the different logic between the above 2 concepts.
Considering case 1 where the
Number of threads = 100, Ramp-up time = 20 and loop count =1.
This means that for every one sec 5 threads will be induced.
Depending on the time taken for the response the test will run for at least 20 sec or more.
Considering case 2 where the
Number of threads = 100, Ramp-up=1, loop count=1 and
Adding Synchronization Timer with No of Simulated user to group = 5
A batch of 5 users will be added at once.
Both the tests don't take the same amount of time to complete.
In case 1, 5 users are distributed over a period of one sec. i.e. per 0.2 sec one user is added.
In case 2, 5 users are added to the active thread in one go.
To know more about Synchronizing Timer
To best understand the difference between the 2 use cases, increase the number of threads to 200 while keeping the remaining parameters as is. Then the difference can be noticed easily.
means every 1 second = 5 requests will be processed. Loop Count =1 - this is wrong, JMeter will start 5 threads each second, the processing time depends on application response time. The concurrency (number of requests at the same moment of time) is not guaranteed
A quote from JMeter documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
See JMeter Ramp-Up - The Ultimate Guide article for more comprehensive explanation.
Synchronizing Timer guarantees that 5 requests will be executed at the same moment so in your 2nd scenario it would be sending requests in "batches" of 5. Normally people go for Synchronizing Timer to do some form of Spike Testing
Whats the difference in JMeter between:
(a) 50 threads 5 second ramp up and 1 loop count
(b) 10 threads 1 second ramp up and 5 loop count
For me (a) and (b) seem to be the same thing, 10 threads will be instantiated every second for a total of 5 seconds.
Am I missing something here?
ramp-up period doesn't effect total number of threads. see user manual:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen
So both scenarios will be executed 50 times, the difference is the order of the threads. In your case let's assume each thread takes ~5 minutes
Scenario a (without encountering any bottleneck) will take ~5 minutes (+5 seconds ramp up)
While Scenario b will take approximately 5 times more because each loop will start after threads ends. 5 minutes will take for each loop - ~25 minutes (+1 second ramp up)
Also you can have limitation on your server that doesn't allow 50 threads executing in 5 seconds same operation so Scenario a will be impossible to execute.
I am under the impression that I use the 3 settings; Threads, Ramp-up and Loop, to simulate X (Threads) amount of users/thread over Y (Ramp-up) seconds and do these for Z (Loop) amount of times. For example I want 10 users every 1 second for 1 hour which equates to 10 Threads, 1 Second Ramp-up and 3600 Loops.
But :)
Others seems to be using it differently ... as in if they want the same as above they would set Threads to 36000, Ramp-up to 3600 seconds and Loop to 1.
I tend to think the first approach is correct based on #a it reads better :) and #b why would you have a setting based in seconds to indicate the length of your test
Can anyone give me a definitive answer or are both options plausible?
Firstly 36000 threads in the second example seems very high! http://wiki.apache.org/jmeter/HowManyThreads reports people using 1000. So the 2nd scenario may not even work.
The two scenarios you describe are not exactly the same, and I'm not sure if either are exactly what you want.
In the first the 10 Threads plus 1 second Ramp-up means all 10 Threads will be in use after 1 second. The 10 Threads will then do their actions in parallel 3600 times. You have not mentioned anything that will mean the test will take 1 hour, it will take as long as it takes to loop through 3600 times. To make it take an hour (assuming the actions don't take longer than 1 second) you would need something like a Constant Throughput Timer within your loop which control the speed of the loop so that it takes an hour.
In the second, 10 threads would be created in second #1 and start doing their loop, another 10 in the second#2, etc, all the way up to 1 hour (second#3600). If the actions take longer than one second, then you would have more than 10 threads running in any one second.
The first approach is much clearer. The 2nd one is a misuse of the Ramp-Up; it's not being used to provide a ramp-up to 360000 threads, but to try and schedule 10 threads to run at once.