Recently start learning Jmeter. I noticed that when I set thread group as:
threads: 2,
ramp-up: 10 seconds,
loops: 5.
Part1: it takes 1~5 sec to run the first thread, part2: 6-10sec to run 2nd thread.
And during part1, it loops and sends 5 requests, part2 the same.
My question one is: during each loop, are the 5 requests being sent one after another, and will only start after the previous one is fully finished?
In above example there are plenty of time for each thread to loop. But what will be like if the time is not enough to finish the loop before next thread start:
if threads: 2,
ramp-up: 10 secs,
loops: 500.
Let's say in first 5 seconds the server cannot finish 500 requests, and from sec 6 the thread2 starts. So there will be active requests from both part1 and part2 running at the same time.
Question 2 I'd assume, as within the loop it's sending one after another done, so there will always be 2 active requests (each 1 from 2 threads). By this way it has approximate 500 times of 2 concurrent threads. Is this understanding correct?
Does the above understanding sound correct?
JMeter starts all threads according to what you set in ramp-up period.
2 threads and 10 seconds ramp-up means that after 10 seconds you will have 2 threads up and running, JMeter will start 1 thread in 5 seconds.
Once started the thread begins executing Samplers upside down (or according to the Logic Controllers)
When thread executes last sampler it starts the new iteration immediately. When there are no more samplers to execute and loops to iterate the thread is being shut down.
When there are no more running threads the test ends.
Threads are absolutely independent. Each thread waits for the previous sampler to finish before starting the next sampler. Thread starts next iteration when the last sampler is executed, the other thread can still be running the previous iteration. If you want all of them to complete everything before starting the next iteration take a look at Synchronizing Timer or Inter-Thread Communication Plugin
Related
Example:
`1. No of Threads = 100
Ramp-up time = 20, means every 1 second = 5 requests will be processed.
Loop Count =1
In the same time:
If i will do like below
No of Threads = 100
Ramp-up time = 1,
Loop Count =1`
And put Synchronization Timer : No of Simulated user to group = 5.
In this case as well, J meter process 5 requests at one time.
So what is the different logic between the above 2 concepts.
Considering case 1 where the
Number of threads = 100, Ramp-up time = 20 and loop count =1.
This means that for every one sec 5 threads will be induced.
Depending on the time taken for the response the test will run for at least 20 sec or more.
Considering case 2 where the
Number of threads = 100, Ramp-up=1, loop count=1 and
Adding Synchronization Timer with No of Simulated user to group = 5
A batch of 5 users will be added at once.
Both the tests don't take the same amount of time to complete.
In case 1, 5 users are distributed over a period of one sec. i.e. per 0.2 sec one user is added.
In case 2, 5 users are added to the active thread in one go.
To know more about Synchronizing Timer
To best understand the difference between the 2 use cases, increase the number of threads to 200 while keeping the remaining parameters as is. Then the difference can be noticed easily.
means every 1 second = 5 requests will be processed. Loop Count =1 - this is wrong, JMeter will start 5 threads each second, the processing time depends on application response time. The concurrency (number of requests at the same moment of time) is not guaranteed
A quote from JMeter documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
See JMeter Ramp-Up - The Ultimate Guide article for more comprehensive explanation.
Synchronizing Timer guarantees that 5 requests will be executed at the same moment so in your 2nd scenario it would be sending requests in "batches" of 5. Normally people go for Synchronizing Timer to do some form of Spike Testing
I ran a test plan with 5 users for a total of 20 seconds and what I am not able to understand is what ramp-up period is in actual. Does it means that each user will get 4 seconds or 20 seconds will be used in total for 5 users?
If case 1 is true(4 second for each user) then the first thread should be completed in 4 seconds but it took 6 seconds to complete it and still the result is passed and next user gets executed? This gets much confusing. I need to clear my doubt as I am not able to find any answers from all the inputs that are available here
As per JMeter Thread Group Documentation:
Ramp-up Period
How long JMeter should take to get all the threads started. If there are 10 threads and a ramp-up time of 100 seconds, then each thread will begin 10 seconds after the previous thread started, for a total time of 100 seconds to get the test fully up to speed.
You have 5 users
if you set ramp-up period to 0 - all 5 users will start at once
if you set ramp-up period to 5 - JMeter will start with 1 user and will add an extra 1 user each second
if you set ramp-up period to 10 - JMeter will start with 1 user and will add an extra 1 user each 2 seconds
etc.
Once user is started it starts executing Samplers upside down (or according to Logic Controllers) when there are no more samplers to execute or loops to iterate - the thread is being shut down.
Check out JMeter Ramp-Up - The Ultimate Guide article for more information on configuring users arrival rate.
You might also be interested in Ultimate Thread Group which makes workload definition easier, moreover you will have a chart representing anticipated load. You can install Ultimate Thread Group using JMeter Plugins Manager
If you want threads not to effect test expected time then use instead Thread group's Loop Count as the number of repeatitions you need.
If you want/must use threads calculate ramp period time as
(test expected time + 1 second) * number of threads
I am new to Jmeter. Here is one thing that is puzzling me
There are two scenarios that I am testing to calculate performance numbers using Jmeter
Having a 1 thread group set with loop count set to 50 and having one https sampler.
Having a 1 thread group with loop count as only 1 but instead using a loop controller having loop count set to 50. In this case my https sampler is inside the loop count.
In both cases the https sampler is run 50 times but I am noticing a lot of performance difference.
For 1 - 100 ms ( Average of 50 calls)
For 2 - 30 ms ( Average of 50 calls)
Question is why am I seeing this difference. Ideally both scenario should give the same performance number.
Also to add in my Jmeter setting I have "Use keep alive" as off and also https.use.cached.ssl.context is set to false in jmeter.properties.
What is your Ramp-up setting for scenario where 50 loops are defined on Thread Group level? As per documentation
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds
By default ramp-up is 1. If you're looking for the same behavior for Thread Group and Loop Controller you need to set it to 0.
Going forward, sending requests as fast as JMeter can is not something that could be called a real-life load scenario. Real users aren't hammering this or that endpoint, they need some time to "think" before doing next action. This "think time" can be simulated using i.e. Constant Timer. However the smarter way to set the desired load is using Constant Throughput Timer
I am new to Jmeter and confused about those 4 settings:
number of threads,
ramp up period,
loop count,
duration in scheduler.
Thread Group 1: # of threads:1, ramp up period:0, loop count:5, no scheduler
Thread Group 2: # of threads:5, ramp up period:0, loop count:1, no scheduler
Thread Group 3: # of threads:1, ramp up period:1s, loop count:5, no scheduler
Thread Group 4: # of threads:5, ramp up period:5s, loop count:1, no scheduler
Thread Group 5: # of threads:1, ramp up period:1s, loop count:forever, duration in scheduler:5s
Thread Group 6: # of threads:5, ramp up period:5s, loop count:forever, duration in scheduler:5s
Is there any difference between thread group 1 and 2?
How long is the time cap between each loop?
Is there any difference among thread group 3, 4, 5 and 6?
Should group 3, 4, 5 and 6 all take 5 seconds to finish sending all requests? Should the total # of results be 5 in each group?
When I tried thread group 5, I got around 83 responses, why is there not just 5 responses? Does this mean the ramp up period is not working?
I am so confused about the differences between those groups, hoping someone can help me out. Thanks a lot in advance!
Firstly, short definition of each parameter:
Number of Threads: simulates the number of users
Ramp up period : how much time Jmeter takes to get all threads started
Loop count: number of times each thread performs the test case
Duration in scheduler: the duration for which the test is to be run. In this case, the value in the end-time field shall be ignored and the relative end-time will be calculated from the start-time and duration
A important thing to understand here is that the Ramp-up period has no relation to the Loop count. It is only related to the Number of Threads. If there are 10 threads and a ramp-up time of 100 seconds, then each thread will begin 10 seconds after the previous thread started, for a total time of 100 seconds to start all threads in the thread group.
Now the answer to each Thread Group Case:
Thread Group 1: # of threads:1, ramp up period:0, loop count:5, no scheduler
Thread Group 2: # of threads:5, ramp up period:0, loop count:1, no scheduler
Thread Group 3: # of threads:1, ramp up period:1s, loop count:5, no scheduler
Thread Group 4: # of threads:5, ramp up period:5s, loop count:1, no scheduler
Thread Group 5: # of threads:1, ramp up period:1s, loop count:forever, duration in scheduler:5s
Thread Group 6: # of threads:5, ramp up period:5s, loop count:forever, duration in scheduler:5s
Q1. Is there any difference between thread group 1 and 2?
Thread Group 1 has a single thread that will repeat the same test 5 times i.e. as one test finishes repeat the test till a loop count of 5 is reached
Thread Group 2 has 5 threads that will start simultaneously as the Ramp-up Period is 0, so there it shall mock 5 users simultaneously on the system
Q2. How long is the time cap between each loop?
The time cap is a factor of the ramp-up period that is divided equally between the number of threads in the thread group. Loop is a continuous process that shall continue till the loop count is reached. "Time cap" is not a term used in the JMeter docs, but assuming what you mean, it can be set between loops by using Test Action or other JMeter mechanisms if you need fine-grained control.
Q3. Is there any difference among thread group 3, 4, 5 and 6?
Yes, all the parameters are different, so as described above, the behavior is different.
Q4. Should group 3, 4, 5 and 6 all take 5 seconds to finish sending all requests? Should the total # of results be 5 in each group?
No, it is all subjective to each condition.
In Thread Group 3, a single thread shall execute the same thread 5 times, so no control over time
In Thread Group 4, there are 5 thread and based on the ramp-up period, each thread shall start 1 s after the other, so it will take at-least 4-5 seconds to complete the test (after the last test)
Thread Group 5 and 6 shall each only run for a duration for 5 seconds. If the test is completed before that then fine, otherwise the test shall be terminated and stopped
Q5. When I tried thread group 5, I got around 83 responses, why is there not just 5 responses? Does this mean the ramp up period is not working?
Why should there be only 5 responses??? Thread Group 5 shall create a single thread that shall execute the test in loop for ever. If the duration was not defined the test would run in an infinite loop. Since the duration is defined, the test shall only run for the duration period.
Note to All: The loop count is thread specific. It specifies the number of times the test should be repeated for each thread. The ramp-up period is related to the #Thread (User) and has no relation to the loop count
Yes, 1 simulates one user doing 5 times the test scenario inside Thread Group1, 2, simulates 5 users executing test scenario one time
No time if you don't use any Timer
Yes, 3 same as Thread Group 1, 4 same as Thread Group 2 but users are started 1s after each others, 5, same as Thread Group 1 but running for 5 s, so you don't know how many times it will execute. 6 same as Thread Group 2 but runs for 5 s
5 and 6 will run for 5 s, 3 and 4 as explained in 3. No
5.Because it runs for 5 seconds without any pause as you didn't set any timer. As you have only one thread, it starts immediately and runs for 5 seconds.
Read:
http://jmeter.apache.org/usermanual/test_plan.html
I want to run a JMeter test with a number of concurrent threads with each thread sending a request every 10 seconds.
These are my thread properties.
Number of Threads: 10
Ramp-Up Period: 10
Loop Count: 1
Result: 10 requests divided over 10 seconds, so every second a request and exactly what I want.
Now I want to run this test for 3 times(30 seconds). So I set the Loop Count to 3.
But the result is: 30 requests in 10 seconds. This is strange, because I would expect to run this for 30 seconds and get 1 request per second.
How can I achieve this with JMeter?
My final goal is to run this test for a long period and also increase the Number of Threads.
How to do this with JMeter?
I researched this today and came to this conclusion: The Loop Count setting is a complete misnomer. It doesn't actually loop in any sort of chronological sense, even if your Test Plan has Run Thread Groups consecutively checked. What it does do is multiply your thread group and run all multiples concurrently. Therefore, the Ramp-Up Period is only respected once, and NOT once per "loop" - there is no temporal loop!
An explanation with graphs can be found here: http://pro-programmers.blogspot.com/2009/07/jmeter-max-threads-with-rump-up-and.html
Seems that the most simplest ways to control throughput in your tests is using either standard "out-of-box" Constant Throughput Timer or custom Throughput Shaping Timer from jmeter-plugins collection.
In both the cases structure of the test will be like the following:
Thread Group
Number of Threads = N
Ramp-up Period = N
Loop Count = 1
Constant Throughput Timer
Target Throughput = 60
Calculate Throughput based on = "all active threads in current thread group"
. . .
Loop Controller
Loop Count = M
. . .
HTTP Request
. . .
Here Loop Controller defines number of iterations.
Looks like both the timers are not absolutely precise as well as both are a bit differently configurable:
Here is also a kind of practical example how to vary the throughput.
In my experience with Jmeter if you set
Number of Threads: 10
Ramp-Up Period: 10
Loop Count: 1
you create 10 threads into 10 seconds so you create 1 thread every second.
With loop count of 1 you repeat this once.
But if you increase loop count I think that you don't create new threads but repeat jmeter elements procedure in the Thread Group therefore the time beetween the request isn't 30 seconds but just over 10s.
If you want to create 30 threads within 30 seconds you have to set
Number of Threads: 30
Ramp-Up Period: 30
Loop Count: 1
If you want to repeat 10 threads for 3 times with ramp- up of 10 seconds you should insert a Timert->Constant Timer with thread delay of 10000 ms so you obtain 30 requests in 30 seconds (really you should consider the time of executions of the task)
refer link: http://www.testingjournals.com/5-must-know-features-thread-group-jmeter/
scenario 1 :
number of threads = 20
Ramp-up period =100
loop count=1
Every 5 seconds(100/20) 1 request/thread will hit the server,Execution will start with 1 request at a time
scenario 2 :
number of threads = 20
Ramp-up period =100
loop count=4
Every 5 seconds(100/20) 4 request/thread will hit the server, once the 1st request/thread completes , it will start 2nd loop by executing same HTTP request,Execution lasts until all 20 threads executes all HTTP requests 4 times
My understanding of Ramp-Up is that a value of 0 will fire all threads at the same time (concurrently).
You might be able to achieve what you want by setting the following:
Number of Threads: 10
Ramp-Up Period: 0
Loop Count: N
and then using a Controller to determine when to end each loop.
Yup, the Loop Count parameter is not intuitive. From what I figured out it is actually the number of times the thread/user performs a particular test
So if I understand correctly your intention that you want:
to run N number of concurrent threads/users
each of them sends a request every 10 seconds
you want to run this scenario for around 30 seconds
the configuration should be the following:
Number of Threads: N
Ramp-up Period: 0
Loop Count: 3
And in Constant Throughput Timer (within the Thread Group) you should set Target Throughput (samples per minute)=6, which means request every 10 seconds