I ran a test plan with 5 users for a total of 20 seconds and what I am not able to understand is what ramp-up period is in actual. Does it means that each user will get 4 seconds or 20 seconds will be used in total for 5 users?
If case 1 is true(4 second for each user) then the first thread should be completed in 4 seconds but it took 6 seconds to complete it and still the result is passed and next user gets executed? This gets much confusing. I need to clear my doubt as I am not able to find any answers from all the inputs that are available here
As per JMeter Thread Group Documentation:
Ramp-up Period
How long JMeter should take to get all the threads started. If there are 10 threads and a ramp-up time of 100 seconds, then each thread will begin 10 seconds after the previous thread started, for a total time of 100 seconds to get the test fully up to speed.
You have 5 users
if you set ramp-up period to 0 - all 5 users will start at once
if you set ramp-up period to 5 - JMeter will start with 1 user and will add an extra 1 user each second
if you set ramp-up period to 10 - JMeter will start with 1 user and will add an extra 1 user each 2 seconds
etc.
Once user is started it starts executing Samplers upside down (or according to Logic Controllers) when there are no more samplers to execute or loops to iterate - the thread is being shut down.
Check out JMeter Ramp-Up - The Ultimate Guide article for more information on configuring users arrival rate.
You might also be interested in Ultimate Thread Group which makes workload definition easier, moreover you will have a chart representing anticipated load. You can install Ultimate Thread Group using JMeter Plugins Manager
If you want threads not to effect test expected time then use instead Thread group's Loop Count as the number of repeatitions you need.
If you want/must use threads calculate ramp period time as
(test expected time + 1 second) * number of threads
Related
I have a requirement like API should be capable of handle 5k Concurrent user in 5 Sec.
How can i test this scenario in jmeter.
For 1k Concurrent user what should the RampUp time to be set in Jmeter.
5000 for 5 seconds is quite easy to achieve:
With normal Thread Group
With Concurrency Thread Group
With regards to the ramp-up, I can only give you:
a quote from JMeter documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
and a piece of advice on ramp-up in general: if you're doing some form of a stress test it's a good idea to increase the load gradually, this way you will be able to correlate increasing load with the throughput and response time because running 5000 threads for 5 seconds will give you only a basic information on the response time for this workload and you won't be able to tell what is the maximum number of users/requests per second the application can support, when performance starts degrading, etc. So you could consider a load pattern like this:
and look at Response Times Over Time chart to see the relationship between the number of concurrent users and the application response time
I am testing an API endpoint. I would like to know if the server can handle 100 users in one minute.
This is the thread configuration that I am currently using.
My question is:
did I configure the threads, ramp-up period, and duration parameter correctly?
How much ramp up period is ideal, against the number of threads?
I would recommend either decreasing the ramp-up or increasing the test duration because with your current settings you will have 100 users online only during the last second of the test
So set "Duration" to 120 seconds so during first minute your threads would be ramping up followed by another minute of 100 online users or go for something like:
30 seconds of ramp-up
180 seconds of duration
You may find i.e. Ultimate Thread Group easier to use, it's default workload pattern seems to match your scenario:
and you will be able to play with ramp-up, ramp-down and time to hold the load and see the anticipated threads count in the chart.
You can install Ultimate Thread Group as a part of "Custom Thread Groups" bundle using JMeter Plugins Manager
The whole point of ramp-up is that your should be increasing the load gradually so you could correlate the increasing load with other metrics (throughout, response times, errors, etc.)
As per JMeter Documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
Example:
`1. No of Threads = 100
Ramp-up time = 20, means every 1 second = 5 requests will be processed.
Loop Count =1
In the same time:
If i will do like below
No of Threads = 100
Ramp-up time = 1,
Loop Count =1`
And put Synchronization Timer : No of Simulated user to group = 5.
In this case as well, J meter process 5 requests at one time.
So what is the different logic between the above 2 concepts.
Considering case 1 where the
Number of threads = 100, Ramp-up time = 20 and loop count =1.
This means that for every one sec 5 threads will be induced.
Depending on the time taken for the response the test will run for at least 20 sec or more.
Considering case 2 where the
Number of threads = 100, Ramp-up=1, loop count=1 and
Adding Synchronization Timer with No of Simulated user to group = 5
A batch of 5 users will be added at once.
Both the tests don't take the same amount of time to complete.
In case 1, 5 users are distributed over a period of one sec. i.e. per 0.2 sec one user is added.
In case 2, 5 users are added to the active thread in one go.
To know more about Synchronizing Timer
To best understand the difference between the 2 use cases, increase the number of threads to 200 while keeping the remaining parameters as is. Then the difference can be noticed easily.
means every 1 second = 5 requests will be processed. Loop Count =1 - this is wrong, JMeter will start 5 threads each second, the processing time depends on application response time. The concurrency (number of requests at the same moment of time) is not guaranteed
A quote from JMeter documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
See JMeter Ramp-Up - The Ultimate Guide article for more comprehensive explanation.
Synchronizing Timer guarantees that 5 requests will be executed at the same moment so in your 2nd scenario it would be sending requests in "batches" of 5. Normally people go for Synchronizing Timer to do some form of Spike Testing
I have a problem with JMeter that I can't figure out.
I have 3 requests in one thread group. Each request needs to be executed 5000 times in 5 minutes which means I need to execute 15 000 requests in 5 minutes altogether.
The thread group is set up like this:
Number of threads (users): 50
Ramp-up period: 1
Loop count: forever
Duration (seconds): 300
Each request has a Constant throughput Timer:
Target throughput: 1000.0
Calculate Throughput based on: all active threads in current thread group
And the results after the test run is complete are around 2000-3000 requests and that is too low. I can't figure out which settings are wrong.
Tests are ran in non-GUI mode on an Azure virtual machine with a Intel Xeon E5-2673 #2.3 GHz with 16 cores and 32gb of RAM and the server is more than capable of executing more requests since the response time is about 1 second so hardware should not pose a problem. JMeter heap memory is set to 12 GB.
Any help is appreciated!
I have 3 requests in one thread group. Each request needs to be executed 5000 times in 5 minutes which means I need to execute 15 000 requests in 5 minutes altogether.
That means you have to execute thread group for 5000 times in 5 mins => 300 seconds. If you execute thread group that much of time, you will automatically generate 15k requests.
From Documentation:-
Loop Count Number of times to perform the test case. Alternatively, "forever" can be selected causing the test to run until manually stopped.
When you set loop count: forever or any value in loop count for that matter, it means you want next set of users(threads) [which you have defined as 50 now], only when you are done with the first set 50 users request (150 requests as each thread group as 3 requests). Now, this 50 users request may take unexpected time and your next set will be returned only after this 50 users are done.
So to solve the problem at hand, you should have the following configuration for the thread group :
Number of threads (users): 5000
Ramp-up period: 300
Loop count: 1
So this means after every second around 17 Threads would be on-boarded.
To achieve your desired value configure the following
Number of threads (users): 50 / 60
Ramp-up period: 30
Loop count: forever
Duration (seconds): 300
It is good practice to give some ramp up time.
Also Constant Throughput Timer works on minute level and your desired throughput should be
Target throughput (in minutes): 3000.0
Calculate Throughput based on: all active threads in current thread group
So now, in 1 minute, JMeter will try to send 3000 requests and in 5 minutes, request count will be more or less around 15000.
Here are a few recommendations to tackle tests like this:
Use Ultimate Thread Group to create the threads required to run your tests.
Increase the total number of threads to 150-200 - Depending on the response time of your requests, 50 threads may be busy and you are not able to scale beyond certain throughput.
Add throughput shaping timer under Ultimate thread group, to maintain the throughput of 50RPS for 5 minutes(300 Seconds).
The combination of Ultimate Thread Group and Throughput Shaping Timer will allow you to have granular control over your requests and RPS you want to achieve.
Set Ulitmate Thread Group and Throughput Shaping Timer as below.
Put all your requests below Throughput Shaping Timer and you should be able to achieve the expected load.
Try to use Stepping thread group. It will allow you to send 50 user per second upto 300 second.
Download jar from below link.
https://jmeter-plugins.org/wiki/SteppingThreadGroup/
I am new to Jmeter. Here is one thing that is puzzling me
There are two scenarios that I am testing to calculate performance numbers using Jmeter
Having a 1 thread group set with loop count set to 50 and having one https sampler.
Having a 1 thread group with loop count as only 1 but instead using a loop controller having loop count set to 50. In this case my https sampler is inside the loop count.
In both cases the https sampler is run 50 times but I am noticing a lot of performance difference.
For 1 - 100 ms ( Average of 50 calls)
For 2 - 30 ms ( Average of 50 calls)
Question is why am I seeing this difference. Ideally both scenario should give the same performance number.
Also to add in my Jmeter setting I have "Use keep alive" as off and also https.use.cached.ssl.context is set to false in jmeter.properties.
What is your Ramp-up setting for scenario where 50 loops are defined on Thread Group level? As per documentation
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds
By default ramp-up is 1. If you're looking for the same behavior for Thread Group and Loop Controller you need to set it to 0.
Going forward, sending requests as fast as JMeter can is not something that could be called a real-life load scenario. Real users aren't hammering this or that endpoint, they need some time to "think" before doing next action. This "think time" can be simulated using i.e. Constant Timer. However the smarter way to set the desired load is using Constant Throughput Timer