Can someone explain to me if the loop sets the number of requests or it`s there to have an average?
If I have 100 users, ramp - up set as 0 and loop 1, does it mean that I have only 100 users and if I increase the loop to 2, does it mean that there will be 200 users doing the request?
If I needed to test with 200 users, why would I not set the users to 200? What does the loop do differently and how does it affect the result?
If you have only one request (sampler) under the Thread Group:
With loop count 1 - 100 users will execute 1 request 1 time each, 100 requests in total
With loop count 2 - 100 users will execute 1 request 2 times each, 200 requests in total
With regards to setting ramp-up to 0 - it's not the best idea, it is good to increase the load gradually, this way you will be able to correlate increasing load with other performance metrics, in particular with response time and number of requests per second.
More information: JMeter Ramp-Up - The Ultimate Guide
P.S. It might be easier to consider i.e. Ultimate Thread Group which makes the process of defining the workload easier, for example here is how you can configure JMeter to start with 1 user, increase the load up to 200 users in 60 seconds, hold the load for another 60 seconds and then gradually decrease the load:
You can install Ultimate Thread Group as a part of Custom Thread Groups bundle using JMeter Plugins Manager
Related
I am using Jmeter normal Thread Group to simulate 1 user to loop and send a total of 1000 requests. Is the total time taken to send 1000 requests the total runtime?
Eg. 1 user, 1000 requests (looped to do so), on top right after it finishes I see a timing of 1 min 40 seconds. 1 user spends 100 seconds to send 1000 requests?
So average of 1 requests per 0.1 second see
Yes it's a viable approach. However if you want the total time to execute 1000 requests in the loop to appear in the .jtl results file and/or HTML Reporting Dashboard you could amend your test plan a little bit like:
Thread Group with 1 user and 1 loop and 0 ramp-up
Transaction Controller
Loop Controller with 1000 loops
Your Sampler
This way the Transaction Controller will generate a synthetic Sample Result and its elapsed time will be the sum of all its children (1000 Sampler executions)
i've started distributed performance testing using jmeter. If i give scenario 1:
no.of threads: 10
ramp up period: 1
loop count: 300
Everything runs smooth, as scenario 1 translates to 3000 requests in 300 seconds. i.e. 10 requests per second.
If i give scenario 2:
no.of threads: 100
ramp up period: 10
loop count: 30
Afaik, scenario2 is also executing 3000 requests in 300 seconds, i.e. 10 requests per second.
But things started failing i.e. sever is facing heavy load and requests fail. In theory both scenario1 and scenario2 should be same, right? Am i missing something?
All of these are heavy calls, each one will take 1-2 seconds under normal load.
In ideal world for scenario 2 you would have 100 requests per second and the test would finish in 30 seconds.
The fact that in 2nd case you have the same execution time indicates that your application cannot process incoming requests faster than 10 per second.
Try increasing ramp-up time for 2nd scenario and look into the following charts:
Active Threads Over Time
Response Times Over Time
Transactions Per Second
Normally when you increase the load the number of "Transactions Per Second" should increase by the same factor and "Response Time" should remain the same. Once response time starts growing and number of transactions per second starts decreasing it means that you passed the saturation point and discovered the bottleneck. You should report the point of maximum performance and investigate the reasons of the first bottleneck
More information: What is the Relationship Between Users and Hits Per Second?
In scenario 2 after 10 seconds you have 100 concurrent users which execute requests in parallel, your server may not handle well or prevent such load
Concurrent user load testing sends simultaneous artificial traffic to a web application in order to stress the infrastructure and record system response times during periods of sustained heavy load.
In scenario 1 after 10 seconds you have 10 concurrent users looping through the flow, without causing a load on server
Notice your server may have restriction on number of users requesting only on specific request(s)
We shall be very clear about the Rampup time
Following is extract from the official documentation
Scenario 1 : no.of threads: 10
ramp up period: 1
loop count: 300
In the above scenario 10 threads(virtual users) are to be created in 1 seconds. Each user will loop 300 times. Hence there will be 3000 requests to the server. Throughput cannot be calculated in advance with above configuration. It fluctuates based on the server capability, network etc. You could control the throughput with some components and plugins.
Scenario 2 : no.of threads: 100
ramp up period: 10
loop count: 30
In scenario 2 100 threads (virtual users) are created in 10 seconds. 100 virtual users will send requests concurrently to the server. Each user will send 30 requests. In the second scenario you will have higher throughput (number of requests per seconds) as compared to the scenario 1. Looks like server cannot handle the 100 users sending requests concurrently.
Ramp up time is applicable for the first cycle of each thread. It will simulate delays between first request of each user in their first iteration.
I have a problem with JMeter that I can't figure out.
I have 3 requests in one thread group. Each request needs to be executed 5000 times in 5 minutes which means I need to execute 15 000 requests in 5 minutes altogether.
The thread group is set up like this:
Number of threads (users): 50
Ramp-up period: 1
Loop count: forever
Duration (seconds): 300
Each request has a Constant throughput Timer:
Target throughput: 1000.0
Calculate Throughput based on: all active threads in current thread group
And the results after the test run is complete are around 2000-3000 requests and that is too low. I can't figure out which settings are wrong.
Tests are ran in non-GUI mode on an Azure virtual machine with a Intel Xeon E5-2673 #2.3 GHz with 16 cores and 32gb of RAM and the server is more than capable of executing more requests since the response time is about 1 second so hardware should not pose a problem. JMeter heap memory is set to 12 GB.
Any help is appreciated!
I have 3 requests in one thread group. Each request needs to be executed 5000 times in 5 minutes which means I need to execute 15 000 requests in 5 minutes altogether.
That means you have to execute thread group for 5000 times in 5 mins => 300 seconds. If you execute thread group that much of time, you will automatically generate 15k requests.
From Documentation:-
Loop Count Number of times to perform the test case. Alternatively, "forever" can be selected causing the test to run until manually stopped.
When you set loop count: forever or any value in loop count for that matter, it means you want next set of users(threads) [which you have defined as 50 now], only when you are done with the first set 50 users request (150 requests as each thread group as 3 requests). Now, this 50 users request may take unexpected time and your next set will be returned only after this 50 users are done.
So to solve the problem at hand, you should have the following configuration for the thread group :
Number of threads (users): 5000
Ramp-up period: 300
Loop count: 1
So this means after every second around 17 Threads would be on-boarded.
To achieve your desired value configure the following
Number of threads (users): 50 / 60
Ramp-up period: 30
Loop count: forever
Duration (seconds): 300
It is good practice to give some ramp up time.
Also Constant Throughput Timer works on minute level and your desired throughput should be
Target throughput (in minutes): 3000.0
Calculate Throughput based on: all active threads in current thread group
So now, in 1 minute, JMeter will try to send 3000 requests and in 5 minutes, request count will be more or less around 15000.
Here are a few recommendations to tackle tests like this:
Use Ultimate Thread Group to create the threads required to run your tests.
Increase the total number of threads to 150-200 - Depending on the response time of your requests, 50 threads may be busy and you are not able to scale beyond certain throughput.
Add throughput shaping timer under Ultimate thread group, to maintain the throughput of 50RPS for 5 minutes(300 Seconds).
The combination of Ultimate Thread Group and Throughput Shaping Timer will allow you to have granular control over your requests and RPS you want to achieve.
Set Ulitmate Thread Group and Throughput Shaping Timer as below.
Put all your requests below Throughput Shaping Timer and you should be able to achieve the expected load.
Try to use Stepping thread group. It will allow you to send 50 user per second upto 300 second.
Download jar from below link.
https://jmeter-plugins.org/wiki/SteppingThreadGroup/
There are many questions/answers available here to understand Ramp up time but I want to get something in detail for my test case.
Test Case : Expecting 1200 users per 5 minutes on the home page. So it will be like 5 users/second.
I have set following thread properties :
No. of Threads : 1200
Ramp-up Time - ? [I am not sure what to set]
Loop count - Forever
Scheduler - 300 Seconds[5 Minutes]
Can anyone please help me to set ramp up time for my test case? I am running the test on my local machine.
I want to check that how many users server can handle in 5 minutes. Our expectation is 1200 users.
As per you test case and duration settings, I think, you are making confusion with the Threads and Requests.
In short: You must use Ramp-up as 240 seconds for this scenario.
Let me describe you in details:
Ramp up is the time in which all the users arrive on your tested application server.
Requests are simulated by samplers but threads are the simulation of users.
Please note- The total number of requests are related to throughput, Whereas the number of active threads performing the same activity is related to concurrency.
From your requirements, I assume you want to measure the throughput which is related to the requests/second not the users per second. To achieve this, you can use a Constant Throughput Timer at your test plan level.
Constant Throughput timer allows you maintain throughput of your server (requests/sec). Here requests are samplers. Threads are users/clients which are requesting server using samplers.
You can achieve this by using both "Thread Group" or "Ultimate Thread Group".
As far as I know, it doesn't matter which type of Thread group you are using with Constant Throughput timer, at the end of the test in the results, you will get your desired Throughput which you mentioned in Constant Throughput Timer i.e. if you have mentioned 1200/min in Constant Throughput Timer with "Calculate Throughput based on" value as "All active threads" and there are 3 requests (samplers) in your thread group then JMeter will manage the requests in a way that it will generate only 6.6 requests/sec for each sampler i.e. 1200/min is divided among 3 requests, it doesn't matter whether you are using Ultimate Thread Group or the Thread Group; Throughput timer works in same way for all types of thread groups.
Now as per your requirement: (Using Thread Group):
No. of Threads : 1200
Ramp-up Time - 240 (Since you want 5 users per second, 1200/5= 240)
Loop count - Forever
Scheduler (Duration) - 780 Seconds [13 Minutes]
IMPORTANT:
Thread Group configurations will be the same which you have already mentioned. Number of threads will be the client load which you want to put(in you case it's 1200), duration : (240+300+240), loop : forever, ramp-up : 240 yes when you ramp up i.e. in the first minute generally you get more requests as threads are starting so it is normal behavior. To test your scenario run for exactly 5 minutes, you should test for 13 minutes. Exclude first 4 min and last 4 min of the test as that is warm up time for your test as well as server and last 4 min are warm down period.
Now as per your requirement: (Using Ultimate Thread Group):
Start Thread Count: 1200
Initial delay, sec: 0
Startup, sec: 240 (I assume you want 5 users come to your application for requesting expected samples per second)
Hold Load for: 300 sec (Since you required 5 minutes to test for 1200 users)
Shutdown Time, sec: 240
You can set just like this:
Don't get confuse the Thread number with the number of requests, each thread will create multiple requests for seconds in field Hold Load for.
Rampup is the speed at which users arrive on your application.
The number you put means that once it is reached all threads have been started.
The tiniest it is , the faster users arrive.
So nobody can answer that for you, you need to analyse your traffic.
Regarding the number of threads, 1200 threads might not be the good number, as 1200 users can navigate on application slowly or rapidly.
What you need is add a Constant Throughput Timer to control how much transactions per second you'll be reaching.
You should use the Ultimate Thread Group plugins available in Standard set.
To maintain concurrency the configuration should be like this:
Start Thread Count: 1200
Initial delay, sec: 0
Startup, sec: 60
Hold Load: 180
Shutdown Time, sec: 60
Ultimate Thread Group
Here all the 1200 threads will be generated and up within 60 sec of Startup time(Ramp up time) .Those 1200 threads will be active for next 180 sec, and after that, those threads will shut down within next 60 sec.
So overall duration of the test is 300 sec as you desire. Throughout the Hold Load time(180sec) the requests will be looped.
This ultimate thread group will give you more control over Threads.
Now if you want to control the rate of transactions per second or the number of samples to execute you can use Constant Throughput Timer. Constant Throughput Timer will let you control the flow of requests to be sent. The default "Thread Group" only control the way the users or threads will be generated.It doesn't control the rate of requests to be hit your application.So for this purpose, you could use "Constant Throughput Timer" to control your requests per second. For further detail, you could go through JMeter documentation.
I am new to Jmeter. Here is one thing that is puzzling me
There are two scenarios that I am testing to calculate performance numbers using Jmeter
Having a 1 thread group set with loop count set to 50 and having one https sampler.
Having a 1 thread group with loop count as only 1 but instead using a loop controller having loop count set to 50. In this case my https sampler is inside the loop count.
In both cases the https sampler is run 50 times but I am noticing a lot of performance difference.
For 1 - 100 ms ( Average of 50 calls)
For 2 - 30 ms ( Average of 50 calls)
Question is why am I seeing this difference. Ideally both scenario should give the same performance number.
Also to add in my Jmeter setting I have "Use keep alive" as off and also https.use.cached.ssl.context is set to false in jmeter.properties.
What is your Ramp-up setting for scenario where 50 loops are defined on Thread Group level? As per documentation
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds
By default ramp-up is 1. If you're looking for the same behavior for Thread Group and Loop Controller you need to set it to 0.
Going forward, sending requests as fast as JMeter can is not something that could be called a real-life load scenario. Real users aren't hammering this or that endpoint, they need some time to "think" before doing next action. This "think time" can be simulated using i.e. Constant Timer. However the smarter way to set the desired load is using Constant Throughput Timer