I've noticed that when load testing with JMeter, if I do a single loop I get a fairly long average time for my test. If I have say a Loop Count of 10, my average time peaks early on and then drops way down. For example if I setup a test on a simple get request for a page with the following settings:
Number of Threads (users): 500
Ramp-up Period(in seconds): 5
Loop Count: 1
My average time is about 4 seconds. If I change it to 10 loops:
Number of Threads (users): 500
Ramp-up Period(in seconds): 5
Loop Count: 10
I get an average time of 1.4 seconds.
Apache's documentation states that the Loop Count is:
The number of times the subelements of this controller will be
iterated each time through a test run.
Is it possible that this means the first request will actually do something on the server and the subsequent 9 requests will be pulling from cache?
How exactly is the Loop Count being used that would cause the results I'm seeing?
Yes, Remaining 9 requests must be pulling from cache.
Loop controller is simple loop executor doing nothing magic inside.
Improved performance is because of use of cached results on server.
If you want one thing you can try, use the loop controller but use different substituted parameters so that every
time different requests will be sent to server (I know that loop controller is for repeating same values, but this is just to confirm the effect of caching).
then compare the results.
I hope this clears the doubt :)
Related
I have written a Jmeter script with 10 threads and 100 loops. So basically, it's 1000 samples. If you look at the Transactions per second PNG, it's approximately 1 transaction in 1 second.
Looking at the Bytes per Second, it's a big figure. Why is that?
What's the relation between these two images? I am bit confused here. Can someone put some light here?
You have 1000 samples in 15 minutes, 15 minutes == 900 seconds so it is absolutely expected that throughput would be something like 1 transaction per second.
Bytes per second is basically the size of response data you are getting, looking into the graph it appears that you are receiving around 300 kilobytes for each request.
If you need to apply more load - increase number of virtual users (threads) and decrease ramp-up time, you should be seeing higher throughput (given your server can handle this load, otherwise it will be the bottleneck).
See JMeter Test Results: Why the Actual Users Number is Lower than Expected article to get more understanding on how does JMeter work in terms of applying the load.
I am not able to find out the specific answer of how to calculate the number of threads for running a load test in JMeter?
How to identify the loop count ?
Is there any formula?
What are the parameters to consider for the calculation?
Say if you want to fire 100 request to server with 2 tps. then your threas properties should be like below:
Number of threads(users) :2.
Ramp up period: 100
Loop count :50
Based on above example.Please find below explaination.
• Number of Threads (N): Sets the number of threads the JMeter will use to execute our test plan. We must know that each thread will execute the whole test plan, which effectively utilizes the number of users that could use the tested service at any given time simultaneously.
• Ramp-Up Period R: Specifies how much time (in seconds) it will take for JMeter to start all the threads (simultaneous user connections). If the number of users is 5 and the ramp-up time is 10 seconds, then each thread will be started in a 2 second delayed interval. We need to be careful when setting this value, because if the value is too high the first thread will already finish processing the whole test plan before the second thread will even begin. This is important because that would effectively reduce the number of concurrent users using the testing server application at any given time. But the ramp-up period also needs to be high enough to avoid starting all of the thread at a single time, which could overload the target application.
• Loop Count (L): How many times each thread group will loop through all configured elements belonging to that thread group.
Hope it helps!
I am under the impression that I use the 3 settings; Threads, Ramp-up and Loop, to simulate X (Threads) amount of users/thread over Y (Ramp-up) seconds and do these for Z (Loop) amount of times. For example I want 10 users every 1 second for 1 hour which equates to 10 Threads, 1 Second Ramp-up and 3600 Loops.
But :)
Others seems to be using it differently ... as in if they want the same as above they would set Threads to 36000, Ramp-up to 3600 seconds and Loop to 1.
I tend to think the first approach is correct based on #a it reads better :) and #b why would you have a setting based in seconds to indicate the length of your test
Can anyone give me a definitive answer or are both options plausible?
Firstly 36000 threads in the second example seems very high! http://wiki.apache.org/jmeter/HowManyThreads reports people using 1000. So the 2nd scenario may not even work.
The two scenarios you describe are not exactly the same, and I'm not sure if either are exactly what you want.
In the first the 10 Threads plus 1 second Ramp-up means all 10 Threads will be in use after 1 second. The 10 Threads will then do their actions in parallel 3600 times. You have not mentioned anything that will mean the test will take 1 hour, it will take as long as it takes to loop through 3600 times. To make it take an hour (assuming the actions don't take longer than 1 second) you would need something like a Constant Throughput Timer within your loop which control the speed of the loop so that it takes an hour.
In the second, 10 threads would be created in second #1 and start doing their loop, another 10 in the second#2, etc, all the way up to 1 hour (second#3600). If the actions take longer than one second, then you would have more than 10 threads running in any one second.
The first approach is much clearer. The 2nd one is a misuse of the Ramp-Up; it's not being used to provide a ramp-up to 360000 threads, but to try and schedule 10 threads to run at once.
I am new to Jmeter. Here is one thing that is puzzling me
There are two scenarios that I am testing to calculate performance numbers using Jmeter
Having a 1 thread group set with loop count set to 50 and having one https sampler.
Having a 1 thread group with loop count as only 1 but instead using a loop controller having loop count set to 50. In this case my https sampler is inside the loop count.
In both cases the https sampler is run 50 times but I am noticing a lot of performance difference.
For 1 - 100 ms ( Average of 50 calls)
For 2 - 30 ms ( Average of 50 calls)
Question is why am I seeing this difference. Ideally both scenario should give the same performance number.
Also to add in my Jmeter setting I have "Use keep alive" as off and also https.use.cached.ssl.context is set to false in jmeter.properties.
What is your Ramp-up setting for scenario where 50 loops are defined on Thread Group level? As per documentation
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds
By default ramp-up is 1. If you're looking for the same behavior for Thread Group and Loop Controller you need to set it to 0.
Going forward, sending requests as fast as JMeter can is not something that could be called a real-life load scenario. Real users aren't hammering this or that endpoint, they need some time to "think" before doing next action. This "think time" can be simulated using i.e. Constant Timer. However the smarter way to set the desired load is using Constant Throughput Timer
I need a little help on how to debug the matter. My current jMeter scenario seems to run fine as long as I keep the loop count at 1, when I add more loops the performance starts to degrade a lot.
I have a thread group with 225 threads, 110s ramp up, loop count 1 - my total response time is ca. 8-9secs. I run this several times to confirm, each run shows similar response times.
Now, I did the same test , just changed the loop count to 3, all other parameters unchanged, and the performance went south, total response time is ca. 30-40s.
I was under the impression that 3x 1 loop runs would be, more or less, equivalent to 1x 3 loops run. It seems that is not the matter. Anyone could explain to me why is that?
Or, if this should be equivalent, any idea where to search for the culprit of degrading performance?
What you're saying is that the response times degrade if you increase the throughput (as in requests per second).
Based on 225 threads making a single request with a rampup of 110 seconds your throughput is going to be in the region of 2 requests every second. Increasing the loop count to 3 is going to up that by around a factor of 3 to 6 requests a second (assuming no timers). Except of course if the response times are increasing then you will not reach this level of throughput which is you problem.
Given that this request is already taking 8-9 seconds, which is not especially fast, it could be assumed that there is some heavy thinking going on behind the scenes and that you have simply hit a bottleneck, somewhere...
Try using less threads and a longer rampup and then monitor the response times and the throughput rate. At some point, as the load increases, you will see response times start to degrade and then at this point you need to roll up your sleeves and have a look at what is happening in your AUT.
Note. 3 x 1 loop is not the same as 1 x 3 loops. The delay between iterations will cause one thread with multiple iterations to have a different throughput vs. more threads with one iteration where the throughput is decided by the rampup, not the delay. That said, this is not what you describe in your question - you mention that the number of threads is consistent.
In addition to the answer from Oliver: try to use custom listener like Active Threads Over Time Listener - to monitor your load-scenario.
You can also retry both your scenarios described above, with this listener - sure, you'll see the difference in graphs.