I have a problem with my load test on JMeter. I set:
Number of Threads = 10
Ramp-Up-Period = 1
Loop Count = 1
So normally the Test goes up to the 10 Threads (10/10) (You can find this in the right upper corner!) and then goes back to zero. My Problem is, that my JMeter Test just goes up to (4/10) and then back to zero. I have no idea why it doesn't work normally.
I tried this with another Number of Threads, e.g. with 20. And the problem also occurs in that example. The test goes up to (7/20) and then back to zero. But it should go up to (20/20).
Your test is setup for loopcount=1 and your threads start up gradually. Do you think your first thread will still be active by the time the last thread starts up?
For your particular test, apparently there are at-most 4 active threads at some point of the test execution. If your test runs for a very short time, if the first thread exits before second thread starts, you wont even get that, the max you'll see is 1/10
If you need all threads to be active at the same time, either
Increase the length of your transaction beyond ramp up time
Have a larger loop count so the threads are active longer
Reduce or eliminate your ramp-up time so all threads become active simultaneously (Though this is usually a bad idea)
Related
How to set up the stepping thread group? If my application gives average response time as 2sec for 100VU by using "ThreadGroup".
Actually, it depends of your performance test goals. Stepping Thread Group won't allow parameters smaller than 1 second. You have to deal with this limitation.
According to JMeter documentation:
Ramp-up needs to be long enough to avoid too large a work-load at the
start of a test, and short enough that the last threads start running
before the first ones finish (unless one wants that to happen).
Defining the ramp-up time is a very important step of your performance test. In your case, I recommend start with 1 thread per second using these parameters:
This group will start 100 threads;
First, wait for 0 seconds;
Then start 1 threads;
Next, add 1 threads every 1 seconds using ramp-up 0 seconds,
Then hold the load for 900 seconds.
You can choose to stop all threads at once then. It is up to you.
Why am I suggesting to run a test for almost 20 minutes? Because you are interested in the performance running with 100 threads and you want to maximize the number of samples with such load level. On the suggested setup, you'll have approximately 90% of your test time running with the ideal number of threads.
Once you have those numbers, you can experiment by starting more than 1 thread per second and decrease the overall ramp-up time. Always look at the resource usage (e.g. CPU utilization, memory available, etc.) to understand the system limits.
I am not able to find out the specific answer of how to calculate the number of threads for running a load test in JMeter?
How to identify the loop count ?
Is there any formula?
What are the parameters to consider for the calculation?
Say if you want to fire 100 request to server with 2 tps. then your threas properties should be like below:
Number of threads(users) :2.
Ramp up period: 100
Loop count :50
Based on above example.Please find below explaination.
• Number of Threads (N): Sets the number of threads the JMeter will use to execute our test plan. We must know that each thread will execute the whole test plan, which effectively utilizes the number of users that could use the tested service at any given time simultaneously.
• Ramp-Up Period R: Specifies how much time (in seconds) it will take for JMeter to start all the threads (simultaneous user connections). If the number of users is 5 and the ramp-up time is 10 seconds, then each thread will be started in a 2 second delayed interval. We need to be careful when setting this value, because if the value is too high the first thread will already finish processing the whole test plan before the second thread will even begin. This is important because that would effectively reduce the number of concurrent users using the testing server application at any given time. But the ramp-up period also needs to be high enough to avoid starting all of the thread at a single time, which could overload the target application.
• Loop Count (L): How many times each thread group will loop through all configured elements belonging to that thread group.
Hope it helps!
This is the first time I had to perform tests and I'm kind of confused about how to interpret the results I got when setting "Ramp-up Period" greater than zero.
To start, when I set a value of zero, if I'm not mistaken, this means that all the threads are launched at the same time. Therefore, one can conclude that the max number of threads that your application can successfully accept is equivalent to the max number of concurrent requests the tested application can accept.
Now that problem comes when I have to do some tests using a "Ramp-up Period" greater that zero, as I'm not sure what conclusions can I come to, or rather I don't know whether they are correct.
Based on what the documentation says:
Start with Ramp-up = number of threads and adjust up or down as
needed.
This is more or less what I've written:
1) To start our test we set that each thread will start 1 (15/15) second after the previous thread was begun. To do this we set the same value for both “Number of threads” and “Ramp-up Period”.
As the 15 requests are done successfully we start adjusting down the value for "Ramp-up Period", until we find the minimum interval of time that must exist between each request
2) Ramp-up Period = 13
3) Ramp-up Period = 10
4) Ramp-up Period = 7
5) Ramp-up Period = 6
So far all the requests were performed successfully, but with a "Ramp-up Period" of 6 only three requests had a response.
With this result we can conclude that the minimum time that must elapse between each call to the web service must be greater or equal to 0.46 seconds
This value is calculated by dividing the value of "Number of threads" and "Ramp-up (7 / 15)
What do you think about my conclusion? Is it right? What else one can conclude using jmeter in the way I showed.
As always your comments suggestions would be greatly appreciated.
Thanks
Ramp up is nothing more than virtual users arrival rate. JMeter starts from 1 thread and adds more threads as per ramp-up specified. 15 threads and 15 seconds ramp-up means that each 1 second 1 thread is being added. In case of 7 seconds ramp-up and 15 threads each second 2 threads are added. When thread doesn't have any work to do it's being shut down so make sure that you have enough loops in the parent Thread Group or Loop Controller.
There are at least 3 options of defining load scenario:
Ultimate Thread Group (free 3rd-party plugin) - provides convenient way of setting ramp-up, ramp-down and hold the load times.
Constant Throughput Timer - provides capability to set the load in "requests per minute"
Synchronizing Timer - provides the way of pausing requests until specified thread number is reached and firing them off at the same moment creating simultaneous concurrent requests.
Also:
Please don't use JMeter GUI for anything apart from tests development and debugging
Avoid View Results in Table Listener
The same for View Results Tree listener.
Follow other advices from JMeter Performance and Tuning Tips guide.
I've noticed that when load testing with JMeter, if I do a single loop I get a fairly long average time for my test. If I have say a Loop Count of 10, my average time peaks early on and then drops way down. For example if I setup a test on a simple get request for a page with the following settings:
Number of Threads (users): 500
Ramp-up Period(in seconds): 5
Loop Count: 1
My average time is about 4 seconds. If I change it to 10 loops:
Number of Threads (users): 500
Ramp-up Period(in seconds): 5
Loop Count: 10
I get an average time of 1.4 seconds.
Apache's documentation states that the Loop Count is:
The number of times the subelements of this controller will be
iterated each time through a test run.
Is it possible that this means the first request will actually do something on the server and the subsequent 9 requests will be pulling from cache?
How exactly is the Loop Count being used that would cause the results I'm seeing?
Yes, Remaining 9 requests must be pulling from cache.
Loop controller is simple loop executor doing nothing magic inside.
Improved performance is because of use of cached results on server.
If you want one thing you can try, use the loop controller but use different substituted parameters so that every
time different requests will be sent to server (I know that loop controller is for repeating same values, but this is just to confirm the effect of caching).
then compare the results.
I hope this clears the doubt :)
I need a little help on how to debug the matter. My current jMeter scenario seems to run fine as long as I keep the loop count at 1, when I add more loops the performance starts to degrade a lot.
I have a thread group with 225 threads, 110s ramp up, loop count 1 - my total response time is ca. 8-9secs. I run this several times to confirm, each run shows similar response times.
Now, I did the same test , just changed the loop count to 3, all other parameters unchanged, and the performance went south, total response time is ca. 30-40s.
I was under the impression that 3x 1 loop runs would be, more or less, equivalent to 1x 3 loops run. It seems that is not the matter. Anyone could explain to me why is that?
Or, if this should be equivalent, any idea where to search for the culprit of degrading performance?
What you're saying is that the response times degrade if you increase the throughput (as in requests per second).
Based on 225 threads making a single request with a rampup of 110 seconds your throughput is going to be in the region of 2 requests every second. Increasing the loop count to 3 is going to up that by around a factor of 3 to 6 requests a second (assuming no timers). Except of course if the response times are increasing then you will not reach this level of throughput which is you problem.
Given that this request is already taking 8-9 seconds, which is not especially fast, it could be assumed that there is some heavy thinking going on behind the scenes and that you have simply hit a bottleneck, somewhere...
Try using less threads and a longer rampup and then monitor the response times and the throughput rate. At some point, as the load increases, you will see response times start to degrade and then at this point you need to roll up your sleeves and have a look at what is happening in your AUT.
Note. 3 x 1 loop is not the same as 1 x 3 loops. The delay between iterations will cause one thread with multiple iterations to have a different throughput vs. more threads with one iteration where the throughput is decided by the rampup, not the delay. That said, this is not what you describe in your question - you mention that the number of threads is consistent.
In addition to the answer from Oliver: try to use custom listener like Active Threads Over Time Listener - to monitor your load-scenario.
You can also retry both your scenarios described above, with this listener - sure, you'll see the difference in graphs.