When I'm doing Jmeter benchmark, the throughput and transfer bytes/sec keep decreasing as time goes by? Who can tell me why?
When the test begins, it has a high throughput , but it decreases all time, and the stable status can't be accessed.
JMeter kicks off threads representing virtual users according to your ramp-up. Each thread starts executing samplers upside down (or according to parent Logic Controllers). When thread doesn't have more samplers to execute and no more loops to iterate it's being shut down.
You can consider the following approaches to implement desired load scenario:
Provide enough iterations via Thread Group or Loop Contoller
Use Ultimate Thread Group which provides easy way of defining a load scenario so you will be able to set initial users number, ramp up duration and users arrival rate, time to hold the load, and define ramp-down.
JMeter provides Constant Throughput Timer which can be used to set precise load in "requests per minute" so you'll be able to control target throughput. You can also use it in conjunction with Beanshell Server - this way you can dynamically change throughput on-the-fly.
Related
How can I acheive below transaction per hour. I tried to control login by using once only controller ,But transaction per hour is still more than 70. How to handle this?
Overall User count is 70
Transaction per hour for login -- 70
Transaction per hour for homepage -- 100
If you need JMeter to execute exact amount of transactions, not more, not less - go for Throughput Controller
Similarly you can configure 100 homepage transactions.
In order to evenly distribute 70/100 transactions for the duration of one hour time frame you can play with ramp-up period and Constant Throughput Timer
Be aware that you won't be able to achieve different throughputs for different samplers under the same Thread Group as JMeter will always wait for the previous sampler to finish prior to executing the next one hence it will act at the speed of the slowest sampler.
More information: Running JMeter Samplers with Defined Percentage Probability
A couple of other options:
Use the Arrivals Thread Group. This TG will allow you to
configure the desire average throughput (ATP); the TG will
instantiate the required threads needed to achieve the ATP goal (no
guessing)
Use the Concurrency Thread Group together with Throughput
Shaping Timer. These option will also autoscale the number of Vusers
see more information here
Be aware that the downside of these options is that vusers is are instantiate using a fixed pacing. In general, this is not how rusers interface with an application in the real world.
I have created a performance test script as below. I am running 4 Thread Groups in parallel (Even though there are 14 Thread Groups, only 4 are enabled and I am running only those enabled 4 Thread Groups). I have used default Thread Groups.
I have used Flow Control Action to simulate the user think time and set it as 3 seconds.
My requirement is to achieve Throughput as 6.6/seconds. What is the best way to achieve it? Also, does user think time creates any impact on Throughput?
As per JMeter Glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So basically throughput is the number of requests which JMeter was able to within test duration time frame. If you introduce artificial delay of 3 seconds - it will make the overall throughput lower
Other (and the main factor) is your application response time because JMeter waits for previous request to finish before executing new one so there could be 2 options:
The amount of threads in your Thread Groups is not sufficient to create the desired throughput. If this is the case - just add more threads (virtual users) in Thread Group(s)
The amount of threads in your Thread Groups is too high and you're getting higher throughput that you expect. If this is the case you can pause JMeter even more by adding Constant Throughput Timer and specifying the desired number of the requests per minute you want to have. If you need 6.6 requests per second it will mean 396 requests per minute
Also remember that JMeter itself must be able to send requests fast enough so make sure to follow JMeter Best Practices
My suggestion is to consider using the Arrivals Thread Group. This TG will allow you to configure the desire average throughput (ATP); the TG will instantiate the required threads needed to achieve the ATP goal.
As for the Think Time (TT), it should be a business requirement that emulates the pauses that users take when using the application in the real world.
I want to understand the actual JMeter throughput behavior achieved at runtime.
Scenario - I'm increasing the JMeter throughput at runtime using Constant Throughput Timer and beanshell script as described here - https://www.blazemeter.com/blog/how-to-change-jmeters-load-during-runtime.
Test plan - Along with CTT as described above, simple threadgroup with fixed #threads and infinite loop iterations is configured. HTTP Sampler for GET call is used. No other timer or plugin is added in the test plan.
As I keep on increasing the target throughput of JMeter at runtime, I noted that the actual achieved throughput value is limited by mainly 2 factors -
The threads in my thread group.
The performance bottleneck of target app.
I have questions regarding both the limitations -
Once the highest throughput is achieved using all threads in current threadgroup (assuming there're no errors from the target app yet), is there a way to increase the #threads at this point dynamically at runtime to achieve a higher JMeter throughput?
Now as I keep on increasing JMeter throughput, it can't be increased further due to errors from the target app. How does the JMeter identify performance bottleneck of my target app and react to it? Does it add any delay or kill threads or apply any such mechanism to reduce it's throughput to the max that the target app can sustain?
In continuation with point #2, if JMeter identifies and reacts to the performance bottleneck by any method, what are the factors (like error rate, response latency etc.) that control it's throughput to keep it within the max limit of the target app? Are these factors configurable or extensible?
You can play the same trick with the number of threads in the thread group, just define it using __P() function and you will be able to manipulate it using Beanshell server. Another option is using a JSR223 Test Element and Groovy language to add the new thread(s) where/when required like:
ctx.getThreadGroup().addNewThread(0, ctx.getEngine())
JMeter doesn't "identify" anything, it just tries to execute Samplers as fast as it can and the number of requests per second depends on 2 factors:
your application needs to be able to respond fast enough
JMeter itself needs to be able to send requests fast enough, i.e. you need to follow JMeter Best Practices
there are no "mechanisms" which detect the application under test behaviour, the closest solution is Auto Stop Listener
See point 2
Our requirement is to execute
1 user/thread, 10 requests/sec, Total requests = 10,000
With current Thread Properties not able to achieve.
Only one user/thread send 10 request/sec and total should be 10000 requests. Any other way to achieve this in Jmeter ?
Is following approach is correct ? We used Loop controller. So each request will repeat 10 times
You will be able to achieve 10 requests/second with 1 virtual user only if your response time will be 100 milliseconds. If response time will be more than 100 milliseconds - you will not be able to reach the desired load.
If your application fails to respond in 100 milliseconds time frame - most probably you've found a performance bottleneck and you can report this to your application developers.
If you have some time to investigate the issue you can try to provide more information, i.e. what is the average response time, what are minimum, maximum, quantiles, the actual number of requests per second, etc, all this information can be retrieved using Aggregate Report listener.
Normally when people are looking for the answer if the application under test can support X requests per second they use > 1 virtual user as load test should represent real life usage of the system under test and 1 thread (virtual user) isn't normally associated with performance testing and derivatives.
So probably you should reconsider your approach to testing and try increasing the number of threads (virtual users). The throughput can be controlled using Precise Throughput Timer or Constant Throughput Timer. However be aware that the above timers can only pause JMeter to slow it down to the desired throughput. Another approach is using Concurrency Thread Group and Throughput Shaping Timer combination, they can be connected using Feedback Function so JMeter will kick off extra threads in order to reach/maintain the desired number of requests per second.
You can install Concurrency Thread Group and Throughput Shaping Timer using JMeter Plugins Manager
how does the delay between requests is computed. I have simple Jmeter script with following 4 components :
thread group - number of threads set to 10, reset other fields have default values.
Http Sampler - named as Thread - ${__threadNum} Req -${__counter(TRUE)}
hitting URL- google.co.in
Constant throughput Timer : Target throughput- 60, Calculate Throughput on -All Active threads.
View result in table listener.
Summarized test results are below :
Please explain how the delay between the requests and number of requests to be created are computed . Please Explain other field values of " Calculate Throughput on ".
Also If i have ultimate Thread group instead of thread group, Have hold for load set to some value, so in that case how the number of requests to be sent during hold load time period are calculated?
Constant Throughput Timer pauses active threads to reach throughput level you define.
Remember 2 things:
Throughput Timer isn't able to create threads, it can only slow them down so make sure that you provide enough threads to reach desired target throughput
Throughput Timer is quite accurate on "minute" level, you need to "wait" for it to start working as expected for 60 seconds.
See How to use JMeter's Throughput Constant Timer for more details.
In some cases for some people it's more convenient to use Throughput Shaping Timer which comes with JMeter Plugins, you can configure target throughput providing multiple steps and seeing the impact on a graph.
From JMeter Cookbook by Bayo Erinle, The goal of the Constant Throughput Timer component is to get your test plan samples as
close as possible to a specified desired throughput. It achieves this by introducing variable
pauses to the test plan in such a manner that will keep numbers as close as possible to
the desired throughput. That said, throughput will be lowered if the server resources of the
system under test can't handle the load. Also, other elements (for example, other timers,
the number of specified threads, and so on) within the test plan can affect attaining the
desired throughput.