I created a simple test of calling an HTTP endpoint, that is configured to spam 1000 requests at once as it is in this picture:
The test took 75 seconds to run which means the throughput should be 1000/75 = 13.3 requests per second however, the Summary Report says it is 4.8 requests per second.
Why?
which means the throughput should be 1000/75 = 13.3 requests
According to the glossary:
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So it should be something like: 4150 / 75 = 55.3, however given you have 1000 threads and no loops you should have 1000 results only so try "clearing" the results and re-running:
Also consider running your test running JMeter in command-line non-GUI mode as GUI mode is supposed to be used for tests development and debugging, when it comes to test execution you should not be using JMeter GUI
Related
I am currently using Jmeter for API Performance Testing, but recently I started to look into Gatling as a potential replacement of Jmeter. Below is the PoC I'm doing for Gatling but I notice the performance result is very different.
Setup:
where we hit a https endpoint with a concurrent user of 10 for 60 seconds.
Results
Jmeter: 10 threads (no ramp up), 60 seconds
Result: 150 TPS
Gatling: 10 concurrent users, also 60 seconds
Result: 27 TPS(cnt/s?)
Question:
first I want to confirm the terminology of Gatling; in Gatling result chart, I see a column named "mean cnt/s" I hovered over it and it says "count of event per second", I imagine that's the same thing as Jmeter's TPS?
Jmeter:
summary + 2386 in 00:00:16 = 153.1/s Avg
Gatling:
Mean cnt/s: 26.652
if above assumption is correct, can someone share some insight on why Gatling's number is much lower than Jmeter's?
Thank You!
Gatling: 10 concurrent users, also 60 seconds
Do you understand what this does?
This is going to spawn a new user every time an existing one finishes, and hence create new connections. Assuming it takes 100ms for a virtual user to complete the scenario, you're going to spawn 101060 = 6,000 virtual users and as many connections.
Is that really what you want and is it the same thing as you do with JMeter?
If you actually want the same 10 users to loop for 60 seconds, you have to inject atOnceUsers(10) and add a during(60) loop in your scenario.
https://gatling.io/docs/gatling/reference/current/core/injection/#open-model
https://gatling.io/docs/gatling/reference/current/core/scenario/#loop-statements
Many things can cause deviations.
I assume you use the same setup for both in terms of load generator/target insance. You can start with fixed number of requests first.
Use loops in Jmeter and repeat in Gatling.
Sending for example 60 x 10 = 600 requests in total.
Gatling will be able to generate much higher load than Jmeter if properly used.
I want to put the load of 20000 user requests on my server to test can it handle this number of requests in a second.
I have put the Number of threads(user) - 20000
Ramp-up period (seconds) - 1
Loop count - 1
Output - It is showing 20 seconds Avg time after completing the script.
Interpreting the output:
It is stating that the average server response time is 20 sec.
To check whether your server can handle 20000 users per sec,
run the test plan from non-GUI mode (Generating HTML dashboard report). Because 20000 is a huge number.
Go get the best of the non-GUI mode add the JMeter plugin manager. Add the required reports needed by you.
If I am not wrong what you are trying to achieve is a throughput of 20000 per sec. The same can be found using transactions per sec graph from the HTML dashboard report generated in non-GUI mode.
This plugin might help your use case: Throughput Shaping Timer
To achieve the desired 20000 users per sec you need to experiment with ramp-up and loop counts based on the output. The end goal should be that your transactions per sec graph should show 20000 or more at one point in time and at the same time the error should be zero then your server is able to handle 20000 users per sec. But average server response time is also an important metric to watch out for. Because no user wants a slow response from the server.
You can also consider the Concurrency thread group. This thread group is specifically designed to generate the desired number of concurrent users.
If you tried to do a Spike Test, like access application with 20 000 users for 1 second - your configuration is fine and and application failed the test so you can raise an issue.
However I'm under impression that your test is kind of short and doesn't tell the full story
20 000 users will produce 20 000 requests per second only if your application response time is 1 second sharp. If your application response time is 0.5 seconds - the load will be 40 000 requests per second, if response time is 2 seconds - the load will be 10 000 requests per second, if response time is 20 seconds - the load will be 1000 requests per second, etc. See What is the Relationship Between Users and Hits Per Second? article for more details if needed.
Normally you should increase the load gradually, this way you will have possibility to correlate increasing load with other metrics (response time, throughput, number of errors, etc) so I would recommend going for ramp-up -> plateau -> ramp-down setting and check the number of requests per second during this "plateau" phase
20000 virtual users is quite a number so make sure to follow JMeter Best Practices and it might be the case you will have to go for Distributed Testing
Good day.
I am new to J-Meter and here is the requirement I need to verify through J-meter TP.
I am using Jmeter: 5.1 version.
I need to get the results for 500 throughput per seconds.
Number of target users : 100
Time to run the test 1 hr.
Start the test with 5 users and increase the load by setting every 3 seconds 5 more users up to add till we reach reaching 100 users.
Once we reach 100 users >> Keep the load for 10 seconds hold and decrease the load by every 3 seconds to reduce 5 number of usrs.
Please help me on what are the parameters I have to set in J-Meter test plan.
Thanks.
Siva
The easiest way of controlling the number of requests per second (throughput) in JMeter test is using Constant Throughput Timer
Add Constant Throughput Timer to your Test plan and configure it to send 30000 requests per minute (500 requests per second)
Make sure that all the Samplers are in the Constant Throughput Timer's scope and you have all active threads selected in the dropdown
You can only achieve 500 requests per second with 100 users if your application response time is 200 ms or less, if it's more - you will have to increase the number of threads accordingly.
And last but not the least, according to JMeter Best Practices you should always be using the latest version of JMeter (5.3 as of now) so consider upgrading at next available opportunity
Below are some suggestions that might help you to start with:
You might need to calculate the pacing required to achieve target total transactions:
Pacing = 3600(Test duration time in seconds) * ( target no.of user) / (no.of total transactions to achieve).
To setup ramp up, ramp down more effectively you will need one of the plugin available in Jmeter Plugins Manager (Plugin name : Stepping Thread Group by blazemeter) which comes with a preview.
How to setup Plugins manager (https://jmeter-plugins.org/wiki/PluginsManager/)
I am using JMeter to load test my application. This is how my thread group configuration looks like:
I am expecting to hit the application with 200 threads (requests) and keep up that load for 30 minutes.
I am using 5 throughput controllers which use percentage executions and each HTTP request (under each controller) uses a specific percentage (totals to a 100).
I use the non-GUI mode to start the test and output the results to a csv. When I plot this though (using Kibana), I see a graph which looks like this
I see a peak and drop every 5 minutes. I am expecting to see uniform load across the 30 minutes I am running my test. Any idea why this is happening ? Is there something wrong with my configuration ?
Looking into "Loop Count" of 1 in your Thread Group I don't think your tests lasts 30 minutes, my expectation is that it takes seconds (or whatever time is needed to execute all your Samplers with 200 threads)
So tick Forever box or set "Loop Count" to -1 and re-run your test.
More information: JMeter Test Results: Why the Actual Users Number is Lower than Expected
I have created a set and just confused with the Loop Count and Ramp Up period.
I have a test set with the following parameters.
Threads = 30
Ramp Up Period = 30
Loop Count = 100
As per the page on
quora.
I suppose:
a) If Loop count is Zero, then each of the 30 threads will be starting every second. As per the shared the web page, I guess 30/30 * 100 ie 100 threads/requests will be hitting the server every second. Please correct me if I am wrong.
b) As per the above parameters, there will be a total of 30 * 100 threads/requests. Does this mean all the 3000 threads/requests will be sent within 30 seconds [ Ramp Up period ]
Assuming you have 30 users and 30 seconds ramp-up
JMeter will start each virtual user each second
Each virtual user will start executing samplers upside down (or according to the logic controllers, if any) as fast as it can (if you don't use timers) so the delivered load can be either more or less than 30 requests/second, it depends on how fast JMeter is executing requests and on your application response time as JMeter will wait for response from previous sampler before starting new one
When virtual user finishes executing all the samplers defined in test plan it will start over and do point 2 for 99 more iterations
When virtual user won't have more samplers to execute and loops to iterate it will shut down
A couple of tips:
You can use Server Hits Per Second listener to see how many requests per second you are actually making given your test plan configuration
You can control the number of requests per second via Constant Throughput Timer
Consider upgrading to JMeter 3.2 as newer JMeter versions normally contain new features, performance improvements and bug fixes
according to jmeter manual ramp up is:
How long JMeter should take to get all the threads started.
If there are 10 threads and a ramp-up time of 100 seconds, then each
thread will begin 10 seconds after the previous thread started, for a
total time of 100 seconds to get the test fully up to speed.
So if your goal is to reach 3000 request within 30 seconds the above wont do, it might take more than that depending on how much it takes to finish the requests you are sending.
If you are looking for Throughput you can add an aggregate report listener which calculates the throughput for you and depending on the results you can configure your thread properties to reach your goal.
Reference :
Jmeter user manual