I have a test suite which has many HTTP requests. Each HTTP requests have different number of threads but with 30 seconds as Ramp up time.
Set 1:
Set 2:
The difference between Set 1 and Set 2 are only in the number of Threads. Its exactly the double number of requests in Set 2. But you can see the total count is reduced. Why is this? i was expecting the number of requests also to go up when the number of Threads are increased.
Can someone please put some light into this?
Your tables don't tell the full story and the could be multiple explanations, for example:
You increase number of threads by factor of 2
Your application becomes overloaded hence response time increases
So assuming the same test duration JMeter is able to execute less requests as it waits for response from previous request prior to sending a new one
So pay attention not only to number of requests but also check response time for all the samplers and correlation between increased amount of active users and response time by looking into i.e. Response Times vs Threads and Transaction Throughput vs Threads charts.
Aforementioned graphs can be installed using JMeter Plugins Manager
Related
I am conducting a performance test (TPS) using jmeter.
I am requesting about 10,000 TPS, but the following two results are different.
(Position that 10,000 TPS responds normally)
1000 thread x 600 target throughput(in samples per minute)
100 thread x 6000 target throughput(in samples per minute)
I think the two results should be the same, but why is the response time delayed as the thread increases?
I think the two results should be the same - why they would be the same?
Let's imagine your system has fixed response time of 1 second, in that case:
With 1000 threads you will get 1000 requests per second and you can limit the throughput to 10 requests per second using the Constant Throughput Timer
With 100 threads you will get 100 requests per second, no limiting is required
And what if response time is 2 seconds?
With 1000 threads you will get 500 requests per second
With 100 threads you will get 50 requests per second
Constant Throughput Timer:
acts precise enough on "minute" scale, if your test lasts less than minute it might not apply the throughput
can only pause the threads to limit the throughput (requests per minute) to the desired value. If current number of threads is not enough in order to conduct the required load - the time won't have any effect.
If you want to send requests at the rate of 10000 TPS it worth considering going for the Throughput Shaping Timer and Concurrency Thread Group combination connected via the Feedback Function in this case JMeter will be able to kick off extra threads if current number is not sufficient.
But also be informed that:
JMeter should be able to start as many threads as needed to send 10000 TPS so make sure to follow JMeter Best Practices or even consider going for Distributed Testing Mode
Application needs to be able to handle the load and respond fast enough, JMeter waits for the previous response before starting the new request so if application is able to serve i.e. 5000 requests per second only you won't be able to reach 10000 by any means
I would like to load test https://app-staging.servespark.com site. I have completed scripts on Jmeter for login and am able to go to any page.
How can find out the max number of concurrent requests per second that the server could handle in Jmeter?
Is it possible in the Jmeter? Please advise.
It looks like you need to conduct a Stress Test, something like:
Start with 1 user
Gradually increase the load at the same time looking into the following charts:
Active Threads Over Time
Response Times Over Time
Transactions Per Second
At the beginning the response time should not change and the throughput (number of transactions per second) should increase by the same factor as the number of users increase
At certain stage of test you will notice that response time will start growing and the number of transactions per second will go down. This will indicate a bottleneck
You may continue increasing the load to see at which stage the errors will start occurring
And finally you can decrease the load gradually as well to see if the application gets back to normal when the load comes down (i.e. errors disappear, throughput grows, etc.)
You could try this;
https://jmeter-plugins.org/wiki/ConcurrencyThreadGroup/
And ramp up the users to a value higher than expected.
I am trying to stress test my server.
To do so I am using Jmeter and here is my set up:
I use
my Setup
Thread: 1000
schedule for 3 mins
So as you see I keep going with 1000 thread for a period of 3 mins.
But when I look at the throughput I only get around 230 per second
results
So what should I do to increase the through put to for example 1000000 per second? How come increasing the thread which I assume means more load does not increase throughput?
According to JMeter Glossary
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
Throughput explicitly relies on the application response time. Looking into your results, the average response time is 3.5 seconds therefore you will not get more than 1000 / 3.5 = 285 requests per second
Theoretically you could use Throughput Shaping Timer and Concurrency Thread Group combination, this way JMeter will kick off extra threads if the current amount is not enough to reach/maintain the desired throughput, however looking into 8.5% error rate and maximum response time for your application > 2 minutes my expectation is that you will not be able to get more throughput because most probably your application is overloaded and cannot respond faster.
Throughput measures the number of transactions or requests that can be made in a given period of time. basically, it lists the number of requests server managed to serve in a given time period. Throughput value depends on lot of factors and maybe your application under test not able to cater the expected load.
So with 1000 threads, you can't expect a 1000 throughput.
It's up to you to find out how much throughput your application can handle. For that maybe you need to do different optimizations on your side like optimize your script, distribute load via JMeter execution, increase theard count,...etc
I'm trying to understand a significant performance increase in my Jmeter test.
In a multi-tenancy database environment, I have a single RESTful service test containing a Thread Group with a single HTTP Request sampler posting an XML payload. The XML payload is then evaluated via stored procedures, and a response is received stating if the claim was qualified. I run this test from a .bat file (non-gui mode) in an Apache 7 environment with a single JVM running.
Test Thread Group Properties
# of Threads: ${__P(test.threads,200)}
Ramp-Up Period: ${__P(test.rampup,1)}
Loop Count: Forever
Delay Thread: Enabled
Scheduler: Enabled
Duration: ${__P(test.duration,1800)}
HTTP Request
Method: POST
https://serverName:port/database/.../${__P(tenant,1111)}/Claim/${__property(contractId)}
When I duplicate the HTTP Request sampler within the TG and change the tenant ID within the URL, for some reason the performance seems to increase by > 55%. (i.e., the # of claims/second is increased by 55%) It appears the test did not fail, so I cannot attribute the performance increase to an increased error rate.
I would have expected an increase if I had enabled another JVM to let the Load Balancer perform optimization, but this is not the case. (still using only 1 JVM)
HTTP Request 1
https://serverName:port/database/.../${__P(tenant,1111)}/Claim/${__property(contractId)}
HTTP Request 2
https://serverName:port/database/.../${__P(tenant,2222)}/Claim/${__property(contractId)}
The theory going around here is that Jmeter generates a workload at a higher rate for multiple requests than for a single request. I'm skeptical, but haven't found anything "solid" to support my skepticism.
Is this theory true? If so, why would two HTTP Requests increase the performance?
In short: it's OK.
Longer version:
Here is how JMeter works:
JMeter starts all the threads during ramp-up period
Each thread starts executing samplers upside down (or according to the Logic Controllers)
When request doesn't have more samplers to execute and no more loops to iterate it's being shut down.
So how does number of virtual users correlate with the "performance". When you increase virtual users number of requests number for a load test it affects Throughput
Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server.
The formula is: Throughput = (number of requests) / (total time).
So if you increase load on well-behaved system throughput should increase by the same factor or linearly.
When you increase load but throughput does not increase, such situation is known as "saturation point" when you get the maximum performance from the system. Further load increasing will lead to throughput going down.
References:
Apache JMeter Glossary
An extended Glossary version
And how do you messure your performance? According to your "theory" your messurements includes jmeter overhead and this would be wrong. More over, is the response the same for both cases? What I mean, is the backend doing the same work work both cases?
Maybe first request returns different output then the other one. Maybe it is more expensive to generate output in one of the request. That is why you will notice "incresed" performance as normally you would do N x Heavy task in X seconds, and in second case G x heavy tasks + H X light tasks in the same time where G < N/2 - more requests in the same time? Sure! Incresed perfomance? Nope.
So to completly investigate what is happening, you need to review your measurement method. I would start with comparing the the actual time for both requests.
I am doing API load testing by sending 250 requests at once.
1. Configuration
Naturally, server takes longer to respond when a lot of users requests it simultaneously, this is what it says here.. As per http://jmeter-plugins.org/wiki/ResponseTimesVsThreads/. However when testing this is what I found..
2. Test
The plot above starts from right to left and as the number of active threads decrease, the response time increases.
Is active threads same as number of user requests, if so why this is happening on a consistent basis?
Update-1
Ran another test and increased the ramp-up period this time
No of threads: 200
Ramp-Up Period: 200 secs
Loop Count: 200
There are at least 2 possible explanations:
you don't have a problem, and your improvement in response times comes from caching effect related to your data being in cache after some time. Only you can validate as we don't know if you are using a large enough dataset and how long is your test lasting
you have a problem, your server is rejecting connections under load, so you have very rapid failed responses that have a very good response time. To know if it's your problem, check the response code over time or transactions over time as long as error percentage