I have a spring boot application which has a post endpoint which takes the request and send it to another service and get the response back and save it to mongo database and returned the response back to user. The application is deployed on embedded tomcat of spring boot. I am using jmeter to see the max response time, throughput etc.
When i ran a test from jmeter with 500 threads for 10 minutes, i got maximum time as around 3500ms.
When i repeat the test from jmeter the maximum time gets reduced to 900ms.
Again, if i run the test after a long time, the maximum again goes upto 3500ms.
I am not able to get any information regarding this behavior of tomcat.
Could you please help me with understanding this behavior of tomcat?
What do you mean by "unexpected"? Lower response time when you repeat the test can be explained by either your application implementation, like when you start load test against the application which is just deployed it's performance might not be optimal and when you repeat the test the cache is "warmed up" so you're getting better performance.
Another explanation could be JIT optimization as JVM is analyzing the pattern of your application usage and does inner improvements of the bytecode to better serve the given load pattern.
Third possible explanation is MongoDB caching, if 500 users are sending the same responses it might be the case the database stores the result sets in memory and when you repeat the test it doesn't actually access the storage but returns the results directly from the memory which is fast and cheap. Consider properly parameterizing your JMeter test so each thread (virtual user) would use its own credentials and perform different query than the other thread(s), but keep in mind that the test needs to be repeatable so don't use unique data each time, it's better to have sufficient set of pre-defined test data
Related
I have been running my JMeter script with the following setup
Users: 100
Loop Controller: 5
I used Loop Controller on the http request where transactions are needed to iterate.
My question is, there is a particular request which is searching where after 5 successful search the proceeding searches displayed “Customer API. Unable to establish connection”
Please see image below:
1 First Image displays lower load time while second image displays higher load time
It looks like you discovered a bottleneck in your application or at least it is not properly configured for high loads. Almost a minute response time for transfering 600 kilobytes is not something I would expect from a well-behaved application.
The error message you're getting is very specific so I would recommend checking your application logs as the very first step. The remaining steps would be inspecting application and middleware configuration and ensuring that it's properly set up for high performance and it has enough headroom to operate in terms of CPU, RAM, Network, Disk, etc.
I want to know how many HTTP requests per second my server can handle using Jmeter.
I got throughput 128/mins which means 2/3 req per sec. but something is wrong in my sampler can you tell what I need to do to test the login URL?
Hammering a login page doesn't have anything in common with the load testing as load testing is checking your application against the anticipated load.
If you're trying to determine the maximum number of requests per second your server can handle - this is some form of stress testing
In both cases you should come up with a realistic test scenario which would represent real users using real browsers like:
Record your scenario using JMeter's HTTP(S) Test Script Recorder
Perform parameterization and correlation
Run your test with several virtual users/iterations and inspect request/response details using View Results Tree listener to ensure that your test is doing what it is supposed to be doing
Once you're happy with your test script - disable the View Results Tree listener and run full load test using command-line non-GUI mode
Analyze your results using JMeter Reporting Dashboard
With regards to your goal I would recommend increasing the load gradually so you would be able to correlate the increasing number of users with the increasing throughput.
When you reach the point where you're adding more users and the throughput doesn't increase - it means that the application cannot handle more load hence that would be the maximum number of users/requests per second your application can handle.
We are creating a new hosted server for one of our APIs on managed containers (Kubernetes) and we're trying to validate that it can handle at least the same amount of traffic load requests.
We've started with one of the APIs, where we would need to handle at least 140k requests per minute, all endpoints combined.
To verify this, I created a simple JMeter test as follows:
-Test Plan
---Thread Group Endpoint1
-----HTTP Request -> a GET request with query params for /path1
---Thread Group Endpoint2
-----HTTP Request -> a GET request with query params for /path2
For a local test, I used the following setup:
Thread Groups Endpoint1 and Endpoint2 are set to 200 threads (users), ramp-up period of 1s, loop count = forever and duration 60s.
Using a Summary Report listener when running the test gets me a total of ~9300 # Samples.
Using this approach, is it safe to just increase the number of threads (users) for the Thread Groups until I reach the desired 140k requests per minute?
Note: I only used JMeter a little before, so I'm aware that the entire approach may be wrong, therefore any suggestions and steering to the right path are more than welcomed.
Your approach is viable as long as it represents real-life application usage. If it has 2 endpoints with equally/evenly distributed load - your setup is just fine. If there are more endpoints and some of them are used more than the others - consider defining the workload correspondingly either using different Thread Groups or other distribution mechanism such as Throughput Controller
Increasing the number of threads is also fine, however consider increasing the load gradually, to wit increase ramp-up time so your test could have:
Arrivals phase
Time to hold the load
Ramp-down phase
This way you will be able to correlate various metrics like increasing response time, throughput, number of errors, etc. with the increasing load. Also you will be able to state what was the number of threads/requests per second when the system reached saturation point/breaking point and does it recover when the load gets back.
Also make sure you're following JMeter Best Practices as 2300/2500 requests per second is not something JMeter can support out of the box and you will need to do some tuning, at least increase JVM Heap size allocated to JMeter.
You may not be able to achieve the desired 140k requests per minute using a single Jmeter Machine, in that case you'll need Distributed Load Testing approach here.
refer: http://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.html
Also keeping the ramp-up period of 1 second will lead to spike and unrealistic load in the system which will not give proper result unless you've pre-warmed your server, you should gradually increase the load as per real/estimated traffic pattern.
Getting the following error in JMeter while running the list of APIs (with no of threads:1-140 with ramp up period-1).
Response code:500
Response message: Internal Server Error
How should I overcome this Error Response code in order to get the accurate response?
What should do to decrease amount of response with this response code?
In general a 500 is an unhandled response on the part of a developer. Usually on the backend but also on the performance testing tool front end.
Ask yourself, are you validating responses that come back from the server for appropriate content? I am not just suggesting an HTTP200 is valid. You need to check response content to ensure that it is what you expect is valid for the business process, for you can have a completely valid HTTP200 class page which contains a response which will send your business process off the rails. If you do not handle the exception on the part of the unexpected response then you will find that one to two steps down the road in the business process then you are pretty much guaranteed that you will find a 500 as your request is completely out of context with the state of the application at that point.
Test101, for every step there is an expected and positive result which allows the business process to continue. Check for that result and branch your code when you do not find that the result is true.
Or, if this is a single step business process then you are likely handing the service poor data and the developer has not fully fleshed out the graceful part of dealing with your poor data.
The general advice in JMeter is Ramp-up = number of threads, in your case 140
Start with Ramp-up = number of threads and adjust up or down as needed.
Currently you are sending every 1/140 seconds new thread which is almost simultaneously, the reason to the change is:
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test
Status code - 500 comes from server/API's and it's not an issue of Jmeter. Sometimes the concurrent requests are rejected by server as it's too weak to handle that number of requests.In my case, I asked my server team to scale up servers so that we can test the underlying API . It's worth mentioning that sometimes Jmeter also runs out of memory. You can do some tweaking in set HEAP=-Xms512m -Xmx512m property of jmeter execuble file. Also listeners consume too much resources.Try not to use them.
I have recorded my web application through template & just to confirm that load test result which i am getting is correct? Just by increasing No of users does it give proper results? Is it enough for load testing of web application?
First of all you need to ensure that your test does what it is supposed to be doing. Recorded tests can rarely be successfully replayed, so normally you should be acting as follows:
Add View Results Tree listener and run your test with 1 user. Inspect request and response details to verify your test steps.
Perform correlation and parametrization if required.
Correlation: the process of identifying and handling any dynamic parameters. Most often people use Regular Expression Extractor for it.
Parametrization: the process of making your test data driven. For example, if your application assumes multiple authenticated users you need to store the credentials somewhere. Most commonly used test element for this is CSV Data Set Config
Make your test realistic. Virtual users simulated by JMeter need to represent real users using real browsers as close as possible with all the related stuff: cookies, headers, cache, etc. See How To Make JMeter Behave More Like A Real Browser to learn how to configure JMeter to act closer to real users. Also real users need some time to "think" between operations so make sure you are using Timers to simulate this behaviour as well.
Only after you apply the above points you should add more virtual users. Again, run your test with 2-3 users and iterations to ensure your test funcitons as designed. Once you are happy with it you can increase the load, but don't overkill your server, increase the load gradually and check the impact of the increasing load on your application, i.e. how response time, throughput and number of errors change as you increase the load. The same is applicable for decreasing the load, don't turn it off at once, decrease the number of virtual users gradually.
Building a Web Test Plan
Building an Advanced Web Test Plan