I am testing a web application login page loading time with 300 thread users and ramp up period of 300 secs.Most of my samples return response code 200.But few of them return response code 400,503.
My goal is to just check the performance of the web application if 300 users start using it.
I am new to Jmeter and have basic knowledge of programming.
My Question :-
1.Can i ignore these errors and focus just on timings from the summary report ?
2.If i really need to fix these errors, how to fix it ?
There are 2 different problems indicated by these errors:
HTTP Status 400 stands for Bad Request - it means that you're sending malformed requests which cannot be understood by the server. You should inspect request details and amend JMeter configuration as it is the problem in your script.
HTTP Status 503 stands for Service Unavailable - it indicates the problem on server side, i.e. server is not capable of handling the load you're generating. This is something you can already report as the application issue. You can try to identify the underlying cause by:
looking into your application log files
checking whether your application has enough headroom to operate in terms of CPU, RAM, Network, Disk, etc. It can be done using APM tool or JMeter PerfMon Plugin
re-running your test with profiler tool telemetry to deep dive into what's under the hood of the longest response times
So first of all you should ensure that your test is doing what it is supposed to be doing by running it with 1-2 users/loops and inspecting requests/response details. At this stage you should not be having any errors.
Going forward you should increase the load gradually and correlate the increasing number of virtual users with the increasing response time/number of errors
`
Performance testing is different from load testing. What you are doing is load testing.
Performance testing is more about how quickly an action takes. I typically capture performance on a system not under load for a given action.
This gives a baseline that I can then refer to during load tests.
Hopefully, you’ve been given some performance figures to test. E.g. must be able to handle 300 requests in two minutes.
When moving onto load, I run a series of load tests with increasing number of users/threads and capture the results from each test.
Armed with this, I can see how load degrades performance to the point where errors start to show up. This gives you an idea of how much typical load the system can handle.
I’d also look to run soak tests too. This where I’d run JMeter for a long period with typical (not peak) load to make sure the system can handle sustained load.
In terms of the errors you’re seeing, no I would not ignore them. Assuming your test is calling the same endpoint, it seems safe to say the code is fine, its the infrastructure struggling with the load you’re throwing at it.
Related
I have excecated a test and got the following report and duration the analysis I noticed that the sum of the individual request data is not matching up to the transaction enter image description here
Help me around here to identify the cause of this issue
Note: For running one or few users I am not facing this issue, only during on higher user count this issue is coming up
Most probably there is an issue with the way you "excecate" the test (whatever it means)
Given the issue is not reproducible with "few" users and only happens under the load my expectation is that JMeter doesn't have enough headroom to operate in terms of CPU/RAM/etc. or it's not properly configured for high loads.
Make sure to follow JMeter Best Practices
Make sure to set up monitoring of resources like CPU, RAM, Network, Disk IO, swap, etc. If you don't have other/better monitoring toolchain you can consider using JMeter PerfMon Plugin
If after following JMeter Best Practices the error is still there or resource consumption is way too high - consider going for Distributed Testing
If even after switching to distributing testing the issue is still there check jmeter.log for any suspicious entries
I am new to performance testing. I have a task on measuring the web application performance. I need to find out which modules/calls are causing deadlock, timeout and memory issues.
Q1. How can I use JMeter to find out deadlock, memory and timeout issues? If I do the following steps, it is the right way to trace those issues?
create a test plan in JMeter, which contains multiple Thread Group.
In each thread group, it contains multiple HTTP requests and 200 or
more users plus infinite loop.
Monitor JMeter results and SQL
profiler for deadlock.
Q2. JMeter is the right tool for tracking those issues? Or, should I use browser based performance testing tool such as LoadNinja, LoadView?
Thanks
Bonnie
Q1 JMeter per se doesn't provide any toolchain to detect deadlock and memory issues, the HTTP Request sampler (or even better HTTP Request Defaults) provides possibility to set the timeouts, if the value is blank - it will default to operating system timeout or web server timeout, whatever comes the first
If you conduct some form of stress test, i.e. start with 1 virtual user and gradually increase the load at some point you will see that response time starts growing and number of requests per second starts decreasing. So it's the point of maximum system performance and after that the performance will be degrading.
To monitor application under test memory you can use JMeter PerfMon Plugin, it will allow you to state whether the lack of RAM is the cause of the performance issue
With regards to deadlocks, it should result in HTTP Request sampler failure (or timeout), JMeter won't give you the underlying reason, but it will give you the timestamp and you should be able to check what happened with your application/database at that moment.
Q2 well-behaved JMeter test must produce the same network footprint as a real browser, if your test plan is good enough the system under test shouldn't be able to distinguish whether it's being hit by JMeter or by a real user using the real browser. JMeter will not give you client-side performance metrics like page rendering time or JavaScript execution time as:
JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).
Getting 503 Error while Running the JMeter for the Thread User 400,Is it Because of Server issues.? When I run the thread group for 100 user with ramp up period 25 seconds then it will be working fine but for the user 400 users its giving 503 error.
Given you don't experience any issues with 100 users and have issues with 400 users most probably it's a server issue connected with the overload so congratulations on finding the bottleneck.
You can either report it as is or perform a little bit deeper investigation in order to find the cause, suggested steps:
Instead of kicking off 400 users at once try increasing the load gradually at the same time looking at Response Times vs Threads and Transaction Throughput vs Threads charts. Ideally response time should remain the same and throughput should be growing as the number of threads increase. When response time starts increasing and throughput starts decreasing it indicates the saturation point and at this stage you can state that this is the maximum number of users your application can support
Check your application logs and configuration as it might be not properly tuned for the high loads, you can use 15 Simple ASP.NET Performance Tuning Tips as a reference or look for a similar guide for your application technology stack
Ensure that your application has enough headroom to operate in terms of CPU, RAM, Network, etc. as it might be the case that it's basically a lack of resources, it can be done using i.e. JMeter PerfMon Plugin
Repeat your test with profiler tool telemetry in place, this way you will be able to localize the problem and state where is the problematic piece of code or inefficient algo lives.
If server isn't down/restarted, then yes, 503 indicate overload
Common causes are a server that is down for maintenance or that is overloaded
You need to find what stop server from serving 400 concurrent requests/users
Notice that if you are testing on a test environment which isn't equal/similar to production environment, it may not reflect the load that production server can endure
I execute jmeter script via Blazemeter and I got 2% error. Is it a acceptable rate?
In detail report I observed 1353 request are failed. I just run the script for one user. In that case non of the request doesn't fail. So is that failures are due to performance issue?
Following is summary of the report. Kindly help me to anlyse this.
Most probably your application simply cannot handle the load of 50 concurrent users. With regards to "acceptable" - we don't know. If you're load testing a fan page of your local hip hop star - even 90% error rate will be acceptable. If you're testing an algorithm which will be deployed on a Mars rover and will have to work without errors and modifications for 20 years - it is not.
Normally maximum response time, minimum throughput, acceptable number of errors, etc. are defined in SLA or NFR. If you don't have those and performing some form of stress testing of your application and want to figure out the root cause of the performance bottleneck - take the next steps:
Check your application log file(s), they should have some information regarding the failure
Check status message and code in .jtl results file. Sometimes it also makes sense to "tell" JMeter to save response data for failed samplers by adding the next lines to user.properties file:
jmeter.save.saveservice.output_format=xml
jmeter.save.saveservice.response_data.on_error=true
Make sure you add the load gradually, this way you will be able to correlate increasing error rate with increasing number of users and will be able to determine exact point of time when first error occurred
Get used to monitoring whether your application under test has enough headroom to operate in terms of CPU, RAM, Network, Disk, etc. It can be done using JMeter PerfMon Plugin
If you have ability to read and understand the code in the language your application is written in - it would be beneficial if you could run your test with profiler tool telemetry enabled, this is probably the most efficient way to identify the performance problem in your application.
I am performing baseline performance test on a project. Average load time reported by jmeter is much higher than actual load time in browser(fresh- no cache and cookies).
What will be the issue?
I suggest to check the following:
Load generator overload. Re-run JMeter test with one user/thread and compare with Firefox. If the results will be comparable then the response time in JMeter may be excessive due to its overload. Try to address it by adding more load generators.
Inaccurate browser emulation. If even with one user the response time in the load test is higher, then it can be caused by inaccurate emulation of browser paralel connections. To troubleshoot it, compare waterfall diagrams. To get it from Firefox, use Firebug. Route JMeter traffic through Fiddler which displays the waterfall on the Timeline tab. If the waterfalls are different, you may have the following issue: a web browser downloads resources in parallel, while by default JMeter replays recorded traffic sequentially. To fix it, add these settings: Simulating browsers using JMeter.
Are you checking with browser during the load test ? or at another time ?
In the latter case, you would be comparing apples and oranges.
Are you using JMeter GUI mode ? if yes, it's a bad practice, GUI mode is for scripting, NON GUI mode for load testing:
http://www.ubik-ingenierie.com/blog/jmeter_performance_tuning_tips/
How much threads are you using ? and which version of Jmeter ?
For embedded resources testing, 3.0 is the most realistic and performing:
https://jmeter.apache.org/changes.html
Whenever you run performance tests instead of Average response times, always consider 90th percentile. In some cases avg. response time is skewed even if one request takes long to respond. So please check 90th Percentile.
If you are running the test with multiple users, try to hit the application from browser while load test is going on and check the response time on browser. This will tell you if your observation is correct.
The load generator may not be able to establish enough connections due to which you might see higher response time. Check load generator utilization in case if you doubt. In some cases load generator itself can't generate enough load.
Check your sever utilization when you run the performance test. This will give you an idea if the application is not able to handle the load or if it's the issue with load generator.
If you are running tests from UI mode, please try to run tests with non-UI mode. (Can you specify with how many users you are running these tests?)
Increase JMeter memory if you see issue with load generator and keep eye on load generator CPU usage too.
Check if load generator and browser from which you are hitting the application are on the same network and check network latency to check if there is network problem.