I am using Visual Studio 2010 Ultimate to perform loadtests. These loadtest use recorded webtests.
When running a loadtest with an increasing number of concurrent users, some steps in my webtests will start to fail. The first error is often an internal server error 500. This will give a wrong impression of the average page_load, because these internal server errors are often returned very fast, in contrast to the generation a succesful response. So, when the load increases, the average page_load drops.
Of course, I need to attend to these internal server errors, but in the meantime, I would like to exclude failed webtests from my measurements.
Does anybody know if this can be done?
Thanks in advance.
It may be possible to run your own query on the test results database that ignores errors, but even that will be inaccurate.
Remember that the page return stats are really only useful when read in conjunction with the load on the hardware.
Essentially, the load test is recording the effect on your hardware of a given load. If you website is returning a large number of 500 error pages quickly, the load on the hardware will be affected and any page stats will reflect the change in server loading.
You will have to investigate the cause of the 500 errors and either fix the issue or report in your load testing results that once a load of 'x' is reached on the servers, the pages 'y' will give an internal server error 500 result instead of the requested page.
This gives the business owners of you app some information to make the decision whether to fix the problem or live with it.
Related
Getting 503 Error while Running the JMeter for the Thread User 400,Is it Because of Server issues.? When I run the thread group for 100 user with ramp up period 25 seconds then it will be working fine but for the user 400 users its giving 503 error.
Given you don't experience any issues with 100 users and have issues with 400 users most probably it's a server issue connected with the overload so congratulations on finding the bottleneck.
You can either report it as is or perform a little bit deeper investigation in order to find the cause, suggested steps:
Instead of kicking off 400 users at once try increasing the load gradually at the same time looking at Response Times vs Threads and Transaction Throughput vs Threads charts. Ideally response time should remain the same and throughput should be growing as the number of threads increase. When response time starts increasing and throughput starts decreasing it indicates the saturation point and at this stage you can state that this is the maximum number of users your application can support
Check your application logs and configuration as it might be not properly tuned for the high loads, you can use 15 Simple ASP.NET Performance Tuning Tips as a reference or look for a similar guide for your application technology stack
Ensure that your application has enough headroom to operate in terms of CPU, RAM, Network, etc. as it might be the case that it's basically a lack of resources, it can be done using i.e. JMeter PerfMon Plugin
Repeat your test with profiler tool telemetry in place, this way you will be able to localize the problem and state where is the problematic piece of code or inefficient algo lives.
If server isn't down/restarted, then yes, 503 indicate overload
Common causes are a server that is down for maintenance or that is overloaded
You need to find what stop server from serving 400 concurrent requests/users
Notice that if you are testing on a test environment which isn't equal/similar to production environment, it may not reflect the load that production server can endure
I am testing a web application login page loading time with 300 thread users and ramp up period of 300 secs.Most of my samples return response code 200.But few of them return response code 400,503.
My goal is to just check the performance of the web application if 300 users start using it.
I am new to Jmeter and have basic knowledge of programming.
My Question :-
1.Can i ignore these errors and focus just on timings from the summary report ?
2.If i really need to fix these errors, how to fix it ?
There are 2 different problems indicated by these errors:
HTTP Status 400 stands for Bad Request - it means that you're sending malformed requests which cannot be understood by the server. You should inspect request details and amend JMeter configuration as it is the problem in your script.
HTTP Status 503 stands for Service Unavailable - it indicates the problem on server side, i.e. server is not capable of handling the load you're generating. This is something you can already report as the application issue. You can try to identify the underlying cause by:
looking into your application log files
checking whether your application has enough headroom to operate in terms of CPU, RAM, Network, Disk, etc. It can be done using APM tool or JMeter PerfMon Plugin
re-running your test with profiler tool telemetry to deep dive into what's under the hood of the longest response times
So first of all you should ensure that your test is doing what it is supposed to be doing by running it with 1-2 users/loops and inspecting requests/response details. At this stage you should not be having any errors.
Going forward you should increase the load gradually and correlate the increasing number of virtual users with the increasing response time/number of errors
`
Performance testing is different from load testing. What you are doing is load testing.
Performance testing is more about how quickly an action takes. I typically capture performance on a system not under load for a given action.
This gives a baseline that I can then refer to during load tests.
Hopefully, you’ve been given some performance figures to test. E.g. must be able to handle 300 requests in two minutes.
When moving onto load, I run a series of load tests with increasing number of users/threads and capture the results from each test.
Armed with this, I can see how load degrades performance to the point where errors start to show up. This gives you an idea of how much typical load the system can handle.
I’d also look to run soak tests too. This where I’d run JMeter for a long period with typical (not peak) load to make sure the system can handle sustained load.
In terms of the errors you’re seeing, no I would not ignore them. Assuming your test is calling the same endpoint, it seems safe to say the code is fine, its the infrastructure struggling with the load you’re throwing at it.
I execute jmeter script via Blazemeter and I got 2% error. Is it a acceptable rate?
In detail report I observed 1353 request are failed. I just run the script for one user. In that case non of the request doesn't fail. So is that failures are due to performance issue?
Following is summary of the report. Kindly help me to anlyse this.
Most probably your application simply cannot handle the load of 50 concurrent users. With regards to "acceptable" - we don't know. If you're load testing a fan page of your local hip hop star - even 90% error rate will be acceptable. If you're testing an algorithm which will be deployed on a Mars rover and will have to work without errors and modifications for 20 years - it is not.
Normally maximum response time, minimum throughput, acceptable number of errors, etc. are defined in SLA or NFR. If you don't have those and performing some form of stress testing of your application and want to figure out the root cause of the performance bottleneck - take the next steps:
Check your application log file(s), they should have some information regarding the failure
Check status message and code in .jtl results file. Sometimes it also makes sense to "tell" JMeter to save response data for failed samplers by adding the next lines to user.properties file:
jmeter.save.saveservice.output_format=xml
jmeter.save.saveservice.response_data.on_error=true
Make sure you add the load gradually, this way you will be able to correlate increasing error rate with increasing number of users and will be able to determine exact point of time when first error occurred
Get used to monitoring whether your application under test has enough headroom to operate in terms of CPU, RAM, Network, Disk, etc. It can be done using JMeter PerfMon Plugin
If you have ability to read and understand the code in the language your application is written in - it would be beneficial if you could run your test with profiler tool telemetry enabled, this is probably the most efficient way to identify the performance problem in your application.
I am performing baseline performance test on a project. Average load time reported by jmeter is much higher than actual load time in browser(fresh- no cache and cookies).
What will be the issue?
I suggest to check the following:
Load generator overload. Re-run JMeter test with one user/thread and compare with Firefox. If the results will be comparable then the response time in JMeter may be excessive due to its overload. Try to address it by adding more load generators.
Inaccurate browser emulation. If even with one user the response time in the load test is higher, then it can be caused by inaccurate emulation of browser paralel connections. To troubleshoot it, compare waterfall diagrams. To get it from Firefox, use Firebug. Route JMeter traffic through Fiddler which displays the waterfall on the Timeline tab. If the waterfalls are different, you may have the following issue: a web browser downloads resources in parallel, while by default JMeter replays recorded traffic sequentially. To fix it, add these settings: Simulating browsers using JMeter.
Are you checking with browser during the load test ? or at another time ?
In the latter case, you would be comparing apples and oranges.
Are you using JMeter GUI mode ? if yes, it's a bad practice, GUI mode is for scripting, NON GUI mode for load testing:
http://www.ubik-ingenierie.com/blog/jmeter_performance_tuning_tips/
How much threads are you using ? and which version of Jmeter ?
For embedded resources testing, 3.0 is the most realistic and performing:
https://jmeter.apache.org/changes.html
Whenever you run performance tests instead of Average response times, always consider 90th percentile. In some cases avg. response time is skewed even if one request takes long to respond. So please check 90th Percentile.
If you are running the test with multiple users, try to hit the application from browser while load test is going on and check the response time on browser. This will tell you if your observation is correct.
The load generator may not be able to establish enough connections due to which you might see higher response time. Check load generator utilization in case if you doubt. In some cases load generator itself can't generate enough load.
Check your sever utilization when you run the performance test. This will give you an idea if the application is not able to handle the load or if it's the issue with load generator.
If you are running tests from UI mode, please try to run tests with non-UI mode. (Can you specify with how many users you are running these tests?)
Increase JMeter memory if you see issue with load generator and keep eye on load generator CPU usage too.
Check if load generator and browser from which you are hitting the application are on the same network and check network latency to check if there is network problem.
I'm new to load testing using VS 2015. Right now, I'm working on load testing for a web project which will need recorded web performance tests for each interaction that users would typically do with our application.
I recorded a web performance test for the simple logging-in of user in the website. After clicking the stop button in browser, the web performance test was generated in the VS 2015 but with an error.
Although I successfully logged in during the recorded web performance test, I was wondering if should I be worried with the error displayed and would affect the load testing which I will be using the recorded web performance test for.
Error message: 1 primary requests, 1 dependant requests and 0 conditional rules failed
When the error message is clicked, the following details would show up:
Access denied (authentication_failed)
404 - File or Directory not found SERVER ERROR
Please help. Thanks
After the stop button on the browser is pressed, Visual Studio runs the test once to help it find dynamic data, etc. Commonly this execution of the test fails, so do not worry about this failure.
You should then expect to run the test yourself a number of times, to make it work properly. Before each execution you may need to make changes, for example:
for data driving
for adding credentials
for adding verification rules
to sort out dynamic data that Visual Studio has not detected, this will probably include adding extraction rules