How do you begin with setting a Load environment for an Enterprise application (traditional MVC application with connectivity to some dependent enterprise systems)? For example, it would be ideal to have similar amount of Servers with same configuration as Production environment, also the Database should have similar size and capacity as Production to make Load environment similar to Production.
This does not happen in many organization, and I have seen organization using a trimmed down version of Production infra for the Load testing. Does that seems to be correct approach? Can we run a load test on the trimmed down version of Prod infra? Will this approach produce the result that can be used to predict Production application performance?
If you need to measure how many users / requests per second your application can support the only way of doing it is running your test against production or production-like environment.
There are some things you could check against a scaled-down environment, for example:
Running a soak test, this way you can discover memory leaks
Running a load test having profiler tools telemetry enabled on the application under test side, this way you will identify the slowest functions, largest objects, etc.
Running a database load test, this way you can find out slow queries subject to optimization
More information: Performance Testing in a Scaled Down Environment. Part Two: 5 Things You Can Test
Related
i want to use the tool Apache JMeter in order to make an http flood in a website that i have designed in Visual Studio 2017. My question is, if i start running the website and in the same time start the http flood, is it "safe" for my PC? I mean, is it possible to cause a damage because of the resource consumption? Or is it depended from the number of threads i'll use at the JMeter script parameters?
Thank you.
From the hardware perspective I don't think you will be able to damage your machine as most probably it will throttle or simply turn off when temperatures will exceed acceptable threshold.
From the test results point of view you might be getting inaccurate metrics especially if you run JMeter at the same machine as when it comes to high loads both JMeter and your website will become very resource intensive will "fight" for the resources like CPU, RAM, etc.
So I would recommend considering deploying your website on prod-like environment and use separate JMeter load generator(s) for conducting the load. This way you will get confidence that test results are not impacted by mutual interference.
If you cannot set up proper load testing environment unfortunately your test scenario will not make a lot of sense, there are still some areas which you can test on a scaled-down environment from performance perspective like:
integration test
soak test
interoperability test
How to proceed with performing performance testing for an application that is not in production.
If the environment of the application under test is the same as it will be in production - the approach should not be different.
If the environment of the application under test is different, i.e. has less servers, the servers has less memory, etc. - in the absolute majority of cases you will not be able to calculate and predict the performance on more powerful hardware as there are too many factors to consider. You can inform the client about it right away.
However there are still some types of testing you can perform, i.e.:
You could run an integration test to check whether your application is configured for high load
You could run a scalability test to check whether and how does your application scale
You could run a soak test to check for possible memory leaks
You could use profiling tools to check bottlenecks in terms of code quality
You could identify slow DB queries and look for a way to optimise them
etc.
More information: Performance Testing in a Scaled Down Environment. Part Two: 5 Things You Can Test
If you meant how to judge on how much volume to test if application is not already in production, the answer is simple, you have to predict. The prediction can be based on survey, reports from your business analysts.
If none of the above are available. Just test how much your application can withstand in a test environment of similar configuration as production. This will give you an idea of when you need to start worrying while the application is live.
I need to undergo performance testing for my project and I have learned how to Handle the Jmeter for Performance testing through online, but still, i was unable to find the solution how to analyze the result? from the report.I do know how to analyze the result so that I can't able to find the Performance Issue I n m application, where the error had been occurring, so from that how I can improve that performance.Is there is any article or video tutorial to learn how to analyze the result?
There are 2 possible test outcomes:
Positive: the performance of your application matches SLA and/or NFR. In this case you need to provide the report as a proof, it might be i.e. HTML Reporting Dashboard
Negative: the performance of your application is beyond the expectations. In this case you need to perform some investigation on the reasons, which could be in:
Simply lack of resources, i.e. your application doesn't have enough headroom to operate due to not enough CPU, RAM, Network or Disk bandwidth. So make sure you are monitoring these resources on the application under test side, you can do it using i.e. JMeter PerfMon Plugin.
The same but on JMeter load generator(s) side. If JMeter cannot send requests fast enough the application won't be able to serve more requests so if JMeter machine doesn't have enough resources - the situation will be the same so make sure to monitor the same metrics on JMeter host(s) as well
You have some software configuration problem (i.e. application server thread pool, db connection pool, memory settings, caching settings, etc. are not optimal). In the majority of cases web, application and database servers default configuration is not suitable for high loads and it needs to be tuned so try playing with different settings and see the impact.
Your application code is not optimal. In case when there is a plenty of free hardware resources and you are sure that infrastructure is properly set up (i.e. other applications behave fine) it might be a problem with your application under test code. In this case you will need to re-run your test with profiler tool telemetry to see what are the most time and resources consuming methods and how they can be optimised.
It might also be a networking related problem, i.e. faulty router or bad cable or whatever.
There are too many possible reasons however the approach should be the same: the whole system acts at the speed of its slowest component so you need to identify this component and determine why it is slow. See Understanding Your Reports posts series to learn how to read JMeter load test results and identify the bottlenecks from them.
I have to test my rest API such that 100k API calls are made simultaneously(within 500ms).Any Idea how to simulate it?Utility to use?
I would suggest to use JMeter too, it enable your test to create multiple concurrent jmeter server. Just to be clear you can control multiple remote JMeter engines from a single JMeter client and replicate a test across many computers and thus simulate a larger load on the target server.
To be honest, your target is quite high (100k API calls simultaneously within 500ms), i.e. you'll need a lot of jmeter servers. When you create stress tests, there are not magical recipes, guides or manuals. Trial and error is a fundamental method of solving this kind of problems.
In my experience, I first try with few concurrent users and see how the server react. Then increase the number of concurrent users till to reach an intolerable performance decrease or, worst, a bottleneck .
http://jmeter.apache.org/usermanual/remote-test.html
You will obviously need a load testing tool which can be run in a distributed mode, i.e. 1 controller and X load generators executing the same test.
Grinder - scripts are written in some Python dialect
Apache JMeter - this guy doesn't require any specific knowledge, you can create tests using simple GUI
Tsung - is written in Erlang, known for capability to produce high loads even on low-end hardware.
See Open Source Load Testing Tools: Which One Should You Use? article for more information on above tools.
JMeter
The Apache JMeter™ application is open source software, a 100% pure
Java application designed to load test functional behavior and measure
performance. It was originally designed for testing Web Applications
but has since expanded to other test functions.
Been trying to understand the overall performance of our application by comparing the benchmark we get from our dev environment and our prod environment.
Interestingly, in our dev environment, which is our local machine, we get application run time as fast as 98ms.
The same application runs on avg at 400ms in our production server, which is a VPS with CentOS 5.8 running.
I'm assuming that this increase must be because of network connection lag between web server and database server, since we didn't have this gap in the dev environment, everything is local.
We're using Doctrine 2.0 as an ORM for our application, we haven't really gotten into optimizing it by caching.
Is there a way to optimize this lag time? Or am I completely wrong about the case?
Your best bet to measure actual database and query time used is to setup a database profiler.
You can read this: Profiling Doctrine 2 with Zend Framework
Just as a note, running a profiler should only be done for testing. You shouldn't run it all the time, especially if your production server is high volume. It will add some processing time, but it will give you more information about your queries and connection time.
Your assumption of it being latency between the boxes could be true and this will verify it for you.