I have a J2EE web-application running on Sun hardware with OpenSolaris/Glassfish stack. We're starting our performance bench-marking tests, to prepare for our scalability requirements later.
Any guidelines/best practices would be very useful..
If you are using JUnit? For simple Time Measurement in JUnit there is JUnitPerf which also supports LoadTests. JUnit 4 is not supported very well but it works for me. ContiPerf is an alternative which also supports annotations for easy configuration. I use this before using JMeter.
For a free Open Source tool I've found Apache's Jmeter pretty good:
http://jmeter.apache.org/
As Joe said you'll need to figure out how to model user behaviour and simulate user load.
I'm sure there is many generic ways to test server performance to be found on the web.
But have you thought that perhaps this an opportunity to gather stories and write a test tuned to exactly how your users will use the server? Even if it doesn't perfectly reflect the way that the server will ultimately work, it will let you start learning something rather than being only formal exercise. I think this is the spirit of agile development.
Related
I have 28 micro-services, some of which communicates with each other. All of them are built with SpringBoot 2x and they use their own resources (database, rabbitmq etc.). They are deployed in PCF.
I need to identify weakness of the overall system. That's when I resorted to Chaos Engineering. As this is my first time, I can use some help regarding how to design the effort, what metrics should I collect, tools that I can use, how long should I run such tests etc.
TIA
Done with my research. Posting it here so that someone else might find it useful.
Very good introductions including how to start your first chaos experiment: https://www.gremlin.com/community/tutorials/chaos-engineering-the-history-principles-and-practice/
Good summary of people, tools, companies doing chaos experiments:
https://coggle.it/diagram/WiKceGDAwgABrmyv/t/chaos-engineeringcompanies%2C-people%2C-tools-practices/0a2d4968c94723e48e1256e67df51d0f4217027143924b23517832f53c536e62
Tools:
Spinnaker: https://www.spinnaker.io/. Netflix Chaos Monkey does not support deployments that are managed by anything other than Spinnaker. That makes it pretty hard to use Chaos Monkey from Netflix.
ChaosMonkey for SpringBoot: https://docs.chaostoolkit.org/drivers/cloudfoundry/. Very easy to follow instructions. Easy to turn on/off using Spring profile.
Chaos Toolkit - https://docs.chaostoolkit.org/drivers/cloudfoundry/. This tool is particularly helpful to my situation since my applications are deployed in Cloud Foundry and this tool has a CloudFoundry extension. Pretty elaborate, but easy to follow instructions. My preferred tool so far.
Chaos Lemur - https://content.pivotal.io/blog/chaos-lemur-testing-high-availability-on-pivotal-cloud-foundry. This tool has promise but network admin won't share AWS credentials for me to muck with Pivotal cells.
You already found Chaos Toolkit, which is what I would recommend for the actual experimenting, especially when starting out.
The basis of Chaos is that you need to plan it out and analyse properly before starting out. And then you need to find out what kind of information you will need to monitor the results of the experiment. What happens to service X if we implement latency? Monitor requests, errors etc. Is there a functioning degraded state or is it just dead?
Your question is very broad and as such it's difficult to answer.
Make sure to try to prevent cascading failures whenever you start an experiment and always have stop that terminates the experiment and rolls everything back should something break really bad.
I have taken as an example for learning and gathered some information about tools, objectives,scenarios, but I need your inputs. Please assist me.
I am new to Performance testing and would like to test the following website www.volkswagen.co.nz
Can you tell me, what are need to be tested? What are the scenarios and activities for each scenario? What metrics do I need to add? Which is the best and free tool for testing it? How to test if it is deployed in cloud like AWS?
Please let me know, Thanks in advance.
Performance testing needs,
Identify critical/heavy/important scenario in your webapp (irrespective of deployment cloud/standalone)
Identify service level agreements in terms of response times, throughput, latency etc.
Identify workload model i.e. how much user load application is expecting. this should be as fine grained as possible (avg users per transaction/workflow at a point of time)
Identify tools (JMeter is freeware and best but if you can afford paid then look at loadrunner, neoload etc.)
Record the script for workflows and parameterise and correlate.
generate test setup for load test and execute the load test.
monitor system utilization, collect metrics like response time, throughput, error rate, latency etc.
This all comes in load testing. For more you can read http://www.guru99.com/performance-testing.html
I am new to Performance testing and would like to test the following website www.volkswagen.co.nz
That is a recipe for disaster. No one new should be allowed to work on their own without a full period of training and internship with a master in the field. This is true of stone masons, electricians, plumbers, barbers, accountants, engineers and physicians. And it is most certainly true of performance testers/engineers.
There are dozens of foundation skills you need to master before you touch any tool, open source or otherwise. Until you show mastery of those items along with tool mechanics for your tool you should not be allowed to test any website, particularly a production website. And, if you don't work for this company what you are engaging in is a denial of service attack and could leave you with exposed legal liability.
I strongly agree with James on this one.
Do not touched the site if:
it's not yours
not sure what you are doing
the owner gave you explicit (and sounds like irresponsible) permission
don't know or don't have the support to restore the environment into a working state
If you do work for the company then you need to have a test environment first, a playground where you can mess around and nobody would mind if you take it down.
Firstly get information from the business on which use cases needs to
be tested.
Get response times target for user actions and for environments utilisation.
Get response time targets for environments utilisation: define environment monitoring tactics.
Found a tool that can fit for purpose: Jmeter, Gattling,etc, lot's of free ones available.
Get a test environment, preferably similar scale to production
Create scripts to cover critical use cases
Comply scripts into scenarios
Create a reporting framework
Kick off monitoring
Kick off scenario
Collect and analyse results
Be mindful of the free editions of load testing tools: they tend to be easy to use at first but soon as you start to outgrow it it can cost a fortune and more often then not it's hard to port scripts/scenarios to another tool.
What are the best tools to test the performance of a (not deployed) application using Play framework? Things like, how long takes a request to execute, with different parameters, simulating a lot of requests (stress test), etc.
I'm searching a while but the problem is that the keyword "performance", "benchmarks" etc. lead me to pages about the performance of Play framework.
I thought maybe functional tests, could be used to measure performance (print difference between method start time and end...). But this doesn't look suitable for this kind of task.
I could just write a script, that triggers the requests, writes the timestamps to a log file... but maybe there's something finished, with extras, like e.g. charts, etc.
Any hint in the right direction greatly appreciated.
Iago is a load generation tool by Twitter written in Scala. Also, I've used the Loader.io addon on Heroku to do performance testing. Loader.io also has a non-heroku service that I have not used. Iago is probably your best bet for local testing of a non-public app.
A good example is a project used by Versal to choose their Scala stack for production.
The project is Scamper.
I've got a PHP site up and running, and the db is mysql. before launching the site, I would like to test the traffic handling. Now am assuming that there are soe softwares that would simulate the traffic and log the processes running on my site. Any recommendation of software I should use? the traffic doesn't have to be real, but nonetheless, I would like to generate a high traffic to investigate the threshold of the site.
Appreciate the help
You can use Gatling https://github.com/excilys/gatling.
It's a stress tool written in Scala which aims at being more efficient and lighter than Jmeter.
Basically you record a scenario on your website and then run it 'n' times in parallel.
Here is the wiki for more infos https://github.com/excilys/gatling/wiki/Basic-Usage
You can use Jmeter:
It's free.
it's easy to Start with lot of documentation on its Website and on internet
it has a proxy feature to easily create test plan from browser navigation
It is easy to start up processes on other machines. It remote testing, can be done from GUI or console.
The scripts can be written in beanshell, java, or any jsr223 language ( groovy, Javascript, scala, jexl ...)
it has a lot of built- in samplers and thanks to its plugin architecture it's very Easy to add new ones or use any scripting engine to do what's missing
it has great user mailing list
it has very reactive support
it's now a top Level Apache
it can run thousands of users
professional solutions exist to run it from cloud
...
See:
Performing a Stress Test on Web Application?
Best way to stress test a website
How do you test the performance of a website?
I am doing performance test against our site. The site exposes many Web services APIs. Our product has several workflows and each complete workflow is composed of calling more than one APIs.
I am wondering which of the following apporoaches is reasonable:
Test against each single API in a standalone way.
Test against each complete workflow which involves several APIs.
I think the latter one make more sense since it mimic the real scenario.
I'd like to hear your comments.
Thanks.
There are (as usual) pro's and cons of each approach. Personally I'd consider doing both, starting with the single API test.
The single API is probably the easiest to build, and it has the benefit of exactly pinpointing out where your performance issues are. It's also useful to so spot regressions in performance during development. If there are unit tests for the application consider using those instead. When there is a performance regression there usually also is a unit test which suddenly became slower.
Once you've done that you will still need to do the more complex tests. Firstly because you need to know if the performance of a certain flow is acceptable, but also because there can be unexpected interactions between different API's. Depending on you application there may be nasty concurrency issues, throughput bottlenecks etc. Make sure you run several flows concurrently, that's what happens in really live and it's only way to find issues related to locking in the database, I/O bottlenecks etc.
But before you start make sure you have a realistic idea of what the performance should be, how many concurrent users there will be and what the hardware requirements are. There is no limit to improving performance, so you have to decide what is good enough or you will never stop optimizing.