I'm pretty new to performance testing.
I've got a bunch of endpoints to test and I wonder what is the best approach to do it.
I thought about grouping tests for endpoints in separate .jmx files, for example:
Suppose that there are 2 endpoints: A with simple GET method and B that can serve GET, PUT, POST and DELETE requests. In this scenario I would create 2 .jmx files.
Any helpful links to some tutorials appreciated :)
Normally people use Test Fragments for organizing large JMeter scripts.
Depending on whether you want to keep the requests in a single .jmx script or separate you can use:
Module Controller - for re-using requests in the same scripts (i.e. to avoid code duplication)
Include Controller - to "pull" external .jmx script into the current one
Related
For functional automation we use to create a framework which is reusable for automating application. Is there any way to create Performance testing framework in jmeter. So that we can use same framework for Performance testing of different applications.
Please help if any one knows and provide more information regarding it.
You can consider JMeter as a "framework" which already comes with test elements to build requests via different protocols/transports, applying assertions, generating reports, etc.
It is highly unlikely you will be able to re-use existing script for another application as JMeter acts on protocol level therefore there will be different requests for different applications.
There is a mechanism in JMeter allowing to re-use pieces of test plan as modules so you won't have to duplicate your code, check out Test Fragments and Module Controller, however it is more applicable for a single application.
The only "framework-like" approach I can think of is adding your JMeter tests into continuous integration process so you will have a build step which will execute performance tests and publish reports, basically you will be able to re-use the same test setup and reporting routine and the only thing which will change from application to application will be .jmx test script(s). See JMeter Maven Plugin and/or JMeter Ant Task for more details.
You must first ask yourself, how dynamic is my conversation that I am attempting to replicate. If you have a very stable services API where the exposed external interface is static, but the code to handle it on the back end is changing, then you have a good shot at building something which has a long life.
But, if you are like the majority of web sites in the universe then you are dealing with developers who are always changing something, adding a resource, adding of deleting form values (hidden or not), headers, etc.... In this case you should consider that your scripts are perishable, with a limited life, and you will need to rebuild them at some point.
Having noted the limited lifetime of a piece of code to test a piece of code with a limited lifetime, are there some techniques you can use to insulate yourself? Yes. Rule of thumb is the higher up the stack you go to build your test scripts the more insulated you are from changes under the covers ( assuming the layer you build to is stable ). The trade off is with more of the intelligence under the covers of your test interface, the higher the resource cost for any individual virtual user which then dictates more hosts for test execution and more skew from client side code which can distort the view of what is coming from the server. An example, run a selenium script instead of a base jmeter script. A browser is invoked, you have the benefit of all of the local javascript processing to handle the dynamic changes and your script has a longer life.
If there are many jmeter scripts(jmx files) in a stress project for different cases run, how to maintain the common jmeter code which can be shared by each jmx?
For example, many jmx files have same thread group, http sampler, config elements (header manager, Authorization manager etc) ?
Any way to only edit once and apply for all jmx scripts? I couldn't find the related jmeter doc about this? wonder if jmeter jmx file supports any share code dynamic load/management?
Any suggestions are very appreciated!
My expectation is that you're looking for the Module Controller. In conjunction with Test Fragments you will be able to re-use sequences of the Test Elements either in current Test Plan or even in different .jmx scripts.
See Using JMeter Module Controller article for detailed information if needed
I think you mean the Include Controller. That way you can combine different jmx files to be run from one 'master' file. There are some things to keep in mind, but check the manual for that.
I have a (slightly complex) spring webservice which communicates with multiple frontends via a RESTful API (JSON) and additionally with other devices via SOAP or REST. I'd like setup an automated test environment which is capable of the following things:
create preconditions via fixtures (POSTGRES DB)
send REST or SOAP messages against the API
is able to run certain task (requests against the API) at a specific
time/date
assert and validate the produced results (return of the API call or
check the DB)
run all tests independet from any frontend/UI
integrate the testing environment in my infrastructure (i.e. create a
docker container which runs all tests deployed by Jenkins)
preferably I'd like to build reusable components (i.e. for creating a user that is needed in multiple different tests and so on...). I know there are a lot of tools and frameworks (SoapUI, JMETER,...). But before trying them all and getting lost, I'd like to get an experience report from someone who has a simular setup.
we are using JMeter for API testing. We tried SOAPui but it had some memory issues. So we are pushing forward with JMeter and so far so good.
For your questions:
We are using MySQL, but this post seems to show how to set up a postgres connection in JMeter: https://hiromia.blogspot.com/2015/03/how-to-perform-load-testing-on.html
JMeter can send REST API requests
I'm not sure if this is possible but you could probably have your Jenkins job scheduled to run when you need the API to run the specific task at the specific time.
There are quite a few assertions in JMeter. I use the Response and the BeanShell Assertions a lot.
JMeter is independent from any front end UI which helps pinpoint bugs.
I have not run docker but I am running via Jenkins. This jenkins plugin has been helpful: https://wiki.jenkins.io/display/JENKINS/Log+Parser+Plugin
A few more Tips:
Use the HTTP Request Defaults element. It will save you from having to update all your HTTP requests.
Use the User Defined Variables to define variables you need.
You can combine user defined variables like: ${namePrefix}${myTime} but it will have to be in a 2nd User Defined Variable element(you cant combine them in the same element)
If you have multiple test environments, set up a user defined variable with a value like this: ${__P(testenv,staging)} This way, you can change it from a CLI like this: -Jtestenv=HOTFIX
We are using ExtentReports for pretty html results reports with a custom JSR223 Listener(find my old post on this site).
If your site uses cookies, use the HTTP Cookie Manager.
If you need things to happen in order, on the Test Plan element, check this option: Run Threat Groups consecutively. If you dont, JMeter runs them in a random order.
Hope this is helpful. Happy Testing!
I'm looking to abstract the sequence of REST calls for complicated behaviors in my company's app into a series of classes that are instantiated as needed and the methods would effectively create the sequence of HTTP request calls. It's my hope that doing this would make the tests more compact and readable (as well as providing more reusable code). I would need to utilize the StandardJmeterEngine and export the test to JMX format after the HashTree test plan is created.
To cut on development time, I'm hoping to find a nice example of this; I'm sure someone's done it, but I've yet to stumble onto it.
If you are looking into the way of programmatic creation JMeter test take a look into the following sources:
JMeter API
How to Write a plugin for JMeter
Five Ways To Launch a JMeter Test without Using the JMeter GUI
If you are looking for an example project you can check out jmeter-from-code solution which demonstrates creating a JMeter Test Plan programmatically, storing it into a .jmx script file, running it and getting the .jtl results file.
I am new with JMeter and I am trying to understand how to work with multiple tests plans.
Is it possible to define multiple tests plans in a single ".jmx" file, each with its own thread groups, etc. ? Or should I save each test plan in a separate ".jmx" file?
What is the correct and common usage?
It depends on your workflow and what you are trying to achieve, the most common use case assumes collaborative work of more than one people on the JMeter test scenario. In this case Test Fragments would be useful.
If you are trying to run several JMeter tests in parallel, the easiest option is using Taurus tool as a wrapper and just pass several .jmx tests to it either in command-line or through YAML configuration file.
I think you are looking for an approach for creating a modular JMeter test scripts.
There are many ways to achieve reusability/modularity in JMeter.
Test Fragment
Module Controller
Parameterized Controller
Include Controller
Check this post for more detailed information.
Keyword driven framework in JMeter.
I am here because I'm asking the same thing than you #SomethingSomething
From my understanding and my point of view, to have multiple tests plans, you must create several .jmx file. For instance, if you need to test different servers or application.
If you need to test multiple scenario in a same application, so combining them into in a single .jmx file should be the better option (by using Fragments or Controllers).