I have a general question about Jmeter. I am using Jmeter and I want to create Continuous integration solution using Jmeter. The problem is that there are several QA, that each one of them have test plans according his development. the problem is that if using Jmeter all test plan need to run as one test plan, and for each of test plan to create report, and than manually over each report and check for failed scenarios. So all the handling is much harder than I thought, Jmeter is wonderful solution but how to use it with several test plans, and if I want it to run nightly? and in the mornng to get single report with all test plans that created? Can someone please advise if Jmeter can be solution for CI nightly? is it possible to merge all reports to single report?, each test plan is for unique development, and we will have multiple testplans over time
Did you have a look at those plugins:
https://github.com/jmeter-maven-plugin/jmeter-maven-plugin which allows you to fail a build based on presence of errors in load test results which you could mix with https://jmeter-plugins.org/wiki/AutoStop/
https://plugins.jenkins.io/performance
JMeter per se is just a load testing tool, you need a continuous integration server. If you don't have one in your company there are several free and open source solutions you can consider like:
Jenkins
Buildbot
Cruise Control
Check out Jenkins vs. Other Open Source Continuous Integration Servers article to learn more about aforementioned tools, see sample build dashboards and commands to kick off a JMeter test.
Related
Please could someone suggest:
I need to performance test a few APIs. Which of the below methods will be better way of conducting the tests (trying to understand the pros and cons of various ways to run JMeter tests):
Create Ultimate Concurrency Thread groups using only Jmeter.
Buy Blazemeter, add the Jmeter scripts, create scenario and run the tests
Run the JMeter tests using Maven plugin as part of the build.
Thanks,
N
Testing the performance of an API can be achieved by the above mentioned 3 ways.
Open Source: (Standalone JMeter/Taurus)
Cloud based solutions (Octoperf/Blazemeter)
If you are looking for an open source tool + you have your LG machine allocated + not gonna test with massive number of users - I would suggest to go with standalone JMeter/ taurus for your load testing solution.
If you are looking for an hassle free environment to manage all your test, LGs, reports, monitoring, etc then cloud based solutions like Blazemeter/Octoperf seems promising. This is going to cost you money and requires a port needs to be open for the solution to communicate with AUT.
For Scripting: You can use the same JMeter scripting in all 3 cases. Choosing a thread group requires what kinda test you wanna perform on your microsevrices.
In any case you will be using JMeter so points 2 and 3 don't really differ. The pro is that you don't have to pay. The con is that you need to manage your infrastructure yourself.
BlazeMeter provides easy way of launching a JMeter test so you don't have to worry about creating and configuring machines, installing JMeter, setting up distributed testing, real time monitoring and results comparison.
JMeter Maven Plugin makes the following processes easier:
installing JMeter
generating reporting dashboard
returning non-zero exit status code when acceptance criteria are not met
I am trying to create a custom test report using Maven as my build tool, JUnit as my framework, along with selenium test cases. I was using maven's surefire report plugin but I need to include more information in my report. Can anyone direct me to a good tutorial on how to create a custom maven reporting tool?
We have a much better test automation dashboard based on just few api calls, ARES dashboard (built under Testastra and owned by ZenQ) is a much better option to try and it's absolutely free.
ARES, is an acronym for Test Automation Results dashboard. It's a TestAutomation framework/tool agonistic solution, that simplifies the collection of Test automation results and their analysis via live dashboard, daily/weekly trends, frequent failures etc. Website: http://www.testastra.com/#ares
Below repo has some code samples, documentation and usage of ARES test automation dashboard:https://github.com/testastra/ARES
Give it a try.
I am just wondering is there any way or possible to generate JMeter script on the fly for the existing test automation script.
Basically the idea is to use the same Test automation script let's say Test automation script was created using selenium for the performance testing too using JMeter.
Any ideas or thoughts?
-Raj
Thanks,
Rajani
The solution is to record your Selenium test.
Start JMeter's HTTP(S) Test Script Recorder
Configure your Selenium scripts to use JMeter as a proxy
Start the test - JMeter will be recording.
This way you will get test "skeleton" however you will still need to handle dynamic parameters. To avoid this step you can try out an alternative recording solution, it is capable of exporting JMeter tests in "SmartJMX" mode with automated correlation applied. See How to Convert Selenium Scripts into the JMX article for more details.
The only way I am aware to achieve this is to record the selenium run and then save the jmeter script.
https://guide.blazemeter.com/hc/en-us/articles/206733979-Recording-a-user-session-with-the-proxy
Yes, it is.
You can run the JMeter stuff programmatically using JMeter API.
Using it, you can write your own test harness that creates the test items on the fly, then build the test plan out of them, and run it.
Or loads existing plan "template", access & alter the test elements in it, then, again run it.
Though that is kinda tricky part, so first ask yourself (then share with us here) what exactly do you need it for, and then try to consider the possibility to implement that using JMeter scripting components.
Basically, in your case, you can call the Selenium stuff in the JSR223/Beanshell components through Selenium API.
And moreover, there exist a plugin to do the things with Selenium stuff.
PS Although the very idea of using something all-purpose "to rule 'em all" usually doesn't pay off, 'cause the effort needed to maintain that "universality" often overcome the one smartly made for separate functional/load suites.
I am new with JMeter and I am trying to understand how to work with multiple tests plans.
Is it possible to define multiple tests plans in a single ".jmx" file, each with its own thread groups, etc. ? Or should I save each test plan in a separate ".jmx" file?
What is the correct and common usage?
It depends on your workflow and what you are trying to achieve, the most common use case assumes collaborative work of more than one people on the JMeter test scenario. In this case Test Fragments would be useful.
If you are trying to run several JMeter tests in parallel, the easiest option is using Taurus tool as a wrapper and just pass several .jmx tests to it either in command-line or through YAML configuration file.
I think you are looking for an approach for creating a modular JMeter test scripts.
There are many ways to achieve reusability/modularity in JMeter.
Test Fragment
Module Controller
Parameterized Controller
Include Controller
Check this post for more detailed information.
Keyword driven framework in JMeter.
I am here because I'm asking the same thing than you #SomethingSomething
From my understanding and my point of view, to have multiple tests plans, you must create several .jmx file. For instance, if you need to test different servers or application.
If you need to test multiple scenario in a same application, so combining them into in a single .jmx file should be the better option (by using Fragments or Controllers).
I am working on a project where the core application needs a harness to send messages which will process the message and feed to a portal web application. I am writing the automated testing for the portal application using cucumber/ruby/capybara.
I have never used Jmeter but through a colleague i just understand the basics of the Jmeter GUI tool. I have been asked me to use Jmeter as harness tool so that I can later extend the functionality to do performance testing as well. How can I do this ? How can i use Jmeter in my automated scripts using cucumber/ruby/selenium or capybara to do my testing ?
I dont want to use GUI as the whole test case should be automated end to end, I have to figure out a way to integrate Jmeter functionality to do test data set up and then run my browser test to check whether the data exists.
i hope my question is clear, ask me if you are not. Hoping to get a solution as i dont think it is a uncommon question.
Note: i have seen the ruby-jmeter gem, i think it does partly what i need but it is actually a DSL to use jmeter in ruby.the problem with that is I have to stick the syntax of gem and have to figure all the options in gem. I have also read a blog that this can be done using Jmeter recording proxy and running the browser tests at the proxy which I am not sure how to do that. here is the link where this is mentioned:
https://groups.google.com/forum/#!topic/cukes/4ZGkf3a234Y
As per my understanding of the problem, please go through the below links
for recording the script using Jmeter:
http://jmeter.apache.org/usermanual/jmeter_proxy_step_by_step.pdf
for integrating selenium in Jmeter:
There are actually 2 ways of achieving this:
exporting all your selenium scripts as *.jar and copying it to /junit folder under /lib folder of Jmeter.
Use JUnit Sampler to execute your test cases:
http://intensetesting.wordpress.com/2013/09/25/integrating-jmeter-with-selenium-code/
use WebDeriver plugin for Jmeter and write your selenium test cases in Jmeter:
http://jmeter-plugins.org/wiki/WebDriverTutorial/
Also you can run Jmeter via command line.