In our company, we use a framework for test automation created with Jmeter.
All the test cases are loaded by .csv files in Jmeter and then executed: The results are written in a separated .csv file.
After this, I have to import the results file into excel so I can create statistics and reports.
I'm looking for a tool that can import the csv files and create statistics automatically (better if web-based). A tool that integrates with Jira/Zephyr would be great - something that directly integrates JMeter output into Jira/Zephyr and links the results to test cases would be perfect.
Are there any tools that can integrate JMeter with Jira/Zephyr, and if there are, what are their advantages and disadvantages?
Jira has buch of REST APIs you can use to create a task after your performance test and attach what ever you want (i.e jmeter results) to the task.
I'm looking for a tool that can import the csv files and create statistics automatically (better if web-based)
Since JMeter 3.0 it's possible to generate HTML Reporting Dashboard out of .jtl/.csv result file
There is BM.Sense online JMeter (and more) results analysis service.
Are there any tools that can integrate JMeter with Jira/Zephyr, and if there are, what are their advantages and disadvantages?
Jira has web APIs so you should be able to perform any operation you need by sending simple HTTP Requests to the JIRA API endpoints.
There is Runscope Zapier which can integrate with a lot of external services including but not limited to Jira, Bitbucket, Github, Trello, etc.
Related
I am trying to create a custom test report using Maven as my build tool, JUnit as my framework, along with selenium test cases. I was using maven's surefire report plugin but I need to include more information in my report. Can anyone direct me to a good tutorial on how to create a custom maven reporting tool?
We have a much better test automation dashboard based on just few api calls, ARES dashboard (built under Testastra and owned by ZenQ) is a much better option to try and it's absolutely free.
ARES, is an acronym for Test Automation Results dashboard. It's a TestAutomation framework/tool agonistic solution, that simplifies the collection of Test automation results and their analysis via live dashboard, daily/weekly trends, frequent failures etc. Website: http://www.testastra.com/#ares
Below repo has some code samples, documentation and usage of ARES test automation dashboard:https://github.com/testastra/ARES
Give it a try.
I have a general question about Jmeter. I am using Jmeter and I want to create Continuous integration solution using Jmeter. The problem is that there are several QA, that each one of them have test plans according his development. the problem is that if using Jmeter all test plan need to run as one test plan, and for each of test plan to create report, and than manually over each report and check for failed scenarios. So all the handling is much harder than I thought, Jmeter is wonderful solution but how to use it with several test plans, and if I want it to run nightly? and in the mornng to get single report with all test plans that created? Can someone please advise if Jmeter can be solution for CI nightly? is it possible to merge all reports to single report?, each test plan is for unique development, and we will have multiple testplans over time
Did you have a look at those plugins:
https://github.com/jmeter-maven-plugin/jmeter-maven-plugin which allows you to fail a build based on presence of errors in load test results which you could mix with https://jmeter-plugins.org/wiki/AutoStop/
https://plugins.jenkins.io/performance
JMeter per se is just a load testing tool, you need a continuous integration server. If you don't have one in your company there are several free and open source solutions you can consider like:
Jenkins
Buildbot
Cruise Control
Check out Jenkins vs. Other Open Source Continuous Integration Servers article to learn more about aforementioned tools, see sample build dashboards and commands to kick off a JMeter test.
I'm looking to abstract the sequence of REST calls for complicated behaviors in my company's app into a series of classes that are instantiated as needed and the methods would effectively create the sequence of HTTP request calls. It's my hope that doing this would make the tests more compact and readable (as well as providing more reusable code). I would need to utilize the StandardJmeterEngine and export the test to JMX format after the HashTree test plan is created.
To cut on development time, I'm hoping to find a nice example of this; I'm sure someone's done it, but I've yet to stumble onto it.
If you are looking into the way of programmatic creation JMeter test take a look into the following sources:
JMeter API
How to Write a plugin for JMeter
Five Ways To Launch a JMeter Test without Using the JMeter GUI
If you are looking for an example project you can check out jmeter-from-code solution which demonstrates creating a JMeter Test Plan programmatically, storing it into a .jmx script file, running it and getting the .jtl results file.
Is it possible to record JMeter scripts for performance testing Siebel?
Siebel uses a custom format for ENCODING its requests plus some necessary technical parameters.
So it's possible provided it's the http module but rather hard job as there are a lot of correlations to do on technical IDs.
For recording, see this:
JMeter record Siebel CRM System
You can use this commercial plugin that will do auto-correlation for you:
https://ubikloadpack.com/
Yes, it is possible to test Siebel with JMeter. I have done so for one of my projects with my client TataSky where I have used JMeters distributed testing approach for generating upto 2000 users with 4 - 5 normal desktops. Siebel uses http protocol so it is finally about understanding the requests sent by browser and responses by the web-server and correlating between them. It gets really messy though, so my suggestion is correlate parameters which are mandatory.
You can use JMeter-Siebel-plugin which provides Siebel HTTP(S) Test Script Recorder test element handy for capturing and correlating Siebel traffic.
The plugin can be installed using JMeter Plugins Manager
Check out Introducing the Siebel CRM Correlation Plugin in JMeter article for comprehensive instructions.
I want to benchmark a new server using historical HTTP-request data. I have a textfile that contains one day's worth of real historical requests to a production server. What is the best tool for sending that list of requests on the server I'm testing? The tool I use should be able to configure the following:
Number of threads making the requests
Number of requests/second sent
A list of request URLs to use when making the requests.
Apache Bench seems like a close fit. However, Bench does not seem to be able to take in a list of request URLs as a parameter. What would you recommend?
I have been using http_load to pretty good success.
http://acme.com/software/http_load/
Http_load is a Unix command line tool that allows you to specify the number of requests/second and the number of threads to use when running. It pulls urls from a text file that you specify in the command. The tool is very similar to Apache Bench, with the big difference being that http_load allows you to use a list of URLs to be used when making requests. Apache Bench makes request to a single URL only.
If all your requests are GET requests (no POST) then you can use the JMeter Access Log Sampler. Here are some straight forward step-by-step instructions on how to set it up. It will run through your requests either in order or using a number of concurrent threads and you can specify how many requests should run. Then you can use the other features of JMeter like reports to analyze the results.
I would reccomend Visual Studio Test Edition. It would be a relativley simple matter to create a coded webtest that loads your url's for testing.
This advice predicates a knowledge of C# or VB for coding and the ability to install and licence Visual Studio. Visual Studio does have a trial edition available so you can have a taste of what you are getting first.
Visual Studio does not require the target site to be running any particular hardware or software, but it does provide more information on the load of the server due to the use of Perfmon counters and any ASP.Net application will provide more detail on the running app.
The caveat to this is that I have not actually used any other web testing software.