I have gone thru multiple posts available for reporting with maven. I'm trying to figure out ways to generate a clear report which has performance data, results and browser details and a good representation of the summary. So far, I've not been able to find out any posts which are helpful. If someone can point me to a way to generate reports just like XSLT format, that would be awesome.
any help would be greatly appreciated!
Thank you,
NickJS
There is no straightforward solution as far as I'm concerned, however you may find following helpful:
If you're not using any testing framework you may collect reports from surefire as described in here: http://maven.apache.org/surefire/maven-surefire-report-plugin/usage.html
Also, you may want to report browsers related info using WebDriver's logs as described in here:
How to obtain native logger in Selenium WebDriver
If you're using testing frameworks there is a little more you can get out of them, for example using testNG you may add listeners to your tests and report everything as html using following reporters:
http://reportng.uncommons.org/
OR
http://extentreports.com/
If you consider using BDD approach there is a nice Cucumber Json reporter for Jenkins, also available as maven dependency:
https://github.com/jenkinsci/cucumber-reports-plugin
Hope you'll find this helpful.
Related
I did some research, this is my second time I try to tackle this problem and still couldn't find a proper tutorial for ES plugin writing.
https://www.elastic.co/guide/en/elasticsearch/plugins/current/plugin-authors.html
I found this page that shows some plugin structure and links some plugin examples, but when I try and create those java projects I get a load of errors in it and I get nowhere.
Some other pages show maven project examples but none have been clear enough, either I got errors or some steps were missing.
Does anyone know a step-by-step tutorial page for ES plugin writing? I have Java coding knowledge and a few months of ES experience in smaller environment.
The closest thing I've found is their Elasticsearch Plugins and Integrations
page, which doesn't help. I was following their out of date tutorial to make a "hello world" plugin, but I can't find any documentation on the classes used. Is there a page with information on what each class does, or do I have to look at the code to figure it out on my own?
To quote elastic search developer Adrien Grand(jpountz) "No, there is no guide about writing plugins and the API is actually quite unstable. The plugin API is mainly a way for us to provide additional functionality through plugins so that we do not have to fold everything into a single release artifact that would be quite huge. Some community membors have writter plugins by taking inspiration of existing plugins but we do not want to commit on a stable API for plugins as this might prevent us from improving other areas of elasticsearch."
The only real way of learning about developing a plugin is trying to find something similar and modifying it to suite your needs. Developing anything remotely complex will require an extensive knowledge of the ES codebase. I suggest you look into some plugins developed by jprante to get started.
We are looking at cucumber for our automation test framework because everyone including business people can understand it.
We use Angualr JS frontend and Java REST backend. Our team that is going to write the step definitions likes Ruby so we want to stick with Ruby for that.
Also we would like to use Maven to tie this process into our build process.
Will cucumber be a good fit given that story above ?
Hui Peztherez, from my prospective cucumber is a great choice, using it with the same architecture expect for Angular.
We are using Maven too, and it's so useful to orchestrate them with Jenkins, using maven to run the tags..
mvn test -Dcucumber.options="--tags #smoke"
ref: https://cucumber.io/docs/reference/jvm
Also Jenkins have several plugin to report the Cucumber Analysis, so useful for testers, and in the end, we are now working about the HPQ server integration with a plugin called Bumblebee (this part is still under development for both sides, our and bumblebee)
Another good choice is Ruby, you can take the step definition so easily defined with Ruby...
We also have a integration with Selenium for the front end side, and it works as well...
So go further!
We are using Cucumber in Java with gradle in past, It was in Maven and It works fine. We have framework for UI and API, In UI we used WebDriver to write step definition and In API, We used RestAssured to write step definition. You can do same thing in Java what you can do in Ruby.
Maven for Java Cucumber :
http://mvnrepository.com/artifact/info.cukes/cucumber-java/1.2.4 - Please add other dependency as per requirement.
Jenkin Plugin : https://wiki.jenkins-ci.org/display/JENKINS/Cucumber+Reports+Plugin
Will cucumber be a good fit given that story above ?
- Yes It is good fit. I will request you to show POC(Proof of concept) to management. I had experience in past that management have no clue about BDD and they have very hard to time to understand coverage. We did very deep dive to provide all information to them. It is very important to answer following question to management
BDD report is providing accurate test converage idea to management ?
Everyone in team is able to write feature file and able to provide same quality of feature file
Feature file and BDD report will be starting tool for check test converage
Thank you.
Please be aware that Cucumber is a BDD framework that can be used on top of a browser automation framework like Selenium WebDriver/Watir/Protractor they are two distinct things. Most of them implements Selenium WebDriver's protocol.
My only concern is for you using Maven in that project setup, I know that you can run ruby code in a JVM by using JRuby. But I'm not sure which plugin you'd use to trigger that from Maven.
I would like to integrate SourceMonitor into my Jenkins CI-system. Since there is no SourceMonitor plugin how can i make the results of SourceMonitor visible on my Jenkins Server.
There is an open issue associated with the Violations plugin.
You can vote up for the implementation of this issue: https://issues.jenkins-ci.org/browse/JENKINS-5741
I guess you're out of luck. You could take a look at similar plugins (probably FindBugs, PMD, Checkstyle plugins should be comparable) and write a plugin yourself.
You could also give a try to post to the Jenkins user mailing list, maybe someone there can help you.
Don't know what kind of output SourceMonitor produces, but if it produces HTML Reports, there is an HTML Publish Plugin, which would let you link and display those results in your job. Not as integrated as a real plugin would be, but perhaps that would be enough for you.
You only realistically have the following options:
Create a Hudson plugin for SourceMonitor yourself.
Contact the SourceMonitor development team and ask them to create a Hudson plugin.
Use an existing Hudson plugin that is similar to SourceMonitor.
I'm interested in using the Lightweight Testing Automation Framework (LTAF) to create integration tests for my web application. However, I need it to be run on the build server. Does anyone know if this can set up to do this?
There does not seem to be a whole lot of information on the web on this right now :-)
There's a blog post (Lightweight Test Automation Framework – Automated Build Support) that describes how to implement the runner as a console application, it can then be integrated into a build server fairly simply by redirecting the build output and setting the return code appropriately.
The author posted the code used in the article, you can download it from here.
I don't have experience with LTAF, but found this nice article:
First steps with Lightweight Test Automation Framework
Quoting the author (Steve Sanderson, from the comments):
Lightweight Test Automation Framework
can be invoked with query string
parameters (to specify which tests to
run) and can emit a log of the results
to a text file, so it would be
possible to integrate it. However,
this is certainly not as easy as using
Selenium RC which as you say works
through a traditional test runner.