I'm trying to use ReportNG with Maven (and Maven Surefire) to generate reports for the results of TestNG tests, within Eclipse. I believe I have followed the instructions here:
http://solidsoft.wordpress.com/2011/01/23/better-looking-html-test-reports-for-testng-with-reportng-maven-guide/
When I run a Maven "site:site", and then look at the generated website, I see the normal TestNG report rather than the ReportNG report.
Poking around the file system, I see that ReportNG has actually created a report, in target/surefire-reports/html. But in target/site, there's a "surefire-report.html" which is the regular TestNG report.
I am guessing that to get the ReportNG report to show up on the generated site, it would have to put an HTML page in target/site, like the regular TestNG report is. How can I make it that happen?
Thanks in advance.
Related
I use Maven to build a multi-component project. One of the components is a web app with HTML and JavaScript. Maven invokes NPM and Karma to run that component's unit and integration tests, written in QUnit. This yields two independent Cobertura coverage reports that I then combine into one with istanbul-combine.
As a final step, I'd like the Maven build to fail if that combined Cobertura report doesn't satisfy my coverage requirements. But how?
I've tried the Cobertura Maven Plugin, but it seems to be unfit to simply read an existing report and apply its check configuration. First, it needlessly repeats the build's tests ("Invokes the execution of the lifecycle phase test prior to executing itself."), then it finds that it cannot instrument the non-Java resources, then it simply stops, without any log output whatsoever.
Are there other plugins that fit? Could changing the report format help? (Doesn't have to be Cobertura.)
Which is the best and featured way to generate reports either by using Maven or ANT? What are the features and differences ? I am using selenium webDriver with TestNG. By using testNG should I prefer "build.xml" or "pom.xml" ?
Maven and ant are build tools. What generates the reports is Testng. So if you use ant or maven, the report generated will still be the same. You can probably do a comparison between maven and ant as build tools, not as a report generation framework, to pick up the one you find better. If you might find this any help.. A blog written long back to describe the ecosystem..http://niharikawhatsnext.wordpress.com/2012/02/15/test-automation-to-building-to-execution-to-scheduling-java/
You can use both maven and ANT to produce XSLT reports which uses TestNg generated output.html. Both are mainly used for build purpose not for report generation.
The XSLT reports generated from maven and ANT both looks same. But there are many advantages in Maven over ANT in testing frame works (I am assuming you know what are those). So I would personally prefer Maven. Please go through the below links which you might found helpful.
http://www.javatpoint.com/difference-between-ant-and-maven
http://blog.varunin.com/2012/05/generate-testng-xslt-report-using-maven.html
In my company we develope eclipse plugins continuously built by a Jenkins CI Server (with tycho & maven) which also executes the tests (Unit Tests and SWTBot Tests). As a post build action we send the test data (JaCoCo & Surefire reports) via maven to a SonarQube server for analysis. Our sources and tests are each located in an own module.
In src/com.mycompany.projectA/ we have the source from Project A.
In test/com.mycompany.projectA.tests.ut/ we have the unit tests from Project A.
In test/com.mycompany.projectA.tests.swtbot/ we have the swtbot tests from Project A.
In order to allow Sonar to find the Jacoco and Surefire reports for the tests of each source plugin we set the report paths in our pom.xml:
<sonar.jacoco.reportPath>${project.basedir}/../../test/${project.artifactId}.tests.ut/target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.surefire.reportsPath>${project.basedir}/../../test/${project.artifactId}.tests.ut/target/surefire-reports</sonar.surefire.reportsPath>
But this only allows one test per plugin. To also see the SwtBot test coverage in SonarQube, we added the SWTBot tests as integration tests (yeah I know, not very neat...)
<sonar.jacoco.itReportPath>${project.basedir}/../../test/${project.artifactId}.tests.swtbot/target/jacoco.exec</sonar.jacoco.itReportPath>
Like this, we can at least see the unit test coverage as well as the SWTBot test coverage in SonarQube. But as we are only able to set one report path for the surefire reports, those are ignored by Sonar and we can not see how many SWTBot test passed/failed.
Is there a convenient way to add multiple locations of surefire reports to be considered in the analysis?
This is not possible yet, but there's a ticket open for that: https://jira.codehaus.org/browse/SONAR-4101
Feel free to watch it and vote for it.
Maybe you can append the jacoco.exec of each project together and put the integrated jacoco report to Sonar. The following link is a good example to integrate the jacoco report.
http://www.lordofthejars.com/2012/07/jacoco-in-maven-multi-module-projects.html
I created selenium test cases using TestNG. After that I created testng.xml and ran all classes parallely. Now the problem is to generate Selenium HTML Report in Jenkins. I exactly have to do that is mentioned in this link.
For that how to generate Selenium Html Report. I used Maven as build tool.
Selenium (RC) by itself doesn't generate any reports (as far as I know). It relies on the testing framework that it gets coupled with. Since you are using TestNG as your testing framework, TestNG does generate the html reports. You can go to your test-output folder and check for index.html which would be a link to the results. In case you are invoking through maven, check the target\surefire-reports folder for reports.
For integration with Jenkins, you can use the testng plugin (https://wiki.jenkins-ci.org/display/JENKINS/testng-plugin) which would give you the testng results on your job page.
You can also try if the html report generated by TestNG can be parsed by the plugin you mentioned, in case you want to use that plugin only.
Hope it helps.
You can also use Allure Report with pretty view and easy-to-use with maven
Try this http://www.wakaleo.com/thucydides/
This library generates well reports, but needs some changes in code.
I have the following setup. I use maven /soapUI maven plugin /soapUI to execute tests on a webservice. My target is to display the test results in sonar. As there is no real build going on in maven, there is nothing to analyze for sonar (if i understood that correct). Is there a a soapUI report format, that can be reused by sonar? Has anyone made experiences with such a setup?
Thanks.
Why not try and run soapui from Junit?
http://www.soapui.org/Test-Automation/integrating-with-junit.html
Maven will run them automatically as part of the build and Sonar should display the results normally.