Maven complete surefire-report including passed tests logging events - maven

I want to generate a report using surefire-report plugin of maven which will give me a detailed report of all the tests that ran with the log events that were logged during each test run instead of just the events where my test failed. This will also give me the full context where my test failed.
Here is my pom.xml file:
<artifactId>SampleTests</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<name>sampleTests</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.4</version>
</dependency>
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.11</version>
<type>maven-plugin</type>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20090211</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/java</sourceDirectory>
<outputDirectory>${basedir}/target/classes</outputDirectory>
<testSourceDirectory>src/main/test</testSourceDirectory>
<testOutputDirectory>${basedir}/target/test-classes</testOutputDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<includes>
<include>com/mypackage/**</include>
</includes>
<test>**/*Tests</test>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
</plugin>
</plugins>
</build>
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.6</version>
<reportSets>
<reportSet>
<reports>
<report>report-only</report>
</reports>
</reportSet>
</reportSets>
</plugin>
</plugins>
</reporting>
</project>
And i use mvn surefire-report:report to generate the report here.
So I want to know if there is anything additional that i can do so as to see a detailed report of what happened during the tests run?

As a reference from the maven site
surefire-report:report Generates the test results report into HTML format.
surefire-report:report-only This goal does not run the tests, it only builds the reports.
You are trying the report-only. Try the other one report as a goal

I ended up using ant to get to what I needed. Thanks to this blog: http://blog.varunin.com/2010/03/ant-script-for-generating-junit-report.html#comment-form
If you want to get a detailed report of all the test events then please use this

Related

Javadoc Plugin Module Error with Site target

I'm trying to use the site target for maven to generate javadoc, but I'm getting inconsistent results. I've created a stripped down sample project to test with, and when I execute the site target along with a build target (e.g., mvn package site, or mvn compile site) it works fine. But if I execute with just the site target (i.e., mvn site) I get the error:
Exit code: 1 - error: module not found: foo.bar
I've checked the generated options files, and when a build target is included the --module-path argument includes my target/classes path, but with just the site target that path isn't included.
Does anyone know how the javadoc plugin configures those arguments and how I can get it to work both ways?
Here's my test pom.xml file:
<modelVersion>4.0.0</modelVersion>
<groupId>foo.bar</groupId>
<artifactId>MyProgram</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<name>MyProgram</name>
<description>
This is a sample program.
</description>
<inceptionYear>2020</inceptionYear>
<properties>
<java.source>11</java.source>
<java.target>11</java.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<clean.plugin.version>3.1.0</clean.plugin.version>
<compiler.plugin.version>3.8.0</compiler.plugin.version>
<dependency.plugin.version>3.1.2</dependency.plugin.version>
<install.plugin.version>3.0.0-M1</install.plugin.version>
<jar.plugin.version>3.2.0</jar.plugin.version>
<javadoc.plugin.version>3.2.0</javadoc.plugin.version>
<resources.plugin.version>3.1.0</resources.plugin.version>
<surefire.plugin.version>3.0.0-M3</surefire.plugin.version>
<versions.plugin.version>2.7</versions.plugin.version>
<jakarta.version>2.3.2</jakarta.version>
<project.site.root.path>${CATALINA_HOME}/webapps/projects</project.site.root.path>
</properties>
<distributionManagement>
<site>
<id>foo.bar</id>
<url>file:${project.site.root.path}/${project.artifactId}</url>
</site>
</distributionManagement>
<dependencies>
<!-- https://mvnrepository.com/artifact/jakarta.xml.bind/jakarta.xml.bind-api -->
<dependency>
<groupId>jakarta.xml.bind</groupId>
<artifactId>jakarta.xml.bind-api</artifactId>
<version>${jakarta.version}</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.junit.jupiter/junit-jupiter-engine -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.7.0-M1</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.junit.platform/junit-platform-commons -->
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-commons</artifactId>
<version>1.7.0-M1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-dependency-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>${dependency.plugin.version}</version>
</plugin>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-resources-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>${resources.plugin.version}</version>
</plugin>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-site-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.9.0</version>
<configuration>
<locales>en</locales>
<generateSiteMap>true</generateSiteMap>
</configuration>
</plugin>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-javadoc-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>${javadoc.plugin.version}</version>
<configuration>
<source>${java.source}</source>
<show>protected</show>
<nohelp>true</nohelp>
<detectLinks>false</detectLinks>
<detectOfflineLinks>false</detectOfflineLinks>
<detectJavaApiLink>true</detectJavaApiLink>
<failOnWarnings>true</failOnWarnings>
<additionalparam>${javadoc.opts}</additionalparam>
<sourcepath>${project.basedir}/src/main/java</sourcepath>
<subpackages>${javadoc.package}</subpackages>
<additionalJOption>-Xdoclint:all</additionalJOption>
<links>
<link>https://javadoc.io/doc/jakarta.xml.bind/jakarta.xml.bind-api/${jakarta.version}/</link>
</links>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-clean-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-clean-plugin</artifactId>
<version>${clean.plugin.version}</version>
</plugin>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-jar-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>${jar.plugin.version}</version>
</plugin>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-install-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>${install.plugin.version}</version>
</plugin>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-surefire-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.plugin.version}</version>
</plugin>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-checkstyle-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<configLocation>google_checks.xml</configLocation>
<encoding>${project.build.sourceEncoding}</encoding>
<consoleOutput>true</consoleOutput>
<failsOnError>false</failsOnError>
<failsOnViolation>true</failsOnViolation>
<linkXref>false</linkXref>
</configuration>
</plugin>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-compiler-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${compiler.plugin.version}</version>
<configuration>
<source>${java.source}</source>
<target>${java.target}</target>
<showWarnings>true</showWarnings>
<encoding>${project.build.sourceEncoding}</encoding>
</configuration>
</plugin>
</plugins>
</build>
<reporting>
<plugins>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-project-info-reports-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-javadoc-plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<reportSets>
<reportSet>
<reports>
<report>javadoc</report>
</reports>
</reportSet>
</reportSets>
</plugin>
<!-- https://mvnrepository.com/artifact/org.codehaus.mojo/versions-maven-plugin -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>versions-maven-plugin</artifactId>
<version>${versions.plugin.version}</version>
<reportSets>
<reportSet>
<reports>
<report>dependency-updates-report</report>
<report>plugin-updates-report</report>
<report>property-updates-report</report>
</reports>
</reportSet>
</reportSets>
</plugin>
</plugins>
</reporting>
To make matters even more confusing, I have another project that behaves exactly the opposite way: mvn site works correctly, but mvn package site creates the error:
Exit code: 2 - javadoc: error - No source files for package org.larrick.commons
Any insights on this would be greatly appreciated.
BTW: I'm using JDK 11 and Maven 3.6.3.
This appears to be an issue with the Javadoc plugin. Without the build step there's no artifact defined for the project, so the target/classes path (or the target .jar file) isn't added to the --module-path argument for Javadoc.
I'm reaching out to the Maven plugin team to identify a fix.

Parallel execution with maven-surefire-plugin is throwing PluginResolutionException

I am trying to execute my feature files via TestRunner.java files (by mentioning them in pom.xml) , in parallel using maven-surefire-plugin, for which i have set up pom.xml as below, but when i run pom.xml as maven test, its throwing PluginResolutionException when the version is 3.0.0-M3, when i have update the version to 2.19.1, the maven test is not running my feature files but the build is shown as successful
I have tried with different versions but not worked
Also I have tried replacing the configuration part with below changes
still my feature files are not executed but the build is
successful
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<!--
<parallel>classes</parallel>
<forkMode>perthread</forkMode>
<threadCount>3</threadCount>
-->
<argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
<includes>
<include>**/*TestRunner.java</include>
</includes>
</configuration>
PS: After reading the below artical
https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html#Running_tests_in_parallel
I understand that there is link between the Junit version and surefireflugin i use in my project, bow one thing is for sure, the correct combination of Junit and maven-surefire-plugin is very much necessary, i have tried with below combinations
JUnit 4.7
plugin 3.0.0-M3
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>3.0.0-M3</version>
</dependency>
</dependencies>
</plugin>
JUnit 4.12
plugin 2.20
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<parallel>classes</parallel>
<threadCount>3</threadCount>
<argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
<includes>
<include>**/*TestRunner.java</include>
</includes>
</configuration>
</plugin>
but its not helpful, I suppose i am doing mistake in choosing this versions and the config of plugin with proper parameters, please help me
My complete pom is as below
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.practise.raja</groupId>
<artifactId>SeleniumConcepts</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>4.7.1</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-picocontainer</artifactId>
<version>4.7.1</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>3.5.3</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<fork>true</fork>
<executable>C:\Program Files (x86)\Java\jdk1.8.0_211\bin\javac</executable>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<parallel>classes</parallel>
<forkMode>perthread</forkMode>
<threadCount>3</threadCount>
<argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
<includes>
<include>**/*TestRunner.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</project>
Expected:
My feature files should run in parallel
Actual:
My feature files are not executed
After changing the dependencies and plugin as suggested by sureshmani, this is how it looks
Finally i am able to solve this, to my initial pom, have started doing below changes. Which ran my feature files in parallel
Change 1: I happen to add the dependency for cucumber-jvm-parallel-plugin along with plugin ,so i have deleted the plugin
Change 2: I have realized that the cucumber-jvm-parallel-plugin is not able to recognize the feature files when I have them placed src/main/java , some of the posts said i have to move all feature files to src/main/resources/feature , where features is package
Change 3: I have realized that cucumber-jvm-parallel-plugin is not able to recognize the resources like step def's and drivers etc, so i have used build-helper-maven-plugin where i have declared the resources as below
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/main/java/</source>
<source>src/main/resources/</source>
<source>features</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
Change 4: cucumber-jvm-parallel-plugin in the maven life cycle in my IDE, because for some reason maven is not able to consider this plugin in it execution
Eclipse --> Windoes --> Preferences --> Maven->LifeCycleMappings-> copy paste below code
<pluginExecution>
<pluginExecutionFilter>
<groupId>com.github.temyers</groupId>
<artifactId>cucumber-jvm-parallel-plugin</artifactId>
<goals>
<goal>generateRunners</goal>
</goals>
<versionRange>[4.2.0,)</versionRange>
</pluginExecutionFilter>
<action>
<ignore/>
</action>
</pluginExecution>
Then , click on "Reload workspace lifecycle mapping metadata" button in Preference
Maven modal
My final pom looks like this
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.practise.raja</groupId>
<artifactId>JUnitCucumberParallelExecutionPractise</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>JUnitCucumberParallelExecutionPractise</name>
<description>JUnitCucumberParallelExecutionPractise</description>
<dependencies>
<!-- <dependency>
<groupId>com.github.temyers</groupId>
<artifactId>cucumber-jvm-parallel-plugin</artifactId>
<version>5.0.0</version>
</dependency>
-->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>4.7.1</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-picocontainer</artifactId>
<version>4.7.1</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>3.5.3</version>
</dependency>
</dependencies>
<properties>
<src.main.java>src/main/java</src.main.java>
</properties>
<build>
<resources>
<resource>
<directory>${src.main.java}</directory>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/main/java/</source>
<source>src/main/resources/</source>
<source>features</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<fork>true</fork>
<executable>C:\Program Files (x86)\Java\jdk1.8.0_211\bin\javac</executable>
</configuration>
</plugin>
<plugin>
<groupId>com.github.temyers</groupId>
<artifactId>cucumber-jvm-parallel-plugin</artifactId>
<version> 4.2.0</version>
<executions>
<execution>
<id>generateRunners</id>
<phase>generate-test-sources</phase>
<goals>
<goal>generateRunners</goal>
</goals>
<configuration>
<featuresDirectory>src/main/resources/features</featuresDirectory>
<glue>
<package>com.qa.stepdef</package>
</glue>
<outputDirectory>${project.build.directory}/generated-test-
sources/cucumber</outputDirectory>
<cucumberOutputDir>${project.build.directory}</cucumberOutputDir>
<format> json </format>
<strict>true</strict>
<monochrome>false</monochrome>
<useTestNG>false</useTestNG>
<namingScheme>simple</namingScheme>
<namingPattern>Parallel{c}IT</namingPattern>
<parallelScheme>FEATURE</parallelScheme>
</configuration>
</execution>
</executions>
</plugin>
<!-- Specify a custom template for the generated sources (this is a path
relative to the project base directory) -->
<!-- <customVmTemplate>src/test/resources/custom-runner-template.java.vm
</customVmTemplate> -->
<!-- Specify a custom package name for generated sources. Default is no
package. -->
<!--<packageName></packageName> <plugins> <plugin> <name>json</name>
</plugin>
<plugin> <name>html</name> </plugin> <plugin> <name>pretty</name> </plugin>
</plugins> -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<includes>
<include>**/*IT.class</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</project>
Even after these changes, an error started showing at <execution> in pom, it says
Plugin execution not covered by lifecycle configuration: com.github.temyers:cucumber-
jvm-parallel-plugin:4.2.0:generateRunners (execution: generateRunners, phase:
generate-
But, its fine, i am able to run the feature files in parallel even with this above
error
test-sources)

SpotBugs site report inconsistent with spotbugs:gui goal

I have set up SpotBugs to help us adhere to some standards, but I get different results in my generated site compared to SpotBugs user interface. This is my configuration of SpotBugs in my pom file:
<build>
<plugins>
<plugin>
<groupId>com.github.spotbugs</groupId>
<artifactId>spotbugs-maven-plugin</artifactId>
<version>${spotbugs-maven-plugin.version}</version>
<configuration>
<excludeFilterFile>
src/test/resources/SpotBugsExcludeFilter.xml
</excludeFilterFile>
</configuration>
<dependencies>
<dependency>
<groupId>com.github.spotbugs</groupId>
<artifactId>spotbugs</artifactId>
<version>${spotbugs.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
...
<reporting>
<plugins>
<plugin>
<groupId>com.github.spotbugs</groupId>
<artifactId>spotbugs-maven-plugin</artifactId>
<version>${spotbugs-maven-plugin.version}</version>
<configuration>
<excludeFilterFile>
src/test/resources/SpotBugsExcludeFilter.xml
</excludeFilterFile>
</configuration>
</plugin>
</plugins>
</reporting>
When I run mvn clean install site, the site reports 5 bugs.
When I run mvn clean install spotbugs:gui, the gui reports 0 bugs.
How come there is a difference? It seems as if the site goal disregards my exclusion filter, but I can't see why. I am also not entirely sure how the build/plugins section correlates with the reporting/plugins section. If someone could tell me what I'm messing up here I'd be very grateful.
spotbugs:gui is not triggering the reporting scope where your configuration is defined.
You should be specifying your SpotBugs configuration in the <build> section.
<build>
<plugins>
[...]
<!-- SpotBugs Static Analysis -->
<plugin>
<groupId>com.github.spotbugs</groupId>
<artifactId>spotbugs-maven-plugin</artifactId>
<version>${spotbugs-maven-plugin.version}</version>
<configuration>
<excludeFilterFile>src/test/resources/SpotBugsExcludeFilter.xml</excludeFilterFile>
</configuration>
</plugin>
</plugins>
</build>
https://github.com/find-sec-bugs/find-sec-bugs/wiki/Maven-configuration

Running "mvn test site" giving [ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project

I'm trying to generate xslt report using reporty-ng with Testng+maven but getting error as
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project Selenium Project: failed to get report for org.reportyng:reporty-ng: Plugin org.reportyng:reporty-ng:1.2 or one of its dependencies could not be resolved: Failure to find org.reportyng:reporty-ng:jar:1.2 in https://github.com/cosminaru/reporty-ng/raw/master/dist/maven was cached in the local repository, resolution will not be reattempted until the update interval of reporty-ng has elapsed or updates are forced -> [Help 1]
Below is the part of the pom.xml:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.48.1</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.9.6</version>
</dependency>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-api</artifactId>
<version>2.19</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.5</version>
</dependency>
</dependencies>
<reporting>
<plugins>
<!-- TestNG-xslt related configuration. -->
<plugin>
<groupId>org.reportyng</groupId>
<artifactId>reporty-ng</artifactId>
<version>1.2</version>
<configuration>
<!-- Output directory for the testng xslt report -->
<outputDir>/target/testng-xslt-report</outputDir>
<sortTestCaseLinks>true</sortTestCaseLinks>
<testDetailsFilter>FAIL,SKIP,PASS,CONF,BY_CLASS</testDetailsFilter>
<showRuntimeTotals>true</showRuntimeTotals>
</configuration>
</plugin>
</plugins>
</reporting>
<pluginRepositories>
<pluginRepository>
<id>reporty-ng</id>
<url>https://github.com/cosminaru/reporty-ng/raw/master/dist/maven</url>
</pluginRepository>
</pluginRepositories>
In my understanding of this, if you run "mvn clean verify", that will include the "site" phase of the build as well as the tests. You shouldn't need to explicitly call the site phase and so don't do it. Also, if you were to explicitly call the "site" phase, you should probably include the site plugin as a dependency, don't you think?
In my experience with both TestNG reports and ReportNG reports, the site plugin is not related to the report output. The site plugin is only related to the "Surefire" (or Failsafe) plugins reporting phase output. In other words, you will notice that the default output folder of Surefire plugin is directly related to the standard output location of "site" plugin, in general. This can be very confusing, I agree.
Also, I would configure it (possibly) like so:
<plugin>
<groupId>org.reportyng</groupId>
<artifactId>reporty-ng</artifactId>
<version>1.4</version>
<configuration>
<outputDirectory>${project.build.directory}/surefire/reportng</outputDirectory>
<sortTestCaseLinks>true</sortTestCaseLinks>
<testDetailsFilter>FAIL,SKIP,PASS,CONF,BY_CLASS</testDetailsFilter>
<showRuntimeTotals>true</showRuntimeTotals>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>2.1</version>
<configuration>
<outputDirectory>${basedir}/target/site</outputDirectory>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.9</version>
<configuration>
<outputDirectory>${basedir}/target/site</outputDirectory>
<reportsDirectories>
<reportsDirectories>${basedir}/target/surefire</reportsDirectories>
</reportsDirectories>
</configuration>
<reportSets>
<reportSet>
<id>integration-tests</id>
<reports>
<report>report-only</report>
</reports>
</reportSet>
</reportSets>
</plugin>

No forking observed with surefire 2.16 despite forkCount being > 1

Here is my setting:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<forkCount>1C</forkCount>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<reuseForks>true</reuseForks>
<argLine>-Xmx256m</argLine>
</configuration>
</plugin>
And I'm using junit 4.12:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
<version>4.12</version>
</dependency>
But when launching my tests I can't see any forking going on?
I'm using maven 3.2.5 on linux with JDK 8.
I ran a little debug (mvn -X test) and it turns out that despite using version 4.12 surefire isn't using the junit47 provider, but the junit4 provider. Unfortunately the junit4 provider does not seem to handle forkCount.
Crawling through the documentation I could find the following algo that selects the provider : https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html
The parallel attribute must be set in order to activate the junit47 provider (a little counter-intuitive...).
I've created a JIRA for that: https://issues.apache.org/jira/browse/SUREFIRE-1171
Now my config looks like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<parallel>classes</parallel>
<threadCount>1</threadCount>
<forkCount>1C</forkCount>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<reuseForks>true</reuseForks>
<argLine>-Xmx256m</argLine>
</configuration>
</plugin>
Edit
Turns out, redirectTestOutputToFile is now rendered useless... I've added a comment to an existing issue : https://issues.apache.org/jira/browse/SUREFIRE-703?focusedCommentId=14653289&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14653289
I came up with the following to have both forks and redirectTestOutputToFile work:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.18.1</version>
</dependency>
</dependencies>
<configuration>
<forkCount>1C</forkCount>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<reuseForks>true</reuseForks>
<argLine>-Xmx512m</argLine>
</configuration>
</plugin>

Resources