No forking observed with surefire 2.16 despite forkCount being > 1 - maven

Here is my setting:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<forkCount>1C</forkCount>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<reuseForks>true</reuseForks>
<argLine>-Xmx256m</argLine>
</configuration>
</plugin>
And I'm using junit 4.12:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
<version>4.12</version>
</dependency>
But when launching my tests I can't see any forking going on?
I'm using maven 3.2.5 on linux with JDK 8.

I ran a little debug (mvn -X test) and it turns out that despite using version 4.12 surefire isn't using the junit47 provider, but the junit4 provider. Unfortunately the junit4 provider does not seem to handle forkCount.
Crawling through the documentation I could find the following algo that selects the provider : https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html
The parallel attribute must be set in order to activate the junit47 provider (a little counter-intuitive...).
I've created a JIRA for that: https://issues.apache.org/jira/browse/SUREFIRE-1171
Now my config looks like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<parallel>classes</parallel>
<threadCount>1</threadCount>
<forkCount>1C</forkCount>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<reuseForks>true</reuseForks>
<argLine>-Xmx256m</argLine>
</configuration>
</plugin>
Edit
Turns out, redirectTestOutputToFile is now rendered useless... I've added a comment to an existing issue : https://issues.apache.org/jira/browse/SUREFIRE-703?focusedCommentId=14653289&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14653289
I came up with the following to have both forks and redirectTestOutputToFile work:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.18.1</version>
</dependency>
</dependencies>
<configuration>
<forkCount>1C</forkCount>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<reuseForks>true</reuseForks>
<argLine>-Xmx512m</argLine>
</configuration>
</plugin>

Related

Maven skip generating reports when executing Maven build

I have several reporting plugins defined in the project and when I do Maven → build fails with FileNotFoundException as the reports are cleared when I do Maven → clean. Is there a way to ignore reports when doing Maven build so that build passes without any issue? I tried using Surefire plugin but not luck.
My pom.xml looks like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.1</version>
<configuration>
<skip>${maven-site-plugin.skip}</skip>
<skipDeploy>true</skipDeploy>
</configuration>
</plugin>
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.19.1</version>
</plugin>
</plugins>
</reporting>
<dependency>
<groupId>com.aventstack</groupId>
<artifactId>extentreports</artifactId>
<version>3.0.5</version>
</dependency>
<dependency>
<groupId>net.masterthought</groupId>
<artifactId>cucumber-reporting</artifactId>
<version>3.2.0</version>
</dependency>
For maven-surefire
add after version
<configuration>
<skipTests>true</skipTests>
</configuration>
I got the fix for this now. I updated selenium version to 3.14 and I see no issue when building the project

maven-failsafe-plugin does not execute test cases from single class in parallel mode

I've single test class with multiple test cases, which I would like to execute in parallel mode.
I've below setup in pom.xml
But instead of executing in parallel mode, test cases are being executed in sequence.
Please clarify what may be going wrong here ?
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
Directly adding parallel configuration in maven-failsafe-plugin is not working (May be a bug.)
But we can set parallel attribute and thread count in surefire plugin configuration.
Internally maven-failsafe-plugin downloads maven-surefire-plugin plugin.
So we can explicitly provide dependency on maven-surefire-plugin having parallel configuration.
I have tested , method are getting executed in parallel.
For more know how on running testcase in parallel and more configuration parameters click here.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<parallel>methods</parallel>
<useUnlimitedThreads>true</useUnlimitedThreads>
</configuration>
<plugin>
Working pom.
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany.app</groupId>
<artifactId>my-module</artifactId>
<version>1</version>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<parallel>methods</parallel>
<useUnlimitedThreads>true</useUnlimitedThreads>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
</plugin>
</plugins>
</build>

Running "mvn test site" giving [ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project

I'm trying to generate xslt report using reporty-ng with Testng+maven but getting error as
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project Selenium Project: failed to get report for org.reportyng:reporty-ng: Plugin org.reportyng:reporty-ng:1.2 or one of its dependencies could not be resolved: Failure to find org.reportyng:reporty-ng:jar:1.2 in https://github.com/cosminaru/reporty-ng/raw/master/dist/maven was cached in the local repository, resolution will not be reattempted until the update interval of reporty-ng has elapsed or updates are forced -> [Help 1]
Below is the part of the pom.xml:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.48.1</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.9.6</version>
</dependency>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-api</artifactId>
<version>2.19</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.5</version>
</dependency>
</dependencies>
<reporting>
<plugins>
<!-- TestNG-xslt related configuration. -->
<plugin>
<groupId>org.reportyng</groupId>
<artifactId>reporty-ng</artifactId>
<version>1.2</version>
<configuration>
<!-- Output directory for the testng xslt report -->
<outputDir>/target/testng-xslt-report</outputDir>
<sortTestCaseLinks>true</sortTestCaseLinks>
<testDetailsFilter>FAIL,SKIP,PASS,CONF,BY_CLASS</testDetailsFilter>
<showRuntimeTotals>true</showRuntimeTotals>
</configuration>
</plugin>
</plugins>
</reporting>
<pluginRepositories>
<pluginRepository>
<id>reporty-ng</id>
<url>https://github.com/cosminaru/reporty-ng/raw/master/dist/maven</url>
</pluginRepository>
</pluginRepositories>
In my understanding of this, if you run "mvn clean verify", that will include the "site" phase of the build as well as the tests. You shouldn't need to explicitly call the site phase and so don't do it. Also, if you were to explicitly call the "site" phase, you should probably include the site plugin as a dependency, don't you think?
In my experience with both TestNG reports and ReportNG reports, the site plugin is not related to the report output. The site plugin is only related to the "Surefire" (or Failsafe) plugins reporting phase output. In other words, you will notice that the default output folder of Surefire plugin is directly related to the standard output location of "site" plugin, in general. This can be very confusing, I agree.
Also, I would configure it (possibly) like so:
<plugin>
<groupId>org.reportyng</groupId>
<artifactId>reporty-ng</artifactId>
<version>1.4</version>
<configuration>
<outputDirectory>${project.build.directory}/surefire/reportng</outputDirectory>
<sortTestCaseLinks>true</sortTestCaseLinks>
<testDetailsFilter>FAIL,SKIP,PASS,CONF,BY_CLASS</testDetailsFilter>
<showRuntimeTotals>true</showRuntimeTotals>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>2.1</version>
<configuration>
<outputDirectory>${basedir}/target/site</outputDirectory>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.9</version>
<configuration>
<outputDirectory>${basedir}/target/site</outputDirectory>
<reportsDirectories>
<reportsDirectories>${basedir}/target/surefire</reportsDirectories>
</reportsDirectories>
</configuration>
<reportSets>
<reportSet>
<id>integration-tests</id>
<reports>
<report>report-only</report>
</reports>
</reportSet>
</reportSets>
</plugin>

Maven complete surefire-report including passed tests logging events

I want to generate a report using surefire-report plugin of maven which will give me a detailed report of all the tests that ran with the log events that were logged during each test run instead of just the events where my test failed. This will also give me the full context where my test failed.
Here is my pom.xml file:
<artifactId>SampleTests</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<name>sampleTests</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.4</version>
</dependency>
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.11</version>
<type>maven-plugin</type>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20090211</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/java</sourceDirectory>
<outputDirectory>${basedir}/target/classes</outputDirectory>
<testSourceDirectory>src/main/test</testSourceDirectory>
<testOutputDirectory>${basedir}/target/test-classes</testOutputDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<includes>
<include>com/mypackage/**</include>
</includes>
<test>**/*Tests</test>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
</plugin>
</plugins>
</build>
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.6</version>
<reportSets>
<reportSet>
<reports>
<report>report-only</report>
</reports>
</reportSet>
</reportSets>
</plugin>
</plugins>
</reporting>
</project>
And i use mvn surefire-report:report to generate the report here.
So I want to know if there is anything additional that i can do so as to see a detailed report of what happened during the tests run?
As a reference from the maven site
surefire-report:report Generates the test results report into HTML format.
surefire-report:report-only This goal does not run the tests, it only builds the reports.
You are trying the report-only. Try the other one report as a goal
I ended up using ant to get to what I needed. Thanks to this blog: http://blog.varunin.com/2010/03/ant-script-for-generating-junit-report.html#comment-form
If you want to get a detailed report of all the test events then please use this

Maven & Junit Theories

I want to run JUnit Theories under Maven3,
The Theories have the Annotation #Theory before the method.
Running the Tests in Eclipse makes no problem.
Running it under Maven with the surfire Plugin I get "java.lang.Exception: No runnable methods"
How can I handle this Problem
It works for me. My configuration in plugin section:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<includes>
<include>**/*Junit*.java</include>
<include>**/*Test*.java</include>
<include>**/*Test.java</include>
<include>**/*TestCase.java</include>
</includes>
<excludedGroups combine.self="override" />
</configuration>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.12</version>
</dependency>
</dependencies>
</plugin>
I used junit version 4.10

Resources