Well, I'm using Spring Boot 2.4.0 with Surefire and Failsafe, so the following plugins:
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>11</source>
<target>11</target>
<annotationProcessorPaths>
<path>
<groupId>org.mapstruct</groupId>
<artifactId>mapstruct-processor</artifactId>
<version>${org.mapstruct.version}</version>
</path>
<!-- other annotation processors -->
</annotationProcessorPaths>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
I have Unit tests (*Test.java) and integration tests (*IT.java). But running mvn clean package just run unit tests, I need run mvn clean instasll to run integration tests + unit tests.
Any tips? This is correct?
Related
I am using Sonarqube to keep track of both unit and integration test coverage for a multi-module Maven project.
This was the existing profile in the parent pom.xml that was used to generate the Sonarqube report locally before I made the change:
Profile that generates all unit test coverage locally in Sonarqube
<profiles>
<profile>
<id>coverage</id>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<sonar.jacoco.reportPaths>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPaths>
<sonar.projectName>plan-advantage-serverless-${project.artifactId}</sonar.projectName>
<sonar.projectKey>${project.groupId}-MPA-${project.artifactId}</sonar.projectKey>
<sonar.exclusions>file:**/generated-sources/**,**/*Model.java,**/models/**/*</sonar.exclusions>
<sonar.test.exclusions>**/test/*</sonar.test.exclusions>
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.language>java</sonar.language>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.2</version>
<configuration>
<append>true</append>
<excludes>
<exclude>**/test/*</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>default-instrument</id>
<goals>
<goal>instrument</goal>
</goals>
</execution>
<execution>
<id>default-restore-instrumented-classes</id>
<goals>
<goal>restore-instrumented-classes</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<showWarnings>true</showWarnings>
<compilerArgs>
<arg>-Xlint:all</arg>
<arg>-Xlint:-processing</arg>
</compilerArgs>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<argLine>-XX:-UseSplitVerifier</argLine>
<systemPropertyVariables>
<jacoco-agent.destfile>${sonar.jacoco.reportPaths}</jacoco-agent.destfile>
</systemPropertyVariables>
</configuration>
<dependencies>
<!-- needed for powermock to run correctly with surefire-->
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.22.0</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.sonarsource.scanner.maven</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>3.5.0.1254</version>
</plugin>
</plugins>
</build>
</profile>
This is generating the expected test coverage (sans integration tests) in
Sonarqube locally when I run mvn clean install -P coverage sonar:sonar.
I've so far been able to get integration coverage added as a proof of concept using the following addition
to the parent pom.xml:
pom.xml that includes integration test coverage in Sonarqube but excludes some unit tests
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<jacoco.version>0.7.9</jacoco.version> . <sonar.jacoco.reportPaths>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPaths>
<sonar.jacoco.itReportPath>${project.basedir}/../target/jacoco-it.exec</sonar.jacoco.itReportPath>
<sonar.language>java</sonar.language>
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
</properties>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M3</version>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>3.0.0-M3</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<executions>
<execution>
<id>agent-for-ut</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${sonar.jacoco.reportPaths}</destFile>
</configuration>
</execution>
<execution>
<id>agent-for-it</id>
<phase>package</phase>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${sonar.jacoco.itReportPath}</destFile>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
This was inspired by the example found here.
However, when I run it with the command mvn clean install failsafe:integration-test sonar:sonar, it causes some of the unit tests which were previously being covered to not show up in the Sonarqube output. I believe that the prepare-agent and prepare-agent integrationgoals are using on-the-fly instrumentation. According to JaCoCo's docs, on-the-fly instrumentation is not possible while using PowerMock (which my project is utilizing), so we have to use the offline instrumentation for JaCoCo.
I looked at this example for using offline instrumentation and used the following pom.xml with the command mvn clean install test sonar:sonar:
parent pom.xml that fails to build due to NoClassDefFound errors
<build>
...
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<executions>
<execution>
<id>default-instrument</id>
<goals>
<goal>instrument</goal>
</goals>
</execution>
<execution>
<id>default-restore-instrumented-classes</id>
<goals>
<goal>restore-instrumented-classes</goal>
</goals>
</execution>
</executions>
</plugin>
</build>
And here's the resulting error:
Any ideas for the proper pom.xml configuration to enable offline instrumentation to get integration and unit test coverage to show up in Sonarqube?
I know this question is 10 months old, but for anyone else coming across this, you need to add this to your dependencies in your pom.xml file.
<dependency>
<groupId>org.jacoco</groupId>
<artifactId>org.jacoco.agent</artifactId>
<classifier>runtime</classifier>
<version>${jacoco.version}</version>
<scope>test</scope>
</dependency>
I have a groovy project in Eclipse. I have to change from Gradle to Maven and now I cannot run the tests that exist in it.
Here is the structure of the project:
Here are the contents of the pom.xml:
<dependencies>
...
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-core</artifactId>
<version>1.0-groovy-2.4</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<compilerId>groovy-eclipse-compiler</compilerId>
<compilerArguments>
<indy />
<!-- optional; supported by batch 2.4.12-04+ -->
...
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<execution>
<id>add-source</id>
...
</execution>
<execution>
<id>add-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/test/groovy</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
However, if I do a mvn clean install, the tests are not being ran.
Here is the output of the console:
The test cases we have are called Spec and not Test, this is fixed like this:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<includes>
<include>**/*Spec.class</include>
</includes>
</configuration>
</plugin>
I've noticed that sometimes when running maven builds on Jenkins the number of Jbehave tests that are run vary from one run to another. When analyzing the logs I see the following snippet:
Failed to run story stories/cancel.story
java.lang.InterruptedException: stories/cancel.story
at org.jbehave.core.embedder.StoryRunner$RunContext.interruptIfCancelled(StoryRunner.java:616)
at org.jbehave.core.embedder.StoryRunner.runStepsWhileKeepingState(StoryRunner.java:514)
at org.jbehave.core.embedder.StoryRunner.runScenarioSteps(StoryRunner.java:479)
at org.jbehave.core.embedder.StoryRunner.runStepsWithLifecycle(StoryRunner.java:445)
at org.jbehave.core.embedder.StoryRunner.runCancellable(StoryRunner.java:305)
at org.jbehave.core.embedder.StoryRunner.run(StoryRunner.java:220)
at org.jbehave.core.embedder.StoryRunner.run(StoryRunner.java:181)
at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:235)
at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:207)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
The problem is that when tests are skipped or fail to run in this way the build is still considered a success.
Is there a maven surefire plugin configuration that will ensure that whenever tests fail to run the build results in a failure? Here are the maven surefire build configurations
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.11</version>
<configuration>
<includes>
<include>**/*TestSuite.java</include>
</includes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9</version>
</plugin>
<plugin>
<groupId>net.thucydides.maven.plugins</groupId>
<artifactId>maven-thucydides-plugin</artifactId>
<version>${thucydides.version}</version>
<executions>
<execution>
<id>thucydides-reports</id>
<phase>post-integration-test</phase>
<goals>
<goal>aggregate</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.2</version>
<configuration>
<reportPlugins>
<plugin>
<groupId>net.thucydides.maven.plugins</groupId>
<artifactId>maven-thucydides-plugin</artifactId>
<version>${thucydides.version}</version>
</plugin>
</reportPlugins>
</configuration>
</plugin>
</plugins>
</build>
Your maven-surefire-plugin is set to skip tests completely (with <skip>true</skip>), so tests are running with maven-failsafe-plugin. That plug-in is supposed to not stop on failure during integration-test, and then only fail on verify phase.
So if you really want this question answered:
Is there a maven surefire plugin configuration that will ensure that whenever tests fail to run the build results in a failure?
That is: you want maven-surefire-plugin to run the tests, and not the maven-failsafe-plugin, then the answer is: remove
<configuration>
<skip>true</skip>
</configuration>
from your POM. In this case you also don't need maven-failsafe-plugin configuration, because it would just make your tests run twice.
But if your goal is to get maven-failsafe-plugin to work, then I think you may have one of the following issues:
Not running the right goal. As plug-in help states, you should invoke it as
mvn verify
An old plug-in, which is not compatible with test framework you are using (current version is 2.19.1)
Or this help recommendation:
For very complex builds, it may be better to separate the executions for the integration-test and verify goals:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
I have an issue with my pom:
There are two types of tests running within the project:
Main set of tests using TestNG and JUnit code coverage tests. What I would like is to run JUnit tests on test-compile phase and do not run TestNG tests (which are run on test phase) if previous failed.
As of now I have the following which runs only TestNG:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<argLine>
-javaagent:${settings.localRepository}/org/aspectj/aspectjweaver/1.8.4/aspectjweaver-1.8.4.jar
</argLine>
<properties>
<property>
<name>listener</name>
<value>org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter</value>
</property>
</properties>
<suiteXmlFiles>
<suiteXmlFile>src/test/_all.xml</suiteXmlFile>
</suiteXmlFiles>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
I tried to play with <execution> tag but I can't provide path to TestNG suite then. Any ideas how to combine these tests using surefire plugin and specify different goals?
Solution was to add maven-failsafe-plugin separately from maven-surefire-plugin.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.17</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.17</version>
</dependency>
</dependencies>
<configuration>
<includes>
<include>junitcodecoverage/**Test.java</include>
</includes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<phase>test-compile</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
i'm writing tests via selenium web driver here's my code :
Pom.xml
<project>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.24.1</version>
</dependency>
</dependencies>
<build>
<finalName>SeleniumebDriverProject</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.11</version>
<configuration>
<!-- Skip the normal tests, we'll run them in the integration-test phase-->
<skip>false</skip>
</configuration>
<executions>
<execution>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<includes>
<include>**/*Test.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
My test class named GoogleTest.java.
after reading this post : failsafe plugin won't run on one project but will run on another -- why?
I changed the name of my class so it's:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.11</version>
<configuration>
<!-- Skip the normal tests, we'll run them in the integration-test phase-->
<skip>false</skip>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
<executions>
</plugin>
But the problem persist.
The goal of the maven-failsafe-plugin is named integration-test instead of test. Furthermore if you changed your naming convention to the convention of maven-failsafe-plugin than you don't need any configuration which includes etc. files. Just use the defaults.
Furthermore i assume you have started running the integration tests via:
mvn verify