I'm generating spring-restdocs snippets with rest assured in my integration tests.
With maven and the failsafe plugin that defaults to the integration-test and verify phase. Problem is, that asciidoctor-maven-plugin (to generate the final HTML file out of the snippets) and maven-resources-plugin (to copy the final HTML file into the correct place) run before the integration tests in prepare-package.
With that asciidoctor naturally fails, because the snippets are not generated yet.
If I configure asciidoctor to run at post-integration-test, it succeeds, but then the finished HTML page is not in my jar, cause the jar was already created in the package phase.
So I feel like the only option is to run my integration tests already in the tests phase (probably with surefire instead of failsafe).
I could also split out the Documentation related tests from the rest of the integration tests, but I actually like to have them in the seemingly correct place.
I wonder if there is a best practice I'm ignoring?
Should integration tests never produce something to put inside the jar?
It looks to me, that integration tests in maven are meant for cross-jar tests (hence running them after package). And not like I'm using them, just for bigger tests that involve multiple parts (especially DB) all within one jar.
What I would like:
run all tests
compile documentation
package everything into the jar
Excerpt from my pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<skip>${skipSurefire}</skip>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/*IT.java</include>
<include>**/*E2ET.java</include>
</includes>
</configuration>
</plugin>
<!-- Compile API documentation -->
<plugin>
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctor-maven-plugin</artifactId>
<version>1.5.8</version>
<executions>
<execution>
<id>generate-docs</id>
<phase>prepare-package</phase>
<goals>
<goal>process-asciidoc</goal>
</goals>
<configuration>
<backend>html</backend>
<doctype>book</doctype>
</configuration>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.springframework.restdocs</groupId>
<artifactId>spring-restdocs-asciidoctor</artifactId>
<version>${spring-restdocs.version}</version>
</dependency>
</dependencies>
</plugin>
<!-- Package API documentation -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.7</version>
<executions>
<execution>
<id>copy-resources</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.outputDirectory}/public/docs
</outputDirectory>
<resources>
<resource>
<directory>
${project.build.directory}/generated-docs
</directory>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
The phases integration-test and verify are just the default bindings of the integration-test and the verify goal. You can bind your integration tests to the test phase, as well:
...
<artifactId>maven-failsafe-plugin</artifactId>
...
<executions>
<execution>
<id>integration-tests</id>
<phase>test</phase>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
and such achieve the order you like/need.
Related
I am generating failsafe and surefire reports with maven with the JaCoCo plugin, but I can only manage to get them in separate reports. I would like to have a overall coverage view (merge between both Unit Tests and Integration Tests) as well.
After what I think was a thorough google search I could only found a way to do this with Sonar. Is there any simpler way to do this?
Related question: Maven separate Unit Test and Integration Tests
I recently implemented this: after some headaches and a lot of testing, I have a configuration that works beautifully.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
<propertyName>testArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
<execution>
<id>merge-results</id>
<phase>verify</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}/coverage-reports</directory>
<includes>
<include>*.exec</include>
</includes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/coverage-reports/aggregate.exec</destFile>
</configuration>
</execution>
<execution>
<id>post-merge-report</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/aggregate.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-aggregate</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${surefireArgLine}</argLine>
<skipTests>${skip.unit.tests}</skipTests>
<includes>
<include>**/*UT.java</include>
<include>**/*MT.java</include>
<include>**/*Test.java</include>
</includes>
<skipTests>${skipUTMTs}</skipTests>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<skipTests>${skipTests}</skipTests>
<skipITs>${skipITs}</skipITs>
<argLine>${testArgLine}</argLine>
<excludes>
<exclude>**/*UT*.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
As you can see, there are 6 distinct Jacoco executions to run the tests, merge the files and create an aggregate report. On top of the Jacoco config, you also need to configure Surefire and Failsafe to take an argument from Jacoco (Surefire runs the unit tests and Failsafe runs the integration tests).
All of the configuration that I used should be there, what you do with it is your design architecture to make it fit within your desired requirements. Personally, I recommend a look into what I exclude and include within surefire and failsafe if you are having issues with files not being read.
Not the answer you're looking for, but still...
In most cases, you should not merge coverage from unit and integration tests.
The value of unit tests is that they improve the design of your application and ensure that corner cases of your code is working correctly. You should try to have a high branch coverage of your unit tests.
The value of your integration tests is that they ensure that the main use cases of your application are working correctly and that the whole stack is integrated correctly. You should try to have a high functional coverage for your integration tests. (But it is fairly hard to measure functional coverage with a tool).
If you need integration tests to improve your branch coverage, that's a strong indication that you should review the design of your code. And if you already have a high branch coverage without integration tests, adding them should not modify significantly your metrics.
You should take a look into the documentation of JaCoCo Maven plugin which contains a merge goal.
Maybe more nice way is to achieve is use same jacoco file, but let it another tests, which works for us - unit and it tests.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.1</version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<skip>${maven.surefire.skipTests}</skip>
<propertyName>maven.surefire.argLine</propertyName>
<!-- using the same dest file for both UT and IT -->
<destFile>${sonar.jacoco.reportPath}</destFile>
</configuration>
</execution>
<execution>
<id>default-prepare-agent-integration</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<skip>${maven.failsafe.skipTests}</skip>
<propertyName>maven.failsafe.argLine</propertyName>
<!-- append to the UT dest file -->
<destFile>${sonar.jacoco.reportPath}</destFile>
<append>true</append>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>${maven.surefire.skipTests}</skipTests>
<failIfNoTests>${maven.surefire.failIfNoTests}</failIfNoTests>
<!-- allow argLine to be modified by other plugins, e.g. jacoco -->
<argLine>#{maven.surefire.argLine}</argLine>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<skipTests>${maven.failsafe.skipTests}</skipTests>
<failIfNoTests>${maven.failsafe.failIfNoTests}</failIfNoTests>
<!-- allow argLine to be modified by other plugins, e.g. jacoco -->
<argLine>#{maven.failsafe.argLine}</argLine>
</configuration>
</plugin>
Building on top of the answer of Chad, here is my configuration. It is configured as a separate profile named jacoco so I can turn it on and off easily. It uses only the default configuration. It can create a separate code coverage report for unit tests, separate code coverage report for integration tests and also a combined code coverage report.
<profile>
<id>jacoco</id>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
</execution>
<execution>
<id>post-integration-test</id>
<goals>
<goal>report-integration</goal>
</goals>
</execution>
<execution>
<id>merge-results</id>
<phase>verify</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<includes>
<include>*.exec</include>
</includes>
<excludes>
<exclude>aggregate.exec</exclude>
</excludes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/aggregate.exec</destFile>
</configuration>
</execution>
<execution>
<id>post-merge-report</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/aggregate.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-aggregate</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
I get the point where Unit tests should really be at the source of code coverage... but sometimes, one would like to have the ability to know right? Here is what I did (this is from using gradle).
plugins {
[...]
id 'jacoco'
}
[...]
test {
jacoco { /* specify the "exec" data file name (see the "Unit" in there) */
destinationFile = file("$buildDir/jacoco/jacocoTestUnit.exec")
classDumpDir = file("$buildDir/jacoco/classpathdumpsUnit")
}
useJUnitPlatform { /* I use the JUnit Jupiter #Tag Annotation to create my suites... you are free to do something else */
excludeTags 'IT'
}
description = "Run unit tests"
group = "verification"
}
task intTest(type: Test) {
jacoco {
destinationFile = file("$buildDir/jacoco/jacocoTestInt.exec")
classDumpDir = file("$buildDir/jacoco/classpathdumpsInt")
}
useJUnitPlatform {
includeTags 'IT'
}
description = "Run integration tests"
group = "verification"
}
jacocoTestReport {
/*
* find all test exec files and feed them to jacoco
*/
def list = []
def dir = new File("${buildDir}/jacoco/")
if(dir.exists()) {
dir.eachFileRecurse(FileType.FILES) { file ->
/* gather all the "exec" files available */
if (file.getName().startsWith("jacocoTest") && file.getName().endsWith(".exec")) {
list << file.getAbsolutePath()
}
}
/* provide all the "exec" files to jacoco */
executionData.from = files(list)
}
/*
* you must run all tests before running jacoco.
* We want the liberty to run partial tests instead of all,
* so this task doesn't depend on any test task.
*/
reports {
xml.enabled true
}
}
With this, you can have the the possibility to get the coverage from Unit tests with:
./gradlew clean test jacocoTestReport
Or you can have the coverage from Integration tests with:
./gradlew clean intTest jacocoTestReport
Or you can have the overall coverage of both unit and Integration tests with:
./gradlew clean test inTest jacocoTestReport
Disclaimer: I'm no Jacoco or Gradle expert... feel free to comment anything I may have forgotten. This worked beautifully so far for my needs.
In order to merge the reports here, I have a complete working solution.
Please note that in order to work properly the merge strategy, the phases should be executed sequentially (if the mvn test and mvn verify -DskipUnitTests will be executed in parallel there is possible to not work fine).
<!-- Jacoco is used to generate the reports for SonarQube -->
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.6</version>
<configuration>
<skip>${skipTests}</skip>
</configuration>
<executions>
<!-- Prepares the property pointing to the JaCoCo runtime agent which
is passed as VM argument when Maven the Surefire plugin is executed. -->
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<!-- Sets the name of the property containing the settings for JaCoCo runtime agent. -->
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!-- Make sure that after the Unit Tests execution, the jacoco-ut.exec file is generated,
will be merged to the aggregation file -->
<execution>
<id>post-unit-merge</id>
<phase>test</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<includes>
<include>**/coverage-reports/*.exec</include>
</includes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/coverage-reports/jacoco_aggregation.exec</destFile>
</configuration>
</execution>
<!-- Ensures that the code coverage report is created/updated after unit tests have been run. -->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/coverage-reports/jacoco_aggregation.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}/jacoco</outputDirectory>
</configuration>
</execution>
<!-- Prepares the property pointing to the JaCoCo runtime agent which
is passed as VM argument when Maven the Failsafe plugin is executed. -->
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
<!-- Sets the name of the property containing the settings for JaCoCo runtime agent. -->
<propertyName>failsafeArgLine</propertyName>
</configuration>
</execution>
<!-- Make sure that after the Integration Test execution, the jacoco-it.exec file is generated,
will be merged to the aggregation file -->
<execution>
<id>post-integration-merge</id>
<phase>post-integration-test</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<includes>
<include>**/coverage-reports/*.exec</include>
</includes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/coverage-reports/jacoco_aggregation.exec</destFile>
</configuration>
</execution>
<!-- Ensures that the code coverage report is created/updated after integration tests have been run. -->
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/coverage-reports/jacoco_aggregation.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}/jacoco</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<!-- Sets the VM argument line used when unit tests are run. -->
<argLine>${surefireArgLine}</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<!-- Sets the VM argument line used when integration tests are run. -->
<argLine>${failsafeArgLine}</argLine>
</configuration>
</plugin>
Now, since the reports are generated, can be executed the sonar command in order to publis the report:
mvn sonar:sonar -Dsonar.coverage.jacoco.xmlReportPaths="target/site/jacoco/jacoco.xml"
This works out-of-the-box.
Explanations
The prepare-agent goal integrates by default with both SureFire and FailSafe plugins (to be exact the argLine parameter set by prepare-agent is consumed by both plugins). Additionally, prepare-agent by default configures Jacoco so that it appends coverage results if the file already exists, so in the end you get single target/jacoco.exec file which contains results from both unit test and integration tests, combined.
Just in case this is the relevant config, as you see nothing changed from default:) :
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<executions>
<execution>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.6</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
<goal>report</goal>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
Using Eclipse Luna with m2eclipse, I have a parent Maven project (facturas_root) and two Maven modules inheriting from it (sharepoint_ws and api_sharepoint).
sharepoint_ws was to be used only to generate JAXWS classes to connect to the Sharepoint WebServices, so I downloaded the related WSDL and included those as resources of the project. At generate-sources phase, it works correctly and generates the sources in target\generated-sources\ws-consume\mypackage\.
Now, the issue is that I made api_sharepoint import the sharepoint_ws dependency, but it does not detect any class. I assumed that it was because the generated classes were not at src/main/java, so I added a plugin to copy them there. Now, the problem is that at the compile phase of sharepoint_ws, it finds twice the source file of each class and throws an error.
My pom.xml -> build
<plugins>
<!-- clean /src/main/java and /target/generated-sources -->
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>2.6.1</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>clean</goal>
</goals>
<configuration>
<filesets>
<fileset>
<directory>${basedir}/src/main/java/es</directory>
</fileset>
<fileset>
<directory>${basedir}/target/generated-sources</directory>
</fileset>
</configuration>
</execution>
</executions>
</plugin>
<!-- generate jaxws -->
<plugin>
<groupId>org.jboss.ws.plugins</groupId>
<artifactId>maven-jaxws-tools-plugin</artifactId>
<version>1.1.2.Final</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>wsconsume</goal>
</goals>
<configuration>
<wsdls>
<wsdl>${basedir}/resources/lists.wsdl</wsdl>
</wsdls>
<targetPackage>es.ssib.otic.facturas.sharepoint_ws</targetPackage>
<outputDirectory>${basedir}/target/generated-sources</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<!-- copy sources -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.7</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.shared</groupId>
<artifactId>maven-filtering</artifactId>
<version>1.3</version>
</dependency>
</dependencies>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${basedir}/src/main/java</outputDirectory>
<resources>
<resource>
<directory>${basedir}/target/generated-sources/wsconsume</directory>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
In order to try to exclude target/generated-sources I have addded this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<executions>
<execution>
<id>default-compile</id>
<phase>compile</phase>
<configuration>
<excludes>
<exclude>**/target/generated-sources/*.java</exclude>
</excludes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
As stated above, I do comment the "copy" plugin, the module depending on sharepoint_ws does not have any ot its classes available; I do use it I get errors in the tune of
[ERROR] /C:/Users/s004256/workspace/facturas_root/sharepoint_ws/src/main/java/es/ssib/otic/facturas/sharepoint_ws/DeleteList.java:[34,8] duplicate class: es.ssib.otic.facturas.sharepoint_ws.DeleteList
for each generated list.
In the first place, I recommend you'd better declare target/generated-sources as a source folder, instead of copying files here and there:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-source</id>
<phase>initialize</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>${project.build.directory}/generated-sources</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
This should be enough to make Maven compile the target/generated-sources/*.java and package them all in the library, and also for Eclipse to recognize target/generated-sources as a source directory (after you execute Maven/Update Project).
By the way: You should take care of binding the plugins to a phase in the correct order: If you bound all tasks to "generate-sources", you have no gurantee about in which order will they be executed. And the same goes for the "compile" phase: You have to set properly the source folders, with its inclusions and exclusions, before the compile phase.
Take a look a the Default Maven Lifecycle and try to chose different, sequential phases for your tasks.
When we say mvn test, usual way is that maven will look for the tests present in src/test/java folder. But I have my tests in some different folder, namely src/integration-test/java. How do I run the tests present in this folder through command line?
Thanks in advance,
Manoj.
First you shouldn't run those integration test via the test life cycle, cause
pre-integration-test, integration-test and post-integration-test life cycle phase exist. Apart from that for integration tests the maven-failsafe-plugin is responsible.
There are several options to handle your situations. First you should follow the naming conventions for integration tests
<includes>
<include>**/IT*.java</include>
<include>**/*IT.java</include>
<include>**/*ITCase.java</include>
</includes>
which means to put the integration tests into the default folder src/test/java. If you have a multi-module build it would be the best having a separate module which contains the integration-tests only or you can go the path you decided to use a separate folder (which is not the best):
First you need to add the folder using the buildhelper-maven-plugin to get those integration tests being compiled like this:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>process-resources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/integration-test/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
and you have to configure the maven-failsafe-plugin like this:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.14.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
After you have configured you can run your integration tests via:
mvn verify
#khmarbaise is right with his recommendation (so +1 for that) but I want to answer your question without speculating about the reasons why the test source are located somewhere else.
If your tests are located in another directory than the standard src/test/java directory, the most simple solution is to change the default value of the testSourceDirectory configuration parameter which is defined in the Super POM.
e.g. for src/foobar/java use
<build>
<testSourceDirectory>src/foobar/java</testSourceDirectory>
</build>
then you can simply run mvn test to execute the tests.
More complex solution...
If you do not want to change the pom.xml configuration you can specifiy the testSourceDirectory parameter on the command line like this:
mvn -DtestSourceDirectory=src/foobar/java clean test
But be sure that your sources are compiled. Otherwise they will not be found and executed. In the above example the test sources are not placed at a location that gets compiled by default, so we nevertheless have to change the pom and add the directory to the list of test sources by using the buildhelper plugin:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/foobar/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
If you do not want to change the configuration of the default value in the pom and not want to pass the new directory at the commandline you have to configure the path in the maven-buildhelper-plugin and the maven-surefire-plugin in your pom.xml like this:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/foobar/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.14.1</version>
<configuration>
<testSourceDirectory>src/foobar/java</testSourceDirectory>
</configuration>
</plugin>
</plugins>
</build>
Now again the simple usage of mvn test will execute the test at the not standard location.
I've searched up and down the internet for this one. There's lots of half-answers out there, to do with Maven properties such as ${sonar.jacoco.reportPath}, or org.jacoco:jacoco-maven-plugin:prepare-agent or setting maven-surefire-plugin argLine with -javaagent.
Some how, none of these answers, either on their own, or in combination, are producing what I'm after:
A coverage report which shows a class as covered if it is used in tests higher up the stack, such as entities being used by DAOs, even though it was not fully covered by tests in its own module.
Is there a definitive config somewhere, to achieve this, please?
I was in the same situation as you, the half answers scattered throughout the Internet were quite annoying, since it seemed that many people had the same issue, but no one could be bothered to fully explain how they solved it.
The Sonar docs refer to a GitHub project with examples that are helpful. What I did to solve this was to apply the integration tests logic to regular unit tests (although proper unit tests should be submodule specific, this isn't always the case).
In the parent pom.xml, add these properties:
<properties>
<!-- Sonar -->
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.language>java</sonar.language>
</properties>
This will make Sonar pick up unit testing reports for all submodules in the same place (a target folder in the parent project). It also tells Sonar to reuse reports ran manually instead of rolling its own. We just need to make jacoco-maven-plugin run for all submodules by placing this in the parent pom, inside build/plugins:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.6.0.201210061924</version>
<configuration>
<destFile>${sonar.jacoco.reportPath}</destFile>
<append>true</append>
</configuration>
<executions>
<execution>
<id>agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
</plugin>
destFile places the report file in the place where Sonar will look for it and append makes it append to the file rather than overwriting it. This will combine all JaCoCo reports for all submodules in the same file.
Sonar will look at that file for each submodule, since that's what we pointed him at above, giving us combined unit testing results for multi module files in Sonar.
NEW WAY SINCE VERSION 0.7.7
Since Version 0.7.7 there is a new way to create an aggregated report:
You create a separate 'report' project which collects all the necessary reports (Any goal in the aggregator project is executed before its modules therefore it can't be used).
aggregator pom
|- parent pom
|- module a
|- module b
|- report module
The root pom looks like this (don't forget to add the new report module under modules):
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
The poms from each sub module doesn't need to be changed at all. The pom from the report module looks like this:
<!-- Add all sub modules as dependencies here -->
<dependencies>
<dependency>
<module a>
</dependency>
<dependency>
<module b>
</dependency>
...
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<executions>
<execution>
<id>report-aggregate</id>
<phase>verify</phase>
<goals>
<goal>report-aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
A full exmple can be found here.
FAQ
Questions from the top of my head since that time I gone crazy with jacoco.
My application server (jBoss, Glassfish..) located in Iraq, Syria, whatever.. Is it possible to get multi-module coverage when running integration tests on it? Jenkins and Sonar are also on different servers.
Yes. You have to use jacoco agent that runs in mode output=tcpserver, jacoco ant lib. Basically two jars. This will give you 99% success.
How does jacoco agent works?
You append a string
-javaagent:[your_path]/jacocoagent.jar=destfile=/jacoco.exec,output=tcpserver,address=*
to your application server JAVA_OPTS and restart it. In this string only [your_path] have to be replaced with the path to jacocoagent.jar, stored(store it!) on your VM where app server runs. Since that time you start app server, all applications that are deployed will be dynamically monitored and their activity (meaning code usage) will be ready for you to get in jacocos .exec format by tcl request.
Could I reset jacoco agent to start collecting execution data only since the time my test start?
Yes, for that purpose you need jacocoant.jar and ant build script located in your jenkins workspace.
So basically what I need from http://www.eclemma.org/jacoco/ is jacocoant.jar located in my jenkins workspace, and jacocoagent.jar located on my app server VM?
That's right.
I don't want to use ant, I've heard that jacoco maven plugin can do all the things too.
That's not right, jacoco maven plugin can collect unit test data and some integration tests data(see Arquillian Jacoco), but if you have for example rest assured tests as a separated build in jenkins, and want to show multi-module coverage, I can't see how maven plugin can help you.
What exactly does jacoco agent produce?
Only coverage data in .exec format. Sonar then can read it.
Does jacoco need to know where my java classes located are?
No, sonar does, but not jacoco. When you do mvn sonar:sonar path to classes comes into play.
So what about the ant script?
It has to be presented in your jenkins workspace. Mine ant script, I called it jacoco.xml looks like that:
<project name="Jacoco library to collect code coverage remotely" xmlns:jacoco="antlib:org.jacoco.ant">
<property name="jacoco.port" value="6300"/>
<property name="jacocoReportFile" location="${workspace}/it-jacoco.exec"/>
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
<classpath path="${workspace}/tools/jacoco/jacocoant.jar"/>
</taskdef>
<target name="jacocoReport">
<jacoco:dump address="${jacoco.host}" port="${jacoco.port}" dump="true" reset="true" destfile="${jacocoReportFile}" append="false"/>
</target>
<target name="jacocoReset">
<jacoco:dump address="${jacoco.host}" port="${jacoco.port}" reset="true" destfile="${jacocoReportFile}" append="false"/>
<delete file="${jacocoReportFile}"/>
</target>
</project>
Two mandatory params you should pass when invoking this script
-Dworkspace=$WORKSPACE
use it to point to your jenkins workspace and -Djacoco.host=yourappserver.com host without http://
Also notice that I put my jacocoant.jar to ${workspace}/tools/jacoco/jacocoant.jar
What should I do next?
Did you start your app server with jacocoagent.jar?
Did you put ant script and jacocoant.jar in your jenkins workspace?
If yes the last step is to configure a jenkins build. Here is the strategy:
Invoke ant target jacocoReset to reset all previously collected data.
Run your tests
Invoke ant target jacocoReport to get report
If everything is right, you will see it-jacoco.exec in your build workspace.
Look at the screenshot, I also have ant installed in my workspace in $WORKSPACE/tools/ant dir, but you can use one that is installed in your jenkins.
How to push this report in sonar?
Maven sonar:sonar will do the job (don't forget to configure it), point it to main pom.xml so it will run through all modules. Use sonar.jacoco.itReportPath=$WORKSPACE/it-jacoco.exec parameter to tell sonar where your integration test report is located. Every time it will analyse new module classes, it will look for information about coverage in it-jacoco.exec.
I already have jacoco.exec in my `target` dir, `mvn sonar:sonar` ignores/removes it
By default mvn sonar:sonar does clean and deletes your target dir, use sonar.dynamicAnalysis=reuseReports to avoid it.
I found another solution for new Sonar versions where JaCoCo's binary report format (*.exec) was deprecated and the preferred format is XML (SonarJava 5.12 and higher).
The solution is very simple and similar to the previous solution with *.exec reports in parent directory from this topic: https://stackoverflow.com/a/15535970/4448263.
Assuming that our project structure is:
moduleC - aggregate project's pom
|- moduleA - some classes without tests
|- moduleB - some classes depending from moduleA and tests for classes in both modules: moduleA and moduleB
You need following maven build plugin configuration in aggregate project's pom:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.5</version>
<executions>
<execution>
<id>prepare-and-report</id>
<goals>
<goal>prepare-agent</goal>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>report-aggregate</id>
<phase>verify</phase>
<goals>
<goal>report-aggregate</goal>
</goals>
<configuration>
<outputDirectory>${project.basedir}/../target/site/jacoco-aggregate</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
Then build project with maven:
mvn clean verify
And for Sonar you should set property in administration GUI:
sonar.coverage.jacoco.xmlReportPaths=target/site/jacoco/jacoco.xml,../target/site/jacoco-aggregate/jacoco.xml
or using command line:
mvn sonar:sonar -Dsonar.coverage.jacoco.xmlReportPaths=target/site/jacoco/jacoco.xml,../target/site/jacoco-aggregate/jacoco.xml
Description
This creates binary reports for each module in default directories: target/jacoco.exec. Then creates XML reports for each module in default directories: target/site/jacoco/jacoco.xml. Then creates an aggregate report for each module in custom directory ${project.basedir}/../target/site/jacoco-aggregate/ that is relative to parent directory for each module. For moduleA and moduleB this will be common path moduleC/target/site/jacoco-aggregate/.
As moduleB depends on moduleA, moduleB will be built last and its report will be used as an aggregate coverage report in Sonar for both modules A and B.
In addition to the aggregate report, we need a normal module report as JaCoCo aggregate reports contain coverage data only for dependencies.
Together, these two types of reports providing full coverage data for Sonar.
There is one little restriction: you should be able to write a report in the project's parent directory (should have permission). Or you can set property jacoco.skip=true in root project's pom.xml (moduleC) and jacoco.skip=false in modules with classes and tests (moduleA and moduleB).
I'll post my solution as it it subtly different from others and also took me a solid day to get right, with the assistance of the existing answers.
For a multi-module Maven project:
ROOT
|--WAR
|--LIB-1
|--LIB-2
|--TEST
Where the WAR project is the main web app, LIB 1 and 2 are additional modules the WAR depends on and TEST is where the integration tests live.
TEST spins up an embedded Tomcat instance (not via Tomcat plugin) and runs WAR project and tests them via a set of JUnit tests.
The WAR and LIB projects both have their own unit tests.
The result of all this is the integration and unit test coverage being separated and able to be distinguished in SonarQube.
ROOT pom.xml
<!-- Sonar properties-->
<sonar.jacoco.itReportPath>${project.basedir}/../target/jacoco-it.exec</sonar.jacoco.itReportPath>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.language>java</sonar.language>
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<!-- build/plugins (not build/pluginManagement/plugins!) -->
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.6.201602180812</version>
<executions>
<execution>
<id>agent-for-ut</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${sonar.jacoco.reportPath}</destFile>
</configuration>
</execution>
<execution>
<id>agent-for-it</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${sonar.jacoco.itReportPath}</destFile>
</configuration>
</execution>
</executions>
</plugin>
WAR, LIB and TEST pom.xml will inherit the the JaCoCo plugins execution.
TEST pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<skipTests>${skip.tests}</skipTests>
<argLine>${argLine} -Duser.timezone=UTC -Xms256m -Xmx256m</argLine>
<includes>
<includes>**/*Test*</includes>
</includes>
</configuration>
</execution>
</executions>
</plugin>
I also found Petri Kainulainens blog post 'Creating Code Coverage Reports for Unit and Integration Tests With the JaCoCo Maven Plugin' to be valuable for the JaCoCo setup side of things.
There is a way to accomplish this. The magic is to create a combined jacoco.exec file. And with maven 3.3.1 there is an easy way to get this. Here my profile:
<profile>
<id>runSonar</id>
<activation>
<property>
<name>runSonar</name>
<value>true</value>
</property>
</activation>
<properties>
<sonar.language>java</sonar.language>
<sonar.host.url>http://sonar.url</sonar.host.url>
<sonar.login>tokenX</sonar.login>
<sonar.jacoco.reportMissing.force.zero>true</sonar.jacoco.reportMissing.force.zero>
<sonar.jacoco.reportPath>${jacoco.destFile}</sonar.jacoco.reportPath>
<jacoco.destFile>${maven.multiModuleProjectDirectory}/target/jacoco_analysis/jacoco.exec</jacoco.destFile>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${jacoco.destFile}</destFile>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.sonarsource.scanner.maven</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>3.2</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
</plugin>
</plugins>
</pluginManagement>
</build>
</profile>
If you add this profile to your parent pom and call mvn clean install sonar:sonar -DrunSonar you get the complete coverage.
The magic here is maven.multiModuleProjectDirectory. This folder is always the folder where you started your maven build.
The configuration I use in my parent level pom where I have separate unit and integration test phases.
I configure the following properties in the parent POM
Properties
<maven.surefire.report.plugin>2.19.1</maven.surefire.report.plugin>
<jacoco.plugin.version>0.7.6.201602180812</jacoco.plugin.version>
<jacoco.check.lineRatio>0.52</jacoco.check.lineRatio>
<jacoco.check.branchRatio>0.40</jacoco.check.branchRatio>
<jacoco.check.complexityMax>15</jacoco.check.complexityMax>
<jacoco.skip>false</jacoco.skip>
<jacoco.excludePattern/>
<jacoco.destfile>${project.basedir}/../target/coverage-reports/jacoco.exec</jacoco.destfile>
<sonar.language>java</sonar.language>
<sonar.exclusions>**/generated-sources/**/*</sonar.exclusions>
<sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
<sonar.coverage.exclusions>${jacoco.excludePattern}</sonar.coverage.exclusions>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.jacoco.reportPath>${project.basedir}/../target/coverage-reports</sonar.jacoco.reportPath>
<skip.surefire.tests>${skipTests}</skip.surefire.tests>
<skip.failsafe.tests>${skipTests}</skip.failsafe.tests>
I place the plugin definitions under plugin management.
Note that I define a property for surefire (surefireArgLine) and failsafe (failsafeArgLine) arguments to allow jacoco to configure the javaagent to run with each test.
Under pluginManagement
<build>
<pluginManagment>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<fork>true</fork>
<meminitial>1024m</meminitial>
<maxmem>1024m</maxmem>
<compilerArgument>-g</compilerArgument>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
<encoding>${project.build.sourceEncoding}</encoding>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<forkCount>4</forkCount>
<reuseForks>false</reuseForks>
<argLine>-Xmx2048m ${surefireArgLine}</argLine>
<includes>
<include>**/*Test.java</include>
</includes>
<excludes>
<exclude>**/*IT.java</exclude>
</excludes>
<skip>${skip.surefire.tests}</skip>
</configuration>
</plugin>
<plugin>
<!-- For integration test separation -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.19.1</version>
</dependency>
</dependencies>
<configuration>
<forkCount>4</forkCount>
<reuseForks>false</reuseForks>
<argLine>${failsafeArgLine}</argLine>
<includes>
<include>**/*IT.java</include>
</includes>
<skip>${skip.failsafe.tests}</skip>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<!-- Code Coverage -->
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.plugin.version}</version>
<configuration>
<haltOnFailure>true</haltOnFailure>
<excludes>
<exclude>**/*.mar</exclude>
<exclude>${jacoco.excludePattern}</exclude>
</excludes>
<rules>
<rule>
<element>BUNDLE</element>
<limits>
<limit>
<counter>LINE</counter>
<value>COVEREDRATIO</value>
<minimum>${jacoco.check.lineRatio}</minimum>
</limit>
<limit>
<counter>BRANCH</counter>
<value>COVEREDRATIO</value>
<minimum>${jacoco.check.branchRatio}</minimum>
</limit>
</limits>
</rule>
<rule>
<element>METHOD</element>
<limits>
<limit>
<counter>COMPLEXITY</counter>
<value>TOTALCOUNT</value>
<maximum>${jacoco.check.complexityMax}</maximum>
</limit>
</limits>
</rule>
</rules>
</configuration>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${jacoco.destfile}</destFile>
<append>true</append>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${jacoco.destfile}</dataFile>
<outputDirectory>${sonar.jacoco.reportPath}</outputDirectory>
<skip>${skip.surefire.tests}</skip>
</configuration>
</execution>
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${jacoco.destfile}</destFile>
<append>true</append>
<propertyName>failsafeArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report-integration</goal>
</goals>
<configuration>
<dataFile>${jacoco.destfile}</dataFile>
<outputDirectory>${sonar.jacoco.reportPath}</outputDirectory>
<skip>${skip.failsafe.tests}</skip>
</configuration>
</execution>
<!-- Disabled until such time as code quality stops this tripping
<execution>
<id>default-check</id>
<goals>
<goal>check</goal>
</goals>
<configuration>
<dataFile>${jacoco.destfile}</dataFile>
</configuration>
</execution>
-->
</executions>
</plugin>
...
And in the build section
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
</plugin>
<plugin>
<!-- for unit test execution -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
<plugin>
<!-- For integration test separation -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
<plugin>
<!-- For code coverage -->
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
</plugin>
....
And in the reporting section
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>${maven.surefire.report.plugin}</version>
<configuration>
<showSuccess>false</showSuccess>
<alwaysGenerateFailsafeReport>true</alwaysGenerateFailsafeReport>
<alwaysGenerateSurefireReport>true</alwaysGenerateSurefireReport>
<aggregate>true</aggregate>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.plugin.version}</version>
<configuration>
<excludes>
<exclude>**/*.mar</exclude>
<exclude>${jacoco.excludePattern}</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</reporting>
As Sonars sonar.jacoco.reportPath, sonar.jacoco.itReportPath and sonar.jacoco.reportPaths have all been deprecated, you should use sonar.coverage.jacoco.xmlReportPaths now. This also has some impact if you want to configure a multi module maven project with Sonar and Jacoco.
As #Lonzak pointed out, since Sonar 0.7.7, you can use Sonars report aggragation goal. Just put in you parent pom the following dependency:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.5</version>
<executions>
<execution>
<id>report</id>
<goals>
<goal>report-aggregate</goal>
</goals>
<phase>verify</phase>
</execution>
</executions>
</plugin>
As current versions of the jacoco-maven-plugin are compatible with the xml-reports, this will create for every module in it's own target folder a site/jacoco-aggregate folder containing a jacoco.xml file.
To let Sonar combine all the modules, use following command:
mvn -Dsonar.coverage.jacoco.xmlReportPaths=full-path-to-module1/target/site/jacoco-aggregate/jacoco.xml,module2...,module3... clean verify sonar:sonar
To keep my answer short and precise, I did not mention the maven-surefire-plugin and
maven-failsafe-plugin dependencies. You can just add them without any other configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
</plugin>
<sonar.language>java</sonar.language>
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.jacoco.reportPath>${user.dir}/target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.jacoco.itReportPath>${user.dir}/target/jacoco-it.exec</sonar.jacoco.itReportPath>
<sonar.exclusions>
file:**/target/generated-sources/**,
file:**/target/generated-test-sources/**,
file:**/target/test-classes/**,
file:**/model/*.java,
file:**/*Config.java,
file:**/*App.java
</sonar.exclusions>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.9</version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${sonar.jacoco.reportPath}</destFile>
<append>true</append>
<propertyName>surefire.argLine</propertyName>
</configuration>
</execution>
<execution>
<id>default-prepare-agent-integration</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${sonar.jacoco.itReportPath}</destFile>
<append>true</append>
<propertyName>failsafe.argLine</propertyName>
</configuration>
</execution>
<execution>
<id>default-report</id>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>default-report-integration</id>
<goals>
<goal>report-integration</goal>
</goals>
</execution>
</executions>
</plugin>
This sample works very well for me :
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.2</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
<!--<excludes>
<exclude>com.asimio.demo.rest</exclude>
<exclude>com.asimio.demo.service</exclude>
</excludes>-->
<propertyName>testArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
<execution>
<id>merge-results</id>
<phase>verify</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}/coverage-reports</directory>
<includes>
<include>*.exec</include>
</includes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/coverage-reports/aggregate.exec</destFile>
</configuration>
</execution>
<execution>
<id>post-merge-report</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/aggregate.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-aggregate</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${surefireArgLine}</argLine>
<!--<skipTests>${skip.unit.tests}</skipTests>-->
<includes>
<include>**/*Test.java</include>
<!--<include>**/*MT.java</include>
<include>**/*Test.java</include>-->
</includes>
<!-- <skipTests>${skipUTMTs}</skipTests>-->
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<!--<skipTests>${skipTests}</skipTests>
<skipITs>${skipITs}</skipITs>-->
<argLine>${testArgLine}</argLine>
<includes>
<include>**/*IT.java</include>
</includes>
<!--<excludes>
<exclude>**/*UT*.java</exclude>
</excludes>-->
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
You can call a ant task called merge on maven, to put all coverage files (*.exec) together in the same file.
If you are run unit tests use the phase prepare-package, if you run integration test so use post-integration-test.
This site has an example to how call jacoco ant task in maven project
You can use this merged file on sonar.
to have unit-testing AND integration-testing you can use maven-surefire-plugin and maven-failsafe-plugin with restricted includes/excludes. I was playing with CDI while getting in touch with sonar/jacoco, so i ended up in this project:
https://github.com/FibreFoX/cdi-sessionscoped-login/
Maybe it helps you a little bit. in my pom.xml i use "-javaagent" implicit by setting the argLine-option in the configuration-section of the specified testing-plugins.
Explicit using ANT in MAVEN projects is something i would not give a try, for me its to much mixing two worlds.
I only have a single-module maven project, but maybe it helps you to adjust yours to work.
note: maybe not all maven-plugins are up2date, maybe some issues are fixed in later versions
As stated in the Jacoco Wiki, you could also generate a new reporting-module:
Strategy: Module with Dependencies: The problems with aggregator
projects can be solved with an extra "reporting" module. In a
multi-module Maven project a separate module is defined that does not
contribute actual content but creates a combined coverage report. It
defines a dependency to all the modules that should be included in the
combined report. The "reporting" module would be built after its
dependencies and have access to the exec files as well as class and
source files from projects it depends on. This strategy seems to work
best with the current Maven architecture. From a user's perspective
one could argue that such a separate module would bloat the build
definition. Or, that the separate module cannot have any submodules
from which it could consume exec or class files. However, compared to
the other strategies these drawbacks seem rather minor and can be
handled in a consistent manner.
This is especially helpful if your modularisation is more complex than one parent with some child modules.
Your reporting pom could look something like:
<dependencies>
<dependency>
<groupId>org.sonarqube</groupId>
<artifactId>module1</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.sonarqube</groupId>
<artifactId>module2</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>report</id>
<goals>
<goal>report-aggregate</goal>
</goals>
<phase>verify</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
AFAIK, the maven failsafe plugin fails safe because it has separate goals for running the tests and failing the build based on the tests. These are designed to be bound to the integration-test and verify goals respectively. This allows post-integration-test bound goals to run (shutting down the build) before the build fails.
My question is, how do I do this with the maven-soapui-plugin? I thought I could simply specify <testFailIgnore>true</testFailIgnore> in my soapui plugin config and then call the failsafe plugin verify goal, but that isn't working. I don't think I'm not sure if I'm getting a summary file out of the soapui plugin or not. I keep getting Expected root element 'failsafe-summary' but found 'testsuite' Here is a snippet of the POM:
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>4.0.0</version>
<configuration>
<junitReport>true</junitReport>
<exportAll>true</exportAll>
<outputFolder>${project.build.directory}/surefire-reports</outputFolder>
<testFailIgnore>true</testFailIgnore>
<printReport>true</printReport>
</configuration>
<executions>
<execution>
<id>FailingTest</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<projectFile>${basedir}/testData/soapui-integration-tests.xml</projectFile>
<host>localhost</host>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.9</version>
<executions>
<execution>
<phase>verify</phase>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
<configuration>
<summaryFiles>
<summaryFile>${project.build.directory}/surefire-reports/TEST-TestSuite_1.xml</summaryFile>
</summaryFiles>
</configuration>
</execution>
</executions>
</plugin>
Is there something wrong with my POM or is this a bad approach? Are there any better approaches?
There is an open source extension of the soapui plugin which has a separate test-verify goal for exactly this purpose.
https://github.com/redfish4ktc/maven-soapui-extension-plugin
The following example shows the required configuration:
https://github.com/redfish4ktc/maven-soapui-extension-plugin/blob/master/src/it/test-verify_goal/one_failing_project/pom.xml
AFAIK maven-failsafe-plugin can only verify success status of tests run by maven-failsafe-plugin and not by maven-soapui-plugin. It does that by reading test summary report file (failsafe-summary.xml) which has specific format.
maven-soapui-plugin could be adjusted to separate running tests from verifying test success status, to support running post-integration-test tasks (stop server, undeploy artifacts, etc.) before verification. Create a support ticket for this to soapUI guys.
Maybe even maven-failsafe-plugin, it's verify mojo, could be extended to allow specifying different test report format (JUnit style reports are supported by soapUI) or even an xpath expression which would be used by maven-failsafe-plugin to determine if there were failed tests or not. Create a support ticket for this on maven-failsafe-plugin issue tracker.
Until those extensions are supported, and you need to do tasks on post-integration-test phase you can use soapUI JUnit integration and have maven-failsafe-plugin run those soapUI JUnit tests.
I am trying this solution, and it doesn't work.
But I have found one.
To obtain de tests report of SOAPUI tests in Jenkins, I using the failsafe plugin with this configuration in the pom.xml of my Soap tests folder :
<build>
<plugins>
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>2.6.1</version>
<configuration>
<projectFile>${basedir}/soap_project_tests.xml</projectFile>
<outputFolder>${filePath.reports.soap}</outputFolder>
<testFailIgnore>true</testFailIgnore>
<junitReport>true</junitReport>
<exportwAll>true</exportwAll>
<printReport>true</printReport>
</configuration>
<executions>
<execution>
<id>run-soap-integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.11</version>
<configuration>
<reportsDirectory>${filePath.reports.soap}</reportsDirectory>
<printSummary>true</printSummary>
<argLine>-Xmx512m</argLine>
</configuration>
<executions>
<execution>
<id>soap-integration-test-verify</id>
<phase>post-integration-test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
The tests cases status are up to Jenkins.