How to run UnitTests in maven which is in src/test-integration/java folder - maven

When we say mvn test, usual way is that maven will look for the tests present in src/test/java folder. But I have my tests in some different folder, namely src/integration-test/java. How do I run the tests present in this folder through command line?
Thanks in advance,
Manoj.

First you shouldn't run those integration test via the test life cycle, cause
pre-integration-test, integration-test and post-integration-test life cycle phase exist. Apart from that for integration tests the maven-failsafe-plugin is responsible.
There are several options to handle your situations. First you should follow the naming conventions for integration tests
<includes>
<include>**/IT*.java</include>
<include>**/*IT.java</include>
<include>**/*ITCase.java</include>
</includes>
which means to put the integration tests into the default folder src/test/java. If you have a multi-module build it would be the best having a separate module which contains the integration-tests only or you can go the path you decided to use a separate folder (which is not the best):
First you need to add the folder using the buildhelper-maven-plugin to get those integration tests being compiled like this:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>process-resources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/integration-test/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
and you have to configure the maven-failsafe-plugin like this:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.14.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
After you have configured you can run your integration tests via:
mvn verify

#khmarbaise is right with his recommendation (so +1 for that) but I want to answer your question without speculating about the reasons why the test source are located somewhere else.
If your tests are located in another directory than the standard src/test/java directory, the most simple solution is to change the default value of the testSourceDirectory configuration parameter which is defined in the Super POM.
e.g. for src/foobar/java use
<build>
<testSourceDirectory>src/foobar/java</testSourceDirectory>
</build>
then you can simply run mvn test to execute the tests.
More complex solution...
If you do not want to change the pom.xml configuration you can specifiy the testSourceDirectory parameter on the command line like this:
mvn -DtestSourceDirectory=src/foobar/java clean test
But be sure that your sources are compiled. Otherwise they will not be found and executed. In the above example the test sources are not placed at a location that gets compiled by default, so we nevertheless have to change the pom and add the directory to the list of test sources by using the buildhelper plugin:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/foobar/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
If you do not want to change the configuration of the default value in the pom and not want to pass the new directory at the commandline you have to configure the path in the maven-buildhelper-plugin and the maven-surefire-plugin in your pom.xml like this:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/foobar/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.14.1</version>
<configuration>
<testSourceDirectory>src/foobar/java</testSourceDirectory>
</configuration>
</plugin>
</plugins>
</build>
Now again the simple usage of mvn test will execute the test at the not standard location.

Related

Integration tests create files for the packaged jar

I'm generating spring-restdocs snippets with rest assured in my integration tests.
With maven and the failsafe plugin that defaults to the integration-test and verify phase. Problem is, that asciidoctor-maven-plugin (to generate the final HTML file out of the snippets) and maven-resources-plugin (to copy the final HTML file into the correct place) run before the integration tests in prepare-package.
With that asciidoctor naturally fails, because the snippets are not generated yet.
If I configure asciidoctor to run at post-integration-test, it succeeds, but then the finished HTML page is not in my jar, cause the jar was already created in the package phase.
So I feel like the only option is to run my integration tests already in the tests phase (probably with surefire instead of failsafe).
I could also split out the Documentation related tests from the rest of the integration tests, but I actually like to have them in the seemingly correct place.
I wonder if there is a best practice I'm ignoring?
Should integration tests never produce something to put inside the jar?
It looks to me, that integration tests in maven are meant for cross-jar tests (hence running them after package). And not like I'm using them, just for bigger tests that involve multiple parts (especially DB) all within one jar.
What I would like:
run all tests
compile documentation
package everything into the jar
Excerpt from my pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<skip>${skipSurefire}</skip>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/*IT.java</include>
<include>**/*E2ET.java</include>
</includes>
</configuration>
</plugin>
<!-- Compile API documentation -->
<plugin>
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctor-maven-plugin</artifactId>
<version>1.5.8</version>
<executions>
<execution>
<id>generate-docs</id>
<phase>prepare-package</phase>
<goals>
<goal>process-asciidoc</goal>
</goals>
<configuration>
<backend>html</backend>
<doctype>book</doctype>
</configuration>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.springframework.restdocs</groupId>
<artifactId>spring-restdocs-asciidoctor</artifactId>
<version>${spring-restdocs.version}</version>
</dependency>
</dependencies>
</plugin>
<!-- Package API documentation -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.7</version>
<executions>
<execution>
<id>copy-resources</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.outputDirectory}/public/docs
</outputDirectory>
<resources>
<resource>
<directory>
${project.build.directory}/generated-docs
</directory>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
The phases integration-test and verify are just the default bindings of the integration-test and the verify goal. You can bind your integration tests to the test phase, as well:
...
<artifactId>maven-failsafe-plugin</artifactId>
...
<executions>
<execution>
<id>integration-tests</id>
<phase>test</phase>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
and such achieve the order you like/need.

Generate Javadoc for multimodule project

I have read everything I can find on solving this and my attempts still fail. The best I can do is to get the Javadoc of exactly one module to show up--the last one built. (For now, I'm not trying to bundle Javadoc into any JARs. I'm also not trying to do anything "site".) I just want to put Javadoc for easy access into a subdirectory under the project root.
Here's what's in my parent pom.xml:
<build>
<plugins>
.
.
.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>3.2.0</version>
<configuration>
<noqualifier>all</noqualifier>
<reportOutputDirectory>${user.dir}/documents</reportOutputDirectory>
<destDir>javadoc</destDir>
</configuration>
<executions>
<execution>
<id>attach-javadocs</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
What I'm putting into subordinate pom.xml files is identical to the above except for
<goals>
<goal>javadoc</goal>
</goals>
I have played with replacing the <execution> in the parent and sometimes subordinate pom.xml files with:
<execution>
<id>aggregate</id>
<goals>
<goal>aggregate</goal>
</goals>
</execution>
but it makes no difference.
I think the following configuration is the reason your reports get overwritten:
<configuration>
<reportOutputDirectory>${user.dir}/documents</reportOutputDirectory>
</configuration>
All module builds will be written to the same directory, hence overwriting the previous build.
The solution is to use the default output directory and configure the output directory for the aggregated javadoc instead. This way the reactor build will create javadoc output files in each module's target directory. These can then be used by the aggregate goal to be combined.
This can be done by configuring your parent POM as follows:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>3.2.0</version>
<configuration>
<!-- Default configuration for all reports -->
<noqualifier>all</noqualifier>
<destDir>javadoc</destDir>
</configuration>
<executions>
<execution>
<id>aggregate</id>
<goals>
<goal>aggregate</goal>
</goals>
<configuration>
<!-- Specific configuration for the aggregate report -->
<reportOutputDirectory>${user.dir}/documents</reportOutputDirectory>
<destDir>javadoc</destDir>
</configuration>
</execution>
...
</executions>
</plugin>
...
</plugins>
</build>
(there is no need for any additional configuration in the module POM files)
The aggregated javadoc can now be created by running
mvn compile javadoc:javadoc javadoc:aggregate
(note that the compile or package goal is required for reactor to resolve inter-module dependencies)

Trouble launching chromedriver in Spock test

I have a Maven web project for which I'm trying to run a couple of simple Spock web UI tests under Eclipse with Selenium and Chromedriver. I can run each test class individually by clicking on it and selecting "Run As>Junit Test" if I add the following to the file's run configuration VM arguments:
-Dwebdriver.chrome.driver=/Users/mht/ChromeDriver/2.3.1/chromedriver
Obviously, this is the location of the Chromedriver on my system. Then I tried to set up a Maven build goal at the project level to run the tests. I click on the project name and select "Run Configurations>Maven Build" and create a "Verify" configuration to match the verify goal defined in my pom. In the "Goals, box I enter "verify" and on the JRE tab I also enter the above location for the chromedriver in the VM arguments box. When I run this goal, or on the command line type "mvn verify", I get the following error:
geb.driver.DriverCreationException: failed to create driver from callback 'script15040527017471153797989$_run_closure1#1046d517'
at com.google.common.base.Preconditions.checkState(Preconditions.java:754)
...
This tells me the test can not locate the chromedriver. If I copy the chromedriver into the base directory of the project, the then the test will run using the verify goal or by typing "mvn verify" on the command line. Why isn't setting the chromedriver location not working at the maven goal level?
I don't think this really matters, but the build section of my pom is
<build>
<plugins>
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>2.6</version>
<configuration>
<webXml>src/main/webapp/WEB-INF/web.xml</webXml>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<useFile>false</useFile>
<includes>
<include>**/*Spec.java</include>
</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<id>tomcat-run</id>
<goals>
<goal>run-war-only</goal>
</goals>
<phase>pre-integration-test</phase>
<configuration>
<port>9081</port>
<fork>true</fork>
</configuration>
</execution>
<execution>
<id>tomcat-shutdown</id>
<goals>
<goal>shutdown</goal>
</goals>
<phase>post-integration-test</phase>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
The maven arguments are not passed to the surefire and failsafe configurations per default. Both fork new JVMs which doesn't get these arguments. See the docs Surefire and Failsafe regarding argLine.
So mvn verify -DargLine="-Dwebdriver.chrome.driver=/Users/mht/ChromeDriver/2.3.1/chromedriver" should work for you.
However, the better way is to use the WebDriver Extensions Maven Plugin which can be used to automatically download the appropriate driver. You can then do some simple scripting to locate the driver via your geb config, or hard code the known relative location.
BTW the gmaven-plus plugin is outdated.

How do I specify a separate maven goal for running (Cucumber) acceptance tests?

I have the following project structure:
MyProject
--src
--test
--acceptance
--step_definitions
--features
--unit
I would like to be able to run my cucumber tests (in test/acceptance) separately in Maven from the unit tests declared in test/unit, so that they can be run in different CI build plans etc. I am using cucumber-junit so the 'runners' for each acceptance test are written with JUnit.
Is this possible?
Is this possible?
Yes, it is possible. I believe you should separate your unit from the acceptance/integration tests having:
Slightly modified folders structure for both of these, placing your integration test files in the standard location of src/it:
MyProject/
src/main/java/ (SUT)
src/test/ (unit test code)
java/
resources/
src/it/ (acceptance/integration tests)
java/ (steps definitions)
resources/ (feature files)
Moreover, by design, different Maven plugins are intended for unit and integration tests:
for unit tests: maven-surefire-plugin
for acceptance/integration tests: maven-failsafe-plugin
You must also bind execution of maven-failsafe-pulgin. To run the integration tests separately, you can define a new profile:
<profiles>
<profile>
<id>acceptance-tests</id>
<build>
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
You will also need to configure the plugin to search the src/it directory tree for test cases.
The acceptance tests can be run afterwards using:
mvn clean verify -Pacceptance-tests
For complete sample, I'd suggest you to follow http://www.weblogism.com/item/334/integration-tests-with-cucumber-jvm-selenium-and-maven
The other answer suggested modifying your folder structure to have shared folder for integration and acceptance tests, but you can have the original folder structure as well. Also you mentioned in the comment you want to keep all three (including not-mentioned integration tests) separate, which is possible, though hackish.
Since you seem to have test/unit for unit tests and test/acceptance for acceptance test, I'm assuming test/integration for integration tests.
<profiles>
<profile>
<id>acceptance-test</id>
<build>
<plugins>
<plugin>
<!-- to run directly: mvn failsafe:integration-test -P acceptance-test -->
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12</version>
<configuration>
<testSourceDirectory>test/acceptance</testSourceDirectory>
<includes>
<include>**/*Acceptance.java</include>
</includes>
<excludes>
<exclude>**/*IT.java</exclude>
<exclude>**/*Test.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>test/unit</source>
<source>test/integration</source>
<source>test/acceptance</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<testSourceDirectory>test/unit</testSourceDirectory>
</configuration>
</plugin>
<plugin>
<!-- to run directly: mvn failsafe:integration-test -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<testSourceDirectory>test/integration</testSourceDirectory>
</configuration>
<!-- execution below can be used, if tests are needed on
mvn:integration-test -->
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
Note though that separation only applies for sources: compiled files will all go to the same folder, and AFAIK that's something you can't change. This means you need to have naming strategy for your tests to separate them from each other.

How to configure multi-module Maven + Sonar + JaCoCo to give merged coverage report?

I've searched up and down the internet for this one. There's lots of half-answers out there, to do with Maven properties such as ${sonar.jacoco.reportPath}, or org.jacoco:jacoco-maven-plugin:prepare-agent or setting maven-surefire-plugin argLine with -javaagent.
Some how, none of these answers, either on their own, or in combination, are producing what I'm after:
A coverage report which shows a class as covered if it is used in tests higher up the stack, such as entities being used by DAOs, even though it was not fully covered by tests in its own module.
Is there a definitive config somewhere, to achieve this, please?
I was in the same situation as you, the half answers scattered throughout the Internet were quite annoying, since it seemed that many people had the same issue, but no one could be bothered to fully explain how they solved it.
The Sonar docs refer to a GitHub project with examples that are helpful. What I did to solve this was to apply the integration tests logic to regular unit tests (although proper unit tests should be submodule specific, this isn't always the case).
In the parent pom.xml, add these properties:
<properties>
<!-- Sonar -->
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.language>java</sonar.language>
</properties>
This will make Sonar pick up unit testing reports for all submodules in the same place (a target folder in the parent project). It also tells Sonar to reuse reports ran manually instead of rolling its own. We just need to make jacoco-maven-plugin run for all submodules by placing this in the parent pom, inside build/plugins:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.6.0.201210061924</version>
<configuration>
<destFile>${sonar.jacoco.reportPath}</destFile>
<append>true</append>
</configuration>
<executions>
<execution>
<id>agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
</plugin>
destFile places the report file in the place where Sonar will look for it and append makes it append to the file rather than overwriting it. This will combine all JaCoCo reports for all submodules in the same file.
Sonar will look at that file for each submodule, since that's what we pointed him at above, giving us combined unit testing results for multi module files in Sonar.
NEW WAY SINCE VERSION 0.7.7
Since Version 0.7.7 there is a new way to create an aggregated report:
You create a separate 'report' project which collects all the necessary reports (Any goal in the aggregator project is executed before its modules therefore it can't be used).
aggregator pom
|- parent pom
|- module a
|- module b
|- report module
The root pom looks like this (don't forget to add the new report module under modules):
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
The poms from each sub module doesn't need to be changed at all. The pom from the report module looks like this:
<!-- Add all sub modules as dependencies here -->
<dependencies>
<dependency>
<module a>
</dependency>
<dependency>
<module b>
</dependency>
...
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<executions>
<execution>
<id>report-aggregate</id>
<phase>verify</phase>
<goals>
<goal>report-aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
A full exmple can be found here.
FAQ
Questions from the top of my head since that time I gone crazy with jacoco.
My application server (jBoss, Glassfish..) located in Iraq, Syria, whatever.. Is it possible to get multi-module coverage when running integration tests on it? Jenkins and Sonar are also on different servers.
Yes. You have to use jacoco agent that runs in mode output=tcpserver, jacoco ant lib. Basically two jars. This will give you 99% success.
How does jacoco agent works?
You append a string
-javaagent:[your_path]/jacocoagent.jar=destfile=/jacoco.exec,output=tcpserver,address=*
to your application server JAVA_OPTS and restart it. In this string only [your_path] have to be replaced with the path to jacocoagent.jar, stored(store it!) on your VM where app server runs. Since that time you start app server, all applications that are deployed will be dynamically monitored and their activity (meaning code usage) will be ready for you to get in jacocos .exec format by tcl request.
Could I reset jacoco agent to start collecting execution data only since the time my test start?
Yes, for that purpose you need jacocoant.jar and ant build script located in your jenkins workspace.
So basically what I need from http://www.eclemma.org/jacoco/ is jacocoant.jar located in my jenkins workspace, and jacocoagent.jar located on my app server VM?
That's right.
I don't want to use ant, I've heard that jacoco maven plugin can do all the things too.
That's not right, jacoco maven plugin can collect unit test data and some integration tests data(see Arquillian Jacoco), but if you have for example rest assured tests as a separated build in jenkins, and want to show multi-module coverage, I can't see how maven plugin can help you.
What exactly does jacoco agent produce?
Only coverage data in .exec format. Sonar then can read it.
Does jacoco need to know where my java classes located are?
No, sonar does, but not jacoco. When you do mvn sonar:sonar path to classes comes into play.
So what about the ant script?
It has to be presented in your jenkins workspace. Mine ant script, I called it jacoco.xml looks like that:
<project name="Jacoco library to collect code coverage remotely" xmlns:jacoco="antlib:org.jacoco.ant">
<property name="jacoco.port" value="6300"/>
<property name="jacocoReportFile" location="${workspace}/it-jacoco.exec"/>
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
<classpath path="${workspace}/tools/jacoco/jacocoant.jar"/>
</taskdef>
<target name="jacocoReport">
<jacoco:dump address="${jacoco.host}" port="${jacoco.port}" dump="true" reset="true" destfile="${jacocoReportFile}" append="false"/>
</target>
<target name="jacocoReset">
<jacoco:dump address="${jacoco.host}" port="${jacoco.port}" reset="true" destfile="${jacocoReportFile}" append="false"/>
<delete file="${jacocoReportFile}"/>
</target>
</project>
Two mandatory params you should pass when invoking this script
-Dworkspace=$WORKSPACE
use it to point to your jenkins workspace and -Djacoco.host=yourappserver.com host without http://
Also notice that I put my jacocoant.jar to ${workspace}/tools/jacoco/jacocoant.jar
What should I do next?
Did you start your app server with jacocoagent.jar?
Did you put ant script and jacocoant.jar in your jenkins workspace?
If yes the last step is to configure a jenkins build. Here is the strategy:
Invoke ant target jacocoReset to reset all previously collected data.
Run your tests
Invoke ant target jacocoReport to get report
If everything is right, you will see it-jacoco.exec in your build workspace.
Look at the screenshot, I also have ant installed in my workspace in $WORKSPACE/tools/ant dir, but you can use one that is installed in your jenkins.
How to push this report in sonar?
Maven sonar:sonar will do the job (don't forget to configure it), point it to main pom.xml so it will run through all modules. Use sonar.jacoco.itReportPath=$WORKSPACE/it-jacoco.exec parameter to tell sonar where your integration test report is located. Every time it will analyse new module classes, it will look for information about coverage in it-jacoco.exec.
I already have jacoco.exec in my `target` dir, `mvn sonar:sonar` ignores/removes it
By default mvn sonar:sonar does clean and deletes your target dir, use sonar.dynamicAnalysis=reuseReports to avoid it.
I found another solution for new Sonar versions where JaCoCo's binary report format (*.exec) was deprecated and the preferred format is XML (SonarJava 5.12 and higher).
The solution is very simple and similar to the previous solution with *.exec reports in parent directory from this topic: https://stackoverflow.com/a/15535970/4448263.
Assuming that our project structure is:
moduleC - aggregate project's pom
|- moduleA - some classes without tests
|- moduleB - some classes depending from moduleA and tests for classes in both modules: moduleA and moduleB
You need following maven build plugin configuration in aggregate project's pom:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.5</version>
<executions>
<execution>
<id>prepare-and-report</id>
<goals>
<goal>prepare-agent</goal>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>report-aggregate</id>
<phase>verify</phase>
<goals>
<goal>report-aggregate</goal>
</goals>
<configuration>
<outputDirectory>${project.basedir}/../target/site/jacoco-aggregate</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
Then build project with maven:
mvn clean verify
And for Sonar you should set property in administration GUI:
sonar.coverage.jacoco.xmlReportPaths=target/site/jacoco/jacoco.xml,../target/site/jacoco-aggregate/jacoco.xml
or using command line:
mvn sonar:sonar -Dsonar.coverage.jacoco.xmlReportPaths=target/site/jacoco/jacoco.xml,../target/site/jacoco-aggregate/jacoco.xml
Description
This creates binary reports for each module in default directories: target/jacoco.exec. Then creates XML reports for each module in default directories: target/site/jacoco/jacoco.xml. Then creates an aggregate report for each module in custom directory ${project.basedir}/../target/site/jacoco-aggregate/ that is relative to parent directory for each module. For moduleA and moduleB this will be common path moduleC/target/site/jacoco-aggregate/.
As moduleB depends on moduleA, moduleB will be built last and its report will be used as an aggregate coverage report in Sonar for both modules A and B.
In addition to the aggregate report, we need a normal module report as JaCoCo aggregate reports contain coverage data only for dependencies.
Together, these two types of reports providing full coverage data for Sonar.
There is one little restriction: you should be able to write a report in the project's parent directory (should have permission). Or you can set property jacoco.skip=true in root project's pom.xml (moduleC) and jacoco.skip=false in modules with classes and tests (moduleA and moduleB).
I'll post my solution as it it subtly different from others and also took me a solid day to get right, with the assistance of the existing answers.
For a multi-module Maven project:
ROOT
|--WAR
|--LIB-1
|--LIB-2
|--TEST
Where the WAR project is the main web app, LIB 1 and 2 are additional modules the WAR depends on and TEST is where the integration tests live.
TEST spins up an embedded Tomcat instance (not via Tomcat plugin) and runs WAR project and tests them via a set of JUnit tests.
The WAR and LIB projects both have their own unit tests.
The result of all this is the integration and unit test coverage being separated and able to be distinguished in SonarQube.
ROOT pom.xml
<!-- Sonar properties-->
<sonar.jacoco.itReportPath>${project.basedir}/../target/jacoco-it.exec</sonar.jacoco.itReportPath>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.language>java</sonar.language>
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<!-- build/plugins (not build/pluginManagement/plugins!) -->
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.6.201602180812</version>
<executions>
<execution>
<id>agent-for-ut</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${sonar.jacoco.reportPath}</destFile>
</configuration>
</execution>
<execution>
<id>agent-for-it</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${sonar.jacoco.itReportPath}</destFile>
</configuration>
</execution>
</executions>
</plugin>
WAR, LIB and TEST pom.xml will inherit the the JaCoCo plugins execution.
TEST pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<skipTests>${skip.tests}</skipTests>
<argLine>${argLine} -Duser.timezone=UTC -Xms256m -Xmx256m</argLine>
<includes>
<includes>**/*Test*</includes>
</includes>
</configuration>
</execution>
</executions>
</plugin>
I also found Petri Kainulainens blog post 'Creating Code Coverage Reports for Unit and Integration Tests With the JaCoCo Maven Plugin' to be valuable for the JaCoCo setup side of things.
There is a way to accomplish this. The magic is to create a combined jacoco.exec file. And with maven 3.3.1 there is an easy way to get this. Here my profile:
<profile>
<id>runSonar</id>
<activation>
<property>
<name>runSonar</name>
<value>true</value>
</property>
</activation>
<properties>
<sonar.language>java</sonar.language>
<sonar.host.url>http://sonar.url</sonar.host.url>
<sonar.login>tokenX</sonar.login>
<sonar.jacoco.reportMissing.force.zero>true</sonar.jacoco.reportMissing.force.zero>
<sonar.jacoco.reportPath>${jacoco.destFile}</sonar.jacoco.reportPath>
<jacoco.destFile>${maven.multiModuleProjectDirectory}/target/jacoco_analysis/jacoco.exec</jacoco.destFile>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${jacoco.destFile}</destFile>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.sonarsource.scanner.maven</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>3.2</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
</plugin>
</plugins>
</pluginManagement>
</build>
</profile>
If you add this profile to your parent pom and call mvn clean install sonar:sonar -DrunSonar you get the complete coverage.
The magic here is maven.multiModuleProjectDirectory. This folder is always the folder where you started your maven build.
The configuration I use in my parent level pom where I have separate unit and integration test phases.
I configure the following properties in the parent POM
Properties
<maven.surefire.report.plugin>2.19.1</maven.surefire.report.plugin>
<jacoco.plugin.version>0.7.6.201602180812</jacoco.plugin.version>
<jacoco.check.lineRatio>0.52</jacoco.check.lineRatio>
<jacoco.check.branchRatio>0.40</jacoco.check.branchRatio>
<jacoco.check.complexityMax>15</jacoco.check.complexityMax>
<jacoco.skip>false</jacoco.skip>
<jacoco.excludePattern/>
<jacoco.destfile>${project.basedir}/../target/coverage-reports/jacoco.exec</jacoco.destfile>
<sonar.language>java</sonar.language>
<sonar.exclusions>**/generated-sources/**/*</sonar.exclusions>
<sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
<sonar.coverage.exclusions>${jacoco.excludePattern}</sonar.coverage.exclusions>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.jacoco.reportPath>${project.basedir}/../target/coverage-reports</sonar.jacoco.reportPath>
<skip.surefire.tests>${skipTests}</skip.surefire.tests>
<skip.failsafe.tests>${skipTests}</skip.failsafe.tests>
I place the plugin definitions under plugin management.
Note that I define a property for surefire (surefireArgLine) and failsafe (failsafeArgLine) arguments to allow jacoco to configure the javaagent to run with each test.
Under pluginManagement
<build>
<pluginManagment>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<fork>true</fork>
<meminitial>1024m</meminitial>
<maxmem>1024m</maxmem>
<compilerArgument>-g</compilerArgument>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
<encoding>${project.build.sourceEncoding}</encoding>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<forkCount>4</forkCount>
<reuseForks>false</reuseForks>
<argLine>-Xmx2048m ${surefireArgLine}</argLine>
<includes>
<include>**/*Test.java</include>
</includes>
<excludes>
<exclude>**/*IT.java</exclude>
</excludes>
<skip>${skip.surefire.tests}</skip>
</configuration>
</plugin>
<plugin>
<!-- For integration test separation -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.19.1</version>
</dependency>
</dependencies>
<configuration>
<forkCount>4</forkCount>
<reuseForks>false</reuseForks>
<argLine>${failsafeArgLine}</argLine>
<includes>
<include>**/*IT.java</include>
</includes>
<skip>${skip.failsafe.tests}</skip>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<!-- Code Coverage -->
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.plugin.version}</version>
<configuration>
<haltOnFailure>true</haltOnFailure>
<excludes>
<exclude>**/*.mar</exclude>
<exclude>${jacoco.excludePattern}</exclude>
</excludes>
<rules>
<rule>
<element>BUNDLE</element>
<limits>
<limit>
<counter>LINE</counter>
<value>COVEREDRATIO</value>
<minimum>${jacoco.check.lineRatio}</minimum>
</limit>
<limit>
<counter>BRANCH</counter>
<value>COVEREDRATIO</value>
<minimum>${jacoco.check.branchRatio}</minimum>
</limit>
</limits>
</rule>
<rule>
<element>METHOD</element>
<limits>
<limit>
<counter>COMPLEXITY</counter>
<value>TOTALCOUNT</value>
<maximum>${jacoco.check.complexityMax}</maximum>
</limit>
</limits>
</rule>
</rules>
</configuration>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${jacoco.destfile}</destFile>
<append>true</append>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${jacoco.destfile}</dataFile>
<outputDirectory>${sonar.jacoco.reportPath}</outputDirectory>
<skip>${skip.surefire.tests}</skip>
</configuration>
</execution>
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${jacoco.destfile}</destFile>
<append>true</append>
<propertyName>failsafeArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report-integration</goal>
</goals>
<configuration>
<dataFile>${jacoco.destfile}</dataFile>
<outputDirectory>${sonar.jacoco.reportPath}</outputDirectory>
<skip>${skip.failsafe.tests}</skip>
</configuration>
</execution>
<!-- Disabled until such time as code quality stops this tripping
<execution>
<id>default-check</id>
<goals>
<goal>check</goal>
</goals>
<configuration>
<dataFile>${jacoco.destfile}</dataFile>
</configuration>
</execution>
-->
</executions>
</plugin>
...
And in the build section
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
</plugin>
<plugin>
<!-- for unit test execution -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
<plugin>
<!-- For integration test separation -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
<plugin>
<!-- For code coverage -->
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
</plugin>
....
And in the reporting section
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>${maven.surefire.report.plugin}</version>
<configuration>
<showSuccess>false</showSuccess>
<alwaysGenerateFailsafeReport>true</alwaysGenerateFailsafeReport>
<alwaysGenerateSurefireReport>true</alwaysGenerateSurefireReport>
<aggregate>true</aggregate>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.plugin.version}</version>
<configuration>
<excludes>
<exclude>**/*.mar</exclude>
<exclude>${jacoco.excludePattern}</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</reporting>
As Sonars sonar.jacoco.reportPath, sonar.jacoco.itReportPath and sonar.jacoco.reportPaths have all been deprecated, you should use sonar.coverage.jacoco.xmlReportPaths now. This also has some impact if you want to configure a multi module maven project with Sonar and Jacoco.
As #Lonzak pointed out, since Sonar 0.7.7, you can use Sonars report aggragation goal. Just put in you parent pom the following dependency:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.5</version>
<executions>
<execution>
<id>report</id>
<goals>
<goal>report-aggregate</goal>
</goals>
<phase>verify</phase>
</execution>
</executions>
</plugin>
As current versions of the jacoco-maven-plugin are compatible with the xml-reports, this will create for every module in it's own target folder a site/jacoco-aggregate folder containing a jacoco.xml file.
To let Sonar combine all the modules, use following command:
mvn -Dsonar.coverage.jacoco.xmlReportPaths=full-path-to-module1/target/site/jacoco-aggregate/jacoco.xml,module2...,module3... clean verify sonar:sonar
To keep my answer short and precise, I did not mention the maven-surefire-plugin and
maven-failsafe-plugin dependencies. You can just add them without any other configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
</plugin>
<sonar.language>java</sonar.language>
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.jacoco.reportPath>${user.dir}/target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.jacoco.itReportPath>${user.dir}/target/jacoco-it.exec</sonar.jacoco.itReportPath>
<sonar.exclusions>
file:**/target/generated-sources/**,
file:**/target/generated-test-sources/**,
file:**/target/test-classes/**,
file:**/model/*.java,
file:**/*Config.java,
file:**/*App.java
</sonar.exclusions>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.9</version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${sonar.jacoco.reportPath}</destFile>
<append>true</append>
<propertyName>surefire.argLine</propertyName>
</configuration>
</execution>
<execution>
<id>default-prepare-agent-integration</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${sonar.jacoco.itReportPath}</destFile>
<append>true</append>
<propertyName>failsafe.argLine</propertyName>
</configuration>
</execution>
<execution>
<id>default-report</id>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>default-report-integration</id>
<goals>
<goal>report-integration</goal>
</goals>
</execution>
</executions>
</plugin>
This sample works very well for me :
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.2</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
<!--<excludes>
<exclude>com.asimio.demo.rest</exclude>
<exclude>com.asimio.demo.service</exclude>
</excludes>-->
<propertyName>testArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
<execution>
<id>merge-results</id>
<phase>verify</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}/coverage-reports</directory>
<includes>
<include>*.exec</include>
</includes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/coverage-reports/aggregate.exec</destFile>
</configuration>
</execution>
<execution>
<id>post-merge-report</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/aggregate.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-aggregate</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${surefireArgLine}</argLine>
<!--<skipTests>${skip.unit.tests}</skipTests>-->
<includes>
<include>**/*Test.java</include>
<!--<include>**/*MT.java</include>
<include>**/*Test.java</include>-->
</includes>
<!-- <skipTests>${skipUTMTs}</skipTests>-->
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<!--<skipTests>${skipTests}</skipTests>
<skipITs>${skipITs}</skipITs>-->
<argLine>${testArgLine}</argLine>
<includes>
<include>**/*IT.java</include>
</includes>
<!--<excludes>
<exclude>**/*UT*.java</exclude>
</excludes>-->
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
You can call a ant task called merge on maven, to put all coverage files (*.exec) together in the same file.
If you are run unit tests use the phase prepare-package, if you run integration test so use post-integration-test.
This site has an example to how call jacoco ant task in maven project
You can use this merged file on sonar.
to have unit-testing AND integration-testing you can use maven-surefire-plugin and maven-failsafe-plugin with restricted includes/excludes. I was playing with CDI while getting in touch with sonar/jacoco, so i ended up in this project:
https://github.com/FibreFoX/cdi-sessionscoped-login/
Maybe it helps you a little bit. in my pom.xml i use "-javaagent" implicit by setting the argLine-option in the configuration-section of the specified testing-plugins.
Explicit using ANT in MAVEN projects is something i would not give a try, for me its to much mixing two worlds.
I only have a single-module maven project, but maybe it helps you to adjust yours to work.
note: maybe not all maven-plugins are up2date, maybe some issues are fixed in later versions
As stated in the Jacoco Wiki, you could also generate a new reporting-module:
Strategy: Module with Dependencies: The problems with aggregator
projects can be solved with an extra "reporting" module. In a
multi-module Maven project a separate module is defined that does not
contribute actual content but creates a combined coverage report. It
defines a dependency to all the modules that should be included in the
combined report. The "reporting" module would be built after its
dependencies and have access to the exec files as well as class and
source files from projects it depends on. This strategy seems to work
best with the current Maven architecture. From a user's perspective
one could argue that such a separate module would bloat the build
definition. Or, that the separate module cannot have any submodules
from which it could consume exec or class files. However, compared to
the other strategies these drawbacks seem rather minor and can be
handled in a consistent manner.
This is especially helpful if your modularisation is more complex than one parent with some child modules.
Your reporting pom could look something like:
<dependencies>
<dependency>
<groupId>org.sonarqube</groupId>
<artifactId>module1</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.sonarqube</groupId>
<artifactId>module2</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>report</id>
<goals>
<goal>report-aggregate</goal>
</goals>
<phase>verify</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>

Resources