Jacoco agent for integration tests on remote machine - maven

I'm using maven and jacoco-maven-plugin for test coverage reports
I need to configure the jacoco-agent to be on a remote machine (Linux), on which I'm running script that runs the integration tests
How do I make it?
Here's what I tried:
<execution>
<id>prepare-agent-integration</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${sonar.jacoco.itReportPath}</destFile>
<jacoco.address>172.X.X.X</jacoco.address><!-- This is the machine where integration tests runs-->
<jacoco.port>22</jacoco.port>
</configuration>
</execution>
<execution>
<id>report-integration</id>
<goals>
<goal>report-integration</goal>
</goals>
<configuration>
<dataFile>${sonar.jacoco.itReportPath}</dataFile>
<outputDirectory>${project.basedir}/target/site/jacoco-it</outputDirectory>
</configuration>
</execution>

This is how the plugin is configured to see the results in a Sonar, the surefire plugin and the failsafe plugin are used in combination too. Maybe this configuration could help you.
http://maven.apache.org/surefire/maven-surefire-plugin/
http://maven.apache.org/surefire/maven-failsafe-plugin/
Properties
<sonar.jacoco.reportPath>target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.jacoco.outReportPath>target/jacoco</sonar.jacoco.outReportPath>
<sonar.jacoco.itReportPath>target/jacoco-it.exec</sonar.jacoco.itReportPath>
<sonar.jacoco.outItReportPath>target/jacoco-it</sonar.jacoco.outItReportPath>
<!-- Only unit tests are run by default. -->
<skip.integration.tests>true</skip.integration.tests>
<skip.unit.tests>false</skip.unit.tests>
maven-surefire-plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.15</version>
<configuration>
<!-- Sets the VM argument line used when unit tests are run. -->
<argLine>${surefireArgLine}</argLine>
<!-- Skips unit tests if the value of skip.unit.tests property is true -->
<skipTests>${skip.unit.tests}</skipTests>
<!-- Excludes integration tests when unit tests are run. -->
<excludes>
<exclude>**/*IntegrationTest.java</exclude>
</excludes>
</configuration>
</plugin>
maven-failsafe-plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.15</version>
<executions>
<!-- Ensures that both integration-test and verify goals of the Failsafe Maven plugin are executed. -->
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<!-- Sets the VM argument line used when integration tests are run. -->
<argLine>${failsafeArgLine}</argLine>
<!-- Skips integration tests if the value of skip.integration.tests property is true -->
<skipTests>${skip.integration.tests}</skipTests>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
jacoco-maven-plugin
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.0.201403182114 </version>
<executions>
<!-- Prepares the property pointing to the JaCoCo runtime agent which is passed as VM argument when Maven the Surefire plugin is executed. -->
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${sonar.jacoco.reportPath}</destFile>
<!-- Sets the name of the property containing the settings for JaCoCo runtime agent. -->
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!-- Ensures that the code coverage report for unit tests is created after unit tests have been run. -->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${sonar.jacoco.reportPath}</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${sonar.jacoco.outReportPath}</outputDirectory>
</configuration>
</execution>
<!-- Prepares the property pointing to the JaCoCo runtime agent which is passed as VM argument when Maven the Failsafe plugin is executed. -->
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${sonar.jacoco.itReportPath}</destFile>
<!-- Sets the name of the property containing the settings for JaCoCo runtime agent. -->
<propertyName>failsafeArgLine</propertyName>
</configuration>
</execution>
<!-- Ensures that the code coverage report for integration tests after integration tests have been run. -->
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${sonar.jacoco.itReportPath}</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${sonar.jacoco.outItReportPath}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>

Related

Why 0% in Jacoco coverage for integration tests

I have setup maven project to get the integration test coverage in my it profile. But the jacoco report shows 0% for all the classes.
My main POM Jacoco integration setup section in the profile and failsafe plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<executions>
<!--
Invokes both the integration-test and the verify goals of the
Failsafe Maven plugin
-->
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<!--
Skips integration tests if the value of skip.it.tests
property is true
-->
<argLine>#{failsafeArgLine}</argLine>
<skipTests>${skip.it.tests}</skipTests>
<testFailureIgnore>true</testFailureIgnore>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<!-- Include your Cucumber tests, as an example -->
<include>**/*CucumberTests.java</include>
<include>**/*IT.java</include>
</includes>
<systemPropertyVariables>
<it.server.port>${random.http.port}</it.server.port>
<it.jmx.port>${random.jmx.port}</it.jmx.port>
</systemPropertyVariables>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
.....
<profiles>
<profile>
<id>it</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<packaging.type>jar</packaging.type>
<run.profiles>it</run.profiles>
<skip.it.tests>true</skip.it.tests>
<skipTests>true</skipTests>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<configuration>
<append>true</append>
<!-- Sets the path to the file which contains the execution data. ;-->
<destFile>${user.dir}/target/jacoco-it.exec</destFile>
<!-- Sets the path to the file which contains the execution data.-->
<dataFile>${user.dir}/target/jacoco-it.exec</dataFile>
<!-- Sets the name of the property containing the settings for JaCoCo runtime agent.-->
<propertyName>failsafeArgLine</propertyName>
<excludes>
<exclude>**/*App*</exclude>
<exclude>**/test/**/*</exclude>
<exclude>**/it/**/*</exclude>
<exclude>**/config/*Config*</exclude>
</excludes>
<!-- Sets the output directory for the code coverage report.-->
<outputDirectory>${user.dir}/target/coverage-reports/jacoco-it</outputDirectory>
</configuration>
<executions>
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<!--
Ensures that the code coverage report for integration tests after
integration tests have been run.
-->
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
The profile setting of the module POM which has api tests where the command is executed
<profile>
<id>it</id>
<properties>
<skip.it.tests>false</skip.it.tests>
<spring.profile>it</spring.profile>
<skipTests>true</skipTests> -- This is for unit tests
</properties>
</profile>
However I can see jacoco-it.exec file has been generated and the report file as below.
Version:
<jacoco.plugin.version>0.8.3</jacoco.plugin.version>
<maven-failsafe-plugin.version>3.0.0-M5</maven-failsafe-plugin.version>
Command to execute tests: mvn clean verify -P it. This will spin up a server and run all integration tests (api end point tests) there.
Further when setup for unit tests that work as expected with surefire plugin with showing proper coverage in jacoco reports.

Unit tests are getting executed twice when performing mvn install

Unit tests are getting executed twice.
When i am removing goal report and phase prepare-package from maven plugins in pom, test are getting executed once but then coverage are not getting generated in the console.
But when i am adding goal report and phase prepare-package from maven plugins in pom,i am getting coverage in the console but unit tests are getting executed twice.
I need to have goal report and phase prepare-package in my pom in order to get coverage but need to run test cases only once. What is the way to to get the test case executed only once with coverage as well.
<plugins>
<!-- Configure maven-compiler-plugin to use the desired Java version -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<!-- Use build-helper-maven-plugin to add Scala source and test source
directories -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/main/scala</source>
</sources>
</configuration>
</execution>
<execution>
<id>add-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/test/scala</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<!-- Use scala-maven-plugin for Scala support -->
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.2</version>
<executions>
<execution>
<goals>
<!-- Need to specify this explicitly, otherwise plugin won't be called
when doing e.g. mvn compile -->
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- disable surefire -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
<plugin>
<groupId>org.scoverage</groupId>
<artifactId>scoverage-maven-plugin</artifactId>
<version>${scoverage.plugin.version}</version>
<configuration>
<aggregate>true</aggregate>
<highlighting>true</highlighting>
<scalacPluginVersion>1.3.0</scalacPluginVersion>
<minimumCoverage>30</minimumCoverage>
<failOnMinimumCoverage>false</failOnMinimumCoverage>
</configuration>
<executions>
<execution>
<goals>
<goal>report</goal> <!-- or integration-check -->
</goals>
<phase>prepare-package</phase> <!-- or any other phase -->
</execution>
</executions>
</plugin>
<!-- enable scalatest -->
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<argLine>
-javaagent:"${settings.localRepository}"/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar
</argLine>
<forkMode>once</forkMode>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
<filereports>WDF TestSuite.txt</filereports>
<!-- The scalatest-maven-plugin seems broken for the spanScaleFactor,
so pass it via system property, instead -->
<!-- <spanScaleFactor>${scalatest.span.scale.factor}</spanScaleFactor> -->
<systemProperties>
<spanScaleFactor>${scalatest.span.scale.factor}</spanScaleFactor>
</systemProperties>
</configuration>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>enter code here
You need to do 2 things:
Add following line in 'scoverage-maven-plugin' plugin's 'configuration' section.
<additionalForkedProjectProperties>skipTests=false</additionalForkedProjectProperties>
Set following property in your root pom.xml
<properties>
<!-- Add other properties here... -->
<!-- skipTests is set to 'true' here to avoid duplicate runs of all test cases. It's set to false in scoverage-maven-plugin below. -->
<skipTests>true</skipTests>
</properties>

Reports are not generated when the build is failed in Maven Cucumber Reports

Reports are generating successfully when the build is successful but when there are any failed cases which cause build failure, reports are not generated.
checkBuildResult is already set to false
pom file plugin
<plugin>
<groupId>net.masterthought</groupId>
<artifactId>maven-cucumber-reporting</artifactId>
<version>3.13.0</version>
<executions>
<execution>
<id>execution</id>
<phase>verify</phase>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<projectName>Simplify360 Automation Test Report</projectName>
<outputDirectory>${project.build.directory}/site/cucumber-reports</outputDirectory>
<cucumberOutput>${project.build.directory}/cucumber.json</cucumberOutput>
<!-- <jsonFiles>
<param>${project.build.directory}/cucumber.json</param>
</jsonFiles> -->
<!-- <parallelTesting>false</parallelTesting> -->
<buildNumber>8.4.1.2</buildNumber>
<checkBuildResult>false</checkBuildResult>
</configuration>
</execution>
</executions>
</plugin>
And the runner class is as below,
#RunWith(Cucumber.class)
#CucumberOptions(
features = {"classpath:features"},
plugin = {"pretty","json:target/cucumber.json"},
tags = {"#currentTest"},
glue={"helpers","stepDefinitions"},
monochrome = true
)
public class RunCukesTest{
}
Add the following configuration to the sure fire plugin. It will not stop the maven execution after the failure. Then it will generate the report.
<testFailureIgnore>true</testFailureIgnore>
as given below with your existing configuration.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
There is flag present for same which is as below:
<checkBuildResult>true</checkBuildResult>
<!-- Set true to fail build on test failures -->
<!-- Set false to pass build on test failures -->
You need to set in the configuration tag as below:
<configuration>
<projectName>oasys-confirmations</projectName>
<outputDirectory>${project.build.directory}</outputDirectory>
<cucumberOutput>${project.build.directory}/cucumber.json</cucumberOutput>
<checkBuildResult>true</checkBuildResult>
</configuration>
New config:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
<plugin>
<groupId>net.masterthought</groupId>
<artifactId>maven-cucumber-reporting</artifactId>
<version>3.20.0</version>
<executions>
<execution>
<id>execution</id>
<phase>verify</phase>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<projectName>cucumber-jvm-example</projectName>
<!-- output directory for the generated report -->
<outputDirectory>${project.build.directory}</outputDirectory>
<!-- optional, defaults to outputDirectory if not specified -->
<inputDirectory>${project.build.directory}/</inputDirectory>
<jsonFiles>
<!-- supports wildcard or name pattern -->
<param>**/*.json</param>
</jsonFiles>
<!-- optional, defaults to outputDirectory if not specified -->
<classificationDirectory>${project.build.directory}/</classificationDirectory>
<classificationFiles>
<!-- supports wildcard or name pattern -->
<param>sample.properties</param>
<param>other.properties</param>
</classificationFiles>
<parallelTesting>false</parallelTesting>
<checkBuildResult>true</checkBuildResult>
</configuration>
</execution>
</executions>
</plugin>
Old Config working till 3.16.0 version:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
<plugin>
<groupId>net.masterthought</groupId>
<artifactId>maven-cucumber-reporting</artifactId>
<version>3.16.0</version>
<executions>
<execution>
<id>execution</id>
<phase>verify</phase>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<projectName>>cucumber-jvm-example</projectName>
<outputDirectory>${project.build.directory}</outputDirectory>
<cucumberOutput>${project.build.directory}/cucumber.json</cucumberOutput>
<checkBuildResult>true</checkBuildResult>
</configuration>
</execution>
</executions>
</plugin>
NOTE If you get same problem After this configuration then
1. Run RunnerFile Normally TestNG
2. Run Pom.xml as Maven install.
3. Check target folder there is a TagName.html file open it and view the result.

Merging Integration and Unit test reports with JaCoCo

I am generating failsafe and surefire reports with maven with the JaCoCo plugin, but I can only manage to get them in separate reports. I would like to have a overall coverage view (merge between both Unit Tests and Integration Tests) as well.
After what I think was a thorough google search I could only found a way to do this with Sonar. Is there any simpler way to do this?
Related question: Maven separate Unit Test and Integration Tests
I recently implemented this: after some headaches and a lot of testing, I have a configuration that works beautifully.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
<propertyName>testArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
<execution>
<id>merge-results</id>
<phase>verify</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}/coverage-reports</directory>
<includes>
<include>*.exec</include>
</includes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/coverage-reports/aggregate.exec</destFile>
</configuration>
</execution>
<execution>
<id>post-merge-report</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/aggregate.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-aggregate</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${surefireArgLine}</argLine>
<skipTests>${skip.unit.tests}</skipTests>
<includes>
<include>**/*UT.java</include>
<include>**/*MT.java</include>
<include>**/*Test.java</include>
</includes>
<skipTests>${skipUTMTs}</skipTests>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<skipTests>${skipTests}</skipTests>
<skipITs>${skipITs}</skipITs>
<argLine>${testArgLine}</argLine>
<excludes>
<exclude>**/*UT*.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
As you can see, there are 6 distinct Jacoco executions to run the tests, merge the files and create an aggregate report. On top of the Jacoco config, you also need to configure Surefire and Failsafe to take an argument from Jacoco (Surefire runs the unit tests and Failsafe runs the integration tests).
All of the configuration that I used should be there, what you do with it is your design architecture to make it fit within your desired requirements. Personally, I recommend a look into what I exclude and include within surefire and failsafe if you are having issues with files not being read.
Not the answer you're looking for, but still...
In most cases, you should not merge coverage from unit and integration tests.
The value of unit tests is that they improve the design of your application and ensure that corner cases of your code is working correctly. You should try to have a high branch coverage of your unit tests.
The value of your integration tests is that they ensure that the main use cases of your application are working correctly and that the whole stack is integrated correctly. You should try to have a high functional coverage for your integration tests. (But it is fairly hard to measure functional coverage with a tool).
If you need integration tests to improve your branch coverage, that's a strong indication that you should review the design of your code. And if you already have a high branch coverage without integration tests, adding them should not modify significantly your metrics.
You should take a look into the documentation of JaCoCo Maven plugin which contains a merge goal.
Maybe more nice way is to achieve is use same jacoco file, but let it another tests, which works for us - unit and it tests.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.1</version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<skip>${maven.surefire.skipTests}</skip>
<propertyName>maven.surefire.argLine</propertyName>
<!-- using the same dest file for both UT and IT -->
<destFile>${sonar.jacoco.reportPath}</destFile>
</configuration>
</execution>
<execution>
<id>default-prepare-agent-integration</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<skip>${maven.failsafe.skipTests}</skip>
<propertyName>maven.failsafe.argLine</propertyName>
<!-- append to the UT dest file -->
<destFile>${sonar.jacoco.reportPath}</destFile>
<append>true</append>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>${maven.surefire.skipTests}</skipTests>
<failIfNoTests>${maven.surefire.failIfNoTests}</failIfNoTests>
<!-- allow argLine to be modified by other plugins, e.g. jacoco -->
<argLine>#{maven.surefire.argLine}</argLine>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<skipTests>${maven.failsafe.skipTests}</skipTests>
<failIfNoTests>${maven.failsafe.failIfNoTests}</failIfNoTests>
<!-- allow argLine to be modified by other plugins, e.g. jacoco -->
<argLine>#{maven.failsafe.argLine}</argLine>
</configuration>
</plugin>
Building on top of the answer of Chad, here is my configuration. It is configured as a separate profile named jacoco so I can turn it on and off easily. It uses only the default configuration. It can create a separate code coverage report for unit tests, separate code coverage report for integration tests and also a combined code coverage report.
<profile>
<id>jacoco</id>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
</execution>
<execution>
<id>post-integration-test</id>
<goals>
<goal>report-integration</goal>
</goals>
</execution>
<execution>
<id>merge-results</id>
<phase>verify</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<includes>
<include>*.exec</include>
</includes>
<excludes>
<exclude>aggregate.exec</exclude>
</excludes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/aggregate.exec</destFile>
</configuration>
</execution>
<execution>
<id>post-merge-report</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/aggregate.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-aggregate</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
I get the point where Unit tests should really be at the source of code coverage... but sometimes, one would like to have the ability to know right? Here is what I did (this is from using gradle).
plugins {
[...]
id 'jacoco'
}
[...]
test {
jacoco { /* specify the "exec" data file name (see the "Unit" in there) */
destinationFile = file("$buildDir/jacoco/jacocoTestUnit.exec")
classDumpDir = file("$buildDir/jacoco/classpathdumpsUnit")
}
useJUnitPlatform { /* I use the JUnit Jupiter #Tag Annotation to create my suites... you are free to do something else */
excludeTags 'IT'
}
description = "Run unit tests"
group = "verification"
}
task intTest(type: Test) {
jacoco {
destinationFile = file("$buildDir/jacoco/jacocoTestInt.exec")
classDumpDir = file("$buildDir/jacoco/classpathdumpsInt")
}
useJUnitPlatform {
includeTags 'IT'
}
description = "Run integration tests"
group = "verification"
}
jacocoTestReport {
/*
* find all test exec files and feed them to jacoco
*/
def list = []
def dir = new File("${buildDir}/jacoco/")
if(dir.exists()) {
dir.eachFileRecurse(FileType.FILES) { file ->
/* gather all the "exec" files available */
if (file.getName().startsWith("jacocoTest") && file.getName().endsWith(".exec")) {
list << file.getAbsolutePath()
}
}
/* provide all the "exec" files to jacoco */
executionData.from = files(list)
}
/*
* you must run all tests before running jacoco.
* We want the liberty to run partial tests instead of all,
* so this task doesn't depend on any test task.
*/
reports {
xml.enabled true
}
}
With this, you can have the the possibility to get the coverage from Unit tests with:
./gradlew clean test jacocoTestReport
Or you can have the coverage from Integration tests with:
./gradlew clean intTest jacocoTestReport
Or you can have the overall coverage of both unit and Integration tests with:
./gradlew clean test inTest jacocoTestReport
Disclaimer: I'm no Jacoco or Gradle expert... feel free to comment anything I may have forgotten. This worked beautifully so far for my needs.
In order to merge the reports here, I have a complete working solution.
Please note that in order to work properly the merge strategy, the phases should be executed sequentially (if the mvn test and mvn verify -DskipUnitTests will be executed in parallel there is possible to not work fine).
<!-- Jacoco is used to generate the reports for SonarQube -->
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.6</version>
<configuration>
<skip>${skipTests}</skip>
</configuration>
<executions>
<!-- Prepares the property pointing to the JaCoCo runtime agent which
is passed as VM argument when Maven the Surefire plugin is executed. -->
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<!-- Sets the name of the property containing the settings for JaCoCo runtime agent. -->
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!-- Make sure that after the Unit Tests execution, the jacoco-ut.exec file is generated,
will be merged to the aggregation file -->
<execution>
<id>post-unit-merge</id>
<phase>test</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<includes>
<include>**/coverage-reports/*.exec</include>
</includes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/coverage-reports/jacoco_aggregation.exec</destFile>
</configuration>
</execution>
<!-- Ensures that the code coverage report is created/updated after unit tests have been run. -->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/coverage-reports/jacoco_aggregation.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}/jacoco</outputDirectory>
</configuration>
</execution>
<!-- Prepares the property pointing to the JaCoCo runtime agent which
is passed as VM argument when Maven the Failsafe plugin is executed. -->
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
<!-- Sets the name of the property containing the settings for JaCoCo runtime agent. -->
<propertyName>failsafeArgLine</propertyName>
</configuration>
</execution>
<!-- Make sure that after the Integration Test execution, the jacoco-it.exec file is generated,
will be merged to the aggregation file -->
<execution>
<id>post-integration-merge</id>
<phase>post-integration-test</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<includes>
<include>**/coverage-reports/*.exec</include>
</includes>
</fileSet>
</fileSets>
<destFile>${project.build.directory}/coverage-reports/jacoco_aggregation.exec</destFile>
</configuration>
</execution>
<!-- Ensures that the code coverage report is created/updated after integration tests have been run. -->
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/coverage-reports/jacoco_aggregation.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}/jacoco</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<!-- Sets the VM argument line used when unit tests are run. -->
<argLine>${surefireArgLine}</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<!-- Sets the VM argument line used when integration tests are run. -->
<argLine>${failsafeArgLine}</argLine>
</configuration>
</plugin>
Now, since the reports are generated, can be executed the sonar command in order to publis the report:
mvn sonar:sonar -Dsonar.coverage.jacoco.xmlReportPaths="target/site/jacoco/jacoco.xml"
This works out-of-the-box.
Explanations
The prepare-agent goal integrates by default with both SureFire and FailSafe plugins (to be exact the argLine parameter set by prepare-agent is consumed by both plugins). Additionally, prepare-agent by default configures Jacoco so that it appends coverage results if the file already exists, so in the end you get single target/jacoco.exec file which contains results from both unit test and integration tests, combined.
Just in case this is the relevant config, as you see nothing changed from default:) :
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<executions>
<execution>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.6</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
<goal>report</goal>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>

Jenkins jacoco plugin empty report

I have a project where I use Jacoco to calculate the code coverage.
I use maven configuration like here :
http://www.petrikainulainen.net/programming/maven/creating-code-coverage-reports-for-unit-and-integration-tests-with-the-jacoco-maven-plugin/
With Jenkins, I run "mvn clean install test". It generates the report in the /target/site/jacoco-ut/ folder. If I open the index.html file, I see this :
But when I open JaCoCo Coverage Report in jenkins job, I see this :
It says that every test cover 0% of the code. I don't understand why I don't have the same result that I have in the html report.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.5.201505241946</version>
<executions>
<!--
Prepares the property pointing to the JaCoCo runtime agent which
is passed as VM argument when Maven the Surefire plugin is executed.
-->
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<!--
Sets the name of the property containing the settings
for JaCoCo runtime agent.
-->
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!--
Ensures that the code coverage report for unit tests is created after
unit tests have been run.
-->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.15</version>
<configuration>
<skip>true</skip>
</configuration>
<executions>
<execution>
<id>surefire-unit-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<!-- Sets the VM argument line used when unit tests are run. -->
<argLine>${surefireArgLine}</argLine>
<!-- Skips unit tests if the value of skip.unit.tests property is true -->
<skipTests>${skip.unit.tests}</skipTests>
<!-- Excludes integration tests when unit tests are run. -->
<skip>false</skip>
<excludes>
<exclude>**/*IntegrationTests.java</exclude>
</excludes>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.15</version>
<executions>
<!--
Ensures that both integration-test and verify goals of the Failsafe Maven
plugin are executed.
-->
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<!-- Sets the VM argument line used when integration tests are run. -->
<argLine>${failsafeArgLine}</argLine>
<!--
Skips integration tests if the value of skip.integration.tests property
is true
-->
<skipTests>${skip.integration.tests}</skipTests>
</configuration>
</execution>
</executions>
</plugin>
And my jenkins configuration :
What is my error ?
I found a jenkins issue similar to my problem :
https://issues.jenkins-ci.org/browse/JENKINS-28652
The error is caused by jacoco version.
The version 0.7.5.201505241946 is bugged.
I switch for the version 0.7.4.201502262128 and now it works.
I had similar issue and it is fixed by just updating the "jacoco-maven-plugin" version to 0.7.7.201606060606 and "maven-surefire-plugin" version to 2.19.1
I hope this will work for you.
Try to use in Jenkins Plugin :
Path to exec files = /.exec
And I use this pom :
<jacoco.it.execution.data.file>${project.build.directory}/coverage-reports/jacoco-it.exec</jacoco.it.execution.data.file>
<jacoco.ut.execution.data.file>${project.build.directory}/coverage-reports/jacoco-ut.exec</jacoco.ut.execution.data.file>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<destFile>${jacoco.it.execution.data.file}</destFile>
<propertyName>failsafeArgLine</propertyName>
</configuration>
</execution>
<!-- Ensures that the code coverage report for integration tests after -->
<!-- integration tests have been run. -->
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report-integration</goal>
</goals>
<configuration>
<dataFile>${jacoco.it.execution.data.file}</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>

Resources