How to measure Code Coverage of a RestFul API - maven

I have a maven based multi-module Jersey project which exposes few RestFul API. Project Structure is like -
Project
Module1
----------src
--------- unit test
Module2
----------src
--------- unit test
Module3
----------src
--------- unit test
ModuleN - this module contains Integration Tests which will hit endpoints exposed by project and test the whole service like a black box
----------src
--------- unit test
I want to build this project and execute unit tests in during build phase then create a jar, deploy this jar somewhere, execute integration tests (which is present in one module of project and this will hit the REST end points) then I wanted to measure combine coverage (unit+integration tests).
I have gone though a lot of blogs and articles but everywhere we have half information. can someone point me or guide me how can I do it.
Thanks
-Shahid

As I've answered here: You can put all the reports together in one folder (don't forget to call the different!) and use the merge mojo for that, or use a central unique file to all your reports by adding the flag "append":
-javaagent:append=true,destFile=/home/YourProject/report.exec
[Here][2] you'll find more information about how to configure the agent.
Hope it helps!
(this is a copy of my other answer to the same problem)
EDIT:
As long as you were asking for different kind of execution (unit and integration) I assume you are using different plugins for the execution of your tests.
What you have to do is to prepare two JaCoCo agents and give the argLine of them to the plugin that will run your test:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.7.201606060606</version>
<executions>
<!--
Prepares the property pointing to the JaCoCo runtime agent which
is passed as VM argument when Maven the Surefire plugin is executed.
-->
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/jacoco.exec</destFile>
<!--
Sets the name of the property containing the settings
for JaCoCo runtime agent.
-->
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!--
Ensures that the code coverage report for unit tests is created after
unit tests have been run.
-->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/jacoco.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}</outputDirectory>
</configuration>
</execution>
<!-- The Executions required by unit tests are omitted. -->
<!--
Prepares the property pointing to the JaCoCo runtime agent which
is passed as VM argument when Maven the Failsafe plugin is executed.
-->
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/jacoco-it.exec</destFile>
<!--
Sets the name of the property containing the settings
for JaCoCo runtime agent.
-->
<propertyName>failsafeArgLine</propertyName>
</configuration>
</execution>
<!--
Ensures that the code coverage report for integration tests after
integration tests have been run.
-->
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/jacoco-it.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
Then you have to configure your plugins (normally used failsafe and surefire) for accepting the argLine created for the JaCoCo agents:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>${surefireArgLine}</argLine>
</configuration>
</plugin>
The same for the failsafe plugin.

Related

Getting “Skipping JaCoCo execution due to missing execution data file” in local

I have the below pom.xml which is fine.I am able to generate jacoco.exec and see the coverage in sonarqube.But the problem is when i am run the code in local (my laptop). target/site folder is not generated and jacoco reports are not generated jacoco.exe is generated is starting folder of my app and not in target folder.
How can i solve this issue for local ?
I get the below error when i run maven install or maven test
Skipping JaCoCo execution due to missing execution data file
I have maven spring boot application
pom.xml
<properties>
<!-- Sonar -->
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.language>java</sonar.language>
</properties>
-- snipper of jacoco plugin tag
<configuration>
<destFile>${sonar.jacoco.reportPath}</destFile>
</configuration>
Quite a long time ago, we replaced the use of the "jacoco.exec" file with the use of the "jacoco.xml" file. The former is supposedly deprecated now.
We set the following properties:
sonar.coverage.jacoco.xmlReportPaths: ${basedir}/target/jacoco_report/jacoco.xml
jacoco.path: ${basedir}/target/jacoco_report
In the jacoco plugin configuration, have the following:
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${sonar.jacoco.reportPath}</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!-- Ensures that the code coverage report for unit tests is created
after unit tests have been run. -->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${sonar.jacoco.reportPath}</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${jacoco.path}</outputDirectory>
</configuration>
</execution>
In the surefire plugin configuration, we have this:
<argLine>${surefireArgLine}</argLine>

Can we execute exec-maven-plugin before maven-surefire-plugin?

Is it possible to run exec-maven-plugin before maven-surefire-plugin, what I observe during my run is maven-surefire-plugin is executing first even though the sequence in tag is second. My scenario is to execute JAVA CLASS (using exec-maven-plugin ) which generates my testng.xml and can run that using (maven-surefire-plugin).
First of all, if you have an execution of the exec-maven-plugin bound to the test phase, it is normal that this execution is performed after the one of the maven-surefire-plugin. The reason is that you're likely dealing with a project of packaging jar, which has a default binding of the Surefire Plugin to the test phase. This default execution is always the first one invoked, regardless of where the plugin is declared in the POM. In the logs, you will spot this execution with an id of default-test.
There is a way to perform actions before the tests are run by leveraging the phases invoked before the phase test. In your case, your goal is to generate a test resource, the testng.xml, so it would be appropriate to use the generate-test-resources phase, whose purpose is to create resources that are needed for the tests. Therefore, you just need to specify
<phase>generate-test-resources</phase>
to the execution of the exec-maven-plugin generating the testng.xml.
Then, you can use the generated testng.xml with the suiteXmlFiles element, see Using Suite XML Files
This is how I have implemented:
I have added test script/java main class, I want to execute before Cucumber Test Suite, in following folder:
enter image description here
Added following in POM.xml in ...
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/test/java/BeforeSuite</source> <!-- source folder where Before Suite scripts are saved -->
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<executions>
<execution>
<id>before-test-suite-scripts</id>
<phase>generate-test-resources</phase>
<goals>
<goal>java</goal>
</goals>
<configuration>
<mainClass>BeforeSuite.HelloBeforeSuiteScript</mainClass> <!-- <packagename>.<className> -->
</configuration>
</execution>
</executions>
</plugin>
When running mvn clean verify, before test suite script will run prior the test suite execution.
enter image description here

No test coverage in sonar

I have a multimodule maven project. I have 2 tests in it. Tests run during build and sonarqube shows that 100% tests (2) passed. But coverage is empty. Why is it? My other simple maven project uploads coverage successfully in sonarqube. I made the test in the same way in both projects.
There are probably more ways to do it, but the best way I found is to use the Jacoco maven plugin, configure it to add it's agent to the Surefire execution (and, optionally, if you use integration tests, Failsafe) and then add these files to the SonarQube config.
A good explanation for this I've found here. The basic stuff is quite simply (here only for surefire):
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.5.201505241946</version>
<executions>
<!--
Prepares the property pointing to the JaCoCo runtime agent which
is passed as VM argument when Maven the Surefire plugin is executed.
-->
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<!--
Sets the name of the property containing the settings
for JaCoCo runtime agent.
-->
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!--
Ensures that the code coverage report for unit tests is created after
unit tests have been run.
-->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.15</version>
<configuration>
<!-- Sets the VM argument line used when unit tests are run. -->
<argLine>${surefireArgLine}</argLine>
<!-- Skips unit tests if the value of skip.unit.tests property is true -->
<skipTests>${skip.unit.tests}</skipTests>
</configuration>
</plugin>
With that you can configure your SonarQube (Settings -> Java) to find the jacoco unittest file at /target/coverage-reports/jacoco-ut.exec, which should be enough to get the coverage reliably when running the sonar:sonar goal afterwards.

Merging sonar metrics across a maven multi-module project

I've inherited an odd maven multi module project. The standard business classes and unit tests are in a module called 'code' and we've nice simple working sonar metrics for this module. The second module holds various integration style tests which run against a deployed glassfish server, these run as unit test in the integ-test module.
pom.xml // root level,
code // business code and unit tests :-)
moduleA
moduleB
integ-test // ;-(
moduleA
moduleB
I know the simplest option is to move the 'integ-test' classes into the 'code' module and refactor them as real integration tests but for the moment this isn't an option.
I've figured out a way to record the level of code coverage for the 'integ-tests' module by using the jacoco:dump goal to download the jacococ.exec to the integ-test modules.
My maven pom structure now looks like this
pom.xml // defined jacoco verify -> report goal
code // business code and unit test
moduleA
target/jacoco.exec - unit test coverage
pom.xml
pom.xml - defines standard jacoco prepare-agent goal
integ-test //
moduleA
target/jacoco.exec - really integration test coverage
pom.xml
pom.xml- defines jacoco dump and merge
My current approach is to merge all the jacococ.exec files in the 'integ-test' module and use the 'destFile' value to save these details the 'code' module as a jacoco-it.exec file. The standard sonar:sonar goal should then pick up this file and use it as the integration test coverage.
pom.xml
integ-test
moduleA
target/jacoco.exec - really integration test coverage modA
moduleB
target/jacoco.exec - really integration test coverage modB
code
moduleA
moduleB
target/jacoco-it.exec - This should be the merged content of the two files above
pom.xml
My root level pom.xml defines this jacoco-maven-plugin config
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<configuration>
<append>true</append>
<includes>
<include>com/a/b/c/**</include>
</includes>
</configuration>
<executions>
<execution>
<id>jacoco-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>jacoco-dump</id>
<phase>pre-integration-test</phase>
<goals>
<goal>dump</goal>
</goals>
</execution>
<execution>
<id>jacoco-merge</id>
<phase>post-integration-test</phase>
<goals>
<goal>merge</goal>
</goals>
<configuration>
<fileSets>
<fileSet implementation="org.apache.maven.shared.model.fileset.FileSet">
<directory>${project.basedir}/integ-test</directory>
<includes>
<include>*.exec</include>
</includes>
</fileSet>
</fileSets>
<destFile>${sonar.jacoco.itReportPath}</destFile>
</configuration>
</execution>
<execution>
<id>jacoco-site</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<sonar.jacoco.itReportPath>${project.basedir}/../target/jacoco-it.exec</sonar.jacoco.itReportPath>
I've linked the jacoco:merge goal to the maven post-integration-test phase to ensure the merge happens after the tests are run, but the 'jacoco-it.exec' file is always created within the 'integ-test' folder.
Any suggestions?
What about pointing to destFile in target directory of the master pom? Just define
<configuration>
<append>true</append>
<destFile>${project.basedir}/../target/jacoco-it.exec</destFile>
<propertyName>jacoco.integrationtest.arguments</propertyName>
</configuration>
in the sub-projects. All coverage data will be appended in the jacoco-it.exec file.
This file is picked up by sonarqube and you will see the coverage as IT coverage.
You can use jacoco.exec as file name as well.

Getting wrong coverage data with jacoco and maven

I have a project using jacoco version 0.7.1.201405082137 and maven 3.0.5. In the project I have some unit tests, and some tests created using arquillian.
To distinguish between the unit tests and integration ones, I created two junit categories: one called FastTest and another called SlowTest.
In the maven profile that I use to run all tests I have this plugins configured:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<groups>SlowTest,FastTest</groups>
<systemPropertyVariables>
<arquillian.launch>wildfly_8_x</arquillian.launch>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${version.jacoco}</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
When I leave both categories in I get only the coverage for the tests annotated with SlowTest. But all the tests run. If I run only the ones annotated with FastTest I get their correct coverage too.
How can I set up jacoco to get the correct coverage when running both kinds of tests?
Small tip:
<groups> tag takes full class name. So my question: is SlowTest.class interface placed in the default package? If not you should provide full path, something like: <groups>com.mycompany.project.SlowTest</groups>
And little advice:
Good practice is to distinguish unit and integration tests - and thus run them separately. Maven accomplishes this by two plugins: maven-surefire-plugin and maven-failsafe-plugin.
First is designed to run unit tests with mvn test. Second is designed to run your integration tests with mvn failsafe:integration-test. This answer may be useful to shed some light.

Resources