We have a small multi-module Java Spring REST application that is building with the maven surefire plugin. Because the application is fairly small, we wrote a mix of unit tests and integration tests and run them all as unit tests, so we get a single set of jacoco.exec files.
When I pull up the test coverage in SonarQube, the coverage is only being calculated for the module in which the test was run. So the integration tests that we have in the REST controller layer are not generating any coverage in the business or data layers.
I have tried multiple configurations of jacoco and I'm tired of staring at it. Any suggestions would be helpful.
jacoco configuration in parent pom.xml:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.7.201606060606</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
surefire configuration in parent pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<systemPropertyVariables>
<org.jboss.logging.provider>slf4j</org.jboss.logging.provider>
</systemPropertyVariables>
</configuration>
</plugin>
the coverage is only being calculated for the module in which the test was run
This is how the import of JaCoCo report in SonarQube works.
If you want to overcome this, you may want to merge the reports before feeding it to SonarQube.
Related
This question already has answers here:
How to configure multi-module Maven + Sonar + JaCoCo to give merged coverage report?
(13 answers)
Closed 6 years ago.
I was trying to generate code coverage reports using jacoco plugin in maven for a multi module project that I was working on.
I added the following in my parent pom.xml within the build tags.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8-SNAPSHOT</version>
<configuration>
<output>file</output>
<append>true</append>
</configuration>
<executions>
<execution>
<id>jacoco-initialize</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>jacoco-site</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${argLine}</argLine>
</configuration>
</plugin>
On running mvn verify, the respective jacoco reports were generated for each module at "project-root\module\target\site\jacoco\"
Is it possible to generate a consolidated jacoco report at the project-root containing the test coverage details of each module?
Please suggest the best possible way to merge the individual module reports.
Sure is!
Took me a while and a few sources to cook this pattern up, but has worked well.
For a multi-module Maven project:
ROOT
|--LIB-1
|--LIB-2
The LIB projects both have their own unit tests.
ROOT pom.xml
<!- properties-->
<jacoco.reportPath>${project.basedir}/../target/jacoco.exec</jacoco.reportPath>
<!-- build/plugins (not build/pluginManagement/plugins!) -->
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.6.201602180812</version>
<executions>
<execution>
<id>agent-for-ut</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${jacoco.reportPath}</destFile>
</configuration>
</execution>
</executions>
</plugin>
LIB projects pom.xml will inherit the the JaCoCo plugins execution, so just need to wire up the argline in the Surefire plugin.
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${argLine}</argLine>
</configuration>
</plugin>
I have an extended answer for rolling up integration tests as well as unit tests for JaCoCo being reported via Sonar, you can see my detailed answer here.
In addition to the steps suggested in markdsievers detailed answer, I had to setup sonarqube-5.3 (supports jdk 7+) in localhost:9000
Setup SonarQube
And use mvn package to generate jacoco.exec files.
Then mvn sonar:sonar to generate report in sonar dashboard.
I have the following task at hand:
-- find IT code coverage for a project
Given situation:
-- IT code resides in a repository separate to the actual production code
-- Production code that the tests were created for reside in more than one git repository.
-- all of the above uses maven and are written in Java.
I have tried following different tutorial and blogs but couldnt find a simpler answer.
Can anyone either point me towards the right resource or give me hints for a kick start?
I will try to answer. I will post example with UT (IT is the same thing just not at the same place in the maven livecycle build, and instead of the surefire plugin its the failsafe plugin)
Lets say we use JaCoCo for the code coverage agent.
In my Parent pom, in the profile section (it is a multi module project)
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.15</version>
<configuration>
<argLine>${surefireArgLine}</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.4.201502262128</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
Now when we build our maven project, we add the profile to inject the JaCoCo Agent
clean install -Psonar-coverage
Then we may tell sonar to analyse our project and to use the JaCoCo report with the following command
mvn org.codehaus.mojo:sonar-maven-plugin:2.4:sonar -Dsonar.dynamicAnalysis=reuseReports -Dsonar.java.coveragePlugin=jacoco -Dsonar.forceAnalysis=true -Dsonar.jacoco.reportMissing.force.zero=true -Dsonar.binaries=target/classes -Dsonar.junit.reportsPath=target/surefire-reports -Dsonar.jacoco.reportPath=target/coverage-reports/jacoco-ut.exec -Dsonar.jdbc.driver=com.mysql.jdbc.Driver -Dsonar.jdbc.url=jdbc:<YOUR SONAR INSTALLATION DB> -Dsonar.host.url=<YOUR SONAR INSTALLATION>
i'm trying to put some bdd into my eclipse plugin project, but can't figure out how to run my integration tests during maven build fase. To write my tests i'm using SWTBot framework.
I already did the feature generation fase, and setup my tests. How do i setup my pom to run my integration tests?
I use below configuration and run mvn clean verify. If you don't want to run tests in parallel, remove parallel, perCoreThreadCount and threadCountClasses tags.
Make sure to update the regular expression to match your test naming convention <include>**/Run*.java</include>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<id>acceptance-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<outputEncoding>UTF-8</outputEncoding>
<parallel>classes</parallel>
<perCoreThreadCount>true</perCoreThreadCount>
<threadCountClasses>10</threadCountClasses>
<argLine>-Xmx1024m</argLine>
<argLine>-XX:MaxPermSize=256m</argLine>
<includes>
<include>**/Run*.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
Having following Maven project structure:
-project1 <-- parent pom with two children.
|--module1 <-- web services
\--module1-itest <-- integration tests written with TestNG
What we do today:
Run mvn sonar:sonar in module1, which shows code coverage from
the Unit tests in Sonar's dashboard.
Run mvn jetty:run on module1, and immediately after that run mvn test on module1-itests to test it.
I know this is far from the ideal scenario... it's more like an intermediate step while we try to improve a legacy project with almost no tests at all...
My question is: what would be the best way to get the code coverage done by the execution of the integration tests in Sonar's dashboard of module1?
Initially, I'm inclined to move the code in module1-itest to module1, and run them using the Failsafe plugin and the well-documented integration with JaCoCo. In this way, Module1 will have a mix of JUnit unit tests and TestNG integration tests, each group to be run by Surefire and Failsafe, respectively, starting up a Jetty server in the pre-integration phase.
However, we have reasons to keep both projects separated, so I'm wondering:
Is the approach above a good one?
Is there any other recommended way we can use to keep both projects separated, but including the code coverage done by module1-itest in module1 Sonar's dashboard?
Thanks,
This is how we solved it:
Summary:
Service and Service-itest projects are two independent modules.
Service has unit tests which coverage is reported by Sonar.
Service-itests uses Cargo to load the service WAR and run the integration tests. The code coverage collected in this execution is reported one level above. In this way, the coverage for the ITests is collected and merged cross-module and reported at the parent project pom level. We also like this approach becasue it allows us to have different ITests projects (for example, one with Cargo loading the app and another using JBehave), and they can evolve independently.
In the parent pom:
<modules>
<module>service</module>
<module></module>
<module>service-itest</module>
</modules>
<!-- Sonar -->
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<!-- The destination file for the code coverage report has to be set to
the same value in the parent pom and in each module pom. Then JaCoCo will
add up information in the same report, so that, it will give the cross-module
code coverage. -->
<sonar.jacoco.itReportPath>${project.basedir}/../target/jacoco-itests.exec</sonar.jacoco.itReportPath>
...
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.6.3.201306030806</version>
</plugin>
</plugins>
</pluginManagement>
...
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<configuration>
<includes>
<include>our.project.packages.*</include>
</includes>
</configuration>
<executions>
<execution>
<id>pre-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>post-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
And in the pom.xml file of the -itest project:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<configuration>
<!-- The destination file for the code coverage report has to be set
to the same value in the parent pom and in each module pom. Then JaCoCo will
add up information in the same report, so that, it will give the cross-module
code coverage. -->
<destFile>${project.basedir}/../target/jacoco-itests.exec</destFile>
<includes>
<include>our.packages.*</include>
</includes>
</configuration>
<executions>
<execution>
<id>post-test</id>
<configuration>
<skip>true</skip>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<version>1.2.2</version>
<configuration>
<skip>${cargo.skip}</skip>
<container>
<containerId>jetty7x</containerId>
<artifactInstaller>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-distribution</artifactId>
<version>7.6.12.v20130726</version>
<type>zip</type>
</artifactInstaller>
</container>
<configuration>
<type>standalone</type>
<properties>
<cargo.servlet.port>${jetty.port}</cargo.servlet.port>
<cargo.jvmargs>${argLine}</cargo.jvmargs>
</properties>
<deployables>
<deployable>
<artifactId>pam_filetask_manager_service</artifactId>
<groupId>${project.groupId}</groupId>
<type>war</type>
<pingURL>http://server:22000/ping</pingURL>
<properties>
</properties>
</deployable>
</deployables>
</configuration>
</configuration>
<executions>
<execution>
<id>start-server</id>
<phase>pre-integration-test</phase>
<goals>
<goal>install</goal>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop-server</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
AFAIK, the maven failsafe plugin fails safe because it has separate goals for running the tests and failing the build based on the tests. These are designed to be bound to the integration-test and verify goals respectively. This allows post-integration-test bound goals to run (shutting down the build) before the build fails.
My question is, how do I do this with the maven-soapui-plugin? I thought I could simply specify <testFailIgnore>true</testFailIgnore> in my soapui plugin config and then call the failsafe plugin verify goal, but that isn't working. I don't think I'm not sure if I'm getting a summary file out of the soapui plugin or not. I keep getting Expected root element 'failsafe-summary' but found 'testsuite' Here is a snippet of the POM:
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>4.0.0</version>
<configuration>
<junitReport>true</junitReport>
<exportAll>true</exportAll>
<outputFolder>${project.build.directory}/surefire-reports</outputFolder>
<testFailIgnore>true</testFailIgnore>
<printReport>true</printReport>
</configuration>
<executions>
<execution>
<id>FailingTest</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<projectFile>${basedir}/testData/soapui-integration-tests.xml</projectFile>
<host>localhost</host>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.9</version>
<executions>
<execution>
<phase>verify</phase>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
<configuration>
<summaryFiles>
<summaryFile>${project.build.directory}/surefire-reports/TEST-TestSuite_1.xml</summaryFile>
</summaryFiles>
</configuration>
</execution>
</executions>
</plugin>
Is there something wrong with my POM or is this a bad approach? Are there any better approaches?
There is an open source extension of the soapui plugin which has a separate test-verify goal for exactly this purpose.
https://github.com/redfish4ktc/maven-soapui-extension-plugin
The following example shows the required configuration:
https://github.com/redfish4ktc/maven-soapui-extension-plugin/blob/master/src/it/test-verify_goal/one_failing_project/pom.xml
AFAIK maven-failsafe-plugin can only verify success status of tests run by maven-failsafe-plugin and not by maven-soapui-plugin. It does that by reading test summary report file (failsafe-summary.xml) which has specific format.
maven-soapui-plugin could be adjusted to separate running tests from verifying test success status, to support running post-integration-test tasks (stop server, undeploy artifacts, etc.) before verification. Create a support ticket for this to soapUI guys.
Maybe even maven-failsafe-plugin, it's verify mojo, could be extended to allow specifying different test report format (JUnit style reports are supported by soapUI) or even an xpath expression which would be used by maven-failsafe-plugin to determine if there were failed tests or not. Create a support ticket for this on maven-failsafe-plugin issue tracker.
Until those extensions are supported, and you need to do tasks on post-integration-test phase you can use soapUI JUnit integration and have maven-failsafe-plugin run those soapUI JUnit tests.
I am trying this solution, and it doesn't work.
But I have found one.
To obtain de tests report of SOAPUI tests in Jenkins, I using the failsafe plugin with this configuration in the pom.xml of my Soap tests folder :
<build>
<plugins>
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>2.6.1</version>
<configuration>
<projectFile>${basedir}/soap_project_tests.xml</projectFile>
<outputFolder>${filePath.reports.soap}</outputFolder>
<testFailIgnore>true</testFailIgnore>
<junitReport>true</junitReport>
<exportwAll>true</exportwAll>
<printReport>true</printReport>
</configuration>
<executions>
<execution>
<id>run-soap-integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.11</version>
<configuration>
<reportsDirectory>${filePath.reports.soap}</reportsDirectory>
<printSummary>true</printSummary>
<argLine>-Xmx512m</argLine>
</configuration>
<executions>
<execution>
<id>soap-integration-test-verify</id>
<phase>post-integration-test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
The tests cases status are up to Jenkins.