IT code coverage with sonar - maven

I have the following task at hand:
-- find IT code coverage for a project
Given situation:
-- IT code resides in a repository separate to the actual production code
-- Production code that the tests were created for reside in more than one git repository.
-- all of the above uses maven and are written in Java.
I have tried following different tutorial and blogs but couldnt find a simpler answer.
Can anyone either point me towards the right resource or give me hints for a kick start?

I will try to answer. I will post example with UT (IT is the same thing just not at the same place in the maven livecycle build, and instead of the surefire plugin its the failsafe plugin)
Lets say we use JaCoCo for the code coverage agent.
In my Parent pom, in the profile section (it is a multi module project)
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.15</version>
<configuration>
<argLine>${surefireArgLine}</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.4.201502262128</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
Now when we build our maven project, we add the profile to inject the JaCoCo Agent
clean install -Psonar-coverage
Then we may tell sonar to analyse our project and to use the JaCoCo report with the following command
mvn org.codehaus.mojo:sonar-maven-plugin:2.4:sonar -Dsonar.dynamicAnalysis=reuseReports -Dsonar.java.coveragePlugin=jacoco -Dsonar.forceAnalysis=true -Dsonar.jacoco.reportMissing.force.zero=true -Dsonar.binaries=target/classes -Dsonar.junit.reportsPath=target/surefire-reports -Dsonar.jacoco.reportPath=target/coverage-reports/jacoco-ut.exec -Dsonar.jdbc.driver=com.mysql.jdbc.Driver -Dsonar.jdbc.url=jdbc:<YOUR SONAR INSTALLATION DB> -Dsonar.host.url=<YOUR SONAR INSTALLATION>

Related

Jacoco.exec not displaying full test coverage in SonarQube

We have a small multi-module Java Spring REST application that is building with the maven surefire plugin. Because the application is fairly small, we wrote a mix of unit tests and integration tests and run them all as unit tests, so we get a single set of jacoco.exec files.
When I pull up the test coverage in SonarQube, the coverage is only being calculated for the module in which the test was run. So the integration tests that we have in the REST controller layer are not generating any coverage in the business or data layers.
I have tried multiple configurations of jacoco and I'm tired of staring at it. Any suggestions would be helpful.
jacoco configuration in parent pom.xml:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.7.201606060606</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
surefire configuration in parent pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<systemPropertyVariables>
<org.jboss.logging.provider>slf4j</org.jboss.logging.provider>
</systemPropertyVariables>
</configuration>
</plugin>
the coverage is only being calculated for the module in which the test was run
This is how the import of JaCoCo report in SonarQube works.
If you want to overcome this, you may want to merge the reports before feeding it to SonarQube.

Maven Jacoco Configuration for multi-module projects [duplicate]

This question already has answers here:
How to configure multi-module Maven + Sonar + JaCoCo to give merged coverage report?
(13 answers)
Closed 6 years ago.
I was trying to generate code coverage reports using jacoco plugin in maven for a multi module project that I was working on.
I added the following in my parent pom.xml within the build tags.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8-SNAPSHOT</version>
<configuration>
<output>file</output>
<append>true</append>
</configuration>
<executions>
<execution>
<id>jacoco-initialize</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>jacoco-site</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${argLine}</argLine>
</configuration>
</plugin>
On running mvn verify, the respective jacoco reports were generated for each module at "project-root\module\target\site\jacoco\"
Is it possible to generate a consolidated jacoco report at the project-root containing the test coverage details of each module?
Please suggest the best possible way to merge the individual module reports.
Sure is!
Took me a while and a few sources to cook this pattern up, but has worked well.
For a multi-module Maven project:
ROOT
|--LIB-1
|--LIB-2
The LIB projects both have their own unit tests.
ROOT pom.xml
<!- properties-->
<jacoco.reportPath>${project.basedir}/../target/jacoco.exec</jacoco.reportPath>
<!-- build/plugins (not build/pluginManagement/plugins!) -->
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.6.201602180812</version>
<executions>
<execution>
<id>agent-for-ut</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<append>true</append>
<destFile>${jacoco.reportPath}</destFile>
</configuration>
</execution>
</executions>
</plugin>
LIB projects pom.xml will inherit the the JaCoCo plugins execution, so just need to wire up the argline in the Surefire plugin.
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${argLine}</argLine>
</configuration>
</plugin>
I have an extended answer for rolling up integration tests as well as unit tests for JaCoCo being reported via Sonar, you can see my detailed answer here.
In addition to the steps suggested in markdsievers detailed answer, I had to setup sonarqube-5.3 (supports jdk 7+) in localhost:9000
Setup SonarQube
And use mvn package to generate jacoco.exec files.
Then mvn sonar:sonar to generate report in sonar dashboard.

How to integrate cucumber-jvm with maven build

i'm trying to put some bdd into my eclipse plugin project, but can't figure out how to run my integration tests during maven build fase. To write my tests i'm using SWTBot framework.
I already did the feature generation fase, and setup my tests. How do i setup my pom to run my integration tests?
I use below configuration and run mvn clean verify. If you don't want to run tests in parallel, remove parallel, perCoreThreadCount and threadCountClasses tags.
Make sure to update the regular expression to match your test naming convention <include>**/Run*.java</include>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<id>acceptance-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<outputEncoding>UTF-8</outputEncoding>
<parallel>classes</parallel>
<perCoreThreadCount>true</perCoreThreadCount>
<threadCountClasses>10</threadCountClasses>
<argLine>-Xmx1024m</argLine>
<argLine>-XX:MaxPermSize=256m</argLine>
<includes>
<include>**/Run*.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>

maven-site-plugin not generating apidocs folder with javadocs

I inherited a project that is supposed to build javadoc files and place them in the site directory. This is not being done. I have looked at all the examples I can find and I can't figure out where the configuration is broken.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.0</version>
<executions>
<execution>
<phase>prepare-package</phase>
<goals>
<goal>site</goal>
</goals>
<configuration>
<reportPlugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.8</version>
</plugin>
</reportPlugins>
</configuration>
</execution>
</executions>
</plugin>
Any and all help is greatly appreciated.
You have bound maven site plugin's site goal to prepare-package phase. You have configured javadoc generation in this plugin configuration.
As such, if you run maven's default lifecycle goals like mvn package or mvn install you should get site report with javadocs.
If you ran mvn site, it would skip prepare-package phase to which your plugin configuration is bound and hence would not generate javadoc.

How do you get the soapUI maven plugin to fail safe?

AFAIK, the maven failsafe plugin fails safe because it has separate goals for running the tests and failing the build based on the tests. These are designed to be bound to the integration-test and verify goals respectively. This allows post-integration-test bound goals to run (shutting down the build) before the build fails.
My question is, how do I do this with the maven-soapui-plugin? I thought I could simply specify <testFailIgnore>true</testFailIgnore> in my soapui plugin config and then call the failsafe plugin verify goal, but that isn't working. I don't think I'm not sure if I'm getting a summary file out of the soapui plugin or not. I keep getting Expected root element 'failsafe-summary' but found 'testsuite' Here is a snippet of the POM:
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>4.0.0</version>
<configuration>
<junitReport>true</junitReport>
<exportAll>true</exportAll>
<outputFolder>${project.build.directory}/surefire-reports</outputFolder>
<testFailIgnore>true</testFailIgnore>
<printReport>true</printReport>
</configuration>
<executions>
<execution>
<id>FailingTest</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<projectFile>${basedir}/testData/soapui-integration-tests.xml</projectFile>
<host>localhost</host>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.9</version>
<executions>
<execution>
<phase>verify</phase>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
<configuration>
<summaryFiles>
<summaryFile>${project.build.directory}/surefire-reports/TEST-TestSuite_1.xml</summaryFile>
</summaryFiles>
</configuration>
</execution>
</executions>
</plugin>
Is there something wrong with my POM or is this a bad approach? Are there any better approaches?
There is an open source extension of the soapui plugin which has a separate test-verify goal for exactly this purpose.
https://github.com/redfish4ktc/maven-soapui-extension-plugin
The following example shows the required configuration:
https://github.com/redfish4ktc/maven-soapui-extension-plugin/blob/master/src/it/test-verify_goal/one_failing_project/pom.xml
AFAIK maven-failsafe-plugin can only verify success status of tests run by maven-failsafe-plugin and not by maven-soapui-plugin. It does that by reading test summary report file (failsafe-summary.xml) which has specific format.
maven-soapui-plugin could be adjusted to separate running tests from verifying test success status, to support running post-integration-test tasks (stop server, undeploy artifacts, etc.) before verification. Create a support ticket for this to soapUI guys.
Maybe even maven-failsafe-plugin, it's verify mojo, could be extended to allow specifying different test report format (JUnit style reports are supported by soapUI) or even an xpath expression which would be used by maven-failsafe-plugin to determine if there were failed tests or not. Create a support ticket for this on maven-failsafe-plugin issue tracker.
Until those extensions are supported, and you need to do tasks on post-integration-test phase you can use soapUI JUnit integration and have maven-failsafe-plugin run those soapUI JUnit tests.
I am trying this solution, and it doesn't work.
But I have found one.
To obtain de tests report of SOAPUI tests in Jenkins, I using the failsafe plugin with this configuration in the pom.xml of my Soap tests folder :
<build>
<plugins>
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>2.6.1</version>
<configuration>
<projectFile>${basedir}/soap_project_tests.xml</projectFile>
<outputFolder>${filePath.reports.soap}</outputFolder>
<testFailIgnore>true</testFailIgnore>
<junitReport>true</junitReport>
<exportwAll>true</exportwAll>
<printReport>true</printReport>
</configuration>
<executions>
<execution>
<id>run-soap-integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.11</version>
<configuration>
<reportsDirectory>${filePath.reports.soap}</reportsDirectory>
<printSummary>true</printSummary>
<argLine>-Xmx512m</argLine>
</configuration>
<executions>
<execution>
<id>soap-integration-test-verify</id>
<phase>post-integration-test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
The tests cases status are up to Jenkins.

Resources