I have a built a Mule API with Munits. I would like to build and deploy the application using jenkins pipeline. However, before build & deploy, I would like to execute maven clean test and check if code coverage is less than some x%. if yes, then I would like to fail the build/deployment to Mule cloud hub.
Is there a Jenkins plugin or any option to fail the jenkins pipeline when the code coverage is less than pre-defined threshold (let's say 80%)?
Thanks,
Bala
Looks like the topic is covered in MUnit documentation for coverage in Maven:
<plugin>
<groupId>com.mulesoft.munit.tools</groupId>
<artifactId>munit-maven-plugin</artifactId>
<version>${munit.version}</version>
<executions>
<execution>
<id>test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
<goal>coverage-report</goal>
</goals>
</execution>
</executions>
<configuration>
<coverage>
<runCoverage>true</runCoverage>
<failBuild>false</failBuild>
<requiredApplicationCoverage>75</requiredApplicationCoverage>
<requiredResourceCoverage>50</requiredResourceCoverage>
<requiredFlowCoverage>50</requiredFlowCoverage>
<formats>
<format>console</format>
<format>html</format>
</formats>
</coverage>
</configuration>
</plugin>
If you set the appropriate threshold (for example requiredApplicationCoverage) to the desired value it will fail the Maven test phase if coverage is not high enough. Since Jenkins just executes Maven it will fail too.
I am having issues with sonar picking up the jacoco analysis report. Jenkins however is able to pick up the report and display the results.
My project is a maven build, built by Jenkins. The jacoco report is generated by maven (configured in the pom). Sonar is executed by using the Jenkins plugin.
This is what I see on SonarQube:
This is the report i can see of the project in jenkins.
The maven plugin config:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.6.4.201312101107</version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>default-report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>default-check</id>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
Jenkins Sonar Plugin config
You were missing a few important sonar properties, Here is a sample from one of my builds:
sonar.jdbc.dialect=mssql
sonar.projectKey=projectname
sonar.projectName=Project Name
sonar.projectVersion=1.0
sonar.sources=src
sonar.language=java
sonar.binaries=build/classes
sonar.tests=junit
sonar.dynamicAnalysis=reuseReports
sonar.junit.reportsPath=build/test-reports
sonar.java.coveragePlugin=jacoco
sonar.jacoco.reportPath=build/test-reports/jacoco.exec
The error in Jenkins console output can be pretty useful for getting code coverage to work.
Project coverage is set to 0% since there is no directories with classes. Indicates that you have not set the Sonar.Binaries property correctly
No information about coverage per test Indicates you have not set the Sonar.Tests property properly
Coverage information was not collected. Perhaps you forget to include debug information into compiled classes? Indicates that the sonar.binaries property was set correctly, but those files were not compiled in debug mode, and they need to be
Based on https://github.com/SonarSource/sonar-examples/blob/master/projects/tycho/pom.xml, the following POM works for me:
<properties>
<sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.0.201403182114</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
<configuration>
<destFile>${sonar.jacoco.reportPath}</destFile>
</configuration>
</plugin>
</plugins>
</build>
Setting the destination file to the report path ensures that Sonar reads exactly the file JaCoCo generates.
The report path should be outside the projects' directories to take cross-project coverage into account (e.g. in case of Tycho where the convention is to have separate projects for tests).
The reuseReports setting prevents the deletion of the JaCoCo report file before it is read! (Since 4.3, this is the default and is deprecated.)
Then I just run
mvn clean install
mvn sonar:sonar
I had the similar issue, 0.0% coverage & no unit tests count on Sonar dashboard with SonarQube 6.7.2:
Maven : 3.5.2,
Java : 1.8,
Jacoco : Worked with 7.0/7.9/8.0,
OS : Windows
After a lot of struggle finding for correct solution on maven multi-module project,not like single module project here we need to say to pick jacoco reports from individual modules & merge to one report,So resolved issue with this configuration as my parent pom looks like:
<properties>
<!--Sonar -->
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.language>java</sonar.language>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
<plugin>
<groupId>org.sonarsource.scanner.maven</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>3.4.0.905</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.9</version>
<configuration>
<destFile>${sonar.jacoco.reportPath}</destFile>
<append>true</append>
</configuration>
<executions>
<execution>
<id>agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
I've tried few other options like jacoco-aggregate & even creating a sub-module by including that in parent pom but nothing really worked & this is simple. I see in logs <sonar.jacoco.reportPath> is deprecated,but still works as is and seems like auto replaced on execution or can be manually updated to <sonar.jacoco.reportPaths> or latest. Once after doing setup in cmd start with mvn clean install then mvn org.jacoco:jacoco-maven-plugin:prepare-agent install (Check on project's target folder whether jacoco.exec is created) & then do mvn sonar:sonar , this is what I've tried please let me know if some other best possible solution available.Hope this helps!! If not please post your question..
Jenkins does not show coverage results as it is a problem of version compatibilities between jenkins jacoco plugin and maven jacoco plugin.
On my side I have fixed it by using a more recent version of maven jacoco plugin
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.9</version>
</plugin>
<plugins>
<pluginManagement>
<build>
I was facing the same problem and the challenge in my case was to configure Jacoco correctly and to configure the right parameters for Sonar. I will briefly explain, how I finally got SonarQube to display the test results and test coverage correctly.
In your project you need the Jacoco plugin in your pom or parent pom (you already got this). Moreover, you need the maven-surefire-plugin, which is used to display test results. All test reports are automatically generated when you run the maven build. The tricky part is to find the right parameters for Sonar. Not all parameters seem to work with regular expressions and you have to use a comma separated list for those (documentation is not really good in my opinion). Here is the list of parameters I have used (I used them from Bamboo, you might omit the "-D" if you use a sonar.properties file):
-Dsonar.branch.target=master (in newer version of SQ I had to remove this, so that master branch is analyzed correctly; I used auto branch checkbox in bamboo instead)
-Dsonar.working.directory=./target/sonar
-Dsonar.java.binaries=**/target/classes
-Dsonar.sources=./service-a/src,./service-b/src,./service-c/src,[..]
-Dsonar.exclusions=**/data/dto/**
-Dsonar.tests=.
-Dsonar.test.inclusions=**/*Test.java [-> all your tests have to end with "Test"]
-Dsonar.junit.reportPaths=./service-a/target/surefire-reports,./service-b/target/surefire-reports,
./service-c/target/surefire-reports,[..]
-Dsonar.jacoco.reportPaths=./service-a/target/jacoco.exec,./service-b/target/jacoco.exec,
./service-c/target/jacoco.exec,[..]
-Dsonar.projectVersion=${bamboo.buildNumber}
-Dsonar.coverage.exclusions=**/src/test/**,**/common/**
-Dsonar.cpd.exclusions=**/*Dto.java,**/*Entity.java,**/common/**
If you are using Lombok in your project, than you also need a lombok.config file to get the correct code coverage. The lombok.config file is located in the root directory of your project with the following content:
config.stopBubbling = true
lombok.addLombokGeneratedAnnotation = true
Include the sunfire and jacoco plugins in the pom.xml and
Run the maven command as given below.
mvn jacoco:prepare-agent jacoco:report sonar:sonar
<properties>
<surefire.version>2.17</surefire.version>
<jacoco.version>0.7.2.201409121644</jacoco.version>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.version}</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals><goal>prepare-agent</goal></goals>
</execution>
<execution>
<id>default-report</id>
<phase>prepare-package</phase>
<goals><goal>report</goal></goals>
</execution>
</executions>
</plugin>
</plugins>
The presence of argLine configurations in either of surefire and jacoco plugins stops the jacoco report generation. The argLine should be defined in properties
<properties>
<argLine>your jvm options here</argLine>
</properties>
For me percentage coverage was not appearing in the sonarqube dashboard. I have added below property in our pom.xml to work.
<sonar.binaries>${project.basedir}/../target/classes</sonar.binaries>
Make sure that Sonarqube can find your test coverage file.
As somebody already mentioned the Sonarqube output should really be helpful here.
e.g.
WARN: No coverage information will be saved because LCOV file cannot be found.
My code contains tests and those tests run fine upon mvn clean install
I have included a cobertura prlugin in order to produce the reports as
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.5.1</version>
<configuration>
<formats>
<format>xml</format>
<format>html</format>
</formats>
<check/>
</configuration>
<executions>
<execution>
<phase>clean</phase>
<goals>
<goal>cobertura</goal>
</goals>
</execution>
</executions>
</plugin>
At the end, HTML is produced in the target as expected. index.html shows all classes and packages but ... no coverage information (0% coverage for all)
If i export coverage via IntelliJ it shows coverage appropriately. Is my setup missing something obvious?
Please help me understand.
It looks like the problem is that you're trying to execute cobertura in the clean phase of the project.
According to the documentation, Cobertura uses its own lifecycle: http://mojo.codehaus.org/cobertura-maven-plugin/cobertura-mojo.html
Please remove the <phase>clean</phase> from your execution configuration and try again.
Further references here:
How do I configure when cobertura tests run in maven-cobertura-plugin?
http://mojo.codehaus.org/cobertura-maven-plugin/usage.html
I need some help in setting up code quality plugin for maven project.
I have a multi module project. While I have configured pmd, checkstyle, findbugs and cobertura in my build process, and I can generate xml reports for each plugin, I am facing some challenges configuring the sonar plugin in my project.
I am not sure how to approach this problem:
Should I reuse the reports generated by these plugins while executing sonar? if so what should my sonar plugin configuration be?
If I run sonar with embedded pmd, checkstyle, findbugs and cobertura plugins, how do I configure them to run only for specific packages or make findbugs analyze on com.mycompany.- structure.
Lastly I cannot get coverage report in sonar, either running cobertura external to sonar or within sonar.
I have my pom below for review. Any help will be tremendously appreciated.
This is in the plugins section of the build section in my root pom:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.5.1</version>
<configuration>
<instrumentation>
<includes>
<include>com/mycompany/**/*.class</include>
</includes>
</instrumentation>
<formats>
<format>xml</format>
</formats>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>2.7.1</version>
<configuration>
<sourceEncoding>utf-8</sourceEncoding>
<minimumTokens>100</minimumTokens>
<targetJdk>1.6</targetJdk>
<includes>
<include>com/mycompany/**/*.java</include>
</includes>
<excludeRoots>
<excludeRoot>target/generated-sources/*</excludeRoot>
</excludeRoots>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>2.5.2</version>
<configuration>
<onlyAnalyze>com.mycompany.-</onlyAnalyze>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<version>2.9.1</version>
<configuration>
<includes>com/mycompany/**/*.java</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>2.0</version>
</plugin>
1) should I reuse the reports generated by these plugins while
executing sonar? if so what should my sonar plugin configuration be?
Sonar does not provide a mechansim to reuse reports generated by these plugins. You could configure the rules through the quality profiles.
2) if I run sonar with embedded pmd, checkstyle, findbugs and cobertura plugins, how do I configure them to run only for specific packages or make findbugs analyze on "com.mycompany.-" structure.
Sonar web UI allows you to specify the exclusion for your findbugs filters. Likewise for cobertura. Not sure about pmd, checkstyle.
3) lastly i cannot get coverage report in sonar, either running cobertura external to sonar or within sonar.
It could be possible due to jacoco being the default code coverage engine. You could run mvn clean verify/install sonar:sonar as instructed and see if it works.
We try to configure maven for run with sonar. At the moment sonar and maven in our project runs very well but now we try to exclude some packages of code coverage analysis.
It's possible to skip this packages only for code coverage but no for analysis of code with checkstyle?
We try this in our pom.xml:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>sonar-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.5.1</version>
<configuration>
<instrumentation>
<excludes>
<exclude>com/gribas/web/**/*.class</exclude>
<exclude>com/gribas/web/**/*.java</exclude>
</excludes>
</instrumentation>
</configuration>
</plugin>
But doesn't run.
Anyone have an idea for do this?
Thanks!!
If you used JaCoco as coverage engine in Sonar, then you'd be able to achieve what you want.
To use JaCoCo, set "sonar.core.codeCoveragePlugin=jacoco".
And then, you can set exclusions with "sonar.jacoco.excludes=...".