Can't get Gradle OpenCover plugin to produce coverage results - sonarqube

I'm not getting any results from opencover. My nunit tests all run and pass, just no coverage results. the problem seems to be opencover filters, but we aren't setting any. Any suggestions?
The CodeCoverage.xml file contains a group of lines like the following which indicate that the plugin is telling opencover to filter out the DLLs we are trying to measure.
<Module hash="A3-F0-3A-1A-FF-38-D7-EF-A2-55-C9-8B-84-37-CF-CF-00-80-70-23" skippedDueTo="Filter">
<FullName>C:\gitlab-runner\builds\83ebc972\0\active\scrpt\output\Scrpt.Core.dll</FullName>
<ModuleName>Scrpt.Core</ModuleName>
<Classes/></Module>
which has the correct path for the dll file, but I don't see why it is skipped due to filtering. The unit tests are contained in a dll called Scrpt.Test.dll and the rest of the code is in other DLLs all of which are being filtered out.
I'm using the following plugins
plugins {
id 'com.ullink.msbuild' version '2.15'
id 'com.ullink.nunit' version '1.8'
id 'com.ullink.opencover-nunit' version '1.6'
}
and the plugin definitions for nunit and opencover are:
nunit {
testAssemblies = [file('output/Scrpt.Tests.dll')]
shadowCopy = false
useX86 = true
ignoreFailures = false
}
opencover {
targetAssemblies = [file('output/Scrpt.dll'),file('output/Scrpt.Core.dll'),file('output/Scrpt.SourceCitations.dll'),file('output/ScrptUtilLib.dll')]
ignoreFailures = false
}
Thank you for your help,
-herb

You need to set an opencover filter. Open cover filter works off of inclusive and exclusive filters.
The first filter should always be something like :
+[*]*
meaning include every assembly and every class.
then add in your exclusive filters:
+[*]* -[AssemblyName]* -[*AnotherName]*
It's very simple. just add in the universal inclusive filter first, get your results, then start to incrementally exclude stuff in your filter one by one.

This was answered thru the help of Francois Valdy over at https://github.com/Ullink/gradle-opencover-plugin/issues/17
The problem turned out to be that the opencover plugin was not generating the coverage.xml results. There was an error message in the xml saying that the files I was interested in were skipped because of filters, but I couldn't find what filters were causing that.
I ended up replacing the opencover plugin with a task that does the same thing. Not sure why the plugin didn't work, but the replacement is essentially the same thing in that it calls nunit, creates the output coverage xml which sonarqube then uploads. Plus doing it this way gives you more control over where the files end up.
A bit awkward, but it work. The following replaced the opencover block in my original question.
task opencover(type:Exec) {
executable 'C:\\Program Files (x86)\\OpenCover\\OpenCover.Console.exe'
workingDir file('output')
args "-target:c:\\Program Files (x86)\\NUnit 2.6.4\\bin\\nunit-console-x86.exe", "-targetargs:c:\\gitlab-runner\\builds\\83ebc972\\0\\active\\scrpt\\output\\scrpt.tests.dll /noshadow", '-output:coverage.xml', '-filter:+[*]*'
}
btw, I did try replacing the file() uses in the targetAssemblies for the opencover gradle plugin, but that had no effect. From what I've read, the following should've worked.
opencover {
targetAssemblies = ['output/Scrpt.dll','output/Scrpt.Core.dll','output/Scrpt.SourceCitations.dll','output/ScrptUtilLib.dll']
ignoreFailures = false
}

Related

sonarqube ignores sonar.junit.reportPaths

I have a gradle project which applies the sonarqube gradle plugin, version 2.6.
I run it against my team's sonarqube server, version 6.4 (build 25310).
According to documentation, new versions of sonarqube accept the property sonar.junit.reportPaths instead of sonar.junit.reportsPath.
My build runs 2 test tasks: test and integrationTest. Each test task outputs its xml into a different directory: build/test-results/test and build/test-results/integrationTest respectively.
I configured the sonarqube plugin to pick up both these directories:
project.sonarqube {
properties {
property 'sonar.junit.reportsPath', ['build/test-results/test',
'build/test-results/integrationTest']
// configure additional integration test properties that seem to be required
Collection<File> integrationTestSourceDirs = project.sourceSets.integrationTest.allJava.srcDirs.findAll { File dir -> dir.exists() }
properties['sonar.tests'] += integrationTestSourceDirs
Collection<File> integrationTestsClasses = project.sourceSets.integrationTest.output.classesDirs.files.findAll { File file -> file.exists() }
properties['sonar.java.test.binaries'] += integrationTestsClasses
}
}
This does not work. In sonarqube UI I only see unit tests (from the test directory) and don't see any integration tests.
I made sure that my integrationTest directory contains valid test reports, and it does.
It seems like sonarqube still uses the old parameter sonar.junit.reportsPath (which by default is assigned by the gradle plugin with the value build/test-results/test). I can tell this because if I remove this parameter I don't see any unit tests at all in the UI. This is how I removed the old parameter:
project.sonarqube {
properties {
properties.remove("sonar.junit.reportsPath")
}
}
As a workaround, I configured my integrationTest task to put its output into the same directory as unit tests: build/test-results/test. After doing this, all tests, including integration tests are picked up by sonarqube, and I can see them all in the UI.
However, I would prefer to keep outputs of different test tasks in different directories.
Is the described behavior intentional, or is it a bug?
Your SonarJava plugin is too old. The new property is only available from 4.11 on. In 4.10 only the old property is evaluated, so the new one is ignored. The Gradle plugin just sets the properties. The evaluation happens in the code that is downloaded from the SonarQube server and thus ignored.

how to let jacoco know sources/classes are outside the module

Module A is a separate module having all the integration tests. Maven-Jacoco-Sonar works fine, if sources and tests are located at one module.
Since module A is a different module altogether, we couldn't be able to generate the jacoco reports easily. Because, jacoco is unable to find the sources within module A to measure the coverage for report. Source files are all residing at different modules. jacoco needs to have the class files and source files available in order to generate a report. We need to link up them to module A accordingly while generating the jacoco report so that, jacoco-it.exec can be easily pulled to Sonar to display the coverage analysis.
Currently in this case,
Classes triggered according to the jacoco agent but no source/class files available
Please help me if anyone has as solution how to achieve the above functionality. It would be so helpful to me.
try adding the following as your main src
def debugTree = [fileTree(dir: "${buildDir}/intermediates/classes/debug", excludes: fileFilter),
fileTree(dir: "${project(":your_module_name").projectDir}/build/intermediates/classes/debug", excludes: fileFilter)]
def mainsrc = [fileTree(dir:"${project(":your_module_name").projectDir}/src/main/java"),
fileTree(dir:"${project.projectDir}/src/main/java")]
sourceDirectories = files(mainsrc)
classDirectories = files(debugTree)
This will include your external sub module files to be used while creating the jacoco report.
Try setting sourcesets variable.
jacocoTestReport {
sourceSets project(':module_name').sourceSets.main
reports {
xml.enabled true
// enable html report for debugging in local env
// html.enabled true
// html.destination = file("${buildDir}/jacocoHtml")
}
}

Exclude packages from Jacoco report using Sonarrunner and Gradle

Is there a way to exclude packages from SonarQube(instrumented by gradle + sonar-runner) coverage reports(generated by jacoco) without excluding them completely from the project ?
Below is what i tried so far:
Version information
SonarQube 4.5.1
Gradle 2.2.
Jacoco configuration
// JaCoCo test coverage configuration
tasks.withType(Test) { task ->
jacoco {
append = false
// excluded classes from coverage defined in above configuration
excludes = excludedClasses()
}
jacocoTestReport {
doFirst {
classDirectories = fileTree(dir: "${buildDir}/classes/main/").exclude(excludedClasses())
}
}
}
Sonarrunner configuration
Property setting to exclude package from Sonar analysis. Adding this to my configuration lead to the situation that the packages do not show-up at all in Sonar.
property 'sonar.exclusions', excludedClasses().join(',')
Property setting to exclude packages from jacoco. Setting this leads to the situation that packages are excluded from coverage analysis however show up having 0% which accumulates to bad total scores.
property 'sonar.jacoco.exclusions', excludedClasses().join(',')
I have managed to exclude particular packages from coverage reports by using sonar.coverage.exclusions property in sonar-project.properties. Property is described in official documentation
To combine #Mikalai's answer and #pavel's comment into something that's a bit easier to copy and paste:
To exclude a package or class from all Sonar checks (coverage, code smells, bugs etc), add the following to build.gradle:
sonarqube {
properties {
property 'sonar.exclusions', "**/com/some/package/**"
}
}
To exclude a package or class from only Sonar code coverage checks, add the following to build.gradle:
sonarqube {
properties {
property 'sonar.coverage.exclusions', "**/com/some/package/**"
}
}
To exclude res, assets, custom packages and auto generated classes from sonar code coverage for android project.
Create Exclusion list like below
exclusionList = [
//Res and Assets
"src/main/res/**/*.*",
"src/main/assets/**/*.*",
//Auto-Generated
'**/R.class',
'**/R$*.class',
'**/BuildConfig.*',
'**/*Manifest.*',
'android/**/*.*',
'androidx/**/*.*',
// excluded packages
**/com/<your-package-path>/**/*]
Provide the exclusion list to sonar coverage property
property 'sonar.coverage.exclusions', exclusionList
NOTE:
Keep the full path to exclude the full directory
**/* includes all the files and sub directories under the parent directory Refer Here
*Bean includes all the class names contains "Bean".
**/ starting with it covers the parent directories for the particular package.
Run Jacoco command and check sonar portal's coverage section after doing above changes.
If you are using Gradle 5+ with Kotlin DSL, you can exclude the files from coverage like this:
// configure the SonarQube plugin
sonarqube {
val exclusions = listOf(
"**/com/some/package/**",
"**/all/files/under/package/*",
"**/com/some/package/OneClass.kt"
)
// exclude the directories only from coverage (but not from other analysis)
// https://docs.sonarqube.org/latest/analysis/scan/sonarscanner-for-gradle/
properties {
property("sonar.coverage.exclusions", exclusions)
}
}
exclude multiple module or class from Sonarqube. add below code to
build.gradle:
example :
package name= com.student.result.detail , com.customer.order ,com.student
sonarqube {
properties {
property 'sonar.exclusions', "**/com/student/result/details/**",
"**/com/customer/order/**",
"**/com/student/**";
}
}
This worked for me with gradle 6, sonarqube plugin 2.8
property "sonar.coverage.exclusions" , "**/com/some/package/entity/**/*," + "**/com/some/package/config/**/*,**/com/some/package/dto/**/*,**/com/some/package/dao/**/*"
NOTE: packages are comma separated but in a single string
Most of the answer I have seen here didn't work for me, resulting on the following error:
Execution failed for task ':sonarqube'.
> Could not find method property() for arguments [sonar.coverage.exclusions, **/package/path/Class1**, **/package/path/Class2**, **/package/path/Class3**, **/package/path/Class4**] on object of type org.sonarqube.gradle.SonarQubeProperties.
The reason is that sonar is asking for a string instead of several comma-separated strings as property value. So instead of:
sonarqube {
properties {
property 'sonar.exclusions', "**/com/student/result/details/**",
"**/com/customer/order/**",
"**/com/student/**";
}
}
The solution for me was:
sonarqube {
properties {
property 'sonar.exclusions', "**/com/student/result/details/**,"+
"**/com/customer/order/**,"+
"**/com/student/**";
}
}

How to generate the code coverage report using jacoco where source code and tests are in different projects?

I have a maven java project in some folder (it has some unit tests), and tests for the same code in another different project (different directory). Both source code and test share the same parent pom. Now I want to generate the code coverage report using JaCoCo.
How to instrument sources? How to run tests on instrumented code? And how to integrate and get the result report?
Say Project ABC contains the code and project XYZ contains the test cases.
Note:
Project ABC and Project XYZ are independent projects
Both ABC and XYZ contains multiple sub projects(Need to integrate everything).
I had a similar problem. I found a solution by changing the path of the jacoco report path:
<sonar.surefire.reportsPath>${project.basedir}/../target/surefire-reports</sonar.surefire.reportsPath>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
I added these properties and configured the jacoco plugin to append reports and not overwrite them by:
<configuration>
<append>true</append>
</configuration>
This way jacoco write the reports to the base directory of the multi module project. The sonar plugin finds these and analyses them.
I had a problem when building a "reference application" with multiple sub-modules and trying to generate test coverage for the sub-modules and have them push to sonarqube. The issue I was having was that since the sub-modules referenced each other, the resulting jacoco reports were getting overwritten and I'd end up with results for a single module. This may not be the same issue as posted above, but I did solve it by using "append" in gradle, so just want to show how to do that.
In the build.gradle file for each module, I have:
testOptions {
unitTests.returnDefaultValues = true
unitTests.includeAndroidResources = true
unitTests.all {
jacoco {
append = true
includeNoLocationClasses = true
}
systemProperty 'robolectric.enabledSdks', '28'
}
}
s

OSGi bundle build issue in Gradle

I have a simple use case of building an OSGi bundle using Gradle build tool. The build is successful if there are java files present in the build path, but it fails otherwise.
I am using 'osgi' plugin inside the gradle script and trying to build without any java files. The build always fails with following error:
Could not copy MANIFEST.MF to
I am sure there must be some way to do it in Gradle but not able to fine. Any idea what can be done to resolve this depending on your experience.
I ran into this today as well, and #Peter's fix didn't work for me (I hadn't applied the java plugin in the first place...). However, after hours of Googling I did find this thread, which helped me find the problem.
Basically, it seems that the error occurs (as Peter stated) when no class files are found in the jar - my guess is because the plugin then cannot scan the classes for package names on which to base all the Import and Export information.
My solution was to add the following to the manifest specification:
classesDir = theSourceSet.output.classesDir
classpath = theSourceSet.runtimeClasspath
In my actual build code, I loop over all source sets to create jar tasks for them, so then it looks like this:
sourceSets.each { ss ->
assemble.dependsOn task("jar${ss.name.capitalize()}", type: Jar, dependsOn: ss.getCompileTaskName('Java')) {
from ss.output
into 'classes'
manifest = osgiManifest {
classesDir = ss.output.classesDir
classpath = ss.runtimeClasspath
// Other properties, like name and symbolicName, also set based on
// the name of the source set
}
baseName = ss.name
}
}
Running with --stacktrace indicates that the osgi plugin doesn't deal correctly with the case where both the osgi and the java plugins are applied, but no Java code is present. Removing the java plugin should solve the problem.
I had the same issue also when java code was present.
Adding these two lines to the osgiManifest closure fixed the problem:
classesDir = sourceSets.main.output.classesDir
classpath = sourceSets.main.runtimeClasspath
-- erik

Resources