I'm using:
Sonar version: 2.10
Emma version: 2.1.5320
Sonar Emma plugin version: 1.2
I'm able to generate an Emma report showing coverage of the tests themselves (ideally this would be 100% but in practice it's not always), but Sonar shows only the coverage of the src files. How do I get it to show the coverage of the test files, too?
Would switching to Cobertura help?
AFAICT from:
public final class NewCoverageFileAnalyzer {
public boolean shouldDecorate(Resource resource) {
return Scopes.isFile(resource) && !Qualifiers.UNIT_TEST_FILE.equals(resource.getQualifier());
}
}
it looks like coverage of test files can't be shown in Sonar without changing the Sonar code.
Comment:
Counting test classes as coverage might inflate coverage ratio.
Using more test files would allow coverage > 100 % (lines covered / lines of production code).
It might still be usefull to see, if there is some dead test code somewhere.
Related
I have a maven project and the below step is mentioned right after executing surefire tests(for JUnit) and failsafe (for Integration tests). However, I am not able to exclude the files from generated-sources folder. However, if I use a single exclusionPattern:'/tomcat/', it is excluding tomcat folder from the report
I have tried below option:
**```
post {
always {
junit allowEmptyResults: true, testResults: '**/target/failsafe-reports/*.xml'
step( [ $class: 'JacocoPublisher', exclusionPattern: '**/target/generated-sources/**,**/tomcat/**'] )
}
}
```**
but it is only excluding the tomcat folders and not the generated-sources. Still seeing files from this folder in coverage report.
As a workaround, I am targeting the java packages inside target directory to improve coverage. It's not a very efficient way to do it as I had to add entries for multiple packages inside exclusionPattern. But, it works well for my requirement.
I have a gradle project which applies the sonarqube gradle plugin, version 2.6.
I run it against my team's sonarqube server, version 6.4 (build 25310).
According to documentation, new versions of sonarqube accept the property sonar.junit.reportPaths instead of sonar.junit.reportsPath.
My build runs 2 test tasks: test and integrationTest. Each test task outputs its xml into a different directory: build/test-results/test and build/test-results/integrationTest respectively.
I configured the sonarqube plugin to pick up both these directories:
project.sonarqube {
properties {
property 'sonar.junit.reportsPath', ['build/test-results/test',
'build/test-results/integrationTest']
// configure additional integration test properties that seem to be required
Collection<File> integrationTestSourceDirs = project.sourceSets.integrationTest.allJava.srcDirs.findAll { File dir -> dir.exists() }
properties['sonar.tests'] += integrationTestSourceDirs
Collection<File> integrationTestsClasses = project.sourceSets.integrationTest.output.classesDirs.files.findAll { File file -> file.exists() }
properties['sonar.java.test.binaries'] += integrationTestsClasses
}
}
This does not work. In sonarqube UI I only see unit tests (from the test directory) and don't see any integration tests.
I made sure that my integrationTest directory contains valid test reports, and it does.
It seems like sonarqube still uses the old parameter sonar.junit.reportsPath (which by default is assigned by the gradle plugin with the value build/test-results/test). I can tell this because if I remove this parameter I don't see any unit tests at all in the UI. This is how I removed the old parameter:
project.sonarqube {
properties {
properties.remove("sonar.junit.reportsPath")
}
}
As a workaround, I configured my integrationTest task to put its output into the same directory as unit tests: build/test-results/test. After doing this, all tests, including integration tests are picked up by sonarqube, and I can see them all in the UI.
However, I would prefer to keep outputs of different test tasks in different directories.
Is the described behavior intentional, or is it a bug?
Your SonarJava plugin is too old. The new property is only available from 4.11 on. In 4.10 only the old property is evaluated, so the new one is ignored. The Gradle plugin just sets the properties. The evaluation happens in the code that is downloaded from the SonarQube server and thus ignored.
I'm not getting any results from opencover. My nunit tests all run and pass, just no coverage results. the problem seems to be opencover filters, but we aren't setting any. Any suggestions?
The CodeCoverage.xml file contains a group of lines like the following which indicate that the plugin is telling opencover to filter out the DLLs we are trying to measure.
<Module hash="A3-F0-3A-1A-FF-38-D7-EF-A2-55-C9-8B-84-37-CF-CF-00-80-70-23" skippedDueTo="Filter">
<FullName>C:\gitlab-runner\builds\83ebc972\0\active\scrpt\output\Scrpt.Core.dll</FullName>
<ModuleName>Scrpt.Core</ModuleName>
<Classes/></Module>
which has the correct path for the dll file, but I don't see why it is skipped due to filtering. The unit tests are contained in a dll called Scrpt.Test.dll and the rest of the code is in other DLLs all of which are being filtered out.
I'm using the following plugins
plugins {
id 'com.ullink.msbuild' version '2.15'
id 'com.ullink.nunit' version '1.8'
id 'com.ullink.opencover-nunit' version '1.6'
}
and the plugin definitions for nunit and opencover are:
nunit {
testAssemblies = [file('output/Scrpt.Tests.dll')]
shadowCopy = false
useX86 = true
ignoreFailures = false
}
opencover {
targetAssemblies = [file('output/Scrpt.dll'),file('output/Scrpt.Core.dll'),file('output/Scrpt.SourceCitations.dll'),file('output/ScrptUtilLib.dll')]
ignoreFailures = false
}
Thank you for your help,
-herb
You need to set an opencover filter. Open cover filter works off of inclusive and exclusive filters.
The first filter should always be something like :
+[*]*
meaning include every assembly and every class.
then add in your exclusive filters:
+[*]* -[AssemblyName]* -[*AnotherName]*
It's very simple. just add in the universal inclusive filter first, get your results, then start to incrementally exclude stuff in your filter one by one.
This was answered thru the help of Francois Valdy over at https://github.com/Ullink/gradle-opencover-plugin/issues/17
The problem turned out to be that the opencover plugin was not generating the coverage.xml results. There was an error message in the xml saying that the files I was interested in were skipped because of filters, but I couldn't find what filters were causing that.
I ended up replacing the opencover plugin with a task that does the same thing. Not sure why the plugin didn't work, but the replacement is essentially the same thing in that it calls nunit, creates the output coverage xml which sonarqube then uploads. Plus doing it this way gives you more control over where the files end up.
A bit awkward, but it work. The following replaced the opencover block in my original question.
task opencover(type:Exec) {
executable 'C:\\Program Files (x86)\\OpenCover\\OpenCover.Console.exe'
workingDir file('output')
args "-target:c:\\Program Files (x86)\\NUnit 2.6.4\\bin\\nunit-console-x86.exe", "-targetargs:c:\\gitlab-runner\\builds\\83ebc972\\0\\active\\scrpt\\output\\scrpt.tests.dll /noshadow", '-output:coverage.xml', '-filter:+[*]*'
}
btw, I did try replacing the file() uses in the targetAssemblies for the opencover gradle plugin, but that had no effect. From what I've read, the following should've worked.
opencover {
targetAssemblies = ['output/Scrpt.dll','output/Scrpt.Core.dll','output/Scrpt.SourceCitations.dll','output/ScrptUtilLib.dll']
ignoreFailures = false
}
I have several tests generating coverage reports with istanbul. One of them is generated by karma-coverage plugin. I am merging these reports with istanbul report but files from karma-coverage report are not included in the merged report.
There was an issue with file paths that had the same symptoms but it seems to have been fixed: https://github.com/karma-runner/karma-coverage/pull/163
So it is probably something else.
I have tried using grunt-istanbul that instruments source files separately and then I browserify them in the bundle. I also tried using preprocessor from karma-coverage plugin to instrument the bundle. In both cases karma-coverage generates reports that look ok, but in both cases these reports are not included in the merged report.
What am I doing wrongly here? Is there some workaround maybe?
Package versions:
"karma": "^0.13.10",
"karma-coverage": "^0.5.2",
"grunt-istanbul": "^0.6.1"
karma.conf.js
reporters: ['coverage', 'spec'],
coverageReporter: {
type: 'lcov',
dir: 'coverage'
}
Coverage reporter should have type: 'lcov' - then you can merge reports
If you are able to generate coverage separately then you can merge them as specified here:
link
I have a maven java project in some folder (it has some unit tests), and tests for the same code in another different project (different directory). Both source code and test share the same parent pom. Now I want to generate the code coverage report using JaCoCo.
How to instrument sources? How to run tests on instrumented code? And how to integrate and get the result report?
Say Project ABC contains the code and project XYZ contains the test cases.
Note:
Project ABC and Project XYZ are independent projects
Both ABC and XYZ contains multiple sub projects(Need to integrate everything).
I had a similar problem. I found a solution by changing the path of the jacoco report path:
<sonar.surefire.reportsPath>${project.basedir}/../target/surefire-reports</sonar.surefire.reportsPath>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
I added these properties and configured the jacoco plugin to append reports and not overwrite them by:
<configuration>
<append>true</append>
</configuration>
This way jacoco write the reports to the base directory of the multi module project. The sonar plugin finds these and analyses them.
I had a problem when building a "reference application" with multiple sub-modules and trying to generate test coverage for the sub-modules and have them push to sonarqube. The issue I was having was that since the sub-modules referenced each other, the resulting jacoco reports were getting overwritten and I'd end up with results for a single module. This may not be the same issue as posted above, but I did solve it by using "append" in gradle, so just want to show how to do that.
In the build.gradle file for each module, I have:
testOptions {
unitTests.returnDefaultValues = true
unitTests.includeAndroidResources = true
unitTests.all {
jacoco {
append = true
includeNoLocationClasses = true
}
systemProperty 'robolectric.enabledSdks', '28'
}
}
s