We are starting to implement code coverage in our CI process and my task is to examine NCover from this perspective.
Specifically, we have SonarQube and a CI build in Azure DevOps that runs the unit tests and reports the coverage to SonarQube as described in https://docs.sonarqube.org/latest/analysis/scan/sonarscanner-for-azure-devops/
Currently it works with the DotCover command line tool.
I would like to check NCover. While googling for it I came across this cheerful page - https://www.ncover.com/support/docs/extras/sonar-integration and sure enough I clicked the link to the respective Jira issue - https://jira.sonarsource.com/browse/SONARCS-653 Oops. Closed with Won't Fix.
According to https://docs.sonarqube.org/latest/analysis/coverage/ only VS Coverage, DotCover and OpenCover format are supported. So, if NCover is supported, it would be through the Generic Test Data format or if NCover knows to produce coverage results in one of the other 3 formats.
So far I do not see how NCover can play with SonarQube, but maybe I am missing something here.
Anyone?
Got an answer from an NCover representative. There is no integration between SonarQube and NCover. Full stop.
Related
I have several java projects with no unit test cases. When I try to analyse these projects using SonarQube I am unable to get unit test code coverage because it doesn't have any test cases. As per my research I saw that I can force it to show it as zero if there are no unit test cases. The official Jacoco Java documentation mentions sonar.jacoco.reportMissing.force.zero=true.
My SonarQube analysis will be done using maven builds and SonarQube 5.6.6 and sonar-java 4.11.0.10660.
How can I set this parameter in maven so that it can show as zero unit test cases in my coverage?
From version 6.2 SonarQube does this automatically where the behavior is supported by the analyzers. Before that version, availability of the feature is spotty and not entirely reliable. For best results you should upgrade SonarQube.
That said, you can set that property on the analysis command line like so:
mvn sonar:sonar -Dsonar.jacoco.reportMissing.force.zero=true
Or by setting a property in your pom, as shown in the docs
Update 16 Dec 2019
Most language analyzers now calculate Executable Lines, so files omitted from the coverage report are included in calculations anyway. But that only kicks in if you provide coverage reports. No coverage reports -> no coverage metrics at all.
Since we updated to SonarQube 6.2 it seems code coverage plugin got merged in the core.
It shows red flags everywhere and I can’t find how to turn it off, we do not use code coverage.
You don't specify what language(s) you're analyzing. I'll assume Java and/or JavaScript. Starting from 6.2, SonarQube supports "force coverage to 0", which marks as uncovered executable lines in files that don't show up in any coverage reports. (That's assuming the underlying code analyzers support the feature, and Java and JavaScript already do.) The purpose is to have a more accurate picture of what's missing when you actually are using unit tests. Without this feature, it's impossible to tell whether a file that's omitted from coverage reports is missing because it has no executable code or because there are no tests on it - even when there should be.
Since you're not using unit tests (really?) you can exclude all the source files in your project from coverage calculations via the UI: Administration > Analysis Scope > Coverage Exclusions. A pattern value of **/*.* ought to do it for you.
Code coverage feature is in SonarQube misleading. To turn this off:
Under Quality gates create new or copy existing profile.
There delete Coverage metric
SonarQube Version: 9.2
I am using SonarQube 4.3 version to show Unit tests and other code metrics analysis results. Now I am looking for incremental preview or mode for my analysis results as described here.
I am looking to see whether the last code checked-in has a 100% unit test coverage, no security violations and has other quality gates passed.
Currently sonar gives overall combined report for all the files within project. Can we have one for the changed files in the last build as well?
I am analysing a Java project that has been unit tested and Cobertura coverage.xml reported. I am using SonarQube 4.1.1 and latest Sonar Runner. I have successfully imported Cobertura coverage results to Sonar 3.2 and Ant analyzer, but with this new version I am running into problems. In the new Sonar analysis execution (through Jenkins) I see no reference in logs that it would have started any Cobertura engine or anything. My settings in Runner Jenkins project:
sonar.dynamicAnalysis=reuseReports
sonar.java.coveragePlugin=cobertura
sonar.cobertura.reportPath=[mypath]/coverage.xml
sonar.junit.reportsPath=[mypath]/
No mentioning of Cobertura in the analysis output (except my own property values) and SonarQube page shows "-" in coverage report. Unit test results are shown fine.
I have also added all source, bin, and test directories. Any ideas? Thanks.
Update
I wonder if the reason why Cobertura coverage is not reported on SonarQube page, is because in Jenkins my SonarQube project clones (Clone plugin) the workspace from a previous Project build? If the coverage.xml file contains static paths, then maybe it goes wrong somehow.
Ok the solution was quite obvious. SonarCube 4.1.1 does not come with Cobertura preinstalled, so I installed it and now it works :) Maybe it was preinstalled in 3.2 version, I can't remember.
For any "reuse reports" feature in SonarQube (may it be for code coverage, test execution or any other third-party tool that generates a intermediate report), the report must be generated based on the same source files (in terms of file system location) than the ones that will be used for the SonarQube analysis. Otherwise the SonarQube plugins won't be able to match paths from the reports with paths of the analyzed sources.
So you guessed it right: cloning the workspace and running the SonarQube analysis on this clone is the reason why the coverage can't be computed.
The root cause of this problem is that SonarQube does not support Cobertura format and you need OpenCover, or dotCover format. Using reportgenerator is one of the solutions but it's unnecessary complicated: you can make "dotnet test" command to return the report in Opencover format by passing special parameter and then pick up the Opencover report.
Use
dotnet test --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura,opencover
to get two reports: one in OpenCover format for SonarQube and default Cobertura for Azure DevOps.
Now you can specify the path to OpenCover report in SonarQubePrepare task.
There is also detailed article on this here
I would like TeamCity to run my mSpec tests and report on the code covered by the tests.
I would also like TeamCity to report that a build has failed if code coverage in certain namespaces doesn't meet a threshold (e.g. MyProduct.ImportantStuff must be 100%, but MyProduct.LegacyStuff must be [23% or whatever it currently is to ensure we don't add new stuff without covering tests].
I initially looked at dotCover as it's integrated into TeamCity. I have since been looking at OpenCover as I couldn't get TC to fail the build on low coverage.
I got OpenCover working but I would still like to know (as I'm sure a lot of people would) how to get TC to fail a build if code coverage is too low.
Are you using the latest TeamCity, ie version 7?
When setting up a build configuration you can specify this:
There are lots more options in the dropdown related to code coverage. You can also force your build to fail if you're using some other code coverage tool.
For example you can echo a line to the console that will then be picked up by teamcity :
##teamcity[buildStatus status='FAILURE' text='something failed']
see official docs on this here