I have been tasked with introducing automated unit test coverage reporting to our CI build environment (we use Hudson and MSBuild). I have used dotCover to produce the coverage data across several projects within a build job, merge that data, produce a single HTML report and integrated that with the Hudson dashboard for that job using the HTML Publisher Hudson add-in.
So far so good. But, what we ultimately want is a graphical display of unit test coverage metrics across the entire system code base (half a dozen build jobs, circa 50 projects) up on one of our team displays; a highly visible 'at a glance' test coverage status. I can build this myself with an app that will merge the dotCover coverage reports for all builds, report to xml and build my own UI around that, but is there a product out there that does this already? We are not married to dotCover, that can change, and it's doesn't have to be free or open source either.
We are using NCover to get the test coverage and a bunch of other code metrics(symbol coverage, branch coverage and cyclomatic complexity).
It is pretty good for highlighting the coverage, you can browse through the source code as part of the reports and see which lines are covered and which are not.
You can also exclude test categories or specific namespaces from your reports.
As far as I know it integrates with Hudson.
OpenCover and PartCover are both open source code coverage tools that can be integrated into build automation systems.
With both you can use ReportGenerator to display results; though PartCover does come with it's own viewer I prefer to use the ReportGenerator one myself as the HTML can be integrated quite nicely into a build status report.
You may also build your own reports using XSLT or such like as both tools have an XML output.
Our C# Test Coverage Tool tool can combine test coverage vectors for separate C# artifacts into a single overall view, out of the box.
In fact, our family of test coverage tools will can combine results from multiple different languages (e.g., C#, VB.net, C++, Java, PHP, COBOL, ...) in the same way.
The tool(s) provide both visual view of the covered code, and reports of the coverage data including in an XML form.
I like the treemap functionality from SonarQube a lot, but this only works for one 'project' in SonarQube, nevertheless, a great tool to look into.
Google for SonarQube and treemap for examples.
Related
I want to include code coverage in my Jenkins pipeline. I am building a COBOL code and want to know if there are any plugins which support code coverage for COBOL on Jenkins.
I don't use Compuware Tool set so I cannot use Compuware Xpeditor Code coverage plugin.
Please suggest.
Many big financial companies relies on SONAR to evaluate their COBOL source code
A pluggin exists for Jenkins. See here for the installation and setup
Our COBOL test coverage tool handles (IBM Enterprise, implied by your Xpeditor note) COBOL and produces test coverage results. You'd probabaly have to reformat those results to plug into Jenkins but that's probably pretty minor string hacking.
It has been cleared here that from Sonarqube version 6.2 that coverage reports are merged and there won't be separate unit and integration coverage report anymore.
We still interested to have these two coverage reports separately. So, We have three sonar projects: unit-tests, integration-tests, whole-project(which is responsible to create overall coverage report)
Problem: All source files are analysed in all three projects. Since the number of files are too many, it takes several minutes to perform the analysis.
Question: Is it possible to turn off sonar issue analyser somehow in a project? It is desired to report only test coverage in the the first two projects(unit-tests & integration-tests) without analysing all files, and then run the issue analyzer only on the last project(whole-project). It could help us to analyse all files once instead of three times.
Additional info: We use sonar gradle plugin version 2.6.2 and sonarqube version 7.4
SonarQube/SonarCloud main responsibility is informing users about issues. Displaying code coverage is just an additional feature. It means there is no flag/parameter which allows you to do it.
Luckily, there is a workaround. You can create empty quality profiles, and use them to scan those two projects (unit-tests & integration-tests). You will get 0 issues because there are zero rules enabled.
The following feature request should be interesting for you: Making test coverage measures mode useful. Feel free to vote on it.
I was working on team internal KPI's lately and I thought that code coverage/number of unit tests will be a great measure.
In our CI build pipeline we're using NUnit to run test and JetBrains dotCover to generate coverage report - everything works great and we have that fancy "statistics" page in Team City control panel:
What I would like to do is to export this coverage data so I can generate HTML output for the current coverage status and keep it under control not necessarily in the Team City console?
Thanks in advance.
For any reports that teamcity shows, the convention is that it needs a zip file with an index.html page at the root. So for coverage report it would be coverage.zip and so on.
If you go the workout directory for your builds, you can find the artefacts that you want (coverage, test results etc) in the artefacts folder.
We use the build in coverage application in TeamCity 6 (about to upgrade to 7.1)
If we wish to see the code coverage (or other metrics) of a particular build it is fine as we can navigate to that build, but it would be great if we could pluck out a few interesting metrics from all/some of the current projects/build configurations and display them all together.
For convenience I would expect the new display to be accessible from within TeamCity itself, however if there are solutions that require a separate solution we could look at them.
If you want to compare a set of common metrics (e.g. code coverage) across different projects and over time then SonarQube is probably what you want.
You can integrate it with TeamCity by adding a sonar-project.properties file to each project and calling sonar-runner from a command line build step.
I am writing some unit tests for one of my DLL libraries.
The 'Code Coverage Results' pane shows a breakdown of the assemblies covered and tested.
For some strange reason - my test project itself appears in the coverage results ! (at approx. 90% covered).
This seems stupid... what's the deal with this ?
The reason the percentage is so high is that projects for code coverage are instrumented to keep track of which lines are hit by a test run, since you are running the tests from this project, almost all lines of code in the project will be run.
You can choose which projects/DLLs to collect Coverage statistics on in the Test Settings.
So if you don't need to capture stats on the test project (which you shouldn't really), you can simply remove this project from the settings you're using for coverage.
See http://msdn.microsoft.com/en-us/library/ms182534.aspx (steps 5 - 7 in particular) for more details.