I was working on team internal KPI's lately and I thought that code coverage/number of unit tests will be a great measure.
In our CI build pipeline we're using NUnit to run test and JetBrains dotCover to generate coverage report - everything works great and we have that fancy "statistics" page in Team City control panel:
What I would like to do is to export this coverage data so I can generate HTML output for the current coverage status and keep it under control not necessarily in the Team City console?
Thanks in advance.
For any reports that teamcity shows, the convention is that it needs a zip file with an index.html page at the root. So for coverage report it would be coverage.zip and so on.
If you go the workout directory for your builds, you can find the artefacts that you want (coverage, test results etc) in the artefacts folder.
Related
In sonarQube, how to reference the unit test report residing in the server pertaining to Continuous Integration tool? Please note that the sonarQube and CI tools are in different servers. This is needed to show unit test coverage in sonarQube.
Usually, the build of a project does operations like:
Checkout source code
Build source code
Run unit test and generate coverage reports
Run a SonarQube Scanner to trigger the analysis
All this happens within the CI server. This means that you just need to configure your SonarQube analysis to point to the local coverage report files that have been generate just before.
For example, take a look at the following project:
Sample project that reads coverage reports
, and especially to its sonar-project.properties file, in which you will see that coverage report files are referenced using relative paths.
so I set up a none-XAML-build on TFS 2015 to build and test my Maven project. Which works as expected. Then I added a build step to consolidate test results via **/TEST-*.xml and publish them as a test run to tfs, which also works, but...
on the testrun summary page it shows the correct number of passed tests and it also attached the JUnit XML-files, though on the test results page it does not show anything :( Those .xml-files are JUnit <testsuite> results with multiple <testcase> entries and I expected for TFS to display the result for each individual test including it's duration, but it does only show a blank page.
Anybody has the same problem or even a solution (except publishing by tfs api :p) Cheers!
There are no additional settings to show the test result info. It will automatically show the every detail information about each test.
Since the test run summary show the right info. The test must run successfully. Please check your build log whether there are some valuable info. And also double check your build template configuration. Here is a blog telling you how to use vnext build with Marven and run the tests for your reference:
Building Java code on Linux using VSO Build.vNext
We use the build in coverage application in TeamCity 6 (about to upgrade to 7.1)
If we wish to see the code coverage (or other metrics) of a particular build it is fine as we can navigate to that build, but it would be great if we could pluck out a few interesting metrics from all/some of the current projects/build configurations and display them all together.
For convenience I would expect the new display to be accessible from within TeamCity itself, however if there are solutions that require a separate solution we could look at them.
If you want to compare a set of common metrics (e.g. code coverage) across different projects and over time then SonarQube is probably what you want.
You can integrate it with TeamCity by adding a sonar-project.properties file to each project and calling sonar-runner from a command line build step.
I have been tasked with introducing automated unit test coverage reporting to our CI build environment (we use Hudson and MSBuild). I have used dotCover to produce the coverage data across several projects within a build job, merge that data, produce a single HTML report and integrated that with the Hudson dashboard for that job using the HTML Publisher Hudson add-in.
So far so good. But, what we ultimately want is a graphical display of unit test coverage metrics across the entire system code base (half a dozen build jobs, circa 50 projects) up on one of our team displays; a highly visible 'at a glance' test coverage status. I can build this myself with an app that will merge the dotCover coverage reports for all builds, report to xml and build my own UI around that, but is there a product out there that does this already? We are not married to dotCover, that can change, and it's doesn't have to be free or open source either.
We are using NCover to get the test coverage and a bunch of other code metrics(symbol coverage, branch coverage and cyclomatic complexity).
It is pretty good for highlighting the coverage, you can browse through the source code as part of the reports and see which lines are covered and which are not.
You can also exclude test categories or specific namespaces from your reports.
As far as I know it integrates with Hudson.
OpenCover and PartCover are both open source code coverage tools that can be integrated into build automation systems.
With both you can use ReportGenerator to display results; though PartCover does come with it's own viewer I prefer to use the ReportGenerator one myself as the HTML can be integrated quite nicely into a build status report.
You may also build your own reports using XSLT or such like as both tools have an XML output.
Our C# Test Coverage Tool tool can combine test coverage vectors for separate C# artifacts into a single overall view, out of the box.
In fact, our family of test coverage tools will can combine results from multiple different languages (e.g., C#, VB.net, C++, Java, PHP, COBOL, ...) in the same way.
The tool(s) provide both visual view of the covered code, and reports of the coverage data including in an XML form.
I like the treemap functionality from SonarQube a lot, but this only works for one 'project' in SonarQube, nevertheless, a great tool to look into.
Google for SonarQube and treemap for examples.
TFS2010 database is growing fast (50GB per month) because of our TFS builds.
We noticed that tbl_attachmentcontent table contains some binary (dll) files and some files related to instrumentation and TQAgents... All content seems to be unit test related.
I tried to find something how to stop growing of TFS DB because of unit tests.
I found this suggestion to disable publishing of unit test results to the TFS.
How can I disable publishing of binary files to the TFS DB?
Is it possible to disable publishing of specific content?
What will be missing in the TFS when I disable publising unit tets results to the TFS?
At about 400+ unit tests are running every build including code coverage results and huge build log (500+ C# compiled projects with a lot of dependencies).
I'd like to have build retention policy set to "Kepp all", because of build statistics.
I did some testing and here are the results:
1) You can disable unit test publishing. Your unit tests results file (trx) is not uploaded to the TFS DB. This is not problem for me, because I'm using Trx2Html tool to convert results to the HTML format. If test failed, converted HTML formatted report is sent by E-Mail to notify developer team.
2) You can enable compression on the dbo.tbl_AttachmentContent table. In our case compressed content is 5% from original.
3) Use this tool to clean up dbo.tbl_AttachmentContent table
All I know that test result is not a binary file, but it's XML file with (.trx) extension, for your question about what happen if you stop publish test result, the build server just attache the test result with the build so you can view the result by click on the test result on the build result, so you just download the file and start viewing the result by the visual studio, otherwise you will not be able to see it.
This is what I think.
Update
You can see this link:
http://blogs.msdn.com/b/anutthara/archive/2011/10/30/gsjgd.aspx