SonarQube Code Coverage from VSTS - sonarqube

I'm using VSTS hosted agent VS2017 to run the build, tests, and SonarQube analysis. I'm using the SonarQube extension from the marketplace (https://marketplace.visualstudio.com/items?itemName=SonarSource.sonarqube). Task version in use is 4 which uses SonarQube Scanner for MSBuild 4.0.2.
The analysis works well, but it doesn't translate the code coverage results to the SonarQube. I can see the Code Coverage analysis in VSTS (build details), but not in SonarQube.
In my understanding, the problem is that VSTS generates a binary file with an extension .codecoverage. SonarQube doesn't know how to handle this file. I'm having trouble converting this to XML format as it seems I have no means to do it.
Has someone used VSTS successfully with SonarQube and got the Code Coverage results to SonarQube as well? How have you set it up?
EDIT 1
SonarQube version that I'm using is: Version 6.7 (build 33306), Community Edition
EDIT 2
The end of analysis actually generates the xml-file, like was stated in the comments below. This is from the logs:
2018-03-06T11:14:56.4189055Z 11:14:56.417 Attempting to locate the
CodeCoverage.exe tool...
2018-03-06T11:14:56.4210147Z 11:14:56.42 Attempting to locate the
CodeCoverage.exe tool using setup configuration...
2018-03-06T11:14:56.4345085Z 11:14:56.433 Code coverage command line tool:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team
Tools\Dynamic Code Coverage Tools\CodeCoverage.exe
2018-03-06T11:14:56.4349101Z 11:14:56.434 Fetching code coverage report
information from TFS...
2018-03-06T11:14:56.4363869Z 11:14:56.435 Attempting to locate a test
results (.trx) file...
2018-03-06T11:14:57.0458245Z 11:14:57.044 Looking for TRX files in:
D:\a\1\TestResults, D:\a\1\s\TestResults
2018-03-06T11:14:57.0462747Z 11:14:57.045 Located a test results file:
D:\a\1\s\TestResults\VssAdministrator_factoryvm-az436_2018-03-
06_11_13_09.trx
2018-03-06T11:14:57.0600587Z 11:14:57.059 One code coverage attachment was
found in the trx file: factoryvm-az436\VssAdministrator_factoryvm-az436
2018-03-06 11_11_34.coverage
2018-03-06T11:14:57.0602504Z 11:14:57.059 Absolute path to coverage file:
D:\a\1\s\TestResults\VssAdministrator_factoryvm-az436_2018-03-
06_11_13_09\In\factoryvm-az436\VssAdministrator_factoryvm-az436 2018-03-06
11_11_34.coverage
2018-03-06T11:14:57.0691948Z 11:14:57.068 Executing file C:\Program Files
(x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code
Coverage Tools\CodeCoverage.exe
2018-03-06T11:14:57.0692731Z Args: analyze
/output:D:\a\1\s\TestResults\VssAdministrator_factoryvm-az436_2018-03-
06_11_13_09\In\factoryvm-az436\VssAdministrator_factoryvm-az436 2018-03-06
11_11_34.coveragexml D:\a\1\s\TestResults\VssAdministrator_factoryvm-
az436_2018-03-06_11_13_09\In\factoryvm-az436\VssAdministrator_factoryvm-
az436 2018-03-06 11_11_34.coverage
2018-03-06T11:14:57.0694963Z Working directory:
D:\a\1\s\TestResults\VssAdministrator_factoryvm-az436_2018-03-
06_11_13_09\In\factoryvm-az436
2018-03-06T11:14:57.0695792Z Timeout (ms):60000
2018-03-06T11:14:57.0699007Z Process id: 3540
2018-03-06T11:14:58.7847582Z 11:14:58.783 Process returned exit code 0
2018-03-06T11:14:58.7858908Z 11:14:58.784 Updating project info files with
code coverage information...

The VSTS extension should automatically import the coverage results, but there is currently a regression that will be fixed (hopefully) soon: SONARMSBRU-339
Meanwhile, there is a workaround explained in the VSTS extension documentation in "Analysing a .NET solution": in the Additional Properties text area, add the following property:
sonar.cs.vscoveragexml.reportsPaths=**/*.coveragexml

I was unable to see the 'coveragexml' file even though I set 'sonar.verbose=true', 'system.debug=true'. Although, I found another way that works just fine, i.e. converting '.coverage' files into '.coveragexml' with PowerShell script:
Get-ChildItem -Path $(Agent.TempDirectory) -Include "*.coverage" -File -Recurse | % {
$outfile = "$([System.IO.Path]::GetFileNameWithoutExtension($_.FullName)).coveragexml"
$output = [System.IO.Path]::Combine([System.IO.Path]::GetDirectoryName($_.FullName), $outfile)
"Analyse '$($_.Name)' with output '$outfile'..."
.$env:USERPROFILE\.nuget\packages\microsoft.codecoverage\15.8.0\build\netstandard1.0\CodeCoverage\CodeCoverage.exe analyze /output:$output $_.FullName
}
To make it work, make sure you have 'Inline PowerShell' task installed for the Marketplace and place it before 'Run Code Analysis' step in your build pipeline.

For those who use a self-hosted agent, you can also install Visual Studio Test Agent 2019.
It contains a CodeCoverage.exe file that the SonarQubeAnalyze#4 Azure DevOps task will find. SonarQube will be able to convert .coverage file into .coveragexml, according to this SonarQube documentation webpage.
Visual Studio Test Agent seems to be free. No need to pay for Visual Studio Enterprise.

Related

0% Code Coverage of Blazor component source file included in my SonarQube report

I have a Blazor app with many components. Running unit test for those components on Visual Studio gives me the fully coverage report.
As scanning code with SonarQube using MSBuild Scanner, my unit test command is:
dotnet test .\Tests\App.Test.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=opencover --logger trx
yet there’s 0% of Blazor code coverage in my SonarQube Web Report .
My Scanner commands:
dotnet "C:\Program Files\sonar-scanner\SonarScanner.MSBuild.dll" begin /k:"My-Project" /d:sonar.cs.opencover.reportsPaths="Tests\coverage.opencover.xml"
dotnet build
dotnet "C:\Program Files\sonar-scanner\SonarScanner.MSBuild.dll" end
I am wondering if there’s any missing item in my config file SonarQube.Analysis.xml.
Unfortunately SonarQube doesn't analyze Blazor files (yet). So it doesn't process the result of your coverage report.

Missing coverage results for Dotcover in Sonarqube

I am able to generate the HTML reports with the help of DotCover and passing the same to Sonarqube using "/d:sonar.cs.dotcover.reportsPaths=.\dotCover\dotCoverResults.html" (dotcover html has coverage data in it). In Sonar log I do not see any error message but Sonar dashboard shows no coverage; it is blank. Similar configuration for another application is working perfectly fine in other application but not in this. Blow is the configuration used in Jenkins for this.
Sonarqube version: 4.5.7
MSBuild SonarQube Runner: 2.0
C#: 5.2
dotnet restore --configfile .\.nuget\NuGet.Config
MSBuild myApplication.sln /t:Clean /t:Rebuild /p:Configuration=Debug /p:Platform=x64
nunit3-console.exe test\UnitTests\bin\Debug\net461\UnitTests.dll --result=nunit-test-results.xml;format=nunit2 --labels=On
dotCover.exe analyse .\dotCover\coverageDebug.xml
Any help is greatly appreciated.

No ProjectInfo.xml files were found

i am using visual studio 2015 and tfs for build the project,
I started the build but got stuck with the error described below,
2016-03-17T09:38:15.2135195Z ##[error]No ProjectInfo.xml files were found. Possible causes:
2016-03-17T09:38:15.2135195Z ##[error]1. The project has not been built - the end step was called right after the begin step, without a build step in between
2016-03-17T09:38:15.2135195Z ##[error]2. An unsupported version of MSBuild has been used to build the project. Currently MSBuild 12.0 upwards are supported
2016-03-17T09:38:15.2135195Z ##[error]3. The build step has been launched from a different working folder
2016-03-17T09:38:15.2135195Z Generation of the sonar-properties file failed. Unable to complete SonarQube analysis.
2016-03-17T09:38:15.2135195Z 15:08:15.15 Creating a summary markdown file...
2016-03-17T09:38:15.2135195Z ##[error]Post-processing failed. Exit code: 1
2016-03-17T09:38:15.2135195Z ##[error]Unexpected exit code received from batch file: 1
Can any one help me in this?
Assuming you are using TFS2015 vNext build, and you are trying to add build tasks below to execute a SonarQube analysis:
· SonarQube for MSBuild – Begin Analysis
· SonarQube for MSBuild – End Analysis
From your error log, you need to make sure the End Analysis task should be used to create a step that is executed after the “Visual Studio Build” task step. Review blog Build Tasks for SonarQube Analysis get more information of how to use SonarQube analysis tasks.

Configuring TFS 2013 has build issues with SonarQube

I have configured SonarQube to work with my TFS 2013. I followed the steps as mentioned in the setup doc. When i try to give a build from TFS online or Visual Studio, I am able to successfully complete the steps before Finish the analysis and upload the results to SonarQube.
I get the error No ProjectInfo.xml files were found. Check that the analysis targets are referenced by the MSBuild projects being built.
I am unable to figure out what might me causing the issue. I checked one of the links : No ProjectInfo.xml files were found error from command prompt using MSBuild SonarQube Runner v1.0 which speaks about the command prompt. Can someone point out where the issue is?
Below is the error log:
2015-09-12T11:44:10.9194896Z Output directory: C:\a\6fb08ad4\.sonarqube\out
2015-09-12T11:44:10.9194896Z Config file: C:\a\6fb08ad4\.sonarqube\out
2015-09-12T11:44:10.9204886Z 11:44:10 AM Loading the SonarQube analysis config from C:\a\6fb08ad4\.sonarqube\conf\SonarQubeAnalysisConfig.xml
2015-09-12T11:44:10.9204886Z 11:44:10 AM Attempting to locate the CodeCoverage.exe tool...
2015-09-12T11:44:11.3231261Z 11:44:11 AM Multiple versions of VS are installed: 11, 12, 14
2015-09-12T11:44:11.3240831Z 11:44:11 AM Code coverage command line tool: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe
2015-09-12T11:44:11.3240831Z 11:44:11 AM Fetching code coverage report information from TFS...
2015-09-12T11:44:11.3250829Z 11:44:11 AM Attempting to locate a test results (.trx) file...
2015-09-12T11:44:11.3250829Z 11:44:11 AM No test results files found
2015-09-12T11:44:11.3260828Z 11:44:11 AM Generating SonarQube project properties file to C:\a\6fb08ad4\.sonarqube\out\sonar-project.properties
2015-09-12T11:44:11.3270789Z ##[error]11:44:11 AM No ProjectInfo.xml files were found. Check that the analysis targets are referenced by the MSBuild projects being built.
2015-09-12T11:44:11.3270789Z 11:44:11 AM Writing processing summary to C:\a\6fb08ad4\.sonarqube\out\ProjectInfo.log
2015-09-12T11:44:11.3270789Z 11:44:11 AM Generation of the sonar-properties file failed. Unable to complete SonarQube analysis.
2015-09-12T11:44:11.3280797Z 11:44:11 AM Creating a summary markdown file...
2015-09-12T11:44:11.3280797Z Process returned exit code 1
2015-09-12T11:44:11.3290776Z ##[error]Post-processing failed. Exit code: 1
2015-09-12T11:44:11.5740896Z ##[error]Unexpected exit code received from batch file: 1
2015-09-12T11:44:11.5790878Z ##[debug]summaryMdPath = C:\a\6fb08ad4\.sonarqube\out\summary.md
2015-09-12T11:44:11.5820880Z ##[debug]Uploading the summary.md file
Thanks!

How to get NCover from TestDriven.NET to generate code coverage via the command line

I have VS 2010 Premium SP1 and using MSTest
The project has a web reference to a Web Service
The tests are integration tests
TestDriven.NET-3.8.2877 RTM which includes NCover 1.5.8
My problem is that it runs the MSTests and creates the pass/fail *.trx file but will not create a coverage report. I get the coverage report if I right click and select test with\NCover. The problem is I want to run in in the command line so I can just make it a task in my continuous integration pipeline.
The error returned is:
Profiled process terminated. Profiler connection not established.
I searched for solutions and tried the following:
Using the //reg in the command line
Updating the config.exe file to use .NET 4.0
Forcing the NCover DLL to register
Here is my command line:
ncover.console.exe //x mstest_coverage.nccov "c:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe" /testcontainer:D:\Perforce3666\serviceengine\SessionStorage\dev\SQATesting\SessionStorage.SQA.Testing\bin\Release\SessionStorage.SQA.Testing.dll

Resources