VS 2015 cannot open .coveragexml generated by CodeCoverage.exe - visual-studio

I am following Code Coverage Results Import (C#, VB.NET) documentation for SonarQube.
For my project, I am able to generate test coverage binary file as "SonarQubeTests.csproj.coverage"
Visual Studio can open this file with no problem. I can see all the details.
Then when I try to generate ".coveragexml" file with following command;
"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" "analyze /output:C:\SonarQubeTests.csproj.coveragexml C:\SonarQubeTests.csproj.coverage"
It does generate ".coveragexml" file, but when I try to open Visual Studio it gives me following error:
I can open this file with text editor and content seems all fine. But neither Visual Studio 2015 nor sonarqube does not read this file.
What I am missing here?
P.S I have also runner settings for Ms Test runner as below.
Not sure if this will effect something;
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<MaxCpuCount>1</MaxCpuCount>
<!-- Path relative to solution directory -->
<ResultsDirectory>C:\</ResultsDirectory>
<!-- [x86] | x64
- You can also change it from menu Test, Test Settings, Default Processor Architecture -->
<TargetPlatform>x86</TargetPlatform>
<!-- Framework35 | [Framework40] | Framework45 -->
<TargetFrameworkVersion>Framework45</TargetFrameworkVersion>
<!-- Path to Test Adapters -->
<TestAdaptersPaths>%SystemDrive%\Temp\foo;%SystemDrive%\Temp\bar</TestAdaptersPaths>
</RunConfiguration>
<!-- Configurations for data collectors -->
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Exclude>
<ModulePath>.*CPPUnitTestFramework.*</ModulePath>
</Exclude>
</ModulePaths>
<!-- We recommend you do not change the following values: -->
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
<!-- Parameters used by tests at runtime -->
<TestRunParameters>
<Parameter name="webAppUrl" value="http://localhost" />
<Parameter name="webAppUserName" value="Admin" />
<Parameter name="webAppPassword" value="Password" />
</TestRunParameters>
<!-- Adapter Specific sections -->
<!-- MSTest adapter -->
<MSTest>
<MapInconclusiveToFailed>True</MapInconclusiveToFailed>
<CaptureTraceOutput>false</CaptureTraceOutput>
<DeleteDeploymentDirectoryAfterTestRunIsComplete>False</DeleteDeploymentDirectoryAfterTestRunIsComplete>
<DeploymentEnabled>False</DeploymentEnabled>
<AssemblyResolution>
<Directory Path="D:\myfolder\bin\" includeSubDirectories="false"/>
</AssemblyResolution>
</MSTest>
</RunSettings>

I think you could use vstest.console to get code coverage directly. We do not sure whether VS could open the code coverage that generated by third party tools.
I think if we could vstest.console to get the code coverage will has not problems. please try it to see the results.

Related

DeploymentEnabled false is not working in my runsettings file

I currently updated to visual studio 2017 enterprise edition version 15.9.24 .
The issue that i'm running across is i'm not able to debug my test using the
runsettings file. When I debug the test its access any dlls and deployed files from my TestResults Out folder instead of bin/debug folder since i'm in debug mode. When i was running Visual studio 2015 I was using .testsettings file
and was able to run my test from the bin/debug folder. Since .testsettings is
going to be deprecated i decided to move to runsettings.
Searching the web and stack someone stated that i needed to have the settings located in the runsettings file that is labeled DeploymentEnabled to false to achieve this. I did this and still getting error because the application is running and trying to access the files from my TestResults Out folder. I also changed the run settings to forced legacy mode to true and added directory of my old test settings file but still not working.
Is there another setting that i'm not taken into consideration
or am i missing something or is something wrong with my run settings.
Please see my example of my current runsettings file below
Thanks for any help or direction given.
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<MaxCpuCount>1</MaxCpuCount>
<!-- Path relative to directory that contains .runsettings file
<ResultsDirectory>.\TestResults</ResultsDirectory> -->
<!-- x86 or x64 -->
<!-- You can also change it from the Test menu; choose "Processor Architecture for AnyCPU Projects" -->
<TargetPlatform>x86</TargetPlatform>
<!-- Framework35 | [Framework40] | Framework45 -->
<TargetFrameworkVersion>Framework45</TargetFrameworkVersion>
<!-- Path to Test Adapters -->
<!--<TestAdaptersPaths>%SystemDrive%\Temp\foo;%SystemDrive%\Temp\bar</TestAdaptersPaths>-->
<!-- TestSessionTimeout was introduced in Visual Studio 2017 version 15.5 -->
<!-- Specify timeout in milliseconds. A valid value should be greater than 0 -->
<TestSessionTimeout>15000000</TestSessionTimeout>
</RunConfiguration>
<!-- Configurations for data collectors -->
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Exclude>
<ModulePath>.*CPPUnitTestFramework.*</ModulePath>
</Exclude>
</ModulePaths>
<!-- We recommend you do not change the following values: -->
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
<!-- Parameters used by tests at run time -->
<TestRunParameters>
<Parameter name="webAppUrl" value="http://localhost" />
<Parameter name="xxxxUser" value="xxxxm,xxxxtAuto01,xxxxtAuto02,xxxxxtAuto03,xxxxtAuto04" />
<Parameter name="xxxxxxxxxnbr" value="xx86,xx90,xx91,xx92,xx93" />
</TestRunParameters>
<!-- MSTest adapter -->
<MSTest>
<MapInconclusiveToFailed>true</MapInconclusiveToFailed>
<CaptureTraceOutput>false</CaptureTraceOutput>
<DeleteDeploymentDirectoryAfterTestRunIsComplete>true</DeleteDeploymentDirectoryAfterTestRunIsComplete>
<DeploymentEnabled>false</DeploymentEnabled>
<!---->
<ForcedLegacyMode>true</ForcedLegacyMode>
<SettingsFile>%xxxx_Automation_Test\xxxx_Automation_Test\xxx_Automation_Test\xxxxAutomation.testsettings;</SettingsFile>
<InProcMode>true</InProcMode>
<KeepExecutorAliveAfterLegacyRun>False</KeepExecutorAliveAfterLegacyRun>
<AssemblyResolution>
<!--
<Directory path="C:\xxx_Automation\xxxx_Automation_Test\xxx_Automation_Test\bin\" includeSubDirectories="false"/>-->
</AssemblyResolution>
</MSTest>
</RunSettings>
I updated my runsettings to have the file name of of run settings and to
update the ForcedLegacyMode to true.
<ForcedLegacyMode>true</ForcedLegacyMode>
<SettingsFile>xxxxxAutomation.testsettings</SettingsFile>
<KeepExecutorAliveAfterLegacyRun>False</KeepExecutorAliveAfterLegacyRun>
<AssemblyResolution>

Brackets are not included as Code Coverage for Unit Tests c# in VSO

There are some code coverage inconsistencies between the VSO build pipeline and Visual Studio local. We found that the code coverage in Visual Studio is higher because it counts brackets as part of the code coverage. But it doesnt count in VSO build pipeline for some reason. Anyone knows why? Is this a known bug in VSO?
We use Xunit and c# for the tests and have test.runsettings to exclude code. Below is the test.runsettings file
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<!-- Path relative to directory that contains .runsettings file-->
<ResultsDirectory>.\TestResults</ResultsDirectory>
<TargetPlatform>x64</TargetPlatform>
<MaxCpuCount>10000</MaxCpuCount>
</RunConfiguration>
<!-- Configurations for data collectors -->
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<Attributes>
<Exclude>
<Attribute>^System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute$</Attribute>
</Exclude>
</Attributes>
<ModulePaths>
<Exclude>
<ModulePath>.*unittests.dll$</ModulePath>
<ModulePath>.*integrationtests.dll$</ModulePath>
<ModulePath>.*xunit.*.dll</ModulePath>
<ModulePath>.*fluentassertions.*$</ModulePath>
<ModulePath>.*moq.*$</ModulePath>
<ModulePath>.*telemetry.extensions.ai.netcore.*$</ModulePath>
<ModulePath>.*newtonsoft.*$</ModulePath>
<ModulePath>.*codeanalysis.*$</ModulePath>
</Exclude>
</ModulePaths>
<!-- We recommend you do not change the following values: -->
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
The reason why it's showing inconsistent code coverage is because in Visual Studio, I was running Debug mode instead of Release mode. After changing it to Release mode, the number is consistent to the VSO number.

MS Test settings > TeamCity > 64bit

I've configured my .testsettings file as per instructions to run Hosts in 64bit.
But I don't see anything actually written to the settings file.
<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="accounting" id="907240cb-5a90-45f4-a41f-2e73533c8c25" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Description>These are default test settings for a local test run.</Description>
<Deployment enabled="false" />
<Execution>
<Timeouts runTimeout="1800000" testTimeout="40000" />
<TestTypeSpecific>
<UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
<AssemblyResolution>
<TestDirectory useLoadContext="true" />
</AssemblyResolution>
</UnitTestRunConfig>
</TestTypeSpecific>
<AgentRule name="LocalMachineDefaultRole">
</AgentRule>
</Execution>
<Properties />
</TestSettings>
And so my build server (Teamcity) continues to build in 32bit.
i've also tried setting it in VS but that definitely doesn't get persisted anywhere for Teamcity to know
how do i force Teamcity to use 64bit ?

Is there any .runsettings documentation?

I'm looking for documentation for .runsettings files as used with vstest. Is there an xsd?
I can't seem to find much except for a few example files in the msdn documentation.
Runsettings (VS2012) are similar to testsettings (VS2010) where testsettings is specific to tests written for MSTest. VS2012 supports settings for different adapters. As such, the schema isn't a closed system so an XSD would not be comprehensive.
This MSDN article lists some high level details (https://msdn.microsoft.com/en-us/library/jj635153.aspx) for the elements in the runsettings file.
Here's a snippet from that article.
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<MaxCpuCount>1</MaxCpuCount>
<!-- Path relative to solution directory -->
<ResultsDirectory>.\TestResults</ResultsDirectory>
<!-- [x86] | x64
- You can also change it from menu Test, Test Settings, Default
Processor Architecture -->
<TargetPlatform>x86</TargetPlatform>
<!-- Framework35 | [Framework40] | Framework45 -->
<TargetFrameworkVersion>Framework40</TargetFrameworkVersion>
<!-- Path to Test Adapters -->
<TestAdaptersPaths>%SystemDrive%\Temp\foo;%SystemDrive%\Temp\bar</TestAdaptersPaths>
</RunConfiguration>
<!-- Configurations for data collectors -->
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Exclude>
<ModulePath>.*CPPUnitTestFramework.*</ModulePath>
</Exclude>
</ModulePaths>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
<!-- Parameters used by tests at runtime -->
<TestRunParameters>
<Parameter name="webAppUrl" value="http://localhost" />
<Parameter name="webAppUserName" value="Admin" />
<Parameter name="webAppPassword" value="Password" />
</TestRunParameters>
<!-- Adapter Specific sections -->
<!-- MSTest adapter -->
<MSTest>
<MapInconclusiveToFailed>True</MapInconclusiveToFailed>
<CaptureTraceOutput>false</CaptureTraceOutput>
<DeleteDeploymentDirectoryAfterTestRunIsComplete>False</DeleteDeploymentDirectoryAfterTestRunIsComplete>
<DeploymentEnabled>False</DeploymentEnabled>
<AssemblyResolution>
<Directory Path>"D:\myfolder\bin\" includeSubDirectories="false"/>
</AssemblyResolution>
</MSTest>
</RunSettings>
I found further information on runsettings (specific to code coverage) here:
http://msdn.microsoft.com/en-us/library/jj159530%28v=vs.110%29.aspx

When & How to use ILMerge with Visual Studio Project / Solution

I'm developing a medium sized enterprise application.
There are many projects / solutions to this.
For example:
Company.Data
Company.Data.LinqToSql
Company.Entities (business objects)
Company.BLL
I then have some applications - for example a windows service:
MyWindowsService.
When i deploy this (by creating a setup project) it installs a load of DLL's from the output of the above mentioned projects.
Is this where i should be using ILMerge? to create one assembly.... Company.dll for example?
How would i go about integrating this into my build process?
The question ILMerge Best Practices
has good info on why.
When I use ILMerge, I use it to build a single DLL, to simplify deployment.
As to How, I define a separate, custom VS project, "Converged.csproj" if you like. In that .csproj file I define a custom Compile target. It is boilerplate code, that performs an ILMerge on all the referenced assemblies for the project.
It looks like this:
<Target Name="Compile">
<!-- Outputs="$(IntermediateOutputPath)$(TargetFileName)" -->
<!-- Outputs="$(TargetPath)" -->
<Message Text="Performing the Ilmerge." />
<!-- in this CreateItem stanza, we collect all the DLLs for the referenced projects -->
<CreateItem Include="#(_ResolvedProjectReferencePaths)">
<Output TaskParameter="Include" ItemName="AssembliesToMerge" />
</CreateItem>
<!-- This weird bit of hieroglyphics is the assemblies to merge, quoted, and separated by spaces -->
<!-- Example: "c:\foo\project1\bin\Debug\ProjectOne.dll" "c:\foo\project2\bin\Debug\ProjectTwo.dll" -->
<Message Text="AssembliesToMerge= #(AssembliesToMerge -> '"%(Fullpath)"', ' ')" />
<!-- Message Text="TargetPath= $(TargetPath)" / -->
<Message Text="TargetFileName= $(TargetFileName)" />
<!-- produce the merged assembly - putting the output in the "IntermediateOutputPath" eg obj\Debug. -->
<!-- it will be copied later by the CopyFilestoOutputDirectory task defined in Microsoft.Common.Targets -->
<Error
Text="ILMerge cannot be found. You need to download and install ILMerge in order to build DotNetZip."
Condition="!Exists('$(ProgramFiles)\Microsoft\Ilmerge\Ilmerge.exe')" />
<Exec Command=""$(ProgramFiles)\Microsoft\Ilmerge\Ilmerge.exe" /t:library /xmldocs /out:"$(IntermediateOutputPath)$(TargetFileName)" #(AssembliesToMerge -> '"%(Fullpath)"', ' ') " />
<!-- for some reason the XML doc file does not get copied automatically from obj\Debug to bin\Debug. -->
<!-- we do it here explicitly. -->
<Copy SourceFiles="$(IntermediateOutputPath)$(AssemblyName).XML" DestinationFolder="$(OutDir)" SkipUnchangedFiles="true" OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)" />
</Target>

Resources