Code Coverage Results periodically gives: Empty results generated - visual-studio

I've run into a recurring problem with a few different projects using MSTest in VS2012, where every now and then my code coverage stops working (seemingly at random) and instead gives me:
Empty results generated: No binaries were instrumented. Make sure the
tests ran, required binaries were loaded, had matching symbol files,
and were not excluded through custom settings. For more information
see http://go.microsoft.com/fwlink/?LinkID=253731
I've checked the obvious (what it's suggested) but can't seem to figure out what is causing it.
Here is my runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage"
uri="datacollector://Microsoft/CodeCoverage/2.0"
assemblyQualifiedName=" Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector,
Microsoft.VisualStudio.TraceCollector,
Version=11.0.0.0,
Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Include>
<ModulePath>.*\.dll$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*\.tests.dll</ModulePath>
</Exclude>
</ModulePaths>
<Attributes>
<Exclude>
<Attribute>.*ExcludeFromCodeCoverageAttribute$</Attribute>
<Attribute>.*GeneratedCodeAttribute$</Attribute>
</Exclude>
</Attributes>
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>

I just ran into this using Visual Studio 2019. The solution was to go to the "Test" menu item in VS and then update the Test -> Processor Architecture for AnyCPU Projects setting from X86 to X64.

This link solved my issue: Issue with Code Coverage in VS 2012
Close Visual Studio 2012, find the .suo file, delete (or rename) it, restart. Worked fine. No idea what is in the .suo file that prevented proper coverage analysis.

If you can't make the Code Coverage to work even after you've deleted the *.suo file, please check your Event Viewer for errors. In my case, after each run I had the following error:
"TraceLog Profiler failed in initialization due to a lack of instrumentation methods, process vstest.executionengine.x86.exe"
I've found the answer here.
In case the link is no longer available, I'm pasting the content in here:
​
If you find yourself with a an empty .coverage file and see errors
similar to the below in your event logs you most probably have a
corrupt install
(info) .NET Runtime version 4.0.30319.17929 - The profiler has
requested that the CLR instance not load the profiler into this
process. Profiler CLSID: '{b19f184a-cc62-4137-9a6f-af0f91730165}'.
Process ID (decimal): 12624. Message ID: [0x2516].
(Error) TraceLog Profiler failed in initialization due to a lack of
instrumentation methods, process vstest.executionengine.x86.exe
Check
a) Environment variable VS110COMNTOOLS is set to
\common7\tools
b) Regkey HKLM\SOFTWARE\Microsoft\VisualStudio\11.0\InstallDir is set
to your \Common7\IDE\
c) covrun32.dll and covrun64.dll exist in "\Team
Tools\Dynamic Code Coverage"
Good luck,
Nadav

I had a similar problem after running PerfView.
Re-running perfview having copied it into a folder of it's own and starting a collection run, followed by stopping it seems to have fixed the issue.
I was getting 0x8007007e errors loading the profiler with a guid of {9999995d-2cbb-4893-be09-fce80abc7564} (Vs2015 profiler) and {6652970f-1756-5d8d-0805-e9aad152aa84} (perfview profiler)
Hope that helps someone else.

There appears to be a bug in Microsoft.NET.Test.Sdk 16.3.0+ which results in the same error output and currently the workaround is to downgrade to 16.2.0 which worked for me. In addition to the troubleshooting tips MS provides here there may be SDK issues.

In my case, the issue was that my test dll path contained the string "DataCollector" and seems coverlet has an internal ignore over any path that matches something like .*DataCollector.*

I modified the Property group associated with my debug environment so that it looks like this:
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<DocumentationFile>OurProjectName.xml</DocumentationFile>
<DebugType>full</DebugType>
<DebugSymbols>true</DebugSymbols>
</PropertyGroup>
For my core.net 3.1 application running xUnit 2.4.1 this was the only thing that fixed it in visual studio 2019 preview.

Configure Test -> Processor Architecture >> AnyCPU Projects setting >>> X86 to X64.This works for me

I was also facing the same issue and tried out all the above options but it did not work for me.
I could solve this problem only after creating full debug information. The settings for full debug information may vary from version to version. As I am using MSVS 2017, in the build menu there is a submenu which allows to have the full debug version.
I hope this will help others as well.

Related

No tests found to run - while debugging/running Unit test cases -- Visual studio 2017 15.5.1

Error : No tests found to run - while debugging/running C# Unit test
cases -- Visual studio 2017 15.5.1 not discovering test cases.
Framework
Microsoft.NET framework 4.6.1
Added Test adapters as below
MSTest.TestAdapter.1.2.0
MSTest.TestFramework.1.2.0
Test explorer not showing the test cases.
This started happening after upgrading visual studio.
Some unit test projects test cases do not get discovered in the test explorer.
When try to debug or run it says "No tests found to run."
I tried deleting project and recreating it again .. but didn't worked.
Output
[4/3/2018 2:59:25 PM Informational] No tests found to run.
I have updated VS to 15.6.4
Deleting VS test cache files DEL %TEMP%\VisualStudioTestExplorerExtensions
Restarted Visual studio
Test cases got discovered but not debugging
Problem
If I understand you correctly, this is something you observe:
when you run the tests, the following can be seen in the console:
[4/3/2018 2:26:13 PM Informational] ------ Run test started ------
[4/3/2018 2:26:14 PM Warning] No test is available in d:\...\UnitTestProject1.dll. Make sure that test discoverer & executors are
registered and platform & framework version settings are appropriate and try
again.
and the test explorer shows:
What could have happened, and it happened to me a couple of times, that the test discovery is set up for another architecture than your currently set one.
Example
Your current configuration is x64
but, the default test settings might have another one (e.g. x86):
Solution
If you _align the test settings with your current architecture (x86 → x64), the tests should run.
Manual cleanups should not be necessary nowadays, if the solution is configured consistently
I had same problem, make sure you set TestClass to public !
[TestClass]
public class CSharp_JavaScript_ServerResponse_Test
{
}
I had may versions of the Nuget Package installed, I consolidated them into the latest one and it fixed the problem
MSTest.TestAdapter
MSTest.TestFramework
As of today, me using NUnit # VSCode and .NET Core 5.0
was missing Microsoft.NET.Test.SDK NuGet package..
As soon as I added it to ALL test assemblies (i.e.: all .csproj files in solution .sln), everything works fine.
HTH to many who reach this far.
These PackageReferences are in my TestProject.csproj
<ItemGroup>
<PackageReference Include="NUnit" Version="3.13.1"/>
<PackageReference Include="NUnit.Console" Version="3.12.0"/>
<PackageReference Include="NUnit3TestAdapter" Version="4.0.0-beta.2"/>
<PackageReference Include="NUnitTestAdapter" Version="2.3.0"/>
<PackageReference Include="NUnit.Extension.NUnitProjectLoader" Version="3.6.0"/>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.10.0-release-20210422-02"/>
</ItemGroup>
The accepted answer didn't work for me in isolation but along with this it did:
Exit MSVS
Delete the .vs folder (you may lose a few settings related only to that solution)
Restart MSVS
From the accepted answer, ensure the Test -> Settings -> Def Arch-> matches the code
Load playlist (if applicable)
Build
Test!
Try clearing temp files located in the %TEMP% directory
Delete folder : VisualStudioTestExplorerExtensions in %temp% directory.
Note: This path is generally at C:\Users\(yourusername)\AppData\Local\Temp
In my case, "Build -> Clean Solution" did the trick.
Also, make sure you have both MSTest.TestFramework AND MSTest.TestAdapter Nuget packages installed
Guys
I was having the same problem, and fix it by adding the [TestClass] and [TestMethod], and got my Selenium Automated test library working on VS 2015.
Cheers,
[TestClass]
public class GigHubAutomatedTest
{
[TestMethod]
public void StartApplication()
{
using(IWebDriver googleDriver = new ChromeDriver())
{
// 1. Maximize the browser
googleDriver.Manage().Window.Maximize();
googleDriver.Navigate().GoToUrl("https://www.google.com/");
}
}
}
I did below thing and it resolved my problem
I have updated VS to latest version 15.6.4
Deleting VS test cache files DEL %TEMP%\VisualStudioTestExplorerExtensions
Restarted Visual Studio
Then test cases got discovered but not debugging.
It was giving error as:
Dependency finder domain): Loaded 'C:\PROGRAM FILES (X86)\MICROSOFT
VISUAL
STUDIO\2017\ENTERPRISE\COMMON7\IDE\EXTENSIONS\TESTPLATFORM\Microsoft.VisualStudio.TestPlatform.ObjectModel.dll'.
Cannot find or open the PDB file.
and similar error for other DLLs as below:
C:\PROGRAM FILES (X86)\MICROSOFT VISUAL
STUDIO\2017\ENTERPRISE\COMMON7\IDE\EXTENSIONS\TESTPLATFORM\Microsoft.VisualStudio.TestPlatform.Common.dll'.
Cannot find or open the PDB file.
So PDB is a debug information file used by Visual Studio. These are system DLLs, which you don't have debug symbols for.
Go to Tools->Options->Debugging->Symbols and select checkbox "Microsoft Symbol Servers", Visual Studio will download PDBs automatically.
Then I downgraded:
MSTest.TestAdapter.1.2.0 to 1.1.18
MSTest.TestFramework.1.2.0 to 1.1.18
Then restarted VS studio and rebuild.
Still got below error:
System.IO.FileNotFoundException: Could not load file or assembly
'System.Net.Http, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The
system cannot find the file specified.
Then I set version to newVersion="4.0.0.0"
<dependentAssembly>
<assemblyIdentity name="System.Net.Http" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-4.1.1.0" newVersion="4.0.0.0" />
</dependentAssembly>
And it started debugging.
My issue was I was using async-await, and the return type was void. It needs to be Task.
Changed this:
public async void MyTest()
To this:
public async Task MyTest()
You can get this error message if you make the mistake of right-clicking inside the editor then choosing Debug Test(s) when you are NOT in a unit test method (e.g. you have a method Foo() and you put a breakpoint in there and want to run the tests so they cause the breakpoint to be hit...nothing will happen if you choose Debug Test(s) while inside the body of Foo()...you have to go to Test Explorer and choose Debug or use the shortcut Ctrl+T,Ctrl+R).

Team foundation server auto build

We have a large non Visual Studio C++ project which is build via a batch file. I have integrated this up in Visual Studio as a makefile project with customised 'Build Command Line' in the project properties, this works well and has done for over a year.
I am now looking to introduce Team Foundation Server and configure automated builds.
If I build the project in Visual Studio then the project build as normal.
If I use the automated build sever to build the project the build fails every time
The option to view the log file in the build report is greyed out. The report given is
The command "Autobuild.cmd" exited with code 1.
Where autobuild is the cmd file specified in the solution's 'Build Command Line'.
Can anyone think of a reason why this might be the case, I assumed that the build sever would just run the Build command with the project specified in the 'Items to Build' box. If this is the case then I cannot see why it would fail.
There are loads of possible reasons; unfortunately you've not given enough information to answer that questions.
The first thing to try is changing your build definition logging level to Diagnostic. (you can also do this when you queue a new build).
What I also recommend is actually wrapping your makefile in an msbuild.proj file.
Something simple like:
<Project ToolsVersion="4.0" DefaultTargets="build"
xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="build">
<Exec Command="autobuild.cmd" />
</Target>
</Project>
The Exec task has loads of options to customize how the cmd is called and should help you diagnose your error.
In general if you want to take advantaged of TFS builds I'd recommend migrating as much of your autobuild.cmd to msbuild as possible. MSBuild is TFS's "makefile" language, so-to-speak.
In your build definition you have an option of logging, there you can change that to Verbose logging. Maybe you get some information in your log file then.
And for the issue of exited with code 1 I think you will need to add some extra informational lines to be printed in the cmd program and when your logging in TFS Build is active you maybe see more information on it.

TFS Build failing because of Code Analysis settings

I've turned on Code Analysis for all of my projects, and have verified code analysis is on for each configuration (Debug, Release, QA, etc). My local builds are of course fine, my TFS builds are failing still.
Here is the failed output XML file in the drop location for my builds:
<?xml version="1.0" encoding="utf-8"?>
<Tests xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<BuildConfigurationSettings name="Debug" ProjectName="MyProject" Platform="AnyCPU">
<Test>
<MSBuildSchemaName>CodeAnalysisRuleSet</MSBuildSchemaName>
<ExpectedValue>MinimumRecommendedRules.ruleset</ExpectedValue>
<ActualValue />
<Verify>true</Verify>
<TestExecuted>true</TestExecuted>
<IsEqual>false</IsEqual>
</Test>
</BuildConfigurationSettings>
<BuildConfigurationSettings name="Release" ProjectName="MyProject" Platform="AnyCPU">
<Test>
<MSBuildSchemaName>CodeAnalysisRuleSet</MSBuildSchemaName>
<ExpectedValue>MinimumRecommendedRules.ruleset</ExpectedValue>
<ActualValue />
<Verify>true</Verify>
<TestExecuted>true</TestExecuted>
<IsEqual>false</IsEqual>
</Test>
</BuildConfigurationSettings>
</Tests>
For some reason, even though Code Analysis was turned on in the 'Properties' for the failing projects, the changes were not being reflected in the .csproj file. After hand-editing the .csproj file to include the minimum ruleset, everything was working fine:
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
Visual Studio edition on build server should support for code analysis And there are some known issues:
Unfortunately due to a bug discovered very late in the release cycle
the install for Code Analysis does not execute on x64 build servers.
This means you won’t see code analysis running despite correct
configuration. The work around for now is to install a Visual Studio
SKU that includes Code Analysis on the build machine to get the FxCop
installation.

QualityToolsPackage failed to load in build?

I am using Bamboo [from Altassian] and it uses the devenv.com builder to build solution files. Currently, I seem to be getting a "false" error in my builds - that I've tried to solve by myself but just can't - so I thought I would ask.
Each build succeeds normally - without errors stemming from code - but seems to instead give this error
Package 'Microsoft.VisualStudio.TestTools.TestCaseManagement.QualityToolsPackage, Microsoft.VisualStudio.QualityTools.TestCaseManagement, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' failed to load.
I've no idea why this is causing problems in the devenv.com environment and I can't figure out either how to "ignore" this error by some build command?
Try this ( from MSFT support )
Can you check if the dll is in the global assembly cache (open a VS 2010 command prompt
(Start | All Programs | Microsoft Visual Studio 10.0 | Visual Studio Tools))
Type in
Gacutil –l > list.txt
Notepad list.txt
Do you see an entry like
Microsoft.VisualStudio.QualityTools.TestCaseManagement, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL
If so, from the same command prompt, run fuslogvw, go to settings, select log bind failures to disk, select ok
Run your command line
C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\devenv.com /Build "Release|Mixed Platforms" "D:\Builds\19\Test VizXView\VizXView v1.8 Test Release Build - Weekly\Sources\VizXView v1.8\VizXView 1.8.sln"
Click on the refresh button in fuslogvw. Do we get any bind errors?
I had a similar issue related to running unit tests on a build server that was using devenv.exe (not devenv.com). I had a premium VS installed on the build server but I'd written the unit tests with ultimate (which 'has' load testing, even tho I'd not used it). I used fusion viewer to work out that the missing dll was LoadTest.dll which I copied from my laptop to the buildserver, I also removed references in my solution that were in version control to: *.vsmdi and *.testsettings (they're deprecated) and I removed from the build def a reference to a .testsettings file.
My guess would be that even though I wasn't running a load unit test the build agent was trying to load the LoadTest libs just in case. Hope that helps
edit ---
ok, my own problem here was that I've got both vs2010 and vs2013 on my laptop. When I added the first test project to my VS2010 solution, the project added actually has a dependency on the 2013 version of the UnitTestFramework.dll. My build server only has vs2010 on it so I got the missing assembly error. Switching the reference to the 2010 version fixed this.

VSTS 2010 SGEN : error : Could not load file or assembly (Exception from HRESULT: 0x80131515)

I am experiencing a strange issue with VS2010. We use TFS to build our API dlls and we used to reference them in our projects usign a mapped network drive that was fully trusted. We have been working like that for at least two years and everything worked perfectly.
Today, I converted a webapp to vs2010 and when I compile it in Release, it's giving me:
SGEN : error : Could not load file or
assembly 'file:///L:\Api\Release
API_20100521.1\Release\CS.API.Exceptions.dll' or one of its dependencies. Operation
is not supported. (Exception from
HRESULT: 0x80131515)
The strange thing is that it's working when it's under the Debug profile...
I tried adding the
<runtime>
<loadFromRemoteSources enabled="true" />
</runtime>
into app.config and still no luck (See http://social.msdn.microsoft.com/Forums/en/msbuild/thread/d12f6301-85bf-4b9e-8e34-a06398a60df0 and http://msdn.microsoft.com/en-us/library/dd409252(VS.100).aspx)
I am pretty sure that this issue is from visual studio or msbuild, as our code won't run from a network share when in prod because all the referenced dll's are copied into the bin folder.
If anyone has an solution (or just an idea for a search path) please let me know !
Edit : It turns out that it was working in Debug mode because generation of serialisation assemblies was turned Off. As the title say, it's really a SGEN problem since it is this utility that says that the path is not trusted...
I was able to fix this error by finding the assembly DLL in Windows Explorer, right clicking, choosing Properties, and then pressing the "unblock" button. The DLL has a stream that is marking it as an external file - and by clicking unblock you remove that designation.
I just had the same/similar issue on a TFS build server where a build was referencing dll's from a network share.
The problems is that the CLR v4 security policy model has changed since previous versions and are not sandboxing assemblies as before.
To fix your issue just find the location of sgen.exe and create a sgen.exe.config in the same folder with following contents:
<configuration>
<runtime>
<loadFromRemoteSources enabled="true" />
</runtime>
</configuration>
sgen.exe is usually at
"C:\Program Files\Microsoft SDKs\Windows\v[current version]\bin\NETFX 4.0 Tools"
You can read about some of the changes around CAS policies in .NET 4.0 in this blogpost: Link
Had the same problem and the config change didnt work. Only when i set Generate Serialization Assembly to off in the project properties did it work.
I had the same error and found my DLL was "blocked". Open up the DLL in explorer, right click -> properties -> press 'Unblock'.
http://cantgrokwontgrok.blogspot.com/2009/10/visual-studio-unknown-build-error.html
I had this exact same problem and fixed it by adding the sgen.exe.config under C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools
with this simple config as others have said
<?xml version ="1.0"?>
<configuration>
<runtime>
<loadFromRemoteSources enabled="true" />
</runtime>
</configuration>
For those of you running a 64bit version of the TFS build service, I had to create the config file in the following path:
C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools\x64
And the file contents:
<?xml version ="1.0"?>
<configuration>
<runtime>
<loadFromRemoteSources enabled="true" />
</runtime>
</configuration>
I had the same issue, loaded the assembly in the GAC and worked
Adding the snippet below to the app.config file worked in my case. I'm Running Windows XP, with VS2010 service pack 1.
<configuration>
<runtime>
<loadFromRemoteSources enabled="true" />
</runtime>
</configuration>
In my case bunch of dlls were blocked.
To unblock all files in folder I used power shell with following command
dir -Path [directory path] -Recurse | Unblock-File
Just as an FYI if you are running Windows 7 the sgen.exe file can be found at:
C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools
I had to create a sgen.exe.config and place it there and then this problem went away.
Neither the unblock nor the config worked for me.
What did the trick for me was this tip about caspol.
I ran
%windir%\Microsoft.NET\Framework\v2.0.50727\CasPol.exe -m -ag 1.2 -url file://UncPathName/UncSubPath/* FullTrust
And I was ready to go, not even a VisualStudio restart required.
I got a similar problem and I finally got over with it by removing the licenses.licx file in the Properties folder of the solution.
Just in case like me, Unblock was not a solution, as Unblock does not appear on my dll file properties. Kept looking and ended up closing my solution file and re-opening using the local C: copy instead of network UNC path to project sln file. Was able to publish after going this route.

Resources