So far I've been developing my project using CLion on Linux and everything worked fine, but now I'm trying to setup my project on Windows to get familiar with the Visual Studio IDE, and I'm having trouble getting Visual Studio to discover my tests.
I'm using the CMake project directly in Visual Studio and things like building, running and etc. work fine, the problem beings when I try to run the individual tests using the Run Test option in the editor - the Test Explorer doesn't see them.
I've installed the Catch2 Test Adapter and added the required .runsettings, as described here but even with that the only things that I see in Tests output is
No tests found to run.
Has anyone tried a setup similar to this and can guide me on how to solve it?
I'm using Visual Studio 2019.
Ok, I got it working by generating a Visual Studio Solution and using it instead of the CMake project, my .runsettings look like this:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<MaxCpuCount>4</MaxCpuCount>
<ResultsDirectory>.\TestResults</ResultsDirectory><!-- Path relative to solution directory -->
<TestSessionTimeout>60000</TestSessionTimeout><!-- Milliseconds -->
</RunConfiguration>
<!-- Adapter Specific sections -->
<!-- Catch2 adapter -->
<Catch2Adapter disabled="false">
<FilenameFilter>test</FilenameFilter>
<WorkingDirectoryRoot>Solution</WorkingDirectoryRoot>
</Catch2Adapter>
</RunSettings>
Select it in Test Explorer > Configure Run Settings > Select Solution wide runsettings File
We have our TFS (2013) build configured to run javascript qunit tests under Chutzpah (4.0.3) by following this post. By default, Chutzpah is executing with a parallelism set to 2 (according to the chutzpah.log generated during a build).
Is there a way to adjust the amount of parallelism? I'd like to set it to 1 to help diagnose some random failures.
You need to set the MaxDegreeOfParallelism property in the .runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<ChutzpahAdapterSettings>
<MaxDegreeOfParallelism>2</MaxDegreeOfParallelism>
</ChutzpahAdapterSettings>
</RunSettings>
See https://github.com/mmanela/chutzpah/wiki/Running-Unit-Tests-in-a-TFS-Build
I've run into a recurring problem with a few different projects using MSTest in VS2012, where every now and then my code coverage stops working (seemingly at random) and instead gives me:
Empty results generated: No binaries were instrumented. Make sure the
tests ran, required binaries were loaded, had matching symbol files,
and were not excluded through custom settings. For more information
see http://go.microsoft.com/fwlink/?LinkID=253731
I've checked the obvious (what it's suggested) but can't seem to figure out what is causing it.
Here is my runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage"
uri="datacollector://Microsoft/CodeCoverage/2.0"
assemblyQualifiedName=" Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector,
Microsoft.VisualStudio.TraceCollector,
Version=11.0.0.0,
Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Include>
<ModulePath>.*\.dll$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*\.tests.dll</ModulePath>
</Exclude>
</ModulePaths>
<Attributes>
<Exclude>
<Attribute>.*ExcludeFromCodeCoverageAttribute$</Attribute>
<Attribute>.*GeneratedCodeAttribute$</Attribute>
</Exclude>
</Attributes>
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
I just ran into this using Visual Studio 2019. The solution was to go to the "Test" menu item in VS and then update the Test -> Processor Architecture for AnyCPU Projects setting from X86 to X64.
This link solved my issue: Issue with Code Coverage in VS 2012
Close Visual Studio 2012, find the .suo file, delete (or rename) it, restart. Worked fine. No idea what is in the .suo file that prevented proper coverage analysis.
If you can't make the Code Coverage to work even after you've deleted the *.suo file, please check your Event Viewer for errors. In my case, after each run I had the following error:
"TraceLog Profiler failed in initialization due to a lack of instrumentation methods, process vstest.executionengine.x86.exe"
I've found the answer here.
In case the link is no longer available, I'm pasting the content in here:
If you find yourself with a an empty .coverage file and see errors
similar to the below in your event logs you most probably have a
corrupt install
(info) .NET Runtime version 4.0.30319.17929 - The profiler has
requested that the CLR instance not load the profiler into this
process. Profiler CLSID: '{b19f184a-cc62-4137-9a6f-af0f91730165}'.
Process ID (decimal): 12624. Message ID: [0x2516].
(Error) TraceLog Profiler failed in initialization due to a lack of
instrumentation methods, process vstest.executionengine.x86.exe
Check
a) Environment variable VS110COMNTOOLS is set to
\common7\tools
b) Regkey HKLM\SOFTWARE\Microsoft\VisualStudio\11.0\InstallDir is set
to your \Common7\IDE\
c) covrun32.dll and covrun64.dll exist in "\Team
Tools\Dynamic Code Coverage"
Good luck,
Nadav
I had a similar problem after running PerfView.
Re-running perfview having copied it into a folder of it's own and starting a collection run, followed by stopping it seems to have fixed the issue.
I was getting 0x8007007e errors loading the profiler with a guid of {9999995d-2cbb-4893-be09-fce80abc7564} (Vs2015 profiler) and {6652970f-1756-5d8d-0805-e9aad152aa84} (perfview profiler)
Hope that helps someone else.
There appears to be a bug in Microsoft.NET.Test.Sdk 16.3.0+ which results in the same error output and currently the workaround is to downgrade to 16.2.0 which worked for me. In addition to the troubleshooting tips MS provides here there may be SDK issues.
In my case, the issue was that my test dll path contained the string "DataCollector" and seems coverlet has an internal ignore over any path that matches something like .*DataCollector.*
I modified the Property group associated with my debug environment so that it looks like this:
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<DocumentationFile>OurProjectName.xml</DocumentationFile>
<DebugType>full</DebugType>
<DebugSymbols>true</DebugSymbols>
</PropertyGroup>
For my core.net 3.1 application running xUnit 2.4.1 this was the only thing that fixed it in visual studio 2019 preview.
Configure Test -> Processor Architecture >> AnyCPU Projects setting >>> X86 to X64.This works for me
I was also facing the same issue and tried out all the above options but it did not work for me.
I could solve this problem only after creating full debug information. The settings for full debug information may vary from version to version. As I am using MSVS 2017, in the build menu there is a submenu which allows to have the full debug version.
I hope this will help others as well.
I'm facing a situation where I'm getting the following error:
The solution is to set this in the app.config:
<startup useLegacyV2RuntimeActivationPolicy="true">
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
I've put this into the app.config file however, I'm running unit tests at the moment. I've added it to the test library config file but the Visual Studio unit test tools seem to be ignoring this setting and still throwing the error.
How or where do I need to set this in order to get Quality Tools to actually use this setting when running tests?
I try to configure the MSTest Settings ssection [Step 3 Runner: Visual Studio (sln)]. My particular configuration require to call a set of scripts (TestSetup.cmd) before the tests. Everything works fine in Visual Studio, but I don't know how to call the scripts from TeamCity.
I tried to add in the MSTest metadata the .vsmdi file but without success. The script are not executed.
My question is how to call the command from TeamCity in the MSTest Settings section?
Thank you and best regards!
testrunconfig file:
This is a default test run configuration for a local test run.
MSTest Settings screen shot TestSetup.cmd
Versions :
TeamCity Enterprise 5.1.3
Microsoft Visual Studio Team System 2008 Development Edition
I
Add a build step before the MSTest runner which runs your testsetup.cmd.