So far I've been developing my project using CLion on Linux and everything worked fine, but now I'm trying to setup my project on Windows to get familiar with the Visual Studio IDE, and I'm having trouble getting Visual Studio to discover my tests.
I'm using the CMake project directly in Visual Studio and things like building, running and etc. work fine, the problem beings when I try to run the individual tests using the Run Test option in the editor - the Test Explorer doesn't see them.
I've installed the Catch2 Test Adapter and added the required .runsettings, as described here but even with that the only things that I see in Tests output is
No tests found to run.
Has anyone tried a setup similar to this and can guide me on how to solve it?
I'm using Visual Studio 2019.
Ok, I got it working by generating a Visual Studio Solution and using it instead of the CMake project, my .runsettings look like this:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<MaxCpuCount>4</MaxCpuCount>
<ResultsDirectory>.\TestResults</ResultsDirectory><!-- Path relative to solution directory -->
<TestSessionTimeout>60000</TestSessionTimeout><!-- Milliseconds -->
</RunConfiguration>
<!-- Adapter Specific sections -->
<!-- Catch2 adapter -->
<Catch2Adapter disabled="false">
<FilenameFilter>test</FilenameFilter>
<WorkingDirectoryRoot>Solution</WorkingDirectoryRoot>
</Catch2Adapter>
</RunSettings>
Select it in Test Explorer > Configure Run Settings > Select Solution wide runsettings File
Related
I'm using MSTest with Playwright in Visual Studio, and am having trouble with a selector. I want to debug using Playwright dev tools, Playwright inspector, etc. This page https://playwright.dev/docs/debug-selectors#using-playwright-inspector suggests setting PWDEBUG=1 or PWDEBUG=console. All the examples show running tests using VS Code, or command line test execution. I'm using Visual Studio 2022 with Test Explorer. Where do I set these variables?
I've tried:
Using the debug settings environment variables (nothing happens)
Creating a .runsettings file with the following contents:
<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
<EnvironmentVariables>
<PWDEBUG>1</PWDEBUG>
</EnvironmentVariables>
</RunSettings>
Then I used Test -> Configure Run Settings to select this file.
Still no debug console.
What am I missing?
I take this approach:
https://playwright.dev/docs/debug#browser-developer-tools
Just add page.pause() before the step where you need that selector and it will pause execution and allow you to debug
I've run into a recurring problem with a few different projects using MSTest in VS2012, where every now and then my code coverage stops working (seemingly at random) and instead gives me:
Empty results generated: No binaries were instrumented. Make sure the
tests ran, required binaries were loaded, had matching symbol files,
and were not excluded through custom settings. For more information
see http://go.microsoft.com/fwlink/?LinkID=253731
I've checked the obvious (what it's suggested) but can't seem to figure out what is causing it.
Here is my runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage"
uri="datacollector://Microsoft/CodeCoverage/2.0"
assemblyQualifiedName=" Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector,
Microsoft.VisualStudio.TraceCollector,
Version=11.0.0.0,
Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Include>
<ModulePath>.*\.dll$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*\.tests.dll</ModulePath>
</Exclude>
</ModulePaths>
<Attributes>
<Exclude>
<Attribute>.*ExcludeFromCodeCoverageAttribute$</Attribute>
<Attribute>.*GeneratedCodeAttribute$</Attribute>
</Exclude>
</Attributes>
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
I just ran into this using Visual Studio 2019. The solution was to go to the "Test" menu item in VS and then update the Test -> Processor Architecture for AnyCPU Projects setting from X86 to X64.
This link solved my issue: Issue with Code Coverage in VS 2012
Close Visual Studio 2012, find the .suo file, delete (or rename) it, restart. Worked fine. No idea what is in the .suo file that prevented proper coverage analysis.
If you can't make the Code Coverage to work even after you've deleted the *.suo file, please check your Event Viewer for errors. In my case, after each run I had the following error:
"TraceLog Profiler failed in initialization due to a lack of instrumentation methods, process vstest.executionengine.x86.exe"
I've found the answer here.
In case the link is no longer available, I'm pasting the content in here:
If you find yourself with a an empty .coverage file and see errors
similar to the below in your event logs you most probably have a
corrupt install
(info) .NET Runtime version 4.0.30319.17929 - The profiler has
requested that the CLR instance not load the profiler into this
process. Profiler CLSID: '{b19f184a-cc62-4137-9a6f-af0f91730165}'.
Process ID (decimal): 12624. Message ID: [0x2516].
(Error) TraceLog Profiler failed in initialization due to a lack of
instrumentation methods, process vstest.executionengine.x86.exe
Check
a) Environment variable VS110COMNTOOLS is set to
\common7\tools
b) Regkey HKLM\SOFTWARE\Microsoft\VisualStudio\11.0\InstallDir is set
to your \Common7\IDE\
c) covrun32.dll and covrun64.dll exist in "\Team
Tools\Dynamic Code Coverage"
Good luck,
Nadav
I had a similar problem after running PerfView.
Re-running perfview having copied it into a folder of it's own and starting a collection run, followed by stopping it seems to have fixed the issue.
I was getting 0x8007007e errors loading the profiler with a guid of {9999995d-2cbb-4893-be09-fce80abc7564} (Vs2015 profiler) and {6652970f-1756-5d8d-0805-e9aad152aa84} (perfview profiler)
Hope that helps someone else.
There appears to be a bug in Microsoft.NET.Test.Sdk 16.3.0+ which results in the same error output and currently the workaround is to downgrade to 16.2.0 which worked for me. In addition to the troubleshooting tips MS provides here there may be SDK issues.
In my case, the issue was that my test dll path contained the string "DataCollector" and seems coverlet has an internal ignore over any path that matches something like .*DataCollector.*
I modified the Property group associated with my debug environment so that it looks like this:
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<DocumentationFile>OurProjectName.xml</DocumentationFile>
<DebugType>full</DebugType>
<DebugSymbols>true</DebugSymbols>
</PropertyGroup>
For my core.net 3.1 application running xUnit 2.4.1 this was the only thing that fixed it in visual studio 2019 preview.
Configure Test -> Processor Architecture >> AnyCPU Projects setting >>> X86 to X64.This works for me
I was also facing the same issue and tried out all the above options but it did not work for me.
I could solve this problem only after creating full debug information. The settings for full debug information may vary from version to version. As I am using MSVS 2017, in the build menu there is a submenu which allows to have the full debug version.
I hope this will help others as well.
I did quite a lot of research on the web and tried a few settings, but I couldn't reproduce the behavior of running MsTest in Visual Studio 2012 on the command line.
Our solution consists of many projects that build to the same bin folder residing at the solution level (e.g. C:\MySolution\bin) - this is the code-under-test (CUT). The tests are grouped in a separate project that resides in its own solution and builds in its own bin folder (e.g. C:\MySolution\Tests\bin). There are really a lot of plugins thus we want MsTest to reference the CUT bin folder when running the test intead of copying everything to the TestResults folder. We did achieve this in Visual Studio 2012 by editing the .testrunconfig and specifying ".\bin" as "root folder for the assemblies to be loaded" (in tab "Unit Test" when editing the testrunconfig). So we can load the test solution in VS2012 and run the tests there without having to copy the bin folder contents to the TestResults directory.
Now I wanted to create a .bat file that would run MsTest the same way as in VS2012 so that we can omit launching Visual Studio just for running the tests. I'm now working on how to execute MsTest on the command line, but have been quite frustrated. This is what I tried (command executed on the solution level in a VS command prompt):
MsTest /testcontainer:Tests\bin\Tests.dll
This did not work at all, it couldn't even find the referenced dlls that Tests.dll need to run. So I re-used the configuration and ran it again:
MsTest /runconfig:LocalTestRun.testrunconfig /testcontainer:Tests\bin\Tests.dll
Still it did not work. It could start the tests, but all of them failed. I got a lot of warnings of the kind
Warning: Test Run deployment issue: The assembly or module '....' directly or indirectly referenced by the test container 'C:\MySolution\Tests\bin\tests.dll' was not found.
and in the end it said:
The configured application base directory 'C:\MySolution\TestResults\User_Machine 2013-07-28 13_16_59\Out\bin' does not exist. The test directory will be used instead.
When I changed the option applicationBaseDirectory in the testrunconfig to an absolute path (C:\MySolution\bin), it worked. Still I get many warnings such as:
Warning: Test Run deployment issue: The assembly or module '....' directly or indirectly referenced by the test container 'C:\MySolution\Tests\bin\tests.dll' was not found.
But anyway, it's not really a feasible solution to specify an absolute path. How can I run MsTest on the command line with a different, but relative assembly base directory?
My LocalTestRun.testrunconfig is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local Test Run" id="...." xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Description>This is a default test run configuration for a local test run.</Description>
<Deployment>
<DeploymentItem filename="Tests\....\Resources\" />
</Deployment>
<Execution hostProcessPlatform="MSIL">
<TestTypeSpecific>
<UnitTestRunConfig testTypeId="....">
<AssemblyResolution applicationBaseDirectory=".\bin">
<TestDirectory useLoadContext="true" />
</AssemblyResolution>
</UnitTestRunConfig>
<WebTestRunConfiguration testTypeId="....">
<Browser name="Internet Explorer 7.0">
<Headers>
<Header name="User-Agent" value="Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" />
<Header name="Accept" value="*/*" />
<Header name="Accept-Language" value="{{$IEAcceptLanguage}}" />
<Header name="Accept-Encoding" value="GZIP" />
</Headers>
</Browser>
</WebTestRunConfiguration>
</TestTypeSpecific>
<AgentRule name="LocalMachineDefaultRole">
</AgentRule>
</Execution>
</TestSettings>
After more searching we changed to use the test console runner that is included with VS2012:
VSTest.Console.exe Tests\bin\Tests.dll /Framework:framework40 /Settings:LocalTestRun.testrunconfig
This works with a relative path as applicationBaseDirectory.
This is due to an MSTest bug that sets the current directory to its own working directory, rather than the test project's bin (or deployment) folder. The workaround is to execute the following code in the constructor of your test class:
Environment.CurrentDirectory(AppDomain.CurrentDomain.BaseDirectory);
I got the idea from http://www.ademiller.com/blogs/tech/2008/01/gotchas-mstest-appdomain-changes-in-vs-2008/; however, note that, in my case at least, it required setting Environment.CurrentDirectory, rather than the reverse, as suggested in the article.
I get following error everytime I try to create a new Test Project:
Microsoft Visual Studio
Failed to get service ITestToolSettingsManager.
OK
I am currently running Visual Studio 2010 Ultimate with Feature Pack 2 on Windows 7 Ultimate 64-bit.
I have also tried following with no avail:
Run VS under admin mode
Reset VS settings by going to Tools --> Import and Export Settings
Run "devenv \setup" --> this fails
Run "devenv \InstallTFSTemplates"
Run "devenv \ResetSettings" --> this fails
Your help in resolving is much appreciated.
Kind regards,
HG
Probably same deal I had File => new project test projects Fails.
Installed Templates => Visual C# => Test => Test Project
works
found answer at
http://social.msdn.microsoft.com/Forums/en-US/vsunittest/thread/8739096e-afb3-4760-baeb-ae707fb3ab09
When I try to run my test suite, which works fine under Visual Studio 2008, I get the following error in the Output window:
Error loading J:\VBProjects2010\Libraries\PFTest\PFTest\bin\Debug\PFTest.dll: Could not load file or assembly 'file:///J:\VBProjects2010\Libraries\PFTest\PFTest\bin\Debug\PFTest.dll' or one of its dependencies. Operation is not supported. (Exception from HRESULT: 0x80131515)
Note that these same tests can be viewed in the Test View windows, and run well under Visual Studio 2008, but fail only under VS2010.
Note that PFTest.dll is the test project's name, and the dll is the file created when I compile (which compiles ok).
strong textIn addition to the above, I realised I could not step through code either since the compiled dlls and exe could not be loaded from the shared drive.
This solved the problem.
Edit C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\devenv.exe.config
Add <loadFromRemoteSources enabled="true"/> under <runtime> so that it looks like this:
<runtime>
<loadFromRemoteSources enabled="true" />
<UseSmallInternalThreadStacks enabled="true" />
...
The gouderadrian's anwser did not worked for me.
I am on a VirtualBox machine with Windows 7 64 bits and Visual Studio 2012.
I had to move the output to a local folder : open the unit tests project properties and in the Build tab, browse the Output path to a local folder. The tests are now detected and can be run.
For Visual Studio 2010 Express for Windows Phone the file you're looking for has the name VDPExpress.exe.config.
MS's docs here: http://msdn.microsoft.com/en-us/library/dd409252.aspx