I'm using MSTest with Playwright in Visual Studio, and am having trouble with a selector. I want to debug using Playwright dev tools, Playwright inspector, etc. This page https://playwright.dev/docs/debug-selectors#using-playwright-inspector suggests setting PWDEBUG=1 or PWDEBUG=console. All the examples show running tests using VS Code, or command line test execution. I'm using Visual Studio 2022 with Test Explorer. Where do I set these variables?
I've tried:
Using the debug settings environment variables (nothing happens)
Creating a .runsettings file with the following contents:
<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
<EnvironmentVariables>
<PWDEBUG>1</PWDEBUG>
</EnvironmentVariables>
</RunSettings>
Then I used Test -> Configure Run Settings to select this file.
Still no debug console.
What am I missing?
I take this approach:
https://playwright.dev/docs/debug#browser-developer-tools
Just add page.pause() before the step where you need that selector and it will pause execution and allow you to debug
So far I've been developing my project using CLion on Linux and everything worked fine, but now I'm trying to setup my project on Windows to get familiar with the Visual Studio IDE, and I'm having trouble getting Visual Studio to discover my tests.
I'm using the CMake project directly in Visual Studio and things like building, running and etc. work fine, the problem beings when I try to run the individual tests using the Run Test option in the editor - the Test Explorer doesn't see them.
I've installed the Catch2 Test Adapter and added the required .runsettings, as described here but even with that the only things that I see in Tests output is
No tests found to run.
Has anyone tried a setup similar to this and can guide me on how to solve it?
I'm using Visual Studio 2019.
Ok, I got it working by generating a Visual Studio Solution and using it instead of the CMake project, my .runsettings look like this:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<MaxCpuCount>4</MaxCpuCount>
<ResultsDirectory>.\TestResults</ResultsDirectory><!-- Path relative to solution directory -->
<TestSessionTimeout>60000</TestSessionTimeout><!-- Milliseconds -->
</RunConfiguration>
<!-- Adapter Specific sections -->
<!-- Catch2 adapter -->
<Catch2Adapter disabled="false">
<FilenameFilter>test</FilenameFilter>
<WorkingDirectoryRoot>Solution</WorkingDirectoryRoot>
</Catch2Adapter>
</RunSettings>
Select it in Test Explorer > Configure Run Settings > Select Solution wide runsettings File
I want to add multiple Ms unit test project in single solution in VS 2010 and want to execute all projects test case in single shot in Bamboo CI.
In Bamboo i am able to test Single project by using project dll but not able to execute Test written in multiple project .
I tried on google and found .vsmdi file will do that functionality but not crack how to all test cases add in to vsmdi file .
Please help me.
The Vsmdi File is look like this :
<?xml version="1.0" encoding="UTF-8"?>
<TestLists xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<TestList name="Lists of Tests" id="8c43106b-9dc1-4907-a29f-aa66a61bf5b6">
<RunConfiguration id="68cd1a40-fbe7-4a52-bc91-47f0ca5ea70a" name="Local" storage="local.testsettings" type="Microsoft.VisualStudio.TestTools.Common.TestRunConfiguration, Microsoft.VisualStudio.QualityTools.Common, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</TestList>
</TestLists>
I've run into a recurring problem with a few different projects using MSTest in VS2012, where every now and then my code coverage stops working (seemingly at random) and instead gives me:
Empty results generated: No binaries were instrumented. Make sure the
tests ran, required binaries were loaded, had matching symbol files,
and were not excluded through custom settings. For more information
see http://go.microsoft.com/fwlink/?LinkID=253731
I've checked the obvious (what it's suggested) but can't seem to figure out what is causing it.
Here is my runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage"
uri="datacollector://Microsoft/CodeCoverage/2.0"
assemblyQualifiedName=" Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector,
Microsoft.VisualStudio.TraceCollector,
Version=11.0.0.0,
Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Include>
<ModulePath>.*\.dll$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*\.tests.dll</ModulePath>
</Exclude>
</ModulePaths>
<Attributes>
<Exclude>
<Attribute>.*ExcludeFromCodeCoverageAttribute$</Attribute>
<Attribute>.*GeneratedCodeAttribute$</Attribute>
</Exclude>
</Attributes>
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
I just ran into this using Visual Studio 2019. The solution was to go to the "Test" menu item in VS and then update the Test -> Processor Architecture for AnyCPU Projects setting from X86 to X64.
This link solved my issue: Issue with Code Coverage in VS 2012
Close Visual Studio 2012, find the .suo file, delete (or rename) it, restart. Worked fine. No idea what is in the .suo file that prevented proper coverage analysis.
If you can't make the Code Coverage to work even after you've deleted the *.suo file, please check your Event Viewer for errors. In my case, after each run I had the following error:
"TraceLog Profiler failed in initialization due to a lack of instrumentation methods, process vstest.executionengine.x86.exe"
I've found the answer here.
In case the link is no longer available, I'm pasting the content in here:
If you find yourself with a an empty .coverage file and see errors
similar to the below in your event logs you most probably have a
corrupt install
(info) .NET Runtime version 4.0.30319.17929 - The profiler has
requested that the CLR instance not load the profiler into this
process. Profiler CLSID: '{b19f184a-cc62-4137-9a6f-af0f91730165}'.
Process ID (decimal): 12624. Message ID: [0x2516].
(Error) TraceLog Profiler failed in initialization due to a lack of
instrumentation methods, process vstest.executionengine.x86.exe
Check
a) Environment variable VS110COMNTOOLS is set to
\common7\tools
b) Regkey HKLM\SOFTWARE\Microsoft\VisualStudio\11.0\InstallDir is set
to your \Common7\IDE\
c) covrun32.dll and covrun64.dll exist in "\Team
Tools\Dynamic Code Coverage"
Good luck,
Nadav
I had a similar problem after running PerfView.
Re-running perfview having copied it into a folder of it's own and starting a collection run, followed by stopping it seems to have fixed the issue.
I was getting 0x8007007e errors loading the profiler with a guid of {9999995d-2cbb-4893-be09-fce80abc7564} (Vs2015 profiler) and {6652970f-1756-5d8d-0805-e9aad152aa84} (perfview profiler)
Hope that helps someone else.
There appears to be a bug in Microsoft.NET.Test.Sdk 16.3.0+ which results in the same error output and currently the workaround is to downgrade to 16.2.0 which worked for me. In addition to the troubleshooting tips MS provides here there may be SDK issues.
In my case, the issue was that my test dll path contained the string "DataCollector" and seems coverlet has an internal ignore over any path that matches something like .*DataCollector.*
I modified the Property group associated with my debug environment so that it looks like this:
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<DocumentationFile>OurProjectName.xml</DocumentationFile>
<DebugType>full</DebugType>
<DebugSymbols>true</DebugSymbols>
</PropertyGroup>
For my core.net 3.1 application running xUnit 2.4.1 this was the only thing that fixed it in visual studio 2019 preview.
Configure Test -> Processor Architecture >> AnyCPU Projects setting >>> X86 to X64.This works for me
I was also facing the same issue and tried out all the above options but it did not work for me.
I could solve this problem only after creating full debug information. The settings for full debug information may vary from version to version. As I am using MSVS 2017, in the build menu there is a submenu which allows to have the full debug version.
I hope this will help others as well.
I've turned on Code Analysis for all of my projects, and have verified code analysis is on for each configuration (Debug, Release, QA, etc). My local builds are of course fine, my TFS builds are failing still.
Here is the failed output XML file in the drop location for my builds:
<?xml version="1.0" encoding="utf-8"?>
<Tests xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<BuildConfigurationSettings name="Debug" ProjectName="MyProject" Platform="AnyCPU">
<Test>
<MSBuildSchemaName>CodeAnalysisRuleSet</MSBuildSchemaName>
<ExpectedValue>MinimumRecommendedRules.ruleset</ExpectedValue>
<ActualValue />
<Verify>true</Verify>
<TestExecuted>true</TestExecuted>
<IsEqual>false</IsEqual>
</Test>
</BuildConfigurationSettings>
<BuildConfigurationSettings name="Release" ProjectName="MyProject" Platform="AnyCPU">
<Test>
<MSBuildSchemaName>CodeAnalysisRuleSet</MSBuildSchemaName>
<ExpectedValue>MinimumRecommendedRules.ruleset</ExpectedValue>
<ActualValue />
<Verify>true</Verify>
<TestExecuted>true</TestExecuted>
<IsEqual>false</IsEqual>
</Test>
</BuildConfigurationSettings>
</Tests>
For some reason, even though Code Analysis was turned on in the 'Properties' for the failing projects, the changes were not being reflected in the .csproj file. After hand-editing the .csproj file to include the minimum ruleset, everything was working fine:
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
Visual Studio edition on build server should support for code analysis And there are some known issues:
Unfortunately due to a bug discovered very late in the release cycle
the install for Code Analysis does not execute on x64 build servers.
This means you won’t see code analysis running despite correct
configuration. The work around for now is to install a Visual Studio
SKU that includes Code Analysis on the build machine to get the FxCop
installation.