I'm facing a situation where I'm getting the following error:
The solution is to set this in the app.config:
<startup useLegacyV2RuntimeActivationPolicy="true">
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
I've put this into the app.config file however, I'm running unit tests at the moment. I've added it to the test library config file but the Visual Studio unit test tools seem to be ignoring this setting and still throwing the error.
How or where do I need to set this in order to get Quality Tools to actually use this setting when running tests?
Related
I'm using MSTest with Playwright in Visual Studio, and am having trouble with a selector. I want to debug using Playwright dev tools, Playwright inspector, etc. This page https://playwright.dev/docs/debug-selectors#using-playwright-inspector suggests setting PWDEBUG=1 or PWDEBUG=console. All the examples show running tests using VS Code, or command line test execution. I'm using Visual Studio 2022 with Test Explorer. Where do I set these variables?
I've tried:
Using the debug settings environment variables (nothing happens)
Creating a .runsettings file with the following contents:
<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
<EnvironmentVariables>
<PWDEBUG>1</PWDEBUG>
</EnvironmentVariables>
</RunSettings>
Then I used Test -> Configure Run Settings to select this file.
Still no debug console.
What am I missing?
I take this approach:
https://playwright.dev/docs/debug#browser-developer-tools
Just add page.pause() before the step where you need that selector and it will pause execution and allow you to debug
So far I've been developing my project using CLion on Linux and everything worked fine, but now I'm trying to setup my project on Windows to get familiar with the Visual Studio IDE, and I'm having trouble getting Visual Studio to discover my tests.
I'm using the CMake project directly in Visual Studio and things like building, running and etc. work fine, the problem beings when I try to run the individual tests using the Run Test option in the editor - the Test Explorer doesn't see them.
I've installed the Catch2 Test Adapter and added the required .runsettings, as described here but even with that the only things that I see in Tests output is
No tests found to run.
Has anyone tried a setup similar to this and can guide me on how to solve it?
I'm using Visual Studio 2019.
Ok, I got it working by generating a Visual Studio Solution and using it instead of the CMake project, my .runsettings look like this:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<MaxCpuCount>4</MaxCpuCount>
<ResultsDirectory>.\TestResults</ResultsDirectory><!-- Path relative to solution directory -->
<TestSessionTimeout>60000</TestSessionTimeout><!-- Milliseconds -->
</RunConfiguration>
<!-- Adapter Specific sections -->
<!-- Catch2 adapter -->
<Catch2Adapter disabled="false">
<FilenameFilter>test</FilenameFilter>
<WorkingDirectoryRoot>Solution</WorkingDirectoryRoot>
</Catch2Adapter>
</RunSettings>
Select it in Test Explorer > Configure Run Settings > Select Solution wide runsettings File
I've run into a recurring problem with a few different projects using MSTest in VS2012, where every now and then my code coverage stops working (seemingly at random) and instead gives me:
Empty results generated: No binaries were instrumented. Make sure the
tests ran, required binaries were loaded, had matching symbol files,
and were not excluded through custom settings. For more information
see http://go.microsoft.com/fwlink/?LinkID=253731
I've checked the obvious (what it's suggested) but can't seem to figure out what is causing it.
Here is my runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage"
uri="datacollector://Microsoft/CodeCoverage/2.0"
assemblyQualifiedName=" Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector,
Microsoft.VisualStudio.TraceCollector,
Version=11.0.0.0,
Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Include>
<ModulePath>.*\.dll$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*\.tests.dll</ModulePath>
</Exclude>
</ModulePaths>
<Attributes>
<Exclude>
<Attribute>.*ExcludeFromCodeCoverageAttribute$</Attribute>
<Attribute>.*GeneratedCodeAttribute$</Attribute>
</Exclude>
</Attributes>
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
I just ran into this using Visual Studio 2019. The solution was to go to the "Test" menu item in VS and then update the Test -> Processor Architecture for AnyCPU Projects setting from X86 to X64.
This link solved my issue: Issue with Code Coverage in VS 2012
Close Visual Studio 2012, find the .suo file, delete (or rename) it, restart. Worked fine. No idea what is in the .suo file that prevented proper coverage analysis.
If you can't make the Code Coverage to work even after you've deleted the *.suo file, please check your Event Viewer for errors. In my case, after each run I had the following error:
"TraceLog Profiler failed in initialization due to a lack of instrumentation methods, process vstest.executionengine.x86.exe"
I've found the answer here.
In case the link is no longer available, I'm pasting the content in here:
If you find yourself with a an empty .coverage file and see errors
similar to the below in your event logs you most probably have a
corrupt install
(info) .NET Runtime version 4.0.30319.17929 - The profiler has
requested that the CLR instance not load the profiler into this
process. Profiler CLSID: '{b19f184a-cc62-4137-9a6f-af0f91730165}'.
Process ID (decimal): 12624. Message ID: [0x2516].
(Error) TraceLog Profiler failed in initialization due to a lack of
instrumentation methods, process vstest.executionengine.x86.exe
Check
a) Environment variable VS110COMNTOOLS is set to
\common7\tools
b) Regkey HKLM\SOFTWARE\Microsoft\VisualStudio\11.0\InstallDir is set
to your \Common7\IDE\
c) covrun32.dll and covrun64.dll exist in "\Team
Tools\Dynamic Code Coverage"
Good luck,
Nadav
I had a similar problem after running PerfView.
Re-running perfview having copied it into a folder of it's own and starting a collection run, followed by stopping it seems to have fixed the issue.
I was getting 0x8007007e errors loading the profiler with a guid of {9999995d-2cbb-4893-be09-fce80abc7564} (Vs2015 profiler) and {6652970f-1756-5d8d-0805-e9aad152aa84} (perfview profiler)
Hope that helps someone else.
There appears to be a bug in Microsoft.NET.Test.Sdk 16.3.0+ which results in the same error output and currently the workaround is to downgrade to 16.2.0 which worked for me. In addition to the troubleshooting tips MS provides here there may be SDK issues.
In my case, the issue was that my test dll path contained the string "DataCollector" and seems coverlet has an internal ignore over any path that matches something like .*DataCollector.*
I modified the Property group associated with my debug environment so that it looks like this:
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<DocumentationFile>OurProjectName.xml</DocumentationFile>
<DebugType>full</DebugType>
<DebugSymbols>true</DebugSymbols>
</PropertyGroup>
For my core.net 3.1 application running xUnit 2.4.1 this was the only thing that fixed it in visual studio 2019 preview.
Configure Test -> Processor Architecture >> AnyCPU Projects setting >>> X86 to X64.This works for me
I was also facing the same issue and tried out all the above options but it did not work for me.
I could solve this problem only after creating full debug information. The settings for full debug information may vary from version to version. As I am using MSVS 2017, in the build menu there is a submenu which allows to have the full debug version.
I hope this will help others as well.
I did quite a lot of research on the web and tried a few settings, but I couldn't reproduce the behavior of running MsTest in Visual Studio 2012 on the command line.
Our solution consists of many projects that build to the same bin folder residing at the solution level (e.g. C:\MySolution\bin) - this is the code-under-test (CUT). The tests are grouped in a separate project that resides in its own solution and builds in its own bin folder (e.g. C:\MySolution\Tests\bin). There are really a lot of plugins thus we want MsTest to reference the CUT bin folder when running the test intead of copying everything to the TestResults folder. We did achieve this in Visual Studio 2012 by editing the .testrunconfig and specifying ".\bin" as "root folder for the assemblies to be loaded" (in tab "Unit Test" when editing the testrunconfig). So we can load the test solution in VS2012 and run the tests there without having to copy the bin folder contents to the TestResults directory.
Now I wanted to create a .bat file that would run MsTest the same way as in VS2012 so that we can omit launching Visual Studio just for running the tests. I'm now working on how to execute MsTest on the command line, but have been quite frustrated. This is what I tried (command executed on the solution level in a VS command prompt):
MsTest /testcontainer:Tests\bin\Tests.dll
This did not work at all, it couldn't even find the referenced dlls that Tests.dll need to run. So I re-used the configuration and ran it again:
MsTest /runconfig:LocalTestRun.testrunconfig /testcontainer:Tests\bin\Tests.dll
Still it did not work. It could start the tests, but all of them failed. I got a lot of warnings of the kind
Warning: Test Run deployment issue: The assembly or module '....' directly or indirectly referenced by the test container 'C:\MySolution\Tests\bin\tests.dll' was not found.
and in the end it said:
The configured application base directory 'C:\MySolution\TestResults\User_Machine 2013-07-28 13_16_59\Out\bin' does not exist. The test directory will be used instead.
When I changed the option applicationBaseDirectory in the testrunconfig to an absolute path (C:\MySolution\bin), it worked. Still I get many warnings such as:
Warning: Test Run deployment issue: The assembly or module '....' directly or indirectly referenced by the test container 'C:\MySolution\Tests\bin\tests.dll' was not found.
But anyway, it's not really a feasible solution to specify an absolute path. How can I run MsTest on the command line with a different, but relative assembly base directory?
My LocalTestRun.testrunconfig is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local Test Run" id="...." xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Description>This is a default test run configuration for a local test run.</Description>
<Deployment>
<DeploymentItem filename="Tests\....\Resources\" />
</Deployment>
<Execution hostProcessPlatform="MSIL">
<TestTypeSpecific>
<UnitTestRunConfig testTypeId="....">
<AssemblyResolution applicationBaseDirectory=".\bin">
<TestDirectory useLoadContext="true" />
</AssemblyResolution>
</UnitTestRunConfig>
<WebTestRunConfiguration testTypeId="....">
<Browser name="Internet Explorer 7.0">
<Headers>
<Header name="User-Agent" value="Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" />
<Header name="Accept" value="*/*" />
<Header name="Accept-Language" value="{{$IEAcceptLanguage}}" />
<Header name="Accept-Encoding" value="GZIP" />
</Headers>
</Browser>
</WebTestRunConfiguration>
</TestTypeSpecific>
<AgentRule name="LocalMachineDefaultRole">
</AgentRule>
</Execution>
</TestSettings>
After more searching we changed to use the test console runner that is included with VS2012:
VSTest.Console.exe Tests\bin\Tests.dll /Framework:framework40 /Settings:LocalTestRun.testrunconfig
This works with a relative path as applicationBaseDirectory.
This is due to an MSTest bug that sets the current directory to its own working directory, rather than the test project's bin (or deployment) folder. The workaround is to execute the following code in the constructor of your test class:
Environment.CurrentDirectory(AppDomain.CurrentDomain.BaseDirectory);
I got the idea from http://www.ademiller.com/blogs/tech/2008/01/gotchas-mstest-appdomain-changes-in-vs-2008/; however, note that, in my case at least, it required setting Environment.CurrentDirectory, rather than the reverse, as suggested in the article.
I've turned on Code Analysis for all of my projects, and have verified code analysis is on for each configuration (Debug, Release, QA, etc). My local builds are of course fine, my TFS builds are failing still.
Here is the failed output XML file in the drop location for my builds:
<?xml version="1.0" encoding="utf-8"?>
<Tests xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<BuildConfigurationSettings name="Debug" ProjectName="MyProject" Platform="AnyCPU">
<Test>
<MSBuildSchemaName>CodeAnalysisRuleSet</MSBuildSchemaName>
<ExpectedValue>MinimumRecommendedRules.ruleset</ExpectedValue>
<ActualValue />
<Verify>true</Verify>
<TestExecuted>true</TestExecuted>
<IsEqual>false</IsEqual>
</Test>
</BuildConfigurationSettings>
<BuildConfigurationSettings name="Release" ProjectName="MyProject" Platform="AnyCPU">
<Test>
<MSBuildSchemaName>CodeAnalysisRuleSet</MSBuildSchemaName>
<ExpectedValue>MinimumRecommendedRules.ruleset</ExpectedValue>
<ActualValue />
<Verify>true</Verify>
<TestExecuted>true</TestExecuted>
<IsEqual>false</IsEqual>
</Test>
</BuildConfigurationSettings>
</Tests>
For some reason, even though Code Analysis was turned on in the 'Properties' for the failing projects, the changes were not being reflected in the .csproj file. After hand-editing the .csproj file to include the minimum ruleset, everything was working fine:
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
Visual Studio edition on build server should support for code analysis And there are some known issues:
Unfortunately due to a bug discovered very late in the release cycle
the install for Code Analysis does not execute on x64 build servers.
This means you won’t see code analysis running despite correct
configuration. The work around for now is to install a Visual Studio
SKU that includes Code Analysis on the build machine to get the FxCop
installation.