Running a test using Shims on a Visual Studio 2013 Test Agent - visual-studio

Question

I have found various iterations of this question across a number of websites, however so far I have not found anything that provides a full answer that worked.
I have just set up a number of Visual Studio Test Agents that all appear to be behaving and running most of the unit tests we are planning on initially running on them. However when I came to checking one of the unit tests it is failing on the line:
using (ShimsContext.Create()),
With the stack trace:
Result Message:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: Failed to resolve profiler path from COR_PROFILER_PATH and COR_PROFILER environment variables.
Having read some other posts I installed Visual Studio 2013 on the test agent and confirmed it would run the test locally. I then set COR_PROFILER_PATH to the profiler that was being used on my machine which was installed at: “C:\Program Files\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\IntelliTrace\12.0.0\Microsoft.IntelliTrace.Profiler.12.0.0.dll”.
It then failed in the same place with the trace:
Test method threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: Failed to get profiler module handle 'C:\Program Files\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\IntelliTrace\12.0.0\Microsoft.IntelliTrace.Profiler.12.0.0.dll'. The specified module could not be found ---> System.ComponentModel.Win32Exception: The specified module could not be found
The suggestion is the profiler has been loaded under a separate process and can't be used. Has anyone had any success with this or similar set ups? At a fundamental level can Visual Studio Test Agents run tests with Fakes?
Thanks

Answer 1

I know this is a bit old, so the OP probably got his answer, but for anyone else, I opened a ticket with Microsoft on this issue, and the answer we came to after an hour and a half of looking at my setup for this problem:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: Failed to resolve profiler path from COR_PROFILER_PATH and COR_PROFILER environment variables.
, was that I was using mstest.exe and I should have been using vstest.console.exe. In my case it was because I have Fakes test in my code now.
I'm using Visual Studio 2013 update 4.

Answer 2

I was having the same issue with a project I was working on. I found this issue noted:
https://connect.microsoft.com/VisualStudio/feedback/details/832744/unable-to-debug-shims-based-tests-unittestisolation-exception
There was an attached project, so I downloaded the project and ran the only unit test. Same error - the unit test was not successful. However, I right-clicked on System under the References, and clicked Add Fakes Assembly. Once it had generated the fakes for System and mscorlib, voila! The test turned green.
I was able to reproduce the issue in my project - I did not have fakes being generated for System. Once I generated them for System, my tests went from red to green!
Side note -- If you don't want to generate ALL of the fakes for System and mscorlib, you can modify the .fakes files like so.
mscorlib.fakes:
<Fakes xmlns="http://schemas.microsoft.com/fakes/2011/">
<Assembly Name="mscorlib" Version="4.0.0.0"/>
<StubGeneration Disable="true" />
<ShimGeneration Disable="true" />
</Fakes>
System.fakes:
<Fakes xmlns="http://schemas.microsoft.com/fakes/2011/">
<Assembly Name="System" Version="4.0.0.0"/>
<StubGeneration Disable="true" />
</Fakes>
I could not disable ShimGeneration for System.fakes - the test would fail.
I hope this fixes the problem for someone else - the error is extremely vague!

Answer 3

(I'm assuming your Test Agent is also a Build Agent.)
Is the build service version synced with the Visual Studio version which is installed?
I had the same issue in the following environment:
TFS 2013
Build Server w/ TFS 2013
Visual Studio 2015.3 installed in the build server
When I updated the TFS version on the build server to 2015.3, the issue stopped happening.
I know it may not make sense, but solved the problem as the TFS 2015 XAML Build is compatible with TFS 2013.4 app tier.