How does Microsoft Test Manager run automated Unit tests? - microsoft-test-manager

I have associated some test cases with unit tests in a project. That project is build and copied to a drop location on a share. When I go to run these tests I'm getting a lot of failures due to System.DllNotFoundExceptions from an unmanaged DLL that is included as part of these tests?
Does anyone know why this is happening? I have a lab environment setup and the tests are (supposedly) running on that machine. What is the workflow for when these tests run? Does the TestController copy the unit test DLLs somewhere locally on the machine in the lab environment or is it trying to run directly from the build folder?
Here is the stack trace for the error. Note that c:\Builds\ is NOT on the test machine which seems quite suspect to me.
Test method OutputProcessing.Integration.PanaceaOutputG1.ReactorCoreSimulationResults threw exception:
System.DllNotFoundException: Unable to load DLL 'ge.gnf.cedar06.c.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
Gnf.Cedar.NativeMethods.ATACHF(Int32& fileCode, String fileName, Int32& openMode, Int32[] stat)
Gnf.Cedar.ManagedMethods.Atachf(String fileName, FileAccess access, Int32[] stat)
Gnf.Cedar.Cypress.CedarFile.OpenCedarFile(String path, FileAccess access, FileSizeHint hint, AuditMode auditMode, ChainingOrderOption chainingOrder, Boolean createIfMissing)
Gnf.Cedar.Cypress.CedarFile..ctor(String path, FileMode mode, FileAccess access, FileSizeHint hint, AuditMode auditMode, ChainingOrderOption chainingOrder)
Gnf.Cedar.Cypress.CedarFile..ctor(String path, FileMode mode, FileAccess access)
GE.GNF.Acumen.Cedar.DataAccess.CedarRepository.GetCedarFile() in c:\Builds\11\ACUMEN\DEV Test Build\src\Acumen\Cedar\Cedar.DataAccess\CedarRepository.cs: line 67
GE.GNF.Acumen.Cedar.DataAccess.CedarRepository..ctor(String fullFilePath) in c:\Builds\11\ACUMEN\DEV Test Build\src\Acumen\Cedar\Cedar.DataAccess\CedarRepository.cs: line 40
GE.GNF.Acumen.Cedar.DataAccess.CedarRepository..ctor(String fullFilePath, String qualifier) in c:\Builds\11\ACUMEN\DEV Test Build\src\Acumen\Cedar\Cedar.DataAccess\CedarRepository.cs: line 43
OutputProcessing.Integration.PanaceaOutputG1.ReactorCoreSimulationResults() in c:\Builds\11\ACUMEN\DEV Test Build\src\Acumen\Auto Integration Tests\OutputProcessing.Integration\PanaceaOutputG1.cs: line 58

The DLLs containing test implementation will be copied to the test machine.
You can activate Test Agent logs and check in the log file which files have been deployed.
To activate these logs:
Go to the C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE folder
(this is the path for Test Agent 2012, if you are using older version the path with be slightly
different).
Change file QTAgentService.exe.config (set the EqtTraceLevel to "4" and CreateTraceListener to "yes"):
<configuration>
<system.diagnostics>
<switches>
<!-- You must use integral values for "value".
Use 0 for off, 1 for error, 2 for warn, 3 for info, and 4 for verbose. -->
<add name="EqtTraceLevel" value="4" />
</switches>
</system.diagnostics>
<appSettings>
<add key="CreateTraceListener" value="yes"/>
</appSettings>
</configuration>
Restart Test Agent and you will find the file VSTTAgent.log created in the same directory.
Start your tests now and then check this logfile. You will find entries like this one (just search for "Copying remote file":
I, 5856, 4, 2014/01/31, 09:50:58.283, <computer name>\QTAgentService.exe, Copying remote file: C:\Users\<tfs build user>\AppData\Local\VSEQT\QTController\<test run>\Deployment\<your DLL file> to: C:\Users\<local user>\AppData\Local\VSEQT\QTAgent\<test run>\<test configuration>\Deployment\<your DLL file>
Now you can check if all your DLLs have been deployed.
Note: The source path is located on the machine Test Controller is running on, the destination path is located on the test machine (where Test Agent is running).
As much as I know unmanaged DLLs will not be deployed along with "unit tests DLLs" so you will have to make sure they will be deployed.
You can do it using Test Settings:

Related

How to overcome "Could not load type 'Microsoft.Cct.Services.Sqm.IWatSqmService'" without updating?

I'm getting the same error message as in this question: Could not load type 'Microsoft.Cct.Services.Sqm.IWatSqmService' Unfortunately, I cannot upgrade to Azure SDK 2.9, as the top answer there suggests.
Specifically, I get this error in a message box whenever I try to package my Azure Cloud Service project:
What's interesting is that I can start the same project in the emulator, which I would assume involves packaging it.
Things I've tried:
Ensuring that I only have version 2.8.2 of the emulator installed (I may have had 2.9 installed initially)
Reinstalling version 2.8.2 of the SDK
Manually copying the Windows Azure Tools from C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0 to the v14.0 equivalent.
Ensuring that C:\Program Files\Microsoft SDKs\Azure\.NET SDK\v2.8 exists
Reinstalling Visual Studio (so you know I'm desperate!)
And I've no idea what to try next. I can't work out which DLL is supposed to hold the Microsoft.VisualStudio.WindowsAzure.Services assembly, I can't find any references in any config files, and all the Google hits seem to point back to the SO question linked above.
What's going on? And how can I fix it?
In case I'm being an idiot, here's all the Azure-related things I've got installed:
Also (just in case it's relevant) I'm on Windows 7 SP1.
Update 17/01/16
Following some advice from the excellent Rest Azured Slack channel, I've tried using ProcMon and Fusion Log to diagnose this (spoiler: without success)
ProcMon filtered to that assembly returns nothing:
Fusion log (set to log everything) yields lots of entries like this:
* Assembly Binder Log Entry (17/01/2017 # 11:52:02) *
The operation was successful. Bind result: hr = 0x0. The operation
completed successfully.
Assembly manager loaded from:
C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll Running under
executable C:\Program Files (x86)\Microsoft Visual Studio
14.0\Common7\IDE\devenv.exe
--- A detailed error log follows.
=== Pre-bind state information === LOG: DisplayName = Microsoft.VisualStudio.WindowsAzure.Services, Version=1.1.0.0,
Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (Fully-specified)
LOG: Appbase = file:///C:/Program Files (x86)/Microsoft Visual Studio
14.0/Common7/IDE/ LOG: Initial PrivatePath = NULL LOG: Dynamic Base = NULL LOG: Cache Base = NULL LOG: AppName = devenv.exe Calling assembly
: Microsoft.VisualStudio.WindowsAzure, Version=2.9.0.0,
Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a.
=== WRN: Native image will not be probed in LoadFrom context. Native image will only be probed in default load context, like with
Assembly.Load(). WRN: No matching native image found. LOG: IL assembly
loaded from C:\Program Files (x86)\Microsoft Visual Studio
14.0\Common7\IDE\Extensions\Microsoft\Windows Azure Tools\Microsoft.VisualStudio.WindowsAzure.Services.dll.
When I explore the DLL mentioned, I can find Microsoft.Cct.Services.Sqm, but it doesn't contain a type IWatSqlService:
This feels like it's the crux of the issue, but I've no idea what to do next. Presumably I'd need to replace the DLL with a version that does define IWatSqlService, but I'm not sure where I'd get one. Or, for that matter, why it would even be necessary.
Next stop is probably a full wipe of the machine.
I've had the same problem, and solved it by not using the Web Platform Installer.
Download all parts (Microsoft Azure SDK for .NET - 2.8.2) from the Microsoft site: https://www.microsoft.com/en-us/download/details.aspx?id=50041 and follow the instructions how to install them.
For anyone else with this problem. I ended up resolving this by nuking it from orbit. Fresh install of Windows, reinstall VS and the SDK.
Confusingly, you can also see this error when you add a new
<setting name="TODO" />
element to your *.csdef file but either:
forget to add the corresponding <setting name="TODO" value="TODO" /> element to every *.cscfg file, or
accidentally add a duplicate <setting name="TODO" value="TODO" /> element to any of the *.cscfg files.

Could not execute automatic test using mstest.exe under Virtual Windows 8 Machine

I have the following issue: can not execute automatic test using mstest command under Virtual Windows 8 Machine. I enabled mstest logging and I came upon the following error:
LocalControllerProxy: Exception Initializing ControllerObject or starting agent process(es): System.ArgumentException: The path is not of a legal form.
at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
at System.IO.Path.GetFullPathInternal(String path)
at System.IO.Path.GetFullPath(String path)
at Microsoft.VisualStudio.TestTools.Common.ControllerDefaults.GetControllerWorkingDirectory()
at Microsoft.VisualStudio.TestTools.Controller.ControllerObject..ctor(Boolean remote, ControllerConfiguration controllerConfiguration, AgentManager agentManager)
at Microsoft.VisualStudio.TestTools.TestManagement.LocalControllerProxy.InitializeController(TestRun run)
The error in the trx file is:
System.ArgumentException: The path is not of a legal form.
at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
at System.IO.Path.GetFullPathInternal(String path)
at System.IO.Path.GetFullPath(String path)
at Microsoft.VisualStudio.TestTools.Common.ControllerDefaults.GetControllerWorkingDirectory()
at Microsoft.VisualStudio.TestTools.Controller.ControllerObject..ctor(Boolean remote, ControllerConfiguration controllerConfiguration, AgentManager agentManager)
at Microsoft.VisualStudio.TestTools.TestManagement.LocalControllerProxy.InitializeController(TestRun run)
at Microsoft.VisualStudio.TestTools.TestManagement.ControllerProxy.QueueTestRunWorker(Object state)
The issues are gone and the test could be executed if I disable Admin Approval Mode, but in this case I could not run Metro apps. That's why I need to keep this option enabled.
Anyone experiencing such an issue? I've been trying to find a solution over the internet for the last 2 days, but with no avail.So I suppose this is not a common error.
I found a solution for the issue.
I started the virtual machine, navigated to mstest.exe location -> right click -> Properties -> Compatibility tab -> Change settings for all users -> Run this program as an administrator.
It seems to me this is the most appropriate solution for now.

VSHost crash, REGDB_E_INVALIDVALUE loading Specific Project

Whenever I load a solution in Visual Studio with a specific project set as the startup project, I get a VSHost32.exe crash. If I keep on going and launch the application, I get a COMException:
{"Invalid value for registry (Exception from HRESULT: 0x80040153 (REGDB_E_INVALIDVALUE))"}
With a stacktrace:
at System.Runtime.InteropServices.RuntimeEnvironment.GetDeveloperPath()
at System.AppDomain.SetupFusionStore(AppDomainSetup info)
at System.AppDomain.SetupDomain(Boolean allowRedirects, String path, String configFile)
This can occur when a project's config file has developmentMode set, but the machine doesn't have a devPath set.
<runtime>
<developmentMode developerInstallation="true"/>
</runtime>
Removing that will fix it up.
Navigate to C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config and remove the below setting from your machine config
<developmentModedeveloperinstallation=”true”/>

VS2010 DB Project Deploy Error SQL01268 Directory Lookup Failed

My VS2010 database deploy is failing on my build server, which has SQL Server 2008 R2 installed on it. Here's the output that's relevant:
Creating Acme.Database...
D:\TeamCity\BuildAgent\Work\da2fc5ebd5805d44\src\Acme.Database\sql\Acme.Database.sql(30,0): Error SQL01268: .Net SqlClient Data Provider: Msg 5133, Level 16, State 1, Line 1 Directory lookup for the file "C:\Program Files\Microsoft SQL Server**MSSQL10.MSSQLSERVER**\MSSQL\DATA\Acme_Database_log.ldf" failed with the operating system error 3(The system cannot find the path specified.).
D:\TeamCity\BuildAgent\Work\da2fc5ebd5805d44\src\Acme.Database\sql\Acme.Database.sql(30,0): Error SQL01268: .Net SqlClient Data Provider: Msg 1802, Level 16, State 1, Line 1 CREATE DATABASE failed. Some file names listed could not be created. Check related errors.
An error occurred while the batch was being executed.
The actual path is here:
C:\Program Files\Microsoft SQL Server**MSSQL10_50.MSSQLSERVER**\MSSQL\DATA\Acme_Database.mdf
The only reference to the path I could find in the database project was in the root folder, a file called:
PRIMARY.Acme_Database.sqlfile.sql
It includes an ALTER DATABASE statement that I changed to use the _50 path, but this did not fix the issue.
I'm not sure why the db project needs to know where the actual database MDF file is located in any case - I would have thought it could just connect to it vis TSQL, but in any case I can't get the project to deploy with this error and would appreciate any help.
Turns out if you specify a "file" in Schema Objects\Database Level Objects\Storage\Files (use Add Item, File) the values specified in the path override those looked up from the Server. If you must specify them use the variables from sqlcmdvariables $(DefaultdataPath)$(Databasename) to fill things out.
Take a look at the default file path on the server itself.
Screencast: http://screencast.com/t/OWM5ODFjZj
The fact that you don't have a path specified could likely be the issue.
[Update]
The reason I suggest this is that during the creation of the deployment script, if you have not specified the settings in the Project Settings -> Deployment tab, the DefaultDataPath is looked up from the target server's path.
I don't know anything about TeamCity really so this may be a stupid question, but how are you doing the deployment? It looks like the MSBuild deploy task for a DB project can take a property that gives the data path:
http://www.codewrecks.com/blog/index.php/2009/10/06/deploy-a-database-project-with-tfs-build/
Can you set the path that way?
I had the same error, but a different solution.
For me everything was correct (as far as I can tell), but the generated deploy script set the path variables before the databasename variable - resulting in a reference to a variable that did not exist. I updated Database.sqlcmdvars to hardcode the database name value - not best practice, but a sufficient workaround for my needs.

Running a generic test after a successful build using vs and TFS 2010

I'm using team explorer under vs2010 to queue a build that is configured to run an automated test after the build. The automated test section is configured to use a vsmdi file that defines one testlist with one generic test that only opens calc.exe.
Looking in the log, after the successful build, mstest generates the following error log and calc is not running on the build agent:
Run MSTest for Metadata File
The MSTestActivity was invoked without a value for Platform or Flavor. The values Mixed Platforms and Debug were used.
C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /searchpathroot:"C:\Builds\4\CITest\CI_AUT_1\Binaries" /resultsfileroot:"C:\Builds\4\CITest\CI_AUT_1\TestResults" /testmetadata:"C:\Builds\4\CITest\CI_AUT_1\Sources\AUT1.vsmdi" /testlist:"TestList1" /publish:"http://vmftrnd70.devlab.ad:8080/tfs/QTPCollection" /publishbuild:"vstfs:///Build/Build/82" /teamproject:"CITest" /platform:"Mixed Platforms" /flavor:"Debug"
Loading C:\Builds\4\CITest\CI_AUT_1\Sources\AUT1.vsmdi...
Search path(s) for tests:
C:\Builds\4\CITest\CI_AUT_1\Binaries
C:\Builds\4\CITest\CI_AUT_1\Sources
Search path(s) for default test settings:
C:\Builds\4\CITest\CI_AUT_1\Sources
Failed to load tests from 'C:\Builds\4\CITest\CI_AUT_1\Binaries\generictest1.generictest': Microsoft.VisualStudio.TestTools.TestManagement.InvalidStorageExtensionException: File extension specified '.generictest' is not a valid test extension.
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.GetTestTypeInfosForExtension(String ext)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.GetTestTypesNotManagedInStorage(String storage)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestsFromTipsHelper(IEnumerable`1 locations, ProjectData projectData)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(IEnumerable`1 locations, ProjectData projectData, TestConflictHandler vetoingHandler)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(String location, ProjectData projectData, TestConflictHandler vetoingHandler)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestLinkStorageHelper.LoadTests(String fullStoragePath, ProjectData projectData)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.SimpleLoadTestLinkStorageHelper.Load(String fullStoragePath, ProjectData projectData)
Starting execution...
Test GenericTest1 cannot be found.
No tests to execute.
I've tried all possible ways to get the generic test to run after the build with no success...
Nothing about this on msdn\google,
Thank you for any clue you can think of.
You need full Visual Studio installed to recognise the filetype
I haven't used generic tests myself, but from the msdn documentation it looks like they have to be treated as test containers.
In your build definition, change the process to use a test container and use ***.generictest instead of ***test*.dll and see if that works.
As a note, if you are firing up a GUI tool like calc.exe, then your build server will need to be running interactively otherwise you'll have test failures.

Resources