I want to add multiple Ms unit test project in single solution in VS 2010 and want to execute all projects test case in single shot in Bamboo CI.
In Bamboo i am able to test Single project by using project dll but not able to execute Test written in multiple project .
I tried on google and found .vsmdi file will do that functionality but not crack how to all test cases add in to vsmdi file .
Please help me.
The Vsmdi File is look like this :
<?xml version="1.0" encoding="UTF-8"?>
<TestLists xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<TestList name="Lists of Tests" id="8c43106b-9dc1-4907-a29f-aa66a61bf5b6">
<RunConfiguration id="68cd1a40-fbe7-4a52-bc91-47f0ca5ea70a" name="Local" storage="local.testsettings" type="Microsoft.VisualStudio.TestTools.Common.TestRunConfiguration, Microsoft.VisualStudio.QualityTools.Common, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</TestList>
</TestLists>
Related
I developed a single plugin for on-premise crm 2013 (PreCreateOpportunityProduct - nothing special). In my visual studio crm solution I added a project (called BusinessLogic) that includes my generated Entities.cs as well as partial classes for each entity. Therefore I can encapsulate the logic for one special entitiy in one separate class. No big deal so far.
Now I want to use those logic, therefore I add the project reference in my plugin project. I know that I need to merge all needed DLL's to one, so I added a post-build event to merge all dll's with ILMerge.
Overview of Solution / Project with used DLLs:
The post-build command looks as follows:
mkdir "$(TargetDir)Merged"
"$(SolutionDir)Libs/ilmerge.exe" /keyfile:"$(ProjectDir)keypair.snk" /target:library /copyattrs /targetplatform:v4,"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0" /out:"$(TargetDir)Merged\Flag.Plugins.dll" "$(TargetDir)Flag.Plugins.dll" "$(TargetDir)BusinessLogic.dll" "$(TargetDir)Microsoft.Crm.Sdk.Proxy.dll" "$(TargetDir)Microsoft.Xrm.Sdk.dll"
del "$(TargetDir)*.*" /Q
move "$(TargetDir)Merged\*.*" "$(TargetDir)"
The RegisterFile.crmregister:
<?xml version="1.0" encoding="utf-8"?>
<Register xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.microsoft.com/crm/2011/tools/pluginregistration">
<Solutions>
<Solution Assembly="Flag.Plugins.dll" Id="f4dff197-5b7a-e411-80c3-005056ba5a51" IsolationMode="None" SourceType="Database">
<PluginTypes>
<Plugin Description="Plug-in to PreOpportunityProductCreate" FriendlyName="PreOpportunityProductCreate" Name="Flag.Plugins.PreOpportunityProductCreate" Id="0f7bd0bc-2b7b-e411-80c3-005056ba5a51" TypeName="Flag.Plugins.PreOpportunityProductCreate">
<Steps>
<clear />
<Step CustomConfiguration="" Name="PreOpportunityProductCreate" Description="Pre-Operation of Opportunity Product Create" Id="107bd0bc-2b7b-e411-80c3-005056ba5a51" MessageName="Create" Mode="Synchronous" PrimaryEntityName="opportunityproduct" Rank="1" SecureConfiguration="" Stage="PreInsideTransaction" SupportedDeployment="ServerOnly">
<Images />
</Step>
</Steps>
</Plugin>
</PluginTypes>
</Solution>
</Solutions>
<XamlWorkflows />
</Register>
Note: I am Deployment Manager + System administrator -> all permissions
When I click Deploy on CrmPackage project, the error
Error registering plugins and/or workflows. Plug-in assembly does not contain the required types or assembly content cannot be updated. C:\Program Files (x86)\MSBuild\Microsoft\CRM\Microsoft.CrmDeveloperTools.CrmClient.targets
Note 2: When I deploy the plugin.dll without merging all the other DLLs the deployment works like charm, but the plugin throw the exception because the BusinessLogic.dll is not known.
Can anyone help me with this issue?
Thanks in advance
I found the problem: I was merging too many files! In detail: It was the Microsoft.Crm.Sdk.Proxy.dll and Microsoft.Xrm.Sdk.dll. Those libs are already in the crm server GAC. After removing those files from my ilmerge command, the deployment finally works.
Here is the link which gave me the important hints:
https://community.dynamics.com/crm/f/117/p/146347/326928.aspx#326928
I did quite a lot of research on the web and tried a few settings, but I couldn't reproduce the behavior of running MsTest in Visual Studio 2012 on the command line.
Our solution consists of many projects that build to the same bin folder residing at the solution level (e.g. C:\MySolution\bin) - this is the code-under-test (CUT). The tests are grouped in a separate project that resides in its own solution and builds in its own bin folder (e.g. C:\MySolution\Tests\bin). There are really a lot of plugins thus we want MsTest to reference the CUT bin folder when running the test intead of copying everything to the TestResults folder. We did achieve this in Visual Studio 2012 by editing the .testrunconfig and specifying ".\bin" as "root folder for the assemblies to be loaded" (in tab "Unit Test" when editing the testrunconfig). So we can load the test solution in VS2012 and run the tests there without having to copy the bin folder contents to the TestResults directory.
Now I wanted to create a .bat file that would run MsTest the same way as in VS2012 so that we can omit launching Visual Studio just for running the tests. I'm now working on how to execute MsTest on the command line, but have been quite frustrated. This is what I tried (command executed on the solution level in a VS command prompt):
MsTest /testcontainer:Tests\bin\Tests.dll
This did not work at all, it couldn't even find the referenced dlls that Tests.dll need to run. So I re-used the configuration and ran it again:
MsTest /runconfig:LocalTestRun.testrunconfig /testcontainer:Tests\bin\Tests.dll
Still it did not work. It could start the tests, but all of them failed. I got a lot of warnings of the kind
Warning: Test Run deployment issue: The assembly or module '....' directly or indirectly referenced by the test container 'C:\MySolution\Tests\bin\tests.dll' was not found.
and in the end it said:
The configured application base directory 'C:\MySolution\TestResults\User_Machine 2013-07-28 13_16_59\Out\bin' does not exist. The test directory will be used instead.
When I changed the option applicationBaseDirectory in the testrunconfig to an absolute path (C:\MySolution\bin), it worked. Still I get many warnings such as:
Warning: Test Run deployment issue: The assembly or module '....' directly or indirectly referenced by the test container 'C:\MySolution\Tests\bin\tests.dll' was not found.
But anyway, it's not really a feasible solution to specify an absolute path. How can I run MsTest on the command line with a different, but relative assembly base directory?
My LocalTestRun.testrunconfig is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local Test Run" id="...." xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Description>This is a default test run configuration for a local test run.</Description>
<Deployment>
<DeploymentItem filename="Tests\....\Resources\" />
</Deployment>
<Execution hostProcessPlatform="MSIL">
<TestTypeSpecific>
<UnitTestRunConfig testTypeId="....">
<AssemblyResolution applicationBaseDirectory=".\bin">
<TestDirectory useLoadContext="true" />
</AssemblyResolution>
</UnitTestRunConfig>
<WebTestRunConfiguration testTypeId="....">
<Browser name="Internet Explorer 7.0">
<Headers>
<Header name="User-Agent" value="Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" />
<Header name="Accept" value="*/*" />
<Header name="Accept-Language" value="{{$IEAcceptLanguage}}" />
<Header name="Accept-Encoding" value="GZIP" />
</Headers>
</Browser>
</WebTestRunConfiguration>
</TestTypeSpecific>
<AgentRule name="LocalMachineDefaultRole">
</AgentRule>
</Execution>
</TestSettings>
After more searching we changed to use the test console runner that is included with VS2012:
VSTest.Console.exe Tests\bin\Tests.dll /Framework:framework40 /Settings:LocalTestRun.testrunconfig
This works with a relative path as applicationBaseDirectory.
This is due to an MSTest bug that sets the current directory to its own working directory, rather than the test project's bin (or deployment) folder. The workaround is to execute the following code in the constructor of your test class:
Environment.CurrentDirectory(AppDomain.CurrentDomain.BaseDirectory);
I got the idea from http://www.ademiller.com/blogs/tech/2008/01/gotchas-mstest-appdomain-changes-in-vs-2008/; however, note that, in my case at least, it required setting Environment.CurrentDirectory, rather than the reverse, as suggested in the article.
I started using Ordered tests recently in VS 2010. I have created a folder under my test-suite and dragged the tests (required tests in a flow) from test list editor to the ordered test list. I am able to run the ordered test from VS successfully. But when I run the ordered test using MSTest, ordered test gets error (all of its tests gets not executed). When I opened the Ordered test file in notepad, I see everything properly configured like storage path as shown below
*<?xml version="1.0" encoding="UTF-8"?>
<OrderedTest name="MyOrderedTest" storage="OrderedTests\MyOrderedTest.orderedtest" id="afadbaf6-7915-426d-932c-788fc27c7a8f" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<TestLinks>
<TestLink id="c2367ab7-ab71-bdaf-d039-4d55ea87e962" name="TestOne" storage="TestSuiteName.dll" type="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
<TestLink id="cc156aeb-ed8e-12d6-c9ad-24fadc2ca54b" name="TestTwo" storage="TestSuiteName.dll" type="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
<TestLink id="caa09202-b582-e53c-cae9-7fe8b402b4a2" name="TestThree" storage="TestSuiteName.dll" type="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</TestLinks>
</OrderedTest>*
When I ran a single test 'TestOne' using MSTest, it runs successfully.
Is there anyother thing that I need to configure/change??
Thanks in Advance.
Sham_
Did I understand you correctly: you have stored your ordered test in a sub-folder?
If so, just move your ordered test in to the root folder (do not use a sub-folder for storing them).
I had been facing the same problem using ordered tests for test automation. When running them from Microsoft Test Manager I got the error
"Cannot find the test ‘XXX’ with storage ‘..\bin\debug\YYY.dll"
According to this message Test Agent (the component that runs the tests) is looking for them in the root folder and not in a sub-folder I've stored them in.
Moving the ordered tests to the root folder of my solution solved the problem.
It seems to be a bug.
PS: You can check in the Output window why the tests were not executed.
Even with 2012 version this issue exists . Moving the ordered test to root folder solved the issue for me as well.
I've turned on Code Analysis for all of my projects, and have verified code analysis is on for each configuration (Debug, Release, QA, etc). My local builds are of course fine, my TFS builds are failing still.
Here is the failed output XML file in the drop location for my builds:
<?xml version="1.0" encoding="utf-8"?>
<Tests xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<BuildConfigurationSettings name="Debug" ProjectName="MyProject" Platform="AnyCPU">
<Test>
<MSBuildSchemaName>CodeAnalysisRuleSet</MSBuildSchemaName>
<ExpectedValue>MinimumRecommendedRules.ruleset</ExpectedValue>
<ActualValue />
<Verify>true</Verify>
<TestExecuted>true</TestExecuted>
<IsEqual>false</IsEqual>
</Test>
</BuildConfigurationSettings>
<BuildConfigurationSettings name="Release" ProjectName="MyProject" Platform="AnyCPU">
<Test>
<MSBuildSchemaName>CodeAnalysisRuleSet</MSBuildSchemaName>
<ExpectedValue>MinimumRecommendedRules.ruleset</ExpectedValue>
<ActualValue />
<Verify>true</Verify>
<TestExecuted>true</TestExecuted>
<IsEqual>false</IsEqual>
</Test>
</BuildConfigurationSettings>
</Tests>
For some reason, even though Code Analysis was turned on in the 'Properties' for the failing projects, the changes were not being reflected in the .csproj file. After hand-editing the .csproj file to include the minimum ruleset, everything was working fine:
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
Visual Studio edition on build server should support for code analysis And there are some known issues:
Unfortunately due to a bug discovered very late in the release cycle
the install for Code Analysis does not execute on x64 build servers.
This means you won’t see code analysis running despite correct
configuration. The work around for now is to install a Visual Studio
SKU that includes Code Analysis on the build machine to get the FxCop
installation.
I have integrated MSTest with cruise control for single project by following the steps present in the below link.Its working fine
http://www.codeproject.com/KB/tips/VSTS2008_Tests_With_CCNET.aspx?display=Print.
But if i add multiple projects, the result file is getting over wrtitten by 2nd project result. I am not able to see the first project result.
Please let me know how i can display multiple project results in result fie.
... in the meantime I checked the problem in more detail and it seems that my 1st guess was right (although I didn't realize that no real build script but a simple batch is used):
Problem: A single working directory is used for all projects. Each project produces it's own results.xml file but since they are all stored in the same location results are overwritten.
Solution: Use a separate folder for the results of each project (e.g. the project's artifact directory which is passed from CruiseControl to the batch process as environment variable).
RunTests.bat:
del "%CCNetArtifactDirectory%\results.xml"
MSTest.exe /testcontainer:<PathtoTestProject>\Bin\Debug\TestAssembly.dll /resultsfile:"%CCNetArtifactDirectory%\results.xml"
ccnet.config:
<project name="ProjectA">
<cb:define projectArtifactDirectory="C:\path\to\data\of\ProjectA" />
...
<artifactDirectory>$(projectArtifactDirectory)</artifactDirectory>
...
<publishers>
<merge>
<files>
<file>$(projectArtifactDirectory)\results.xml</file>
</files>
</merge>
</publishers>
...
</project>