Using NCover Community edition, I tried:
set ncover="C:\Program Files (x86)\NCover\NCover.Console.exe"
set mstest="C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe"
set testdllfolder="C:\.hudson\jobs\Project\workspace\src\TestProject\bin\Debug"
%ncover% //x coverage.xml //w %testdllfolder% %mstest% /noisolation /testcontainer:TestProject.dll
The output begins like this
Working Directory:
Assemblies:
Coverage Xml: C:.hudson\jobs\Project\workspace\src\coverage.xml
Coverage Log: Coverage.Log
Waiting for profiled application to connect...Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1
and after the MSTest output
Test Run Failed.
Passed 8
Failed 2
Inconclusive 2
Total 103
Results file: C:.hudson\jobs\Project\workspace\src\debug.trx
Test Settings: Local
Connected
Profiled process terminated. Profiler connection not established.
How to fix that?
Can you post the console output to your message? That'd tell if the session completed successfully. It could be that the coverage xml isn't showing because it's going to a directory other than the one you think it should. It's default is the current command prompt directory.
Also, you may want to try using the latest edition of NCover, v.3.4.16. You can get a 21-day trial for free at www.ncover.com/download.
Related
I am using VS2013.
I am trying to run the *.dll from the MSTest, but it fails without any execution.
If I execute the same test case manually, it always pass.
Below is the command line execution lines:
c:>"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest" /testcontainer:"E:\E_Drive\TFS_Work\PlatformSoftware\Embedded\AutomatedRe
gressionTests\Main\bin\Debug\YDOA.dll" /category:"TRLYS1D"
Microsoft (R) Test Execution Command Line Tool Version 12.0.21005.1
Copyright (c) Microsoft Corporation. All rights reserved.
Loading E:\E_Drive\TFS_Work\PlatformSoftware\Embedded\AutomatedRegressionTests\Main\bin\Debug\YDOA.dll...
Starting execution...
Results Top Level Tests
------- ---------------
Failed YDOA.InputOutputFeature.RelayAndRelayFdbkAsUsedNormalAndFALSEAndTRUE_01
Failed YDOA.InputOutputFeature.RelayAsUnusedNormalAndALLRelaysTRUE
0/2 test(s) Passed, 2 Failed
Summary
Test Run Failed.
Failed 2
Total 2
Results file: c:\TestResults\Admin_GE-6QSVBDRLU204 2015-07-08 16_10_38.trx
Test Settings: Default Test Settings
I have the reference "Microsoft.VisualStudio.QualityTools.UnitTestFramework" in my project.
mentioned is my AppConfig.
<specFlow>
<unitTestProvider name="MsTest"/>
<stepAssemblies>
In the SpecFlow - "Test Runner Tool", I have tried with all the options like : AUTO, ReSharper, VS2010MStest, VS2012, SpecRun....still I have the same problem.
I could not able to understand the problem.
Thanks,
Sai
To execute the tests using MSTest I have already copied required dlls to specific folder.
It is not giving me any exception.
Added logging to MsTest.exe.config though no logs are generated in a file mentioned MSTest.exe.config.
Command is:
C:\01Jenkins\MsTest\MSTest.exe /testcontainer:"C:\abc.dll " /resultsfile:C:abc\TestResults\testresult2.trx /category:"xyz"
Loading C:\abc.dll...
Starting execution...
It gets stuck at the above point
Any type of solution would be appreciated.
I am having a problem when trying to run a VS10 test suite through OpenCover, using mstest as my target application.
Running the tests directly with MsTest works with this command:
"C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe" /resultsfile:"<application_root_path>\UnitTestResults\MyProject.vsmdi.trx" /testmetadata:"MyProject.vsmdi" /testlist:"ServiceTests" /testlist:"DatabaseTests"
However, when trying to run the same command under OpenCover, like this:
OpenCover\Opencover.console.exe -register:user -target:"C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe" -targetargs:"/resultsfile:"<application_root_path>\UnitTestResults\MyProject.vsmdi.trx" /testmetadata:"MyProject.vsmdi" /testlist:"ServiceTests" /testlist:"DatabaseTests"" -output:<application_root_path>\UnitTestResults\Coverage
it fails, with the following error, regarding the .vsmdi.trx file (which I gather should be created by MsTest after the tests run):
Error occurred while loading document '<application_root_path>\UnitTestResults\MyProject.vsmdi.trx'.
Code: 0x800c0006
The system cannot locate the object specified.
So, basically, it complains that it cannot find the results file, before running the tests, but that file is supposed to be created at the end of the run.
Might this be an issue related to OpenCover, as the same params work when running directly with mstest?
I checked my paths and they all work out, even the one in the error is the path where the file should be generated by mstest.
Thanks.
It looks like you may need to escape your quotes when passing your data via targetargs
-targetargs:"/resultsfile:"<application...""
becomes
-targetargs:"/resultsfile:\"<application...\""
as detailed in the wiki on handling spaces.
Alternatively put your command to execute your tests in a cmd/bat file and then use opencover to execute that.
I'm using team explorer under vs2010 to queue a build that is configured to run an automated test after the build. The automated test section is configured to use a vsmdi file that defines one testlist with one generic test that only opens calc.exe.
Looking in the log, after the successful build, mstest generates the following error log and calc is not running on the build agent:
Run MSTest for Metadata File
The MSTestActivity was invoked without a value for Platform or Flavor. The values Mixed Platforms and Debug were used.
C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /searchpathroot:"C:\Builds\4\CITest\CI_AUT_1\Binaries" /resultsfileroot:"C:\Builds\4\CITest\CI_AUT_1\TestResults" /testmetadata:"C:\Builds\4\CITest\CI_AUT_1\Sources\AUT1.vsmdi" /testlist:"TestList1" /publish:"http://vmftrnd70.devlab.ad:8080/tfs/QTPCollection" /publishbuild:"vstfs:///Build/Build/82" /teamproject:"CITest" /platform:"Mixed Platforms" /flavor:"Debug"
Loading C:\Builds\4\CITest\CI_AUT_1\Sources\AUT1.vsmdi...
Search path(s) for tests:
C:\Builds\4\CITest\CI_AUT_1\Binaries
C:\Builds\4\CITest\CI_AUT_1\Sources
Search path(s) for default test settings:
C:\Builds\4\CITest\CI_AUT_1\Sources
Failed to load tests from 'C:\Builds\4\CITest\CI_AUT_1\Binaries\generictest1.generictest': Microsoft.VisualStudio.TestTools.TestManagement.InvalidStorageExtensionException: File extension specified '.generictest' is not a valid test extension.
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.GetTestTypeInfosForExtension(String ext)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.GetTestTypesNotManagedInStorage(String storage)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestsFromTipsHelper(IEnumerable`1 locations, ProjectData projectData)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(IEnumerable`1 locations, ProjectData projectData, TestConflictHandler vetoingHandler)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(String location, ProjectData projectData, TestConflictHandler vetoingHandler)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestLinkStorageHelper.LoadTests(String fullStoragePath, ProjectData projectData)
at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.SimpleLoadTestLinkStorageHelper.Load(String fullStoragePath, ProjectData projectData)
Starting execution...
Test GenericTest1 cannot be found.
No tests to execute.
I've tried all possible ways to get the generic test to run after the build with no success...
Nothing about this on msdn\google,
Thank you for any clue you can think of.
You need full Visual Studio installed to recognise the filetype
I haven't used generic tests myself, but from the msdn documentation it looks like they have to be treated as test containers.
In your build definition, change the process to use a test container and use ***.generictest instead of ***test*.dll and see if that works.
As a note, if you are firing up a GUI tool like calc.exe, then your build server will need to be running interactively otherwise you'll have test failures.
I have a weird problem while building VS2010 solution incl. unit tests on a Team Build 2008 server. It builds the solution and also can run the test without an error, but after running the test I get following warning which makes the build breaking:
MSBUILD : warning MSB6003: The
specified task executable "MSTest.exe"
could not be run. The system cannot
find the file specified
[C:\data\builds\FSuite\CI_Project[Debug]\BuildType\TFSBuild.proj]
The previous error was converted to a
warning because the task was called
with ContinueOnError=true. Build
continuing because "ContinueOnError"
on the task "TestToolsTask" is set to
"true".
It does not seem to be the problem that MSTest have not been found, I also checked if the TFSBuild.proj file is on this localtion - it is not missing. So what could it be?
Edit:
In meantime I figured out, that our old build server loaded and start the test assemblies once.
Task "TestToolsTask" Command:
C:\Program Files\Microsoft Visual
Studio 9.0\Common7\IDE\MSTest.exe
/nologo
/runconfig:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Sources\Sources\Products\FSuite.Debug.testrunconfig"
/searchpathroot:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug"
/resultsfileroot:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\TestResults"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.AppConfiguration.Console.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.Common.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.Common.UI.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.Dbms.Handler.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Base.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.BusinessObjects.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Client.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Client.UI.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Enums.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Messages.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Reporting.ReportPackage.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Reporting.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.MasterStation.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.MonitoringStation.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Pac.ProductInspection.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Pac.XmlRpcProtocol.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Server.DomainLayer.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Server.Licensing.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Server.NotificationLayer.Tests.Unit.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Tests.Unit.Ext.dll"
/testcontainer:"C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Tests.Unit.Joint.dll"
/publish:"http://ch06sd01:8080/"
/publishbuild:"V1.0_CI_FSuite[Debug]_20100728.4"
/teamproject:"FSuite" /platform:"x86"
/flavor:"Debug" The
"TestToolsTask" task is using
"MSTest.exe" from "C:\Program
Files\Microsoft Visual Studio
9.0\Common7\IDE\MSTest.exe". Loading C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Sources\Sources\Products\FSuite.Debug.testrunconfig...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.AppConfiguration.Console.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.Common.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.Common.UI.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.Dbms.Handler.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Base.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.BusinessObjects.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Client.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Client.UI.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Enums.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Messages.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Reporting.ReportPackage.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Reporting.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.MasterStation.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.MonitoringStation.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Pac.ProductInspection.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Pac.XmlRpcProtocol.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Server.DomainLayer.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Server.Licensing.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Server.NotificationLayer.Tests.Unit.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Tests.Unit.Ext.dll...
Loading
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Tests.Unit.Joint.dll...
Starting execution...
Results...
2387/2387 test(s) Passed
Summary ------- Test Run Warning.
Passed 2387
------------
Total 2387 Results file: C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\TestResults\ch06-tfsservice_CH06WL2013302
2010-07-28 11_08_03_x86_Debug.trx
Run Configuration: FSuite Debug
Testconfig
Run has the following issue(s): Code coverage instrumentation warning
while processing file
MT.FSuite.Joint.Enums.dll:
TESTTOOLSTASK : warning VSP2007:
C:\data\builds\FSuite\V1.0_CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Joint.Enums.dll
contains no instrumentable code.
TESTTOOLSTASK : warning : Test Run
deployment issue: The assembly or
module
'DevExpress.RichEdit.v10.1.Core'
directly or indirectly referenced by
the test container
'c:\data\builds\fsuite\v1.0_ci_fsuite[debug]\binaries\x86\debug\mt.fsuite.tests.unit.ext.dll'
was not found. Waiting to publish...
Publishing results of test run
ch06-tfsservice#CH06WL2013302
2010-07-28 11:08:03_x86_Debug to
http://ch06sd01:8080/Build/v1.0/PublishTestResultsBuildService2.asmx...
................................Publish
completed successfully. Done executing
task "TestToolsTask".
But the new build server with VS2010 loads all test assemblies except the MT.FSuite.Tests.Unit.Joint.dll and executes the tests, which run successfully, afterwards MSTest get called again with the mentioned assembly and failes.
Command: MSTest.exe /nologo
/runconfig:"C:\data\builds\FSuite\CI_FSuite[Debug]\Sources\Sources\Products\FSuite.Debug.testrunconfig"
/searchpathroot:"C:\data\builds\FSuite\CI_FSuite[Debug]\Binaries\x86\Debug"
/resultsfileroot:"C:\data\builds\FSuite\CI_FSuite[Debug]\TestResults"
/testcontainer:"C:\data\builds\FSuite\CI_FSuite[Debug]\Binaries\x86\Debug\MT.FSuite.Tests.Unit.Joint.dll"
/publish:"http://ch06sd01:8080/"
/publishbuild:"CI_FSuite[Debug]_20100728.2"
/teamproject:"FSuite" /platform:"x86"
/flavor:"Debug" The
"TestToolsTask" task is using
"MSTest.exe" from "MSTest.exe".
I wonder why, because the test container string did not change and furthermore the second time it does not say anymore from where it gets the MSTest.exe...
Thanks for help,
Eny
OK, I've found the source of the problem. The TeamBuild targets file (C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets) has a wrong value - replace the following:
<UsingTask TaskName="TestToolsTask"
AssemblyFile="$(MSTestRefPath)\Microsoft.VisualStudio.QualityTools.MSBuildTasks.dll"
Condition="'$(ProjectFileVersion)' == '2'" />
(the last UsingTask at line 82 if you haven't edited the file already) with:
<UsingTask TaskName="Microsoft.TeamFoundation.Build.Tasks.TestToolsTask"
AssemblyFile="C:\Program Files\Microsoft Visual Studio
10.0\Common7\IDE\PrivateAssemblies\Microsoft.TeamFoundation.Build.ProcessComponents.dll"
Condition=" '$(ProjectFileVersion)' == '2' "/>
This fixed it for me, and now my build lights are all green.