If I use MSTest with the parameter /noisolation my tests fail, but without it they pass.
If I run MSTest without the parameter for the first time a ms office configuration box appears and configures something because my tests depend on excel. this happens only the first time. then the test runs trough without problems.
but if I run my test with the parameter the box doesn't appear and tests fail.
Related
I'm trying to understand how to use visual studio online. I have a user story and it has linked to a test case.
This test case has an automated test case (which is a unit test in my test project).
The test case is:
I have it running my tests on build, but even after a successful test run the little beaker thing tool-tip on the user story shows 0 passed, 0 failed and 1 pending. I have to manually pass the test. I assume this is because I have no steps in the test. I don't want it to be a manual test though.
Is their anyway to have it run and pass my tests and have this reflected on the user story? (If I pass it manually, a little green tick appears next to the beaker).
You need to make sure you specified corresponding test plan and test suite for Run Functional Tests step/task.
You can click open test suite icon to navigate to corresponding test suite.
The MSTest allows to run tests in specify assemblies to test through /testcontainer or metalist through /testmetadata; But is there a way to automatically run all tests in a solution, whithout specifing any name of dlls/meta files? Thanks.
It's not what you want to hear, but the answer is no, by default you are limited to the /testcontainer or /testmetadata options for specifying which tests to run.
You could easily run MSTest multiple times for each DLL in your project which contains tests. You could even run a process that analyzes your DLLs for tests and build up a list of DLLs which contain tests and then run a script that calls mstest for each of those DLLs.
I have been using the Build-Deploy-Test build workflow for TFS 2010 (see here http://msdn.microsoft.com/en-us/vstudio/gg131922.aspx) and have come up against a very annoying limitation.
Here is what I have done:
Setup a build to be queued
Configured it to restore to a test-ready snapshot
Added some deployment steps; in this case starting mongo db
I then came to the step where you configure which tests to run and hit an issue. Firstly you need configure a test plan with some test suites; a test suite consisting of test cases. The problem is that each test case can only be associated with a single unit test.
With having to create a test case per unit test, it means that every time a unit test is added a new test case must be created. Is there any way you can associate many unit tests with a test suite that can be ran in the Build-Deploy-Test build workflow.
One way around this problem is to update your test suite using the tcm.exe testcase /import /syncsuite: command. This command has to be run before test execution begins and a logical place to add it would be into the build template.
I suggest you to use Test List Editor in Test Tab on Visual Studio, create your differents lists of tests, organize your tests based on functionnal after that you can execute on your build list of tests by entering name of list.
link : http://msdn.microsoft.com/en-us/library/ms182463(v=vs.100).aspx
For configuring Build Definition, i suggets you to edit & select Automated Test Tab and insert list.
link : http://support.smartbear.com/articles/testcomplete/testcomplete-and-team-build/
I'm new to load testing in Visual Studio/MSTest, and I created a new Load Test recently to validate some high-traffic scenarios for a WCF service. I want to add this to the tests project for the service, but I don't want the test to be executed whenever I "Run All Tests in Solution" nor as part of our Continuous Integration build-verification process because a) it takes 5 minutes to run, and b) the service call that it is testing generates many thousands of email messages. Basically, I'd like to do the equivalent of adding the [Ignore] attribute to a unit test so that the load test is only executed when I explicitly choose to run it.
This MSDN Article ("How to: Disable and Enable Tests") suggests that the only to disable the test is to use Test Lists (.vsmdi files), but I don't have much experience with them, they seem like a hassle to manage, I don't want to have to modify our CI Build Definition, and this blog post says that Test Lists are deprecated in VS2012. Any other ideas?
Edit: I accepted Mauricio's answer, which was to put the load tests into a separate project and maintain separate solutions, one with the load tests and one without. This enables you to run the (faster-running) unit tests during development and also include the (slower-running) load tests during build verification without using test lists.
This should not be an issue for your CI Build Definition. Why?
To run unit tests as part of your build process you need to configure the build definition to point to a test container (usually a .dll file containint your test classes and methods). Load tests do not work this way, they are defined within .loadtest files (which are just xml files) that are consumed by the MSTest engine.
If you do not make any further changes to your CI Build definition the load test will be ignored.
If you want to run the test as part of a build, then you need to configure the build definition to use the .loadtest file.
Stay away from testlists. Like you said, they are being deprecated in VS11.
Edit: The simplest way to avoid running the load test as part of Visual Studio "Run All" tests is to create a different solution for your load tests.
Why don't you want to use Test Lists. I think is the best way to do that. Create different Test Lists for each test type (unit test, load test...) and then in your MSTest command run the Test List(s) you want:
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests (only UnitTests)
MSTest \testmetadata:testlists.vsmdi \testlist:LoadTests (only LoadTests)
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests \testlist:LoadTests (UnitTests & LoadTests)
I am using the Gallio MSBuildTasks library to run MbUnit tests inside JetBrains TeamCity. When tests pass they report as having passed once, however when they fail they appear as having failed twice. Once with the Namespace and once without. This looks something like:
http://imageshack.us/photo/my-images/191/capturegv.png/
There are two tests here (Test and Test2) and they both fail. However TeamCity reckons they have each failed twice, displaying each one with a namespace (UnitTests) and then without. This is not just the report in the TeamCity website interface, I have pre-test commit enabled and they show up as failing twice in there too.
Has anyone else experienced this and if so how have you solved it?