I work with Coded UI Tests and Visual Studio 2013. Now I have to test if images from a folder are shown correctly inside the application. That’s why I have created a folder containing the images and set build action to none and deployment to always.
Unfortunately all tests are executed in an own test results folder and my images are not deployed correctly. I know I could do this by using the DeploymentItem attribute or a testsettings file but I don't want to do this. I want to avoid the "test results" folder and run the tests from within the output folder of my test project.
I do this with the unit tests for instance. They are written with XUnit.Net which works just fine. I thought it would also work with MS Test but this only seems to work for Unit Tests but not for Coded UI Tests.
So, to sum it up again: How can I get rid of the “Test Results” folder when using Coded UI Tests with Visual Studio 2013 and run my tests simply from within the output directory of the project?
You can do this using a custom entry in the .runsettings file I think.
DeploymentItemAttribute Class
Consider running your unit tests directly in the build output
directory, so that testing runs more rapidly. This is especially
useful on the build server after you have checked in your tests. To do
this, add a .runsettings file to your solution, include
False, and select the file in
the Test, Test Settings menu. The same effect occurs in any test run
in which DeploymentItemAttribute is not used at all.
However, you
might prefer not to do this if you want to be able to inspect the data
files after a failed run.
You cannot avoid using a deployment folder
if you are using a .testsettings file, which is required for web and
load tests, coded UI tests, and any test in which you deploy an
application to remote machines.
As for DeploymentItemAttribue itself I would steer clear of it unless the following issues have been fixed, Gotchas: MSTest’s [DeploymentItem] attribute. +1 for xUnit in TestDriven.NET.
You can try deleting the LocalTestRun.testrunconfig file from the Solution Items folder (directly under the solution). When we did that it started using the \bin\debug\ folder instead of the TestResults folder when running our unit tests using MSTest.
Related
I'm new to Coded UI testing and TFS. I've written a coded UI test for my web application and looking for different options to automatically run it. Apparently the first option is to run it from visual studio. I have also found an article explaining how to associate a test method with a test case however I don't know whether I can run my test method (s) from TFS or not? How about running them with my build? I know the unit tests can be run with the build however I'm not sure about coded UI tests.
Yes, it is possible for you to run Coded UI test in the TFS build process.
Firstly, you need to configure the Team Build Service Host run as an interactive process. See this link.
Secondly, add the Coded UI test project into TFS Version Control.
Thirdly, create a build definition to build the Coded UI test project, and also specify tests to run.
Please have a check of this blog for the detailed steps: http://blogs.msdn.com/b/mathew_aniyan/archive/2009/05/26/coded-ui-test-in-a-team-build.aspx
I am developing automated tests in Visual Studio 2010 (Ultimate). I can execute my test locally trough VS and remotely on a test agent through MS Testing Center. Everything works well.
The problem for me now is that I want to deploy test data with my tests. This test data is ordinary image files that is to be processes by the application I am testing. Deploying these files works fine as long as I'm running the test locally. I can either use the "Deployment" section in my Local.testsettings file, or I can use the DeploymentItems attribute in my code. Of course - the files that I want to deploy are included in my project file.
However, when I execute my test through the test agent - it will not deploy any files. And as I said, the test itself works fine both locally and remote.
I have been facing the same problem recently.
It sounds like you have the DeploymentTest attribute like this:
<DeploymentItem #"MyProject.Tests\TestDataSubfolder>\", "OutputDir">
Just change it to:
<DeploymentItem #"TestDataSubfolder>\", "OutputDir">
(That means remove the project folder, since the project folder does not exit on the remote machine).
I'm new to load testing in Visual Studio/MSTest, and I created a new Load Test recently to validate some high-traffic scenarios for a WCF service. I want to add this to the tests project for the service, but I don't want the test to be executed whenever I "Run All Tests in Solution" nor as part of our Continuous Integration build-verification process because a) it takes 5 minutes to run, and b) the service call that it is testing generates many thousands of email messages. Basically, I'd like to do the equivalent of adding the [Ignore] attribute to a unit test so that the load test is only executed when I explicitly choose to run it.
This MSDN Article ("How to: Disable and Enable Tests") suggests that the only to disable the test is to use Test Lists (.vsmdi files), but I don't have much experience with them, they seem like a hassle to manage, I don't want to have to modify our CI Build Definition, and this blog post says that Test Lists are deprecated in VS2012. Any other ideas?
Edit: I accepted Mauricio's answer, which was to put the load tests into a separate project and maintain separate solutions, one with the load tests and one without. This enables you to run the (faster-running) unit tests during development and also include the (slower-running) load tests during build verification without using test lists.
This should not be an issue for your CI Build Definition. Why?
To run unit tests as part of your build process you need to configure the build definition to point to a test container (usually a .dll file containint your test classes and methods). Load tests do not work this way, they are defined within .loadtest files (which are just xml files) that are consumed by the MSTest engine.
If you do not make any further changes to your CI Build definition the load test will be ignored.
If you want to run the test as part of a build, then you need to configure the build definition to use the .loadtest file.
Stay away from testlists. Like you said, they are being deprecated in VS11.
Edit: The simplest way to avoid running the load test as part of Visual Studio "Run All" tests is to create a different solution for your load tests.
Why don't you want to use Test Lists. I think is the best way to do that. Create different Test Lists for each test type (unit test, load test...) and then in your MSTest command run the Test List(s) you want:
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests (only UnitTests)
MSTest \testmetadata:testlists.vsmdi \testlist:LoadTests (only LoadTests)
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests \testlist:LoadTests (UnitTests & LoadTests)
I have two unit test projects in my VS 2010 solution. Each project has a Data directory with input data needed for the unit tests.
I annotated the test classes that need the data with
[DeploymentItem("Data")]
When I run tests individually, the run fine. However, when I run (or debug) all tests in the solution, I find that only one of the two Data directories is copied to TestResults\MyTestDir-YYYY-MM-DD HH_mm_SS\Out, causing unit tests that rely on the other data directory to fail.
Interestingly if I then use the Test Results window to re-run checked (as in failed) tests, the tests still fail (they do not recognize that the correct Data directory's files are missing). If I then navigate directly to a failed test and select Run Tests in Current Context, the test run gets the correct Data directory and the test succeeds.
My question: How can I cause DeploymentItems from two separate test projects all to be copied to the Out directory where the tests are executed?
For reference, I have reviewed these questions without noting a solution
Problems with DeploymentItem attribute
Do MSTest deployment items only work when present in the project test settings file?
I found that giving each "Data" directory a unique name (e.g. "TestProjectAData") resolved the issue. There seems to be some sort of bug when multiple directories in different paths all have the same relative path to different test projects (i.e. if every test project has a subdirectory called "Data").
I am working with an n-tiered architecture in Visual Studio 2008 (Developer Edition), and I have run into an issue.
We are running unit tests on every method of our services layer, and I am attempting to see the code coverage results, to ensure I'm hitting all the main paths through my methods.
When I attempt to view the results, I am informed that the test run did not have code coverage enabled. I tried editing the test run configuration through test -> edit test run configurations, only to discover that there were none found.
I have yet to discover how to create a test run configuration so that I can enable code coverage results.
How do I create the configuration?
Right-click the solution, choose Add, New item, choose Test run configuration in the resulting dialog. The only tricky part is that you don't get this choice unless you right-click the solution.