I have a XUnit test project. I am planning to use .runsettings file and I need to specify the TEST NAMES which I have to run in .runsettings file.
I am going to run the test in commandline using "dotnet vstest" command and specifying the test dll and settings file. I am using runsettings for the first time.
My issue is I dont have idea about how to specify test names to be run in runsettings file. Can anybody please help me on this?
Related
I'm having a strange problem. I have a bunch of integration tests projects. Each test project has a configuration file (app.config).
Currently, in my "Run Integration Tests" build step, the NUnit3 console runner allows me to specify only one config file (where it says "Path to application configuration file"). I have specified the path to one of my integration test app.configs. The problem is, the tests in other test projects fail because the Nunit3 console doesn't recognize their app.config files.
How can I have TeamCity run each test project with its own app.config file in its bin/release folder?
FYI: This is what I have specified in the NUnit assembly lookup: **\bin\Release\*.Integration.Tests.dll
Turns out, this was my problem. After removing all references between my test projects, I noticed that the build time was reduced by 50%.
I am running the build command like this:
set MSBuildParams=/m:16 /target:Rebuild
/property:Configuration=""Release"";Platform=""x64"" msbuild
%MSBuildParams% C:\path\to\the\sln\Solution.sln
I'm using the following msbuild version:
Microsoft (R) Build Engine version 14.0.25420.1
Solution.sln contains 2 projects,
let's say Project1 and Project2.
Project1 is set as the startup project.
After I build the Solution.sln using this setup, I would like to
rebuild it, but this time using Project2 as the startup project.
Is there a way to do that, without changing the Solution.sln?
What would be the best practice to accomplish that?
You could use the specific command line to build/rebuild the specific project using MSbuild like the following case.
specify project file of a solution using msbuild
But we have to change certain files if you want to change the start up project without using the VS IDE, since the setting was stored into the ".SUO" file.
Actually there is no setting in ".SLN" file for startup project even if you don't want to change it,
In addition, the start up project was the running project of the solution, maybe you don't have to change it:
https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/b6347dce-8449-4cbb-a606-7b19407a1026/how-do-i-set-the-startup-project-in-the-sln-file?forum=vcgeneral
MSTest uses in CI environment to run set of unit tests. These set of unit test need custom binaries to run. I'm not sure how can pass the custom assembly path to the MSTest.exe when I run the unit test using command line for debugging purpose. When it runs via TFS there is a place to configure custom assembly path in build controller as this image.
So, how to run MSTest with custom assembly path using command-line
I wrote a Nant script that executes MSBUILD.exe to compile a project on my dev machine. On my dev machine, the projects builds its output to bin\x86\Release and my Nant script zips up the contents of that directory. I then commit everything to SVN and let TeamCity run the Nant script that executes MSBUILD.exe to compile the project and zip the output, but the output is created in bin\Release and the zip file is empty because it looks in bin\x86\Release. Why does this happen?
When I make changes to the configuration and platform in VS.NET 2008, I do not see the project file light up as being changed. Are these settings stored in the project file, solution file, or user configuration file and therefore not carried over to the build server?
Quick fix: You can use the flag /property:OutDir=bin\x86\Release
You would have to find root cause of this. Probably configuration is incorrect. You can change configuration explicitly to something like /p:Configuration=Release
Are you sure ${project.config} point to the same place in both local and TeamCity environments?
The agent are not always running with the sem environment variables as the local machine. So I would start out to check all properties and see where they point in the local machine as well as TeamCity.
As for the /p:Configuration=${project.config}, you can only have one configuration running but you can specify more properties with ; between them:
/p:Configuration=${project.config};OutDir=bin\x86\Release
I've got a test which requires an XML file to be read in and then parsed. How can I have this file copied into the test run folder each time?
The XML file is set to "Copy if newer" and a compile mode of "none" (since it's not really a compile-able thing)
use a DeploymentItem attribute
using System;
using System.IO;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using CarMaker;
namespace DeploymentTest
{
[TestClass]
public class UnitTest1
{
[TestMethod()]
[DeploymentItem("testFile1.xml")]
public void ConstructorTest()
{
string file = "testFile1.xml";
Assert.IsTrue(File.Exists(file), "deployment failed: " + file +
" did not get deployed");
}
}
}
It seems that if you provide a TestSettings file for the Solution then you can uncheck the "Enable deployment" option and stop mstest from trying to run from the ...TestResults\...\out folder where it doesn't copy your extra files (unless you make them a deployment option).
This is also useful if you depend on the extra files being in a preserved folder structure because Deployment items all seem to be copied directly (flat) into the temporary run folder (out) if you use the Deployment, Add Folder option in the TestSettings (answers above suggest you can keep the structure if you add each item as its own DeploymentItem).
For me it worked fine running tests directly in Visual Studio (i.e. my extra files in their structure were found and used by tests) because I had created a TestSettings file for another reason long ago (which has Enable deployment unchecked), but not when TeamCity ran mstest to run tests because I hadn't specified that the TestSettings file should be used.
To create a TestSettings file in Visual Studio, right click on the Solution and choose New Item, and select the TestSettings template. To use the TestSettings file at the command prompt of mstest.exe add the option, /testsettings:C:\Src\mySolution\myProject\local.testsettings (or add as an extra command line option in TeamCity with appropriate path)
The Preet answer is used to deploy items for a single test. If you want to do it at solution level, use the .testrunconfig settings.
Best solution to me is using testsettings, especially if multiple tests need the same datafiles.
First create a testsettings file, and add the deployment items you need (file or folder name):
<TestSettings name="Local" id="00ebe0c6-7b64-49c0-80a5-09796270f111" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Description>These are default test settings for a local test run.</Description>
<Deployment>
<DeploymentItem filename="Folder1\TestScripts\test.xml" outputDirectory="TestScripts"/>
<DeploymentItem filename="Folder2\TestData\" outputDirectory="TestData"/>
</Deployment>
<...../>
Running in visual studio, use "select Test Settings File" from "Test\Test Settings" menu to select new testsettings
Running mstest, use the /testsettings parameter to have mstest use your testsettings.
You can define DeploymentItem in a class that holds a method with AssemblyInitialize attribute. Then you're sure that the files are copied regardless of which test you run.
Unfortunately DeploymentItem attribute is executed only on classes which contain tests you're running. So if you have 10 test classes which use the same set of files, you have to add the attribute to all of them.
Also found out that changes in *.testsettings files are not automatically refreshed in Visual Studio. Therefore after adding files / folders into deployment in testsettings, you have to reopen solution file and then run the tests.
In Visual Studio 2012, vstest.console.exe (the built-in test runner) runs with the output dir as the current path. This means that you only need to include the items in your solution with the 'Copy always' or 'Copy if newer' property for them to be used by your test. You don't need the DeploymentItem attribute for the general case. The same applies when running vstest.console.exe from the command line inside your output/test directory.
There are some cases where a separate folder is used, one of them being when you are using the DeploymentItem attribute. See here for more information.