I have a few tests that need to be fed with external data from excel files. The files are included in the test project, and in Visual Studio, I have edited the test settings file (Local.testsettings) to deploy the data files. This makes it work fine i VS.
We are, however, also running continous integration with TeamCity, and in TeamCity this doesn't work. My data files are unavailable to the test. Seems that the tests are run from a temporary folder named "C:\TeamCity\buildAgent\temp\buildTmp\ciuser_AS40VS6 2009-12-11 09_40_17\Out", and the data files are not copied there.
I have tried changing the build action for the data files to "Resource" and setting copy to output dir to "Always", but that didn't help.
Does anyone know how to make this work?
I am running Visual Studio 2010 beta 2 and TeamCity 4.5.5, which is why I'm running MSTest in the first place, and not NUnit...
I get round this by adding my data files (in my case usually XML) as embedded resources and I extract them from the test assembly.
[TestInitialize]
public void InitializeTests()
{
var asm = Assembly.GetExecutingAssembly();
this.doc = new XmlDocument();
this.doc.Load(asm.GetManifestResourceStream("TestAssembly.File.xml"));
}
This post answers this question: MSTest copy file to test run folder
The accepted answer is technically correct. However, from my experience, I find that the embedding files as resources requires an additional step of remembering to set the property "Embedded Resource". This becomes a challenge when you have a large number of data files. Also, with increasing number of data files, the size of the unit test assembly keeps growing . In my case, I had over 500MB of test data files and packing all them into the assembly was not a good idea.
What is the alternative?
Let the data files remain as they are. Do not use DeploymentItemAttribute, do not use embedded resources. Please refer my proposed solution How do I make a data file available to unit tests?
Related
My program deals with input and output files and therefore need to have an access to a folder (included in Visual Studio Project) with existing files, and also create new files and check what's inside.
Tests run on several machines, so absolute path is not an option. Also, I can not self-generate input files.
How I can tell in an NUnit test that I need a folder which is located inside project source tree? NUnit seems to place exe code in some obscure temp folder.
Edit: NUnit 3 is used
Presuming this question refers to NUnit 3, sounds like you need to use TestContext.CurrentContext.TestDirectory.
This gets the directory the assembly was built to, rather that the temp location where it is ran. (Which Environment.CurrentDirectory would return.) Documented here. I believe CurrentContext can also sometimes be null, we use TestContext.CurrentContext?.TestDirectory.
For NUnit 2.X - I believe tests run where they are build, so Environement.CurrentDirectory should be sufficient.
We’re using Xunit for testing. We’re running our tests via the built-in Visual Studio 2013 Test Runner, using the Xunit plugin.
The issue is that some of the tests need to refer to files on the filesystem. It seems that Xunit (or the VS Test Runner—not sure which), copies the assembles, but not any supporting files in the bin directory, to another directory before executing the tests, hence our test files are not found. [The MS Testing framework specifies attributes for listing files to be copied, but Xunit does not.]
How to either disable this copying behaviour, or else programmatically determine the original bin/ directory location to fetch the files?
It seems that most proposed solutions (including on the Xunit bug-tracker site) suggest storing the files as embedded resources instead of ‘copy always’ files. However, this is not always practical, for example: testing file manipulation code, and (in one case) code which wants a Sqlite database file.
Okay, typical, just as soon as I post the question, I find the answer myself…
The gist is that the copying (Shadow Copying) of assemblies seems to be done by the .NET framework, not by Visual Studio or by Xunit.
We had been using Assembly.Location to locate the assembly file, and hence the test files. However, this was wrong, since it gave us the location of the Shadow-Copied assembles instead of the originals.
Instead you should use Assembly.CodeBase to fetch the base assembly code location. However, this is a (File) URL, so it’s necessary to extract the path from the URL. The new (C#) code looks like this:
var codeBaseUrl = new Uri(Assembly.GetExecutingAssembly().CodeBase);
var codeBasePath = Uri.UnescapeDataString(codeBaseUrl.AbsolutePath);
var dirPath = Path.GetDirectoryName(codeBasePath);
return Path.Combine(dirPath, relativePath);
…where relativePath is the path relative to the Bin\ directory.
After a bit of search I found the solution here: https://msdn.microsoft.com/en-us/library/ms182475.aspx.
Particularly, the first step has been enough for me:
If they are specific to one test project, include them as content files in the Visual Studio test project. Select them in Solution Explorer and set the Copy to Output property to Copy if Newer.
associated to the following code:
var filename = "./Resources/fake.pdf"
File.OpenRead(filename)
As of .NET5 (perhaps earlier), CodeBase no longer works, so the solution is now to copy all of your files to the bin directory first and use this as your known location.
What makes this OK now, which was always a total pain in the past, is that you can copy a directory to your bin folder easily from your csproj file.
<ItemGroup>
<None
Include="TestFiles\**"
CopyToOutputDirectory="PreserveNewest"
LinkBase="TestFiles\" />
</ItemGroup>
Where TestFiles is in the root of your project folder. Now you can access these files with the following helper method.
public static class TestUtils
{
public static string GetTestPath(string relativePath)
{
var codeBaseUrl = new Uri(Assembly.GetExecutingAssembly().Location);
var codeBasePath = Uri.UnescapeDataString(codeBaseUrl.AbsolutePath);
var dirPath = Path.GetDirectoryName(codeBasePath);
return Path.Combine(dirPath, "TestFiles", relativePath);
}
}
I am running .Net Core 1.0 on Mac. Assembly.GetExecutingAssembly is unavailable. I use the following code instead:
var location = typeof(YourClassName).GetTypeInfo().Assembly.Location;
var dirPath = Path.GetDirectoryName(location);
return Path.Combine(dirPath, relativePath);
relativePath is the path relative to the directory of your DLL.
After struggling with trying to identify directories and copy files (there seem to be multiple directories in play and it's hard to pin down the right one), I found that you can just turn off shadow copying; at that point the unit tests reference content in the bin/{configuration} folder again.
To do so, follow the directions here. It seems to be as simple as setting shadowCopy to false:
{
"shadowCopy": false
}
I'm unclear whether this setting has any adverse interactions with other settings (e.g. appDomain). If anyone knows, your comments would be welcome.
I have a visual studio solution of C# projects. Currently, I do not have test projects in my solution. So I do not have file *.vsmdi.
I want to test my projects via command line like this.
MSTest.exe /testmetadata:*.vsmdi /resultsfile:Logfile.log
Is it possible to create .vsmdi without adding project? if possible, how?
To execute in command line, *.vsmdi file MUST be present in the solution?
If I have to test using IDE, the I should have *.vsmdi file in test project?
The only use of *.vsmdi file is for listing the test cases?
1) Technically it’s possible to create a .vsmdi file without adding your test project(s) to your solution. Since the content of the VSMDI files is XML you can write them by hand or generate them with a tool based on the tests in your assembly. However, VSMDI files are less useful when the test projects are not included in the solution because the power of the VSMDI files is managing your tests. But you can still use them (see also answer to question 2).
If you want to write the vsmdi file by hand you need to hash the FQN with SHA1 and create a Guid of the first 128 bits of the hash as shown below (code from Dominic Hopton). See for more information his blog post: Helpful Internals of TRX (and VSMDI files).
private static HashAlgorithm s_provider = new SHA1CryptoServiceProvider();
internal static HashAlgorithm Provider
{
get { return s_provider; }
}
/// Calculates a hash of the string and copies the first 128 bits of the hash
/// to a new Guid.
///
internal static Guid GuidFromString(string data)
{
byte[] hash = Provider.ComputeHash(
System.Text.Encoding.Unicode.GetBytes(data));
byte[] toGuid = new byte[16];
Array.Copy(hash, toGuid, 16);
return new Guid(toGuid);
}
2) No, the vsmdi file doesn’t need to be in your solution folder (or in your solution) as long as the storage attribute of the TestLink element points to the correct assembly containing the test (relative to your vsmdi file). For example:
<TestLink id="0c257cc9-ffed-cd5e-bda1-56df14ad68c2" name="AddOneAndTwo_Test"
storage="TestProject\bin\debug\testproject.dll" />
3) Vsmdi files exists on the solution level and not at project level. If you have more than one test projects you can use one vsmdi file to manage all your tests. Please note that the storage attribute is ignored when you run the tests from Visual Studio assuming that you loaded the projects in your solution
4) Vsmdi files are used for managing the tests in your solution and you can create a hierarchy of several layers within the VSMDI file. For example you have more than one assembly in your solution you want to test. Under the root test list (called: List Of Tests) you create a test list for each of your projects. You can then divide the test list for one of the projects into two or more separate test lists (e.g. one for testing the business logic and one for testing the error handling). When you only want to test the business logic tests from project A you can select the corresponding test list and run the tests in it.
An extra note about vsmdi files:
A) Since Visual Studio 2012 Microsoft decided to make VSMDI files (and with them the test lists) deprecated. Test lists got replaced by Test Categories (already exists in Visual Studio 2010) and Playlists.
Test categories are attributes you can set on individual test methods and can be used in builds. Playlists were introduced in Visual Studio 2012 Update 2 and are currently not supported in MSBuild.
For more information about Test Categories see: http://msdn.microsoft.com/en-us/library/dd286595.aspx
I'm using Visual Studio 2008 with Microsoft test tools. I need to access a text file from within the unit test.
I've already configured the file with build action set to 'Content' and copy to output directory to 'Copy Always', but the file is not being copied to the output directory, which according to System.Environment.CurrentDirectory is
'{project_path}\TestResults\Pablo_COMPU 2009-11-26 15_01_23\Out'
This folder contains all the DLL dependencies of the project, but my text file is not there.
Which is the correct way to access a text file from a unit test?
You have to add the DeploymentItem attribute to your test class. With this attribute you specify the files which are copied into the out directory for the test run.
For example:
[TestMethod]
[DeploymentItem(#"myfile.txt", "optionalOutFolder")]
public void MyTest()
{
...
}
See also: http://msdn.microsoft.com/en-us/library/ms182475.aspx.
Alternatively if you set all your text files to "Copy to build directory" then you could reference their path in your tests by doing this
var directory = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
var path = System.IO.Path.Combine(directory, "myFile.txt");
When I need a chunk of text as part of a unit test and it's more than a line or two, I use an embedded resource. It doesn't clutter your test code, because it's a separate text file in the source code. It gets compiled right into the assembly, so you don't have to worry about copying around a separate file after compilation. Your object under test can accept a TextReader, and you pass in the StreamReader that you get from loading the embedded resource.
I can't answer your question as I don't use MSTest. However, I'd consider whether accessing the file system in a unit test is the right thing to do. If you introduce a dependency on the file system, the test will become slower and less trustworthy (you now depend on something that may not be there/accessible/etc). It is for these reasons that many folk will say "it's not a unit test if it hits the file system".
Although this is not a hard rule, it's always worth considering. Any time I have to touch the file system in tests, I try to avoid it because I find tests that rely on files are harder to maintain and are generally less consistent.
I'd consider abstracting the file operations to some degree. You can do numerous things here, from changing the internal loading strategy (via Dependency Injection) to -- even better -- separating the loading/use of the file so that the consumer of the file's contents doesn't even have to care about the loading strategy.
How are you running your tests?
We use (TestDriven.net -> Run Tests).
From my experience, some test runners (like Junit in Netbeans) won't automatically copy any additional text files that you might need for testing. So in your case you might have to do a full build, and then try running the tests again.
And the correct way for accessing text files from tests is the way you're trying to do it. (Setting the files to "copy always" and "content", and accessing them from the compiled directory).
Also, not sure where people are getting the idea that having tests rely on files is a bad thing. It's not.
If anything, having separate test files, will only clean up your tests and make them more readable. Consider some xml parsing method that returns a large string:
String expectedOutput = fileOperator.ReadStringFromFile("expectedFileContents.xml");
String result = systemUnderTest.Parse(somethingtoparse);
Assert.Equals(expectedOutput, result);
Imagine if the output was 30 lines long, would you clutter your test code with one giant String? or just read it from a file.
I have data files used as input to my unit tests. These files are quite big and I don't want to copy them each time unit tests are executed. Tests are executed without deployment. So I can just put them into folder under my solution, and... how to obtain path to my solution (or test project source code) when unit test is executing?
Because you can run a test project in different ways (TD.NET, Visual Studio, R# etc.), the path used to reference the tests can change.
For this reason, I embed test needed files in my test assembly and draw them out from there.
You can use:
Assembly.GetExecutingAssembly().Location
in your tests to get the path of the assembly containing the unit tests.
Simple, make the location of the files configurable (and testable).
Then either, set it in the unit testing code or set it thru some config file.