I am forcing myself to learn test-driven development, and so far I'm enjoying myself. There's a few quirks that Visual Studio Unit Testing has that is driving me batty though. A bit of background information, my project folder looks like this:
[Root] BitFlex
BitFlex\Code
BitFlex\Debug
BitFlex\Documents
BitFlex\Release
Now of course all the source code is stored in the code folder, and on a build the project output either goes to the debug or release folders depending on the current configuration. Now for my unit testing, I have it setup so the test project is output to either:
BitFlex\Debug\Unit Tests\
BitFlex\Release\Unit Tests\
1) At this point, everything is fine and dandy. There are two problems with this, the first being that when I run a test it cannot find the assembly, as it gives me this error:
Error AssignDefaultProgramTest BitFlex.UnitTests The test assembly 'D:\src\DCOM Productions\BitFlex\Code\TestResults\David Anderson_DCOMPRODUCTIONS 2009-07-31 23_21_00\Out\BitFlex.UnitTests.dll' cannot be loaded. Error details: Could not find file 'D:\src\DCOM Productions\BitFlex\Code\TestResults\David Anderson_DCOMPRODUCTIONS 2009-07-31 23_21_00\Out\BitFlex.UnitTests.dll'.
I cannot seem to find information on this error, or how to resolve it so I suppose that's where everyone's experise around here comes in to play.
2) My other beef is that Visual Studio generates the "Test Results" folder in my code directory, I would prefer to move that to my Unit tests folder in either output configuration. Is there a way to do this, or a better practice to setting up a well organized Unit Test using my folder hierarchy?
By default MSTesting framework runs all tests in an 'isolated' location and not from the binaries directory.
To fix this you can do one of these two:
1. go to the test configuration file and under deployment uncheck deploy the tests.
2. don't use path when looking for external files instead use deploy attribute or the test config to deploy the needed files along with your tests.
For doing TDD with MSTest, turn off deployment. You shouldn't need it for "unit testing."
Also, never ever EVER have VS automatically generate tests for you. What's generated may be fine for some types of functional testing, but are usually very poor unit tests.
Related
We noticed that on a certain dev machine a Visual Studio (2015 update 3) debug build of a C# solution was generating a $RANDOM_SEED$ file alongside every built DLL.
The content of the file is just a single number e.g.
1443972318
Deleting the file(s) then rebuilding resulted in the file being regenerated, with a different number.
This behaviour was also observed when rebuilding a single project in the solution (one which has only the standard C# project refs/dependencies + System.Management).
Note that running a command line build e.g.
msbuild <sln-file>
did not regenerate the file (for build of complete solution or single project).
After a restart of VS, the file is no longer regenerated.
As far as we know this file name is not used in any of our source code, post build steps or internal dependencies.
There are quite a few dependencies on .NET framework classes, including Random and RNGCryptoServiceProvider, and also external dependencies. We don't have complete source code for all these so it's not possible to check exhaustively which if any of the dependencies are responsible.
This is a bit of a shot in the dark but the question is has anyone seen anything similar to this?
EDIT
I'm not surprised this has been downvoted - I appreciate it is pretty open ended, but as I'm currently not able to reproduce this and as it could have potentially serious consequences (random number generator attack?) I have posted it anyway. If I am able to repro I will of course update here.
I have the same file.
After a short investigation I found guilty:
this file is created by NUnit 3.x test adapter.
(You can check it in AdapterSettings.cs from NUnit adapter source code).
The file is used by NUnit to ensure that we use the same random seed value for generating random test cases in both the discovery and execution processes. This is required because the IDE uses two different processes to execute the adapter. It's not actually required (or created) when running the adapter under vstest.console.exe.
I have a visual studio solution that contains around 18 projects. I want to write unit tests for those projects (by creating a test-project that contains unit tests against each source project).
Should I use a separate solution that contains all the test-projects? Or should I use Partitioned solution approach of Visual Studio 2008 and create a sub-solution for all test-projects?
To put unit tests in a separate solution would seem to me to create problems. Inherently, unit tests must be able to reference the types (classes) they are testing. If in a separate solution then it implies a binary association. That would seem very 'difficult'.
So, to investigate the reasons for the question, and hopefully provide some help, these are the reasons I would put my unit tests in the same solution:
Project referencing. I like to name unit tests projects as .Tests where is the name of the production assembly holding the types being tested. That way they appear next to each other (alphabetical order) in VS.
With TDD development I need rapid switching between production code and unit tests. It is about writing the code, not so much about post code testing.
I can select the solution node at the top of the solution pane and say 'run all tests' (I use Resharper(.
The build farm (TeamCity) always runs all tests on all commits. Much easier with them all in one solution.
As I write this I wonder why put them in another solution. Unit test have no meaning outside of the solution with the production code.
Perhaps you can explain why your asking the question. Can you see a problem with them being in the same solution. 18 projects does not, to me, seem like a lot. The solutions I work on have many more ... but if using Resharper (and why wouldn't you?) you will need sufficient RAM in you box.
Hope this is of some help.
I'm currently creating a paired unit test assembly for every assembly in my project, both are in the same folder.
MyProject/MyProject.csproj
MyProject.Test/MyProject.Test.csproj
Looking at open source projects, I've seen some smaller project put all tests in one assembly, and other split it out like mine. I'm dealing with a large solution so it would be pretty crazy to put all tests in one project.
I currently have msbuild logic to run tests on all *.Test.csproj files. If I had all my tests in a different folder I wouldn't need to do this.
Just wondering if there are any good arguments to do things a certain way.
Thanks
I do it the same way but I change the default namespace for each test project to match the namespace of the production project. So the tests for class X.Y.Foo are in X.Y.FooTest rather than X.Y.Test.FooTest - it means you need fewer using directives, and generally makes things simpler.
My main reason for wanting to keep the two in separate projects is to avoid either including the tests in the production library or having to ship an untested library. With the separate project structure, you can run unit tests against anything you build. It also makes it easier to look through just the production classes without having twice as many files to look at (when getting the "feel" of a library).
Finally, don't forget that if you need to access internal members when testing, there's always [InternalsVisibleTo].
I suggest making as few unit test projects as possible. The reason being is that each one you create adds on at least ten seconds of compile time. In a big project, it starts adding up.
Here's the directory structure I use:
projectName/branches/trunk/projects/code/codeproject1
projectName/branches/trunk/projects/code/codeproject2
projectName/branches/trunk/projects/code/codeproject3
projectName/branches/trunk/projects/Tests/testproject1
projectName/branches/trunk/Dependencies
projectName/prototypes
projectName/...
and within testproject1, the following directory structure:
codeproject1/
codeproject2/
codeproject2/web
codeproject2/web/mvc
codeproject3/
codeproject3/support
I do the same thing except each project is in it's own folder under the same root folder.
Something along the following:
Solution Folder
ProjectA folder
ProjectA.Test folder
ProjectB folder
ProjectB.Test folder
I always have a separate test project for each project. Part of it is simply I like the organization of it, but I've also often run into situations where I've decide to break a library out into it's own solution so that it can be reused by other solutions. In those cases, having the library project have it's own separate test project (rather than all the tests in a single project) makes it much easier to break that library out.
I have data files used as input to my unit tests. These files are quite big and I don't want to copy them each time unit tests are executed. Tests are executed without deployment. So I can just put them into folder under my solution, and... how to obtain path to my solution (or test project source code) when unit test is executing?
Because you can run a test project in different ways (TD.NET, Visual Studio, R# etc.), the path used to reference the tests can change.
For this reason, I embed test needed files in my test assembly and draw them out from there.
You can use:
Assembly.GetExecutingAssembly().Location
in your tests to get the path of the assembly containing the unit tests.
Simple, make the location of the files configurable (and testable).
Then either, set it in the unit testing code or set it thru some config file.
I need some advice as to how I easily can separate test runs for unit tests and integration test in Visual Studio. Often, or always, I structure the solution as presented in the above picture: separate projects for unit tests and integration tests. The unit tests is run very frequently while the integration tests naturally is run when the context is correctly aligned.
My goal is to somehow be able configure which tests (or test folders) to run when I use a keyboard shortcut. The tests should preferably be run by a graphical test runner (ReSharpers). So for example
Alt+1 runs the tests in project BLL.Test,
Alt+2 runs the tests in project DAL.Tests,
Alt+3 runs them both (i.e. all the tests in the [Tests] folder, and
Alt+4 runs the tests in folder [Tests.Integration].
TestDriven.net have an option of running just the test in the selected folder or project by right-clicking it and select Run Test(s). Being able to do this, but via a keyboard command and with a graphical test runner would be awesome.
Currently I use VS2008, ReSharper 4 and nUnit. But advice for a setup in the general is of course also appreciated.
I actually found kind of a solution for this on my own by using keyboard command bound to a macro. The macro was recorded from the menu Tools>Macros>Record TemporaryMacro. While recording I selected my [Tests] folder and ran ReSharpers UnitTest.ContextRun. This resulted in the following macro,
Sub TemporaryMacro()
DTE.Windows.Item(Constants.vsWindowKindSolutionExplorer).Activate
DTE.ActiveWindow.Object.GetItem("TestUnitTest\Tests").Select(vsUISelectionType.vsUISelectionTypeSelect)
DTE.ExecuteCommand("ReSharper.UnitTest_ContextRun")
End Sub
which was then bound to it's own keyboard command in Tools>Options>Environment>Keyboard.
However, what would be even more awesome is a more general solution where I can configure exactly which projects/folders/classes to run and when. For example by the means of an xml file. This could then easily be checked in to version control and distributed to everyone who works with the project.
This is a bit of fiddly solution, but you could configure some external tools for each of group of tests you want to run. I'm not sure if you'll be able to launch the ReSharper test runner this way, but you can run the console version of nunit. Once you have of those tools setup, you can assigned keyboard shortcuts to the commands "Tools.ExternalCommand1", "Tools.ExternalCommand2", etc.
This wont really scale very well, and it's awkward to change - but it will give you keyboard shortcuts for running your tests. It does feel like there should be a much simpler way of doing this.
You can use a VS macro to parse the XML file and then call nunit.exe with the /fixture command line argument to specify which classes to run or generate a selection save file and run nunit using that.
I have never used this but maybe it could help....
http://www.codeplex.com/VS2008UnitTestGUI
"Project Description
This project is about running all unit test inside multiple .NET Unit tests assembly coded with Visual Studio 2008."