I am trying to set up a Microsoft Visual Studio Unit Testing environment for some of my SQL queries. My scenario is somewhat backwards to the typical software development scenario. For any given work product that I work on, I am trying to exercise ONE query through different scenarios, by passing in different parameters and comparing results.
My intention is to load my SQL query from a .sql file in each of my unit tests.
How do I open my SQL file for reading from within my SqlServerUnitTest1.cs file?
Solving this was complicated, so I will describe the general outline of what I did. Some of this might have been unnecessary, but at least my instructions will help you understand all the important steps:
Create a new solution, and create a new Unit Testing project in it. (If you're using the Reporting Services, like I am, make a Reporting project alongside it)
Import the queries to your testing project, and set the properties to "Copy if Newer" or "Always Copy".
Load the SQL file in your test. I wrote a function of the form:
.
public static string LoadSqlFile(string name)
{
string path = Path.Combine( Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)
, #"sql\" + name
);
return File.ReadAllText(path);
}
which loads the SQL file by its filename. This is the answer to my specific question.
Write the test (using the UnitTest template, not the SqlServerUnitTest template). I wrote some code to generate tests from JSON, which makes this easier for the non-C#-users on my team. In my use case, each test has an undeclared parameter which gets passed in to the test. The JSON also takes in a list of expected results, which we compare to the result.
Related
I'm using Visual Studio Online Load Testing to test an API with variable parameters coming from a CSV file.
My setup looks like this:
In properties I set "Show Separate Request Results" to True, hoping that I would be able to see which parameters were used during the test, but I cannot find anything on this in the report?
Is this the way to do this or am I doing something wrong?
Visual Studio load tests are not great at showing how individual test cases worked. The test case logs show the data source values used by a test, look in the context section of the log. By default, logs of the first 200 test cases that fail are retained; altered via Maximum test logs in run settings. Logs of successful tests can also be retained by altering Save log frequency for completed tests in run settings.
Whilst the log files have the data in their context sections, it is hard work (ie lots of mouse waving and mouse clicking) to open each log file, view the context, scroll the right section into view, close the log file, etc.
The mechanism I use to record data source usage etc is to have a web test plugin with a PostWebTest method. It writes useful data to a simple text file as each test case finishes. I write one line per test case, formatted as a CSV so it can easily be read and analysed in a spreadsheet. The data written includes date, time, test outcome, some data source values, and some context parameter values extracted or generated during the run. Tests run with multiple agents will get one file written on each agent. Gathering these files will be a little work but less than viewing individual test case log files. Unfortunately I have not found a way of collecting these files from load tests run with Visual Studio Team Services (previously known as Visual Studio Online).
An early version of the plugins I wrote can be found here.
I have a visual studio solution of C# projects. Currently, I do not have test projects in my solution. So I do not have file *.vsmdi.
I want to test my projects via command line like this.
MSTest.exe /testmetadata:*.vsmdi /resultsfile:Logfile.log
Is it possible to create .vsmdi without adding project? if possible, how?
To execute in command line, *.vsmdi file MUST be present in the solution?
If I have to test using IDE, the I should have *.vsmdi file in test project?
The only use of *.vsmdi file is for listing the test cases?
1) Technically it’s possible to create a .vsmdi file without adding your test project(s) to your solution. Since the content of the VSMDI files is XML you can write them by hand or generate them with a tool based on the tests in your assembly. However, VSMDI files are less useful when the test projects are not included in the solution because the power of the VSMDI files is managing your tests. But you can still use them (see also answer to question 2).
If you want to write the vsmdi file by hand you need to hash the FQN with SHA1 and create a Guid of the first 128 bits of the hash as shown below (code from Dominic Hopton). See for more information his blog post: Helpful Internals of TRX (and VSMDI files).
private static HashAlgorithm s_provider = new SHA1CryptoServiceProvider();
internal static HashAlgorithm Provider
{
get { return s_provider; }
}
/// Calculates a hash of the string and copies the first 128 bits of the hash
/// to a new Guid.
///
internal static Guid GuidFromString(string data)
{
byte[] hash = Provider.ComputeHash(
System.Text.Encoding.Unicode.GetBytes(data));
byte[] toGuid = new byte[16];
Array.Copy(hash, toGuid, 16);
return new Guid(toGuid);
}
2) No, the vsmdi file doesn’t need to be in your solution folder (or in your solution) as long as the storage attribute of the TestLink element points to the correct assembly containing the test (relative to your vsmdi file). For example:
<TestLink id="0c257cc9-ffed-cd5e-bda1-56df14ad68c2" name="AddOneAndTwo_Test"
storage="TestProject\bin\debug\testproject.dll" />
3) Vsmdi files exists on the solution level and not at project level. If you have more than one test projects you can use one vsmdi file to manage all your tests. Please note that the storage attribute is ignored when you run the tests from Visual Studio assuming that you loaded the projects in your solution
4) Vsmdi files are used for managing the tests in your solution and you can create a hierarchy of several layers within the VSMDI file. For example you have more than one assembly in your solution you want to test. Under the root test list (called: List Of Tests) you create a test list for each of your projects. You can then divide the test list for one of the projects into two or more separate test lists (e.g. one for testing the business logic and one for testing the error handling). When you only want to test the business logic tests from project A you can select the corresponding test list and run the tests in it.
An extra note about vsmdi files:
A) Since Visual Studio 2012 Microsoft decided to make VSMDI files (and with them the test lists) deprecated. Test lists got replaced by Test Categories (already exists in Visual Studio 2010) and Playlists.
Test categories are attributes you can set on individual test methods and can be used in builds. Playlists were introduced in Visual Studio 2012 Update 2 and are currently not supported in MSBuild.
For more information about Test Categories see: http://msdn.microsoft.com/en-us/library/dd286595.aspx
For regulatory reason all unit tests in our solution have to have some attributes set. Is there a way to check in visual studio if some specific attributes (like CREATOR, REVIEWER, REVIEWDATE, ...) are set within the VSMDI-list of the solution?
Further I have to check if the "CREATOR" attribute is equal to the "REVIEWER" attribute.
The goal is to deliver a list (Text, Excel, whatever) to my Scrum team with all these "wrong set attributes", so that they can fix them.
You can use reflection to check for the attributes and then create your Excel/Text list. You can do this as a MSBuild task and plug it into pre/post build actions of your Unit test project. See this blog post for an example on how to create a custom MSBuild task.
After running all the needed Tests within Visual Studio 2010 you get a TRX-file containing all the details of the executed tests. If you have put some attribute on these tests,you'll find them also in the TRX-file. Now you have to write an XML parser to read the attributes you need and work with this data. I extracted them into Excel and worked further with the data into the Excel map.
I want to generate some code from my dbml(Linq to Sql) file,the dbml file is placed in many part of my project So I wrote a custom tool for this purpose
But the problem is that dbml already has contained MSLinqToSQLGenerator custom tool ,
So do you know any way to set two custom tools for one file, If no, Let me know your idea about that
Visual Studio will only support a single "Custom Tool" per file, but you can add a pre-compilation step to run other tools against anything you want. For instance, I have the following pre-compile step set on the "Build Events" tab of one of my projects.
"$(DevEnvDir)..\..\..\Common Files\Microsoft Shared\TextTemplating\10.0\TextTransform" "$(ProjectDir)DataContext\Northwind.proxy.tt"
There's a lot of relative pathing going on here in order to find the T4 command-line tool, but you get the idea. This particular T4 file counts on being in the same directory as the .dbml file that it reads to generate its output.
Before the project is compiled, you can run whatever external tool you want. Just make sure that after the first run, you include the tool's output in the project. After that, since the file gets changed as part of a PRE-compile step, it will always be updated in each build.
You'd get proper control on the T4 if you include the LINQ to SQL T4 generator in your template's responsibility.
If I understood properly, you want to keep the default behavior of .dbml generator, but also add your own.
This seemed a bit "old", and I haven't personally used LINQ to SQL for some time, but I did use this as-is replacement of T4 generator, that produced the equivalent of the standard .dbml generator.
https://github.com/damieng/L2ST4
Not sure if that's up to date with VS 2010 version, but you can always compare the standard .dbml generated code and this T4 output and make proper changes to achieve identical outcome.
Of course you can simply have multiple different generators, and simply run them with "Transform All Templates", but based on your question, you'd want the generator to be attached to the file specific custom tool.
You might want to check out (unless its already familiar to you) also T4 Toolbox https://github.com/olegsych/T4Toolbox that adds "T4ScriptFileGenerator" custom tool to a file. It effectively runs the T4 code when the file changes.
I have a few tests that need to be fed with external data from excel files. The files are included in the test project, and in Visual Studio, I have edited the test settings file (Local.testsettings) to deploy the data files. This makes it work fine i VS.
We are, however, also running continous integration with TeamCity, and in TeamCity this doesn't work. My data files are unavailable to the test. Seems that the tests are run from a temporary folder named "C:\TeamCity\buildAgent\temp\buildTmp\ciuser_AS40VS6 2009-12-11 09_40_17\Out", and the data files are not copied there.
I have tried changing the build action for the data files to "Resource" and setting copy to output dir to "Always", but that didn't help.
Does anyone know how to make this work?
I am running Visual Studio 2010 beta 2 and TeamCity 4.5.5, which is why I'm running MSTest in the first place, and not NUnit...
I get round this by adding my data files (in my case usually XML) as embedded resources and I extract them from the test assembly.
[TestInitialize]
public void InitializeTests()
{
var asm = Assembly.GetExecutingAssembly();
this.doc = new XmlDocument();
this.doc.Load(asm.GetManifestResourceStream("TestAssembly.File.xml"));
}
This post answers this question: MSTest copy file to test run folder
The accepted answer is technically correct. However, from my experience, I find that the embedding files as resources requires an additional step of remembering to set the property "Embedded Resource". This becomes a challenge when you have a large number of data files. Also, with increasing number of data files, the size of the unit test assembly keeps growing . In my case, I had over 500MB of test data files and packing all them into the assembly was not a good idea.
What is the alternative?
Let the data files remain as they are. Do not use DeploymentItemAttribute, do not use embedded resources. Please refer my proposed solution How do I make a data file available to unit tests?