I have started to work with Unit Testing in Visual Studio 2010, .NET 4.0.
Some of our tests fail if a value is below a certain threshold, and then we plan to design the proper exception.
For example, let's suppose some method is expected to return a value greater than 10. I run the tests, and when I open the automatically saved .trx file, it says the test failed.
What I want to know is:
Is there a way to save property values during a test, so that I can know their values after the test list execution has finished?
Here are some resources that explain the different ways to log values. from the link:
[TestMethod]
public void CodedUITestMethod1()
{
Console.WriteLine("Console.WriteLine()");
Console.Error.WriteLine("Console.Error.WriteLine()");
TestContext.WriteLine("TestContext.WriteLine()");
Trace.WriteLine("Trace.WriteLine()");
Debug.WriteLine("Debug.WriteLine()");
}
Keep in mind they are not captured during run time, but flushed at the end, so if there is an unhandled exception the values may be lost. In this case we send key values to a DB or file as a backup.
Related
Given the following unit test in Visual Studio (not VSCode)
TEST_METHOD(TestShutdown)
{
stopIt();
destroyIt();
// How do I know that those worked?
}
The functions stopIt and destroyIt stop & terminate a thread; they return no value. How can I guarantee that the have completed, or, indeed, any void function?
I guess that if they thrown an exception, then the unit test framework will catch it and fail the test.
What if one hangs, in an infinite loop? Does the test framework have a time that will catch that eventually?
I don't see that adding Assert::IsTrue(true, ...); after the calls would help.
Is there any way to be certain that those two calls completed - without changing the existing code?
I want to do load testing with Visual Studio but i don't get the idea how to setup load testing with authenticated users.
Imagine my scenario. This should be a quite common problem:
a website where you need to authenticate with username and password.
Perform a action that is only allowed for a authenticated user
What i have done so far:
i have already written UI tests with Selenium:
(this is working quite nice)
UPDATE: My Selenium test class: I want to use this code with the load test.This is a data-driven unit test project as you can see on method TestCase4529
[TestClass]
public class Scenario2
{
private IWebDriver driver;
public TestContext TestContext { get; set; }
[TestInitialize]
public void SetupTest()
{
this.driver = new ChromeDriver();
this.driver.Manage().Timeouts().ImplicitWait = new TimeSpan(0, 0, 30);
}
[TestCleanup]
public void TeardownTest()
{
this.driver.Quit();
}
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "|DataDirectory|\\4529.csv", "4529#csv", DataAccessMethod.Sequential), DeploymentItem("4529.csv"), TestMethod]
public void TestCase4529()
{
var userName = TestContext.DataRow["UserName"].ToString();
var password = TestContext.DataRow["Password"].ToString();
// UI Test logic
var loginPage = new LoginPage(this.driver);
loginPage.FillForm(userName, password);
loginPage.LoginButton.Click();
// Some assertions
}
}
Now when i setup the load test in Visual Studio i am asked how many users should do something:
I am not getting what this number means:
Does it only mean number of simultaneous threads?
How can i get a connection between a user (defined in the load test) an a authenticated user in my Selenium test?
What i would like to achieve:
Each user defined in the load test should be a authenthenticated user in my selenium UI test.
May somebody give me an idea how to do that or what i am thinking wrong...
A Visual Studio load test provides a way of running other tests repeatedly and simultaneously. It works best with Web Performance Tests but it can run unit tests and Coded UI tests.
When a "constant load" of (say) 25 users is selected then 25 test cases chosen from the "test mix" of the load test will be started. Whenever one of those test cases finishes another test will be chosen and started so that there are always 25 test cases being executed. That will continue until the end of the test run, which is normally either a test duration or a number of iterations. (Here "iterations" means number of test cases executed.)
Assuming "Web Performance Tests" are being used then those tests are responsible for providing the user authentication. A common way of doing that is to data drive the test and provide user names and corresponding passwords in that data. See here for more detail.
You question asks whether the "constant load" of 25 users means 25 threads. It means that 25 tests cases will be running at the same time, but it does not use Windows threads.
In response to comments:
I think you are misusing or misunderstanding the terminology of Microsoft's test environments. You may be able to have a Selenium test within the test mix of a load test although I have never done it. The user count and the data source are independent items. The user count is about how many simulated users are running tests at the same time. The data source is used by the test cases. If you have 25 users and one data driven test then that test should be started 25 times and those 25 executions should use the first 25 lines of the data source (assuming a Sequential or Unique access method).
To provide user and password you have to check for QueryString Parameters in your recorded webtest and pass data through Data Source see the following images for more detail:
Then pass the recorded webtest into load test as follow:
Why would this be happening? I have a no code in my test:
[TestClass]
public class ApiClientTest
{
[TestMethod]
public void ApiTestSampleUseCase()
{
TestCleanup();
}
public void TestCleanup()
{
//DeleteUsers();
}
}
Yet when I run this, it takes ~1 minute to complete. It passes, it just takes forever. I set a breakpoint after DeleteUsers and it gets hit immediately; the delay is after the test has completed.
Check your bin directory - for some incomprehensible reason, MSTest copies all referenced assemblies to a new directory every time you run the tests. (These are safe to delete, though.) Maybe it does something else as well that causes it to slow down after many runs... According to this answer and this documentation, you can disable this behaviour.
I'd recommend switching to NUnit if possible, preferrably in combination with FluentAssertions (due to the not insignificant number of weird design decisions in MSTest, such as this behaviour, the lack of easy-to-use parameterized tests, and the fact that a test will get run even if [TestInitialize] throws an exception).
Currently developing a connector DLL to HP's Quality Center. I'm using their (insert expelative) COM API to connect to the server. An Interop wrapper gets created automatically by VStudio.
My solution has 2 projects: the DLL and a tester application - essentially a form with buttons that call functions in the DLL. Everything works well - I can create defects, update them and delete them. When I close the main form, the application stops nicely.
But when I call a function that returns a list of all available projects (to fill a combo box), if I close the main form, VStudio still shows the solution as running and I have to stop it.
I've managed to pinpoint a single function in my code that when I call, the solution remains "hung" and if I don't, it closes well. It's a call to a property in the TDC object get_VisibleProjects that returns a List (not the .Net one, but a type in the COM library) - I just iterate over it and return a proper list (that I later use to fill the combo box):
public List<string> GetAvailableProjects()
{
List<string> projects = new List<string>();
foreach (string project in this.tdc.get_VisibleProjects(qcDomain))
{
projects.Add(project);
}
return projects;
}
My assumption is that something gets retained in memory. If I run the EXE outside of VStudio it closes - but who knows what gets left behind in memory?
My question is - how do I get rid of whatever calling this property returns? Shouldn't the GC handle this? Do I need to delve into pointers?
Things I've tried:
getting the list into a variable and setting it to null at the end of the function
Adding a destructor to the class and nulling the tdc object
Stepping through the tester function application all the way out, whne the form closes and the Main function ends - it closes, but VStudio still shows I'm running.
Thanks for your assistance!
Try to add these 2 lines to post-build event:
call "$(DevEnvDir)..\Tools\vsvars32.bat"
editbin.exe /NXCOMPAT:NO "$(TargetPath)"
Have you tried manually releasing the List object using System.Runtime.InteropServices.Marshal.ReleaseComObject when you are finished with it ?
I suspect some dangling threads.
When this happens, pause the process in the debugger and see what threads are still around.
May be try to iterate the list manually using it's count and Item properties instead of using it's iterator, some thing like:
for (int i=1; i <= lst.Count ; ++i)
{
string projectName = lst.Item(i);
}
It might be the Iterator that keeps it alive and not the list object itself, if not using an iterator might not have a problem.
Recently I started using mstest for testing.
Is there any way to write messages to test window if test successed? I don't see the way, messages are alowed only if test fails. What if I want to let say, print little description of a test, so I can see what test means without having to open the test. Or, as now is the case, I'm measuring times of execution for some tests, I want to print that time out.
Is there a way to extend test methods so to easy choose if I want tests with or without time measuring, choosing the mode of test execution?
Thanx
Right click on the columns in the test result window and choose "Add/Remove Columns". Add the columns for "Duration" and "Output (StdOut)". That will give you test timing and let you see what the tests print.
Why not give your tests descriptive names?
[Test]
public void AddsTwoNumbersTogether() {...}
[Test]
public void DividesFirstNumberBySecondNumber() {...}
etc.