We have an issue with our unit tests. At some point, hitting "Run All" started giving us a truncated subset of tests that ran. The others were simply not run, and we have been having to manually run groups and individual tests.
Any recommendations from anyone out there?
Update:
Partial Solve!
So, my first question was early on in my investigation. I had looked for previous solutions and hadn't found anything that worked out.
Some additional details...
This is an Android project using Xamarin and MvvmCross. The problem was a call to .Clear() on an MvxObservableCollection. Apparently the test just wasn't set up properly, and the IoC provider wasn't able to resolve an interface. I solved it temporarily by removing the call to the offending method.
My question now is how to set this up properly. We have a base test class that we are inheriting from MvxIoCSupportingTest, and the required components seem to have been set up correctly, but we were still experiencing this issue. How do we set up our tests so we can use the .Clear and other MvxObservableCollection methods?
Related
I'm currently trying to make UI-Tests in Xamarin, where i want the app to be tested against different server versions.
Originally i wanted to do this like i did in unit-tests with parametrized tests by using [Theory] and [ClassData].
Unfortunately i haven't found anything about this in the web so far regarding this nUnit construct.
Is this even possible? or is there another way to tell my xamarin UI-Test to run all my tests once for each server version?.
Many thanks in advance.
Mav
Soo, the mistake i made was to assume that i could use xunit blocks for nunit tests.
My bad.
The solution is pretty simple and can be looked up here in the gitHub projects documentation:
https://github.com/nunit/docs/wiki/TestFixtureData
I created a console application that was designed to be completely self-reliant and contain all the necessary DLLs to run. I was able to do this and get it to run locally, but when copying to another computer I began to receive this message...
"No test discoverer is registered to perform discovery of test cases. Register a test discoverer and try again."
I found this Microsoft bug report, but it was closed as could not reproduce:
https://connect.microsoft.com/VisualStudio/feedback/details/770093/unit-test-project-not-discovering-test-cases
I didn't want to sit around waiting for Microsoft to say they couldn't reproduce so I dug into it and found that the reason I was getting this error message is because I was missing the DLL file "Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll".
This is the file that contains the test discoverer and it needs to be in directory of the executable that is running the tests as well as the directory of the test files themselves. It is easy to miss because the assemblies to run the tests appear to be completely contained within the "Microsoft.VisualStudio.TestPlatform" namespace and the tests appear to only require a reference to "Microsoft.VisualStudio.TestTools.UnitTesting".
I decided to post this "knowledge share" post because there was no other information about the issue on the internet and Microsoft's documentation of their MSTest framework is bad at best.
Just come across this issue with some unit tests originally written in .Net 3.5 and running (or not) in vs 2013.
The UnitTestFramework.dll mentioned by Love2Learn was present however updating to .Net 4.5 seems to resolve the problem.
I'm sure someone out there can explain why but my gut feeling is that this has something to do with this being a multi-platform solution and that .Net 4.5 is just better at running 32bit unit tests against 64bit projects.
I wanted to know is there any way in IDEA putting debuggers' breakpoint to previous row where debugger has just passed and re-run that scenario again. Because when debugging a loop, it's a bit tedious going to a UI doing desired action for handling that case again. I think such feature is available in Visual Studio. I've tried to google it but unfortunately I could not found anything about it.
I've submitted an issue on IDEA's bug tracker but its status changed to won't fix with this comment:
This functionality is not supported by the JVM. It's not possible to
implement it in IntelliJ IDEA without JVM support.
http://youtrack.jetbrains.com/issue/IDEA-84257?projectKey=IDEA
I'm having a little annoying problem with resharper trying to run my unit tests, using mstest. I can select an individual unit test and run/debug it fine but when I select the parent node to run a bunch of tests, the tests don't run; it just displays pending for a few seconds and then the test goes grey. If I set breakpoints in the tests and try again with the debugger, I get the same result and no breakpoints are hit. Has anyone else experienced this before?
For anyone using VS2012 update 2 (Charlie obviously wasn't when he asked the question, but for anyone else...), then make sure you have R# version 7.1.3 or later. At the time of this writing, the official release is 7.1.2 so you'll have to follow this link to go download it:
http://download.jetbrains.com/resharper/ReSharperSetup.7.1.3000.1964.msi
(as referenced from http://youtrack.jetbrains.com/issue/RSRP-339987)
Try to rebuild your test project/entire solution.
I sometimes have similar problems where the resharper shows tests that I have removed, or doesn't execute tests for new methods in the SUT project. After a rebuild it gets back in sync.
Ran into this problem aswell with ReSharper 5.0.1659 and VS2010. Turns out that the problem lies somewhere in the testrun config files that VS creates when running test the regular VS way. Removed them from both disc as well as references in solution file which solved the problem and I was able to run the tests with ReSharper again.
Is there an API for running Visual Studio Unit Tests programmatically?
Running MSTests.exe with Process.Start() does not work in the current scenario. What I'm looking for is something like the NUnit SimpleTestRunner.
Any ideas?
/Erik
You're correct in that there's no public API for the mstest framework. I wrote a manual replacement for mstest one day to see how hard it was, and it's not as simple as it looks (particularly if you want to take advantage of more than one CPU core), so beware of going down this path.
Personally I've always just run mstest.exe programatically and then parsed the resulting .trx XML file. Are there any particular reasons why you can't use Process.Start to run it?
P.S. Some of the strange behaviour of mstest.exe are solved if you pass the /noisolation command line parameter - give that a go if you feel so inclined :-)
Update: Erik mentions he wants to run the test API in the current thread so he can set the thread culture for globalization issues.
If you run a unit test under the debugger, you'll notice that mstest creates a bunch of threads, and runs all your tests in different threads, so this isn't likely to work even if you could access the API.
What I'd suggest doing is this:
From your test "runner" application, set an environment variable
Run mstest pointing it at the specific tests
Add a [ClassInitialize] (or [TestInitialize]) method which reads this environment variable and sets the culture
Profit!
After taking a deep dive with reflector into MSTest.exe and further down into the Visual Studio Unit Test stack, I found that the API used by MSTest is sealed up and made private so that i cannot be used from the outside.
Why not using Reflector and seeing how NUnit SimpleTestRunner is running the tests... And then use this technique...
You can make use of the Microsoft REST API's for TFS to run ms tests. Please refer to the documentation here.
I've linked to "Call a Rest API" so that you can see how you'd go about calling one of the REST API's for TFS.
Note that if your tests are linked to the build, they should run automatically every time a build is queued.
Here is the link to Run Functional Tests.
I've also discovered an article on using the TFS SDK API to run tests. Here is that link as well: Link to API Article