I recently run into problems when running all my unit tests at once.
I can debug them and run my tests seperate without problems, but when running them all together, the test-run keeps hanging half way through.
This happens:
"Run all tests in Solution"
The first tests parses without problem (slower then usual though)
At some point it gets stuck. Nothing fails, no exceptions, VS just stops running the pending tests.
When stopping the test-run it gets stuck again, and I need to restart VS to abort the test-run.
Normally I would expect a bug in my code, but I haven't made any changes to the code beeing testet since last succesful test-run. The only thing I did was run Performance Wizard - Cuncurrency profiling.
It always stops the same place, when removing some tests from the run it stops a new place (still without actually entering any leftover tests).
I have no clue what is causing this. But seems like I'm having problem with a VS setting rather then a code Error.
Any suggestions? Do Performance Wizard change any settings that might have influenced the way test should be run?
System details:
Windows 7 Ultimate 64-bit,
Visual Studio 10 Premium
This sounds like a concurrency issue. It seems that one test changes the testenvironment in such a way that another test runs into a deadlock. When you remove some tests, the test run order is changed and some other tests get stuck.
So I would look for a concurrency issue regarding your test environment/externall dependencies.
I can't really explain why this works, but it solved the problem!
I reversed the '.csproj' file to an earlier version, in one of the projects that had been in 'contact' with the Performance Wizard, and now my tests works.
ALSO Be aware of that Performance Wizard can change the solution configurations from 'DEBUG' to 'RELEASE' mode in some cases. This was not the case for me, but have been a pain for some of my colleagues.
Related
This question is very similar to other questions that also in some cases literally have the text "tests not running in Test Explorer" in the title. But, my context is a bit different. In those questions, there was a fair bit of investigation into what might be wrong with the tests. I am fairly confident nothing is wrong with the tests in this case.
I am one of hundreds of developers working on a project, and this project has a large bank of automated tests (though perhaps not as large as it ought to be :-P). Everybody is frequently running tests, and triggers fire when pull requests are made and merged to automatically run them then too. Tests were working fine for me as well. But, I have just been given a new laptop with better hardware specs, and I am trying to get it set up. On the new laptop, the project builds just fine (and noticeably faster :-) ), but the automated tests just don't run. I can't figure out why, and I'm looking for suggestions about what to check in this context -- given that there are hundreds of places where the exact same code is working perfectly, I really don't think the tests or test projects themselves are at fault here.
I have observed that the build output, apparently randomly, sometimes does not contain the test adapter files:
Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll
Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll
Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll
xunit.runner.visualstudio.testadapter.dll
If these files are missing, then VSTest.Console.exe also cannot run the test. But, usually rebuilding the project results in the files magically appearing, and then VSTest.Console.exe works just fine.
I haven't been able to ascertain a reason why the adapter files are sometimes put into the build output and sometimes not, and in either case, the Test Explorer within Visual Studio always fails to run the tests -- it discovers the tests just fine, puts several thousand items into the forest of trees, but when told to run tests, it just sits there for a minute or two and then returns to idle state with no output at all in the "Tests" output window.
This is a brand new installation of Visual Studio Enterprise 2019 Preview, the exact same version that is on my old laptop, but on my old laptop the tests run fine. What do?? I don't know what to check next. :-(
Well, I am thoroughly confused. I tried installing new features, I tried checking for system updates, I rebooted multiple times, and tests did not work. So, finally, I decided to make a cut-down minimal test project and see if I could observe any differences in Process Monitor between the two computers. I made a project with two tiny tests, one with NUnit and one with xUnit, and ... they worked. So, I opened up the big project again and hit Run Tests, and ... they worked. I am completely stumped, and the only advice I can offer to anyone finding this question with a similar problem is, just keep trying.
Visual Studio Version 15.2 (26430.6) Release.
Having recently updated to the above version I am running into continuous issues with VS locking up files when trying to build.
Could not copy "obj\Debug\projHype.dll" to "bin\Debug\projHype.dll". Exceeded retry count of 10. Failed.
Also tried running VS2017 with and with out admin priveledges
I tried the suggestions for older versions of visual studio but to no avail. Any ideas how to get around this?
For anyone encountering this. Updating to version 26430.12 will resolve this. Looks like the previous release contained a bug.
While there may be other causes, testhost.exe and testhost.x86.exe can both result in a lock that prevents the build from completing. The symptoms are bewildering-- the test explorer churns indefinitely, sometimes timeout warnings appear in the build. Sometimes the files cannot be accessed even after VS is shut down.
If you are using nUnit or another test framework, make sure that test discovery does not encounter any infinite loops or crashes in your code. If this happens, it can hang the testhost executable. For example, if you use nUnit TestCase or TestCaseSource, if any of these perform an action that can hang, lock up, or crash, they will be invoked before the tests are executed.
This is the tricky bit-- your tests haven't run yet, but your code can lock up VS! While this may not be your problem, if you have unit test discovery in any way, check that it all completes.
One way to make sure all discovery completes is to use the functions from a test itself, and disable them as a TestCase or TestCaseSource (or equivalent in other test frameworks). If the test hangs or crashes, that's the culprit.
For me it helped to run the program again with accepting "run last successful build". After the run, the locks were gone.
The test explorer (MS-Test) kind of stopped working. It won’t discover any of my tests or sometimes only some of it. When I right click on my test and run it, the output window says “Run test finished: 1 run (time)”, but the test explorer would only display “Last Test Run Failed” (see screenshot), without information about which test or assert failed. I’ve tried restarting the VS, which I thought solved the issue once, but not currently. I’ve also tried build, rebuild, but I just can’t get it working again? Any idea what might help? All of this was working fine until early morning today...
Test Explorer has problems understanding test projects if they are build for x64 or x86, and solution is targeting a different platform.
Also, under Test > Test Settings > Default Test Architecture select appropriate value and then clean and rebuild.
I'm using MSTest in Visual Studio 2010 and have the need to restore my database after all tests have run.
What I did was decorate a method with the AssemblyCleanupAttribute attribute.
<AssemblyCleanupAttribute()>
Shared Sub AssemblyCleanup()
' Restore my databases which takes a long time...
End Sub
Problem is the clean up takes a reasonable amount of time, so much so that the timeout is reached.
The only reason I started realizing that a timeout occurred is that in debug mode the Output window reports "...QTAgent32.exe, AgentObject: Cleanup: Timeout reached in cleaning up the agent.". Hence it fails very quietly and I would have loved if MSTest reported a Test Run Error.
What is the best way to detect and report the timeout? My ideal solution would be to report the timeout as a test run error.
In short, you cannot cause MSTest to report an error if AssemblyCleanup times out.
If you are encountering this issue, then at this point you need to consider if this limitation of MSTest is too great for you. There are other, and imho better, test frameworks out there.
If you decide to stick with MSTest and just want to ensure, at least, that the code/script in AssemblyCleanup runs to completion then you can choose to either run the clean up code as a Process. That way even if AssemblyCleanup internally calls a Thread.Abort then your Process runs to completion. It's messy though...
Why not wrap the contents of each test in a transaction, and rollback the transaction at the end of the test? See here for more information: http://msdn.microsoft.com/en-us/library/bb381703(v=vs.80).aspx#dtbunttsttedp_topic7
is there a way? do I have to wait for building every time I start the test? I want to build from visual studio not from test
thanks
Any time your code changes and you run your test it is going to do a build... so technically you can run your test over and over again and they will only build the first time, but once you run your test why would you run them again without making a code change?
Couple of things that I use that make your test run faster are:
Check the box for "Only build startup projects and dependencies on Run", located Options->Projects and Solution->Build and Run.
Learn the short cut keys
a. "Ctrl+R, T" Runs test in current context, so if your cursor is inside a test method it will only run that test, but when you do it inside of a non test class it will run all of your test.
b. "Crtl+R, Ctrl+T" Debug test same except debug.
c. Others can be found here, those are 2008 if you need to reference others you can find them via google.
Make sure your test are not calling the database or other time intensive resources, use mocking and stubbing.
Run only small sets of test, ie if I am working in a service class I run only the service class test.
Edit: Reading your question again if you want to build and not from a test you can just go to the menu and click Build->Build Solution or press F6. Also it would be helpful if you indicated which version of visual studio you are using because 2010 is different in the sense that you have to click refresh. Either way are you able to clarify?
This is an old question, but I keep seeing people ask it and the issue is still true in VS2017, and it's also true of other test frameworks (Xunit, etc) run from within VS.
I don't know how to make VS stop building all the time. But I do know how to circumvent the compile - run your tests from a console runner, not from within VS. If you're using ReSharper, it has one.
If you aren't using ReSharper, for MSTest, you can start here. https://msdn.microsoft.com/en-us/library/ms182489.aspx
If you aren't using ReSharper, for XUnit, you can start here. https://xunit.github.io/docs/getting-started-desktop.html#add-xunit-runner-ref
Any changes to source code cause compilation, because in order to run tests VS needs up to date DLL with tests.
If you have already compiled project then you can run test multiple times without compilation.
PS: I run MSTest using TestDriven.NET as for me it is faster.