I have a TFS build in VS2010. Following the build unit tests are executed. In the Build summary it tells me that "1 test run(s) completed - 100% average pass rate" but below this it states "No tests were impacted". I guess Impacted Tests relate to functionality providing the ability to only run tests that were impacted by code checked in? Is this correct and since I'm happy to run all tests following each build, how can I turn off the "No tests were impacted" message?
Thanks,
Rob
You're right about test impact analysis - it looks at the changesets since the last build and which tests intersect with that code. To disable test impact analysis for a build:
Right-click your build definition in Team Explorer and choose Edit...
Go to the Process category and expand the Advanced section
Set "Analyze Test Impact" to False
You can also set this value on a per-build basis - queue a manual build and the setting is available via the Parameters tab.
Related
Im using SonarQube v6.4. I'm aware that all types of tests (Unit Tests, Integration Tests etc) have been merged together as overall coverage.
However, on the interface I can see statistics for only Unit tests, Is there a way to get the statistics for other types of tests?
Example of statistics available only for unit test
Unit Test Errors
Unit Test Failures
Skipped Unit Tests
Unit Test Success (%)
Unit Test Duration
SonarQube no longer distinguishes between different types of tests. Integration tests, Smoke Tests, Medium Tests, Regression Tests, etc. - all are now called "Unit Tests". This new naming is indeed misleading...
For to see the values navigate to your project, click on the "Measures" tab ("All" page) and scroll down to "Coverage". There you will find the current tests measure values.
Starting from version 6.6 of SonarQube you will be able to show graphs for any metric (see SonarQube's own SonarQube instance with 6.6-SNAPSHOT installed).
Navigate to any SonarQube project, click on the tab "Activity", select "Custom" from the drop down and click on "Add metric". There you can choose "Unit Test Errors", "Skipped Unit Tests", etc.
I discovered that this feature is not supported in Sonarqube. A ticket opened for this issue has already been closed as "Won't Fix" by the Sonarqube Team.
For a workaround you can check this
I'm trying to understand how to use visual studio online. I have a user story and it has linked to a test case.
This test case has an automated test case (which is a unit test in my test project).
The test case is:
I have it running my tests on build, but even after a successful test run the little beaker thing tool-tip on the user story shows 0 passed, 0 failed and 1 pending. I have to manually pass the test. I assume this is because I have no steps in the test. I don't want it to be a manual test though.
Is their anyway to have it run and pass my tests and have this reflected on the user story? (If I pass it manually, a little green tick appears next to the beaker).
You need to make sure you specified corresponding test plan and test suite for Run Functional Tests step/task.
You can click open test suite icon to navigate to corresponding test suite.
Is it possible to stop a TeamCity build (the entire build, i.e. it won't execute subsequent steps) when a unit test fails? Ideally I'd also like it to terminate the currently executing step which in my case would be the Nunit Test Runner. In my circumstance there is no point in continuing the build if a single unit test fails. I've looked at Failure Conditions but I don't think they are applicable as the build continues to run.
Feature requested: http://youtrack.jetbrains.com/issue/YTF-3275
As you noted, TeamCity can run no further build steps on test failures in NUnit tests with "Only if build status is successful" step execution condition. However, that does not make the tests run stop until the step finish.
A related feature request is TW-23766.
The only workaround I can consider currently is not to use NUnit test runner and implement the logic inside the build script. For example, with nunit-console.exe like Manuel noted. If you choose to follow this route, consider using TeamCity Addin for NUNit.
You can do it using nunit-console.exe.
accordingly to the official documentation (http://nunit.org/index.php?p=consoleCommandLine&r=2.6.2) it provides a /stoponerror switch that does exactly what you need.
it can also generate an XML output that can be parsed by teamcity (there is a build feature for that) in order to populate the "test" tab.
on the build step after the unit tests, change the setting to Only if Build status is successful instead of if all previous steps finished successfully
see screenshot
also make sure that under failure conditions menu item the at least one test failed option is also ticked.
I am writing some unit tests for one of my DLL libraries.
The 'Code Coverage Results' pane shows a breakdown of the assemblies covered and tested.
For some strange reason - my test project itself appears in the coverage results ! (at approx. 90% covered).
This seems stupid... what's the deal with this ?
The reason the percentage is so high is that projects for code coverage are instrumented to keep track of which lines are hit by a test run, since you are running the tests from this project, almost all lines of code in the project will be run.
You can choose which projects/DLLs to collect Coverage statistics on in the Test Settings.
So if you don't need to capture stats on the test project (which you shouldn't really), you can simply remove this project from the settings you're using for coverage.
See http://msdn.microsoft.com/en-us/library/ms182534.aspx (steps 5 - 7 in particular) for more details.
I am working with an n-tiered architecture in Visual Studio 2008 (Developer Edition), and I have run into an issue.
We are running unit tests on every method of our services layer, and I am attempting to see the code coverage results, to ensure I'm hitting all the main paths through my methods.
When I attempt to view the results, I am informed that the test run did not have code coverage enabled. I tried editing the test run configuration through test -> edit test run configurations, only to discover that there were none found.
I have yet to discover how to create a test run configuration so that I can enable code coverage results.
How do I create the configuration?
Right-click the solution, choose Add, New item, choose Test run configuration in the resulting dialog. The only tricky part is that you don't get this choice unless you right-click the solution.