Im using SonarQube v6.4. I'm aware that all types of tests (Unit Tests, Integration Tests etc) have been merged together as overall coverage.
However, on the interface I can see statistics for only Unit tests, Is there a way to get the statistics for other types of tests?
Example of statistics available only for unit test
Unit Test Errors
Unit Test Failures
Skipped Unit Tests
Unit Test Success (%)
Unit Test Duration
SonarQube no longer distinguishes between different types of tests. Integration tests, Smoke Tests, Medium Tests, Regression Tests, etc. - all are now called "Unit Tests". This new naming is indeed misleading...
For to see the values navigate to your project, click on the "Measures" tab ("All" page) and scroll down to "Coverage". There you will find the current tests measure values.
Starting from version 6.6 of SonarQube you will be able to show graphs for any metric (see SonarQube's own SonarQube instance with 6.6-SNAPSHOT installed).
Navigate to any SonarQube project, click on the tab "Activity", select "Custom" from the drop down and click on "Add metric". There you can choose "Unit Test Errors", "Skipped Unit Tests", etc.
I discovered that this feature is not supported in Sonarqube. A ticket opened for this issue has already been closed as "Won't Fix" by the Sonarqube Team.
For a workaround you can check this
Related
I know how to check coverage of Unit test cases, we can see coverage for each .swift file in Xcode coverage report. but what about UI test cases?
As per my understanding, In unit test cases the subject being tested is a .swift file. if a file has class with 4 methods/functions. The unit test coverage of that file would be 100% only if all the 4 methods are being called from unit test cases.
In UI test cases the subject is the View, Does interacting with all the UI elements leads to 100% coverage? How does coverage report for UI test work?
Edit:
In Unit tests - I know that when few lines of function are not covered I see red overlay, here I know that I have to write unit tests for screenshot class method, line 56 in above attached image. Is there any similar mechanism in UI tests?
In UI tests - How can we find which UI element is left uncovered?
The code coverage report can be generated for both unit and UI tests in Xcode. In your testing scheme choose Gather coverage for required targets. You can only get coverage for targets in your workspace.
The way coverage report is collected for UI tests is the same as for unit tests.
Even if you interact with all UI elements in your app, some code might be uncovered by tests. If you aim to increase your coverage, add additional tests to execute previously uncovered code.
I'm running a Maven multi-module project, and using Sonar Runner to analyze the project for SonarQube 6.3. This project contains both unit and integration tests in every modules. I succeeded generating reports for UT and IT in target/jacoco-ut.exec and target/jacoco-it.exec.
I think analysis parameters for Sonar Runner are good, as I can see both reports are processed and merged during analysis.
From SonarQube 6.3, there's no difference anymore between unit tests and integration tests, though the only measure reported is "Unit tests", which suggests integration tests are ignored.
When I look at the coverage measures in SonarQube, I'm surprised because the number of tests reported is not the sum of the number of unit tests and the number of integration tests. Integration tests are not listed in the measures. To me, if both unit tests and integration tests were merged in SonarQube, I understand there shall both appear in measures, but that's not the case.
I can't find anything in SonarQube documentation about integration tests inclusion in measures. There are only notes that they are merged during analysis, though I don't see anything about my integration tests in the coverage measures.
How can I see integration tests and unit tests in coverage measures?
All tests are now merged into "Coverage", so those numbers include the sum of UT and IT coverage. However prior to the merger there were no metrics about integrations tests themselves (test count, duration, errors, &etc) so there was nothing there to merge.
In fact, metrics about tests (count, errors...) really aren't seen as relevant in general and remain in the system only because they've been grandfathered.
I am using SonarQube 4.3 version to show Unit tests and other code metrics analysis results. Now I am looking for incremental preview or mode for my analysis results as described here.
I am looking to see whether the last code checked-in has a 100% unit test coverage, no security violations and has other quality gates passed.
Currently sonar gives overall combined report for all the files within project. Can we have one for the changed files in the last build as well?
I have a TFS build in VS2010. Following the build unit tests are executed. In the Build summary it tells me that "1 test run(s) completed - 100% average pass rate" but below this it states "No tests were impacted". I guess Impacted Tests relate to functionality providing the ability to only run tests that were impacted by code checked in? Is this correct and since I'm happy to run all tests following each build, how can I turn off the "No tests were impacted" message?
Thanks,
Rob
You're right about test impact analysis - it looks at the changesets since the last build and which tests intersect with that code. To disable test impact analysis for a build:
Right-click your build definition in Team Explorer and choose Edit...
Go to the Process category and expand the Advanced section
Set "Analyze Test Impact" to False
You can also set this value on a per-build basis - queue a manual build and the setting is available via the Parameters tab.
I am writing some unit tests for one of my DLL libraries.
The 'Code Coverage Results' pane shows a breakdown of the assemblies covered and tested.
For some strange reason - my test project itself appears in the coverage results ! (at approx. 90% covered).
This seems stupid... what's the deal with this ?
The reason the percentage is so high is that projects for code coverage are instrumented to keep track of which lines are hit by a test run, since you are running the tests from this project, almost all lines of code in the project will be run.
You can choose which projects/DLLs to collect Coverage statistics on in the Test Settings.
So if you don't need to capture stats on the test project (which you shouldn't really), you can simply remove this project from the settings you're using for coverage.
See http://msdn.microsoft.com/en-us/library/ms182534.aspx (steps 5 - 7 in particular) for more details.