I use sonarqube to do the line coverage analysis, but the reporting results are fallacious.
For example for the if method below:
if(a != null ){
system.out.print("Hello");
}
the if condition is reported as NOT covered by unit test，which means not executed .
BUT，the logical system.out.print("Hello") inside is reported as covered by unit test. that is illogical, right?
This is not really a question of SonarQube but of your coverage engine. SonarQube only relays what your coverage engine reported.
That said, you're likely misinterpreting the markers in the SonarQube interface, although without a screenshot it's hard to know for certain. If you're seeing a diagonally striped marker next to the if, then SonarQube is telling you that the line is partially covered. That is, there are multiple paths through the code and only some of them are taken in your testing. Specifically, it sounds like you are testing the path where the condition is true. I would guess you're not testing the path where the condition is false.
Related
I am trying to add a quality gate in sonarqube, to fail if the new code coverage % drops below the overall code coverage.
Anyone have tried this ?
You're going to have to be more specific. I'm not sure what "overall" means. Are you referring to the "after merge" value? It's also unclear whether you're referring to a "base project" or a pull request or branch.
If you're looking to ensure that the "after merge" coverage on a branch scan satisfies a required threshold, I'm pretty sure you can't do that out of the box with SonarQube, but you should also specify what version of SonarQube you're using.
I implemented a check for whether the "after merge" coverage value of a branch scan satisfies our required threshold, but I had to do it in script code, using the SonarQube Web API. I had it obtain the project's quality gate, along with the resulting coverage from the scan, and if it's below the required number, I have it fail the build with an appropriate message. There's no way to mark the scan itself to be in violation, but at least we can make the build fail.
i am a Beginner with SonarQube and really tried to google and read a lot of community pages to understand which functions SonarQube offers.
What i dont get is: What does the test coverage in SonarQube refer to?
If it says for example that the coverage on New Code is 30% what does new code mean?
And when does SonarQube say that a issue is a bug? Is the analyzed code compared to a certain standard in order for SonarQube to say that there is a bug?
I hope someone with more knowledge about SonarQube can help me understand it. Thank you very much
Test coverage (also known as code coverage) corresponds to the proportion of the application code (i.e., code without test and sample code) that is executed by test cases out of all application code of the code base.
SonarQube does not compute code coverage itself. Instead coverage is computed and uploaded by external code coverage tools (e.g., cobertura, JaCoCo). SonarQube presents code coverage at different levels (e.g., line coverage, condition coverage); see https://docs.sonarqube.org/latest/user-guide/metric-definitions/#header-9.
Coverage on new code refers to the proportion of code that is both covered and added (or modified) since a certain baseline out of all added and changed code since the same baseline. The baseline can, for example, be the previously analyzed code state or the code state of the previous commit. That is, this metric expresses how extensively changes have been tested. Note that 100% coverage does not mean that code has been perfectly tested; it just says that all code has been executed by test cases.
Issues in SonarQube do not necessarily represent bugs. Usually most issues are actually not bugs but are problems affecting code maintainability in the long term (e.g., code duplications) or violations of best practices. Still, some issues can represent bugs (e.g., potential null-dereferences, incorrect concurrency handling).
Note that an issue can also be false a positive and therefore not be a problem at all.
Most issues are identified with static code analysis by searching the code structure for certain patterns. Some can be uncovered by simple code searches (e.g., violation of naming conventions). Other analyses / issue classes may additionally need data-flow analyses (null-dereferences) or require byte-code information.
SonarQube 7.3 it has inbuilt support for golang where I have found at least 2 issues :-
It does not exclude *_test.go automatically from coverage. In unit tests it also picks up IP addresses and asks them to be made configurable (not constant which also does not fix the error).
It counts structs and const as not covered lines and hence has a significantly lower % covered than the go coverage tool itself making it a bad use case. For example in a medium size project it reports 40% coverage against go tools 70%
Apart from commenting them all to be sonar exclude or putting constants and structs in a common exclude pattern file is there something else that can be done? Is there a plan to address these in a later version of SonarQube?
For now (SonarQube 7.4), SonarGo analyzer does not identify automatically *_test.go as test files. This is a missing feature, this is why the SonarGo documentation describes how to properly identify test files through settings:
sonar.test.inclusions=**/*_test.go
Without proper test identification, the coverage result will be wrong and the analysis result could raise issues that do not make sense (like hard-coded IP addresses in tests).
About the coverage accuracy (for files that are not test files), there's two cases:
If a file has entries related to it in the coverage report, the covered lines shown in SonarQube should match exactly the ones in the report, otherwise it's a major bug. But the percentage shown by go tools (ranges coverage) could be slightly different that the percentage shown in SonarQube (lines coverage), e.g: +2%.
If a file is not in the coverage report, SonarGo generate a 0% coverage based on its definition of executable line of code. If there's a difference with go test definition of executable line of code, it's a bug that will disappear once the file is partially covered.
The best way to have those coverage bugs fixed, is to report them at community.sonarsource.com by creating a Report a Bug topic with a small code reproducer.
Number of critical issues are 4 on my project. I have configured a condition 'Critical Issues' with 2 measure in the following order.
1. 'delta since previous analysis' 'is greater than' 0
2. 'Value' 'is greater than' 6
Please find screenshot
I had forcefully introduced a critical error and run sonar. I thought it should fail because of above 1st condition but to my surprise it got passed.
After doing multiple permutations and combinations I got to know that a unique 'Condition' should only be configured once. If a unique 'Condition' is used more than once then one which is configured later is used by Sonar and the one which is configured earlier is ignored.
According to Sonar documentation - http://www.sonarqube.org/quality-gates-shall-your-projects-pass/
'Quality Gates can be thought of in two different ways. At the practical level, they’re collections of what were once called alerts, and are now known simply as “conditions”. At the abstract level, they’re logic gates, AND-ing together all the conditions in the set to determine whether or not your project can pass.'
I could not find any information on internet to prove my findings and even sonar documentation contradicts it. Has anyone faced this issue before? Could someone confirm that a 'Condition' for example 'Critical Issues' should only be configured once.
Multiple conditions on metric(s) are definitely supported for configuring Quality Gates in SonarQube, and the documentation is in line with that.
The behaviour you are observing is most likely due to a bug present in SonarQube 5.2, 5.3, 5.4 , preventing analysis to succeed if there are multiple Quality Gate conditions on same metric, see SONAR-7276 for more details.
I'm using Visual Studio 2010's code coverage feature. I've made several unit tests to test a method, but the code coverage is telling me that three blocks are not getting completely covered. The problem is, I don't see how these blocks can be only partially covered. Notice that the return statements ARE covered, so clearly the branch has been taken. Any ideas?
The answer turned out to be that endDate is nullable. Even though I handle null cases at the top, the code coverage wanted to see the null situation dealt with at each branch with endDate in it.