I am using Lombok 1.18.2 and JaCoCo 0.8.3, which theoretically recognize/ignore lombok annotations. I did a test, added the lombok.addLombokGeneratedAnnotation = true param in my lombok.config and then I see the generated annotation on my target classes decompiled code.
But I am surprised to open jacoco.exec file and see that things like #Getter increase the "Total Probes" values, but keep untouched the "Executed Probes" one.
Is this expected?
If so, how can SonarQube make smart use of it, if by pointing it to the jacoco.exec files, there is only information on total/hit probes? How can Sonar tell one of this probes has something to do with lombok?
I am using Lombok 1.18.2 and JaCoCo 0.8.3, which theoretically recognize/ignore lombok annotations.
Not theoretically, but practically.
Not "ignore lombok annotations", but methods annotated with #lombok.Generated, or even more precise quoting entry from 0.8.3 changelog:
Classes and methods annotated by annotation whose retention policy is runtime or class and whose simple name contains "Generated" (previously equality was required) are filtered out during generation of report
But I am surprised to open jacoco.exec file and see that things like #Getter increase the "Total Probes" values, but keep untouched the "Executed Probes" one. Is this expected?
Yes, this is expected. Filtering of methods doesn't happen during instrumentation of classes (insertion of probes). exec file is not final report, it contains raw information about all probes inserted in class. Filtering of methods happens during generation of report (analysis of exec and class files), i.e. during execution of report JaCoCo Ant Task for example.
If so, how can SonarQube make smart use of it, if by pointing it to the jacoco.exec files, there is only information on total/hit probes? How can Sonar tell one of this probes has something to do with lombok?
SonarQube embeds JaCoCo as a library and use it to analyze exec and class files for generation of their report.
And that's why JaCoCo release announcements usually contain following statement, e.g. for 0.8.2:
As before - please note
0.8.2 version of integrations developed as part of JaCoCo project by us (Ant Tasks, Maven Plugin and Command Line Interface) provide Java 11 support and new filters,
tools that directly read exec files (which is not a final report) and embed JaCoCo as a library for generation of report will provide ability to analyze Java 11 class files and to use new filters only after they updated to this version of JaCoCo.
So please follow/wait/etc respective vendors such as
SonarQube - https://jira.sonarsource.com/browse/SONARJAVA-2876 , or try to use new plugin that reads XML report - https://github.com/SonarSource/sonar-jacoco
Related
I have upwardes of 200 classes in a Maven java application and I use Jenkin to manage the test and build.
Jenkins continued to report a nice high coverage figure but when I dig into the report I notioced some classed hadf not been analysed.
These classes all took data from a source, did some work created a message and then added it to a database table via a sertvice and JPA repository. I am testing using a mixture of mocking and argument capture.
I found that it I added a "simple method" to the class (add two numbers and return the sum) then PIT analysed and reported on all the methods in the class.
I tried with and without an interface, in different folder and made minor changes to the methods and the mocks.
Switching on verbose loggion in PIT did not help.
I'm using Gradle sonarqube plugin and I need to exclude all test sources from the sonar analysis (main goal is to filter out unit test classes from the issues/code smells report)
To do so, I have used the dedicated sonar.test.exclusions property as follow, to exclude the whole src/test directory from analysis
sonarqube {
properties {
property("sonar.exclusions" , "")
property("sonar.test.exclusions" , "src/test/**/*.java")
// other sonar properties, omitted
}
}
This works as expected (test sources are filtered out) BUT : when this property is set, sonar does not compute/report number of unit tests correctly.
See simple example for a very basic project: 2 main source files, 1 test source file containing 2 Junit tests (and also containing some issues I don"t want to see in report)
Without exclusions:
Sonar properly reports my 2 unit tests, but it also includes code smells from the unit test class
With exclusions:
Now, code smells from the unit test are properly filtered, but I lost the Unit test count information
Notes:
using Gradle 6.7, sonarqube plugin version 3.0, and sonar server Community EditionVersion 8.4.2
also tried with the property sonar.exclusions : same problem
all other sonar properties are correctly set and have same values in both scenarios : specially sonar.tests, sonar.java.test.binaries, sonar.junit.reportPaths, sonar.jacoco.reportPath
Any idea how to configure the sonarqube plugin, to exclude properly test sources, while keeping Unit tests information available?
I agree with Chriki's answer but only the first part : using sonar.test.exclusions is not the good way.
I disagree with the last part : using sonar.issue.ignore.multicriteria is totally possible with Gradle
https://www.baeldung.com/sonar-exclude-violations#using-sonar-projectproperties
Try something like this (not tested from my end though) :
sonarqube {
properties {
property "sonar.issue.ignore.multicriteria", "e1"
property "sonar.issue.ignore.multicriteria.e1.resourceKey", "src/test/java/**/*"
property "sonar.issue.ignore.multicriteria.e1.ruleKey", "*"
}
}
What you’re after doesn’t seem to be possible from a Gradle configuration. Let me elaborate on how I came to that conclusion.
In a (admittedly very) old thread on the Sonarqube mailing list from 2013, somebody asked the same question (albeit for Maven). A Sonarqube consultant has answered it as follows:
When you exclude a file (or a test file) with sonar.exclusions (or sonar.test.exclusions), the file is just ignored by SonarQube. The source code is not imported, no metrics are computed on this file (such as number of tests).
To ignore some specific issues on specific files, see http://docs.codehaus.org/display/SONAR/Narrowing+the+Focus#NarrowingtheFocus-IgnoreIssues. To ignore some files to be taking into account for code coverage computation, see http://docs.codehaus.org/display/SONAR/Narrowing+the+Focus#NarrowingtheFocus-IgnoreCodeCoverage and so on.
The current documentation link corresponding to the first (now broken) one from the quote is this: https://docs.sonarqube.org/latest/project-administration/narrowing-the-focus/#header-3 (or for version 8.4) It’s about ignoring “issues on certain components and against certain coding rules” – which is what you’re after, if I’m not mistaken. However, these docs state right at the beginning:
Note that the properties below can only be set through the web interface because they are multi-valued.
Here “web interface” is meant as opposed to a (Gradle) build configuration for example. In fact, the previously mentioned Sonarqube consultant explicitly states this for Maven, too:
The property is sonar.issue.ignore.multicriteria. I think I read somewhere that you have to do these through the GUI because they are multi-valued? Is that the case? If not, how would I write it as an entry in the pom.xml? We like to keep all config in the one place (pom.xml).
It's unfortunately not possible.
The mailing list discussion is a bit longer and gives more insight, so it may be worth a read in full. It also sheds some more light on the sonar.test.exclusions property for which I could otherwise not find any good documentation.
We have a large Maven project with about 50 modules. Today we have a good setup where we get a combined JaCoCo coverage for the whole project. We also use the final binary generated by JaCoCo for custom coverage logic.
We would like to implement a more strict coverage policy where only tests in the same module contribute to the overall coverage. We want every module to be responsible for its own tests and coverage.
For example. Module A depends on module B. A has code that makes use of code in B. When tests in A is run, we do not want to have coverage from any class in B recorded.
Is this setup possible? Trying around, I find that JaCoCo has many options for filtering when generating the final report, but I'd like to do this in the first step, when the coverage is collected.
Eventually, I'd like to end up with a JaCoCo exec file in each module's target directory that only has classes recorded from its own module. It would be perfectly fine if we had to do some configuration on a module-by-module basis like having an explicit package expression for inclusion.
Trying around, I find that JaCoCo has many options for filtering when generating the final report, but I'd like to do this in the first step, when the coverage is collected.
There is also options for filtering during collection - see includes and excludes options of JaCoCo Java Agent and corresponding options of prepare-agent in jacoco-maven-plugin.
Build of JaCoCo itself uses them exactly for this.
I'm wondering if it is currently possible to ignore the equals and hashcode method for the sonar test coverage? I have heard about the block exclusion, but it didn't work.
(assuming you're using jacoco for coverage reporting)
If you're not using Lombok, you might try adding the #Generated annotation to your methods you want skipped. I'm not sure this will work - but worth a shot!
If you're using Lombok [like I was], here's a solution from Rainer Hahnekamp that marks the code as #Generated, which makes jacoco ignore the methods, and in turn makes sonarqube display a higher coverage percentage.
Luckily, beginning with version 0.8.0, Jacoco can detect, identify, and ignore Lombok-generated code. The only thing you as the developer have to do is to create a file named lombok.config in your directory’s root and set the following flag:
lombok.addLombokGeneratedAnnotation = true
This adds the annotation lombok.#Generated to the relevant methods, classes and fields. Jacoco is aware of this annotation and will ignore that annotated code.
Please keep in mind that you require at least version 0.8.0 of Jacoco and v1.16.14 of Lombok.
https://www.rainerhahnekamp.com/en/ignoring-lombok-code-in-jacoco/
Our project uses Maven as the build tool and we are using Sonar to track quality. JUnit tests are executed by SureFire and the results are displayed in Sonar. We've added some JavaScript tests which are run by the jasmine-maven-plugin and want to include these results in the Sonar project.
The plugin generates a JUnit style XML report. How should we go about including the XML report in Sonar? Do we want to merge the XML reports as part of the build maybe?
You need to be sure the format generated by Jasmine is compliant with the JUnit XML format expected by the JavaScript Plugin : http://docs.sonarqube.org/display/PLUG/JavaScript+Unit+Tests+Execution+Reports+Import
You can also have a look at the format supported by the Generic Test Coverage Plugin : http://docs.sonarqube.org/display/SONAR/Generic+Test+Coverage
You don't need to merge the XML reports, there are different sonar.* properties to feed depending on the way you want to load your data: thru the JavaScript Plugin or the Generic Test Coverage. As a consequence, you don't need to run SonarQube analysis twice.