I'm using Surefire 2.16 and TestNG 6.8.5. The configuration in my pom.xml is really simple : no configuration (so Surefire must find out by itself the tests classes).
But I have 500 unit tests in this project and the generation of the "old" directory content is very very slow (it creates a sub-directory for each class/suite). The tests execution only takes 15 seconds, but with the old directory it raises to more than 800 seconds.
So I would like avoid this "old" directory (but keep the other generated files). I tried the parameter use usedefaultlisteners=false, but I loose all the reports.
Someone to tell me how I can do this ?
Thank you.
As per the Surefire docs listeners may be added by setting the listener property as a CSV list of Class names.
You can understand what the default listeners are by looking at the TestNG source. Method is initializeDefaultListeners. Pick just the listeners that you need and add them yourself.
Related
I'm using Gradle sonarqube plugin and I need to exclude all test sources from the sonar analysis (main goal is to filter out unit test classes from the issues/code smells report)
To do so, I have used the dedicated sonar.test.exclusions property as follow, to exclude the whole src/test directory from analysis
sonarqube {
properties {
property("sonar.exclusions" , "")
property("sonar.test.exclusions" , "src/test/**/*.java")
// other sonar properties, omitted
}
}
This works as expected (test sources are filtered out) BUT : when this property is set, sonar does not compute/report number of unit tests correctly.
See simple example for a very basic project: 2 main source files, 1 test source file containing 2 Junit tests (and also containing some issues I don"t want to see in report)
Without exclusions:
Sonar properly reports my 2 unit tests, but it also includes code smells from the unit test class
With exclusions:
Now, code smells from the unit test are properly filtered, but I lost the Unit test count information
Notes:
using Gradle 6.7, sonarqube plugin version 3.0, and sonar server Community EditionVersion 8.4.2
also tried with the property sonar.exclusions : same problem
all other sonar properties are correctly set and have same values in both scenarios : specially sonar.tests, sonar.java.test.binaries, sonar.junit.reportPaths, sonar.jacoco.reportPath
Any idea how to configure the sonarqube plugin, to exclude properly test sources, while keeping Unit tests information available?
I agree with Chriki's answer but only the first part : using sonar.test.exclusions is not the good way.
I disagree with the last part : using sonar.issue.ignore.multicriteria is totally possible with Gradle
https://www.baeldung.com/sonar-exclude-violations#using-sonar-projectproperties
Try something like this (not tested from my end though) :
sonarqube {
properties {
property "sonar.issue.ignore.multicriteria", "e1"
property "sonar.issue.ignore.multicriteria.e1.resourceKey", "src/test/java/**/*"
property "sonar.issue.ignore.multicriteria.e1.ruleKey", "*"
}
}
What you’re after doesn’t seem to be possible from a Gradle configuration. Let me elaborate on how I came to that conclusion.
In a (admittedly very) old thread on the Sonarqube mailing list from 2013, somebody asked the same question (albeit for Maven). A Sonarqube consultant has answered it as follows:
When you exclude a file (or a test file) with sonar.exclusions (or sonar.test.exclusions), the file is just ignored by SonarQube. The source code is not imported, no metrics are computed on this file (such as number of tests).
To ignore some specific issues on specific files, see http://docs.codehaus.org/display/SONAR/Narrowing+the+Focus#NarrowingtheFocus-IgnoreIssues. To ignore some files to be taking into account for code coverage computation, see http://docs.codehaus.org/display/SONAR/Narrowing+the+Focus#NarrowingtheFocus-IgnoreCodeCoverage and so on.
The current documentation link corresponding to the first (now broken) one from the quote is this: https://docs.sonarqube.org/latest/project-administration/narrowing-the-focus/#header-3 (or for version 8.4) It’s about ignoring “issues on certain components and against certain coding rules” – which is what you’re after, if I’m not mistaken. However, these docs state right at the beginning:
Note that the properties below can only be set through the web interface because they are multi-valued.
Here “web interface” is meant as opposed to a (Gradle) build configuration for example. In fact, the previously mentioned Sonarqube consultant explicitly states this for Maven, too:
The property is sonar.issue.ignore.multicriteria. I think I read somewhere that you have to do these through the GUI because they are multi-valued? Is that the case? If not, how would I write it as an entry in the pom.xml? We like to keep all config in the one place (pom.xml).
It's unfortunately not possible.
The mailing list discussion is a bit longer and gives more insight, so it may be worth a read in full. It also sheds some more light on the sonar.test.exclusions property for which I could otherwise not find any good documentation.
I am using Lombok 1.18.2 and JaCoCo 0.8.3, which theoretically recognize/ignore lombok annotations. I did a test, added the lombok.addLombokGeneratedAnnotation = true param in my lombok.config and then I see the generated annotation on my target classes decompiled code.
But I am surprised to open jacoco.exec file and see that things like #Getter increase the "Total Probes" values, but keep untouched the "Executed Probes" one.
Is this expected?
If so, how can SonarQube make smart use of it, if by pointing it to the jacoco.exec files, there is only information on total/hit probes? How can Sonar tell one of this probes has something to do with lombok?
I am using Lombok 1.18.2 and JaCoCo 0.8.3, which theoretically recognize/ignore lombok annotations.
Not theoretically, but practically.
Not "ignore lombok annotations", but methods annotated with #lombok.Generated, or even more precise quoting entry from 0.8.3 changelog:
Classes and methods annotated by annotation whose retention policy is runtime or class and whose simple name contains "Generated" (previously equality was required) are filtered out during generation of report
But I am surprised to open jacoco.exec file and see that things like #Getter increase the "Total Probes" values, but keep untouched the "Executed Probes" one. Is this expected?
Yes, this is expected. Filtering of methods doesn't happen during instrumentation of classes (insertion of probes). exec file is not final report, it contains raw information about all probes inserted in class. Filtering of methods happens during generation of report (analysis of exec and class files), i.e. during execution of report JaCoCo Ant Task for example.
If so, how can SonarQube make smart use of it, if by pointing it to the jacoco.exec files, there is only information on total/hit probes? How can Sonar tell one of this probes has something to do with lombok?
SonarQube embeds JaCoCo as a library and use it to analyze exec and class files for generation of their report.
And that's why JaCoCo release announcements usually contain following statement, e.g. for 0.8.2:
As before - please note
0.8.2 version of integrations developed as part of JaCoCo project by us (Ant Tasks, Maven Plugin and Command Line Interface) provide Java 11 support and new filters,
tools that directly read exec files (which is not a final report) and embed JaCoCo as a library for generation of report will provide ability to analyze Java 11 class files and to use new filters only after they updated to this version of JaCoCo.
So please follow/wait/etc respective vendors such as
SonarQube - https://jira.sonarsource.com/browse/SONARJAVA-2876 , or try to use new plugin that reads XML report - https://github.com/SonarSource/sonar-jacoco
I have a #Service class where i'm caching some table data. I don't want those queries to run while building mvn install. Is there a way to ignore the file while building and it only execute when i start the server ?
It's a spring-boot application.
Here is background of my issue. I have initialized the spring boot app from http://start.spring.io/ site, which actually adds dummy application test file with SpringBootTest annotation and default contextLoads() with Test annotation, with an intention to initialize and execute all test cases, which needs to initialize and execute all code base. In my opinion this is not required, as we can have respective Test classes per controller/manager, which will give more controlled environment to hook up your Test setups and executions.
I have removed the default application Test file and included respective test classes for code coverage and quality. This way my beans are not executed at server startup time rather build time.
Currently, we are using Sonar to check the quality of our production code. I would Like to check the quality of the test Code either.
How can I do it ?
I tried to change the "sonar.sources" to include test, but in this case i have an error : test folders are defined twice, one in test and one in source.
May be I can configure sonar / maven / jenkins to run 2 analysis : first one, the src code, with the test coverage, thant the tests (only quality, no coverage). But I also need to have everything run in a single jenkins job and displayed in a single Sonar Project.
And I don't know where to configure (pom ? sonar ? jenkins ?) I found some things about profiles, but it seems to be deprecated for my version.
I'm using Sonar 5.1, run from a jenkins job.
our Sonar properties in the pom are :
<project.testresult.directory>${project.build.directory}/test-results</project.testresult.directory>
<run.addResources>false</run.addResources>
<sonar-maven-plugin.version>2.6</sonar-maven-plugin.version>
<sonar.exclusions>src/main/webapp/assets/**/*.*,
src/main/webapp/bower_components/**/*.*,
src/main/webapp/dist/**/*.*</sonar.exclusions>
<sonar.jacoco.itReportPath>${project.testresult.directory}/coverage/jacoco/jacoco-it.exec</sonar.jacoco.itReportPath>
<sonar.jacoco.reportPath>${project.testresult.directory}/coverage/jacoco/jacoco.exec</sonar.jacoco.reportPath>
<sonar.java.codeCoveragePlugin>jacoco</sonar.java.codeCoveragePlugin>
<sonar.javascript.jstestdriver.reportsPath>${project.testresult.directory}/karma</sonar.javascript.jstestdriver.reportsPath>
<sonar.javascript.lcov.reportPath>${project.testresult.directory}/coverage/report-lcov/lcov.info</sonar.javascript.lcov.reportPath>
<sonar.sources>${project.basedir}/src/main/</sonar.sources>
<sonar.surefire.reportsPath>${project.testresult.directory}/surefire-reports</sonar.surefire.reportsPath>
<sonar.tests>${project.basedir}/src/test/</sonar.tests>
In the latest versions of the Java plugin there are some rules to check tests, but they are test-specific rules. I.E. they check if your #Test methods contain assertions &etc. You simply need to add those rules to your profile.
If, however, you're talking about running "code rules" on your tests, then your best bet is probably to define another project where the tests are treated as sources. Since you've already noted the difficulties of doing that with Maven, I would use SonarQube Scanner for this separate, second analysis.
I am developing TestNG classes in a Maven Project and executing the same. I understand the fact that ,for the maven-surefire-plugin the classname should be in either of the following formats
*Test.java
Test*.java
*TestCase.java
But incase I would NOT be naming my class as per this convention , does the plugin pick up the test classes depending upon the Java MetaData , as is the case with TestNG.
If not then is defining the names of the classes in testng.xml the way out or is there some other way ??
Thanks in Advance,
Vivek
As per this documentation, you can explicitly include and exclude tests, as well as specify complex regex patterns. You could try this in lieu of maintaining a testng.xml.