I need to configure the PMD maven plugin to scan test source code with less strict checks comparing to production code.
Is there a way to do it?
You'll probably need to configure two separate executions for the plugin. This allows you to use different configurations. Once execution checks only production code (<includeTests>false</includeTests>) and the other checks test-code only (<includeTests>true</includeTests> and <excludeRoots><excludeRoot>${basedir}/src/main/java</excludeRoot></excludeRoots>).
If you share your solution, we can add it as an example on https://maven.apache.org/plugins/maven-pmd-plugin/
Another way to do it is to use the same code quality rules for all your code, avoiding to make compromises because it's "just test code". While test code doesn't run in production, it's used to test production code, so you might consider the test code then as your weakest link if you use less strict checks...
Here's the complete plugin configuration for this scenario:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>3.14.0</version>
<configuration>
<includeTests>false</includeTests>
<rulesets>
<ruleset>config/pmd/pmdMain.xml</ruleset>
</rulesets>
<linkXRef>false</linkXRef>
</configuration>
<executions>
<execution>
<id>pmd-main</id>
<phase>verify</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
<execution>
<id>pmd-test</id>
<phase>verify</phase>
<goals>
<goal>pmd</goal>
<goal>check</goal>
</goals>
<configuration>
<targetDirectory>${project.build.directory}/pmdTest/</targetDirectory>
<includeTests>true</includeTests>
<excludeRoots>
<excludeRoot>${basedir}/src/main/java</excludeRoot>
</excludeRoots>
<rulesets>
<ruleset>config/pmd/pmdTest.xml</ruleset>
</rulesets>
</configuration>
</execution>
<execution>
<id>cpd</id>
<phase>verify</phase>
<goals>
<goal>cpd-check</goal>
</goals>
</execution>
</executions>
</plugin>
It uses the ruleset config/pmd/pmdMain.xml for the main code without tests. This is configured directly at the plugin level.
The ruleset config/pmd/pmdTest.xml is used for the test code only. This is configured in the execution with id pmd-test.
It has one downside though: PMD is run three times.... twice for the main code, and once for the test code. The reason is that "pmd:check" triggers automatically "pmd:pmd", but it uses only the standard configuration (e.g. ignored the lifecycle/execution id).
The execution "pmd-test" calls once "pmd:pmd" for the test code (which creates target/pmdTest/pmd.xml), then calls pmd:check - which itself calls pmd:pmd and uses the default configuration - and then runs pmd:check actually, which uses the execution configuration and uses target/pmdTest/pmd.xml to decide whether to fail the build.
Related
I'm currently evaluating Sonarqube 6.3 (a big upgrade from my current 5.5 instance) and I'm getting confused trying to work out the functionality of the sonar.test.exclusions setting.
There's this question: Sonar Maven Plugin: How do I exclude test source directories? which seems to indicate that it is used to exclude test files from analysis (which is what I'm after - I don't want my sonar ruleset run over my unit tests). The documentation https://docs.sonarqube.org/display/SONAR/Narrowing+the+Focus also indicates that it is used to 'exclude unit test files' (perhaps this can be expanded upon to make it clearer?)
Thing is, when I add sonar.test.exclusions with a value of **/src/test/** and then run my analysis, I'm still getting code smells and the like being found for:
Foo/src/test/java/foo/bar/BarTest.java
Foo/src/test/java/lah/LahTest.java
etc.
When I use sonar.exclusions instead, they don't show up. Why is sonar.test.exclusions not doing what I expect?
First of all: if you have a Maven project, you should use the scanner for Maven (mvn sonar:sonar). It will simplify your configuration, and will automatically register src/test/java folder as a test directory.
Now if you want to do the configuration manually, or understand what is going on under the hood, here is the explanation: SonarQube scanner work with 2 sets of files, main and test. Main source files are configured using the property sonar.sources. Test source files are configured using sonar.tests.
On top of that, you can filter some content using the sonar.[test.]exclusions properties.
In you case your problem is that Foo/src/test/java/foo/bar/BarTest.java seems to be considered as a main source file. That's why sonar.test.exclusions has no effect.
Using maven with verfication goal (mvn clean verify sonar:sonar install), I have used this configuration without problems:
...
<properties>
....
<sonar.exclusions>
**/generated/**/*,
**/model/**/*
</sonar.exclusions>
<sonar.test.exclusions>
src/test/**/*
</sonar.test.exclusions>
....
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.coverage.exclusions>
**/generated/**/*,
**/model/**/*
</sonar.coverage.exclusions>
<jacoco.version>0.7.5.201505241946</jacoco.version>
....
</properties>
....
Coverage exclusion configuration, inside properties (up) and jacoco plugin configuracion:
.....
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>target/jacoco.exec</dataFile>
<outputDirectory>target/jacoco-ut</outputDirectory>
</configuration>
</execution>
</executions>
<configuration>
<systemPropertyVariables>
<jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</build>
....
Is it possible to run exec-maven-plugin before maven-surefire-plugin, what I observe during my run is maven-surefire-plugin is executing first even though the sequence in tag is second. My scenario is to execute JAVA CLASS (using exec-maven-plugin ) which generates my testng.xml and can run that using (maven-surefire-plugin).
First of all, if you have an execution of the exec-maven-plugin bound to the test phase, it is normal that this execution is performed after the one of the maven-surefire-plugin. The reason is that you're likely dealing with a project of packaging jar, which has a default binding of the Surefire Plugin to the test phase. This default execution is always the first one invoked, regardless of where the plugin is declared in the POM. In the logs, you will spot this execution with an id of default-test.
There is a way to perform actions before the tests are run by leveraging the phases invoked before the phase test. In your case, your goal is to generate a test resource, the testng.xml, so it would be appropriate to use the generate-test-resources phase, whose purpose is to create resources that are needed for the tests. Therefore, you just need to specify
<phase>generate-test-resources</phase>
to the execution of the exec-maven-plugin generating the testng.xml.
Then, you can use the generated testng.xml with the suiteXmlFiles element, see Using Suite XML Files
This is how I have implemented:
I have added test script/java main class, I want to execute before Cucumber Test Suite, in following folder:
enter image description here
Added following in POM.xml in ...
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/test/java/BeforeSuite</source> <!-- source folder where Before Suite scripts are saved -->
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<executions>
<execution>
<id>before-test-suite-scripts</id>
<phase>generate-test-resources</phase>
<goals>
<goal>java</goal>
</goals>
<configuration>
<mainClass>BeforeSuite.HelloBeforeSuiteScript</mainClass> <!-- <packagename>.<className> -->
</configuration>
</execution>
</executions>
</plugin>
When running mvn clean verify, before test suite script will run prior the test suite execution.
enter image description here
I have several levels of nested Maven projects, where every module can participate in the global integration tests. To have a global, multi module coverage, I've configured jacoco to use and share the same file accross modules, using the Maven variable ${session.executionRootDirectory}:
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent-integration</goal>
</goals>
<configuration>
<propertyName>jacoco.failsafeArgLine</propertyName>
<destFile>${session.executionRootDirectory}/target/jacoco-it.exec</destFile>
</configuration>
</execution>
This way, the same data file is used by each module, no matter how deep it is nested in the submodules. I've checked, a correct data file is generated by jacoco when launching "mvn clean install".
Now the problem appears when launching mvn sonar:sonar. It seems that the plugin can not replace that variable with the real path. I can see the following in the logs
[INFO] JaCoCoItSensor: JaCoCo IT report not found: /home/tomcat/.jenkins/jobs/MYJOB/workspace/${session.executionRootDirectory}/target/jacoco-it.exec
It doesn't work better when using #{session.executionRootDirectory}.
Any workaround?
Following a comment in this bug report at SonarSource, advising to use the following configuration:
<plugin>
<groupId>com.github.goldin</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>0.2.5</version>
<executions>
<execution>
<id>set-sonar.jacoco.reportPath</id>
<goals>
<goal>set-properties</goal>
</goals>
<phase>initialize</phase>
<configuration>
<rawProperties>
sonar.jacoco.itReportPath = ${session.executionRootDirectory}/target/jacoco-it.exec
</rawProperties>
<addDollar>true</addDollar>
</configuration>
</execution>
</executions>
</plugin>
... which was unfortunately not compatible with Maven 3.1+, I've used and built from sources that fork, and then I was able to make everything work correctly with Maven 3.2.3.
I have a project using jacoco version 0.7.1.201405082137 and maven 3.0.5. In the project I have some unit tests, and some tests created using arquillian.
To distinguish between the unit tests and integration ones, I created two junit categories: one called FastTest and another called SlowTest.
In the maven profile that I use to run all tests I have this plugins configured:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<groups>SlowTest,FastTest</groups>
<systemPropertyVariables>
<arquillian.launch>wildfly_8_x</arquillian.launch>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${version.jacoco}</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
When I leave both categories in I get only the coverage for the tests annotated with SlowTest. But all the tests run. If I run only the ones annotated with FastTest I get their correct coverage too.
How can I set up jacoco to get the correct coverage when running both kinds of tests?
Small tip:
<groups> tag takes full class name. So my question: is SlowTest.class interface placed in the default package? If not you should provide full path, something like: <groups>com.mycompany.project.SlowTest</groups>
And little advice:
Good practice is to distinguish unit and integration tests - and thus run them separately. Maven accomplishes this by two plugins: maven-surefire-plugin and maven-failsafe-plugin.
First is designed to run unit tests with mvn test. Second is designed to run your integration tests with mvn failsafe:integration-test. This answer may be useful to shed some light.
We're checking our code using the PMD 'check' goal that is bound to the 'verify' life cycle. (http://maven.apache.org/plugins/maven-pmd-plugin/examples/violationChecking.html)
For the 'pmd' goal you can add 'excludes' and 'excludeRoots' but not for the 'check' goal.
How does one exclude eg. generated sources directories?
You need to do the pmd:pmd first and afterwards do a pmd:check. You can configure that simply by using. Bind that to a particular lifecycle-phase which is before verify. For example into package or pre-integration-test phase.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>2.7.1</version>
<executions>
<execution>
<goals>
<goal>pmd</goal>
<goal>check</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
The check goals (check, cpd-check are exactly intended to fail a build if there are some violations. So you can define some exceptions for the pmd goal which folders should be included/excluded.