Java 8 to Java 17 migration + pitest failing - Mutation testing requires a green suite - maven

Recently , the project has been migrated from Java8 to Java17 . The mutation test coverage is also checked for the project and before migration there was no known issues during the command run of mvn clean install org.pitest:pitest-maven:mutationCoverage
After migration , the issue is reported as Execution default-cli of goal org.pitest:pitest-maven:1.7.6:mutationCoverage failed: 6 tests did no
t pass without mutation when calculating line coverage. Mutation testing requires a green suite.
The pom.xml file has
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.6.2</version>
<dependencies>
<dependency>
<groupId>org.pitest</groupId>
<artifactId>pitest-junit5-plugin</artifactId>
<version>0.12</version>
</dependency>
</dependencies>
</plugin>
Note: There are few test cases contain Whitebox.invokeMethod to invoke the private method. There are no issues during the run of the test cases - mvn clean install.
The actual problem occurred during the run of mutation test coverage with the command - mvn clean install org.pitest:pitest-maven:mutationCoverage
I tried to include the option --add-opens java.base/jdk.internal.misc=ALL-UNNAMED in pom.xml file but of no use.
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>
--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
</argLine>
</configuration>
</plugin>
I tried to change the version of pitest-maven as 1.7.3 from 1.6.2 and pitest-junit5-plugin as 0.15 from 0.12. This is also not useful.
Any idea or suggestion to overcome the pitest-maven in Java17 with WhiteBox.invokeMethod in test cases.

The correct version of pitest-junit5-plugin was helpful to get rid of the error.
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.11.0</version>
<groupId>org.pitest</groupId>
<artifactId>pitest-junit5-plugin</artifactId>
<version>1.1.2</version>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.8.2</version>
Along with the junit pitest-junit5-plugin addition , adding --add-opens is also required ,
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.11.0</version>
<dependencies>
<dependency>
<groupId>org.pitest</groupId>
<artifactId>pitest-junit5-plugin</artifactId>
<version>1.1.2</version>
</dependency>
</dependencies>
<configuration>
<argLine>
--add-opens java.base/java.lang=ALL-UNNAMED
</argLine>
</configuration>
</plugin>

Related

Jacoco + Surefire Maven plugins always show 0% coverage

I got a multi-module maven project on which i want to generate a jacoco report with the unit coverage. Currently i am using arqullian surefire and jacoco maven plugins, which both are defined in only one module. The tests run on a JBoss EAP 7.2
Although the jacoco report is generated, i always get a 0% coverage on all my test classes which are deployed as an archive org.jboss.shrinkwrap.api.spec.WebArchive on my JBoss server even though there are several methods covered by the Unit tests.
Important: Other unit tests performed without a deployment, do show some coverage on the report
Below is my maven configuration
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<skip>${skipTests}</skip>
<argLine>${surefireArgLine}</argLine>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.22.2</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.5</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
Below my dependencies used:
<dependency>
<groupId>org.dbunit</groupId>
<artifactId>dbunit</artifactId>
<version>2.7.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.shrinkwrap.resolver</groupId>
<artifactId>shrinkwrap-resolver-api-maven</artifactId>
<version>3.1.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.shrinkwrap.resolver</groupId>
<artifactId>shrinkwrap-resolver-impl-maven</artifactId>
<version>3.1.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.shrinkwrap.resolver</groupId>
<artifactId>shrinkwrap-resolver-spi</artifactId>
<version>3.1.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.junit</groupId>
<artifactId>arquillian-junit-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.junit</groupId>
<artifactId>arquillian-junit-container</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.protocol</groupId>
<artifactId>arquillian-protocol-servlet</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jacoco</groupId>
<artifactId>org.jacoco.core</artifactId>
<version>0.8.5</version>
<scope>test</scope>
</dependency>
Arqulian XML
<?xml version="1.0" encoding="UTF-8"?>
<arquillian xmlns="http://jboss.org/schema/arquillian" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://jboss.org/schema/arquillian http://jboss.org/schema/arquillian/arquillian_1_0.xsd">
<!-- Force the use of the Servlet 3.0 protocol with all containers, as it is the most mature -->
<defaultProtocol type="Servlet 3.0" />
<!-- Example configuration for a remote JBoss EAP 7 instance -->
<container qualifier="jboss" default="true">
<!-- If you want to use the JBOSS_HOME environment variable, just delete the jbossHome property -->
<configuration>
<!-- <property name="javaVmArguments">${jacoco.agent}</property> -->
</configuration>
</container>
</arquillian>
On one of my configurations (not the above), i remember seeing a warning that the ${jacoco.agent} was not used. So i did comment it out again.
When running the mvn clean install goal, in my console logs i see following variables set to the surefireArgLine
[INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (pre-unit-test) # alis-ejb ---
[INFO] surefireArgLine set to -javaagent:C:\\Users\\user\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.5\\org.jacoco.agent-0.8.5-runtime.jar=destfile=c:\\Projects\\Projectname\\ejb-module\\target\\coverage-reports\\jacoco-ut.exec
All my tests are executed normaly and i am able to generate a surefire report command with surefire-report that lists all the unit test executions. I am thinking if am missing a mvn goal for jacoco that calculates the coverage, but it looks that i am still missing something
Update 1: I start to believe that the issue is created due to the usage of Arquilian, and i should use the pre-integration-test phases instead. But not sure yet.
Update 2: I just noticed that for one of my Unit tests i see some coverage. The difference between this unit test and the rest is the one Class containing the tests of which a coverage as shown, are simple Unit tests, whereas for the rest of my Test classes on which i do not get a coverage report from Jacoco a org.jboss.shrinkwrap.api.spec.WebArchive deployment is created. Some more details on this. 90% of my tests, are of this structure. A webarchive is created and deployed on JBoss EAP 7.

JUnit5 run tests by tags

I have a setup Maven+JUnit5+Selenium, my pom.xml https://github.com/13Dima13/G/blob/master/pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<forkCount>4</forkCount>
<reuseForks>false</reuseForks>
<properties>
<includeTags>${tag}</includeTags>
</properties>
</configuration>
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>${junit.platform.version}</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit.jupiter.version}</version>
</dependency>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-logger-api</artifactId>
<version>${surefire-logger-api}</version>
</dependency>
</dependencies>
</plugin>
First i thought that #Tag works fine, because when i put an annotation to my class https://github.com/13Dima13/G/blob/master/src/test/java/com/example/project/TestThreads2.java
#Test
#Tag("smoke")
public void test1() {
open("https://mvnrepository.com/artifact/log4j/log4j/1.2.17");
$("#maincontent > table > tbody > tr:nth-child(1) > th").waitUntil(Condition.appears, 120000);
$("#maincontent > table > tbody > tr:nth-child(1) > th").shouldBe(Condition.text("License"));
assertEquals("Maven Repository: log4j » log4j » 1.2.17", title());
out.println("test1 Passed");
}
and then ran from terminal
mvn test -Dtag=smoke
it executed only the tests which were marked as #Tag("smoke"), so in my case only one test and this made me happy.
But
when I started to use it on my real project I realized that it does not work sometimes.
For example, if my Test.class with tests methods is not placed in the parent project folder (for example inside child of project folder) https://github.com/13Dima13/G/blob/master/src/test/java/com/example/project/test2/GoogleSearch.java, annotation will not work at all.
So annotation #Tag doesn't work for the whole project or did I miss something?
Good example https://ibb.co/gjbSZJ
Give
mvn test -D groups=smoke
a try...
Or, if you like to see results pretty fast (and don't need to have your code recompiled), go for:
mvn surefire:test -D groups=smoke
Note that the space after -D is optional and purely for cosmetic reasons.
N.B. i am also rather new to the java/maven world, but for what it's worth: i never got it to work with -D tag in the first place so far. Would love to know what's up with that (comments linking to an explanation?)
First guess: The name of your test class (test3.java) does not match maven‘s default pattern for test classes and it will therefore be ignored.
The support of JUnit 5 is contained in Maven Surefire Plugin Version 2.22.0
<dependencies>
[...]
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.2.0</version>
<scope>test</scope>
</dependency>
[...]
</dependencies>
and maybe you need to define maven-surefire-plugin like this:
<build>
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<excludes>
<exclude>some test to exclude here</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
So your configuration can be simplified.

Getting cxf-codegen-plugin working on Java 9

I've been trying to get cxf-codegen-plugin working on Java 9 with some resistance. So far I've added java.se.ee to the runtime modules and added the necessary dependencies to maven.
However, when I try to build my sources I get the following error:
DefaultValidationEventHandler: [ERROR]: unexpected element (uri:"http://cxf.apache.org/tools/plugin", local:"databinding"). Expected elements are <{}databinding>,<{}frontend>
Location: node: [databinding: null]
apr. 21, 2018 8:23:57 EM org.apache.cxf.tools.wsdlto.core.PluginLoader loadPlugin
ALLVARLIG: Tools plugin jar:file:/C:/Users/Daniel/.m2/repository/org/apache/cxf/cxf-tools-wsdlto-core/3.2.4/cxf-tools-wsdlto-core-3.2.4.jar!/META-INF/tools-plugin.xml load failed
Any ideas what might be causing this or how to fix it?
Run into the same issue, I ended up changing the plugin configuration and adding the required modules explicitly:
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
<version>3.2.4</version>
<configuration>
<additionalJvmArgs>--add-modules java.xml.bind,java.xml.ws</additionalJvmArgs>
<fork>once</fork>
</configuration>
</plugin>
It seems to work for Java 9/10 but certainly not going to work for Java 11.
As the documentation said you have to start with version 3.2.5 cxf documentation for java 9
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
<version>3.2.5</version>
I tested with java 10:
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
<version>3.2.4</version>
<configuration>
<fork>true</fork>
<additionalJvmArgs>--add-modules java.activation,java.xml.bind,java.xml.ws --add-exports=java.xml.bind/com.sun.xml.internal.bind.v2.runtime=ALL-UNNAMED --add-exports=jdk.xml.dom/org.w3c.dom.html=ALL-UNNAMED --add-exports=java.xml/com.sun.org.apache.xerces.internal.impl.xs=ALL-UNNAMED --add-exports=java.xml.bind/com.sun.xml.internal.bind.marshaller=ALL-UNNAMED --add-opens java.xml.ws/javax.xml.ws.wsaddressing=ALL-UNNAMED --add-opens java.base/java.security=ALL-UNNAMED --add-opens java.base/java.net=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.util.concurrent=ALL-UNNAMED</additionalJvmArgs>
</configuration>
</plugin>
My maven execution file (windows):
set PATH=C:\Users\user\Documents\programs\jdk-10\bin;%PATH%
set JAVA_HOME=C:\Users\user\Documents\programs\jdk-10
set MAVEN_OPTS=--add-modules java.xml.bind
mvn clean install
Using --add-modules fails when running with Java 8. To make the build compatible with both Java 8 and Java 9, I've added javax.xml.bind:jaxb-api as dependency of the plugin instead, which seems to work fine.
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
<version>${cxf.version}</version>
<executions>
<!-- ... -->
</executions>
<dependencies>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>2.3.1</version>
</dependency>
</dependencies>
</plugin>

All jUnit test cases are not running for Maven project using PowerMock with easymock, Surefire

In a Maven Project I am using PowerMock-easymock to run jUnit test cases. but while doing "mvn clean install" , I am getting below output..
T E S T S
Running TestSuite
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.621 sec
Results :
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
But I have many other test cases.
Here is a part of pom.xml
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.easymock</groupId>
<artifactId>easymock</artifactId>
<version>3.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-easymock-release-full</artifactId>
<version>1.4.12</version>
<type>pom</type>
</dependency>
If I remove, PowerMock dependency and do "mvn clean install" , all test cases are running fine. But I have to use PowerMock. How to solve this issue?
I guess that some of your test cases are not running, did you try this
Use mvn surefire plugin, and include test cases in that.
ensure dependancy of powermock-module-junit4.
check out these link: code.google.com/p/powermock/wiki/EasyMock_maven
http://code.google.com/p/powermock/wiki/GettingStarted
I had the same problem, and it took me a while to figure out. My setup was pulling in an older version of jboss.javassist, which oddly was preventing the PowerMockRunner from working at all.
It's worth noting that I also have a mixed JUnit / TestNG environment. I previously tried the solution of adding multiple surefire providers, and that didn't work either (using surefire 2.14.1). After upgrading to surefire 2.17, both my JUnit and TestNG tests started running without needing to declare any surefire providers. In fact, the JUnit provider threw an error for me, because I'm using groups. Apparently the TestNG provider allows free-form text (e.g. "integration") while the JUnit provider expects a classpath (e.g. "com.example.UnitTests").
Here's my plugin section...
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<groups>spring, unit, integration</groups>
<systemPropertyVariables>
<java.awt.headless>true</java.awt.headless>
<org.apache.activemq.default.directory.prefix>target/test/</org.apache.activemq.default.directory.prefix>
<log4j.configuration>file:${project.basedir}/src/test/resources/log4j.properties</log4j.configuration>
</systemPropertyVariables>
<argLine>${surefire.args}</argLine>
</configuration>
</plugin>
... and the relevant testing deps ...
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.9.5</version>
<scope>test</scope>
</dependency>
<!--
PowerMock versions are compatible with specific Mockito versions.
https://code.google.com/p/powermock/wiki/MockitoUsage13
-->
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<version>1.5.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito</artifactId>
<version>1.5.4</version>
<scope>test</scope>
</dependency>
<!-- without this PowerMock tests don't run in maven -->
<dependency>
<groupId>jboss</groupId>
<artifactId>javassist</artifactId>
<version>3.8.0.GA</version>
<scope>test</scope>
</dependency>
As per Dipak,
Solution 1:
Add below code in pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.13</version>
</dependency>
</dependencies>
</plugin>
For Correct ArtifactId in dependency , you can see this link if you are using jUnit.
Then do "mvn clean install"
Solution 2:
Add below code in pom.xml
<properties>
<powermock.version>1.5</powermock.version>
</properties>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-easymock</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
Refer this link for detail
All credits go to Dipak
I had a situation recently where PowerMock tests were run, but not included in the surefire coverage report. We found that the problem had to do with instrumentation.
I'll note that most of our tests run with TestNG. In general, we only use JUnit when we need to leverage PowerMock.
Here is the POM snippet:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<systemPropertyVariables>
<org.apache.activemq.default.directory.prefix>target/test/</org.apache.activemq.default.directory.prefix>
<log4j.configuration>file:${project.basedir}/src/test/resources/log4j.properties</log4j.configuration>
<jacoco-agent.destfile>${project.basedir}/target/jacoco.exec</jacoco-agent.destfile>
</systemPropertyVariables>
<argLine>-Xmx512m -XX:MaxPermSize=256m -Djava.awt.headless=true</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.5.201505241946</version>
<executions>
<execution>
<id>instrument</id>
<phase>process-classes</phase>
<goals>
<goal>instrument</goal>
</goals>
</execution>
<execution>
<id>restore</id>
<phase>test</phase>
<goals>
<goal>restore-instrumented-classes</goal>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
The <systemPropertyVariables> are probably not relevant to the fix.
Also, note that JaCoCo's documentation warns against using this type of configuration unless you find it necessary.
http://www.eclemma.org/jacoco/trunk/doc/instrument-mojo.html
Warning: The preferred way for code coverage analysis with JaCoCo is
on-the-fly instrumentation. Offline instrumentation has several
drawbacks and should only be used if a specific scenario explicitly
requires this mode. Please consult documentation about offline
instrumentation before using this mode.

Can't get QueryDsl / APT to generate Q classes

I'm trying to use QueryDsl in a new Spring project. I'm new to QueryDsl, and pretty new to maven and Spring, so I may be missing something fairly basic, but I can't get QueryDsl / maven-apt-plugin to generate my Q classes. The Querydsl reference makes sound so easy; I think I did exactly what it said:
I configured pom.xml with:
<plugin>
<groupId>com.mysema.maven</groupId>
<artifactId>maven-apt-plugin</artifactId>
<version>1.0.3</version>
<executions>
<execution>
<goals>
<goal>process</goal>
</goals>
<configuration>
<outputDirectory>target/generated-sources/java</outputDirectory>
<processor>com.mysema.query.apt.jpa.JPAAnnotationProcessor</processor>
</configuration>
</execution>
</executions>
</plugin>
and:
<repository>
<id>QUERYDSL</id>
<url>http://source.mysema.com/maven2/releases</url>
<layout>default</layout>
</repository>
and:
<dependency>
<groupId>com.mysema.querydsl</groupId>
<artifactId>querydsl-apt</artifactId>
<version>2.5.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.mysema.querydsl</groupId>
<artifactId>querydsl-jpa</artifactId>
<version>2.5.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.1</version>
</dependency>
I have two #Entity's in that project.
mvn clean install does not result in any output to target/generated-sources/java/
What am I missing?
I tried mvn apt:process, and it results in:
[ERROR] Failed to execute goal com.mysema.maven:maven-apt-plugin:1.0.3:process (default-cli) on project logging-implementation: Either processor or processors need to be given -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.mysema.maven:maven-apt-plugin:1.0.3:process (default-cli) on project logging-implementation: Either processor or processors need to be given
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
Any suggestions?
Thanks!
OK, I got it.
I don't understand it (I'm a Maven noob), but here's what worked:
In the parent pom.xml, I have
<build>
<pluginManagement>
<plugins>
the maven-apt-plugin definition shown above
</plugin>
<pluginManagement>
</build>
and in the project's POM I have:
<build>
<plugins>
the **exact same** maven-apt-plugin definition shown above
</plugin>
</build>
without the <pluginManagement> level betweeen <build> and <plugins>, following the instructions at http://mojo.codehaus.org/apt-maven-plugin/plugin-info.html
You are calling the goal directly, but the configuration is execution specific. So either use apt via the standard maven lifecycle or make the configuration general.

Resources