Why no jacoco.exec produced when a single test is run - but it is produced when all tests are run? - maven

I'm running surefire tests with a jacoco agent. When I run mvn verify a jacoco.exec file is produced.
When I run mvn clean verify -Dtest=com.org.MyTest -DfailIfNoTests=false then no jacoco.exec file is produced.
Here is my surefire config.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
</configuration>
<executions>
<execution>
<phase>test</phase>
<id>testconfig</id>
<configuration>
<argLine>${test.jvm.options} ${jacoco.agent.argLine}</argLine>
<skip>false</skip>
</configuration>
<goals><goal>test</goal></goals>
</execution>
</executions>
</plugin>
Here is my jacoco config
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.5.201505241946</version>
<configuration>
<properties>
<property>
<name>listener</name>
<value>org.sonar.java.jacoco.JUnitListener</value>
</property>
</properties>
</configuration>
<executions>
<execution>
<id>unit_agent</id>
<phase>initialize</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>jacoco.agent.argLine</propertyName>
</configuration>
</execution>
</executions>
</plugin>
My question is: Why no jacoco.exec produced when a single test is run - but it is produced when all tests are run?

Log of execution of mvn clean verify -Dtest=com.org.MyTest -DfailIfNoTests=false shows something like (I'm using Apache Maven 3.3.9):
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) # example ---
[INFO] Surefire report directory: /private/tmp/jacoco-example/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.org.MyTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in com.org.MyTest
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-surefire-plugin:2.18.1:test (testconfig) # example ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
Notice that maven-surefire-plugin executed two times - one time with id default-test and another execution with id testconfig is actually skipped, while only configuration with id testconfig uses ${jacoco.agent.argLine}.
Change of definition for maven-surefire-plugin on
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${jacoco.agent.argLine}</argLine>
</configuration>
</plugin>
solves the issue.

Related

maven - why is dependency check skipped?

In my pom plugins I have...
<plugin>
<groupId>org.owasp</groupId>
<artifactId>dependency-check-maven</artifactId>
<version>6.2.2</version>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
But when I run mvn dependency-check:check or mvn verify in the logs I see
[INFO] --- dependency-check-maven:6.2.2:check (default-cli) # my-project ---
[INFO] Skipping dependency-check
How do I get it to run??
Turned out the dependency-check.skip was evaluating to true in the properties.
I simply set it to false.

Maven failsafe plugin with system property executing the integration test twice

with the below config:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<systemPropertyVariables>
<environment>${env}</environment>
</systemPropertyVariables>
<skipTests>${skip.integration.tests}</skipTests>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
When I execute the command
mvn verify -Denv=http://localhost:8080
integration tests are getting executed twice, one with env value =null resulting in failure of test and other with the desired env value. So it is executing the test twice with different results
I had the same issue.
My maven logs:
[INFO] --- maven-failsafe-plugin:2.22.0:integration-test (integration-tests) # core ---
[...tests...]
[INFO] --- maven-failsafe-plugin:2.22.0:integration-test (default) # core ---
[...tests...]
Integration tests run twice when we specify id for execution. So then we have default inherit execution which runs tests and second one - our custom execution.
I resolved it by removing id element from configuration.
So in your case remove <id>integration-tests</id> line.

Why maven-surefire-plugin would run on my integration test

It is weird that I see surefire plugin run on my integration test. Anyone know why ?
[INFO] --- maven-surefire-plugin:2.17:test (default-test) # xxxx ---
[INFO] Surefire report directory: /Users/jzhang/github/project/module-A/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running package.XXXXIT
Yes. It's natural. You need to tell maven-surefire-plugin to skip execution of your tests ending with *IT.java.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<excludes>
<exclude>**/*IT.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>

Jacoco Report are not generated in the site

Jacoco reports are not generated when mvn builds the site.
Jacoco reports are generated in my target folder at target\site\jacoco , but somehow I am not able to see "Jacoco Tests" option under "Project Reports" when I try to build the jar site
The only thing I see in the log while generating the site is as below :
[INFO] <<< clirr-maven-plugin:2.6.1:clirr # data-binding <<<
[INFO] configuring report plugin com.cerner.engineering:maven-cerner-ml-dependency-plugin:1.1.1
[INFO] configuring report plugin org.jacoco:jacoco-maven-plugin:0.7.4.201502262128
[INFO] Skipping JaCoCo execution due to missing execution data file:C:\MSVCWorkspace\br-hla-april-data-binding\target\jacoco.exec
I am using the command mvn clean install site
This is how I have configure jacoco in my pom.xml in my build section :
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.plugin.version}</version>
<configuration>
<excludes>
<exclude>**/*Suite*.java</exclude>
</excludes>
<forkCount>1</forkCount>
<reuseForks>false</reuseForks>
<argLine>-Xms64m -Xmx512m -XX:MaxPermSize=256M -enableassertions -XX:+HeapDumpOnOutOfMemoryError ${jacoco.argLine}</argLine>
<junitArtifactName>junit:junit-dep</junitArtifactName>
</configuration>
</plugin>
Jacoco plugin is called like :
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<configuration>
<destFile>${basedir}/target/coverage-reports/jacoco-unit.exec</destFile>
<dataFile>${basedir}/target/coverage-reports/jacoco-unit.exec</dataFile>
</configuration>
<executions>
<execution>
<id>jacoco-initialize</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>jacoco.argLine</propertyName>
</configuration>
</execution>
<execution>
<id>jacoco-site</id>
<phase>package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
I am not sure what is causing this issue , not to generate Jacoco reports on my site.
Although I am able to see the below in my logs
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.4.201502262128:prepare-agent (default-cli) # data-binding ---
[INFO] argLine set to -javaagent:C:\\_repository\\org\\jacoco\\org.jacoco.agent\\0.7.4.201502262128\\org.jacoco.agent-0.7.4.201502262128-runtime.jar=destfile=C:\\target\\coverage-reports\\jacoco-unit.exec
Thanks !!!
I was able to fix by calling plugin as below
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<configuration>
<dataFile>${basedir}/target/coverage-reports/jacoco-unit.exec</dataFile>
</configuration>
<executions>
<execution>
<id>jacoco-initialize</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>jacoco.argLine</propertyName>
</configuration>
</execution>
</executions>
</plugin>

maven-failsafe-plugin Errors and BUILD SUCCESS?

my question is very similar to this one: maven-failsafe-plugin Failures and BUILD SUCCESS?
and I manage to set up failsafe plugin to fail if tests fail.
But if test goes into error state, failsafe plugin still does not break the build.
.................
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running xxxxx.IntegrationTierFunctionalTestCase
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.054 sec <<< FAILURE!
Results :
Tests in error:
testException(xxxxx.IntegrationTierFunctionalTestCas
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is
[INFO] [failsafe:verify {execution: functional-test-1024}]
[INFO] Failsafe report directory: C:\projects\oec-integration-server\trunk\oec-integrati
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is
[INFO] [failsafe:integration-test {execution: functional-test-24}]
[INFO] Failsafe report directory: C:\projects\oec-integration-server\trunk\oec-integrati
.............
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 58 seconds
[INFO] Finished at: Tue May 28 17:48:13 BST 2013
[INFO] Final Memory: 114M/781M
[INFO] ------------------------------------------------------------------------
for simplicy IntegrationTierFunctionalTestCase contains only this code
import org.junit.Test;
import static org.junit.Assert.fail;
public class IntegrationTierFunctionalTestCase
{
#Test
public void testException(){
//fail();
throw new RuntimeException("super error");
}
}
if I uncomment fail() whole build fails correctly, with build failed.
but if I just throw an exception, it fails as on shown above.
oour plugin configuration looks like this
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.7</version>
<configuration>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<systemPropertyVariables>
<oec.env>TEST</oec.env>
<mule.test.timeoutSecs>2400</mule.test.timeoutSecs>
</systemPropertyVariables>
<additionalClasspathElements>
<additionalClasspathElement>${basedir}/src/main/resources/config</additionalClasspathElement>
</additionalClasspathElement>
</additionalClasspathElements>
</configuration>
<executions>
<execution>
<id>functional-test-1024</id>
<phase>test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<includes>
<include>**/IntegrationTierFunctionalTestCase.java</include>
</includes>
<forkMode>once</forkMode>
<argLine>-XX:MaxPermSize=256M -Xmx1024M</argLine>
</configuration>
</execution>
</executions>
</plugin>
What am I missing?
And no I do not want to wrap it in try-catch blocks and fail tests manually.
You need having two executions blocks, cause the verify goal of the maven-failsafe-plugin is intended to check the results of the integration tests.
<executions>
<execution>
<id>functional-test-1024</id>
<phase>test</phase>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<includes>
<include>**/IntegrationTierFunctionalTestCase.java</include>
</includes>
<forkMode>once</forkMode>
<argLine>-XX:MaxPermSize=256M -Xmx1024M</argLine>
</configuration>
</execution>
<execution>
<id>verify</id>
<phase>verify</phase>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
Furthermore you should update the version of the maven-failsafe-plugin to 2.14.1 instead of 2.7.
Update: In the meantime update to 2.17.
If you're running the integration tests like this:
mvn test-compile failsafe:integration-test
Then you should know that according to the maven documentation on failsafe:
The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute.
I was able to get the build to fail like this:
mvn test-compile failsafe:integration-test failsafe:verify
And here's my failsafe configuration for reference:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19</version>
<configuration>
<includes>
<include>**/*IT.java</include>
</includes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
</plugin>
Please verify that the maven property "maven.test.failure.ignore" is not set to "true" in any of your maven pom.xml files, as it can be the only reason to not stop the build after test failure.
There is one possible additional reason why it happens. Failsafe verify is looking for test classes and expects them to be in ${project.build.directory}/test-classes - if you have a different setup it will just print "No tests to run." and ends if build success regardless of what the result of integration-test phase was.
You have to set testClassesDirectory in plugin configuration :
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<systemPropertyVariables>
<environmentType>${environmentType}</environmentType>
</systemPropertyVariables>
<summaryFile>${project.build.directory}/failsafe-reports/failsafe-summary.xml</summaryFile>
<testClassesDirectory>${project.build.directory}/classes</testClassesDirectory>
<suiteXmlFiles>
<suiteXmlFile>${suiteXml}</suiteXmlFile>
</suiteXmlFiles>
</configuration>

Resources