with the below config:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<systemPropertyVariables>
<environment>${env}</environment>
</systemPropertyVariables>
<skipTests>${skip.integration.tests}</skipTests>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
When I execute the command
mvn verify -Denv=http://localhost:8080
integration tests are getting executed twice, one with env value =null resulting in failure of test and other with the desired env value. So it is executing the test twice with different results
I had the same issue.
My maven logs:
[INFO] --- maven-failsafe-plugin:2.22.0:integration-test (integration-tests) # core ---
[...tests...]
[INFO] --- maven-failsafe-plugin:2.22.0:integration-test (default) # core ---
[...tests...]
Integration tests run twice when we specify id for execution. So then we have default inherit execution which runs tests and second one - our custom execution.
I resolved it by removing id element from configuration.
So in your case remove <id>integration-tests</id> line.
Related
This question is more focused on understanding maven life circle than in solving a real problem.
We have a project with several maven modules. Both Jacoco and Surefire plugins are configured in the parent pom.xml as follows:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.0</version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/jacoco.exec</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>default-report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/jacoco.exec</dataFile>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<configuration>
<argLine>${surefireArgLine}</argLine>
</configuration>
</plugin>
This configuration works well and the jacoco.exec files are created on the target directory when common goals are executed, for instance:
mvn clean install
or
mvn test
But if we execute the following commands the jacoco.exec files are not created:
mvn clean install -DskipTests
#other actions here...
mvn jacoco:prepare-agent surefire:test
Analyzing the logs with the -X option, surefire plugin indicates that it is going to use the argLine property as it is expected:
<argLine>${surefireArgLine}</argLine>
The goal jococo:prepare-agent generates the value for this variable correctly:
argLine set to -javaagent:s:\\m2_repo\\org\\jacoco\\org.jacoco.agent\\0.8.0\\org.jacoco.agent-0.8.0-runtime.jar=destfile=S:\\sources\\sofia2-s4c\\config\\services\\target\\jacoco.exec
But when surefire:test is executed it does not use the argLine property:
[DEBUG] (s) additionalClasspathElements = []
argLine SHOULD BE HERE!!!!
[DEBUG] (s) basedir = S:\sources\sofia2-s4c\config\services
We have solved this executing:
mvn clean install -DskipTests
#other actions here...
mvn test
Due to mvn test detect that there are no changes in the compiled classes this is efficient. But I would like to understand why the first approach does not work.
The problem is the <argLine> of the surefire plugin configuration.
Solutions from https://github.com/jacoco/jacoco/issues/964:
Define user property <argLine>...</argLine>. surefire will pick it up. No need to use ${surefireArgLine}
Keep the <argLine> in the surefire plugin configuration, but write it as <argLine>#{argLine} ${surefireArgLine}</argLine>
See also https://stackoverflow.com/a/23605812/510583
I'm running surefire tests with a jacoco agent. When I run mvn verify a jacoco.exec file is produced.
When I run mvn clean verify -Dtest=com.org.MyTest -DfailIfNoTests=false then no jacoco.exec file is produced.
Here is my surefire config.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
</configuration>
<executions>
<execution>
<phase>test</phase>
<id>testconfig</id>
<configuration>
<argLine>${test.jvm.options} ${jacoco.agent.argLine}</argLine>
<skip>false</skip>
</configuration>
<goals><goal>test</goal></goals>
</execution>
</executions>
</plugin>
Here is my jacoco config
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.5.201505241946</version>
<configuration>
<properties>
<property>
<name>listener</name>
<value>org.sonar.java.jacoco.JUnitListener</value>
</property>
</properties>
</configuration>
<executions>
<execution>
<id>unit_agent</id>
<phase>initialize</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>jacoco.agent.argLine</propertyName>
</configuration>
</execution>
</executions>
</plugin>
My question is: Why no jacoco.exec produced when a single test is run - but it is produced when all tests are run?
Log of execution of mvn clean verify -Dtest=com.org.MyTest -DfailIfNoTests=false shows something like (I'm using Apache Maven 3.3.9):
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) # example ---
[INFO] Surefire report directory: /private/tmp/jacoco-example/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.org.MyTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in com.org.MyTest
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-surefire-plugin:2.18.1:test (testconfig) # example ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
Notice that maven-surefire-plugin executed two times - one time with id default-test and another execution with id testconfig is actually skipped, while only configuration with id testconfig uses ${jacoco.agent.argLine}.
Change of definition for maven-surefire-plugin on
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>${jacoco.agent.argLine}</argLine>
</configuration>
</plugin>
solves the issue.
The failsafe plugin code below is intended to populate a target server with subscriber records during the pre-integration-test phase and delete them again in the post-integration-test phase .
There are JMeter plugins bound to the integration-test phase which carry out performance tests .
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<id>create-subscribers</id>
<phase>pre-integration-test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<test>AbcIntegrationIT#registerSubscriberGroup</test>
</configuration>
</execution>
<execution>
<id>delete-subscribers</id>
<phase>post-integration-test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<test>AbcIntegrationIT#deleteSubscriberGroup</test>
</configuration>
</execution>
</executions>
</plugin>
The first execution of this plugin runs OK and the performance test run fine .
However the second execution doesn't as , according to Failsafe
[INFO] --- maven-failsafe-plugin:2.16:integration-test (delete-subscribers) # digihost-integration-test-apm ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
!?
I would argue that yes it has run for the pre-integration-test phase but not for the post-integration-test phase .
It looks as if other plugins do support multi-phase executions for a single goal (viz. cobertura plugin Running maven goal in multiple lifecycles ) , which leads me to wonder if this is a bug in Failsafe
Any Thoughts and Feedback greatly appreciated .
Here is the condensed snippet of pom.xml from my project
<profiles>
<profile>
<id>run-tests</id>
<build>
<plugins>
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId>
<version>1.5.2</version>
<executions>
<execution>
<phase>pre-integration-test</phase>
<goals>
<goal>replace</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
......
</includes>
<replacements>
<replacement>
.......
</replacement>
</replacements>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
<configuration>
......
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<phase>integration-test</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
I have two questions:
1) when I execute mvn clean package -Prun-tests, what happens? I expected none of these plugin goals to execute here because they are bound to integration-test phase. But I see these goals executed why?
2) what does having two goals in execution block mean? please see above in failsafe-plugin
Thanks
A partial answer:
1) No Way. Unless you also have these plugins configured in the main build section to be run in phases up to package.
How did you determine that the plugins had run? Do you have something such as the following in the maven output?
[INFO] --- maven-failsafe-plugin:2.18.1:integration-test (default)
[INFO] --- maven-failsafe-plugin:2.18.1:verify (default)
2) That means that the two goals (mojos) will be executed in the integration-test phase. First the integration-test goal, immediately followed by the verify goal.
Comment: integration-test goal is by default bound to the integration-test phase, whereas the verify goal is bound to the verify phase. So you could have configured the failsafe plugin this way:
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
Note that the phase is ommited
my question is very similar to this one: maven-failsafe-plugin Failures and BUILD SUCCESS?
and I manage to set up failsafe plugin to fail if tests fail.
But if test goes into error state, failsafe plugin still does not break the build.
.................
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running xxxxx.IntegrationTierFunctionalTestCase
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.054 sec <<< FAILURE!
Results :
Tests in error:
testException(xxxxx.IntegrationTierFunctionalTestCas
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is
[INFO] [failsafe:verify {execution: functional-test-1024}]
[INFO] Failsafe report directory: C:\projects\oec-integration-server\trunk\oec-integrati
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is
[INFO] [failsafe:integration-test {execution: functional-test-24}]
[INFO] Failsafe report directory: C:\projects\oec-integration-server\trunk\oec-integrati
.............
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 58 seconds
[INFO] Finished at: Tue May 28 17:48:13 BST 2013
[INFO] Final Memory: 114M/781M
[INFO] ------------------------------------------------------------------------
for simplicy IntegrationTierFunctionalTestCase contains only this code
import org.junit.Test;
import static org.junit.Assert.fail;
public class IntegrationTierFunctionalTestCase
{
#Test
public void testException(){
//fail();
throw new RuntimeException("super error");
}
}
if I uncomment fail() whole build fails correctly, with build failed.
but if I just throw an exception, it fails as on shown above.
oour plugin configuration looks like this
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.7</version>
<configuration>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<systemPropertyVariables>
<oec.env>TEST</oec.env>
<mule.test.timeoutSecs>2400</mule.test.timeoutSecs>
</systemPropertyVariables>
<additionalClasspathElements>
<additionalClasspathElement>${basedir}/src/main/resources/config</additionalClasspathElement>
</additionalClasspathElement>
</additionalClasspathElements>
</configuration>
<executions>
<execution>
<id>functional-test-1024</id>
<phase>test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<includes>
<include>**/IntegrationTierFunctionalTestCase.java</include>
</includes>
<forkMode>once</forkMode>
<argLine>-XX:MaxPermSize=256M -Xmx1024M</argLine>
</configuration>
</execution>
</executions>
</plugin>
What am I missing?
And no I do not want to wrap it in try-catch blocks and fail tests manually.
You need having two executions blocks, cause the verify goal of the maven-failsafe-plugin is intended to check the results of the integration tests.
<executions>
<execution>
<id>functional-test-1024</id>
<phase>test</phase>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<includes>
<include>**/IntegrationTierFunctionalTestCase.java</include>
</includes>
<forkMode>once</forkMode>
<argLine>-XX:MaxPermSize=256M -Xmx1024M</argLine>
</configuration>
</execution>
<execution>
<id>verify</id>
<phase>verify</phase>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
Furthermore you should update the version of the maven-failsafe-plugin to 2.14.1 instead of 2.7.
Update: In the meantime update to 2.17.
If you're running the integration tests like this:
mvn test-compile failsafe:integration-test
Then you should know that according to the maven documentation on failsafe:
The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute.
I was able to get the build to fail like this:
mvn test-compile failsafe:integration-test failsafe:verify
And here's my failsafe configuration for reference:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19</version>
<configuration>
<includes>
<include>**/*IT.java</include>
</includes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
</plugin>
Please verify that the maven property "maven.test.failure.ignore" is not set to "true" in any of your maven pom.xml files, as it can be the only reason to not stop the build after test failure.
There is one possible additional reason why it happens. Failsafe verify is looking for test classes and expects them to be in ${project.build.directory}/test-classes - if you have a different setup it will just print "No tests to run." and ends if build success regardless of what the result of integration-test phase was.
You have to set testClassesDirectory in plugin configuration :
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<systemPropertyVariables>
<environmentType>${environmentType}</environmentType>
</systemPropertyVariables>
<summaryFile>${project.build.directory}/failsafe-reports/failsafe-summary.xml</summaryFile>
<testClassesDirectory>${project.build.directory}/classes</testClassesDirectory>
<suiteXmlFiles>
<suiteXmlFile>${suiteXml}</suiteXmlFile>
</suiteXmlFiles>
</configuration>