maven-failsafe-plugin Failures and BUILD SUCCESS? - maven

Question

I want to use maven-failsafe-plugin to run some integration tests. If any test fails, I want Maven to fail the build and not BUILD SUCCESS.
Tests run: 103, Failures: 1, Errors: 0, Skipped: 26
[INFO] BUILD SUCCESS*
how can I configure it, that build not success is?
My failsafe plugin is configured as:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${failsafe.version}</version>
<configuration>
<systemProperties>
<CI_INTEGRATION_OVERRIDE_PATH>${basedir}/..</CI_INTEGRATION_OVERRIDE_PATH>
</systemProperties>
<includes>
<include>**/integration/**/*.java</include>
</includes>
<excludes>
<exclude>**/integration/**/*TestSuite.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>

Answer 1

As Andrew pointed out, the correct solution is to use failsafe as intended. The integration-test goal is specifically designed not to fail the build. If you want to fail the build, call mvn verify or mvn failsafe:verify
For the verify goal to work, it needs to read the results of the integration test which by default are written to ${project.build.directory}/failsafe-reports/failsafe-summary.xml so make sure that is getting generated.
Also, you have to make sure you bind your maven-failsafe-plugin configuration to both the integration-test goal and the verify goal in the executions part.
Failure to add either of those will lead to maven succeeding the build instead of failing it when integration tests fail.

Answer 2

Since you are running mvn clean install both the integration-test and verify phases should be executing. According to the failsafe plugin docs the failsafe:integration-test and failsafe:verify goals are bound to those phases, so I don't believe the extra call to failsafe:integration-test is required.
That said however, I'm not sure I trust the failsafe plugin documentation. I answered a similar question for someone earlier this year. It turned out he had to explicitly bind each goal to the correct phase and then failsafe worked as expected. Might be worth a shot.

Answer 3

I had a similar issue with Jenkins builds.
Running the tests locally resulted in failed tests and thus failed build
Running the tests on Jenkins resulted in failed tests but "BUILD SUCCESS"
Solution: in job configuration set MAVEN_OPTS to -Dmaven.test.failure.ignore=false.
Jenkins by default seems to ignore failed integration tests.
See also https://stackoverflow.com/a/28684048/4412885

Answer 4

solution.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
<executions>
<execution>
<id>unit-test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<excludes>
<exclude>**/*IntegrationTest.java</exclude>
</excludes>
</configuration>
</execution>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<enableAssertions>false</enableAssertions>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
<systemPropertyVariables>
<integration>${integration}</integration>
</systemPropertyVariables>
</configuration>
</execution>
</executions>
</plugin>