I have the next pom.xml
<project>
...
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<argLine>${failsafeArgLine}</argLine>
<includes>
<include>**/dmg/*IT.java</include>
</includes>
<skipTests>${skipTests}</skipTests>
</configuration>
</execution>
</executions>
</plugin>
...
</project>
The problem is that when I'm taking off verify goal then the build is every time successful even if there was test failures.
And when I'm Taking off integration-test goal the integration tests simply do not run
Why failsafe plugin requires both integration-test and verify goals?
In Maven Failsafe plugin reference you can find simple answer why build is always successful
failsafe:integration-test runs the integration tests of an application.
failsafe:verify verifies that the integration tests of an application passed.
Without verify goal test results are not checked at all(but they are executed), so failsafe plugin requires integration-test goal to run tests, and verify to "verify" their results.
In Maven there are two types of test runner plug-ins, Surefire and Failsafe and they both serve different purposes. (I mention Surefire here because I feel it helps explain and contrast different types of failure in builds.)
Surefire
Surefire is a plug-in designed for your pre-deploy tests, such as Unit and Component tests. Surefire is guaranteed to return a failure (and therefore break your build) if any of the tests fail.
You want this behaviour in pre-deploy tests because you want to fail the build early if your tests fail, and as Unit and Component tests don't (or shouldn't) have any external dependencies it's safe to fail a build there and then.
Failsafe
Failsafe is a plug-in designed for post-deploy tests, such as Functional and Smoke tests.
Failsafe, as the name implies handles failures safely, by always returning a success exit code (as you have experienced in your build).
The reason behind this is for post-deploy tests you don't want test failures to immediately break the build because you may have spun up infrastructure, or seeded some test data into the system, which needs clean-up before the build is failed.
Therefore Failsafe splits the failure checking into the failsafe:verify goal which happens in the verify phase.
If you run Maven without parameters (assuming the normal jar lifecycle) you'll notice a phase between integration-test and verify called, post-integration-test. This is the phase which was designed for the tear down of any infrastructure or test data you may have injected into the system to run the tests.
... pre-integration-test, integration-test, post-integration-test, verify, ...
The phase before integration-test, pre-integration-test is designed for the said seeding and infrastructure creation.
To sum it all up, with pre-deploy tests you want to fail the build as early as possible, and so surefire guarantees this by exiting with a failure.
Whereas with post-deploy tests there are often external dependencies which need clean-up before you fail the build. For example in Kubernetes you may want to issue a rollback of the deploy and so failsafe:integration always returns a build success to guarantee you can perform the clean-up before failing the build by checking the status with failsafe:verify.
(The currently accepted answer just quotes the manual and doesn't explain why there are two commands and there are differences which is why I feel this answer is needed.)
Related
When release:clean release:prepare release:perform is executed , the tests are executed twice
Once at release:prepare , stage run-preparation-goals , task clean verify
And another time on release:perform ,stage run-perform-goals , task deploy
Is there any configuration option to make tests run only on first one and not being executed on perform?
BR
Yulian Oifa
That's because 2 full builds are run as part of the commands you issue.
The release:prepare section performs lots of checks on the code & repository, and does a full build to ensure all tests pass.
The release:perform section tags the repo, then pulls that tag. It performs a build based on that tag and the releases that build to your artefact store of choice (Nexus/Artifactory/wherever).
These steps are designed this way to ensure you don't pollute your repo with a tag on code that doesn't build, and tests are an integral part of your build. Once maven is happy your code is good to go, it creates the tag, and pulls it so it knows for sure that the code being released is the code referenced by the tag, and nothing else. Building that tag for release requires a full build, including the tests.
You can skip the tests on one or other of the steps, or on both, but you're missing the point of tests, and of the release plugin, if you do that.
You can override the goals parameter and add -DskipTests=true.
This parameter will skip the tests only on the perform part.
Eventually i had to change a logic.
I have executed mvn clean install release:clean release:prepare release:perform -Pmaven-release
and create a maven-release. As result the tests are executed only at install
BR
Yulian Oifa
<profile>
<id>maven-release</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<configuration>
<autoVersionSubmodules>true</autoVersionSubmodules>
<arguments>-Dmaven.test.skip</arguments>
</configuration>
</plugin>
</plugins>
</build>
</profile>
You can skip running the tests (managed by surefire plugin) with an explicitly specified parameter -DskipTests:
Something like this:
mvn release:perform -DskipTests
For intergration tests (managed by failsafe plugin) you can use a similar flag: -DskipITs
If you don't want even compile the tests you can use -Dmaven.test.skip=true (for both unit and integration tests)
Can I make a Mojo that runs during integration-test phase trigger another Mojo that runs during verify phase?
This question is a follow up on my previous question. I extended the frontend-maven-plugin with a parameter that delays build failure from integration tests until the verify phase to allow the post-integration-test phase to execute properly. This is often needed to clean up the environment (close DB, server, ...). Pull request commit available here.
The solution is rather hacky because it requires the user to add a verify execution phase, even if it doesn't do anything. Instead, I would like an execution on the integration-test phase to automatically execute a Mojo during the verify phase (without the user explicitly adding this to the Maven configuration). So the following:
<execution>
<id>npm run integration tests</id>
<goals>
<goal>npm</goal>
</goals>
<phase>integration-test</phase>
<configuration>
<arguments>run e2e</arguments>
<integrationTestFailureAfterPostIntegration>true</integrationTestFailureAfterPostIntegration>
</configuration>
</execution>
should trigger e2e tests to run during integration-test phase, but also needs to execute another Mojo during verify phase.
Is this possible and how would you do it?
UPDATE: I managed to add another pluginExecution at runtime to the plugin like so:
for (Plugin plugin : project.getBuildPlugins()) {
if("com.github.eirslett".equals(plugin.getGroupId()) && "frontend-maven-plugin".equals(plugin.getArtifactId())){
PluginExecution verifyExecution = new PluginExecution();
verifyExecution.addGoal("verify");
verifyExecution.setId("verify integration tests");
verifyExecution.setPhase("verify");
plugin.addExecution(verifyExecution);
}
}
but this does not seem to affect the runtime executions. The goal is a custom verify Mojo that performs the validation during the verify phase
In my project I have surefire as well as failsafe tests.
If I run with mvn clean install -DskipTests then both kinds of tests are skipped.
If I try to run a single failsafe test using -Dit.test=TestName then first all surefire tests run and then my it test.. but as the surefire tests take a long time this is not good.
I would like to skip the surefire tests in some cases but run the failsafe IT tests.
It's sometimes kind of confusing (annoying?) that, by default, -DskipITs=true will skip Failsafe but -DskipTests=true will skip both Surefire and Failsafe tests.
Anyway, you can change this behaviour by configuring the Surefire plugin to use a different 'skip' parameter. For example:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<skip>${skipSurefire}</skip>
</configuration>
</plugin>
This will allow you to skip Surefire tests but not Failsafe tests by invoking:
mvn clean verify -DskipSurefire=true
Problem statement
How to invoke tests within a given test server(i.e. within the process)?
Explanation
We know that there is surefire plugin, but this uses surefire:test goal which runs unit tests in separate process. In my case, i need to run the tests within the existing process (pre-configured test server and required environment) to run test. In surefire plugin, there is configuration that has forkCount and reuseForks. If i use it with forkCount as 0, reuseForks true, still creates separate server and hence process. I simply want to avoid setup part for test.
Update after Joe's comment saying it as duplicate As my application usage SpringBoot, running mvn clean install build the project and execute tests. While executing tests, it creates server for each running test. Even though i used configuration given below, in console it shows same process id, but starting server every time. In my case i don't want to make server up for every test case, rather used pre-existing test server and the test should use that server.
Configuration used
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<forkCount>0</forkCount>
<reuseForks>true</reuseForks>
</configuration>
</plugin>
I'm using the JBoss and WildFly Maven plugin to deploy my applications.
<plugins>
<plugin>
<groupId>org.wildfly.plugins</groupId>
<artifactId>wildfly-maven-plugin</artifactId>
<version>1.0.0.Beta1</version>
</plugin>
</plugins>
The problem that I have found is that if I have some Junit test in my project they are executed before application deployment, leading to a test with an inconsistent application state. Is it possible to configure somehow this plugin to kick-in before the test phase ?
Thanks
This plugin by default executes alongside the package phase and you'll probably have issues attempting to run the plugin before your application is packaged. As an alternative, you could override the maven-surefire-plugin to run your tests in a later phase, such as integration-test, which would be executed after your application has been packaged.
A practical example can be found on this Maven tutorial where it's show how to include failsafe plugin and bind it to the executions of the integration-test and verify phase. This way tests which are engineered as integration test (e.g ending in *IT) will execute only during the integration-test phase.