The following config does not work. No test is in scope for the goal integration-test.
In case it is unclear, What should happen is that what I do a mvn integration-test the failsafe plugin should run my test. But the surefire plugin configuration is excluding the test. If I uncomment the surefire config block the test is run during the integration-test goal.
Maven config:
<build>
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<groups>spring-container-sanity</groups>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludedGroups>spring-container-sanity</excludedGroups>
</configuration>
</plugin>
</plugins>
</build>
A Java Test class
#SpringApplicationConfiguration(TestApplication.class)
#TestPropertySource("/test.properties")
public class SimpleTest extends AbstractTestNGSpringContextTests {
#Test(groups = "spring-container-sanity")
public void isHessianServiceExported() throws Exception {
/*...*/
}
}
I don't know why the tests are run when surefire is disabled. Failsafe has a file naming convention for integration tests, if the tests you want to run do not follow this they won't be in scope and the group rule will have nothing to match. So, in this case to get failsafe to resolve the tests it should run correctly you first need to add an include filter. This build block will work:
<build>
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/*.java</include>
</includes>
<groups>spring-container-sanity</groups>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludedGroups>spring-container-sanity</excludedGroups>
</configuration>
</plugin>
</plugins>
</build>
When annotating your test, make sure to define an array of groups, even if its only a single one.
#Test(groups = {"spring-container-sanity"})
Related
I am attempting to have my integration tests separated from the usual lifecycle, meaning that I do not want them to be executed during a mvn install. I have the following configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<excludes>
<exclude>**/*IT.java</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<includes>
<include>**/*IT.java</include>
</includes>
</configuration>
<goals>
<goal>integration-test</goal>
</goals>
</plugin>
And my integration tests are all suffixed with IT. I was able to confirm that surefire does exclude the *IT.java tests, however it seems that failsafe is being triggered during mvn install anyway. What am I missing?
Thank you for your help
A typical configuration to handle the issue with running integration tests only if you like is a profile like the following:
<profiles>
<profile>
<id>run-its</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
By using the above configuration you can run
mvn clean verify
which will execute the unit tests. Using the following you can activate the integration tests:
mvn -Prun-its clean verify
Based on the default naming conventions the integration tests which will be executed by maven-failsafe-plugin should be named like *IT.java where as the unit tests can be name like *Test.java (will be executed via maven-surefire-plugin).
I'm setting up a CD pipeline in Jenkins and I want to run my unit tests and integration tests in two different steps. The plan is to have my pipeline script look something like this and have them run separately:
stage('Unit tests') {
steps {
withMaven(maven: 'Maven 3.6.2') {
sh 'mvn test -P coverage'
}
}
}
stage('Integration tests') {
steps {
withMaven(maven: 'Maven 3.6.2') {
sh 'mvn test -P coverage'
}
}
I have tried using the surefire plugin as described here: https://dzone.com/articles/splitting-unit-and-integration-tests-using-maven-a, and running 'mvn test' does run only the unit test as it should, but 'mvn integration-test' runs both unit and integration tests.
I have also tried using the failsafe plugin as described here: Maven separate Unit Test and Integration Tests, but 'mvn verify' runs both unit and integration tests no matter which options I enter.
How can I make my pipeline execute the unit tests and integration tests in two different steps?
Pom with surefire:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId> <!-- surefire plugin version managed by Spring Boot -->
<configuration>
<skipTests>true</skipTests>
</configuration>
<executions>
<execution>
<id>unit-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skipTests>false</skipTests>
<includes>
<include>**/*Test.java</include>
</includes>
</configuration>
</execution>
<execution>
<id>integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skipTests>false</skipTests>
<includes>
<include>**/*IT.*</include>
<include>**/*Tests.*</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
Pom with surefire and failsafe:
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>${surefire.skip}</skip>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
This solved it for me:
Below is example pom.xml which works with the following commands:
mvn clean verify -DskipUTs=true : Runs only integration tests (tests ending in "IT")
mvn clean verify -DskipITs=true : Runs only unit tests (tests ending in "Test")
mvn clean verify -DskipTests=true : Skips all tests
<build>
<finalName>test-app</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId> <!-- surefire plugin version managed by Spring Boot -->
<configuration>
<skipTests>${skipUTs}</skipTests>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M3</version>
<executions>
<execution>
<id>run-integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<skipTests>${skipTests}</skipTests>
<skipITs>${skipITs}</skipITs>
</configuration>
</plugin>
</plugins>
</build>
When we say mvn test, usual way is that maven will look for the tests present in src/test/java folder. But I have my tests in some different folder, namely src/integration-test/java. How do I run the tests present in this folder through command line?
Thanks in advance,
Manoj.
First you shouldn't run those integration test via the test life cycle, cause
pre-integration-test, integration-test and post-integration-test life cycle phase exist. Apart from that for integration tests the maven-failsafe-plugin is responsible.
There are several options to handle your situations. First you should follow the naming conventions for integration tests
<includes>
<include>**/IT*.java</include>
<include>**/*IT.java</include>
<include>**/*ITCase.java</include>
</includes>
which means to put the integration tests into the default folder src/test/java. If you have a multi-module build it would be the best having a separate module which contains the integration-tests only or you can go the path you decided to use a separate folder (which is not the best):
First you need to add the folder using the buildhelper-maven-plugin to get those integration tests being compiled like this:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>process-resources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/integration-test/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
and you have to configure the maven-failsafe-plugin like this:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.14.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
After you have configured you can run your integration tests via:
mvn verify
#khmarbaise is right with his recommendation (so +1 for that) but I want to answer your question without speculating about the reasons why the test source are located somewhere else.
If your tests are located in another directory than the standard src/test/java directory, the most simple solution is to change the default value of the testSourceDirectory configuration parameter which is defined in the Super POM.
e.g. for src/foobar/java use
<build>
<testSourceDirectory>src/foobar/java</testSourceDirectory>
</build>
then you can simply run mvn test to execute the tests.
More complex solution...
If you do not want to change the pom.xml configuration you can specifiy the testSourceDirectory parameter on the command line like this:
mvn -DtestSourceDirectory=src/foobar/java clean test
But be sure that your sources are compiled. Otherwise they will not be found and executed. In the above example the test sources are not placed at a location that gets compiled by default, so we nevertheless have to change the pom and add the directory to the list of test sources by using the buildhelper plugin:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/foobar/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
If you do not want to change the configuration of the default value in the pom and not want to pass the new directory at the commandline you have to configure the path in the maven-buildhelper-plugin and the maven-surefire-plugin in your pom.xml like this:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/foobar/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.14.1</version>
<configuration>
<testSourceDirectory>src/foobar/java</testSourceDirectory>
</configuration>
</plugin>
</plugins>
</build>
Now again the simple usage of mvn test will execute the test at the not standard location.
I have written a JUnit test suite for running multiple test cases.
Now I want to run my test suite class (AllTest.java) at once so that all tests are triggered, carried and managed by one class. I know maven-failsafe-plugin is available, but is there any other easier way to invoke a JUnit test suite from Maven?
I dont want to use another plugin for this.
This is my current maven-failsafe-plugin configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.9</version>
<configuration>
<includes>
<include>**/AllTests.java</include>
</includes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
You can run it with -Dit.test=[package].AllTest (-Dtest with surefire), or configure the included tests in the pom:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<includes>
<include>AllTest.java</include>
</includes>
</configuration>
</plugin>
You can run test suite using following maven command:
mvn test -Dtest=x.y.z.MyTestSuite
Note : x.y.z is the package name.
Bellow a part of my actual pom.
Testng tests for integration tests have been assigned a "integration" group in the #Test annotations.
To do little test I did not exclude the "integration" group during the test phase.
When building using for example mvn verify or mvn install the integration tests get executed in the test phase, but not the verify or integration-test phase.
The number of tests run remains 0. Somehow they are not picked up. Anyone have an idea on what might be wrong?
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.1</version>
<configuration>
<skip>false</skip>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.1</version>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<skip>false</skip>
<excludedGroups>unit</excludedGroups>
</configuration>
</execution>
<execution>
<id>verify</id>
<phase>verify</phase>
<goals>
<goal>verify</goal>
</goals>
<configuration>
<skip>false</skip>
<excludedGroups>unit</excludedGroups>
</configuration>
</execution>
</executions>
<configuration>
<skip>false</skip>
<excludedGroups>unit</excludedGroups>
</configuration>
</plugin>
Update: Adding TestNG as a dependency to the failsafe plugin does not
help
maven-failsafe-plugin by default includes only files matching following patterns:
<includes>
<include>**/IT*.java</include>
<include>**/*IT.java</include>
<include>**/*ITCase.java</include>
</includes>
How did you name your test class(es)?