Splitting unit and integration tests with Maven - maven

I'm setting up a CD pipeline in Jenkins and I want to run my unit tests and integration tests in two different steps. The plan is to have my pipeline script look something like this and have them run separately:
stage('Unit tests') {
steps {
withMaven(maven: 'Maven 3.6.2') {
sh 'mvn test -P coverage'
}
}
}
stage('Integration tests') {
steps {
withMaven(maven: 'Maven 3.6.2') {
sh 'mvn test -P coverage'
}
}
I have tried using the surefire plugin as described here: https://dzone.com/articles/splitting-unit-and-integration-tests-using-maven-a, and running 'mvn test' does run only the unit test as it should, but 'mvn integration-test' runs both unit and integration tests.
I have also tried using the failsafe plugin as described here: Maven separate Unit Test and Integration Tests, but 'mvn verify' runs both unit and integration tests no matter which options I enter.
How can I make my pipeline execute the unit tests and integration tests in two different steps?
Pom with surefire:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId> <!-- surefire plugin version managed by Spring Boot -->
<configuration>
<skipTests>true</skipTests>
</configuration>
<executions>
<execution>
<id>unit-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skipTests>false</skipTests>
<includes>
<include>**/*Test.java</include>
</includes>
</configuration>
</execution>
<execution>
<id>integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skipTests>false</skipTests>
<includes>
<include>**/*IT.*</include>
<include>**/*Tests.*</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
Pom with surefire and failsafe:
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>${surefire.skip}</skip>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>

This solved it for me:
Below is example pom.xml which works with the following commands:
mvn clean verify -DskipUTs=true : Runs only integration tests (tests ending in "IT")
mvn clean verify -DskipITs=true : Runs only unit tests (tests ending in "Test")
mvn clean verify -DskipTests=true : Skips all tests
<build>
<finalName>test-app</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId> <!-- surefire plugin version managed by Spring Boot -->
<configuration>
<skipTests>${skipUTs}</skipTests>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M3</version>
<executions>
<execution>
<id>run-integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<skipTests>${skipTests}</skipTests>
<skipITs>${skipITs}</skipITs>
</configuration>
</plugin>
</plugins>
</build>

Related

Running spock integrations tests in its own folder in maven

I am trying to only run integration tests in maven with mvn. I have my tests in src/it/java for java integration tests and src/it/groovy for Spock integration tests. When I run mvn failsafe:integration-test, only the java integration tests in src/it/java are run.
This is my pom:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<id>add-integration-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/it/java</source>
<source>src/it/groovy</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<testSourceDirectory>src/it/groovy</testSourceDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.6</version>
<configuration>
<testSources>
<testSource>
<directory>src/it/groovy</directory>
<includes>
<include>**/*IT.groovy</include>
</includes>
</testSource>
</testSources>
</configuration>
<executions>
<execution>
<id>groovy-compile</id>
<goals>
<goal>compileTests</goal>
</goals>
</execution>
</executions>
</plugin>
Any idea what I am missing here?
It looks like the issue is that your running only failsafe:integration-test. I am able to run groovy and java tests using the command:
mvn verify
In the failsafe docs it suggests to use the verify phase here - http://maven.apache.org/surefire/maven-failsafe-plugin/index.html
If mvn verify still doesn't run your groovy tests, check your_project/target/test-classes/path_to_your_test for expected class files and make sure that lines up with the includes option in the pom.

Maven Integration Testing Command

I created Maven project using Java for automation testing. When i run mvn verify it does not open any browser. What wrong with it? I can easily run testng class from eclipse but i need to run for CI/CD.
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
Please help. Thanks.

How to separate Unit Tests and Integration Tests for different levels of maven lifecycle

Here is my plugin configuration from pom.xml file
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<id>run-unit-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
<configuration>
<skipITs>true</skipITs>
<skipUTs>false</skipUTs>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<id>run-integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
<configuration>
<skipITs>false</skipITs>
<skipUTs>false</skipUTs>
</configuration>
</plugin>
I have 2 test classes HelloTest.java and WorldITest.java.
When I run mvn clean test i want it to execute only HelloTest. But when i execute mvn clean integration-test, then I want both HelloTest and WorldITest to be executed.
But the problem is that it executes the Integration test also in mvn clean test command.
PS: Original code was taken from the discussion : Prevent unit tests in maven but allow integration tests

Ensure Maven build fails when tests fail to run

I've noticed that sometimes when running maven builds on Jenkins the number of Jbehave tests that are run vary from one run to another. When analyzing the logs I see the following snippet:
Failed to run story stories/cancel.story
java.lang.InterruptedException: stories/cancel.story
at org.jbehave.core.embedder.StoryRunner$RunContext.interruptIfCancelled(StoryRunner.java:616)
at org.jbehave.core.embedder.StoryRunner.runStepsWhileKeepingState(StoryRunner.java:514)
at org.jbehave.core.embedder.StoryRunner.runScenarioSteps(StoryRunner.java:479)
at org.jbehave.core.embedder.StoryRunner.runStepsWithLifecycle(StoryRunner.java:445)
at org.jbehave.core.embedder.StoryRunner.runCancellable(StoryRunner.java:305)
at org.jbehave.core.embedder.StoryRunner.run(StoryRunner.java:220)
at org.jbehave.core.embedder.StoryRunner.run(StoryRunner.java:181)
at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:235)
at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:207)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
The problem is that when tests are skipped or fail to run in this way the build is still considered a success.
Is there a maven surefire plugin configuration that will ensure that whenever tests fail to run the build results in a failure? Here are the maven surefire build configurations
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.11</version>
<configuration>
<includes>
<include>**/*TestSuite.java</include>
</includes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9</version>
</plugin>
<plugin>
<groupId>net.thucydides.maven.plugins</groupId>
<artifactId>maven-thucydides-plugin</artifactId>
<version>${thucydides.version}</version>
<executions>
<execution>
<id>thucydides-reports</id>
<phase>post-integration-test</phase>
<goals>
<goal>aggregate</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.2</version>
<configuration>
<reportPlugins>
<plugin>
<groupId>net.thucydides.maven.plugins</groupId>
<artifactId>maven-thucydides-plugin</artifactId>
<version>${thucydides.version}</version>
</plugin>
</reportPlugins>
</configuration>
</plugin>
</plugins>
</build>
Your maven-surefire-plugin is set to skip tests completely (with <skip>true</skip>), so tests are running with maven-failsafe-plugin. That plug-in is supposed to not stop on failure during integration-test, and then only fail on verify phase.
So if you really want this question answered:
Is there a maven surefire plugin configuration that will ensure that whenever tests fail to run the build results in a failure?
That is: you want maven-surefire-plugin to run the tests, and not the maven-failsafe-plugin, then the answer is: remove
<configuration>
<skip>true</skip>
</configuration>
from your POM. In this case you also don't need maven-failsafe-plugin configuration, because it would just make your tests run twice.
But if your goal is to get maven-failsafe-plugin to work, then I think you may have one of the following issues:
Not running the right goal. As plug-in help states, you should invoke it as
mvn verify
An old plug-in, which is not compatible with test framework you are using (current version is 2.19.1)
Or this help recommendation:
For very complex builds, it may be better to separate the executions for the integration-test and verify goals:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>

exclude Test.java class from maven build war

I've tried to exclude some .java file of src/test/java from being packaged in maven build. My pom.xml is:
plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<inherited>true</inherited>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
<executions>
<execution>
<id>default-testCompile</id>
<phase>test-compile</phase>
<configuration>
< test Excludes >
<exclude>**/Test1.java</exclude>
<exclude>**/Test2.java</exclude>
<exclude>**/Test3.java</exclude>
<exclude>**/Test4.java</exclude>
<exclude>**/Test5.java</exclude>
</testExcludes>
</configuration>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
enter code here
But When I'm trying to run my Test classes that is excluded from build, is giving ClassNotFoundException.
Kindly, guide me..
I have added below plugins so now its skipping the test cases:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.2</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>

Resources