If argument in maven Surefire plugin, skip E2E tests - maven

I couldn't seem to find anything on this but I'm curious if I can pass an argument during runtime to skip all of our projects E2E tests.
Is there anyway for me to do something like the segregated exclude block in the following pom example?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>${argLine}</argLine>
<excludes>
<exclude unless="${skip.E2E.tests}> **/*E2E*.java</exclude>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
</plugin>
Then I could just call mvn clean install -Dskip.E2E.tests=true. Anybody seen anything like this?
I suppose I could do something like...
<exclude>${name.of.tests.to.exclude}</exclude>
and then mvn clean install -Dname.of.tests.to.exclude=**/*E2E*.javabut I would prefer to get an easy true or false argument to set rather than this in case some of the tests I want to skip do not include E2E and I need to add them to a list.

It's hard to tell just from the snippet of your pom that you are showing, but it looks like you are using surefire for both your unit and your e2e tests. Instead, you should consider using the failsafe plugin for e2e.
One benefit is that the e2e tests will run in a different stage so you get the behavior looking for by default. They are run during the verify stage of the project build. So, you can run mvn test to run unit tests only.
You can configure your project to use fail-safe like this:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
Run them using: mvn verify
Running mvn install -DskipITs will skip only integration tests, while still running unit tests.
And running mvn install -DskipTests will skip both integration and unit tests.

If you want to implement such a condition, you could use Maven profiles and have two configuration:
The default one as part of the normal build, not skipping the E2E tests
the profiled one skipping them
The profile could be then activated upon property or direct activation.
As an example you could have:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>${argLine}</argLine>
<excludes>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>skip.E2E.tests</id>
<activation>
<property>
<name>skip.E2E.tests</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>${argLine}</argLine>
<excludes>
<exclude>**/*E2E*.java</exclude>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
Note: the default Maven Surefire Plugin applying to normal build and then a profiled one.
Running:
mvn clean install
Will not activate the profile and your build will skip the tests. While running:
mvn clean install -Pskip.E2E.tests
or
mvn clean install -Dskip.E2E.tests=true
Will activate the profile and as such add the exclusion to the tests execution.
So this is exactly the scenario you were looking for, I presume.
Alternatively and as suggested by #AndrewEisenberg in the another answer, you could use the Maven Failsafe Plugin for different type of tests. The main two differences are that: it has different phase bindings AND when it fails, it does it in a safer way. As from official documentation:
If you use the Surefire Plugin for running tests, then when you have a test failure, the build will stop at the integration-test phase and your integration test environment will not have been torn down correctly.
The Failsafe Plugin is used during the integration-test and verify phases of the build lifecycle to execute the integration tests of an application. The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute

Related

Build parent pom and run specific integration test in Jenkins through maven?

Executing mvn clean install on my parent pom builds all the sub-modules and runs the related junit tests. It doesn't run the integration tests.
After building and running the junits, I want to run a specific integration test in a particular sub-module.
I am using following command:
mvn clean install -DTest=integrationTestName
The job fails with No Test Found error during build phase.
I also tried using
mvn clean install -DTest=integrationTestName -pl=sub-module-name
but this only builds my sub-module having integration test.
Question: How to run a single integration test of a certain module?
I suppose you tried the test option (lower case) of the Maven Surefire Plugin to invoke a specific test, which Surefire couldn't find in the first module of the reactor build and hence fails.
I also suppose integration tests are executed by the Maven Failsafe Plugin. If not, they should, as mentioned by official documentation:
The Failsafe Plugin is designed to run integration tests while the Surefire Plugin is designed to run unit tests. ... If you use the Surefire Plugin for running tests, then when you have a test failure, the build will stop at the integration-test phase and your integration test environment will not have been torn down correctly. .. The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute.
To make it short: it's safer and much more robust to do so.
Although the plugin configuration entry for the Maven Failsafe Plugin is also test, its corresponding command line option is it.test, so you should instead run:
mvn clean install -Dit.test=SampleIT
Where SampleIT is an integration Test (note the standard IT suffix, recognized by default by Failsafe.
The official Running a Single Test documentation also provides further details on running single integration tests.
Also note: the invocation above will work if you don't have other integration tests in other modules of the build, otherwise it will fail not finding it earlier (before hitting the concerned module).
If you are using Surefire to run the concerned integration test (again, you shouldn't) or you have several modules running integration tests, you should then configure a profile in the concerned module which would take care of this specific invocation, something like as following:
<profiles>
<profile>
<id>run-single-it-test</id>
<activation>
<property>
<name>single.it.test</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<test>${single.it.test}</test>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
And then invoke the build as following:
mvn clean install -Dsingle.it.test=SampleIT
As such, Maven will activate the profile based on the existence of a value for the single.it.test property and pass it to the test property of the Failsafe Plugin (or the Surefire, if it was the case), Failsafe will ignore any other integration test for that execution and other modules would not suffer any impact (ignoring that property).
A_Di-Matteo's answer gets you most of the way there, but you need the following config for maven-surefire-plugin to suppress all the unit tests as well.
<profiles>
<profile>
<id>run-single-it-test</id>
<activation>
<property>
<name>single.it.test</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>true</skipTests>
<failIfNoTests>false</failIfNoTests>
</configuration>
<executions>
<execution>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<test>${single.it.test}</test>
<failIfNoTests>false</failIfNoTests>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

exclude maven module with intergration tests from test phase

I have a multi module maven project with a structure
parent
pom.xml
module
pom.xml
core-api
pom.xml
integ-tests
pom.xml
I have maven surefire plugin setup for executing the unit tests '*Test.java' which are houses in the 'core-api' module.
We have slow long-running integration tests housed in a separate 'integ-tests' module. we use '*Test.java' for our integ tests as well.
We need to be able to compile all source code but want to exclude the 'integ-test' from running as part of the default maven 'test' phase. We plan to use a profile to enable the test phase of the 'integ-test' module. I don't want to use the 'failsafe' plugin.
A matrix outlining the combination is here
mvn | core | integ-test
test | run unit tests | exclude
test -PintegTest | unit tests | integ tests
I've defined the surefire plugin in my parent pom, with a property 'skip.integ.tests' which will be controlled via a profile '-PintegTests'.
<properties>
<skip.integ.tests>true</skip.integ.tests>
</properties>
..
<build>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
</build>
..
<profiles>
<profile>
<id>integTests</id>
<properties>
<skip.integ.tests>false</skip.integ.tests>
</properties>
</profile>
</profiles>
In my 'integ-test' pom, i've then overridden the 'maven-surefire-plugin' config and have the 'skipTests' configuration set to look at the value of the property.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>${skip.integ.tests}</skipTests>
</configuration>
</plugin>
</plugins>
</build>
My problem is the integ-test module tests run in every case. Any ideas on where i'm going wrong with the setup?
First you should name your integration tests accordingly to the naming conventions of the maven-failsafe-plugin which is intended to run the integration tests. Furthermore the pre-integration-test, integration-test and post-integration-test life cycle phases are intended for running those tests. This means in your case to configure the maven-failsafe-plugin accordingly to the documentation like this. The maven-failsafe-plugin bounds to the integration-test life cycle phase.
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
I would suggest to add the following profile into your integration test module like this:
<profiles>
<profile>
<id>run-its</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
This gives you simply the following options:
mvn test
running unit tests only
mvn -DskipTests=true test
running compilation etc. but no unit tests.
mvn -Prun-its verify
Running packaging etc. unit tests and integration tests
mvn install
Running installation without integration tests.
mvn -DskipTests=true install
Running installation without running unit tests nor integration tests.
In your Maven build, you can exclude:
compilation and execution of both unit tests (by Surefire plugin) and integration tests (by Failsafe plugin) by adding -Dmaven.skip.test=true
execution of both unit and integration tests via -DskipTests
execution of integration tests via -DskipITs
Instead, if you have your integration tests in a separate Maven module (i.e. integ-test in your case), you can directly exclude that from the Maven build via profile, like in below example -- see extract of aggregator's pom.xml and maven command line to be launched:
<modules>
<!-- remove 'integ-test' from this list -->
</modules>
<profiles>
<profile><id>build-it</id>
<activation><activeByDefault>true</activeByDefault></activation>
<modules><module>integ-test</module></modules>
</profile>
</profiles>
and then mvn install -P !build-it

How can I skip tests in maven install goal, while running them in maven test goal?

I have a multi-module maven project with both integration and unit tests in the same folder (src/test/java). Integration tests are marked with #Category(IntegrationTest.class). I want to end up with the following setup:
If I run mvn install, I want all tests to compile, but I do not want to execute any.
If I run mvn test, I want all tests to compile, but execute only unit tests.
If I run mvn integration-test, I want to compile and execute all tests.
The important point is, I want this configured in the pom.xml without any extra commandline arguments.
Currently I came up with the following setup in my parent pom.xml, where the only problem is #1, where all tests are executed:
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>${project.java.version}</source>
<target>${project.java.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.14.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.14.1</version>
</dependency>
</dependencies>
<configuration>
<includes>
<include>**/*.class</include>
</includes>
<excludedGroups>cz.cuni.xrg.intlib.commons.IntegrationTest</excludedGroups>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.14.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.14.1</version>
</dependency>
</dependencies>
<configuration>
<groups>cz.cuni.xrg.intlib.commons.IntegrationTest</groups>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<includes>
<include>**/*.class</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
All child modules have the following plugin configuration in their pom.xml, which I believe should inherit from the parent pom:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
</plugins>
</build>
I tried using <skipTests>true</skipTests>, but it disables test execution for all goals, which is not what I want (violates #2 and #3). It is also quite weird, that mvn test honors the skipTests=true option...why would I want to run it in the first place??
After hours of googling and trying different combinations, I am hesitant whether it is even possible to not run tests in mvn install, while at the same time run them in mvn test. I hope someone proves this wrong. ;)
I am also willing to accept a solution, where mvn install would execute only unit tests, but I don't think it makes much difference.
It sounds like you didn't understand the concept of the build life-cycle in Maven. If you run mvn install all life-cycle phases (including the install phase itself) run before the install phase. This means running the following phases:
validate
initialize
generate-sources
process-sources
generate-resources
process-resources
compile
process-classes
generate-test-sources
process-test-sources
generate-test-resources
process-test-resources
test-compile
process-test-classes
test
prepare-package
package
pre-integration-test
integration-test
post-integration-test
verify
install
which means in other words the test as well as integration-test life-cycle phases are included. So without any supplemental information it's not possible to change the behaviour as you wish it.
It could be achieved by using a profile in Maven:
<project>
[...]
<profiles>
<profile>
<id>no-unit-tests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
[...]
</project>
So your first requirement:
If I run mvn install, I want all tests to compile, but I do not want to execute any.
can be achieved by using the following:
mvn -Pno-unit-test test
If I run mvn test, I want all tests to compile, but execute only unit tests.
This can simply achieved by using the plain call:
mvn test
cause the integration tests phase is not run (see the build life cycle).
If I run mvn integration-test, I want to compile and execute all tests.
This means running the default which includes running the test phase which will run the unit tests (maven-surefire-plugin) and furthermore running the integration test which are handled by the maven-failsafe-plugin. But you should be aware that if you like to call the integration tests you should using the following command:
mvn verify
instead, cause you missed the post-integration-test phase in your previous call.
Apart from the above you should follow the naming conventions for unit and integration tests where unit tests should be named like the following:
<includes>
<include>**/*Test*.java</include>
<include>**/*Test.java</include>
<include>**/*TestCase.java</include>
</includes>
and integration tests should be named like the following:
<includes>
<include>**/IT*.java</include>
<include>**/*IT.java</include>
<include>**/*ITCase.java</include>
</includes>
I hope you have configured the maven-failsafe-plugin like the following which is needed to bound the maven-failsafe-plugin to the correct life-cycle-phases:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.15</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
as you correctly did, but you should be aware that the include tags work on the source code (.java) and not on the compiled names (.class). I wouldn't use the Category annotation, just simply using the naming conventions makes the pom simpler and shorter.
According to the Failsafe Plugin documentation
mvn install -DskipITs
is what you want.
What OP stated in his question:
If I run mvn install, I want all tests to compile, but I do not want
to execute any.
If I run mvn test, I want all tests to compile, but execute only unit tests.
If I run mvn integration-test, I want to compile and execute all tests.
is perfectly valid and extremely easy to achieve.
EDIT: except first condition, which acts againts the maven nature. The best way here would be simply do mvn install -DskipTests
All you need is following snippet in pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.17</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
and to stick to the maven naming conventions for unit and integration tests (as #khmarbaise already stated). So generally name you integration tests with IT suffix (for example MyIntegrationTestIT.java) and let maven-failsafe do its job.
In that way, you do not even need JUnit categories (although sometimes they can be quite useful).
That's it :)
mvn test executes only unit tests
mvn integration-test executes all tests
mvn failsafe:integration-test runs only integration tests
mvn clean verify when you want to be sure, that whole project just works
Some personal advices
Keeping integration tests separately from unit tests lets you easily run within your IDE all tests in some package. Usually additional directory called test-integration (or integrationtest) is used for this purpose.
This is also easy to achieve with maven:
<plugin>
<!-- adding second test source directory (just for integration tests) -->
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>add-integration-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/test-integration/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
And then move your integration tests to that directory. It should look like:
src
main
test
test-integration
Integration tests usually needs more memory:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
...
<configuration>
<argLine>-Xmx512m -XX:MaxPermSize=256m</argLine>
</configuration>
</plugin>
This post explains how to skip integration tests, no matter what plugin you are using for these tests.
Basically, what you do is define a profile and put all your integration-tests related xml code inside that profile. Than you activate it when a property -DskipIntegrationTests is missing.
You can do the same for unit tests: write a profile and activate it when -DskipUnitTests is missing.
Then, you could do:
mvn install -DskipIntegrationTests -DskipUnitTests # (runs install without any tests)
mvn test # (runs unit tests)
mvn post-integration-test # (runs all tests)
The maven-failsafe-plugin docs has a section titled "Skipping by Default."
Sadly, the steps that page describes don't work as written. However, a slight change to those steps will make it work:
In the properties section of pom.xml, add this:
<skipITs>true</skipITs>
Then add the skipTests property to the plugin section of maven-failsafe-plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<skipTests>${skipITs}</skipTests>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
So now, an mvn install by default will execute unit tests, but not integration tests.
But an mvn install -DskipITs=false will execute both unit tests and integration tests.
Footnote: Bad documentation played a big part on why Maven was so disliked for such a long time.
mvn test-compile does exactly what you are looking for. You can simply replace mvn install with mvn test-compile and you are done. No need to customise the pom file or anything. The below linked question is similar around #1:
Maven - How to compile tests without running them ?
mvn test-compile should be accepted as the best answer as Maven supports exactly what you want to do natively and without any magic. You would end up with this:
If I run mvn test-compile, I want all tests to compile, but I do not want to execute any.
If I run mvn test, I want all tests to compile, but execute only unit tests.
If I run mvn integration-test, I want to compile and execute all tests.
Don't specify the execution step(s) in the configuration of the failsafe plugin. E.g.
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M3</version>
</plugin>
</plugins>
Now, you specifically need to call mvn failsafe:integration-test to run the integration tests; they will be skipped in other mvn targets.

Run Junit Suite using Maven Command

I have multiple Junit test suites (SlowTestSuite, FastTestSuite etc). I would like to run only specific suite using maven command. e.g.
mvn clean install test -Dtest=FastTestSuite -DfailIfNoTests=false
but its not working. Just not running any test at all. Any suggestions please.
I have achieved this by adding property into pom as:
<properties>
<runSuite>**/FastTestSuite.class</runSuite>
</properties>
and maven-surefire-plugin should be:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>${runSuite}</include>
</includes>
</configuration>
</plugin>
so it means by default it will run FastTestSuite but you can run other test e.g. SlowTestSuite using maven command as:
mvn install -DrunSuite=**/SlowTestSuite.class -DfailIfNoTests=false
The keyword you missed is maven-surefire-plugin :http://maven.apache.org/plugins/maven-surefire-plugin/.
Usage is :
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.1</version>
<configuration>
<includes>
<include>**/com.your.packaged.Sample.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
If you make a little search on stack overflow, you may find information :
Running a JUnit4 Test Suite in Maven using maven-failsafe-plugin
Using JUnit Categories with Maven Failsafe plugin
In addition, you may define profile, like fastTest, that will be triggered by adding parameter to cmd line :
mvn package -PfastTests
This profile would include some inclusions too.

Skipping tests in some modules in Maven

I would like my Maven builds to run most unit tests. But there are unit tests in one project which are slower and I'd like to generally exclude them; and occasionally turn them on.
Question: How do I do this?
I know about -Dmaven.test.skip=true, but that turns off all unit tests.
I also know about skipping integration tests, described here. But I do not have integration tests, just unit tests, and I don't have any explicit calls to the maven-surefire-plugin. (I am using Maven 2 with the Eclipse-Maven plugin).
What about skipping tests only in this module ?
In the pom.xml of this module:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
Eventually, you can create a profile that will disable the tests (still the pom.xml of the module) :
<project>
[...]
<profiles>
<profile>
<id>noTest</id>
<activation>
<property>
<name>noTest</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
[...]
</project>
With the latter solution, if you run mvn clean package, it will run all tests. If you run mvn clean package -DnoTest=true, it will not run the tests for this module.
I think this is easier, and also has the benefit of working for non-surefire tests (in my case, FlexUnitTests)
<profile>
<id>noTest</id>
<properties>
<maven.test.skip>true</maven.test.skip>
</properties>
</profile>
If you have a large multi-module project and you would like to skip tests only in certain modules without the need to change each of the module pom.xml file with custom configuration and profiling, you could add the following to the parent pom.xml file:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<execution>
<id>regex-property</id>
<goals>
<goal>regex-property</goal>
</goals>
<configuration>
<name>maven.test.skip</name>
<value>${project.artifactId}</value>
<regex>(module1)|(module3)</regex>
<replacement>true</replacement>
<failIfNoMatch>false</failIfNoMatch>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<modules>
<module>module1</module>
<module>module2</module>
<module>module3</module>
</modules>
Thanks to the build-helper-maven-plugin you would actually dynamically check whether you are in a certain module or not during the build, via the project.artifactId property (pointing at each artifactId module during the build), the regex would then seek matching for certain values (the module names for which you want to skip tests) and populated the maven.test.skip property accordingly (setting it to true).
In this case, tests will be skipped for module1 and module3 while running properly for module2, that is, as expressed by the regex.
The advantage of this approach is to have it dynamic and centralized (in the parent pom.xml) hence better for maintenance: you could add or remove modules at any time simply by changing the simple regex above.
Obviously, if this is not the default behavior of the build (recommended case), you could always wrap the snippet above in a maven profile.
You could also go further and have dynamic behavior based on your input:
<properties>
<test.regex>none</test.regex>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<execution>
<id>regex-property</id>
<goals>
<goal>regex-property</goal>
</goals>
<configuration>
<name>maven.test.skip</name>
<value>${project.artifactId}</value>
<regex>${test.regex}</regex>
<replacement>true</replacement>
<failIfNoMatch>false</failIfNoMatch>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Here we are actually replacing the regex value with a property, test.regex, with default value to none (or whatever would not match any module name or, also, the default skipping matchings required).
Then from command line we could have
mvn clean test -Dtest.regex="(module1)" > will skip tests only for module1
mvn clean test -Dtest.regex="(module1)|(module2)" > will skip tests on module1 and module2
mvn clean test -Dtest.regex="(module1)|(module2)|(module3)" > will skip the three module tests
mvn clean test -Dtest.regex=".+" > will skip all module tests
mvn clean test > would not skip anything (or fall back on default behavior)
That is, then at runtime you decide, without any need to change the pom.xml file or activating any profile.
Using Surefire Plugin 2.19 you can simply exclude the tests you don't want using regular expressions:
mvn '-Dtest=!%regex[.*excludedString.*]' test
The above command will exclude all the tests that contain excludedString.
NB1 If double quotation mark(") is used instead of apostrophe(') the command will not be interpreted properly and will produce unexpected results. (Tested using bash 3.2.57)
NB2 Particular attention should be paid to projects in which multiple version of the surefire plugin is used. Versions of surefire older than 2.19 will not execute any tests because they do not support regular expressions.
Version management(it might be a good idea to add this in the parent pom file):
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
</plugin>
</plugins>
</pluginManagement>
</build>
Examples of build commands that skip tests: https://artbcode.wordpress.com/2016/11/28/how-to-skip-a-subset-of-the-unit-tests/
I had a slightly different need from this question that may prove helpful. I wanted to exclude from the command line a few different tests from different packages, so a single wildcard would not do it.
I found in the Maven Failsafe documentation rules for exclusions that you can specify a comma-separated list of either regex or wildcard exclusions:
https://maven.apache.org/surefire/maven-failsafe-plugin/examples/inclusion-exclusion.html
So my pomfile looked like this:
<excludes>
<exclude>${exclude.slow.tests}</exclude>
</excludes>
and my command line included this:
mvn install "-Dexclude.slow.tests=**/SlowTest1.java, **/package/ofslowtests/*.java, **/OtherSlowTest.java"
For me the key ingredient was getting a bunch of tests into one maven property in a single exclude statement.

Resources