Why won't my run configuration using a maven profile work as intended? Why does the profile fail to skip tests while the parameter does work as intended?
I have a maven run configuration which doesn't run as intended (it runs tests in lib-module which I want skipped):
-pl common,lib-module,my-module test -PdontTestLibProfile
When I run the same run config but using the maven property parameter directly:
-pl common,lib-module,my-module test -DdontTestLib=true the skipping of lib-module tests succeeds.
The maven profile dontTestLibProfile lives in the pom of my-module.
<artifact-id>my-module</artifact-id>
[...]
<profile>
<id>dontTestLibProfile</id>
<properties>
<dontTestLib>true</dontTestLib>
</properties>
</profile>
dontTestLib is a mavenProperty which is checked in the lib-module, where if it is set to true it will not run integration or unit tests.
<artifactId>lib-module</artifactId>
<properties>
<!-- Skip tests: Default is to NOT skip ==> Default is to run -->
<doneTestLib>false</dontTestLib>
</properties>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<!-- Skip Lib-Module Integration tests if skip is set -->
<skipTests>${dontTestLib}</skipTests>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<!-- Skip Lib-Module Unit tests if skip is set -->
<skipTests>${dontTestLib}</skipTests>
</configuration>
</plugin>
</plugins>
</build>
Some corollary questions which might be the cause of this issue:
Do maven profiles run in the same sequence as the project list itself, not allowing my-module to affect the lib-module which comes before it?
Do profiles only affect their own POM/module and I am trying to reach outside of the scope?
Related
I couldn't seem to find anything on this but I'm curious if I can pass an argument during runtime to skip all of our projects E2E tests.
Is there anyway for me to do something like the segregated exclude block in the following pom example?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>${argLine}</argLine>
<excludes>
<exclude unless="${skip.E2E.tests}> **/*E2E*.java</exclude>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
</plugin>
Then I could just call mvn clean install -Dskip.E2E.tests=true. Anybody seen anything like this?
I suppose I could do something like...
<exclude>${name.of.tests.to.exclude}</exclude>
and then mvn clean install -Dname.of.tests.to.exclude=**/*E2E*.javabut I would prefer to get an easy true or false argument to set rather than this in case some of the tests I want to skip do not include E2E and I need to add them to a list.
It's hard to tell just from the snippet of your pom that you are showing, but it looks like you are using surefire for both your unit and your e2e tests. Instead, you should consider using the failsafe plugin for e2e.
One benefit is that the e2e tests will run in a different stage so you get the behavior looking for by default. They are run during the verify stage of the project build. So, you can run mvn test to run unit tests only.
You can configure your project to use fail-safe like this:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
Run them using: mvn verify
Running mvn install -DskipITs will skip only integration tests, while still running unit tests.
And running mvn install -DskipTests will skip both integration and unit tests.
If you want to implement such a condition, you could use Maven profiles and have two configuration:
The default one as part of the normal build, not skipping the E2E tests
the profiled one skipping them
The profile could be then activated upon property or direct activation.
As an example you could have:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>${argLine}</argLine>
<excludes>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>skip.E2E.tests</id>
<activation>
<property>
<name>skip.E2E.tests</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>${argLine}</argLine>
<excludes>
<exclude>**/*E2E*.java</exclude>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
Note: the default Maven Surefire Plugin applying to normal build and then a profiled one.
Running:
mvn clean install
Will not activate the profile and your build will skip the tests. While running:
mvn clean install -Pskip.E2E.tests
or
mvn clean install -Dskip.E2E.tests=true
Will activate the profile and as such add the exclusion to the tests execution.
So this is exactly the scenario you were looking for, I presume.
Alternatively and as suggested by #AndrewEisenberg in the another answer, you could use the Maven Failsafe Plugin for different type of tests. The main two differences are that: it has different phase bindings AND when it fails, it does it in a safer way. As from official documentation:
If you use the Surefire Plugin for running tests, then when you have a test failure, the build will stop at the integration-test phase and your integration test environment will not have been torn down correctly.
The Failsafe Plugin is used during the integration-test and verify phases of the build lifecycle to execute the integration tests of an application. The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute
I have an application contains both Cucumber and JBehave test, I want to be able to run one of them optionally every time, I can do that with JBehave by explicit Maven goal, but the problem is that Cucumber run implicitly with each build or test, is there anyway to stop and run it o choice?
You can achieve this by configuring the Maven Surefire Plugin as part of your default build or/and via a profile.
If your Maven build section, you can skip the Cucumber tests by default (given that they either have all the same suffix or belong all to the same package, alternatively you can arrange them to meet any of these two criterias):
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<excludes>
<!-- classes that include the name CucumberTest, as an example -->
<exclude>**/*CucumberTest*.java</exclude>
<!-- classes in a package whose last segment is named cucumber, as an example -->
<exclude>**/cucumber/*.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
As such, Maven by default (as part of the default build) will skip your Cucumber tests.
Then, you can configure a Maven Profile to run exclusively the Cucumber tests with a counterpart of the Maven Surefire Plugin configuration as following:
<project>
[...]
<profiles>
<profile>
<id>cucumber-tests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<!-- Include your Cucumber tests, as an example -->
<include>**/*CucumberTest.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
<profile>
</profiles>
[...]
</project>
Then running mvn clean install -Pcucumber-tests will run your Cucumber tests.
This approach would give you more flexibility on configuration in both scenarios (default or cucumber tests build) and you could swap the behavior according to your needs.
Alternatively, for a simpler (but less flexible) approach, you could follow the suggestion on another SO answer and use a Maven property to have a switch cucumber tests on/off as following:
<properties>
<exclude.cucumber.tests>nothing-to-exclude</exclude.cucumber.tests>
</properties>
<profiles>
<profile>
<id>exclude-cucumber</id>
<properties>
<exclude.cucumber.tests>**/*Cucumber*.java</exclude.cucumber.tests>
</properties>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>${exclude.cucumber.tests}</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
Using the configuration above, by default Maven will execute Cucumber tests and skip them when executing mvn clean install -Pexclude-cucumber (the profile will change the content of the exclude.cucumber.tests property and as such change the Surefire plugin filter). You can of course swap the behavior is as well and have an include-cucumber profile instead.
I have a multi module maven project with a structure
parent
pom.xml
module
pom.xml
core-api
pom.xml
integ-tests
pom.xml
I have maven surefire plugin setup for executing the unit tests '*Test.java' which are houses in the 'core-api' module.
We have slow long-running integration tests housed in a separate 'integ-tests' module. we use '*Test.java' for our integ tests as well.
We need to be able to compile all source code but want to exclude the 'integ-test' from running as part of the default maven 'test' phase. We plan to use a profile to enable the test phase of the 'integ-test' module. I don't want to use the 'failsafe' plugin.
A matrix outlining the combination is here
mvn | core | integ-test
test | run unit tests | exclude
test -PintegTest | unit tests | integ tests
I've defined the surefire plugin in my parent pom, with a property 'skip.integ.tests' which will be controlled via a profile '-PintegTests'.
<properties>
<skip.integ.tests>true</skip.integ.tests>
</properties>
..
<build>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
</build>
..
<profiles>
<profile>
<id>integTests</id>
<properties>
<skip.integ.tests>false</skip.integ.tests>
</properties>
</profile>
</profiles>
In my 'integ-test' pom, i've then overridden the 'maven-surefire-plugin' config and have the 'skipTests' configuration set to look at the value of the property.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>${skip.integ.tests}</skipTests>
</configuration>
</plugin>
</plugins>
</build>
My problem is the integ-test module tests run in every case. Any ideas on where i'm going wrong with the setup?
First you should name your integration tests accordingly to the naming conventions of the maven-failsafe-plugin which is intended to run the integration tests. Furthermore the pre-integration-test, integration-test and post-integration-test life cycle phases are intended for running those tests. This means in your case to configure the maven-failsafe-plugin accordingly to the documentation like this. The maven-failsafe-plugin bounds to the integration-test life cycle phase.
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
I would suggest to add the following profile into your integration test module like this:
<profiles>
<profile>
<id>run-its</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
This gives you simply the following options:
mvn test
running unit tests only
mvn -DskipTests=true test
running compilation etc. but no unit tests.
mvn -Prun-its verify
Running packaging etc. unit tests and integration tests
mvn install
Running installation without integration tests.
mvn -DskipTests=true install
Running installation without running unit tests nor integration tests.
In your Maven build, you can exclude:
compilation and execution of both unit tests (by Surefire plugin) and integration tests (by Failsafe plugin) by adding -Dmaven.skip.test=true
execution of both unit and integration tests via -DskipTests
execution of integration tests via -DskipITs
Instead, if you have your integration tests in a separate Maven module (i.e. integ-test in your case), you can directly exclude that from the Maven build via profile, like in below example -- see extract of aggregator's pom.xml and maven command line to be launched:
<modules>
<!-- remove 'integ-test' from this list -->
</modules>
<profiles>
<profile><id>build-it</id>
<activation><activeByDefault>true</activeByDefault></activation>
<modules><module>integ-test</module></modules>
</profile>
</profiles>
and then mvn install -P !build-it
I'm trying to use the mvn scm plugin to check out the daily tag, and create an assembly from that version of the code. I configured the scm plugin and everythhing is working well, except that I can not seem to tell it to not run the unittests.
I tried:
Passing the -Dmaven.test.skip=true command line parameter
Creating a profile where the surefire plugin skips test, and list that profile in the scm configuration "profiles" section
setting the "maven.test.skip=true" as an environment variable
In all cases, when the scm plugin starts running the goals I told it to run in the configuration (see below), it also runs the unittests.
Below is the example I used to skip tests by using a profile:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-scm-plugin</artifactId>
<version>1.0</version>
<configuration>
<goals>install,assembly:assembly</goals>
<profiles>skiptest</profiles>
</configuration>
</plugin>
And this is the profile (I defined this in the pom.xml of the project):
<profiles>
<profile>
<id>skiptest</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
The command I use to do the checkout and bootstrap is:
mvn scm:bootstrap -DscmVersion=daily-20110427-421 -DscmVersionType=tag
I'm running mvn 2.2.1 on a Linux machine, and doing a checkout from a CVS repository. It's an existing project, I have continuous integration and tagging all up and running, I just want to check out a daily tag and create an assembly from that.
Any tips are much appreciated.
Edit: Got it to work with the answer below, but only after I upgraded to maven-scm-plugin version 1.1. Apparently, 1.0 did not propagate profiles.
Try this in the profile:
<profiles>
<profile>
<id>skiptest</id>
<properties>
<maven.test.skip>true</maven.test.skip>
</properties>
</profile>
</profiles>
I would like my Maven builds to run most unit tests. But there are unit tests in one project which are slower and I'd like to generally exclude them; and occasionally turn them on.
Question: How do I do this?
I know about -Dmaven.test.skip=true, but that turns off all unit tests.
I also know about skipping integration tests, described here. But I do not have integration tests, just unit tests, and I don't have any explicit calls to the maven-surefire-plugin. (I am using Maven 2 with the Eclipse-Maven plugin).
What about skipping tests only in this module ?
In the pom.xml of this module:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
Eventually, you can create a profile that will disable the tests (still the pom.xml of the module) :
<project>
[...]
<profiles>
<profile>
<id>noTest</id>
<activation>
<property>
<name>noTest</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
[...]
</project>
With the latter solution, if you run mvn clean package, it will run all tests. If you run mvn clean package -DnoTest=true, it will not run the tests for this module.
I think this is easier, and also has the benefit of working for non-surefire tests (in my case, FlexUnitTests)
<profile>
<id>noTest</id>
<properties>
<maven.test.skip>true</maven.test.skip>
</properties>
</profile>
If you have a large multi-module project and you would like to skip tests only in certain modules without the need to change each of the module pom.xml file with custom configuration and profiling, you could add the following to the parent pom.xml file:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<execution>
<id>regex-property</id>
<goals>
<goal>regex-property</goal>
</goals>
<configuration>
<name>maven.test.skip</name>
<value>${project.artifactId}</value>
<regex>(module1)|(module3)</regex>
<replacement>true</replacement>
<failIfNoMatch>false</failIfNoMatch>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<modules>
<module>module1</module>
<module>module2</module>
<module>module3</module>
</modules>
Thanks to the build-helper-maven-plugin you would actually dynamically check whether you are in a certain module or not during the build, via the project.artifactId property (pointing at each artifactId module during the build), the regex would then seek matching for certain values (the module names for which you want to skip tests) and populated the maven.test.skip property accordingly (setting it to true).
In this case, tests will be skipped for module1 and module3 while running properly for module2, that is, as expressed by the regex.
The advantage of this approach is to have it dynamic and centralized (in the parent pom.xml) hence better for maintenance: you could add or remove modules at any time simply by changing the simple regex above.
Obviously, if this is not the default behavior of the build (recommended case), you could always wrap the snippet above in a maven profile.
You could also go further and have dynamic behavior based on your input:
<properties>
<test.regex>none</test.regex>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<execution>
<id>regex-property</id>
<goals>
<goal>regex-property</goal>
</goals>
<configuration>
<name>maven.test.skip</name>
<value>${project.artifactId}</value>
<regex>${test.regex}</regex>
<replacement>true</replacement>
<failIfNoMatch>false</failIfNoMatch>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Here we are actually replacing the regex value with a property, test.regex, with default value to none (or whatever would not match any module name or, also, the default skipping matchings required).
Then from command line we could have
mvn clean test -Dtest.regex="(module1)" > will skip tests only for module1
mvn clean test -Dtest.regex="(module1)|(module2)" > will skip tests on module1 and module2
mvn clean test -Dtest.regex="(module1)|(module2)|(module3)" > will skip the three module tests
mvn clean test -Dtest.regex=".+" > will skip all module tests
mvn clean test > would not skip anything (or fall back on default behavior)
That is, then at runtime you decide, without any need to change the pom.xml file or activating any profile.
Using Surefire Plugin 2.19 you can simply exclude the tests you don't want using regular expressions:
mvn '-Dtest=!%regex[.*excludedString.*]' test
The above command will exclude all the tests that contain excludedString.
NB1 If double quotation mark(") is used instead of apostrophe(') the command will not be interpreted properly and will produce unexpected results. (Tested using bash 3.2.57)
NB2 Particular attention should be paid to projects in which multiple version of the surefire plugin is used. Versions of surefire older than 2.19 will not execute any tests because they do not support regular expressions.
Version management(it might be a good idea to add this in the parent pom file):
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
</plugin>
</plugins>
</pluginManagement>
</build>
Examples of build commands that skip tests: https://artbcode.wordpress.com/2016/11/28/how-to-skip-a-subset-of-the-unit-tests/
I had a slightly different need from this question that may prove helpful. I wanted to exclude from the command line a few different tests from different packages, so a single wildcard would not do it.
I found in the Maven Failsafe documentation rules for exclusions that you can specify a comma-separated list of either regex or wildcard exclusions:
https://maven.apache.org/surefire/maven-failsafe-plugin/examples/inclusion-exclusion.html
So my pomfile looked like this:
<excludes>
<exclude>${exclude.slow.tests}</exclude>
</excludes>
and my command line included this:
mvn install "-Dexclude.slow.tests=**/SlowTest1.java, **/package/ofslowtests/*.java, **/OtherSlowTest.java"
For me the key ingredient was getting a bunch of tests into one maven property in a single exclude statement.