mvn install is running cucumber steps also. in our local development, we need to run only unit tests and not the cucumber scenarios. Tried with -Dtest=!com.mycompany.* no luck.
And at the same time, we need to skip our Unit Tests while executing a cucumber scenario, is that possible?
This could be achieved by using build profiles. You should run your unit tests by using the surefire plugin and the cucumber scenarios with the failsafe plugin. Naming conventions for automatic running of tests for surefire and failsafe.
<profiles>
<profile>
<id>jenkins</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<excludes>
<exclude>**/*Test.java</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M3</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
Run this with mvn clean install -Pjenkins. This will only run the integration tests ie scenarios.
To run the unit tests just use mvn clean install. Surefire is invoked ie unit test, by default but not failsafe.
Related
I am attempting to have my integration tests separated from the usual lifecycle, meaning that I do not want them to be executed during a mvn install. I have the following configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<excludes>
<exclude>**/*IT.java</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<includes>
<include>**/*IT.java</include>
</includes>
</configuration>
<goals>
<goal>integration-test</goal>
</goals>
</plugin>
And my integration tests are all suffixed with IT. I was able to confirm that surefire does exclude the *IT.java tests, however it seems that failsafe is being triggered during mvn install anyway. What am I missing?
Thank you for your help
A typical configuration to handle the issue with running integration tests only if you like is a profile like the following:
<profiles>
<profile>
<id>run-its</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
By using the above configuration you can run
mvn clean verify
which will execute the unit tests. Using the following you can activate the integration tests:
mvn -Prun-its clean verify
Based on the default naming conventions the integration tests which will be executed by maven-failsafe-plugin should be named like *IT.java where as the unit tests can be name like *Test.java (will be executed via maven-surefire-plugin).
The value of property skipITs when overridden on the command line appears to be ignored by maven if used in conjunction with -Dit.test=full.Classname. The specified failsafe test will not be run (case one).
As opposed to this specifiing skipITs without the it.test switch leads to running of existing failsafe tests (case two).
Background: Am trying to isolate surefire tests from failsafe tests runs by namely either running both types of them or one of each only. The maven calls are:
mvn -Pintegration -DskipTests=true -DskipITs=false -Dit.test=full.Classname verify
in the first case and:
mvn -Pintegration -DskipTests=true -DskipITs=false verify
in the second.
The relevant configuration (pom.xml, snippets only) being:
<properties>
<skipTests>false</skipTests>
<skipITs>false</skipITs>
</properties>
(those are defaults) and
<profiles>
<profile>
<id>integration</id>
<build>
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>custom<id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<skipTests>${skipITs}</skipTests>
<skip>${skipITs}</skip>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
Did you observe that too or have you eventually found a better working approach?
By default, the Surefire plugins runs during the test phase and usually you configure the Failsafe plugin to run during the integration-test and verify phase like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
To run the Surefire tests only, use
mvn clean test
To run both the Surefire and the Failsafe tests, use
mvn clean verify
You can completely skip any plugin by using the <skip> configuration option. If you configure the Surefire plugin like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<skip>${skip.surefire}</skip>
</configuration>
</plugin>
you can run only the Failsafe tests by calling
mvn clean verify -Dskip.surefire=true
If the integration test (it.test) has a non-default name I need to add an include pattern accordingly, like in (pom.xml snippet):
<profiles>
<profile>
<id>dbUnit</id>
<build>
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>custom</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<skipTests>${skipITs}</skipTests>
<skip>${skipITs}</skip>
<includes>
<include>**/*DbTest.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
But that appears valid only for the original approach. Also observe that the default inclusion patterns might need to be added too in case for your integration tests you followed different naming schemes.
I have the following project structure:
MyProject
--src
--test
--acceptance
--step_definitions
--features
--unit
I would like to be able to run my cucumber tests (in test/acceptance) separately in Maven from the unit tests declared in test/unit, so that they can be run in different CI build plans etc. I am using cucumber-junit so the 'runners' for each acceptance test are written with JUnit.
Is this possible?
Is this possible?
Yes, it is possible. I believe you should separate your unit from the acceptance/integration tests having:
Slightly modified folders structure for both of these, placing your integration test files in the standard location of src/it:
MyProject/
src/main/java/ (SUT)
src/test/ (unit test code)
java/
resources/
src/it/ (acceptance/integration tests)
java/ (steps definitions)
resources/ (feature files)
Moreover, by design, different Maven plugins are intended for unit and integration tests:
for unit tests: maven-surefire-plugin
for acceptance/integration tests: maven-failsafe-plugin
You must also bind execution of maven-failsafe-pulgin. To run the integration tests separately, you can define a new profile:
<profiles>
<profile>
<id>acceptance-tests</id>
<build>
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
You will also need to configure the plugin to search the src/it directory tree for test cases.
The acceptance tests can be run afterwards using:
mvn clean verify -Pacceptance-tests
For complete sample, I'd suggest you to follow http://www.weblogism.com/item/334/integration-tests-with-cucumber-jvm-selenium-and-maven
The other answer suggested modifying your folder structure to have shared folder for integration and acceptance tests, but you can have the original folder structure as well. Also you mentioned in the comment you want to keep all three (including not-mentioned integration tests) separate, which is possible, though hackish.
Since you seem to have test/unit for unit tests and test/acceptance for acceptance test, I'm assuming test/integration for integration tests.
<profiles>
<profile>
<id>acceptance-test</id>
<build>
<plugins>
<plugin>
<!-- to run directly: mvn failsafe:integration-test -P acceptance-test -->
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12</version>
<configuration>
<testSourceDirectory>test/acceptance</testSourceDirectory>
<includes>
<include>**/*Acceptance.java</include>
</includes>
<excludes>
<exclude>**/*IT.java</exclude>
<exclude>**/*Test.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>test/unit</source>
<source>test/integration</source>
<source>test/acceptance</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<testSourceDirectory>test/unit</testSourceDirectory>
</configuration>
</plugin>
<plugin>
<!-- to run directly: mvn failsafe:integration-test -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<testSourceDirectory>test/integration</testSourceDirectory>
</configuration>
<!-- execution below can be used, if tests are needed on
mvn:integration-test -->
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
Note though that separation only applies for sources: compiled files will all go to the same folder, and AFAIK that's something you can't change. This means you need to have naming strategy for your tests to separate them from each other.
I am trying to run only selenium tests using mvn failsafe plugin. I created a separate profile to run only the selenium tests but mvn is not able to find them. my project structure looks like
moduleA
scr/main/...
src/test/integration/java/...
scr/test/unit/java/...
moduleB
scr/main/...
src/test/integration/java/...
scr/test/unit/java/...
moduleC (only for selenium tests)
scr/main/java/com/selenium/A.java
src/test/java/...
Since I have new directories for the unit and integration tests in moduleA and B. I have defined the following in pom.xml (listingB) to let Maven know about the additional test directories. So far so good but when I add selenium tests in moduleC and I want to run only selenium its not running selenium tests. I created a new profile (listingA) to run selenium tests. Any help is appreciated.
listingA:
<profiles>
<profile>
<id>selenium</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>verify</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/selenium/*.java</include>
</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
listinB:
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>${basedir}/src/test/unit/java</source>
<source>${basedir}/src/test/integration/java</source>
<source>${basedir}/src/test/common/java</source>
</sources>
</configuration>
</execution>
<execution>
<id>add-test-resource</id>
<phase>generate-test-resources</phase>
<goals>
<goal>add-test-resource</goal>
</goals>
<configuration>
<resources>
<resource>
<directory>${basedir}/src/test/integration/resources</directory>
<directory>${basedir}/src/test/unit/resources</directory>
<directory>${basedir}/src/test/common/resources</directory>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
The first thing which came into my mind is why you are separating the unit- and integration tests by using a different folder, cause in Maven the separation between unit- and integration tests is done by naming convention.
The unit tests have to be named like
**/Test*.java
**/*Test.java
**/*TestCase.java
The integration test have to be named like:
**/IT*.java
**/*IT.java
**/*ITCase.java
This means in other words you can put your integration tests and your unit tests into the same folder which is src/test/java without any problem. The execution of the integration tests is not influenced by this.
If you would call
mvn clean package
only the unit tests will be run. If you need to run the integration tests as well you can simply use:
mvn verify
If you wan't to skip the unit tests you can use the following:
mvn -DskipTests=true verify
For the separate selenium module which you have created the best solutin is to put the integration tests into the usual folder src/test/java with the appropriate naming convention.
The problem you might have is if your integration tests needed some special resources you might move those integration tests into the separate module with integration tests only.
If your Selenium tests are in the src/test/java folder, delete the <includes> parameter from the configuration of maven-failsafe-plugin in listingA
Name your Selenium test classes like:
**/IT*.java
**/*IT.java
**/*ITCase.java
(as khmarbaise has suggested)
Failsafe runs integration tests using Surefire, therefore don't name your Selenium test classes like:
**/Test*.java
**/*Test.java
**/*TestCase.java
because this will make them run in test fase and not in integration-test fase as you plan.
I have a configuration similar to yours and it works.
When I run mvn clean install, for the integration-test phase it does not use the failsafe plugin.
However if i explicilty call the plugin to run the integration test, it works (mvn failsafe:integration-test).
How can I make maven use the failsafe plugin when I run mvn clean install during the integration-test phase?
Quote from official documentation:
To use the Failsafe Plugin, you need to add the following configuration
to your pom.xml
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.11</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
Is this what you looking for?