When I run mvn test it executes unit tests only but when I run mvn integration-test it executes both unit test and integration test even after configuring the maven-failsafe-plugin and excluding the *Test.java file. Not sure what I am missing here. Also worth mentioning that I have not put in maven-surefire-plugin in my pom.xml. Not sure if that is creating this problem. Please guide.
pom.xml
<!-- Integration tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18</version>
<configuration>
<includes>
<include>**/*IT.java</include>
</includes>
<excludes>
<exclude>**/*Test.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
Unit Test Class File:
com.study.jenkins.ut.MyUnitTest.java
Integration Test Class File:
com.study.jenkins.it.PageIT.java
The maven-surefire-plugin is part of the lifecycle, which is always bound to the test-phase for Java projects. Calling integration-test means that all lifecycle-phases up to the integration-test phase are executed. So the MyUnitTest will always be executed (which is a good thing).
Your includes/excludes have no effect, these are already the defaults for the maven-failsafe-plugin, see http://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#includes
Related
mvn test
I am looking to initialize certain resources before all test cases executed using maven test and also to destroy them after all test cases are executed.
I have looked into jUnit #BeforeClass, #AfterClass, #Before and #After, but none of them are helpful.
I have tried to use maven life-cycle phases i.e. pre-integration-test as below, but even in this case the expected Test case (TestPostgresqlEmbedded) doesn't get executed first.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.plugin.version}</version>
<executions>
<execution>
<id>test-init</id>
<configuration>
<runOrder>alphabetical</runOrder>
<includes>
<include>**/TestPostgresqlEmbedded.java</include>
</includes>
</configuration>
<phase>pre-integration-test</phase>
</execution>
<execution>
<id>test-all</id>
<configuration>
<runOrder>alphabetical</runOrder>
</configuration>
</execution>
</executions>
</plugin>
How can I achieve this ?
Use the integration-test phase for tests with databases. Then you have pre-integration-test to set up your database resources, and post-integration-test to destroy them.
i'm trying to put some bdd into my eclipse plugin project, but can't figure out how to run my integration tests during maven build fase. To write my tests i'm using SWTBot framework.
I already did the feature generation fase, and setup my tests. How do i setup my pom to run my integration tests?
I use below configuration and run mvn clean verify. If you don't want to run tests in parallel, remove parallel, perCoreThreadCount and threadCountClasses tags.
Make sure to update the regular expression to match your test naming convention <include>**/Run*.java</include>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<id>acceptance-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<outputEncoding>UTF-8</outputEncoding>
<parallel>classes</parallel>
<perCoreThreadCount>true</perCoreThreadCount>
<threadCountClasses>10</threadCountClasses>
<argLine>-Xmx1024m</argLine>
<argLine>-XX:MaxPermSize=256m</argLine>
<includes>
<include>**/Run*.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
I have written a multiple JUnit test suite for running multiple test cases.(multiple JUnit test like AllTest1.java/AllTest2.java is requirement of my web project)
Now, I want to run my multiple test suite classes (AllTest1.java/AllTest2.java) at a time means on same build time.. for this I used maven-surefire-plugin but not able to execute both same time I have used many logic to execute but not successed:(..Is there any way to execute both test-suite parallely .
Any help will be appreciated.
This is my current maven-surefire-plugin configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.8.1/version>
<configuration>
<includes>
<include>**/AllTests1.java</include>
</includes>
<excludes>
<exclude>**/AllTests2.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
This looks indeed like integration tests and unit tests or a combination of both. For such purposes i would suggest to use the maven-failsafe-plugin to run the integration tests which should follow the naming schema:
<includes>
<include>**/IT*.java</include>
<include>**/*IT.java</include>
<include>**/*ITCase.java</include>
</includes>
Furthermore the usual unit tests will be executed by the maven-surefire-plugin which should follow the following naming schema:
<includes>
<include>**/*Test*.java</include>
<include>**/*Test.java</include>
<include>**/*TestCase.java</include>
</includes>
Furthermore you should avoid using test suites, cause based on the naming schema it can be distinguished if it's a unit- or integration tests.
After you changed to the above schema you can run the unit tests by
mvn test
running the integration tests by using:
mvn verify
I am trying to run only selenium tests using mvn failsafe plugin. I created a separate profile to run only the selenium tests but mvn is not able to find them. my project structure looks like
moduleA
scr/main/...
src/test/integration/java/...
scr/test/unit/java/...
moduleB
scr/main/...
src/test/integration/java/...
scr/test/unit/java/...
moduleC (only for selenium tests)
scr/main/java/com/selenium/A.java
src/test/java/...
Since I have new directories for the unit and integration tests in moduleA and B. I have defined the following in pom.xml (listingB) to let Maven know about the additional test directories. So far so good but when I add selenium tests in moduleC and I want to run only selenium its not running selenium tests. I created a new profile (listingA) to run selenium tests. Any help is appreciated.
listingA:
<profiles>
<profile>
<id>selenium</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>verify</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/selenium/*.java</include>
</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
listinB:
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>${basedir}/src/test/unit/java</source>
<source>${basedir}/src/test/integration/java</source>
<source>${basedir}/src/test/common/java</source>
</sources>
</configuration>
</execution>
<execution>
<id>add-test-resource</id>
<phase>generate-test-resources</phase>
<goals>
<goal>add-test-resource</goal>
</goals>
<configuration>
<resources>
<resource>
<directory>${basedir}/src/test/integration/resources</directory>
<directory>${basedir}/src/test/unit/resources</directory>
<directory>${basedir}/src/test/common/resources</directory>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
The first thing which came into my mind is why you are separating the unit- and integration tests by using a different folder, cause in Maven the separation between unit- and integration tests is done by naming convention.
The unit tests have to be named like
**/Test*.java
**/*Test.java
**/*TestCase.java
The integration test have to be named like:
**/IT*.java
**/*IT.java
**/*ITCase.java
This means in other words you can put your integration tests and your unit tests into the same folder which is src/test/java without any problem. The execution of the integration tests is not influenced by this.
If you would call
mvn clean package
only the unit tests will be run. If you need to run the integration tests as well you can simply use:
mvn verify
If you wan't to skip the unit tests you can use the following:
mvn -DskipTests=true verify
For the separate selenium module which you have created the best solutin is to put the integration tests into the usual folder src/test/java with the appropriate naming convention.
The problem you might have is if your integration tests needed some special resources you might move those integration tests into the separate module with integration tests only.
If your Selenium tests are in the src/test/java folder, delete the <includes> parameter from the configuration of maven-failsafe-plugin in listingA
Name your Selenium test classes like:
**/IT*.java
**/*IT.java
**/*ITCase.java
(as khmarbaise has suggested)
Failsafe runs integration tests using Surefire, therefore don't name your Selenium test classes like:
**/Test*.java
**/*Test.java
**/*TestCase.java
because this will make them run in test fase and not in integration-test fase as you plan.
I have a configuration similar to yours and it works.
I want to use maven-failsafe-plugin to run some integration tests. If any test fails, I want Maven to fail the build and not BUILD SUCCESS.
Tests run: 103, Failures: 1, Errors: 0, Skipped: 26
[INFO] BUILD SUCCESS*
how can I configure it, that build not success is?
My failsafe plugin is configured as:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${failsafe.version}</version>
<configuration>
<systemProperties>
<CI_INTEGRATION_OVERRIDE_PATH>${basedir}/..</CI_INTEGRATION_OVERRIDE_PATH>
</systemProperties>
<includes>
<include>**/integration/**/*.java</include>
</includes>
<excludes>
<exclude>**/integration/**/*TestSuite.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
As Andrew pointed out, the correct solution is to use failsafe as intended. The integration-test goal is specifically designed not to fail the build. If you want to fail the build, call mvn verify or mvn failsafe:verify
For the verify goal to work, it needs to read the results of the integration test which by default are written to ${project.build.directory}/failsafe-reports/failsafe-summary.xml so make sure that is getting generated.
Also, you have to make sure you bind your maven-failsafe-plugin configuration to both the integration-test goal and the verify goal in the executions part.
Failure to add either of those will lead to maven succeeding the build instead of failing it when integration tests fail.
Since you are running mvn clean install both the integration-test and verify phases should be executing. According to the failsafe plugin docs the failsafe:integration-test and failsafe:verify goals are bound to those phases, so I don't believe the extra call to failsafe:integration-test is required.
That said however, I'm not sure I trust the failsafe plugin documentation. I answered a similar question for someone earlier this year. It turned out he had to explicitly bind each goal to the correct phase and then failsafe worked as expected. Might be worth a shot.
I had a similar issue with Jenkins builds.
Running the tests locally resulted in failed tests and thus failed build
Running the tests on Jenkins resulted in failed tests but "BUILD SUCCESS"
Solution: in job configuration set MAVEN_OPTS to -Dmaven.test.failure.ignore=false.
Jenkins by default seems to ignore failed integration tests.
See also https://stackoverflow.com/a/28684048/4412885
solution.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
<executions>
<execution>
<id>unit-test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<excludes>
<exclude>**/*IntegrationTest.java</exclude>
</excludes>
</configuration>
</execution>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<enableAssertions>false</enableAssertions>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
<systemPropertyVariables>
<integration>${integration}</integration>
</systemPropertyVariables>
</configuration>
</execution>
</executions>
</plugin>