We have projects where we use jacoco to aggregate code coverage over several modules. We also have some project that use the Fabric8 docker-maven-plugin to run black box test. But for the first time we would now like to run them both i the same project. And it kind of works but not in the same maven command.
We can run mvn test and then jacoco does it job perfecly.
we can run mvn install -Djacoco.skip=true and the black box tests start the necessary docker-containers and run the tests on them.
But running mvn install and thus saying that both blackbox-tests and jacoco codecoverage should run will fail saying that jacoco doesn't find dependencies in a central repository (where they are not supposed to be anyway).
So, my question, what could docker-maven-plugin do to interfere with jacoco? It seems like docker-maven-plugin deletes stuff that jacoco expect to be there.
My configuration for jacoco is using the invoker-plugin (as many of the guides for multi module exampels for jacoco does) with localRepositoryPath, could it be colliding? Or do docker-maven-plugin clean the workspace? Se plugin-definition below.
<plugin>
<!-- To run with different Maven versions use -Dinvoker.mavenHome -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-invoker-plugin</artifactId>
<version>1.5</version>
<configuration>
<skipInvocation>${skipTests}</skipInvocation>
<projectsDirectory>it</projectsDirectory>
<cloneProjectsTo>${project.build.directory}/it</cloneProjectsTo>
<pomIncludes>
<pomInclude>*/pom.xml</pomInclude>
</pomIncludes>
<postBuildHookScript>verify</postBuildHookScript>
<localRepositoryPath>${project.build.directory}/local-repo</localRepositoryPath>
<goals>
<goal>clean</goal>
<goal>install</goal>
</goals>
<settingsFile>it/settings.xml</settingsFile>
<extraArtifacts>
<extraArtifact>org.jacoco:org.jacoco.agent:0.8.3:jar:runtime</extraArtifact>
</extraArtifacts>
</configuration>
<executions>
<execution>
<goals>
<goal>install</goal>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
Sorry about this very basic questions about maven-failsafe-plugin but I am not very much familiar with maven.
Is it mandatory to specify maven-failsafe-plugin to run integration tests?
Why can't mvn verify execute integration tests just like mvn test executes unit tests?
Can integration tests be executed without this plugin?
mvn test executes the unit tests, because Maven has a default binding from test to surefire:test, meaning, if you execute the phase test, Maven will call the surefire plugin with the goal test. However, there is no default binding for the integration test or verify phase, so you have to provide it yourself by specifying the failsafe plugin.
Completely Agree with dunni's answer. Adding few more points.
It would be good practice to use maven-failsafe-plugin to run integration tests. As the Failsafe Plugin is designed to run integration tests while the Surefire Plugin is designed to run unit tests.
This has been correctly answered by dunni.
Addiing additional info, To use the Failsafe Plugin, you need to add the following configuration to your project pom.xml:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin-version}</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Though it is not a good practise, you could configure maven-surefire-plugin to also run the integration tests without the failsafe-plugin.
I have the following directory structure for my maven project:
|-src
|---itest
|-----java
|-------com
|---------corp
|-----------div
|-------------dept
|---------------prod
|-----------------config
|-------------------integration
|---------------------PersistenceConfig.java
|-----------------test
|-------------------LandingPageInfoTest.java
|-----resources
|-------db
|---------migration
|-----------V1__SeedLandingPageInfo.sql
|-------log4j.properties
|-------persistence.properties
(note: src/i test, not src/test)
I have an appropriate set of build-helper-maven-plugin plugin declarations that tell maven to add src/itest/java and src/itest/resources via the add-test-source and add-test-resource goals.
Everything works great when I run mvn test from the command line.
The problem: when I move my plugin declarations into a separate <profile> named integration my tests refuse to run when I execute mvn test -Pintegration and I'm not sure why...
The convention tends to be use the maven-failsafe-plugin (http://maven.apache.org/surefire/maven-failsafe-plugin/) with configuration to include your integration tests whereever they are, even alongside your unit tests, and they should then be excluded from the unit test phase surefire plugin. As a simple example:
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<!-- Include your integration test files/directories/suites here -->
</includes>
</configuration>
</plugin>
And then you can remove the profile you've specified and as mentioned, only unit tests will be executed in the test phase and your integration tests will be executed in the verify phase (included before install/deploy).
Following this convention will let you get rid of all the extra plugin configuration to move files around and point at your itest directory.
I have a multi-module maven project with both integration and unit tests in the same folder (src/test/java). Integration tests are marked with #Category(IntegrationTest.class). I want to end up with the following setup:
If I run mvn install, I want all tests to compile, but I do not want to execute any.
If I run mvn test, I want all tests to compile, but execute only unit tests.
If I run mvn integration-test, I want to compile and execute all tests.
The important point is, I want this configured in the pom.xml without any extra commandline arguments.
Currently I came up with the following setup in my parent pom.xml, where the only problem is #1, where all tests are executed:
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>${project.java.version}</source>
<target>${project.java.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.14.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.14.1</version>
</dependency>
</dependencies>
<configuration>
<includes>
<include>**/*.class</include>
</includes>
<excludedGroups>cz.cuni.xrg.intlib.commons.IntegrationTest</excludedGroups>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.14.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.14.1</version>
</dependency>
</dependencies>
<configuration>
<groups>cz.cuni.xrg.intlib.commons.IntegrationTest</groups>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<includes>
<include>**/*.class</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
All child modules have the following plugin configuration in their pom.xml, which I believe should inherit from the parent pom:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
</plugins>
</build>
I tried using <skipTests>true</skipTests>, but it disables test execution for all goals, which is not what I want (violates #2 and #3). It is also quite weird, that mvn test honors the skipTests=true option...why would I want to run it in the first place??
After hours of googling and trying different combinations, I am hesitant whether it is even possible to not run tests in mvn install, while at the same time run them in mvn test. I hope someone proves this wrong. ;)
I am also willing to accept a solution, where mvn install would execute only unit tests, but I don't think it makes much difference.
It sounds like you didn't understand the concept of the build life-cycle in Maven. If you run mvn install all life-cycle phases (including the install phase itself) run before the install phase. This means running the following phases:
validate
initialize
generate-sources
process-sources
generate-resources
process-resources
compile
process-classes
generate-test-sources
process-test-sources
generate-test-resources
process-test-resources
test-compile
process-test-classes
test
prepare-package
package
pre-integration-test
integration-test
post-integration-test
verify
install
which means in other words the test as well as integration-test life-cycle phases are included. So without any supplemental information it's not possible to change the behaviour as you wish it.
It could be achieved by using a profile in Maven:
<project>
[...]
<profiles>
<profile>
<id>no-unit-tests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
[...]
</project>
So your first requirement:
If I run mvn install, I want all tests to compile, but I do not want to execute any.
can be achieved by using the following:
mvn -Pno-unit-test test
If I run mvn test, I want all tests to compile, but execute only unit tests.
This can simply achieved by using the plain call:
mvn test
cause the integration tests phase is not run (see the build life cycle).
If I run mvn integration-test, I want to compile and execute all tests.
This means running the default which includes running the test phase which will run the unit tests (maven-surefire-plugin) and furthermore running the integration test which are handled by the maven-failsafe-plugin. But you should be aware that if you like to call the integration tests you should using the following command:
mvn verify
instead, cause you missed the post-integration-test phase in your previous call.
Apart from the above you should follow the naming conventions for unit and integration tests where unit tests should be named like the following:
<includes>
<include>**/*Test*.java</include>
<include>**/*Test.java</include>
<include>**/*TestCase.java</include>
</includes>
and integration tests should be named like the following:
<includes>
<include>**/IT*.java</include>
<include>**/*IT.java</include>
<include>**/*ITCase.java</include>
</includes>
I hope you have configured the maven-failsafe-plugin like the following which is needed to bound the maven-failsafe-plugin to the correct life-cycle-phases:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.15</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
as you correctly did, but you should be aware that the include tags work on the source code (.java) and not on the compiled names (.class). I wouldn't use the Category annotation, just simply using the naming conventions makes the pom simpler and shorter.
According to the Failsafe Plugin documentation
mvn install -DskipITs
is what you want.
What OP stated in his question:
If I run mvn install, I want all tests to compile, but I do not want
to execute any.
If I run mvn test, I want all tests to compile, but execute only unit tests.
If I run mvn integration-test, I want to compile and execute all tests.
is perfectly valid and extremely easy to achieve.
EDIT: except first condition, which acts againts the maven nature. The best way here would be simply do mvn install -DskipTests
All you need is following snippet in pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.17</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
and to stick to the maven naming conventions for unit and integration tests (as #khmarbaise already stated). So generally name you integration tests with IT suffix (for example MyIntegrationTestIT.java) and let maven-failsafe do its job.
In that way, you do not even need JUnit categories (although sometimes they can be quite useful).
That's it :)
mvn test executes only unit tests
mvn integration-test executes all tests
mvn failsafe:integration-test runs only integration tests
mvn clean verify when you want to be sure, that whole project just works
Some personal advices
Keeping integration tests separately from unit tests lets you easily run within your IDE all tests in some package. Usually additional directory called test-integration (or integrationtest) is used for this purpose.
This is also easy to achieve with maven:
<plugin>
<!-- adding second test source directory (just for integration tests) -->
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>add-integration-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/test-integration/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
And then move your integration tests to that directory. It should look like:
src
main
test
test-integration
Integration tests usually needs more memory:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
...
<configuration>
<argLine>-Xmx512m -XX:MaxPermSize=256m</argLine>
</configuration>
</plugin>
This post explains how to skip integration tests, no matter what plugin you are using for these tests.
Basically, what you do is define a profile and put all your integration-tests related xml code inside that profile. Than you activate it when a property -DskipIntegrationTests is missing.
You can do the same for unit tests: write a profile and activate it when -DskipUnitTests is missing.
Then, you could do:
mvn install -DskipIntegrationTests -DskipUnitTests # (runs install without any tests)
mvn test # (runs unit tests)
mvn post-integration-test # (runs all tests)
The maven-failsafe-plugin docs has a section titled "Skipping by Default."
Sadly, the steps that page describes don't work as written. However, a slight change to those steps will make it work:
In the properties section of pom.xml, add this:
<skipITs>true</skipITs>
Then add the skipTests property to the plugin section of maven-failsafe-plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<skipTests>${skipITs}</skipTests>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
So now, an mvn install by default will execute unit tests, but not integration tests.
But an mvn install -DskipITs=false will execute both unit tests and integration tests.
Footnote: Bad documentation played a big part on why Maven was so disliked for such a long time.
mvn test-compile does exactly what you are looking for. You can simply replace mvn install with mvn test-compile and you are done. No need to customise the pom file or anything. The below linked question is similar around #1:
Maven - How to compile tests without running them ?
mvn test-compile should be accepted as the best answer as Maven supports exactly what you want to do natively and without any magic. You would end up with this:
If I run mvn test-compile, I want all tests to compile, but I do not want to execute any.
If I run mvn test, I want all tests to compile, but execute only unit tests.
If I run mvn integration-test, I want to compile and execute all tests.
Don't specify the execution step(s) in the configuration of the failsafe plugin. E.g.
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M3</version>
</plugin>
</plugins>
Now, you specifically need to call mvn failsafe:integration-test to run the integration tests; they will be skipped in other mvn targets.
I'm having some issues running my unit tests when my pom is set to packaging type "pom". At first, it was saying no goals needed for this project, so I added the maven-surefire-plugin to my pom.xml to bind the test phase to the maven-surefire-plugin test goal.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.9</version>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
Now the surefire plugin is getting executed, but it says there are no tests to run. If I change the packaging type to jar and run mvn test then it picks up my tests files.
When I run mvn test -X it says "testSourceDirectory = C:\dev\dsl\src\test\java", which is the correct location. Is the test location different for the packaging type "pom" than for "jar"? I tried adding
<configuration>
<testSourceDirectory>src/test/java</testSourceDirectory>
</configuration>
to the surefire plugin, but it didn't help at all.
As commented by Dave, if you are using pom packaging, it only executes the following lifecycle goals. Refer to this related maven documentation.
package
install
deploy
If you need it to run any other goal, you would need to explicitly specify it. For instance,
mvn clean compiler:testCompile surefire:test