I have webapp which provides web service. I want to perform JUnit tests with SoapUI to check if this service is working properly.
But to test web service application has to be deployed to my Tomcat 7 server.
I have no idea how to configure maven to build war, then deploy it to tomcat (ideally: to run separate tomcat instance for this) and then to run JUnit tests.
I will appreciate any help.
I am using maven 2.2.1
There are a number of schools of thought as to how to handle this type of integration test with Maven.
I should point out that when you are deploying an application to an application server, you are not in the realm of unit testing any more. Because the entire application is being deployed within a container, you are testing the integration of those two components.
Now there is nothing wrong with using JUnit for running integration tests (though there are some limitations that you may hit, for example unit tests should not care about the sequencing of individual tests - assuming you are writing them correctly - so JUnit enforces this by not guaranteeing any sequence of execution... prior to Java 1.7 the execution order was accidentally implied by the order of test methods within a class, but it was not part of the JUnit contract... Some people switch to other testing frameworks for their integration tests, e.g. TestNG, if they find the unit test focus of JUnit is getting in the way of their test development)
The key point to keep in mind is that the Maven lifecycle uses the test phase for the execution of unit tests.
When it comes to integration tests there are two (and a half) schools of thought as to the right way to handle the tests with Maven.
School 1 - Failsafe and integration-test/verify
This school of thought uses the phases after package to start up a container, run the integration tests, tear down the container, and finally check the test results and fail the build in the event of test failures.
NEVER EVER RUN mvn integration-test as that will not tear down the container correctly, any time you think you want to type mvn integration-test you actually want to type mvn verify (oh look, it's shorter and easier to type also... bonus)
So with this you do the following:
Bind tomcat7:run to the pre-integration-test phase with fork=true
Bind failsafe:integration-test to the integration-test phase
Bind tomcat7:shutdown to the post-integration-test phase
Bind failsafe:verify to the verify phase.
For extra brownie points you would use build-helper-maven-plugin:reserve-network-port bound to the validate phase to ensure that the test server is started on an unused network port and then either use resource filtering against the test resources to pass the port through to the tests or use a system property passed through systemPropertyVariables to make the port number available to the tests.
Advantages
Clean Maven build
If the tests fail, you cannot release the project
Can move the integration tests into a separate profile (by convention called run-its) if the tests are too slow to run every build.
Disadvantages
Hard to run the tests from an IDE. All the integration tests start/end in IT and while Maven knows to run tests starting/ending in Test with Surefire and run tests starting/ending in IT with Failsafe, your IDE probably doesn't. Additionally, your IDE is not going to start the container for you, so you have to do a lot of work by hand to actually run the tests manually.
Debugging the tests potentially requires attaching two debuggers, e.g. one to debug the application running in container and the other to debug the test cases.
mvnDebug -Dmaven.failsafe.debug=true verify
Couples your tests to the Maven build process.
School 2 - Separate module
This school of thought moves the integration tests into a separate module that depends on the war module and copies the war into the test resources using, e.g. dependency:copy-dependencies bound to the generate-test-resources phase coupled with a Tomcat7 dependency to test against.
The test cases themselves start up the Tomcat7 container using embedded mode
Advantages
Tests can run in IDE
Integration tests are separated from Unit tests, so asking the IDE to run all tests will not kick off the slower tests
Disadvantages
The war artifact is only rebuilt if you go past the package phase, consequently, you need to run at least mvn clean package periodically to refresh the code under test when using the IDE.
The failure of the integration tests does not break the build of the war module, so you can end up releasing a broken war artifact and then have the reactor build fail for the integration test module. Some people counteract this issue by having the integration test module within src/it and using Maven Invoker Plugin to run the tests... though that provides a poorer IDE integration, so I do not recommend that line.
Hard to get a consolidated test coverage report from Maven.
Have to code the container start/stop yourself from within your test cases.
School 2.5 - Failsafe with the test cases starting their own Tomcat7 server
This is a kind of hybrid of the two approaches.
You use Failsafe to execute the tests, but the tests themselves are responsible for starting and stopping the Tomcat7 container that you want to test in.
Advantages
Don't have to configure the server start/stop in Maven pom
IDE can safely run all tests (though the integration tests may be slower and you may want to not run them, but it's not like they will all fail unless there is a test failure)
Easier to debug the tests from your IDE (only one process to attach against, and the IDE usually makes it easy to debug tests by providing a special test runner)
Disadvantages
Have to code the container start/stop yourself from within your test cases
I hope the above helps you understand the options you have. There may be other tweaks but in general the above are considered the best practice(s) for integration testing with Maven at present.
#Stephen Connolly - your answer above was really good. I thought I'd kick in and show a full configuration for what you termed a School 1 response.
This configuration:
runs unit tests separately from integration tests. It uses the #Category annotation on the root classes that unit tests and integration tests extend.
before integration tests it starts the dependent application (loaded as a maven dependency at runtime) on the local machine, by finding an open port
after integration tests it tears down the dependent application
There's other stuff in there like how to set certain system properties on the dependent application only.
So far this configuration is working awesome..
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>reserve-network-port</id>
<goals>
<goal>reserve-network-port</goal>
</goals>
<phase>pre-integration-test</phase>
<configuration>
<portNames>
<portName>tomcat.maven.http.port</portName>
</portNames>
</configuration>
</execution>
<execution>
<id>get-local-ip</id>
<goals>
<goal>local-ip</goal>
</goals>
<configuration>
<!-- if not given, 'local.ip' name is used -->
<localIpProperty>local.ip</localIpProperty>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.2</version>
<configuration>
<!-- http port from reserve-network-port-plugin-->
<port>${tomcat.maven.http.port}</port>
<!-- application path always starts with /-->
<path>/</path>
<webapps>
<webapp>
<groupId>com.company.other.app</groupId>
<artifactId>web-rest</artifactId>
<version>1.0.1-SNAPSHOT</version>
<type>war</type>
<contextPath>/webapi-loopback</contextPath>
<asWebapp>true</asWebapp>
</webapp>
</webapps>
</configuration>
<executions>
<execution>
<id>start-server</id>
<configuration>
<fork>true</fork>
<skip>${skipTests}</skip>
<systemProperties>
<spring.profiles.active>test,h2</spring.profiles.active>
</systemProperties>
</configuration>
<phase>pre-integration-test</phase>
<goals>
<goal>run</goal>
</goals>
</execution>
<execution>
<id>stop-server</id>
<configuration>
<skip>${skipTests}</skip>
</configuration>
<phase>post-integration-test</phase>
<goals>
<goal>shutdown</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<configuration>
<excludedGroups>com.company.app.service.IntegrationTestRootClassAnnotatedWithAtCategory</excludedGroups>
</configuration>
<executions>
<execution>
<id>unit-test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<argLine>-Xmx1024m -XX:MaxPermSize=256m #{jacocoArgLine}</argLine>
<excludedGroups> com.company.app.service.IntegrationTestRootClassAnnotatedWithAtCategory </excludedGroups>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.18</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>start-integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<argLine>-Xmx1024m -XX:MaxPermSize=256m #{jacocoArgLine}</argLine>
<groups>com.company.app.IntegrationTestRootClassAnnotatedWithAtCategory</groups>
<includes>
<include>**/*.java</include>
</includes>
<systemPropertyVariables>
<program.service.url>
http://${local.ip}:${tomcat.maven.http.port}/webapi-loopback
</program.service.url>
</systemPropertyVariables>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
As Stephen Connolly explains there is no a direct way to configure this. I will explain how to solve this using failsafe plugin. In the maven lifecycle type of test can be tested. Unit testing one of them and another one is integrate testing. Unit testing can be run in the test stage of maven lifecycle. When you want to do integrate test it can be done in verify stage. If you want to know the difference between unit testing and integrate testing, this is a good one. By default unit test classes should be in ***/*Test.java, and **/*TestCase.java this format. The failsafe plugin will look for **/IT*.java, **/*IT.java, and **/*ITCase.java.
Here is an example.
Here I have one unit test class and one integrate test class. Now I will explain, how should be the looks like maven pom.xml. Build section of maven configuration should look like this.
<build>
<plugins>
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>2.3</version>
<configuration>
<webXml>src/main/webapp/WEB-INF/web.xml</webXml>
<warName>${name}</warName>
<outputDirectory>/home/jobs/wso2/wso2as-5.3.0/repository/deployment/server/webapps</outputDirectory>
<goal>
</goal>
</configuration>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
Unit tests are run before deploying the web app(war file). But integrate tests are run in verify stage. I hope your requirement is satisfied in this stage.
Related
is there a way to run integration tests independently?
I got a few selenium tests (reworked in java) in "integration" group, in pre-integration-test phase tomcat runs the webapp and in post-integration-test it shuts down. But the problem is I need the tests to be independent on each other. For example when first test runs, it creates a few posts here and there. I need to reset the webapp (clean everything that the first test did) before running the second test (on a default webapp). All of this needs to be automatic...
Here's my failsafe plugin config from pom.xml.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19</version>
<configuration>
<groups>integration</groups>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
And this is the part with the integration phases and tomcat init...
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.2</version>
<dependencies>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.4.190</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>tomcat-run</id>
<goals>
<goal>run</goal>
</goals>
<phase>pre-integration-test</phase>
<configuration>
<webapps>
<webapp>
<groupId>...webapp...</groupId>
<artifactId>...webapp...</artifactId>
<version>${webapp.version}</version>
<type>war</type>
<asWebapp>true</asWebapp>
<contextPath>/</contextPath>
</webapp>
</webapps>
<fork>true</fork>
<port>${tomcat.port}</port>
</configuration>
</execution>
<execution>
<id>tomcat-shutdown</id>
<goals>
<goal>shutdown</goal>
</goals>
<phase>post-integration-test</phase>
</execution>
</executions>
</plugin>
Unfortunately I do not know of any tools out there that simplifies this and a solution will be very specific to the technologies your using (e.g. Tomcat/Jetty/etc., Jersey/Spring/etc., H2/MySQL/etc., etc.) but here are two options:
Do setup/teardown of your web-app from your Java code using a Tomcat Container Java API (instead of using a Maven plugin, e.g. Cargo supports Tomcat and has a Java API (start here although I doubt this will be easy but probably worth it)). This will let you use Before/After methods to set-up and tear-down web-apps on the fly and give you the control you're looking for (you might reuse the same web-container and just reinstall the app, reinitialize the database, etc. or you might tear everything down and start over, your call). This may even allow you to run multiple instances of your web-app at once (assuming you can do this with app) and then run integration tests independently in parallel.
Use the maven plug-in to install multiple instances of your app accessible from different base urls and/or ports.
Technically you'd need to add multiple executions each of which would need to restart app server, deploy the app and run the tests. Which you won't be able to do by standard Maven means. Though you always can write your own plugin.
The issue with your approach though is that it's very complicated (as you already noticed) and very ineffective since you'd need to do multiple deployments. I'd advice to change the approach, here are some suggestions:
Randomize data to isolate tests. In most cases it's possible to prepare and randomize data for tests. E.g. if you create posts on behalf of some user, you can create that user (with random name) for that particular test and create a randomly generated post. This way 2 tests will be isolated from each other. It's not always possible (e.g. you test global configuration of the app), but in 99% of cases it is.
Don't write many System Tests that require real deployment (like those that use Selenium). In most cases it's enough to write Unit tests or Component Tests. E.g. initialize several Spring contexts (or whatever DI you use) and test services/controllers directly. A lot of REST or MVC frameworks have ability to invoke services/controllers directly without real deployments (e.g. Spring MVC has MockMvc). If you still want to deploy the app in tests consider Arquillian instead of Tomcat, but it's still going to be slow. The only case where you need real deployment is when you want to test UI - that requires a full blown app server and browsers.
In general to get testing optimal and isolated you need to build a Test Pyramid where you've got a plenty of unit tests, many component tests and a small number of system tests (which are hard to isolate and take a lot of efforts anyway). See in this article how you can do this in Java apps.
Sorry about this very basic questions about maven-failsafe-plugin but I am not very much familiar with maven.
Is it mandatory to specify maven-failsafe-plugin to run integration tests?
Why can't mvn verify execute integration tests just like mvn test executes unit tests?
Can integration tests be executed without this plugin?
mvn test executes the unit tests, because Maven has a default binding from test to surefire:test, meaning, if you execute the phase test, Maven will call the surefire plugin with the goal test. However, there is no default binding for the integration test or verify phase, so you have to provide it yourself by specifying the failsafe plugin.
Completely Agree with dunni's answer. Adding few more points.
It would be good practice to use maven-failsafe-plugin to run integration tests. As the Failsafe Plugin is designed to run integration tests while the Surefire Plugin is designed to run unit tests.
This has been correctly answered by dunni.
Addiing additional info, To use the Failsafe Plugin, you need to add the following configuration to your project pom.xml:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin-version}</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Though it is not a good practise, you could configure maven-surefire-plugin to also run the integration tests without the failsafe-plugin.
I have a project using jacoco version 0.7.1.201405082137 and maven 3.0.5. In the project I have some unit tests, and some tests created using arquillian.
To distinguish between the unit tests and integration ones, I created two junit categories: one called FastTest and another called SlowTest.
In the maven profile that I use to run all tests I have this plugins configured:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<groups>SlowTest,FastTest</groups>
<systemPropertyVariables>
<arquillian.launch>wildfly_8_x</arquillian.launch>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${version.jacoco}</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
When I leave both categories in I get only the coverage for the tests annotated with SlowTest. But all the tests run. If I run only the ones annotated with FastTest I get their correct coverage too.
How can I set up jacoco to get the correct coverage when running both kinds of tests?
Small tip:
<groups> tag takes full class name. So my question: is SlowTest.class interface placed in the default package? If not you should provide full path, something like: <groups>com.mycompany.project.SlowTest</groups>
And little advice:
Good practice is to distinguish unit and integration tests - and thus run them separately. Maven accomplishes this by two plugins: maven-surefire-plugin and maven-failsafe-plugin.
First is designed to run unit tests with mvn test. Second is designed to run your integration tests with mvn failsafe:integration-test. This answer may be useful to shed some light.
I have the following directory structure for my maven project:
|-src
|---itest
|-----java
|-------com
|---------corp
|-----------div
|-------------dept
|---------------prod
|-----------------config
|-------------------integration
|---------------------PersistenceConfig.java
|-----------------test
|-------------------LandingPageInfoTest.java
|-----resources
|-------db
|---------migration
|-----------V1__SeedLandingPageInfo.sql
|-------log4j.properties
|-------persistence.properties
(note: src/i test, not src/test)
I have an appropriate set of build-helper-maven-plugin plugin declarations that tell maven to add src/itest/java and src/itest/resources via the add-test-source and add-test-resource goals.
Everything works great when I run mvn test from the command line.
The problem: when I move my plugin declarations into a separate <profile> named integration my tests refuse to run when I execute mvn test -Pintegration and I'm not sure why...
The convention tends to be use the maven-failsafe-plugin (http://maven.apache.org/surefire/maven-failsafe-plugin/) with configuration to include your integration tests whereever they are, even alongside your unit tests, and they should then be excluded from the unit test phase surefire plugin. As a simple example:
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<!-- Include your integration test files/directories/suites here -->
</includes>
</configuration>
</plugin>
And then you can remove the profile you've specified and as mentioned, only unit tests will be executed in the test phase and your integration tests will be executed in the verify phase (included before install/deploy).
Following this convention will let you get rid of all the extra plugin configuration to move files around and point at your itest directory.
We have unit tests (mockito) and integration tests (in-memory database)
We'd like maven to not run the integration tests as part of 'mvn install'.
Basically I think this means reconfiguring the lifecyle so that integration-test
comes between install and deploy. Is this possible?
The reason for this would be that the integration tests are somewhat time consuming
and so we don't want them to run every time a developer does an install. But we would
like them to be run before the project can be released, for example.
Check the docs for the plugin you use for running integration tests (possibly Failsafe) - simply exclude the tests, or set the plugin execution to false.
Does integration-tests just execute a single plugin (like surefire)? If so, it is probably easier to just bind the plugin execution to a different phase:
<project>
...
<build>
<plugins>
<plugin>
...
<executions>
<execution>
<id>execution1</id>
<phase>install</phase>
<configuration>
...
</configuration>
<goals>
<goal>test</goal>
</goals>
</execution>