I read Maven Failsafe plugin is designed specifically to run integration tests. Currently I'm working on a multi-module project and integration tests are in its own separate module, written in TestNg and run using Surefire plugin. We don't have conflicts with unit tests since only integration tests are run in the test phase in that module. And to set up the environment before the tests, and clean it after tests are run, #BeforeSuite #AfterSuite TestNg annotations are used. So there's no need to make use of pre-integration-test phase, integration-test phase, post-integration-test phase utilized by Failsafe plugin.
Are there any more benefits I'm missing out on, by not using the Failsafe plugin?
Are there better ways to do my current requirement using Failsafe plugin?
Can I do my server startup, shut down, file unzipping etc. in the pre-integration-test, post-integration-test phases without writing a maven plugin?
If you already have your own test setup/teardown in your suites, which from the looks of it you do, there is not much you can gain from the FailSafe plugin.
The FailSafe plugin is useful in situations where the Setup of your System Under Test is costly or takes a long time such as starting up a Servlet or a distributed system. The way the FailSafe plugin comes handy in these situations is that you can set up this environment in the pre-integration-test phase. This plugin also doesn't stop the execution of the Maven build when a test fails, which allows you to clean up all of your artifacts during the post-integration-test phase, after which it checks the status of your tests and passes or fails the build accordingly during the verify phase.
Failsafe has one big feature vs Surefire: When a test fails, it does not immediately abort. Instead it lets the clean-up code run (which typically takes down the Jetty server).
Addressing your third question as it isn't really answered, imho.
Can I do my server startup, shut down, file unzipping etc. in the pre-integration-test, post-integration-test phases without writing a maven plugin?
Taken from this answer to "Maven Failsafe Plugin: how to use the pre- and post-integration-test phases"
It boils down to: pre-integration-test and post-integration-test do nothing per default. You can bind a plugin specific for your task to those phases. Finding a specific plugin depends on what you're trying to do.
Another important thing to point out is default naming conventions used by the maven-failsafe-plugin: It runs test-classes with names starting or ending with IT (as integration test class)
Related
I am adding basic regression testing to a Maven project that has no automated testing. My initial idea was to create a number of test classes, called IT<whatever>.java, to run in the integration-test phase. However, during packageing we do obfuscation and optimization and I want to be sure that the tests run against the final JAR (or at least the final classes).
The thing is, I can't tell from reading the docs what the actual difference is between the two kinds of test. The docs mention that integration-test is run after package, which sounds promising, but the tests are excluded from the JAR so it's unlikely they're running against the final artifact. Are they run against the packaged classes? Or is the only distinction between the two test types that they are run in different phases of the build lifecycle?
I use Maven to build a multi-component project. One of the components is a web app with HTML and JavaScript. Maven invokes NPM and Karma to run that component's unit and integration tests, written in QUnit. This yields two independent Cobertura coverage reports that I then combine into one with istanbul-combine.
As a final step, I'd like the Maven build to fail if that combined Cobertura report doesn't satisfy my coverage requirements. But how?
I've tried the Cobertura Maven Plugin, but it seems to be unfit to simply read an existing report and apply its check configuration. First, it needlessly repeats the build's tests ("Invokes the execution of the lifecycle phase test prior to executing itself."), then it finds that it cannot instrument the non-Java resources, then it simply stops, without any log output whatsoever.
Are there other plugins that fit? Could changing the report format help? (Doesn't have to be Cobertura.)
My Maven project has multiple surefire executions configured to make sure different test groups get rund with slight modifications to the classpath etc. (see the pom.xml). When running the build job, the tests are executed as expected.
The build job running the Maven Sonar plugin only runs the default test execution and skips the other test configurations which effectively leads to Sonar reporting 0% test coverage. How can I tweak the Maven Sonar plugin to not skip the additional test executions. The relevant section from the log is here, the full log here.
I'd be interested in why the additional executions are skipped as well.
The objective of SonarQube is not to run your unit tests, especially when the configuration is not standard. For you information, our long-term goal is to remove such a feature and only allow one mode: reuseReports.
Therefore you should run your unit tests first and reuse those generated reports while running the SonarQube analysis. See http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Unit+Tests+for+Java+Project#CodeCoveragebyUnitTestsforJavaProject-ReusingExistingReports.
I have a large build where tests sometimes prevent creation of artifacts because they stall or terminate the build due to make module timeout. I'd like my build server to:
build and publish artifacts
run unit / integration tests
At present unit and integration tests execute during each module's test and integration test phases in the typical maven way. This is great for developers but places risk of lack of build. The build server can easily setup a build/publish step by skipping the tests and I'd like to run a testing only test that skips process-resources through package phases just running the tests. However, I'm fairly certain that maven will traverse all the previous stages again.
I'd like to avoid lengthening an already quite long build by needlessly rebuilding and repackaging my artifacts.
Has anyone split a maven build like this so the desktop developers get the benefit of test integration and the build server / continuous integration system get's the benefit of pushing risky tests to the end of the process so we always produce artifacts?
Am I trying to break the build up in the wrong fashion and should consider changing a monolithic end-to-end build in a series of smaller component builds?
Thanks
Peter
Here's how you can do this:
You can run maven and skip all tests to install/deploy your artifacts
Then you can run it again but this time only run the tests
Like so:
mvn install -DskipTests
mvn surefire:test failsafe:integration-test
If you have to do this all on one command line line, I'm sure you can figure something out by having two profiles. One that skips tests and one that only runs tests.
You could put the tests into their own modules which depend on the modules they test then use profiles to include/exclude the test modules as desired.
I am running my Integration Tests using maven and the failsafe plugin. The tests themselves are executed using pax-exam (which uses pax-runner). Failsafe runs at the integration-test phase, and Cobertura at verify.
While I get reports for all my junit tests, I get nothing from the Integration Tests.
Is this even technically feasible, or would pax-runner need to support cobertura directly?
Any idea how I could solve this? I am also open to any other plugin that helps, but I do need to run integration tests on OSGi bundles..
Thanks!
Well, it's been a while. The closest thing I've found that could help out is instrumenting the builds and collecting cobertura.ser files after the pax-exam tests have run. It is far from a simple setup. I'll report as I go! relevant reference