I want to run multiple test suites from my project with in a single execution of maven. Somehow multiple run is not successful.
If I add two executions with the same test goal as below, each test suite is running as individually.
<executions>
<execution>
<goals>
<goal>test</goal>
</goals>
<configuration>
<projectFile>soapui-project.xml</projectFile>
<testSuite>SOAPTests1</testSuite>
</configuration>
</execution>
<execution>
<goals>
<goal>test</goal>
</goals>
<configuration>
<projectFile>soapui-project.xml</projectFile>
<testSuite>SOAPTests2</testSuite>
</configuration>
</execution>
</executions>
But I would like to run in a single execution or in a better way than the above as I have lot of different test suites to run and if I all all my test suites as above, that will just make my file look long.
I want my pom.xml look something like this:
<execution>
<goals>
<goal>test</goal>
</goals>
<configuration>
<projectFile>LocalAdapa-soapui-project.xml</projectFile>
<testSuite>SOAPTests1</testSuite>
<testSuite>SOAPTests2</testSuite>
<testSuite>SOAPTests3</testSuite>
</configuration>
</execution>
You can find some examples here multiple Test Suites or Test Cases when executing tests from Maven
Related
I have written a maven plugin to create documentation from latex.
Now I added some verification and to that end
added another execution, the one with id validate-converters like so:
<executions>
<execution>
<id>process-latex-sources</id>
<goals>
<goal>cfg</goal>
</goals>
</execution>
<execution>
<id>clear-latex-sources</id>
<goals>
<goal>clr</goal>
</goals>
</execution>
<execution>
<id>validate-converters</id>
<goals>
<goal>vrs</goal>
</goals>
<configuration>
<versionsWarnOnly>true</versionsWarnOnly>
</configuration>
</execution>
</executions>
Now the strange thing is, that for mvn site verification is run 3(!!!) times and I cannot figure out why.
I am willing to give you all code or anything to find out...
Is there a way to enforce the post-integration phase to always run after the integration phase? By always I mean in the advent of test failures during integration phase.
I am running an Angular / Springboot application. I use protractor to run e2e tests that test the whole Angular + Springboot chain. I managed to integrate this in my Maven build so that I can:
setup the backend Springboot server
setup a DB with initial data
run protractor during the integration phase
with the following plugins:
spring-boot-maven-plugin which starts and stops a test server for integration testing:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
...
</configuration>
<executions>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>post-integration-test</id>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
and frontend-maven-plugin which runs my protractor tests during the integration phase:
<plugin>
<groupId>com.github.eirslett</groupId>
<artifactId>frontend-maven-plugin</artifactId>
<configuration>
...
</configuration>
<executions>
<execution>
<id>install node and npm</id>
<goals>
<goal>install-node-and-npm</goal>
</goals>
<phase>generate-resources</phase>
</execution>
<execution>
<id>npm install</id>
<goals>
<goal>npm</goal>
</goals>
<phase>generate-resources</phase>
<configuration>
<arguments>install</arguments>
</configuration>
</execution>
<execution>
<id>npm run build</id>
<goals>
<goal>npm</goal>
</goals>
<phase>generate-resources</phase>
<configuration>
<arguments>run build</arguments>
</configuration>
</execution>
<execution>
<id>npm run integration tests</id>
<goals>
<goal>npm</goal>
</goals>
<phase>integration-test</phase>
<configuration>
<arguments>run e2e</arguments>
<testFailureIgnore>true</testFailureIgnore> // this should probably be deleted
</configuration>
</execution>
</executions>
</plugin>
I added testFailureIgnore = true to the frontend-maven-plugin because if any protractor test fails, it will stop my maven build before it gets to execute the post-integration phase. This causes the test server to keep running with that port. Any subsequent runs will fail since the port is already in use until that server is killed (manually). The testFailureIgnore property allows failed tests to be ignored by the build, effectively letting me continue with the post-integration phase.
The obvious downside is that my build will print SUCCESS even when tests have failed. I am looking for behavior similar to the failsafe plugin where failed tests will fail my build, but will still execute the post-integration phase first to cleanup properly.
I can't seem to find a proper solution for this but surely I can't be the first to encounter this problem. What solutions/alternatives are available for this? I imagine using the exec-maven-plugin instead of the frontend-maven-plugin will cause the same issue.
I didn't manage to find a decent solution to this anywhere so I decided to try and create my own. I extended the frontend-maven-plugin with a parameter that logs integration test failures during the integration-test phase, but only fails the build during the verify phase. This allows the post-integration-test phase to finish.
My solution is available from my repository (version 1.9.1-failsafe). This implementation requires a configuration parameter integrationTestFailureAfterPostIntegration to be added. Unfortunately I did not figure out how to make a Mojo execution trigger another Mojo execution at a later phase without user intervention. Because of this the user needs to have an execution that trigger during the verify phase, even if it doesn't do anything useful functionally (ie. npm -version).
My working example:
<execution>
<id>npm run integration tests</id>
<goals>
<goal>npm</goal>
</goals>
<phase>integration-test</phase>
<configuration>
<arguments>run e2e</arguments>
<integrationTestFailureAfterPostIntegration>true</integrationTestFailureAfterPostIntegration>
</configuration>
</execution>
<execution>
<id>fail any integration tests</id>
<goals>
<goal>npm</goal>
</goals>
<phase>verify</phase>
<configuration>
<arguments>-version</arguments>
</configuration>
</execution>
If any IT tests fail, they will be logged during integration-test phase and fail the build at verify. If all IT tests pass, the build will be successful.
I have an open pull request at the frontend-maven-plugin which might get added to the 1.9.2 version. I will still attempt to improve upon the change by removing the need for the verify execution phase to be added manually. Suggestions or improvements on the pull request are welcome!
UPDATE: I already went ahead and released my own version in case the pull request doesn't come through:
<dependency>
<groupId>io.github.alexandertang</groupId>
<artifactId>frontend-maven-plugin</artifactId>
<version>1.9.1-failsafe</version>
</dependency>
In this version I added a verify mojo which simplifies the second execution to:
<execution>
<id>fail any integration tests</id>
<goals>
<goal>verify</goal>
</goals>
<phase>verify</phase> <!--default phase is verify, so this is optional-->
</execution>
I resolved this puting this instructions on package.json
"scripts": {
...
"e2e": "ng e2e && echo Success > e2e/result.txt || echo Error > e2e/result.txt"
}
This will supress the exit code in error situation, and will record a file called result.txt whith Success or Error in your content.
Then, i add the maven-verifier-plugin on maven to verify the content of the file result.txt.
I have the following configuration for the maven failsafe plugin to run my integration tests (based on the documentation at Failsafe Usage Documentation:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
When I try to run an individual test using:
mvn -Dit.test=MyLovelyTest failsafe:integration-test
It does not run my test. It dies with the message: No tests were executed
If I remove the execution definition for verify, then it executes the test as expected. Since I copied the plugin def from the official usage documentation, I'm wondering if there's a bug in the plugin, or is it something I'm doing wrong?
TL;DR: Using maven, I want to run a plugin goal at the beginning of the test phase, before tests actually run. What would be a clean way to do it?
I want to print a message just before the tests actually run. Hence I want to use the echo goal of the echo plugin at the beginning of the test phase (to tell the user that if every tests fail, he'd better have a look at the README since there's a test environment he should set up first)
Attempt n°1
A simple approach could be to run this plugin in the previous phase, process-test-classes.
It works, but it doesn't seem semantically correct to bind this task to this phase...
Attempt n°2
According to Maven documentation, When multiple executions are given that match a particular phase, they are executed in the order specified in the POM, with inherited executions running first., so I tried to set explicitly the surefire plugin:
...
<plugin>
<groupId>com.soebes.maven.plugins</groupId>
<artifactId>maven-echo-plugin</artifactId>
<version>0.1</version>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>echo</goal>
</goals>
</execution>
</executions>
<configuration>
<echos>
<echo>*** If most tests fail, make sure you've installed the fake wiki. See README for more info ***</echo>
</echos>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
...
But tests run before my message is printed.
So, to put it in a nutshell: is there a way to reach my goal, or should I stick to the "process-test-classes solution" even though it seems a bit "hacky"?
Thanks!
As #khmarbaise said, your solution is still hacky, because whole test looks like Integration Test and should be processed by Failsafe Plugin. Failsafe has nice phase pre-integration-test for testing fake wiki etc :)
Based on Guide to Configuring Default Mojo Executions this works for me:
<plugin>
<groupId>com.soebes.maven.plugins</groupId>
<artifactId>maven-echo-plugin</artifactId>
<version>0.1</version>
<executions>
<execution>
<id>1-test</id>
<phase>test</phase>
<goals>
<goal>echo</goal>
</goals>
</execution>
</executions>
<configuration>
<echos>
<echo>*** If most tests fail, make sure you've installed the fake wiki. See README for more info ***</echo>
</echos>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<id>default-test</id>
<configuration>
<skip>true</skip>
</configuration>
</execution>
<execution>
<id>2-test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
This is very odd for me ;)
I have two plugins with executions bound to generate-sources, one listed first in the list of about 6 plugins and the other listed last. However, the one listed last (which depends on the one listed first) always executes first.
How can I execute several maven plugins within a single phase and set their respective execution order?
I have the findbugs plugin working fine in my maven setup. I've setup findbugs to execute during the compile phase. I noticed however that it runs during the test phase as well because the test phase also calls compile. Because I have an automated build pipeline that runs all my targets, I don't need findbugs to run during the test phase. I've tried to exclude findbugs from the test phase with the following but no luck yet.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>2.4.0</version>
<inherited>true</inherited>
<configuration>
<failOnError>${findbugs.failOnError}</failOnError>
<skip>${findbugs.skip}</skip>
<trace>${findbugs.trace}</trace>
</configuration>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
<execution>
<id>findbugs-test-compile</id>
<phase>test</phase>
<goals>
<goal>check</goal>
</goals>
<configuration>
<skip>true</skip>
</configuration>
</execution>
</executions>
</plugin>
It will not be called based on the running through the life-cylcle via compile it simply is running cause you configured to have two executions one in test and one in compile phase. Findbugs should usually run in the reporting area(site).
Just make a single execution:
<executions>
<execution>
<id>findbugs-test-compile</id>
<phase>test</phase>
<goals>
<goal>check</goal>
</goals>
<configuration>
<skip>true</skip>
</configuration>
</execution>
</executions>
The one you like to have. But i recommend to read the documentation cause it should run in reporting area (via site) only.
UPDATE:
If you like to run findbugs only during the site generation than just remove it from the usual build area and put into the reporting area instead.