Currently our pipeline does and this creates a war file and puts in into the server
mvn clean test package
We are now trying to add surefire reports for those tests, everything works well when all tests pass
mvn clean test site package
But if some test fails then site does not get called and we have no report.
Although I know that site has tests inside as well so we tried but even if there was failing tests the site command did not make it to be failing and build succeeded.
mvn clean site package
What is the best approach to get tests and surefire report.
With How to build a jar using maven, ignoring test results?
and https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#testFailureIgnore:
<testFailureIgnore>(element) boolean(type) -((ever) since)
Set this to "true" to ignore a failure during testing. Its use is NOT RECOMMENDED, but quite convenient on occasion.
Default value is: false.
User property is: maven.test.failure.ignore.
Using:
mvn -Dmaven.test.failure.ignore=true clean package site
Will:
clean
package (including tests + test results, unless skipped)
not fail the build, when tests fail
and generate the project report (site)
Related
I'm bit confused with mvn verify phase. I've created a Spring Boot project (a simple project, without any explicit configurations added). I've created a few JUnit unit tests which are run with both the mvn verify and mvn test commands.
There isn't any difference observed in the mvn verify and mvn test command output.
What does mvn verify do different than mvn test?
Also some posts on Stack Overflow mentions that mvn verify runs the integration tests. If this is the case then I have few questions.
How does Maven identify a specific test as a unit test or integration test?
If mvn verify is supposed to run only the integration tests, then why are unit tests executed with it?
First of all, when you run a Maven goal, it will run any previous goal. The order of basic phases is:
Validate
Compile
Test
Package
Verify
Install
Deploy
If you run Test, Maven will execute validate, compile and test. Based on this, the first point is that verify includes test.
Based on official documentation:
TEST - test the compiled source code using a suitable unit testing framework. These tests should not require the code be packaged or deployed
VERIFY - run any checks on results of integration tests to ensure quality criteria are met
To run unit tests, the Surefire plugin is recommended. And Failsafe for integration tests.
The verify command executes each default lifecycle phase in order (validate, compile, package, etc.), before executing verify. In most cases the effect is the same as package. However, in case there are integration tests, these will be executed as well. And during the verify phase some additional checks can be done, e.g. if your code is written according to the predefined checkstyle rules.
Conclusion: if you want to run your integration tests and check it, use verify. If you only want to run unit tests, use test.
My personal advice: if in doubt, use verify.
How does Maven identify a specific test as a unit test or integration test?
Integrations Test always takes a name like IT.java or IT.java or *ITCase.java
mvn clean install in my case is set up to run unit tests and integration tests. When a particular integration test is skipped for some reason, the build is still successful.
I am trying to understand if there is a way to make this fail when a test is skipped or if there is another work around for this?
When generating a skeleton Maven plugin from archetype, the new project includes a Maven project under the src/it directory. It is an integration it (hinted at by the it dir name) and fresh out-of-the-box it passes when run during Maven's integration-test phase.
There are nearly 10 such IT Maven projects, a subset of which intentionally result in BUILD FAILURE, and attendant verify.groovy scripts that ensure those builds fail for the correct reason. Ideally each IT test sub-build that fails for the correct reason results in that IT test passing, but by including any of these failing IT tests as part of the whole integration test suite causes the overall Maven run to fail as well, which is incorrect in my case.
How do I coax Maven to run those failing Maven sub-builds, ignore their build results, but honor the results of their Groovy verification scripts?
Edit: One IT test (disabled) is committed here.
If you like to write an integration test which is intended to fail as a result
you have to express this via the invoker.properties file like this:
invoker.buildResult=failure
The full description of the file can be found in the documentation.
i am trying to build maven command to run specific test:
i want to be able to execute this:
mvn test
mvn integration
mvn specificdata
so each test will go into a folder and run the suite
src/test
src/integration
src/specificdata
mvn test works with the test folder but when i run
mvn specificdata i get
[ERROR] Unknown life-cycle phase "specificdata".
same for integration
how can i make mvn run these tests independently?
This cannot be done in the way you describe it.
test is a phase and as such, part of the standard lifecycle. Calling mvn test does not only run tests, but executes the phases before test as well.
The standard lifecycle also offers phases for integration tests, esp. integration-test. Integration tests are usually put into src/test as well and distinguished by a naming convention. But beware: Calling mvn integration-test will call all previous phases (including test, compileetc.) as well.
https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
Ive included the surefire-report plugin to generate html reports.
When I run "mvn site", it runs my tests and generates the reports but the build passes even if there are test failures.
On the other hand, if I run "mvn install site", the build fails at the test failure and does not even do the site phase. I tried adding -fae but does not help. How can i prevent this?
Ideally I would like it to run through all tests in all my modules, run site, generate the reports and then fail the build.
Note, -Dmaven.test.failure.ignore=false already but no use. Also, if i remove the surefire-report plugin, everything is fine.
On a side note, any good alternatives to surefire-report plugin to generate html junit test reports?