IntelliJ has a nice integration with JUnit that allows me to run some or all of my unit tests and visualize the results of each test and test class with a collapsible/expandable hierarchy that even takes into account JUnit5 nested test classes. I can click on any failed test to see the log output from running just that method. It looks like this:
I have now defined an integration test phase to my project using two popular Maven plugins:
maven-failsafe-plugin separates my JUnit tests into two phases. The test classes ending in "Test" run during the test phase, and those ending in "IT" run during the integration-test phase.
fabric8's docker-maven-plugin which I'm using to spin up a test database in the pre-integration-test phase and shut it down in the post-integration-test phase.
I can tell IntelliJ to run a Maven command (such as maven clean verify) that runs this lifecycle and tells me if the tests have passed. However, a Maven "run configuration" in IntelliJ doesn't produce the same helpful output. All I get is the success or failure status of each phase, and I can navigate to the console output for a phase, but not an individual test. The output looks like this:
So here's my question: Is there any way to configure IntelliJ (or Maven, or both) to give that hierarchical test-by-test output while keeping the Maven lifecycle I've defined, with the temporary use of a docker-container database for my integration tests?
I know that I can manually spin up the test database container, use IntelliJ's JUnit runner for my integration tests, and then manually stop the container. Can I do better?
Unfortunately there is no tool to show test-by-test output while keeping the Maven lifecycle as Maven run configuration is used in this case. Please use Junit run configuration for Class/method output in test runner tab.
Related
I'm bit confused with mvn verify phase. I've created a Spring Boot project (a simple project, without any explicit configurations added). I've created a few JUnit unit tests which are run with both the mvn verify and mvn test commands.
There isn't any difference observed in the mvn verify and mvn test command output.
What does mvn verify do different than mvn test?
Also some posts on Stack Overflow mentions that mvn verify runs the integration tests. If this is the case then I have few questions.
How does Maven identify a specific test as a unit test or integration test?
If mvn verify is supposed to run only the integration tests, then why are unit tests executed with it?
First of all, when you run a Maven goal, it will run any previous goal. The order of basic phases is:
Validate
Compile
Test
Package
Verify
Install
Deploy
If you run Test, Maven will execute validate, compile and test. Based on this, the first point is that verify includes test.
Based on official documentation:
TEST - test the compiled source code using a suitable unit testing framework. These tests should not require the code be packaged or deployed
VERIFY - run any checks on results of integration tests to ensure quality criteria are met
To run unit tests, the Surefire plugin is recommended. And Failsafe for integration tests.
The verify command executes each default lifecycle phase in order (validate, compile, package, etc.), before executing verify. In most cases the effect is the same as package. However, in case there are integration tests, these will be executed as well. And during the verify phase some additional checks can be done, e.g. if your code is written according to the predefined checkstyle rules.
Conclusion: if you want to run your integration tests and check it, use verify. If you only want to run unit tests, use test.
My personal advice: if in doubt, use verify.
How does Maven identify a specific test as a unit test or integration test?
Integrations Test always takes a name like IT.java or IT.java or *ITCase.java
When generating a skeleton Maven plugin from archetype, the new project includes a Maven project under the src/it directory. It is an integration it (hinted at by the it dir name) and fresh out-of-the-box it passes when run during Maven's integration-test phase.
There are nearly 10 such IT Maven projects, a subset of which intentionally result in BUILD FAILURE, and attendant verify.groovy scripts that ensure those builds fail for the correct reason. Ideally each IT test sub-build that fails for the correct reason results in that IT test passing, but by including any of these failing IT tests as part of the whole integration test suite causes the overall Maven run to fail as well, which is incorrect in my case.
How do I coax Maven to run those failing Maven sub-builds, ignore their build results, but honor the results of their Groovy verification scripts?
Edit: One IT test (disabled) is committed here.
If you like to write an integration test which is intended to fail as a result
you have to express this via the invoker.properties file like this:
invoker.buildResult=failure
The full description of the file can be found in the documentation.
I use junit in my java project (developed using intellij idea) for unit test, and I want to configure build step in team city to run my unit tests only. I also use maven to build my project. It works when I set goals for maven to "clean compile" but I dont know how to configure build step to run unit tests.
Also in command line. when i run "maven test" it runs unit tests correctly and shows the failures.
I can't comment on TeamCity, but I'll try to help anyway out of my maven knowledge.
So first of all mvn clean compile will never run your unit tests, because maven has a concept of lifecycle and testing phase is coming after compile phase.
When you run mvn test it will run all the face up to (including) the test phase so the unit tests will run as a part of maven default lifecycle.
Now you're asking about a "build step for running unit tests" from which I conclude that you need a separate step. In maven phase is nothing more than running a series of plugins. In maven plugin has goals, so you can run a plugin responsible for running unit tests directly.
In maven this plugin is called "surefire" and a goal is called "test", so you can run:
mvn surefire:test
Given the classes (production code and tests) are compiled, you will see that this only runs your unit tests. So this probably has to be configured in Team City.
I have both unit and acceptance tests in a maven project (spring boot). I would like to run unit tests only when my build runs in TeamCity. I am using cucumber.
When i run the tests via command line, everything works as expected (only unit tests are run)
c:\apache-maven-3.3.9\bin\mvn package -Dcucumber.options="--tags #unit"
However, in teamcity, all tests are being run (unit and acceptance). It seems teamcity ignores my cucumber.options
In addition, when I double click on the 'test' lifecycle in Intellij, all tests are run as well (not just unit tests) So my guess is that TeamCity is doing exactly what the 'test' lifecycle does.
How can i get around this problem (in TeamCity)?. I have tried using a 'Command line' step, which works, however, i lose all the tests reporting as well as test coverage reports.
i have solved the Intellij problem by creating (or changing) a configuration:
Try writing it this way in TeamCity:
"-Dcucumber.options= --tags #unit"
I have some Unit tests, that when I try to run, it automatically creates Maven run/debug configuration, instead of JUnit (the integrated IDEA tab).
For some tests it does use JUnit run\debug configuration and manually - I can create both Maven and JUnit.
How do I make JUnit to be the default test runner ?
The problems is that I had maven runner plugin installed, causing all my tests to run with Maven
You can use the maven option "-Dskip=true" to suppress the maven test execution, and you can add your own test configuration to the build process. This screenshot was taken form IntelliJ 15.0.2.
Update: maven option