I am writing integration tests in a SpringBootApp with Junit5. It is a must for me to follow a execution order of REST Api calls. In one test class it is ok to use #TestMethodOrder from jUnit5 or another approach for test execution ordering in a class. My question is regard execution test classes in specific order. I found an open issue on the topic in the Junit repo https://github.com/junit-team/junit5/issues/1948 . It's not resolved. Is there a way to achieve this? I have think of #Categories introduced from Junit5 but it will be too hacky approach, it is not an option.
Related
I am working on a large project and need to offer users the ability to optionally enable or disable local integration test cases ( For pipeline, test cases must be enforced).
First of all, welcome to the community.
Next, you can modify the test task inside the build.gradle file or maybe add a new task called integrationTest and implement your custom logic there.
As an instance, you can check this gist on Github: Separating tests from integration tests with Gradle
You can also use #Profile annotation to your integration test classes and run your tests with different profiles. You can read more about profiles using the following link: Spring Profiles
I have an integration test that's using #SpringBootTest to spin up a Spring application context that's testing a simple Spring Boot app. I'm using Spock for writing the tests, my build tool is Maven.
I'm looking for a way to run the same test class multiple times with different configurations for the test (I have a set of config options and I need to ensure consistent behavior for a certain permutation of config options). The first idea I had was to use profiles to define the exact permutation, maybe it could also work by using #TestPropertySource in some way. However, I don't see any way to run a test class multiple times, using different configurations each time.
I know that I could run all tests with a given profile, but in my case I want to only apply the different configs to certain test classes.
I can also use a where block to repeat spock tests as described here, but that doesn't allow me to switch spring configurations for each run
The easiest way is to use simple subclasses, i.e., you define all your tests in an abstract base class and then subclass it for every variation and add the necessary annotations to the subclasses. This approach works well if you only have a limited set of variations, and provides good reporting feedback as every variation is reported as its own specification.
I have a bunch of tests implemented using Spock and that contain the #Stepwise clause, meaning that each test method contained in each class must be executed sequentially (they're dependant on the previous one).
Now I'd like to implement a retry mechanism for flaky tests and I'm trying the one provided by the surefire plugin.
this works, but it only retries the test method that failed, when I want it to rerun the whole class (even the tests that were executed correctly, since they're a pre-condition for the next ones). How can it be configured?
An example of how one of my test classes looks like (I know it's a stupid one, I just wrote it like this way because it is suitable as en example):
#Stepwise
class DummyTest extends loginWithNewUser {
def infoDialogAppears() {
given:
at homePage
when:
infoButton.click()
then:
infoDialog
}
def closeInfoDialog() {
when:
infoDialog.close()
then:
!infoDialog
}
Your test is not a simple Spock test but a Geb test. Anyway, have you considered handling flaky tests with the Spock #Retry annotation available both in Spock 1.3 and Spock 2.0?
I have not tried to do that from Surefire/Failsafe or even tried to do it in Spock at all because I believe in fixing flaky tests and having them red if they do fail instead of hiding the flakiness, eliminating the desire to fix it. But FWIW, Spock has that feature built in and I recommend to use it sparingly and on a per-test basis rather than globally because re-running each single failing integration test (even the non-flaky ones) just makes your test execution slower and keeps developers waiting for feedback.
I have some set of test cases configured with TestNG. I developed preconditions in junit5 and this has to be run before the test starts. So I wanted to run in sequence Line Precondition(junit5) and then Testcases.
I am using dependency for Junit5 and TestNG7 in PM.XML. Below is a snapshot of POM.xml htmlunitTest.java is for junit5 and testng.xml for TestNGTest cases. while running build is terminating successfully without execution of any test.
You probably do not want to hear my answer. It is the best I have anyway: Don’t take that route! Don’t couple two frameworks that are not made to work together.
Although there might be a convoluted and hacky way to achieve what you describe, you’re setting yourself up for unnecessary technical complexity. Stick with one framework and use its own means for preconditions or fixing a test order.
Is there any way to specify the order of the unit tests classes run by a Gradle Test task?
I'd like to get some known longer-running tests at either the front or the back of the list, but don't know if it's possible without splitting my tests' execution between multiple tasks.
Running JUnit 4.12 and Gradle 4.5.
Gradle simply delegates execution to the JUnit runner.
So if you want specific test class ordering, you will need to create a Suite and specify the test classes in the order you want, see the JUnit documentation for this.
Now, given the flexibility of Gradle in terms of having different source roots, I would strongly recommend doing the separation at the Gradle level, by create extra test source roots, test task and the like. This will allow you to effectively control when these long running tests are run in a standard build execution, but also to skip them or run these only when desired. The ordering at the JUnit level will not give that flexibility without much more tweaking. See the Gradle documentation on adding source sets.