I have a project, with tests implemented by junit 4 and 5. Some of the tests require a database to be present, is there any way I can mark these specific tests (is TestSuite the answer?), so gradle continue the build even if those tests fail?
I don't want to skip other tests, but just these specific tests.
And I am using junit 5 vintage, so my test task runs both junit 4 and junit 5 tests together.
Honestly, I wouldn't do this. What is the point of a test if you ignore it when it fails? You might as well delete the test right away. Can't you delay test execution if setup is slow?
That said...
gradle --continue will continue execution when a task fails, collecting all the errors. You may still need to ignore Gradle's exit code (e.g., in a build pipeline).
You can also use the ignoreFailures test property to always ignore failing tests.
Both may be a bit too broad. Depending on your build script, you could add have a separate test target for database tests and add ignoreFailures only to that.
If you want to handle this in JUnit, you could look into the Assume class which lets you skip a test if a certain condition is (or is not) given:
import static org.junit.Assume.*;
public class SomeDatabaseTest {
#Test
public void someThing() {
assumeTrue(Database.isAvailable());
// actual test goes here
}
}
Add this block of code to your module or project level build.gradle file:
project.gradle.teskGraph.whenReady {
connectedDebugAndroidTest {
ignoreFailures = true
}
}
You can use any of the test tasks instead of connectedDebugAndroidTest. You can also specify #Test for only those tests annotated as such. See Android documentation here for example of tests.
Related
I posted this: Maven JUnit 5: Run all unit tests by default or run only a specific test suite from the command line .
I'm simply posting it again with almost exactly the same title because nothing about my base problem has changed, but I will include what I tried to do with Tags, which are not working for me.
We are upgrading our services to a relatively recent SpringBoot version, along with JUnit 5. We normally run our builds with just "mvn package", which runs all of our unit tests by default.
We also have "component tests", which are run from a separate command line, and which are all specified in a single test suite with a specific name. Before JUnit 5, we would run this with something like:
mvn -Dtest=ComponentTestSuite test
This was working fine with JUnit 4.
The class looks like this:
#Suite
#SelectClasses(<testclassname>.class)
public class ComponentTestSuite {
}
With JUnit 5, this ends up saying "No tests found".
We are using v2.3.12 of Spring-Boot, which by default includes a somewhat older version of JUnit 5. I am overriding those defaults and including v1.8.2 of the junit-platform components and v5.8.2 of the jupiter components.
For the service that I'm testing this with, the test suite only has a single component test (which is unusual). I WAS able to simply replace "ComponentTestSuite" with the name of the component test class, and that would run that single component test.
I noticed this thread: Junit5 test suites with #Suite annotation doesn't execute tests with mvn test command .
Unfortunately, what I found was that changing "-Dtest" to "-Dinclude" simply ran all my default unit tests and ignored the test suite.
So, I was told that I'm "not supposed to do it this way", although it's not clear to me exactly which parts I'm not supposed to do, but I am supposed to use "Tags". Ok, so I tried to use tags.
I also noticed this article: https://www.baeldung.com/junit-filtering-tests , although that article appears to use some deprecated mechanisms, like "#RunWith(JUnitPlatform.class)", so I'm not sure how much I can believe from that article.
I've been trying several variations, and my last attempt is this:
#Suite
#Tag("ComponentTest")
#IncludeTags("ComponentTest")
public class ComponentTestSuite {
}
And I added #Tag("ComponentTest") to my one component test class.
This still doesn't work. I try to run mvn -Dtest=ComponentTestSuite test, and it says "No tests found". If I instead try mvn -Dtests=ComponentTest test that just runs all of my unit tests and ignores my component test suite. If I instead run mvn test -Dgroups=ComponentTest, that gives me another variation of "No tests found" in that it doesn't print that, it just literally executes no tests without giving an error.
Looking at that suite class now, it does seem nonsensical, as it doesn't really "contain" anything anymore, which is unfortunate. It was good to have that list of component test classes in one place. Now, it seems like the suite class is pointless, but I can't get anything to work anyway.
Note that I haven't yet added a "groups" element to my surefire config, and I haven't edited all of my unit tests to add a #Tag("UnitTest") annotation. I've only edited the suite class and the one component test.
Update:
To address a comment, I am specifying the following dependencies (among others):
Version 1.8.2 of
junit-platform-suite
junit-platform-suite-api
junit-platform-launcher
junit-platform-commons
junit-platform-engine
junit-platform-runner
junit-platform-suite-engine
Version 5.8.2 of
junit-jupiter
junit-bom (probably unnecessary)
The entire suite class is this:
import org.junit.jupiter.api.Tag;
import org.junit.platform.suite.api.IncludeTags;
import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.Suite;
#Suite
#Tag("ComponentTest")
#IncludeTags("ComponentTest")
public class ComponentTestSuite {
}
My project has two different cucumber test and each one needs a different spring context configuration.
The problem I have is that when I run each test individually from Intellij, they load the right Spring Context and the tests are passing, but I press run all test, none of them are passing.
Running a maven test, both test are passing.
this is my code:
#RunWith(FCSCucumber.class)
#Cucumber.Options(strict = true
// , tags = {"~#ignore"}
// , tags = {"#Only"}
, glue ="feature.scenario1"
, features = "src/test/resources/feature/scenario1/"
)
#FCSApplicationProperties(commonProps="config/scenario1/exec.common.properties",
environmentProps="src/test/resources/scenario1-test.properties")
public class TesScenario1Features {
}
#ContextConfiguration("/cucumber-scenario1.xml")
public class scenario1Steps {
......
}
#RunWith(FCSCucumber.class)
#Cucumber.Options(strict = true
// , tags = {"~#ignore"}
// , tags = {"#Only"}
, glue ="feature.scenario2"
, features = "src/test/resources/feature/scenario2/"
)
#FCSApplicationProperties(commonProps="config/scenario2/exec.common.properties",
environmentProps="src/test/resources/scenario2-test.properties")
public class TesScenario2Features {
}
#ContextConfiguration("/cucumber-scenario2.xml")
public class scenario2Steps {
......
}
Thank you very much for your help
The issue is that the IntelliJ cucumber plugin is using the cucumber cli to run tests, without using the JUnit runner at all. This causes several limitations, like requiring the spring annotations on the step definition classes instead of the runner, or by default requiring the steps definitions to be in the same package as the scenario files.
In your example I would actually expect also running a single test to fail, unless the correct application properties are also referenced by the /cucumber-scenario{1,2}.xml files.
The only option I see with the standard cucumber implementation would be to extract the tests into separate projects.
I'm actually working on an alternative implementation of cucumber with an improved spring integration that you might want to try. It's not fully integrated with IntelliJ yet though.
I guess am trying to get a corner case to work here. In my current project there are about 20 integration tests. One new integration test requires #EnableAsync to make the test work:
#RunWith(SpringRunner.class)
#EnableAsync
#SpringBootTest(webEnvironment = WebEnvironment.NONE)
public class MyITest {
:
}
When run alone, this test works fine.
Considering Maven and Eclipse' execution of tests in one project and knowing that the environment is only created once and reused (or soft-reset) for all integration tests, it's somewhat a requirement that this integration test runs first. However, that's (nearly?) never the case.
Therefore, this integration test (nearly?) always fails. One obvious solution is to add #EnableAsync to all integration tests. However, that's a bad dependency which I bet is broken once somebody adds another integration test and forgets this requirement.
I'm looking for a way to force the SpringRunner to completely reset the context and really start it from scratch also looking at #EnableAsync. Ideally that way includes to flag that SpringRunner has to reset the context (i.e., remove the #EnableAsync) after the test, too. That way any order of execution would ensure that only that very one test has the #EnableAsync.
Is there a way to do this? Or can I manually turn on/off the async-stuff by code in a #Before/#After method?
take a look at DirtiesContext
Not sure if this is what you're looking for.
Possible duplicate of: How do you reset Spring JUnit application context after a test class dirties it?
Whow, I think I just found out by accident... What I have now:
#RunWith(SpringRunner.class)
#EnableAsync
#SpringBootTest(webEnvironment = WebEnvironment.NONE, classes = {
ClassWithAnAutowiredAsyncDependency.class // <=== difference!!! ===>
})
public class MyITest {
:
#Autowired
private ClassWithAnAutowiredAsyncDependency mine;
:
}
It seems as if the given classes are reset (specially?) or at least the autowiring happens in there again or something. I can't explain it any different.
I'm sure that this integration test is not the first integration test being run and still the asynchronous bit seems to be in place.
Well, test is green, it works...
I've tried just about every configuration I can think of (and reviewed some answers on StackOverflow), but all of our tests show the 'Failed to load ApplicationContext' error when run through Hudson. What is interesting is that some tests appear to run and pass, while some run and fail (as expected), but regardless I'm always getting the errors list for all tests. Here is the basic configuration:
#ContextConfiguration(locations = "classpath:/MyTest-context.xml")
#RunWith(SpringJUnit4ClassRunner.class)
public class MyTest {
#Autowired
private ApplicationContext applicationContext;
public MyTest() {}
#Test
public void doSomething() {
// Implementation...
}
}
UPDATE:
There appears to be a duplicate set of tests running, one for Emma coverage reporting, and the other the normal tests. It is when the tests run for Emma coverage that they are showing the errors. If I turn off the "emma:emma package" goal so those don't run then I don't get the errors, and the tests appear to run fine. I'm not sure if that helps any.
The answer ended up being close to what gontard was pointing to, which is an issue that was hidden by the way Emma's classloader works. Between my local JUnit tests, what was running in our DEV environment, and what was running in Hudson with Emma, all of them have a different way in which the classloader orders the loading of libraries and classes. I ended up reviewing the stack trace on the test results, and it turns out on my local, a new version of a library was loaded via the POM, but in Hudson Emma was loading an old version of a library first. I had to find and remove the old version, and everything now works fine.
I want to to integration tests against an http server. So far I have only experiences with junit for unit testing.
I have two requirements: The framework must have a maven plugin and the tests cases code must be clean - so no dirty hacks and no boilerplate code.
Plain JUnit is good for unit testings, #Test methods are individual. But for integration testing I have to process several dependant steps which must exchange some kind of state (variables).
I already read:
Can we use JUNIT for Automated Integration Testing? and Passing JUnit data between tests and came to the conclusion that I don't like static fields in unit test and I don't want to use TestNG and add dependency annotations on tests and I don't want to put my test into one long unreadable test method.
I though more about some syntax like:
public class MyIntegrationTest() {
#Step
public void testCreate(Context context) {context.put("foo");}
#Step
public void testUpdate(Context context) {context.get();}
#Step
public void testDelete(Context context) {context.get()}
}
So I want to enhance/use ?Unit in a way that it executes #Step methods with a context instance as argument. The methods must be called by the framework in order and cannot be called individually. In a perfect world, all ?Unit guis would show the #Step like an #Test but this is optional...
Any hints how to do this?
Jan
The first point is to check the Maven Failsafe Plugin which is intended for doing integration tests with Maven. Second you have to name your Integration tests based on the conventions used by Maven FailSafe Plugin after that you should be able to run your integration tests simply with maven (by mvn clean verify).
So this means you have to name your integration test like MyIntegrationIT.java...To define the order of executions you have to use a different framework than JUnit may be TestNG which supports this kind of needs, but you already excluded it. So the questions is what kind of tests would you like to do? Page-flows etc. may be a look at JWebUnit might be look worth...
You might also want to consider http://httpunit.sourceforge.net/. It's useful for checking to see if responses come back from the http server.
However, it doesn't do the #Step functionality. Normally I'd do that by :
#Test
public void MasterTest() {
step1(..);
step2(..);
....
}
public void step1(...){...}
public void step2(...){...}