Hi is there plan to add support for 'configfailurepolicy' property (http://testng.org/doc/documentation-m...) for TestNG in gradle ?
It's really inconvenient that if some test fails in Before* phase, all other tests are skipped (even unrelated tests in different classes)
Related
I'm creating a fairly large test suite using gradle, geb, and spock in conjunction. Gradle is obviously building and kicking off geb and spock, but I think that spock is where I can control and specify which Spec to run.
I'm building this based off of this starter.
https://github.com/AutomationSchool/geb-and-spock-automation-examples
How can I set this to run just one Spec?
Gradle's Test task takes a tests option. The supported patterns are documented in the javadoc for TestFilter. So if you want to run spec class called MySpecToRun in the project you linked to then you can do it this way:
./gradlew chromeTest --tests=MySpecToRun
When I invoke mvm test, I want to execute a setup.sql script before Surefire JUnit is invoked and then execute a teardown.sql script after.
I know from questions like this how to execute scripts during the test phase, but I have no idea how to define this specific sequence of events. Thank you!
Not with the surefire plugin but with its sibling the failsafe plugin. They both execute Tests but in different life-cycle phases. The surefire plugin in test and the failsafe plugin in integration-test. See life-cycle phases and the default plugin bindings.
The advantage of the failsafe plugin running in the integration test phase is that there are pre- and post- phases.
Since you mention some sql script it seems you want to prepare a database. At that point you are not really doing unit testing anymore but writing an integration test. So using the failsafe plugin makes the most sense here.
Well, I would like to have a maven goal execute-custom-tests inside my custom-maven-plugin that consists of running test methods (This tests are not unit tests). Something similar to test goal of soapui-pro-maven-plugin, for example.
Why? Basically the main objectives of the plugin are testing stuff (not unit testing) and the tests in src/test are for unit testing, right?
Being more specific I was thinking about something like this:
#Mojo (name = "run-custom-tests", LifecyclePhase.TEST)
public class TesterMojo extends AbstractMojo {
#Parameter(property = "someParameter")
private String someParameter;
// [...] parameters for test configuration
#Override
public void execute() throws MojoExecutionException, MojoFailureException {
// Piece of code that executes a set of custom tests which procedure I specified.
}
}
When test fail, I would like them to be marked as failed tests not as failed executions. What's the right thing to do here? Show me the light, please.
Maven conventions support two types of testing out of the box: unit tests (via maven-surefire-plugin) and integration tests (via maven-failsafe-plugin).
By default, maven-surefire-plugin only looks for the following files with unit tests:
**/Test*.java
**/*Test.java
**/*TestCase.java
Similarly, default includes for integration tests run by maven-failsafe-plugin are the following:
**/IT*.java
**/*IT.java
**/*ITCase.java
So, as you can see, Maven lets each plugin figure out which tests it should care about. So it's perfectly fine for src/test/java to contain different types of tests, not just unit tests.
Different folder
You can put tests in a different folder too. One example would be if you have non-Java tests, since then src/test/java location doesn't make sense. Standard Maven plugins get project model from Maven to figure out the src/test/java location and some 3rd party plugins use the same mechanism. Depending on the plugin you use, you might want to check out its configuration or use maven-build-helper-plugin to add-test-source in order for some plugins to pick up another test folder automatically.
Different tests on demand
From the Maven perspective the core difference between unit tests and integration tests is the additional requirements for the later: they often need to have your project already packaged and they often need additional setup or teardown. But you yourself can set up multiple test goals during both test and integration-test phases. All major test frameworks support specifying which test suite should be run when (e.g., via groups). If your framework doesn't, you can still use plugin includes/excludes. It is a standard practice to combine this with Maven profiles in order to only run smoke tests by default (during development) and to run full tests on CI environment. You can use the same approach to enable anyone (a tester?) to run extra tests on demand, e.g., to run extra heavy tests when certain important part of the code has changed.
I need to create a build in such a way that, unit test, integration test and performance test can be executed separately using Maven and Gradle.
Acceptance Criteria :
Seperate Build files (Maven and Gradle) for unit test, Integration test and Performance test.
The test execution flexible enough to be executed on Demand like
Unit test Alone (Default profile)
Unit Test & Integration Test (Profile With Integration Test)
Unit test and Performance test.(Profile With PerformanceTest)
Start reading the documentation:
https://maven.apache.org/guides/
http://books.sonatype.com/mvnref-book/reference/
https://docs.gradle.org/current/userguide/userguide.html
ScalaTest has such a feature as tagging different tests. It would be great somehow to instruct gradle about what type of tests it should run while executing test task(as it done in scalatest maven-plugin). How this can be achieved?
As Gradle doesn't have any specific ScalaTest support, the question is if ScalaTest exposes this feature in a JUnit-compatible way. Alternatively, you could leverage Gradle's support for JUnit categories (see "23.12.5. Test grouping" in the Gradle User Guide).