I'm setting up some tests. I have the following entries in my build.gradle file:
integTest {
useTestNG() {
}
}
integTest2 (type: Test){
useTestNG() {
include 'Group2'
}
}
And in my test I have the following annotation:
#Test (groups={"Group2"})
public void testMethod() {
// Test code here
}
When I run the second task (integTest2) with gradle it only builds my directory and doesn't actually pick up any tests to run (integTest works fine and runs all the tests but I want to be able to run separate suites individually too).
Is there something obvious I have missed?
Figured out the answer. I was using our company's internal version of gradle which only recognized "integTest" as the main task. So as a workaround I had to do something like this:
def allTestGroups = ['all', 'Group1', 'Group2']
def testGroup = project.hasProperty("testGroup") ? project.testGroup : 'all'
integTest {
useTestNG() {
includeGroups "${testGroup}"
excludeGroups allTestGroups.findAll { it != "${testGroup}" }.collect { "'${it}'" }.join(',')
}
}
Just try to change include 'Group2' to includeGroups 'Group2', like this:
integTest2 (type: Test) {
useTestNG {
includeGroups 'Group2'
}
}
Related
I am using TestNG and Gradle, what I am trying to achieve is, if the task that runs the tests fails, run another task which is also of type test which actually sends the test report. If all tests pass, don't do anything, I know about finalizedBy, but that sends the test report either way.
I tried something like this, but no luck.
task uiTest(type: Test) {
useTestNG() {
environment "DOMAIN", "${DOMAIN}"
useDefaultListeners = true
suites "src/test/resources/ui-tests/ThemeA/chrome67.xml"
}
reports {
html {
enabled true
}
reports.html.destination = "build/reports/TestReports/uiTestThemeA"
}
testLogging {
showStandardStreams = true
exceptionFormat "full"
events = ["started", "passed", "skipped", "failed"] //, "standard_out", "standard_error"]
showExceptions = true
showStackTraces = true
}
}
task testing(dependsOn: uiTest, type: Test) {
boolean state = uiTestThemeA.state.executed
if(!state) {
println name
useTestNG() {
suites "src/test/resources/ui-tests/sendReport.xml"
}
}
}
If I understand you correctly, then the following should do what you need. Replace your testing task with the following one and configure your uiTest task to be finalized by the testing task:
task testing(type: Test) {
onlyIf { uiTest.state.failure }
useTestNG() {
suites "src/test/resources/ui-tests/sendReport.xml"
}
}
uiTest {
finalizedBy testing
}
Some notes:
the onlyIf statement does the magic you are looking for
this assumes you execute uiTest
Our company has a Gradle script plugin with a number of tasks in it. For instance, it includes the Jacoco afterEvaluate block from this answer:
def pathsToExclude = ["**/*Example*"]
jacocoTestReport {
afterEvaluate {
classDirectories = files(classDirectories.files.collect {
fileTree(dir: it, exclude: pathsToExclude)
})
}
}
We would like to take that pathsToExclude variable and define that in our build.gradle file and have the rest of the logic in a script plugin (let's call it company-script-plugin.gradle. For instance:
apply from: http://example.com/company-script-plugin.gradle
companyConfiguration {
pathsToExclude = ["**/*Example*"]
}
Our initial thought was to add a task in the build script so that we could get a companyConfiguration
task companyConfiguration {
ext.pathsToExclude = []
}
However, we think that this is a hacky workaround because running the task doesn't do anything. What is the proper way to create my own configuration block?
We'd like it to be as simple as possible, and to be a script plugin (rather than binary plugin) if possible.
Here you've an example how it can be done:
apply plugin: CompanyPlugin
companyConfiguration {
pathsToExclude = ['a', 'b', 'c']
}
class CompanyPlugin implements Plugin<Project> {
void apply(Project p) {
println "Plugin ${getClass().simpleName} applied"
p.extensions.create('companyConfiguration', CompanyConfigurationExtension, p)
}
}
class CompanyConfigurationExtension {
List<String> pathsToExclude
CompanyConfigurationExtension(Project p) {
}
}
task printCompanyConfiguration {
doLast {
println "Path to exclide $companyConfiguration.pathsToExclude"
}
}
Also, please have a look at the docs.
I am using Gradle with TestNG. I have this build.gradle:
useTestNG() {
useDefaultListeners = true
suites "src/test/resources/tests1.xml"
suites "src/test/resources/tests2.xml"
}
}
How can I run for example only tests1.xml from command line?
you can use project properties to add/change/... different suites. In you example you are probably running
gradlew test
which run both suites. If you modify test task in your build.gradle
def suite1 = project.hasProperty("suite1")
def suite2 = project.hasProperty("suite2")
test {
useTestNG() {
dependsOn cleanTest
useDefaultListeners = true
if(suite1) {
suites "src/test/resources/simpleSuite.xml"
}
if(suite2) {
suites "src/test/resources/advancedSuite.xml"
}
}
}
you can choose suite in this way
gradlew test -Psuite1
gradlew test -Psuite2
gradlew test -Psuite1 -Psuite2
you can specify variable let say suiteFile with default value and use it in testNG section. For example:
ext{
set suiteFile, default is 'testrun_config.xml'
if (!project.hasProperty('suiteFile')) {
suiteFile = 'testrun_config.xml'
}
}
test {
useTestNG() {
dependsOn cleanTest
useDefaultListeners = true
suites "src/test/resources/"+suiteFile
}
}
Refer qaf gradle build file
If you want to pass through command line
gradlew test -PsuiteFile=test.xml
In build.gradle update:
test {
useTestNG() {
if (project.hasProperty('suite1')) { suites './src/test/suite1.xml' }
if (project.hasProperty('suite2')) { suites './src/test/suite2.xml' }
}
}
Use command gradlew test -Psuite1 to run suite1 and similarly update for suite2 as well.
build.gradle
test {
jvmArgs "-DisParallel=true"
useTestNG(){
suites testngxml
}
}
when i am trying to run gradlew clean test -Dgroups='sanityTests' will not run tests in that specific group but will run all tests. Can you please provide some insight on this.
test {
jvmArgs "-DisParallel=true"
useTestNG(){
suites testngxml
includeGroups System.properties['groups']
}
}
We can pass by adding includeGroups inside test which is shown below:
test {
useTestNG() {
dependsOn cleanTest
useDefaultListeners = true
includeGroups groupName
}
}
I use the following annotation to tag my integration tests:
#Target({ ElementType.TYPE, ElementType.METHOD })
#Retention(RetentionPolicy.RUNTIME)
#Tag("integration-test")
public #interface IntegrationTest {
}
This is the filter I use in build.gradle to exclude these tests from gradle build:
junitPlatform {
filters {
tags {
exclude 'integration-test'
}
}
}
So far, so good.
Now I would like to offer a Gradle task which specifically runs my integration tests – what's the recommended approach?
Based on https://github.com/gradle/gradle/issues/6172#issuecomment-409883128
Amended in 2020 to take lazy task configuration and Gradle 5 into account. See answer's history for older versions.
plugins {
id "java"
}
def test = tasks.named("test") {
useJUnitPlatform {
excludeTags "integration"
}
}
def integrationTest = tasks.register("integrationTest2", Test) {
useJUnitPlatform {
includeTags "integration"
}
shouldRunAfter test
}
tasks.named("check") {
dependsOn integrationTest
}
Running
gradlew test will run tests without integration
gradlew integrationTest will run only integration test
gradlew check will run test followed by integrationTest
gradlew integrationTest test will run test followed by integrationTest
note: order is swapped because of shouldRunAfter
History
Gradle 4.6+ supports JUnit 5 natively
JUnit 5 deprecated their plugin: https://github.com/junit-team/junit5/issues/1317
JUnit 5 deleted plugin: 'org.junit.platform.gradle.plugin'
JUnit 5 closed junit5#579 (same as OP's question) as won't-fix (due to decommissioning their plugin)
Gradle supports the above feature: https://github.com/gradle/gradle/issues/6172
Tip
Note: while the above works, IntelliJ IDEA has a hard time inferring stuff, so I suggest to use this more explicit version where everything is typed and code completion is fully supported:
... { Test task ->
task.useJUnitPlatform { org.gradle.api.tasks.testing.junitplatform.JUnitPlatformOptions options ->
options.includeTags 'integration'
}
}
build.gradle.kts
Root project Kotlin DSL drop-in for configuring integration tests in all modules in Gradle 5.6.4
allprojects {
plugins.withId("java") {
#Suppress("UnstableApiUsage")
this#allprojects.tasks {
val test = "test"(Test::class) {
useJUnitPlatform {
excludeTags("integration")
}
}
val integrationTest = register<Test>("integrationTest") {
useJUnitPlatform {
includeTags("integration")
}
shouldRunAfter(test)
}
"check" {
dependsOn(integrationTest)
}
}
}
}
I filed an issue: https://github.com/junit-team/junit5/issues/579 (as suggested by Sam Brannen).
Meanwhile, I am using a project property as a workaround:
junitPlatform {
filters {
tags {
exclude project.hasProperty('runIntegrationTests') ? '' : 'integration-test'
}
}
}
Consequently, integrations tests will be skipped with:
gradle test
but will be included with:
gradle test -PrunIntegrationTests
Gradle 6
I am not sure if it is because Gradle behavior has changed, but the highest voted answer did not work for me in Gradle. 6.8.3. I was seeing the integrationTests task run along with the main test task. This simplified version worked for me:
test {
useJUnitPlatform {
excludeTags "integration"
}
}
tasks.register("integrationTests", Test) {
useJUnitPlatform {
includeTags "integration"
}
mustRunAfter check
}
Commands:
./gradlew test or ./gradlew clean build - Runs tests without
'integration' tag.
./gradlew integrationTests - Only runs test with
'integration' tag.
According to me, the best, current working code to solve this, is the one presented by: TWiStErRob found here.
Note that tests must be tagged with ui in the example below (Junit5 tagging):
task uiTest(type: Test) {
useJUnitPlatform {
includeTags 'ui'
excludeTags 'integration'
}
}
Howe ever, I did not managed to get the Junit5 test-suit-thing to be run from gradle directly witch I would think would be an even nicer solution. But I think the solution buy TWiStErRob, is good enough. The down side is that the gradle.build file now also will be bloated with test-suit-things.
Please note that it is fine to create multiple test suites in the gradle file like this:
task firstTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-first-test-suite'
}
}
task secondTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-second-test-suite'
}
}
Then then all could be run separately like this:
gradlew firstTestSuite
gradlew secondTestSuite
gradlew ui
Solution run with Gradle 6.6.1
A similar approach to Rahel Lüthy avoiding the usage of empty strings, in this case to run all tests or just some tags:
test {
useJUnitPlatform() {
if (project.hasProperty("includes")) {
includeTags(project.property("includes") as String)
}
}
}