Grouping JUnit tests with Gradle - gradle

I created a new Groovy project of integration tests and want to be able to be able to group tests together for different components. For example, some tests are only valid for a docker image, while others are for testing a webapp running locally. I'd also like to get reports for the results of the tests and also be able to run the tests over and over if needed.

The first step to grouping tests in JUnit is to create an interface to name the group. Create this in a package your test classes can reach.
If your Test Class looks like this:
package com.example.test.api.get.cert
class ByIssuedBeforeTest {
//awesome tests
}
Then create an empty interface like this:
package com.example.test
interface DockerTest {}
Now change your Test Class to look like this:
package com.example.test.api.get.cert
import org.junit.experimental.categories.Category
#Category(DockerTest.class)
class ByIssuedBeforeTest {
//awesome tests
}
You can also annotate individual tests instead of the whole class.
Now in your build.gradle file:
apply plugin: 'java'
sourceSets {
main {
java {
srcDirs = ["src/main/java", "src/main/groovy"]
}
}
test {
java {
srcDirs = ["src/test/java", "src/test/groovy"]
}
}
}
test {
outputs.upToDateWhen { false }
reports {
junitXml.enabled=true
html.enabled=true
}
}
task nondockerTest(type: Test) {
outputs.upToDateWhen { false }
useJUnit {
excludeCategories 'com.example.test.DockerTest'
}
}
task dockerTest(type: Test) {
outputs.upToDateWhen { false }
useJUnit {
includeCategories 'com.example.test.DockerTest'
}
}
The java plugin for gradle gives you the test task. In the test task we add the outputs line in order to keep the test task from ever being labeled as "UP-TO-DATE", in essence forcing gradle to always run the task when we call it instead of caching the result.
The report block enables JUnit report files to be created. They will be placed in the build/reports/tests directory. If you ran the nonDockerTest it would be build/reports/tests/nondockerTest/index.html.
The useJUnit block tells Gradle to use JUnit to run the tests as well as which category of tests to run (if using includeCategories) or category to not run (if using excludeCategories).
gradle test to run all the test.
gradle nondockerTest to run the tests not labeled as with the DockerTest category.
gradle dockerTest to run just tests labeled with the DockerTest category.
Each of these tasks will create reports in the /build/reports/tests directory.

Related

Generate Jacoco report for integration tests

I've created a new test suite called integrationTest using the jvm-test-suite plugin.
Now I want to generate a jacoco report just for the integration tests.
I create a new gradle task like this:
tasks.create<JacocoReport>("jacocoIntegrationTestReport") {
group = "verification"
description = "Generates code coverage report for the integrationTest task."
executionData.setFrom(fileTree(buildDir).include("/jacoco/integrationTest.exec"))
reports {
xml.required.set(true)
html.required.set(true)
}
}
But the generated HTML/XML report is empty. I have run the integration tests before executing the task and the file integrationTest.exec exists.
Thanks
It seems the important part of the new JaCoCo report task configuration is to wire the execution data via the integrationTest task instead of the exec-file path. The official docs (see last example here) also imply that the source set must be wired as well.
Here is the full build script (Gradle 7.6) that produces a report with command:
./gradlew :app:integrationTest :app:jacocoIntegrationTestReport
// build.gradle.kts
plugins {
application
jacoco
}
repositories {
mavenCentral()
}
testing {
suites {
val integrationTest by creating(JvmTestSuite::class) {
dependencies {
implementation(project(":app"))
}
}
}
}
tasks.register<JacocoReport>("jacocoIntegrationTestReport") {
executionData(tasks.named("integrationTest").get())
sourceSets(sourceSets["integrationTest"])
reports {
xml.required.set(true)
html.required.set(true)
}
}

How do I add testFixures as a dependency of a JvmTestSuite with gradle

I'm using java-test-fixtures in combination with jvm-test-suite. I'd like my testFixtures to be available to both unit tests and my integrationTest suite.
java-test-fixtures adds testFixtures as a dependency to the default unit test suite, along with compile-time and runtime transitive dependencies. What's the right way to add this to integrationTest too?
The following works, but it seems a bit repetitive:
plugins {
id 'java'
id 'application'
id 'java-test-fixtures'
id 'jvm-test-suite'
}
testing {
suites {
integrationTest(JvmTestSuite) {
dependencies {
implementation sourceSets.testFixtures.output
}
configurations {
integrationTestCompileClasspath.extendsFrom testFixturesApi
integrationTestRuntimeClasspath.extendsFrom testFixturesRuntimeClasspath
}
}
}
}
I can also use testFixtures(project), but only if I declare the dependency in a top-level dependency block, with the top-level dependency block appearing after the test suite has been declared:
testing {
suites {
integrationTest(JvmTestSuite) {}
}
}
dependencies {
integrationTestImplementation testFixtures(project)
}
This works, with all the transitive dependencies set up correctly.
Curiously, I can't use testFixtures(project) inside the test suite declaration - the following:
testing {
suites {
integrationTest(JvmTestSuite) {
dependencies {
implementation testFixtures(project)
}
}
}
}
...fails to evaluate.
Is there a preferred way to have a test suite depend upon testFixtures?
I hate to answer my own question! The cleanest solution right now appears to be to fully-qualify the testFixtures function inside the test's dependencies block:
integrationTest(JvmTestSuite) {
dependencies {
implementation project.getDependencies().testFixtures(project)
}
}

How to duplicate/clone a gradle task?

I am using the new JVM Test Suite plugin to create a regression test suite (including a dedicated source set and dedicated dependencies).
testing {
suites {
regressionTest(JvmTestSuite) {
dependencies {
implementation project
implementation 'org.assertj:assertj-core:3.22.0'
}
targets {
all {
testTask.configure {
sourceCompatibility = '17'
targetCompatibility = '17'
useJUnitPlatform {
includeTags 'check'
}
}
}
}
}
}
}
This generates a 'regressionTest' task that runs the tests on my new source set. So far so good.
Now I need to be able to run that task with a different includeTags for JUnit, but I can't find a good way to do so.
I found https://github.com/gradle/gradle/issues/6172 and JUnit5 tag-specific gradle task, but those only work for the duplicating the standard test task. If I do so, it would not run with the correct dependencies and the correct source set.
What's a good way to create a new test task with specific JUnit5 tags when using the JVM Test Suite plugin ?

Run a specific TestNG XML test suite with Gradle from command line?

I am using Gradle with TestNG. I have this build.gradle:
useTestNG() {
useDefaultListeners = true
suites "src/test/resources/tests1.xml"
suites "src/test/resources/tests2.xml"
}
}
How can I run for example only tests1.xml from command line?
you can use project properties to add/change/... different suites. In you example you are probably running
gradlew test
which run both suites. If you modify test task in your build.gradle
def suite1 = project.hasProperty("suite1")
def suite2 = project.hasProperty("suite2")
test {
useTestNG() {
dependsOn cleanTest
useDefaultListeners = true
if(suite1) {
suites "src/test/resources/simpleSuite.xml"
}
if(suite2) {
suites "src/test/resources/advancedSuite.xml"
}
}
}
you can choose suite in this way
gradlew test -Psuite1
gradlew test -Psuite2
gradlew test -Psuite1 -Psuite2
you can specify variable let say suiteFile with default value and use it in testNG section. For example:
ext{
set suiteFile, default is 'testrun_config.xml'
if (!project.hasProperty('suiteFile')) {
suiteFile = 'testrun_config.xml'
}
}
test {
useTestNG() {
dependsOn cleanTest
useDefaultListeners = true
suites "src/test/resources/"+suiteFile
}
}
Refer qaf gradle build file
If you want to pass through command line
gradlew test -PsuiteFile=test.xml
In build.gradle update:
test {
useTestNG() {
if (project.hasProperty('suite1')) { suites './src/test/suite1.xml' }
if (project.hasProperty('suite2')) { suites './src/test/suite2.xml' }
}
}
Use command gradlew test -Psuite1 to run suite1 and similarly update for suite2 as well.

JUnit5 tag-specific gradle task

I use the following annotation to tag my integration tests:
#Target({ ElementType.TYPE, ElementType.METHOD })
#Retention(RetentionPolicy.RUNTIME)
#Tag("integration-test")
public #interface IntegrationTest {
}
This is the filter I use in build.gradle to exclude these tests from gradle build:
junitPlatform {
filters {
tags {
exclude 'integration-test'
}
}
}
So far, so good.
Now I would like to offer a Gradle task which specifically runs my integration tests – what's the recommended approach?
Based on https://github.com/gradle/gradle/issues/6172#issuecomment-409883128
Amended in 2020 to take lazy task configuration and Gradle 5 into account. See answer's history for older versions.
plugins {
id "java"
}
def test = tasks.named("test") {
useJUnitPlatform {
excludeTags "integration"
}
}
def integrationTest = tasks.register("integrationTest2", Test) {
useJUnitPlatform {
includeTags "integration"
}
shouldRunAfter test
}
tasks.named("check") {
dependsOn integrationTest
}
Running
gradlew test will run tests without integration
gradlew integrationTest will run only integration test
gradlew check will run test followed by integrationTest
gradlew integrationTest test will run test followed by integrationTest
note: order is swapped because of shouldRunAfter
History
Gradle 4.6+ supports JUnit 5 natively
JUnit 5 deprecated their plugin: https://github.com/junit-team/junit5/issues/1317
JUnit 5 deleted plugin: 'org.junit.platform.gradle.plugin'
JUnit 5 closed junit5#579 (same as OP's question) as won't-fix (due to decommissioning their plugin)
Gradle supports the above feature: https://github.com/gradle/gradle/issues/6172
Tip
Note: while the above works, IntelliJ IDEA has a hard time inferring stuff, so I suggest to use this more explicit version where everything is typed and code completion is fully supported:
... { Test task ->
task.useJUnitPlatform { org.gradle.api.tasks.testing.junitplatform.JUnitPlatformOptions options ->
options.includeTags 'integration'
}
}
build.gradle.kts
Root project Kotlin DSL drop-in for configuring integration tests in all modules in Gradle 5.6.4
allprojects {
plugins.withId("java") {
#Suppress("UnstableApiUsage")
this#allprojects.tasks {
val test = "test"(Test::class) {
useJUnitPlatform {
excludeTags("integration")
}
}
val integrationTest = register<Test>("integrationTest") {
useJUnitPlatform {
includeTags("integration")
}
shouldRunAfter(test)
}
"check" {
dependsOn(integrationTest)
}
}
}
}
I filed an issue: https://github.com/junit-team/junit5/issues/579 (as suggested by Sam Brannen).
Meanwhile, I am using a project property as a workaround:
junitPlatform {
filters {
tags {
exclude project.hasProperty('runIntegrationTests') ? '' : 'integration-test'
}
}
}
Consequently, integrations tests will be skipped with:
gradle test
but will be included with:
gradle test -PrunIntegrationTests
Gradle 6
I am not sure if it is because Gradle behavior has changed, but the highest voted answer did not work for me in Gradle. 6.8.3. I was seeing the integrationTests task run along with the main test task. This simplified version worked for me:
test {
useJUnitPlatform {
excludeTags "integration"
}
}
tasks.register("integrationTests", Test) {
useJUnitPlatform {
includeTags "integration"
}
mustRunAfter check
}
Commands:
./gradlew test or ./gradlew clean build - Runs tests without
'integration' tag.
./gradlew integrationTests - Only runs test with
'integration' tag.
According to me, the best, current working code to solve this, is the one presented by: TWiStErRob found here.
Note that tests must be tagged with ui in the example below (Junit5 tagging):
task uiTest(type: Test) {
useJUnitPlatform {
includeTags 'ui'
excludeTags 'integration'
}
}
Howe ever, I did not managed to get the Junit5 test-suit-thing to be run from gradle directly witch I would think would be an even nicer solution. But I think the solution buy TWiStErRob, is good enough. The down side is that the gradle.build file now also will be bloated with test-suit-things.
Please note that it is fine to create multiple test suites in the gradle file like this:
task firstTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-first-test-suite'
}
}
task secondTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-second-test-suite'
}
}
Then then all could be run separately like this:
gradlew firstTestSuite
gradlew secondTestSuite
gradlew ui
Solution run with Gradle 6.6.1
A similar approach to Rahel Lüthy avoiding the usage of empty strings, in this case to run all tests or just some tags:
test {
useJUnitPlatform() {
if (project.hasProperty("includes")) {
includeTags(project.property("includes") as String)
}
}
}

Resources