I am using the new JVM Test Suite plugin to create a regression test suite (including a dedicated source set and dedicated dependencies).
testing {
suites {
regressionTest(JvmTestSuite) {
dependencies {
implementation project
implementation 'org.assertj:assertj-core:3.22.0'
}
targets {
all {
testTask.configure {
sourceCompatibility = '17'
targetCompatibility = '17'
useJUnitPlatform {
includeTags 'check'
}
}
}
}
}
}
}
This generates a 'regressionTest' task that runs the tests on my new source set. So far so good.
Now I need to be able to run that task with a different includeTags for JUnit, but I can't find a good way to do so.
I found https://github.com/gradle/gradle/issues/6172 and JUnit5 tag-specific gradle task, but those only work for the duplicating the standard test task. If I do so, it would not run with the correct dependencies and the correct source set.
What's a good way to create a new test task with specific JUnit5 tags when using the JVM Test Suite plugin ?
Related
I've created a new test suite called integrationTest using the jvm-test-suite plugin.
Now I want to generate a jacoco report just for the integration tests.
I create a new gradle task like this:
tasks.create<JacocoReport>("jacocoIntegrationTestReport") {
group = "verification"
description = "Generates code coverage report for the integrationTest task."
executionData.setFrom(fileTree(buildDir).include("/jacoco/integrationTest.exec"))
reports {
xml.required.set(true)
html.required.set(true)
}
}
But the generated HTML/XML report is empty. I have run the integration tests before executing the task and the file integrationTest.exec exists.
Thanks
It seems the important part of the new JaCoCo report task configuration is to wire the execution data via the integrationTest task instead of the exec-file path. The official docs (see last example here) also imply that the source set must be wired as well.
Here is the full build script (Gradle 7.6) that produces a report with command:
./gradlew :app:integrationTest :app:jacocoIntegrationTestReport
// build.gradle.kts
plugins {
application
jacoco
}
repositories {
mavenCentral()
}
testing {
suites {
val integrationTest by creating(JvmTestSuite::class) {
dependencies {
implementation(project(":app"))
}
}
}
}
tasks.register<JacocoReport>("jacocoIntegrationTestReport") {
executionData(tasks.named("integrationTest").get())
sourceSets(sourceSets["integrationTest"])
reports {
xml.required.set(true)
html.required.set(true)
}
}
I'm trying to get a hierarchical test report from Kotlintest using Gradle. I've seen some screenshots allowing it, however, I have no luck. For any type of tests (FunSpec, WordSpec, BehaviorSpec etc) I always see only the class name and then the "leaf" tests.
Gradle 5.6.2
Kotlintest 3.4.2
JUnit Platform 1.5.2
Sample test class
import io.kotlintest.matchers.string.shouldStartWith
import io.kotlintest.specs.FunSpec
class HierarchicalTest : FunSpec({
context("Here is a context 1") {
test("Test 1") {
"abc".shouldStartWith("a")
}
}
context("Here is a context 2") {
test("Test 2") {
"abc".shouldStartWith("b")
}
}
})
build.gradle
plugins {
id 'org.jetbrains.kotlin.jvm' version '1.3.41'
}
sourceCompatibility = 1.8
repositories {
jcenter()
}
dependencies {
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8"
testImplementation 'ch.qos.logback:logback-classic:1.2.3'
testImplementation 'io.kotlintest:kotlintest-runner-junit5:3.4.2'
testImplementation 'org.junit.platform:junit-platform-engine:1.5.2'
}
test {
useJUnitPlatform()
}
IntelliJ result
Gradle report
What do I need to do to have the context level visible in the report?
I'm on Idea 2018.3.2 and I recreated project with your sources and for me it works. I guess this behavious correlates with the way you run tests. Because I ran tests via Idea and it's green arrows for tests.
I guess you run tests via gradle tasks? If so, Idea can show results differently
This is how it looks like when I run gradle test task via gradle plugin in IDEA
I'm using Geb/Spock for automated testing. I'm using Gradle as my build tool.
I'd like to call different gradle tasks to build and run a specific spec(test) or a suite of specs.
I dont know enough about the gradle build lifecycle to completely understand what is going on here: https://github.com/geb/geb-example-gradle/blob/master/build.gradle
plugins {
id "idea"
id "groovy"
id "com.energizedwork.webdriver-binaries" version "1.4"
id "com.energizedwork.idea-base" version "1.4"
}
ext {
// The drivers we want to use
drivers = ["firefox", "chrome", "chromeHeadless"]
ext {
groovyVersion = '2.4.12'
gebVersion = '2.2'
seleniumVersion = '3.6.0'
chromeDriverVersion = '2.32'
geckoDriverVersion = '0.18.0'
}
}
repositories {
mavenCentral()
}
dependencies {
// If using Spock, need to depend on geb-spock
testCompile "org.gebish:geb-spock:$gebVersion"
testCompile("org.spockframework:spock-core:1.1-groovy-2.4") {
exclude group: "org.codehaus.groovy"
}
testCompile "org.codehaus.groovy:groovy-all:$groovyVersion"
// If using JUnit, need to depend on geb-junit (3 or 4)
testCompile "org.gebish:geb-junit4:$gebVersion"
// Drivers
testCompile "org.seleniumhq.selenium:selenium-chrome-driver:$seleniumVersion"
testCompile "org.seleniumhq.selenium:selenium-firefox-driver:$seleniumVersion"
}
webdriverBinaries {
chromedriver chromeDriverVersion
geckodriver geckoDriverVersion
}
drivers.each { driver ->
task "${driver}Test"(type: Test) {
group JavaBasePlugin.VERIFICATION_GROUP
outputs.upToDateWhen { false } // Always run tests
systemProperty "geb.build.reportsDir", reporting.file("geb/$name")
systemProperty "geb.env", driver
}
}
test {
dependsOn drivers.collect { tasks["${it}Test"] }
enabled = false
}
tasks.withType(Test) {
maxHeapSize = "1g"
jvmArgs '-XX:MaxMetaspaceSize=128m'
testLogging {
exceptionFormat = 'full'
}
}
tasks.withType(GroovyCompile) {
groovyOptions.forkOptions.memoryMaximumSize = '256m'
}
I've tried inserting the following into build.gradle:
task dataGen {
include '**com.company.project.spec.util/DataGenerationUtilSpec.groovy'
}
task sanity {
include '**com.company.project.spec.sanity.*'
}
But calling these tasks (gradle sanity) results in a build failure:
Could not find method include() for arguments [**com.company.project.spec.util/DataGenerationUtilSpec.groovy] on task ':dataGen' of type org.gradle.api.DefaultTask
Obviously there's existing build instructions since I can call gradle build and all the specs run on Chrome, I'm just not sure how to add more tasks
I think these 2 tasks are test tasks so it should look like that :
task dataGen (type: Test) {
include '**com.company.project.spec.util/DataGenerationUtilSpec.groovy'
}
task sanity (type: Test) {
include '**com.company.project.spec.sanity.*'
}
You can use Spock annotation to control the test or the Spec, see example here.
You will have to define annotation classes and define the Spock config file to use that annotation. You then annotate the specific Specification (or test).
Now you will have to define the Spock config file in the task or from a parameter.
I created a new Groovy project of integration tests and want to be able to be able to group tests together for different components. For example, some tests are only valid for a docker image, while others are for testing a webapp running locally. I'd also like to get reports for the results of the tests and also be able to run the tests over and over if needed.
The first step to grouping tests in JUnit is to create an interface to name the group. Create this in a package your test classes can reach.
If your Test Class looks like this:
package com.example.test.api.get.cert
class ByIssuedBeforeTest {
//awesome tests
}
Then create an empty interface like this:
package com.example.test
interface DockerTest {}
Now change your Test Class to look like this:
package com.example.test.api.get.cert
import org.junit.experimental.categories.Category
#Category(DockerTest.class)
class ByIssuedBeforeTest {
//awesome tests
}
You can also annotate individual tests instead of the whole class.
Now in your build.gradle file:
apply plugin: 'java'
sourceSets {
main {
java {
srcDirs = ["src/main/java", "src/main/groovy"]
}
}
test {
java {
srcDirs = ["src/test/java", "src/test/groovy"]
}
}
}
test {
outputs.upToDateWhen { false }
reports {
junitXml.enabled=true
html.enabled=true
}
}
task nondockerTest(type: Test) {
outputs.upToDateWhen { false }
useJUnit {
excludeCategories 'com.example.test.DockerTest'
}
}
task dockerTest(type: Test) {
outputs.upToDateWhen { false }
useJUnit {
includeCategories 'com.example.test.DockerTest'
}
}
The java plugin for gradle gives you the test task. In the test task we add the outputs line in order to keep the test task from ever being labeled as "UP-TO-DATE", in essence forcing gradle to always run the task when we call it instead of caching the result.
The report block enables JUnit report files to be created. They will be placed in the build/reports/tests directory. If you ran the nonDockerTest it would be build/reports/tests/nondockerTest/index.html.
The useJUnit block tells Gradle to use JUnit to run the tests as well as which category of tests to run (if using includeCategories) or category to not run (if using excludeCategories).
gradle test to run all the test.
gradle nondockerTest to run the tests not labeled as with the DockerTest category.
gradle dockerTest to run just tests labeled with the DockerTest category.
Each of these tasks will create reports in the /build/reports/tests directory.
I use the following annotation to tag my integration tests:
#Target({ ElementType.TYPE, ElementType.METHOD })
#Retention(RetentionPolicy.RUNTIME)
#Tag("integration-test")
public #interface IntegrationTest {
}
This is the filter I use in build.gradle to exclude these tests from gradle build:
junitPlatform {
filters {
tags {
exclude 'integration-test'
}
}
}
So far, so good.
Now I would like to offer a Gradle task which specifically runs my integration tests – what's the recommended approach?
Based on https://github.com/gradle/gradle/issues/6172#issuecomment-409883128
Amended in 2020 to take lazy task configuration and Gradle 5 into account. See answer's history for older versions.
plugins {
id "java"
}
def test = tasks.named("test") {
useJUnitPlatform {
excludeTags "integration"
}
}
def integrationTest = tasks.register("integrationTest2", Test) {
useJUnitPlatform {
includeTags "integration"
}
shouldRunAfter test
}
tasks.named("check") {
dependsOn integrationTest
}
Running
gradlew test will run tests without integration
gradlew integrationTest will run only integration test
gradlew check will run test followed by integrationTest
gradlew integrationTest test will run test followed by integrationTest
note: order is swapped because of shouldRunAfter
History
Gradle 4.6+ supports JUnit 5 natively
JUnit 5 deprecated their plugin: https://github.com/junit-team/junit5/issues/1317
JUnit 5 deleted plugin: 'org.junit.platform.gradle.plugin'
JUnit 5 closed junit5#579 (same as OP's question) as won't-fix (due to decommissioning their plugin)
Gradle supports the above feature: https://github.com/gradle/gradle/issues/6172
Tip
Note: while the above works, IntelliJ IDEA has a hard time inferring stuff, so I suggest to use this more explicit version where everything is typed and code completion is fully supported:
... { Test task ->
task.useJUnitPlatform { org.gradle.api.tasks.testing.junitplatform.JUnitPlatformOptions options ->
options.includeTags 'integration'
}
}
build.gradle.kts
Root project Kotlin DSL drop-in for configuring integration tests in all modules in Gradle 5.6.4
allprojects {
plugins.withId("java") {
#Suppress("UnstableApiUsage")
this#allprojects.tasks {
val test = "test"(Test::class) {
useJUnitPlatform {
excludeTags("integration")
}
}
val integrationTest = register<Test>("integrationTest") {
useJUnitPlatform {
includeTags("integration")
}
shouldRunAfter(test)
}
"check" {
dependsOn(integrationTest)
}
}
}
}
I filed an issue: https://github.com/junit-team/junit5/issues/579 (as suggested by Sam Brannen).
Meanwhile, I am using a project property as a workaround:
junitPlatform {
filters {
tags {
exclude project.hasProperty('runIntegrationTests') ? '' : 'integration-test'
}
}
}
Consequently, integrations tests will be skipped with:
gradle test
but will be included with:
gradle test -PrunIntegrationTests
Gradle 6
I am not sure if it is because Gradle behavior has changed, but the highest voted answer did not work for me in Gradle. 6.8.3. I was seeing the integrationTests task run along with the main test task. This simplified version worked for me:
test {
useJUnitPlatform {
excludeTags "integration"
}
}
tasks.register("integrationTests", Test) {
useJUnitPlatform {
includeTags "integration"
}
mustRunAfter check
}
Commands:
./gradlew test or ./gradlew clean build - Runs tests without
'integration' tag.
./gradlew integrationTests - Only runs test with
'integration' tag.
According to me, the best, current working code to solve this, is the one presented by: TWiStErRob found here.
Note that tests must be tagged with ui in the example below (Junit5 tagging):
task uiTest(type: Test) {
useJUnitPlatform {
includeTags 'ui'
excludeTags 'integration'
}
}
Howe ever, I did not managed to get the Junit5 test-suit-thing to be run from gradle directly witch I would think would be an even nicer solution. But I think the solution buy TWiStErRob, is good enough. The down side is that the gradle.build file now also will be bloated with test-suit-things.
Please note that it is fine to create multiple test suites in the gradle file like this:
task firstTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-first-test-suite'
}
}
task secondTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-second-test-suite'
}
}
Then then all could be run separately like this:
gradlew firstTestSuite
gradlew secondTestSuite
gradlew ui
Solution run with Gradle 6.6.1
A similar approach to Rahel Lüthy avoiding the usage of empty strings, in this case to run all tests or just some tags:
test {
useJUnitPlatform() {
if (project.hasProperty("includes")) {
includeTags(project.property("includes") as String)
}
}
}