Gradle integration test suite depending on testImplementation dependencies - gradle

I am trying to migrate to the test suites introduced in Gradle 7.3. What I'd like to do is to add testImplementation dependencies to my integration tests.
testing {
suites {
val test by getting(JvmTestSuite::class) {
useJUnitJupiter()
}
val integrationTest by registering(JvmTestSuite::class) {
dependencies {
implementation(project) // This adds dependencies to the prod code
// What to add to automatically use testImplementation deps?
}
...
}
}
}

You’d probably like to make the integrationTestImplementation configuration extend the testImplementation configuration – just like testImplementation already extends implementation by default. See also the docs on configuration inheritance.
Here’s a self-contained example (tested with Gradle 7.3.2):
plugins {
`java-library`
}
repositories {
mavenCentral()
}
testing {
suites {
val test by getting(JvmTestSuite::class) {
useJUnitJupiter()
dependencies {
implementation("org.assertj:assertj-core:3.21.0")
}
}
val integrationTest by registering(JvmTestSuite::class) {
dependencies {
// TODO add any integration test only dependencies here
}
}
}
}
// Here’s the bit that you’re after:
val integrationTestImplementation by configurations.getting {
extendsFrom(configurations.testImplementation.get())
}
If you run ./gradlew dependencies --configuration integrationTestRuntimeClasspath with the configuration inheritance configured, then you’ll get the following output (abbreviated):
integrationTestRuntimeClasspath - Runtime classpath of source set 'integration test'.
+--- org.junit.jupiter:junit-jupiter:5.7.2
| +--- org.junit:junit-bom:5.7.2
| | …
| +--- org.junit.jupiter:junit-jupiter-api:5.7.2
| | …
| +--- org.junit.jupiter:junit-jupiter-params:5.7.2
| | …
| \--- org.junit.jupiter:junit-jupiter-engine:5.7.2
| …
\--- org.assertj:assertj-core:3.21.0
However, if you run the same command without the configuration inheritance, then you’ll get the following output (abbreviated) – note the missing org.assertj:assertj-core:3.21.0 dependency:
integrationTestRuntimeClasspath - Runtime classpath of source set 'integration test'.
\--- org.junit.jupiter:junit-jupiter:5.7.2
+--- org.junit:junit-bom:5.7.2
| …
+--- org.junit.jupiter:junit-jupiter-api:5.7.2
| …
+--- org.junit.jupiter:junit-jupiter-params:5.7.2
| …
\--- org.junit.jupiter:junit-jupiter-engine:5.7.2
As requested in the answer comments, here’s additionally one way of making a test data class from the unit test suite available for integration testing:
sourceSets.named("integrationTest") {
java {
val sharedTestData = project.objects.sourceDirectorySet("testData",
"Shared test data")
sharedTestData.srcDir("src/test/java")
sharedTestData.include("com/example/MyData.java")
source(sharedTestData)
}
}

I came up with following solution
testing {
suites {
val test by getting(JvmTestSuite::class) {
useJUnitJupiter()
}
val integrationTest by registering(JvmTestSuite::class) {
useJUnitJupiter()
dependencies {
implementation(project)
// add testImplementation dependencies
configurations.testImplementation {
dependencies.forEach(::implementation)
}
}
sources {
java {
//...
}
}
targets {
all {
testTask.configure {
shouldRunAfter(test)
}
}
}
}
}
}

I'm a Gradle Dev, currently working on this incubating feature.
One thing to keep in mind is that while currently the names of the configurations created for each test suite are based on the name of the corresponding sourceSet, which matches the name of the Test Suite itself, these names should be treated as an implementation detail. One of the main goals of Test Suites is to build a useful layer of abstraction over these details, and allow you to configure them without detailed knowledge of this sort of plumbing.
Wiring the configurations together with extendsFrom goes against this intent, as does any sort of retrieval by name, and should be seen as anti-patterns.
Better options include using suites.withType(JvmTestSuite).configureEach { suite -> ... } or creating a Closure containing the configuration you want and using it to configure the test suites of interest like this:
testing {
suites {
def applyMockitoAndJupiter = { suite ->
suite.useJUnitJupiter()
suite.dependencies {
implementation('org.mockito:mockito-junit-jupiter:4.6.1')
}
}
/* This is the equivalent of:
test {
applyMockitoAndJupiter(this)
}
*/
test(applyMockitoAndJupiter)
/* This is the equivalent of:
integrationTest(JvmTestSuite)
applyMockitoAndJupiter(integrationTest)
*/
integrationTest(JvmTestSuite, applyMockitoAndJupiter)
}
}
We're working to expand the functionality offered by Test Suites and each suite's dependencies block, as well as to make the docs more explicit about this point as well. You can see a preview (subject to further change) of this advice in a future version of the Gradle docs - you'll have to log in as guest.

This is what I used for not to rewrite the same dependencies twice for test and customTest tasks. Maybe it is not recommended as per #Tom Tresansky, but I find it quite convenient:
configurations {
customTestImplementation {
extendsFrom testImplementation
}
customTestCompileOnly {
extendsFrom testCompileOnly
}
customTestAnnotationProcessor {
extendsFrom annotationProcessor
}
}

Related

How do I add testFixures as a dependency of a JvmTestSuite with gradle

I'm using java-test-fixtures in combination with jvm-test-suite. I'd like my testFixtures to be available to both unit tests and my integrationTest suite.
java-test-fixtures adds testFixtures as a dependency to the default unit test suite, along with compile-time and runtime transitive dependencies. What's the right way to add this to integrationTest too?
The following works, but it seems a bit repetitive:
plugins {
id 'java'
id 'application'
id 'java-test-fixtures'
id 'jvm-test-suite'
}
testing {
suites {
integrationTest(JvmTestSuite) {
dependencies {
implementation sourceSets.testFixtures.output
}
configurations {
integrationTestCompileClasspath.extendsFrom testFixturesApi
integrationTestRuntimeClasspath.extendsFrom testFixturesRuntimeClasspath
}
}
}
}
I can also use testFixtures(project), but only if I declare the dependency in a top-level dependency block, with the top-level dependency block appearing after the test suite has been declared:
testing {
suites {
integrationTest(JvmTestSuite) {}
}
}
dependencies {
integrationTestImplementation testFixtures(project)
}
This works, with all the transitive dependencies set up correctly.
Curiously, I can't use testFixtures(project) inside the test suite declaration - the following:
testing {
suites {
integrationTest(JvmTestSuite) {
dependencies {
implementation testFixtures(project)
}
}
}
}
...fails to evaluate.
Is there a preferred way to have a test suite depend upon testFixtures?
I hate to answer my own question! The cleanest solution right now appears to be to fully-qualify the testFixtures function inside the test's dependencies block:
integrationTest(JvmTestSuite) {
dependencies {
implementation project.getDependencies().testFixtures(project)
}
}

Using testCompile output from other subproject (Gradle Kotlin DSL)

I have some utility files in the test sources in one of my gradle subproject and would like to use them in an other subproject.
My "source" subproject is called core, while the one uses it is called tem.
I try to migrate and integrate the following example:
In your Server project:
configurations {
testArtifacts.extendsFrom testCompile
}
task testJar(type: Jar) {
classifier "test"
from sourceSets.test.output
}
artifacts {
testArtifacts testJar
}
In your ServerWeb project:
testCompile project(path: ":Server", configuration: 'testArtifacts')
As far as I get is making the conversation. I added the following to my core.gradle.kts:
val testConfig = configurations.create("testArtifacts") {
extendsFrom(configurations["testCompile"])
}
tasks.register("testJar", Jar::class.java) {
classifier += "test"
from(sourceSets["test"].output)
}
artifacts {
add("testArtifacts", tasks.named<Jar>("testJar") )
}
And tried to refer to it in tem.gradle.kts:
testImplementation(project(":core", "testArtifacts"))
It compiles, but I still can't access the classes from core.
Where did I miss something?
Most of your code should be OK
But you must define classesDirs for jar
tasks.register<Jar>("testJar") {
dependsOn("testClasses")
archiveBaseName.set("${project.name}-test")
from(sourceSets["test"].output.classesDirs)
}
I also added depends on testClasses to be sure that classes are compiled.
You can test that jar is OK by executing testJar task. Then verify that generated jar contains your classes. If you make mistake with from method call then you get empty jar.
The following configuration worked for me to include both the test classes and test resources:
core build.gradle.kts
val testConfig = configurations.create("testArtifacts") {
extendsFrom(configurations["testCompile"])
}
tasks.register("testJar", Jar::class.java) {
dependsOn("testClasses")
classifier += "test"
from(sourceSets["test"].output)
}
artifacts {
add("testArtifacts", tasks.named<Jar>("testJar") )
}
tem build.gradle.kts
testImplementation(project(":core", "testArtifacts"))

Running specific tests using gradle over multiple browsers

I'm using Geb/Spock for automated testing. I'm using Gradle as my build tool.
I'd like to call different gradle tasks to build and run a specific spec(test) or a suite of specs.
I dont know enough about the gradle build lifecycle to completely understand what is going on here: https://github.com/geb/geb-example-gradle/blob/master/build.gradle
plugins {
id "idea"
id "groovy"
id "com.energizedwork.webdriver-binaries" version "1.4"
id "com.energizedwork.idea-base" version "1.4"
}
ext {
// The drivers we want to use
drivers = ["firefox", "chrome", "chromeHeadless"]
ext {
groovyVersion = '2.4.12'
gebVersion = '2.2'
seleniumVersion = '3.6.0'
chromeDriverVersion = '2.32'
geckoDriverVersion = '0.18.0'
}
}
repositories {
mavenCentral()
}
dependencies {
// If using Spock, need to depend on geb-spock
testCompile "org.gebish:geb-spock:$gebVersion"
testCompile("org.spockframework:spock-core:1.1-groovy-2.4") {
exclude group: "org.codehaus.groovy"
}
testCompile "org.codehaus.groovy:groovy-all:$groovyVersion"
// If using JUnit, need to depend on geb-junit (3 or 4)
testCompile "org.gebish:geb-junit4:$gebVersion"
// Drivers
testCompile "org.seleniumhq.selenium:selenium-chrome-driver:$seleniumVersion"
testCompile "org.seleniumhq.selenium:selenium-firefox-driver:$seleniumVersion"
}
webdriverBinaries {
chromedriver chromeDriverVersion
geckodriver geckoDriverVersion
}
drivers.each { driver ->
task "${driver}Test"(type: Test) {
group JavaBasePlugin.VERIFICATION_GROUP
outputs.upToDateWhen { false } // Always run tests
systemProperty "geb.build.reportsDir", reporting.file("geb/$name")
systemProperty "geb.env", driver
}
}
test {
dependsOn drivers.collect { tasks["${it}Test"] }
enabled = false
}
tasks.withType(Test) {
maxHeapSize = "1g"
jvmArgs '-XX:MaxMetaspaceSize=128m'
testLogging {
exceptionFormat = 'full'
}
}
tasks.withType(GroovyCompile) {
groovyOptions.forkOptions.memoryMaximumSize = '256m'
}
I've tried inserting the following into build.gradle:
task dataGen {
include '**com.company.project.spec.util/DataGenerationUtilSpec.groovy'
}
task sanity {
include '**com.company.project.spec.sanity.*'
}
But calling these tasks (gradle sanity) results in a build failure:
Could not find method include() for arguments [**com.company.project.spec.util/DataGenerationUtilSpec.groovy] on task ':dataGen' of type org.gradle.api.DefaultTask
Obviously there's existing build instructions since I can call gradle build and all the specs run on Chrome, I'm just not sure how to add more tasks
I think these 2 tasks are test tasks so it should look like that :
task dataGen (type: Test) {
include '**com.company.project.spec.util/DataGenerationUtilSpec.groovy'
}
task sanity (type: Test) {
include '**com.company.project.spec.sanity.*'
}
You can use Spock annotation to control the test or the Spec, see example here.
You will have to define annotation classes and define the Spock config file to use that annotation. You then annotate the specific Specification (or test).
Now you will have to define the Spock config file in the task or from a parameter.

Building a Gradle Kotlin project with Java 9/10 and Gradle's Kotlin DSL

This is kind of a follow up to Building a Kotlin + Java 9 project with Gradle. In the linked post Gradle with Groovy is used. In my case Kotlin DSL is used.
Basically I have a gradle project with the following structure (only relevant content here):
src/
| main/
| | kotlin/
| | | com/example/testproject/
| | | | Main.kt
| | | module-info.java
build.gradle.kts
settings.gradle
Usually I would run gradle run on it, but that results in the following error:
module-info.java:3: error: module not found: kotlin.stdlib
requires kotlin.stdlib;
Now this is what my build file currently looks like
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
buildscript {
var kotlin_version: String by extra
kotlin_version = "1.2.41"
repositories {
mavenCentral()
}
dependencies {
classpath(kotlin("gradle-plugin", kotlin_version))
}
}
repositories {
mavenCentral()
}
plugins {
kotlin("jvm") version "1.2.41"
application
}
val kotlin_version: String by extra
dependencies {
implementation(kotlin("stdlib", kotlin_version))
implementation(kotlin("stdlib-jdk8", kotlin_version))
implementation(kotlin("runtime", kotlin_version))
implementation(kotlin("reflect", kotlin_version))
}
val group = "com.example"
application {
mainClassName = "$group.testproject.Main"
}
java {
sourceCompatibility = JavaVersion.VERSION_1_10
targetCompatibility = sourceCompatibility
sourceSets {
"main" {
java.srcDirs("src/main/kotlin")
}
}
}
tasks.withType<KotlinCompile> {
kotlinOptions.jvmTarget = "1.8"
}
And this is my module-info.java:
module com.example.testproject {
// Kotlin compatibility
requires kotlin.stdlib;
exports com.example.testproject;
}
Question: How to get the solution provided in the linked post (or any other solution) running, so that a Kotlin project with Gradle's Kotlin DSL can be compiled using a Java 9/10 environment?
This is kind of a self-answer (I do not full understand that matter, so the explanations might not be correct). The conclusions I draw here are purely empiric and based on a conversion from Kotlin DSL to Gradle's Groovy and back.
The first problem I encountered was that I had two conflicting providers for the Kotlin functions in:
implementation(kotlin("stdlib", kotlin_version))
implementation(kotlin("runtime", kotlin_version))
I solved that by deciding to go with stdlib. All other dependencies did not conflict with each other.
The more severe problem was something different: The compileJava task did not find the correct classes (from the project) and modules (from the distribution). Therefore I needed to adapt the paths as in the following example:
val compileKotlin: KotlinCompile by tasks
val compileJava: JavaCompile by tasks
compileJava.destinationDir = compileKotlin.destinationDir
This basically compiles the Java classes within Kotlins compiled output and makes Java find the classes from the project.
The last problem could finally be solved by the following non-idiomatic piece of Kotlin Script:
tasks {
"compileJava" {
dependsOn(":compileKotlin")
if (JavaVersion.current() >= JavaVersion.VERSION_1_9) {
inputs.property("moduleName", ext["moduleName"])
doFirst {
compileJava.options.compilerArgs = listOf(
// include Gradle dependencies as modules
"--module-path", java.sourceSets["main"].compileClasspath.asPath,
)
java.sourceSets["main"].compileClasspath = files()
}
}
}
}
This basically lets the compileJava task use an empty classpath and sets module path as compiler option to the currently set compileClasspath of the main source set (the Kotlin source set which is also added as Java source set).

JUnit5 tag-specific gradle task

I use the following annotation to tag my integration tests:
#Target({ ElementType.TYPE, ElementType.METHOD })
#Retention(RetentionPolicy.RUNTIME)
#Tag("integration-test")
public #interface IntegrationTest {
}
This is the filter I use in build.gradle to exclude these tests from gradle build:
junitPlatform {
filters {
tags {
exclude 'integration-test'
}
}
}
So far, so good.
Now I would like to offer a Gradle task which specifically runs my integration tests – what's the recommended approach?
Based on https://github.com/gradle/gradle/issues/6172#issuecomment-409883128
Amended in 2020 to take lazy task configuration and Gradle 5 into account. See answer's history for older versions.
plugins {
id "java"
}
def test = tasks.named("test") {
useJUnitPlatform {
excludeTags "integration"
}
}
def integrationTest = tasks.register("integrationTest2", Test) {
useJUnitPlatform {
includeTags "integration"
}
shouldRunAfter test
}
tasks.named("check") {
dependsOn integrationTest
}
Running
gradlew test will run tests without integration
gradlew integrationTest will run only integration test
gradlew check will run test followed by integrationTest
gradlew integrationTest test will run test followed by integrationTest
note: order is swapped because of shouldRunAfter
History
Gradle 4.6+ supports JUnit 5 natively
JUnit 5 deprecated their plugin: https://github.com/junit-team/junit5/issues/1317
JUnit 5 deleted plugin: 'org.junit.platform.gradle.plugin'
JUnit 5 closed junit5#579 (same as OP's question) as won't-fix (due to decommissioning their plugin)
Gradle supports the above feature: https://github.com/gradle/gradle/issues/6172
Tip
Note: while the above works, IntelliJ IDEA has a hard time inferring stuff, so I suggest to use this more explicit version where everything is typed and code completion is fully supported:
... { Test task ->
task.useJUnitPlatform { org.gradle.api.tasks.testing.junitplatform.JUnitPlatformOptions options ->
options.includeTags 'integration'
}
}
build.gradle.kts
Root project Kotlin DSL drop-in for configuring integration tests in all modules in Gradle 5.6.4
allprojects {
plugins.withId("java") {
#Suppress("UnstableApiUsage")
this#allprojects.tasks {
val test = "test"(Test::class) {
useJUnitPlatform {
excludeTags("integration")
}
}
val integrationTest = register<Test>("integrationTest") {
useJUnitPlatform {
includeTags("integration")
}
shouldRunAfter(test)
}
"check" {
dependsOn(integrationTest)
}
}
}
}
I filed an issue: https://github.com/junit-team/junit5/issues/579 (as suggested by Sam Brannen).
Meanwhile, I am using a project property as a workaround:
junitPlatform {
filters {
tags {
exclude project.hasProperty('runIntegrationTests') ? '' : 'integration-test'
}
}
}
Consequently, integrations tests will be skipped with:
gradle test
but will be included with:
gradle test -PrunIntegrationTests
Gradle 6
I am not sure if it is because Gradle behavior has changed, but the highest voted answer did not work for me in Gradle. 6.8.3. I was seeing the integrationTests task run along with the main test task. This simplified version worked for me:
test {
useJUnitPlatform {
excludeTags "integration"
}
}
tasks.register("integrationTests", Test) {
useJUnitPlatform {
includeTags "integration"
}
mustRunAfter check
}
Commands:
./gradlew test or ./gradlew clean build - Runs tests without
'integration' tag.
./gradlew integrationTests - Only runs test with
'integration' tag.
According to me, the best, current working code to solve this, is the one presented by: TWiStErRob found here.
Note that tests must be tagged with ui in the example below (Junit5 tagging):
task uiTest(type: Test) {
useJUnitPlatform {
includeTags 'ui'
excludeTags 'integration'
}
}
Howe ever, I did not managed to get the Junit5 test-suit-thing to be run from gradle directly witch I would think would be an even nicer solution. But I think the solution buy TWiStErRob, is good enough. The down side is that the gradle.build file now also will be bloated with test-suit-things.
Please note that it is fine to create multiple test suites in the gradle file like this:
task firstTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-first-test-suite'
}
}
task secondTestSuite(type: Test) {
useJUnitPlatform {
includeTags 'test-for-second-test-suite'
}
}
Then then all could be run separately like this:
gradlew firstTestSuite
gradlew secondTestSuite
gradlew ui
Solution run with Gradle 6.6.1
A similar approach to Rahel Lüthy avoiding the usage of empty strings, in this case to run all tests or just some tags:
test {
useJUnitPlatform() {
if (project.hasProperty("includes")) {
includeTags(project.property("includes") as String)
}
}
}

Resources