Gradle. Running tests in parallel from one Class - gradle

Using Java 11, Gradle 6.5.1
In the build.gradle there is an option:
maxParallelForks = 8
I have two classes, first has 2 tests and second has 30 tests.
When I run tests using Gradle it runs only in 2 threads:
./gradlew clean test
One thread - tests for the first class, second thread - tests for the second class.
But, how to make Gradle to execute all tests in parallel?
So tests from the same class could be run in parallel in 8 threads.

If you're using JUnit 5, you can do this using JUnit's parallel execution instead of Gradle's. Note that this will be within the same JVM (I believe Gradle's maxParallelForks uses separate JVMs).
E.g. in your Gradle script:
test {
...... (your existing stuff here)
systemProperties = [
'junit.jupiter.execution.parallel.enabled': 'true'
'junit.jupiter.execution.parallel.mode.default': 'concurrent'
]
}
Different test frameworks may have equivalents.
Other than that, a workaround might be to split your tests into a separate class per test case. You can use things like test class inheritance to avoid duplication with this.
If you have more than the 2 test classes you've mentioned here, you'd only need to split up the few longest running classes to get most of the benefit.

Related

Gradle how to exclude 1 test file in a multi module project setup?

So we have a multi module project setup, with test all scattered in multiple modules, now we want to execute all of them but exclude 1 test file.
How could we achieve this?
I tried the following:
gradle test -PexcludeTests=*SpecificTests
but the tests get still executed.
for running a singular test I managed to fix it this way:
gradle :multi-module:test --tests '*SpecificTests'
but unfortunately the equivalent for executing all tests but 1 cannot be made with this.
Condition: we need a command we cannot use the testing filter
You can use a project property and define an optional exclude filter, which you can then use from the command line -PexcludeTests='*SpecificTests'.
test {
if (project.hasProperty('excludeTests')) {
exclude(project.property('excludeTests'))
}
}

how to create a gradle test suite without using built-in frameworks

Lets say I created a new and unique Java test framework, or I am retro-fitting a legacy project where the tests are just main(String[] args)-entrypoint classes and I would like to run those with gradle test and have gradle exit with 0 if none of the tests fail/nothing throws.
"without built-in frameworks" - meaning, without useJUnit(), useJUnitPlatform() et al.

How to run a single test that is excluded

I have integration tests named with "IT" at the end and the tests are excluded by default. My build.gradle contains:
test {
if (!project.hasProperty('runITs')) {
exclude '**/**IT.class'
}
}
The check task, which is part of build, no longer runs integration tests. All tests (unit + integration) are executed with defined runITs (e.g. ./gradlew -PrunITs=1 check) if it is necessary.
It works very well. The only drawback is than I can't run single integration test with --test (used by IDE) without runITs defined. The command fails with message: No tests found for given includes: [**/**IT.class](exclude rules).
Is there any way (e.g. build variable) how to recognize the run of single test with --test and skip the exclusion?
I have found a solution. My build.gradle now contains:
test {
if (!project.hasProperty('runITs') && filter.commandLineIncludePatterns.empty) {
exclude '**/**IT.class'
}
}
When tests are run with --tests then the list commandLineIncludePatterns in filter is not empty and exclude is not applied.
Details are obvious in test task source: https://github.com/gradle/gradle/blob/dc8545eb1caf7ea99b48604dcd7b4693e79b6254/subprojects/testing-base/src/main/java/org/gradle/api/tasks/testing/AbstractTestTask.java
try invoking the test task to the specific sub-project. if the test is root try
gradle -PrunITs=1 :test --tests 'MServiceTest'
else
gradle -PrunITs=1 :sub-prj:test --tests 'MServiceTest'

Exclude file from Gradle test dependency

I have rather expensive tests in my gradle java project, so I would like to avoid running them too often. Unfortunately, gradle reruns the tests on every build, since some log file in the resource-folder is changing.
Is there any way to exclude log-files from the dependency checks of :processTestResources and :test? I tried to include a exclude command in my test task, but this doesn't seem to do anything. My test task is
test {
maxHeapSize = "2048m"
workingDir = "src/test/resources/test-instance"
environment "LD_LIBRARY_PATH", "xpressmp/lib:/opt/gurobi/linux64/lib"
environment "XPRESS", "xpressmp/bin"
environment "XPRESSDIR", "xpressmp"
exclude("*.log")
exclude("*.lp")
}
I think what you are after is
sourceSets {
test {
resources {
exclude '*.log'
}
}
}
Excluding in the task would only exclude the test class from running, not which files are considered input for the task.
Btw. you can also use JUnit Categories to separate your tests into Short-Running and Long-Running tests and then make different tasks or a project property to only run the fast tests or all tests or only the slow tests. Or you can split the tests in different sourcesets and make separate tasks for it.
Define a new Test task:
task notGenericNotFT( type: Test, dependsOn: testClasses ){
filter { excludeTestsMatching 'generic.*' }
// excludes a whole package, "generic". NB this is not a regex:
// '*' is simply "wildcard" and dot means dot ... other more
// sophisticated "ANT-style" patterns are available in class Test
filter { excludeTestsMatching '*_FT' }
// also exclude all test classes ending in "_FT" (e.g. for "functional test")
}
To understand where these things come from, examine class Test and class TestFilter.
Also be aware that the gradle command line parser is quite intelligent and permissive with case-sensitivity (even in Linux!), so you can do this:
...$ ./gradlew notgen
... and it will run (as long as "notgen" designates this task unambiguously).

Why do my Gradle tests run repeatedly?

I have a pretty standard Gradle build that's building a Java project.
When I run it for the first time, it compiles everything and runs the tests. When I run it a second time without changing any files, it runs the tests again.
According to this thread, Gradle is supposed to be lazy by defaut and not bother running tests if nothing has changed. Has the default behaviour here been changed?
EDIT:
If I run gradle test repeatedly, the tests only run the first time and are subsequently skipped. However, if I run gradle build repeatedly, the tests get re-run every time, even though all other tasks are marked as up-to-date.
the gradle uptodate check logs on info level why a task is not considered to be up-to-date. please rerun the "gradle build -i" to run with info logging at check the logging output.
cheers,
René
OK, so I got the answer thanks to Rene prompting me to look at the '-i' output...
I actually have 2 test tasks: the 'test' one from the Java plugin, and my own 'integrationTest' one. I didn't mention this in the question because I didn't think it was relevant.
It turns out that these tasks are writing their output (reports, etc.) to the same directory, so Gradle's task-based input and output tracking was thinking that something had changed, and re-running the tests.
So the next question (which I will ask separately) becomes: how do I cleanly (and with minimal Groovy/Gradle code) completely separate two instances of the test task.
You need to create test tasks in your build.gradle and then call those specific tasks to run a specific set of tests. Here is an example that will filter out classes so that they don't get run twice (such as when running a suite and then re-running its child classes independently):
tasks.withType(Test) {
jvmArgs '-Xms128m', '-Xmx1024m', '-XX:MaxPermSize=128m'
maxParallelForks = 4 // this runs tests parallel if more than one class
testLogging {
exceptionFormat "full"
events "started", "passed", "skipped", "failed", "standardOut", "standardError"
displayGranularity = 0
}
}
task runAllTests(type: Test) {
include '**/AllTests.class'
testReportDir = file("${reporting.baseDir}/AllTests")
testResultsDir = file("${buildDir}/test-results/AllTests")
}
task runSkipSuite(type: Test) {
include '**/Test*.class'
testReportDir = file("${reporting.baseDir}/Tests")
testResultsDir = file("${buildDir}/test-results/Tests")
}
Also, concerning your build question. The "build" task includes a clean step which is cleaning tests from your build directory. Otherwise the execution thinks the tests have already been ran.

Resources