parallelize code in a gradle task - gradle

Question

I have a task which essentially performs the following:
['subproj1', 'subproj2'].each { proj ->
GradleRunner.create()
.withProjectDir(file("./examples/${proj}/"))
.withArguments('check')
.build()
}
The check is a system test and requires connecting to 3rd party services, so I would like to parallelize this.
Can this be done in gradle? If so, how?
I tried using java threading but the builds failed with errors which I can't remember what they were exactly, but they suggested that the gradle internal state had gotten corrupted.

Answer 1

Did you tried to use the experimental parallel task execution? On the first glance it's quite simple. You just call ./gradlew --parallel check (or if it turns out to work fine for you can also define this in your gradle.properties). This will start n threads (where n is the number cpu cores) which will execute your tasks. Each thread owns a certain project so the tasks of one project will never be executed in parallel.
You can override the number of tasks (or worker) by setting the property --max-workers at the command line or by setting org.gradle.workers.max=n at your gradle.properies.
If you are just interested in executing tests in parallel than you might try to set Test.setMaxParallelForks(int). That will cause the to execute the tests of one project (if I understood this right) in parallel (with the number of tasks you defined).
Hope that helps. Maybe the gradle documentation gives you some more insights: https://docs.gradle.org/current/userguide/multi_project_builds.html#sec:parallel_execution

Answer 2

While multiproject build is the right way to go in the long term, my short term approach was to use GPars:
import groovyx.gpars.GParsPool
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath "org.codehaus.gpars:gpars:1.1.0"
}
}
task XXXX << {
GParsPool.withPool(10) {
['subproj1', 'subproj2'].eachParallel { proj ->
GradleRunner.create()
.withProjectDir(file("./examples/${proj}/"))
.withArguments('check')
.build()
}
}
}