I got a Gradle task which calls two methods. The methods are independent of each other so I want to start them parallel. What is the best way to achieve this?
Example:
task cucumber(dependsOn: 'testClasses') {
doLast {
// ...
// I want to call the next 2 methods in parallel
runSequentialTests(testEnvironment, tags)
runParallelTests(testEnvironment, tags)
}
}
def runSequentialTests(testEnvironment, tags) {
// execute cucumber tests via javaexec
}
def runParallelTests(testEnvironment, tags) {
// execute cucumber tests via exec sh script for each feature file (parallelism is done via GParsPool.withPool(5) {...}
}
Worth researching groovy parallel systems library gpars
GParsPool.withPool {
GParsPool.executeAsyncAndWait({runSequentialTests(testEnvironment, tags)}, {runParallelTests(testEnvironment, tags)})
}
Or maybe create a #ParallelizableTask
But I'm not sure what version of gradle you are using. And parallel is/was an incubating feature.
And needs to run build with --parallel if those have no dependencies between themselves Gradle should run them independently.
Alternatively, you can specify a property in your gradle.properties
C:\Users\<user>\.gradle\gradle.properties
org.gradle.parallel=true
I used the asynchronous invocations of GPars: http://www.gpars.org/webapp/guide/#_asynchronous_invocations
I got it to work with this. Thanks
GParsPool.withPool {
GParsPool.executeAsyncAndWait({runSequentialTests(testEnvironment, tags)}, {runParallelTests(testEnvironment, tags)})
}
Related
There is task which can be executed with parameter like this:
./gradlew taskX -Pkey=value
And plugin with custom task which should execute taskX:
class CustomPlugin : Plugin<Project> {
override fun apply(project: Project) {
project.tasks.register("custom", CustomTask::class.java)
.configure {
it.description = "Description"
it.group = "Group"
val taskX = project.getTasksByName("taskX", true).first()
it.dependsOn(taskX)
}
}
}
I would expect something like this for example:
it.dependsOn(taskX, "key=value")
How to pass parameters to dependsOn?
Simple answer: You can't. Task dependencies only express what needs to be done beforehand, not how it needs to be done.
Let me show you a simple example, why something like this is not possible in the Gradle task system:
First, we need to know that in Gradle, every task will be executed only once in a single invocation (often called build). Now imagine a task that needs to be run before two tasks that are unrelated to each other. A good real world example is the task compileJava from the Java plugin that both the test task and the jar task depend on. If dependsOn would support parameters, it could happen that two tasks depend on a single task with different parameters. What parameters should be used in this case?
As a solution, you may configure the other task directly in your plugin. If you want to pass the parameter only if your custom task is run, you may need to add another task that runs as a setup and applies the required configuration to the actual task:
task setup {
doFirst {
// apply configuration
}
}
taskX.mustRunAfter setup
task custom {
dependsOn setup
dependsOn taskX
}
This example uses Groovy, but it should be possible to translate it to Kotlin and use it in your plugin.
Edit regarding actual parameter
To be honest, I am not that familiar with the Android Gradle plugin, but if I get this documentation right, the project property android.testInstrumentationRunnerArguments.annotation is just an alternative to using the following code in the build script:
android {
defaultConfig {
testInstrumentationRunnerArgument 'annotation', '<some-value>'
}
}
You may try to define the following task and then run it using ./gradlew customTest
task customTest {
doFirst {
android.defaultConfig.testInstrumentationRunnerArgument 'annotation', '<some-value>'
}
finalizedBy 'connectedAndroidTest'
}
I created Spring Boot project which uses gradle build system. I want to run one separate test class by custom gradle task to be able depend on it in other tasks. Now I can do it with this code:
import org.apache.tools.ant.taskdefs.condition.Os
def gradleWrapper = Os.isFamily(Os.FAMILY_WINDOWS) ? 'gradlew.bat' : './gradlew'
task runMyTest(type: Exec) {
workingDir "$rootDir"
commandLine gradleWrapper, ':test', '--tests', 'com.example.MyTest'
}
Obviously, this is not a very beautiful solution, because it launches an additional Gradle daemon. I tried before another solution:
task runMyTest(type: Test, dependsOn: testClasses) {
include 'com.example.MyTest'
}
But it is not working (do not execute my test class).
UPD: I tried yet another solution:
task runMyTest(type: Test) {
filter {
includeTestsMatching "com.example.MyTest"
}
}
It fails with this error message:
Execution failed for task ':runMyTest'.
> No tests found for given includes: [com.example.MyTest](filter.includeTestsMatching)
However, obviously, my test exists, since running the test through the command line produces the correct result.
UPD2: I missed useJUnitPlatform() inside my test task. It was in the default test task (written to my build.gradle by Spring Boot initializer), but not in the custom task.
You can do it using a TestFilter.
Using includeTestsMatching you can specify your class.
If you need to specify a single test method, you can use includeTest "com.example.MyTest", "someTestMethod".
task runMyTest(type: Test) {
useJUnitPlatform()
filter {
includeTestsMatching "com.example.MyTest"
}
}
I added a task to my gradle project:
task deploy() {
dependsOn "build"
// excludeTask "test" <-- something like this
doFirst {
// ...
}
}
Now the build task always runs before the deploy task. This is fine because the build task has many steps included. Now I want to explicitly disable one of these included tasks.
Usually I disable it from command line with
gradle deploy -x test
How can I exclude the test task programmatically?
You need to configure tasks graph rather than configure the deploy task itself. Here's the piece of code you need:
gradle.taskGraph.whenReady { graph ->
if (graph.hasTask(deploy)) {
test.enabled = false
}
}
WARNING: this will skip the actions defined by the test task, it will NOT skip tasks that test depends on. Thus this is not the same behavior as passing -x test on the command line
I don't know what your deploy task does, but it probably just shouldn't depend on the 'build' task. The 'build' task is a very coarse grained lifecycle task that includes tons of stuff you probably don't want.
Instead it should correctly define its inputs (probably the artifacts that you wanna deploy) and then Gradle will only run the necessary tasks to build those inputs. Then you no longer need any excludes.
I ran into a similar problem. Here is how I prevent "test" from running when I run "intTest" and want the ITs to run alone:
test {
onlyIf { !gradle.startParameter.taskNames.contains("intTest") }
}
An alternative that doesn't depend on certain tasks being run explicitly:
test {
onlyIf { !gradle.taskGraph.hasTask(":intTest") || gradle.taskGraph.hasTask(":check") }
}
I have a task which essentially performs the following:
['subproj1', 'subproj2'].each { proj ->
GradleRunner.create()
.withProjectDir(file("./examples/${proj}/"))
.withArguments('check')
.build()
}
The check is a system test and requires connecting to 3rd party services, so I would like to parallelize this.
Can this be done in gradle? If so, how?
I tried using java threading but the builds failed with errors which I can't remember what they were exactly, but they suggested that the gradle internal state had gotten corrupted.
Did you tried to use the experimental parallel task execution? On the first glance it's quite simple. You just call ./gradlew --parallel check (or if it turns out to work fine for you can also define this in your gradle.properties). This will start n threads (where n is the number cpu cores) which will execute your tasks. Each thread owns a certain project so the tasks of one project will never be executed in parallel.
You can override the number of tasks (or worker) by setting the property --max-workers at the command line or by setting org.gradle.workers.max=n at your gradle.properies.
If you are just interested in executing tests in parallel than you might try to set Test.setMaxParallelForks(int). That will cause the to execute the tests of one project (if I understood this right) in parallel (with the number of tasks you defined).
Hope that helps. Maybe the gradle documentation gives you some more insights: https://docs.gradle.org/current/userguide/multi_project_builds.html#sec:parallel_execution
While multiproject build is the right way to go in the long term, my short term approach was to use GPars:
import groovyx.gpars.GParsPool
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath "org.codehaus.gpars:gpars:1.1.0"
}
}
task XXXX << {
GParsPool.withPool(10) {
['subproj1', 'subproj2'].eachParallel { proj ->
GradleRunner.create()
.withProjectDir(file("./examples/${proj}/"))
.withArguments('check')
.build()
}
}
}
I know that I can use the -x test option to prevent the test task from getting called. I also have something like this in my gradle script to prevent tests from being executed in certain cases:
plugins.withType(JavaPlugin).whenPluginAdded {
test {
doFirst {
if (env.equals('prod')) {
throw new StopExecutionException("DON'T RUN TESTS IN PROD!!!!")
}
}
}
}
but is there a way to configure the java plugin to removed the dependency between build -> test?
build depends on test via check. You probably don't want to remove the dependency on check as it may do other things, so you could try:
check.dependsOn.remove(test)
Do you mind if I ask why you want to do this?
You can skip tasks via the command line with the -x option:
./gradlew assembleDebug -x taskToSkip
I don't know if it is possible to remove such a dependency.
You can however skip the execution of tasks, eg: skipping all test tasks (in production) goes like this.
tasks.withType(Test).each { task ->
task.enabled = !env.equals('prod')
}