In Gradle, I need to run the tasks in parallel with dependsOn method
task testdepends(dependsOn: ['test1', 'test2']) {
doLast {
println("Final Task Completed!")
}
}
In the above example I want to run the test1 and test2 in parallel.
Is there a way to achieve this ?
My basic need is - I have to run tasks in parallel. After the completion of the parallel tasks I have to run another task.
Gradle runs inter-project tasks in parallel (wherever possible) if you use org.gradle.parallel=true or the --parallel flag. For intra-project tasks, you will need to use #ParallelizableTask for versions < 4 and Worker Api for versions >= 4.
However, be aware that WorkerApi has some limitations and is only useful in certain scenarios.
Related
I ran into the problem using the following situation:
To run different native testplugins in a row I dynamically create some tasks to prepare the environment appropriately for each plugin and then create a plugin_task for each testplugin that hosts the dependencies.
To start the tests on the device, the plugin_task is finalizedBy connectedAndroidTest.
So e.g. the call gradle plugin_name_1 works fine, it prepares the environment and starts the tests.
tasks.register("allTests"){}
name_list.each { test_name ->
def prepare_dir = tasks.register("${test_name}_Prepare_Dir") {...}
def copy_files = tasks.register("${test_name}_Copy_Files") {...}
def plugin_task = tasks.register(test_name){
dependsOn prepare_dir
dependsOn copy_files
finalizedBy connectedAndroidTest
}
allTests.configure {
dependsOn pluginTask
}
}
The problem now occurs, when I try run the allTests task:
gradle then correctly operates over all the dynamicly created tasks "prepare_dir", "copy_files" for each of the plugins, all of them are executed.
But the connectedAndroidTest will only be executed at the very end of allTests, so only runs once. But I need this task to be executed at every end of each plugin_task.
The behavior of gradle is correct, as it sees that all plugin_tasks shall be finalized by connectedAndroidTest and so puts it at the end of the graph.
Is there any possibility to make gradle execute this task multiple times?
Would be glad for help.
Wrapping the connectedAndroidTest task into dynamically created tasks and let the plugin_tasks depend on them, does not help. Also the parameter "rerun-tasks" has no effect.
That's not how Gradle works. Each task will execute either 1 or 0 times per Gradle invocation. If you want to execute the same task multiple times (with different task inputs) then you'll need multiple task instances, not one.
I have these tasks in a multi-project build:
task runA(type: GradleBuild) {
tasks = [
':a:x:bootRun',
':a:y:bootRun',
':c:x:bootRun'
]
}
task runB(type: GradleBuild) {
tasks = [
':b:x:bootRun',
':b:y:bootRun',
':c:x:bootRun'
]
}
Each :a:x etc. is a separate project (build.gradle / build.gradle.kts file). When I invoke gradle runA or gradle runB all of the subtasks are run in parallel just as they should.
However, when I run gradle runA runB only some of the tasks get invoked
while I want all five unique tasks a:x a:y c:x b:x b:y to be run in parallel. I have also tried using dependsOn instead of the GradleBuild task type but that leads to the same outcome (usually only 4 of 5 expected tasks run in parallel).
Note that the tasks never "complete" as they run an application, I'm more or less abusing gradle as a launcher of sorts here. I assume gradle blocks and waits for one of the tasks to complete but there are no dependencies between them.
What do I need to do to force gradle to run all of the tasks in parallel at the same time?
Gradle will use heuristics to decide how many things to run in parallel. So the most likely explanation is that by default it is limiting itself to 4 parallel tasks.
You can control this via gradle.properties:
org.gradle.workers.max=(max # of worker processes)
I got a Gradle task which calls two methods. The methods are independent of each other so I want to start them parallel. What is the best way to achieve this?
Example:
task cucumber(dependsOn: 'testClasses') {
doLast {
// ...
// I want to call the next 2 methods in parallel
runSequentialTests(testEnvironment, tags)
runParallelTests(testEnvironment, tags)
}
}
def runSequentialTests(testEnvironment, tags) {
// execute cucumber tests via javaexec
}
def runParallelTests(testEnvironment, tags) {
// execute cucumber tests via exec sh script for each feature file (parallelism is done via GParsPool.withPool(5) {...}
}
Worth researching groovy parallel systems library gpars
GParsPool.withPool {
GParsPool.executeAsyncAndWait({runSequentialTests(testEnvironment, tags)}, {runParallelTests(testEnvironment, tags)})
}
Or maybe create a #ParallelizableTask
But I'm not sure what version of gradle you are using. And parallel is/was an incubating feature.
And needs to run build with --parallel if those have no dependencies between themselves Gradle should run them independently.
Alternatively, you can specify a property in your gradle.properties
C:\Users\<user>\.gradle\gradle.properties
org.gradle.parallel=true
I used the asynchronous invocations of GPars: http://www.gpars.org/webapp/guide/#_asynchronous_invocations
I got it to work with this. Thanks
GParsPool.withPool {
GParsPool.executeAsyncAndWait({runSequentialTests(testEnvironment, tags)}, {runParallelTests(testEnvironment, tags)})
}
'I'm writing build script using net.foragerr.jmeter plug-in, version 1.0.2-2.13.
What my task does is mostly 1) runs plug-in JMeter task and 2) collects application log.
task perfTest(dependsOn: ['jmClean', 'jmRun'],
description:'Runs (cleanly) performance tests on a deployed application and collects the app log if available. ' +
'Use --no-daemon to see progress. Use --info to see all JMeter command-line arguments.') << {
if (logDir.isDirectory()) {
copy {
from "${logDir}"
into "${buildDir}/jmeter-report"
include 'iRePORT.log'
}
}
}
Now, I need to collect the log even if jmRun fails. When I try to implement a solution for this, I am really stuck:
I can't modify jmRun task, because it's a plug-in task
I can't use try/finally and execute another task directly because Gradle is designed not to support this, e.g. see How do I wrap a gradle task in another task?
I can't use --continue flag gradle.startParameter.continueOnFailure = true because it would not continue execution of a dependent task
The only possible workaround that I could think of is to separate log collection, always include it on a command line, e.g. perfTest collectLogs and set gradle.startParameter.continueOnFailure = true in perfTest.
This is far from ideal.
Are there any better solutions? Shouldn't there be a way for Gradle to support scenarios like this?
You could try creating a task like this
task collectPerfTestLogs(type: Copy) {
from "${logDir}"
into "${buildDir}/jmeter-report"
include 'iRePORT.log'
}
and then have perfTest.finalizedBy(collectPerfTestLogs).
To quote Gradle:
Finalizer tasks will be executed even if the finalized task fails.
See https://docs.gradle.org/current/userguide/more_about_tasks.html
Two questions:
What is the gradle way to specify that 1 task is comprised of several other tasks?
It seems like gradle's taskName.execute() method does not honor the dependsOn relationships of taskName is this true and what is the work-around?
More background:
Right now I have a build script that has no plugins (not Java in other words). I want a task called tests that will run all my test tasks. I have 3 such tasks. Call them task1, task2, and task3.
I could say tests.dependsOn ['task1', 'task2', 'task3']
This is a bit wonky because the relationship seems to be tests.isComprisedOf ['task1', 'task2', 'task3']
I could say:
task tests << {
task1.execute()
task2.execute()
task3.execute()
}
but then task3, which itself depends on taskSetup, runs without running taskSetup. In other words the execute() call does not seem to honor gradle's dependencies resolution strategy.
One last small gripe (I really do love gradle by the way), is that it is hard to search on this topic because dependency means two different things in gradle: dependsOn style dependencies and library style dependencies.
Typically, you do not invoke task.execute().
You can specify that one task is comprised of other tasks in the following manner:
task task1 << {
println "Hello"
}
task task2 << {
println "World"
}
task task3(dependsOn: 'task3dependency') << {
println "QBert"
}
task task3dependency << {
println "MR"
}
task tests(dependsOn: ['task1', 'task2', 'task3'])
This outputs:
$ gradle tests
:task1
Hello
:task2
World
:task3dependency
MR
:task3
QBert
:tests
BUILD SUCCESSFUL
Keep in mind that the order in which your dependency tasks are run is not always guaranteed, but you can mitigate this by specifying the order task2.mustRunAfter task1. Usually though, the tasks are run in the order you would expect.
Also, you should read up on Gradle's Build Lifecycle. When you use the syntax task task1 << {...}, you are specifying a closure that is run in the execution phase of the task. Before execution is run, the configuration phase evaluates your build script and determines the tasks to be run and in what order. When you manually execute tasks, as in:
task tests << {
task1.execute()
task2.execute()
task3.execute()
}
you have bypassed Gradle's ability to evaluate the task dependencies of task3, hence it runs only task3.
Gradle's task model is "flat" and doesn't have a concept of aggregation. (It's important to note that TaskInternal#execute is an internal method, and must not be called from build scripts.) Aggregation is often simulated with a lifecycle task (a task with task dependencies but without any task actions):
task allTests {
dependsOn tasks.withType(Test)
}
Besides dependsOn, the following task relationships are supported: mustRunAfter, shouldRunAfter, and finalizedBy.