Gradle : Specify tasks to be run in parallel - gradle

I am working on an Android project that uses gradle.
I have 5 tasks, say taskA, taskB, task C, taskD, taskE.
Now whenever I run taskA, it should be followed by other 4 tasks.
So, I wrote like this in my build.gradle:
taskA.finalizedBy(taskB)
taskA.finalizedBy(taskC)
taskA.finalizedBy(taskD)
taskA.finalizedBy(taskE)
But I observe that these tasks are being executed sequentially and take much time.
I have set org.gradle.parallel=true in my gradle.properties but still no luck.
Is there a way to specify these tasks to run in parallel?
Note : taskB, task C, taskD, taskE are dynamically generated by Android Gradle Plugin.

Related

How to have Gradle task depend on another task but not re-execute if the other executes?

I would like to have one Gradle task, taskA, to trigger the execution of another task, taskB, if both aren't up-to-date but the execution of taskB shouldn't trigger an execution of taskA if only taskB isn't up-to-date. How can this be done?
IOW, there are two parts of the task dependency involved here, the task hierarchy and the up-to-date checks. I would like to bed able to set up the task hierarchy but not have it imply the up-to-date check for taskA.
Context: taskA must execute if its inputs change or if taskB executes on a dev machine (ie not CI). taskB must execute if it's not up-to-date. This is because the output of taskB is an executable that's run by taskA. The reason taskA shouldn't run if only the executable is updated is because the updated executable can produce different output than the previous executable. That new output can cause issues with other parts of the build. Since the purpose of the output is to help ensure users are following best practices, the new output of the updated executable can be ignored in CI builds.
More concretely, taskA calls a protolock binary which is output by taskB. taskA outputs a proto.lock file which engineers should commit. Auto-committing that file can lead to surprises for engineers and to subversion of the purpose of these checks.
Use onlyIf:
taskB.onlyIf {
trueIffNotCiOrInputsAreNewerThanOutput()
}

Gradle shouldRunAfter not available for a task

I created this repo to reproduce exactly what I'm seeing. I have a project that I'm building with Gradle. I would like to configure my Gradle build so that running:
./gradlew build
Has the exact same effect as running:
./gradlew clean build scalafmt shadowJar fizzbuzz
Meaning Gradle calls tasks in the following order:
clean
build (compile & run unit tests)
scalafmt (run a tool that formats my code to a styleguide)
shadowJar (creates a self-contained executable "fat" jar)
fizzbuzz (prints out "Fizzbuzz!" to the console)
According to the Gradle docs on ordering tasks, it seems that I can use shouldRunAfter to specify ordering on all tasks...
If you clone my repo above and then run ./gradlew build you'll get the following output:
./gradlew build
Fizzbuzz!
FAILURE: Build failed with an exception.
* Where:
Build file '/Users/myUser/thelab/idea-scala-hate-each-other/build.gradle' line: 65
* What went wrong:
A problem occurred evaluating root project 'idea-scala-hate-each-other'.
> Could not find method shouldRunAfter() for arguments [task ':build'] on cz.alenkacz.gradle.scalafmt.PluginExtension_Decorated#6b24ddd7.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 10.983 secs
So even though I specify fizzbuzz to run last...its running first! And there are (obviously) errors with the rest of my config. And I'm not sure how to "hook" clean so that even when I run ./gradlew build it first runs clean.
I'd be OK with a solution that requires me to write my own "wrapper task" to achieve the order I want, and then invoke it, say, via ./gradlew buildMyApp, etc. Just not sure how to accomplish what I want.
Update
I made some changes to build.gradle and am now seeing this:
./gradlew fullBuild
:compileJava UP-TO-DATE
:compileScala
:processResources UP-TO-DATE
:classes
:jar
:startScripts
:distTar
:distZip
:assemble
:compileTestJava UP-TO-DATE
:compileTestScala UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build
:clean
:fizzbuzz
Fizzbuzz!
:scalafmt
:shadowJar
:fullBuild
So when I run ./gradlew fullBuild task execution seems to be:
build (which calls compileJava through check)
clean
fizzbuzz
scalafmt
shadowJar
So the ordering is still wrong even given my most recent changes...
As Oliver already stated, you need to put the console output to a doFirst or doLast closure, otherwise it will be executed when the task is defined (during configuration phase).
The exception is caused by the fact that both extension properties and tasks are added to the scope of the Project object, but if both an extension property and a task with the same name (scalafmt in this case) exist, the extension property will be accessed. As the error message tells you, you are trying to access the shouldRunAfter method on an object of the type PluginExtension, where it does not exist. You need to make sure to access the task:
tasks['scalafmt'].shouldRunAfter build
Update
Actually, I thought that you only need the two specific problems to be solved, but already have a solution for your basic Gradle structure.
First of all, both shouldRunAfter and mustRunAfter do not actually cause a task to be executed, they only define the order if both tasks are executed (caused by command line or a task dependency). This is the reason why the tasks clean, scalafmt, shadowJar and even fizzbuzz are not executed if you call gradle build. So, to solve your first problem, you could let the build task, which you call explicitly, depend on them:
build.dependsOn 'clean', 'scalafmt', 'shadowJar', 'fizzbuzz'
But a task dependency will always run before the parent task, so all tasks will be executed before the build task. This should not be a problem, since the build task does nothing more than collecting task dependencies for all required build steps. You would also need to define the order not only between of your task dependencies, e.g. clean and the parent task build, but mainly between the existing task dependencies, e.g. compileJava. Otherwise clean could run after compileJava, which would delete the compiled files.
Another option would be to define a new task, which then depends on all the tasks you want to execute:
task fullBuild {
dependsOn 'clean', 'build', 'scalafmt', 'shadowJar', 'fizzbuzz'
}
This would still require to define the order between your tasks and the existing task dependencies, e.g.
compileJava.mustRunAfter 'clean'
[...]
Please note, that now you would have to call gradle fullBuild from command line. If you really need to only call gradle build via command line and still execute some tasks after the actual build task, you could use a little trick in your settings.gradle file:
startParameter.with {
if (taskNames == ['build']) {
taskNames = ['clean', 'build', 'scalafmt', 'shadowJar', 'fizzbuzz']
}
}
This piece of code checks the task name input you entered via command line and replaces it, if it only contains the build task. This way you would not have to struggle with the task order, since the command line tasks are executed consecutively.
However, this is not a clean solution. A good solution would include the definition of task dependencies for all tasks that really depend on each other and the invocation of multiple tasks via command line (which you want to avoid). Especially the hard connection between the clean and the build task bypasses a lot of useful features from the Gradle platform, e.g. incremental builds.
Regarding your second point in the update, it is wrong that the fizzbuzz task is running first. It is not running at all. The command line output is printed when the task is configured. Please move the println call to a doFirst / doLast closure:
task fizzbuzz {
doFirst {
println "Fizzbuzz!"
}
}
One of the possible causes of that error could be because Gradle couldn't compile the task of the method shouldRunAfter or mustRunAfter.
Also it could happen when there is another entity of other type (not a task) with the same name of the task of the method shouldRunAfter or mustRunAfter. In this case you could use the syntax tasks['task1_name'].shouldRunAfter tasks['task2_name'] or tasks['task1_name'].mustRunAfter tasks['task2_name'] to make sure Gradle is referencing a task type entity.

Conditionally ordering tasks in Gradle

Consider a Gradle plugin that adds three tasks to a project - a buildZip task to create a distributable zip of the project, a publishZip task to publish that zip to a shared repository, and a cleanZip task to clean up any local version of the zip. For local development, cleanZip buildZip will be used frequently, but the automated build system will be running buildZip publishZip cleanZip.
One of the projects in which this plugin is being used wants to run their build using Gradle's parallel flag to allow the different parts of the project to be built in parallel. Unfortunately, this runs into a problem with the zip tasks - buildZip depends on the project actually building, but cleanZip doesn't have any dependencies so it can run right away, leading to the automated build system not being able to clean up.
Declaring any dependencies between these tasks isn't a good idea because they should be able to be run separately. Also, I can't specify mustRunAfter (at least between buildZip and cleanZip) because sometimes clean should be first and sometimes build should be first.
How can I tell Gradle what order to run these tasks in, in a way that will be honored by --parallel and isn't hardcoded to have a particular one always run before the other?
What you can do is: detect if gradle is run with --parallel and based on this configure dependencies between tasks appropriately. It can be done in the following way:
println project.gradle.startParameter.parallelProjectExecutionEnabled

Calling Gradle commands and tasks with parameters from Gradle Task

I have a current setup in my build.gradle that I'm trying to understand. I need a number of tasks to go off in a very specific order and execute all with one task call. The setup I want looks like this:
1.) Run liquibase changeset into predefined database
2.) Run a number of tests against database
3.) Rollback all changes made with the previous changeset
I want the database in a 'clean' state every time I test it. It should have only the changes I expect and nothing else. The liquibase is set up with the Gradle plugin for it and the changeset is applied/updated. However, I don't want to call the command manually. This will be something that needs to run in continuous integration, so I need to script it so I simply have our CI call one task and it then runs each task, in order, until the end. I'm unsure of how to call the Gradle command-line task from inside of itself (ie inside the build.gradle file) and then also pass parameters to it (since I'll need to call some type of rollback command task to get the database to be what it was before calling the update).
Right now, all I'm doing is calling the command line tasks like this:
$ gradle update
$ gradle test
$ gradle rollbackToDate -PliquibaseCommandValue=2016-05-25
Again, I can't call them by the command line alone. I need a custom task inside Gradle so that I could just call something like:
$ gradle runDatabaseTests
...And I would have it do everything I expect.
There is no gradle way to invoke/call a task from another task directly. What you can do instead is to use dependsOn or finalizedBy to setup task dependencies which will force the prereq tasks to run first.
If you declare a task:
task runDatabaseTests(dependsOn: [update, test, rollbackToDate]) << {
println "I depend on update, test and rollbackToDate"
}
when you call
gradle runDatabaseTests -PliquibaseCommandValue=2016-05-25
it will force update, test and rollbackToDate first. You can control the order in which they're run, if you care about that, by using mustRunAfter and/or shouldRunAfter

Run all microservices in a multi-project gradle build

I have a multi-project gradle build that's roughly set up like this:
RootProject
- ServiceA
- ServiceB
- ServiceC
- UI
Each of these subprojects is using the Spark framework and runs an embedded web server. It's basically a microservices setup, so they all need to be up and running for the system as a whole to work.
They each have a task defined like this:
task runApp(type: JavaExec) {
main = 'App'
classpath = sourceSets.main.runtimeClasspath
}
I know I can manually start each service either through my IDE or by opening a different terminal for each sub-project and running gradlew ServiceA:runApp, but what I'd like to do is create a runSystem task at the root level that will fire up everything to make it easy to run the whole system during development.
How can I do this with Gradle?
If you run a task on the root project, Gradle invokes the same task (if it exists) on all the subprojects. Just execute runApp from the root folder.
In your case however, your runApp task might not exit because it starts a server, so execution will not move on to the next project. You can either enable parallel execution, or modify your tasks to run your server tasks in the background (using ProcessBuilder)

Resources