we have a project structure such that
ParentProject
projA
projB
projC
projD
...
We have a bunch of tasks that we run on these projects from the parent project, calling gradle clean build make tag push remove
clean: removes the build directory for each project
build: compiles the source code
make: uses the compiled code to create a docker image (custom task)
tag: tags the docker image (custom task)
push: pushes the tagged docker image to our nexus repo (custom task)
remove: removes the local docker image (custom task)
all the tasks work as intended, but not exactly in the order we want.
our issue is disk space. creating all the docker images (we have many to make) takes several gigabytes of space. Gradle fails because it runs out of disk space before reaching the end. We thought we could solve this issue by simply removing the image after it is pushed and free the disk space, but the way gradle runs, that does not work.
Current, gradle executes in the following manner:
projA:clean
projB:clean
projC:clean
...
projA:build
projB:build
projC:build
...
projA:make
projB:make
projC:make
...
going through like this, everything tries to build before anything gets removed. The way we would like to run gradle is as follows:
projA:clean
projA:build
projA:make
projA:tag
projA:push
projA:remove
projB:clean
projB:build
projB:make
projB:tag
projB:push
projB:remove
...
this way the project cleans itself up and frees disk space before the next project starts building.
is there any way to do this?
edit: There may be times where not every task needs to be run. The image may already built and simply needs to be retagged. Or may need to be built for local testing, but not tagged or pushed.
Looks like tasks make, tag, push, remove are custom tasks.
If you want dependency among tasks, for example taskX should be executed whenever taskY is started, then you should use dependsOn property of task.It is also called as task dependencies.
below shows the build.gradle on parent project that make task of projA dependsOn clean task of projA. you should do this for remaining tasks of projA.
project('projA') {
task make(dependsOn: ':projA:build') {
doLast {
println 'make'
}
}
}
project('projA') {
task build {
doLast {
println 'build'
}
}
}
...
...
Then you should continue for remaining projects and remaining tasks. For example, ProjB:clean should dependOn projA:remove
project('projB') {
task clean(dependsOn: ':projA:remove') {
doLast {
println 'clean'
}
}
}
continue for other tasks in project B and remaining projects.
For more details please follow this link.
We found we can create a task of type 'GradleBuild' where we can specify what tasks we want to run within a single task.
task deploy(type: GradleBuild){
tasks = ['clean','build','make','tag','push','remove']
}
this solution prevents linking of tasks with dependencies, so we can still build sub projects individually without worrying about other tasks being run unnecessarily.
Related
I have set up some tasks compiling a libGdx project in Android Studio. What I want is, that a dynamic file "version.txt" is created in the assets folder. This file must be available in the final jar-output.
Here is how I did the setup:
libGdx provides me with a dist task.
I created two tasks on top of that: buildDebugVersion and buildReleaseVersion.
Both shall create a version.txt, one containing debug information and the other no debug info.
libGdx' original dist task
task dist(type: Jar) {
manifest {
attributes 'Main-Class': project.mainClassName
}
//processResources.dependsOn tasks.updateVersionFileRelease
dependsOn configurations.runtimeClasspath
from {
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
with jar
}
my debug/release tasks
task createDebugVersion {
outputs.upToDateWhen { false }
doFirst {
modifyVersionFile(1)
}
}
task createReleaseVersion {
outputs.upToDateWhen { false }
doFirst {
modifyVersionFile(0)
}
}
task dependencies
To put that all in the right running order, I created
createDebugVersion.finalizedBy(tasks.dist)
createReleaseVersion.finalizedBy(tasks.dist)
dist.mustRunAfter(tasks.createDebugVersion, tasks.createReleaseVersion)
My expectation was:
createdebug/release runs and creates the file
THEN dist runs
This seems to happen, when I look at the output. You can see, what file is generated:
Here is the output when I run it:
> Task :desktop:createReleaseVersion
Generated version file is:
--------------------------
Version=0.1.113
BuildDate=2020-04-24
--------------------------
BUILD SUCCESSFUL in 5s
10 actionable tasks: 9 executed, 1 up-to-date
...but when I run the program, I see this version information:
The file contained in the jar is alway 1 or 2 versions behind, as if it would've been taken out of some cache folder.
What scares me is the question: How many of the other assets are now 1 or 2 versions behind, too?
From where does gradle take this file?
I was not able to find it out so far.
what I already tried
I run this through a .cmd script. Before starting the gradle task, I already delete all build folders (and tested it, everything is removed before the build starts)
(the variable %TASK% contains either createDebugVersion or createReleaseVersion)
ECHO Forcing resources rebuild
RD /S /Q .\desktop\build
RD /S /Q .\android\build
ECHO Compiling distribution version of %GAMENAME%...
CALL gradlew desktop:clean desktop:build %TASK% --rerun-tasks
But still, out of some "ghost-galaxy-space", a file that is two builds old is taken from somewhere and put into the jar...
Any help greatly appreciated!
Ok, I figured it out - maybe it helps others if they run into a similar issue.
The error was here:
CALL gradlew desktop:clean desktop:build %TASK% --rerun-tasks
If you look closely, you see FIRST desktop:clean runs, THEN desktop:build runs, which causes a full-rebuild due to the deleted caches and build folders, and THEN my %TASK% runs - as third in order!
So, my task runs after the build. Even when I put up the build chain with
createDebugVersion.finalizedBy(tasks.dist)
createReleaseVersion.finalizedBy(tasks.dist)
this all only runs after the build is already done. And the dist task has no clean in this build chain so it uses the output of the build generated by the second parameter of my gradlew call!
The solution
I simply changed how I run gradlew:
my command script now calls
call gradlew desktop:clean %TASK% --rerun-tasks
so a clean is still forced, but the build only happens as a reaction from dist because it can't find any binaries, so it has to build them right now.
And then the build finally runs after my file has been written to the assets folder.
Hope this helps someone some day!
cheers, gris
I’m getting the following error whenever I attempt to use a Copy task to copy a file into the root of a project (the same folder I’m running gradle from):
Failed to create MD5 hash for file content.
I thought this was related to the artifacts I was pulling from Artifactory, but that seems to be unrelated. I was able to get the same results with a minimal script.
Is there something obviously wrong with what I’m doing, or does Gradle intentionally disallow such things?
task fails(type:Copy) {
from 'build/someFile.txt'
into new File('.').absolutePath
}
task works(type:Copy) {
from 'build/someFile.txt'
into new File('.').absolutePath + '/output'
}
Short Answer: Don't copy into the project directory, you are best to use into "$buildDir/someFolder" so that the folder is isolated to this single task, and also so that it will be cleaned by gradle clean
Long Answer: At it's core, Gradle has the concept of an "UP-TO-DATE" check for every single task. If Gradle sees that nothing has changed since last time a task was executed it will use the old result instead of executing again.
UP-TO-DATE checking is implemented by taking a "hash" of the task inputs and task outputs. Since you are using into '.' that means that the entire contents of the project directory is considered a task output (bad)
Gradle uses the .gradle folder for temp files (eg task hashes) It's likely some of these files are locked for writing as Gradle is trying to also read the same files (to calculate the "hash" of the task outputs) causing the error you are seeing
* EDIT *
If you need to copy into the project directory for legacy reasons, you might use Project.copy(...) directly instead of a Copy task. You could manually manage the task inputs/outputs in this case
Eg
task customCopy {
inputs.file "$buildDir/someFile.txt"
outputs.file 'someFile.txt'
doLast {
copy {
from "$buildDir/someFile.txt"
into '.'
}
}
}
Can you believe it, the following works
task myCopy(type: Copy) {
from "$rootDir/app1/src/main/resources/db"
into "$rootDir/app2/src/test/resources/db"
}
test.dependsOn myCopy
and the following doesn't 🤦
task myCopy(type: Copy) {
from '$rootDir/app1/src/main/resources'
into '$rootDir/app2/src/test/resources'
}
test.dependsOn myCopy
I have a gradle composite build project, which runs tasks from an included build. I am not able to make changes to the included build.
For a certain task, I want to first clean the included build and then build it. I.e., I want to do (loosely):
some-project:clean
some-project:build
where some-project is the included build. I tried doing this by setting up tasks which depend on those external tasks:
task cleanIncludedBuild {
dependsOn gradle.includedBuild("some-project").task("clean")
}
task buildIncludedBuild {
dependsOn cleanIncludedBuild
dependsOn gradle.includedBuild("some-project").task("build")
}
However, since there is no ordering between dependencies, this execution order is valid as declared:
some-project:build
some-project:clean
cleanIncludedBuild
buildIncludedBuild
Though this is of course not what I'm after.
I thought of doing something like
gradle.includedBuild("some-project").task("build").dependsOn cleanIncludedBuild
but the value returned by task() above is of type IncludedBuildTaskReference, which essentially are read-only references to the underlying task, and do not allow adding dependencies.
I have a multiproject and after the last subproject is built, I'd like to process all jars.
Therefore I created a task in the root-project:
task install(dependsOn: 'build', type: Copy) {
doLast {
println "exec install task"
}
}
Upon calling ./gradlew install in the root directory, I'm facing this error:
FAILURE: Build failed with an exception.
* What went wrong:
Could not determine the dependencies of task ':install'.
> Task with path 'build' not found in root project 'foo'.
However, calling ./gradlew tasks shows me these tasks:
:tasks
------------------------------------------------------------
All tasks runnable from root project
------------------------------------------------------------
Build tasks
-----------
assemble - Assembles the outputs of this project.
build - Assembles and tests this project.
...
How can I achieve the desired functionality?
I assume, that your root project organizes the build, but does not define build action taken by itself. The build task is often defined by language plugins (in most cases via apply plugin: 'java'), but if your root project does not use any of them, it won't have a build task.
The description of the help task tasks, which you used, says:
Displays the tasks runnable from root project 'projectReports' (some of the displayed tasks may belong to subprojects).
The help task followes the same logic as the task activation via the command line. You can provide a task name and any task with the name in any subproject will be executed (thats why gradle build works).
But if you define a dependsOn dependency, the given string is evaluated as task path for a single task. Since each task name can only be used once in a project, the name is unique for tasks in the root project, but many tasks could be found, if subprojects would be considered. Therefor, one can use the syntax :<projectName>:<taskName> to identify tasks in subprojects.
Now, let's face your specific problem: If the install task should depend on the build task of one single subproject, you could use dependsOn ':<mySubproject>:build'. But I assume you want the install task to depend on each subproject build task, so I'd like to propose this approach:
task install(type: Copy) {
dependsOn subprojects*.tasks*.findByName('build').minus(null)
doLast {
println "exec install task"
}
}
This way, for each registered subproject, findByName('build') is called and the result (the found task or null) is put into a list, which is then used as task dependency list. I added the .minus(null) part to remove null entries from the list, because I am not sure how Gradle handles such entries in a dependency collection. If you are sure, that each subproject provides a build task, you can use getByName('build'), too.
EDIT: OP found the optionally recursive getTasksByName method, which suits this case even better than iterating over all subprojects manually:
dependsOn getTasksByName('build', true)
I am on a team that has a large multi-project build (~350 modules) and we have noticed that doing an individual build (-a) still takes a large amount of time. When building the entire set of things the overhead isn't so bad, but when we do something like:
cd my/individual/project
gradle -a --configure-on-demand --daemon build
that it still takes 30-40 seconds just in the configuration phase (before it starts building anything, which we measure using the --dry-run option)
We have approximately 10 custom tasks and we are setting the inputs and outputs for those tasks but we still see this inordinately long configuration time. We are using gradle 1.10
It's hard to say from a distance. A common mistake is to do work in the configuration phase that should be done in the execution phase. Here is an example:
task myTask {
copy {
from ...
to ...
}
}
Here, the project.copy method will be called (and therefore copying will take place) every single time the task is configured, which is for every single build invocation - no matter which tasks are run! To fix this, you'd move the work into a task action:
task myTask {
doLast {
copy {
from ...
into ...
}
}
}
Or even better, use a predefined task type:
task myTask(type: Copy) {
from ...
into ...
}
Some steps you can take to find the cause(s) for performance problems:
Review the build
Run with --info, --debug, or --profile
Profile the build with a Java profiler
PS: Specifying inputs and outputs doesn't shorten configuration time (but likely execution time).