I have a multi project Gradle build script that runs successfully on Windows 10. It reads and updates a Version.properties file that is located away from project managed directories.
All file manipulations are done using Gradle/groovy. After the Version file has been read, incremented and rewritten it is copied to a build/classes directory where it will be picked up by subsequent jar and shadowjar tasks.
Everything works as advertised if I invoke gradle as follows:
gradle build shadowjar ... etc.
However, if I invoke the clean task prior to build the file is read and incremented properly but the copy of the file fails silently.
The command used is:
gradle clean build shadowjar
My suspicion is that gradle does not wait for the clean task to finish prior to starting the build task. The file gets read and incremented but meanwhile, the multi-project clean activities have not yet finished. I have tried variations on dependencies{} blocks, doFirst{} and doLast{} to try and push the file copy back further in the build process. My main requirement is to have the Version.properties file in place prior to the jar or shadowjar task executing. I'm suspicious of trying to write into gradle's build/ directories in that it might not be possible to put anything into the build directories while gradle is performing its activities. Is there any way to ensure that the Version.properties file (or any generated file) gets copied? Or is there another location that I can use that will not be blown away by gradle at clean time yet still get picked up in the build:jar / build:shadowjar?
You are not supposed to call gradle clean 99.99% of the time, it is redundant due to gradle's incremental build feature. So as long as you correctly define your task inputs and outputs and start from ground up in each task, the problem solves itself.
Anyway in your case the wrong order could be caused by dependency between clean and other tasks, is there any?
I have found a way to write out a generated Version.properties file that will get picked up by the jar and shadowjar tasks. Use the gradle copy task and place the revised Version.properties file into a resources directory. The build activity includes the files found in resources/ in subsequent tasks (jar, shadowjar, test, etc.) My suspicion is that because clean blows away build directories gradle assumes that the activity has fully completed when it starts the build. I think that I've proven that this is not the case. doFirst{}, doLast{} and dependencies{} do not seem to work as modifiers to clean build.
Related
I have this kotlin gradle build script representing my use case.
I am working with Gradle 6.7.
This is my kotlin gradle build file:
plugins {
java
}
tasks.register("createFile"){
doLast{
projectDir.resolve("tmp.txt").createNewFile()
projectDir.resolve("tmp.txt").writeText("tmp")
}
dependsOn("assemble")
}
tasks.register("createExecFile", Exec::class){
workingDir(buildDir)
commandLine("cmd", "/c", "mkdir destdir\\subdir")
dependsOn("createFile")
}
tasks.register("copyBug", Copy::class){
from(projectDir.resolve("tmp.txt"))
into(buildDir.resolve("destDir"))
dependsOn("createExecFile")
}
Now run gradle copyBug -i.
The first time this will give you this output:
> Task :copyBug Deleting stale output file: E:\repo\BugCOpy\build\destDir Caching disabled for task ':copyBug' because: Build cache is disabled Task ':copyBug' is not up-to-date because: No history is available.
The copy task deletes the file created by the previous exec task.
Now if you rerun this command the copy task won't delete stale file.
So what are those stale file? How can I prevent those file to be deleted? My first build and is different than the other build.
Should I file a bug?
In your createExecFile task you produce output files without telling Gradle about them. So Gradle doesn’t know that this task and the copyBug task use the same output directory. Instead, Gradle believes that copyBug is the only task producing outputs under build/destdir and hence it’s safer to assume that any existing files in that directory should not be there (and are “stale”).
The solution is to tell Gradle that your createExecFile task outputs to build/destdir:
tasks.register("createExecFile", Exec::class) {
workingDir(buildDir)
commandLine("cmd", "/c", "mkdir destdir\\subdir")
// tell Gradle about the output directory
outputs.dir(buildDir.resolve("destdir"))
dependsOn("createFile")
}
See this release notes section for why this behavior was introduced and this explanation to a very similar issue if you want some more background information.
My build process looks like
run gradle (part 1)
do something else
run gradle (part 2)
For me annoyingly running gradle in step 3 would not just add a few files to the output directory but delete that folder as stale first. Even adding
outputs.dir(...)
did not prevent Gradle from removing it as stale. Working out the inputs and outputs of my task looked too tedious, but luckily I found a way to tell Gradle not to perform any up to date tracking:
Example 37. Ignoring up-to-date checks mentions to add
doNotTrackState("Comment why this is needed")
which ultimately helped me to keep the files from build step 1.
We wired https://github.com/sherter/google-java-format-gradle-plugin into our project per the readme.
We also wired in a pre-commit hook to run the plugin before committing, which ensures that all of the code in a changelist is formatted before pushing it, which avoids errors in Jenkins when we run the verGJF task.
But we have to keep remembering to run goJF locally before running ./gradlew build, or the build fails with formatting errors.
We worked around this by adding the https://plugins.jetbrains.com/plugin/8527-google-java-format and https://plugins.jetbrains.com/plugin/7642-save-actions plugins for IntelliJ, enabling the google-java-format plugin, and configuring the save-actions plugin to format on save.
But that's a lot of extra configuration a developer has to remember to go through, plus it means they can't format code the way they want while working on it, and only have it be reformatted at the point of build or commit.
We'd prefer an all-Gradle solution, so that the goJF task is run before the build task (and before the verGJF task, which is already bound to the build task by the google-java-format Gradle plugin).
We couldn't figure out how to do that. Does someone else know?
It sounds like you want to essentially always ensure that the code is properly formatted before the verifyGoogleJavaFormat task is run (and could complain). In that case, I’d simply make the googleJavaFormat task a dependency of the verifyGoogleJavaFormat task. In your build.gradle file, after you have applied the google-java-format plugin, simply add the following:
verifyGoogleJavaFormat.dependsOn(tasks.googleJavaFormat)
Alternatively, if you really only want to run the code formatter when the build task is run (as opposed to when the verifyGoogleJavaFormat task is run only), you could add this instead:
build.dependsOn(tasks.googleJavaFormat)
verifyGoogleJavaFormat.mustRunAfter(tasks.googleJavaFormat)
I have the following situation:
I have a project with several subprojects. Today I tried to build the projects with gradle via command line.
The build was successful when I executed ./gradlew clean :build, but not with ./gradlew clean build. It causes different errors, depending on which subprojects are activated. Why is that? Shouldn't it be the same?
Both commands are executed directly after each other, without changes in the code, and from the same directory (the base-directory, where settings.gradle is located.
The gradle-refresh of Intellij works, the build is successful (but fails on our build server, if that is relevant).
According to the documentation https://docs.gradle.org/current/userguide/command_line_interface.html#executing_tasks_in_multi_project_builds I assumed that it would do the same, since no subproject is specified, and the build task is executed for all submodules. There is no folder called build in the root project, so this should not cause confusion. Am I interpreting it wrong?
I searched online, however, I could not find the result, since : is is not recognized by most search engines, and colon leads to not relevant results like What is the colon operator in Gradle?.
The gradle version is 4.10.2
If you need more information, please let me know.
There is a difference between ./gradlew clean :build and ./gradlew clean build, that's why you have a different behavior: in the first case you are using qualified task name, in the other case you are using simple task name. These documentations here and here explain these two approaches for executing tasks:
Using simple task name ( ./gradlew test) :
The first approach is similar to the single-project use case, but Gradle works slightly differently in the case of a multi-project build. The command gradle test will execute the test task in any subprojects, relative to the current working directory, that have that task. So if you run the command from the root project directory, you’ll run test in api, shared, services:shared and services:webservice. If you run the command from the services project directory, you’ll only execute the task in services:shared and services:webservice.
=> so executing ./gradlew build in the root project directory will trigger execution of build task for root project and all subprojects
Using qualified task name ( ./gradlew :build)
For more control over what gets executed, use qualified names (the second approach mentioned). These are paths just like directory paths, but use ‘:’ instead of ‘/’ or ‘\’. If the path begins with a ‘:’, then the path is resolved relative to the root project. In other words, the leading ‘:’ represents the root project itself. All other colons are path separators.
=> executing ./gradlew :build , you will execute "only" the build task for rootProject
As I said in comment, you migth have some issues in one or more of your subproject, but you will not see these errors if you execute only Root project build ( ./gradlew :build )
Consider a Gradle plugin that adds three tasks to a project - a buildZip task to create a distributable zip of the project, a publishZip task to publish that zip to a shared repository, and a cleanZip task to clean up any local version of the zip. For local development, cleanZip buildZip will be used frequently, but the automated build system will be running buildZip publishZip cleanZip.
One of the projects in which this plugin is being used wants to run their build using Gradle's parallel flag to allow the different parts of the project to be built in parallel. Unfortunately, this runs into a problem with the zip tasks - buildZip depends on the project actually building, but cleanZip doesn't have any dependencies so it can run right away, leading to the automated build system not being able to clean up.
Declaring any dependencies between these tasks isn't a good idea because they should be able to be run separately. Also, I can't specify mustRunAfter (at least between buildZip and cleanZip) because sometimes clean should be first and sometimes build should be first.
How can I tell Gradle what order to run these tasks in, in a way that will be honored by --parallel and isn't hardcoded to have a particular one always run before the other?
What you can do is: detect if gradle is run with --parallel and based on this configure dependencies between tasks appropriately. It can be done in the following way:
println project.gradle.startParameter.parallelProjectExecutionEnabled
Why would I run gradle clean build instead of gradle build?
From what I understand, Gradle can detect source changes and update the final artifacts if needed. So why would I still need to clean?
The clean task is defined by the java plugin and it simply removes the buildDir folder, thus cleaning everything including leftovers from previous builds which are no longer relevant. Not doing so may result in an unclean build which may be broken due to build artifacts produced by previous builds.
As an example assume that your build contains several tests that were failed and you decided that these are obsolete thus needs to be removed. Without cleaning the test results (using cleanTest task) or the build entirely (by running the clean task) you'll get stuck with the failed tests results which will cause your build to fail. Similar side effects can happen also with resources/classes removed from the sources but remained in the build folder that was not cleaned.
It removes the build directory. (Build contains the output of the gradle operation)
Other build tools like buck will detect that some tests are removed and won't run them without the needs to run clean target. I think this is pitfall of gradle.
You don't need to run the clean task.
Gradle will track task dependencies and clean appropriate parts for you.
Here's an example Gradle project I created to show that the accepted answer is incorrect.
If custom tasks don't track their dependencies well (they're bugged), then clean is a workaround.