I am using the liquibase plugin.
I want it to participate in incremental builds, so I defined:
tasks.withType(org.liquibase.gradle.LiquibaseTask) {
//for incremental builds
outputs.dirs("$rootDir/liquibase/src/main/resources", 'build/generated')
}
Everything works fine, except that after I do a ./gradlew clean (which deletes the build/generated directory), the task is still considered UP-TO-DATE, so it doesn't execute again the migrations.
Full context of the issue: I am using the docker avast plugin, I am spinning up a PostgreSQL database, I run the liquibase update (db is empty, so all the migrations should run), then I use jooq codegen to generate Java classes based on the schema (they are generated in build/generated). The other tasks (composeUp, generateJooq) have the same outputs.dirs gradle configuration, but after I execute ./gradlew clean they are re-executed.
I fixed it by creating a dummy file in each task, define it as an output file and use that as an input for the next task.
Related
I have a multi-project gradle repo with separate projects for handing database migrations (schema) and code generation (codegen). The schema project has a :migrate task for applying migrations to a database, and the codegen project has a :generate task, which depends on a migrated database, to generate some java code. For the sake of example, lets say a third project exists which depends on codegen:generate, named app.
In CI, the gradle build makes use of a remote build cache. This works well in most cases. Changes within schema cause a run that looks like this:
./gradlew app:build
> schema:migrate
> codegen:generate
> app:build
SUCCESS
Changes to only app also works well:
./gradlew app:build
> schema:migrate UP-TO-DATE
> codegen:generate UP-TO-DATE
> app:build
SUCCESS
The problem exists when there are changes within the codegen project, but no changes to schema:
./gradlew app:build
> schema:migrate UP-TO-DATE
> codegen:generate
(Error here because a migrated database does not exist in CI,
because `:migrate` did no work)
FAILURE
How can I ensure that codegen:migrate is always re-run when there are changes to either schema or codegen?
If migrate should run when there are codegen changes, then those are inputs.
I would merge the schema and codegen projects into one, then configure the inputs such that the files that trigger generate to run are also declared as inputs of migrate.
We wired https://github.com/sherter/google-java-format-gradle-plugin into our project per the readme.
We also wired in a pre-commit hook to run the plugin before committing, which ensures that all of the code in a changelist is formatted before pushing it, which avoids errors in Jenkins when we run the verGJF task.
But we have to keep remembering to run goJF locally before running ./gradlew build, or the build fails with formatting errors.
We worked around this by adding the https://plugins.jetbrains.com/plugin/8527-google-java-format and https://plugins.jetbrains.com/plugin/7642-save-actions plugins for IntelliJ, enabling the google-java-format plugin, and configuring the save-actions plugin to format on save.
But that's a lot of extra configuration a developer has to remember to go through, plus it means they can't format code the way they want while working on it, and only have it be reformatted at the point of build or commit.
We'd prefer an all-Gradle solution, so that the goJF task is run before the build task (and before the verGJF task, which is already bound to the build task by the google-java-format Gradle plugin).
We couldn't figure out how to do that. Does someone else know?
It sounds like you want to essentially always ensure that the code is properly formatted before the verifyGoogleJavaFormat task is run (and could complain). In that case, I’d simply make the googleJavaFormat task a dependency of the verifyGoogleJavaFormat task. In your build.gradle file, after you have applied the google-java-format plugin, simply add the following:
verifyGoogleJavaFormat.dependsOn(tasks.googleJavaFormat)
Alternatively, if you really only want to run the code formatter when the build task is run (as opposed to when the verifyGoogleJavaFormat task is run only), you could add this instead:
build.dependsOn(tasks.googleJavaFormat)
verifyGoogleJavaFormat.mustRunAfter(tasks.googleJavaFormat)
I have a multi project Gradle build script that runs successfully on Windows 10. It reads and updates a Version.properties file that is located away from project managed directories.
All file manipulations are done using Gradle/groovy. After the Version file has been read, incremented and rewritten it is copied to a build/classes directory where it will be picked up by subsequent jar and shadowjar tasks.
Everything works as advertised if I invoke gradle as follows:
gradle build shadowjar ... etc.
However, if I invoke the clean task prior to build the file is read and incremented properly but the copy of the file fails silently.
The command used is:
gradle clean build shadowjar
My suspicion is that gradle does not wait for the clean task to finish prior to starting the build task. The file gets read and incremented but meanwhile, the multi-project clean activities have not yet finished. I have tried variations on dependencies{} blocks, doFirst{} and doLast{} to try and push the file copy back further in the build process. My main requirement is to have the Version.properties file in place prior to the jar or shadowjar task executing. I'm suspicious of trying to write into gradle's build/ directories in that it might not be possible to put anything into the build directories while gradle is performing its activities. Is there any way to ensure that the Version.properties file (or any generated file) gets copied? Or is there another location that I can use that will not be blown away by gradle at clean time yet still get picked up in the build:jar / build:shadowjar?
You are not supposed to call gradle clean 99.99% of the time, it is redundant due to gradle's incremental build feature. So as long as you correctly define your task inputs and outputs and start from ground up in each task, the problem solves itself.
Anyway in your case the wrong order could be caused by dependency between clean and other tasks, is there any?
I have found a way to write out a generated Version.properties file that will get picked up by the jar and shadowjar tasks. Use the gradle copy task and place the revised Version.properties file into a resources directory. The build activity includes the files found in resources/ in subsequent tasks (jar, shadowjar, test, etc.) My suspicion is that because clean blows away build directories gradle assumes that the activity has fully completed when it starts the build. I think that I've proven that this is not the case. doFirst{}, doLast{} and dependencies{} do not seem to work as modifiers to clean build.
Consider a Gradle plugin that adds three tasks to a project - a buildZip task to create a distributable zip of the project, a publishZip task to publish that zip to a shared repository, and a cleanZip task to clean up any local version of the zip. For local development, cleanZip buildZip will be used frequently, but the automated build system will be running buildZip publishZip cleanZip.
One of the projects in which this plugin is being used wants to run their build using Gradle's parallel flag to allow the different parts of the project to be built in parallel. Unfortunately, this runs into a problem with the zip tasks - buildZip depends on the project actually building, but cleanZip doesn't have any dependencies so it can run right away, leading to the automated build system not being able to clean up.
Declaring any dependencies between these tasks isn't a good idea because they should be able to be run separately. Also, I can't specify mustRunAfter (at least between buildZip and cleanZip) because sometimes clean should be first and sometimes build should be first.
How can I tell Gradle what order to run these tasks in, in a way that will be honored by --parallel and isn't hardcoded to have a particular one always run before the other?
What you can do is: detect if gradle is run with --parallel and based on this configure dependencies between tasks appropriately. It can be done in the following way:
println project.gradle.startParameter.parallelProjectExecutionEnabled
I am observing some strange Liquibase behavior when I run simple Maven project on TeamCity build agent.
Maven project structure:
changelogs/
databaseChangeLog.xml
pom.xml
Run command: mvn liquibase:update
databaseChangeLogs.xml contains next line: <includeAll path="changelogs/"/>
But build log contains duplication records:
liquibase: databaseChangeLog.xml: /home/teamcity/BuildAgent/work/28fe713da351c06d/changelogs/1.xml: ChangeSet /home/teamcity/BuildAgent/work/28fe713da351c06d/changelogs/1.xml ran successfully in 40ms
liquibase: databaseChangeLog.xml: Custom SQL executed
liquibase: databaseChangeLog.xml: changelogs/1.xml: ChangeSet changelogs/1.xml ran successfully in 36ms
So seems like Liquibase picked up changeset twice from different locations: from build agent's build folder and root of the project.
Does anybody meet the same issue?
Any ideas how to fix this?
Liquibase has kinda a design flaw which lays in considering at our first glance "identical" change sets as different. To bypass such a peculiarity you can use logicalFilePath attribute either on databaseChangeLog tag or on every changeSet tag. This one will add another level of identity, uniqueness to a your change set.