gradle submodules of submodules not built with :submodule:build syntax - gradle

I have the following project structure correctly found by gradle:
$ ./gradlew projects
Root project 'test-project'
\--- Project ':sub-1'
\--- Project ':sub-1:sub-2'
Which makes me believe that my setups is correct.
Now, I have found that the following gradle syntax:
$ ./gradlew clean :sub-1:build
is not equivalent to:
$ cd sub-1
$ ../gradlew clean build
$ cd ..
The above is equivalence is stated in many places on the gradle website.
Like HERE
Running the former, the result is incorrect:
> Task :sub-1:test
test.project.LibrarySuite > someLibraryMethod is always true PASSED
Running the latter, the result is correct:
> Task :sub-1:test
test.project.LibrarySuite > someLibraryMethod is always true PASSED
> Task :sub-1:sub-2:test
test.project.LibrarySuite > someLibraryMethod is always true PASSED
Please help me understand if I might have assumed or done anything wrong, or if it's a bug that should be raised to the gradle team.

You will find detailed description of how Gradle handles execution of tasks in a multiproject build setup here : https://docs.gradle.org/current/userguide/intro_multi_project_builds.html#sec:executing_a_multiproject_build , specially:
From a user’s perspective, multi-project builds are still collections
of tasks you can run. The difference is that you may want to control
which project’s tasks get executed. You have two options here:
Change to the directory corresponding to the subproject you’re
interested in and just execute gradle as normal.
Use a qualified task name from any directory, although this is usually
done from the root. For example: gradle :services:webservice:build
will build the webservice subproject and any subprojects it depends
on.
The first approach is similar to the single-project use case, but
Gradle works slightly differently in the case of a multi-project
build. The command gradle test will execute the test task in any
subprojects, relative to the current working directory, that have that
task. So if you run the command from the root project directory,
you’ll run test in api, shared, services:shared and
services:webservice. If you run the command from the services project
directory, you’ll only execute the task in services:shared and
services:webservice.
This explains how Gradle behaves in the two examples you gave in your question :
$ ./gradlew clean :sub-1:build
from the root project directory: you execute task clean, which will be executed for current project and each subprojects below, then :sub-1:build (with qualified task name) which executes build tasks ONLY for subproject sub1
Gradle execution log:
> Task :clean UP-TO-DATE
> Task :sub-1:clean
> Task :sub-1:sub-2:clean UP-TO-DATE
> Task :sub-1:compileJava NO-SOURCE
> Task :sub-1:processResources NO-SOURCE
> Task :sub-1:classes UP-TO-DATE
> Task :sub-1:jar
> Task :sub-1:assemble
> Task :sub-1:compileTestJava NO-SOURCE
> Task :sub-1:processTestResources NO-SOURCE
> Task :sub-1:testClasses UP-TO-DATE
> Task :sub-1:test NO-SOURCE
> Task :sub-1:check UP-TO-DATE
> Task :sub-1:build
EDIT to answer #Guido's comment: this will also build any other projects sub-1 depends on, so ./gradlew clean :sub-1:build will also trigger build of sub-2 if sub-1 project dependsOn sub-2:
sub-1 build.gradle
dependencies {
implementation project(":sub-1:sub-2")
}
$ cd sub-1 && ../gradlew clean build
From sub-1 subproject directory , you trigger tasks clean then build, without using qualified names, so these two tasks will both be executed on current project and sub-projects` :
Gralde output:
$ ../gradlew clean build --console=plain
> Task :sub-1:clean
> Task :sub-1:sub-2:clean
> Task :sub-1:compileJava NO-SOURCE
> Task :sub-1:processResources NO-SOURCE
> Task :sub-1:classes UP-TO-DATE
> Task :sub-1:jar
> Task :sub-1:assemble
> Task :sub-1:compileTestJava NO-SOURCE
> Task :sub-1:processTestResources NO-SOURCE
> Task :sub-1:testClasses UP-TO-DATE
> Task :sub-1:test NO-SOURCE
> Task :sub-1:check UP-TO-DATE
> Task :sub-1:build
> Task :sub-1:sub-2:compileJava NO-SOURCE
> Task :sub-1:sub-2:processResources NO-SOURCE
> Task :sub-1:sub-2:classes UP-TO-DATE
> Task :sub-1:sub-2:jar
> Task :sub-1:sub-2:assemble
> Task :sub-1:sub-2:compileTestJava NO-SOURCE
> Task :sub-1:sub-2:processTestResources NO-SOURCE
> Task :sub-1:sub-2:testClasses UP-TO-DATE
> Task :sub-1:sub-2:test NO-SOURCE
> Task :sub-1:sub-2:check UP-TO-DATE
> Task :sub-1:sub-2:build

Related

Ignore file when calculating gradle cache fingerprint

In my project I access the build-info.properties generated by the gradle springboot plugin buildInfo() task during runtime to include my project version in log metadata.
My problem is that this file is included in the fingerprint calculation for gradle tasks such as tests via the classpath fingerprint, but the version in that file changes with every build in my pipelines. Therefore I can never reuse that cache.
I saw this question on how to exclude that file from runtime, but if I follow that advise,
the file's not available during runtime anymore, naturally.
the file /BOOT-INF/classes/META-INF/build-info.properties is empty. Therefore my application fails on startup.
Can I somehow exclude it from the cache fingerprint calculation only?
Edits:
I'm currently on Gradle 6.8.1 and Spring Bot 2.2.2.
This is how I generate the file:
springBoot {
buildInfo()
}
And this is how I add the normalization:
normalization {
runtimeClasspath {
ignore "**/build-info.properties"
}
}
Update: As stated in the comment, this problem appeared due to a missconfiguration of my Gradle build scripts in another location. The normalization approach linked in the question and explained in the accepted answer is the solution to the initial question.
Gradle input normalization should be a solution for it.
normalization {
runtimeClasspath {
ignore '**/build-info.properties'
}
}
Not sure why you are saying " if I follow that advise, the file's not available during runtime anymore". According to documentation
The effect of this configuration would be that changes to build-info.properties would be ignored for up-to-date checks and build cache key calculations. Note that this will not change the runtime behavior of the test task — i.e. any test is still able to load build-info.properties and the runtime classpath is still the same as before.
Here are some tests that proves the above
Running build first time
./gradlew build -Pversion=0.0.1 --console=plain
> Task :bootBuildInfo
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes
> Task :resolveMainClassName
> Task :bootJar
> Task :jar
> Task :assemble
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources UP-TO-DATE
> Task :testClasses UP-TO-DATE
> Task :test
> Task :check
> Task :build
test task was executed because there is no build cache.
Running build second time with different version
./gradlew build -Pversion=0.0.2 --console=plain
> Task :bootBuildInfo
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes
> Task :resolveMainClassName UP-TO-DATE
> Task :bootJar
> Task :jar
> Task :assemble
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources UP-TO-DATE
> Task :testClasses UP-TO-DATE
> Task :test UP-TO-DATE
> Task :check UP-TO-DATE
> Task :build
As you may see only build tasks were executed but test task is still UP-TO-DATE.
build-info.properties is still available under build/resources/main/META-INF/
build.artifact=demo
build.group=com.example
build.name=demo
build.time=2023-02-01T18\:32\:03.871040Z
build.version=0.0.2
and could be accessed using Spring Boot actuator endpoint /actuator/info in case it's enabled
{
"build": {
"artifact": "demo",
"name": "demo",
"version": "0.0.2",
"group": "com.example"
}
}
Consider excluding build time
You could optimize even more by excluding time from the build info.
springBoot {
buildInfo {
excludes = ['time']
}
}
Usually it's a good idea for optimizing local builds. Otherwise build tasks will be always executed. By excluding time all tasks will be cached
./gradlew build --console=plain
> Task :bootBuildInfo UP-TO-DATE
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :resolveMainClassName UP-TO-DATE
> Task :bootJar UP-TO-DATE
> Task :jar UP-TO-DATE
> Task :assemble UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources UP-TO-DATE
> Task :testClasses UP-TO-DATE
> Task :test UP-TO-DATE
> Task :check UP-TO-DATE
> Task :build UP-TO-DATE
but build.time will not be part of the build-info.properties
build.artifact=demo
build.group=com.example
build.name=demo
build.version=0.0.1-SNAPSHOT

Gradle task output not reused in gitlab for kotlin build

I have created simple kotlin project with gradle build tool and pushed it to gitlab. To speed up pipeline I want to reuse tasks output. So I configured gitlab to cache build folder. Here is gitlab-ci.yml:
stages:
- build
build:
stage: build
image: gradle:7.5.1-jdk17
variables:
GRADLE_OPTS: "-Dorg.gradle.daemon=false"
cache:
- key: global-cache # gradle thirdparty artifacts cache
paths:
- /home/gradle/.gradle
- key: $CI_COMMIT_REF_NAME # gradle previous build task output cache
paths:
- build
script:
- ls build/libs
- gradle -version
- gradle build --console=plain
artifacts:
paths:
- build/libs/*.jar
reports:
junit:
- build/test-results/test/TEST-*.xml
expire_in: 1 week
when: always
In pipeline I have executed build job twice(second job run). Since build folder cached my expectation was all tasks will be UP-TO-DATE for second job. Unfortunately, both jobs have same task state:
> Task :compileKotlin
> Task :compileJava NO-SOURCE
> Task :processResources NO-SOURCE
> Task :classes UP-TO-DATE
> Task :jar
> Task :inspectClassesForKotlinIC
> Task :assemble
> Task :compileTestKotlin
> Task :compileTestJava NO-SOURCE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
> Task :check
> Task :build
To debug "jar" task I printed build/lib/ files to console:
$ ls build/libs
gradle-reuse-build-example.jar
You can notice build/libs have cached file inside. Do you have any ideas why jar task isn't UP-TO-DATE? How should I configurate gitlab/gradle to reuse previous job run task output?
Your build fetches the git repo again when running each time(because gitlab uses different shared runner for job execution). That means that the timestamps of the kotlin source files are later than the classes compiled in earlier builds. As a result, the compileKotlin task is not UP-TO-DATE, causing the jar task to also be out of date.
Documentation here: https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:how_does_it_work
As to how to configure the build to do what you want, you may need to look into how to reuse an existing git clone instead of cloning fresh each time. I don't even know if that's possible.

Publishing artifact only after tests were successful

I've been fighting this problem for a long time and genuinely out of ideas. In my project I use the maven-publish plugin for publishing artifact to my repository:
plugins {
kotlin("jvm") version "1.6.20"
id("maven-publish")
}
And also using Kotest to run tests:
tasks.withType<Test> {
useJUnitPlatform()
}
Obviously, I want to publish artifacts only if all tests passed, so I add the following:
tasks.publish {
dependsOn(tasks.test)
mustRunAfter(tasks.test)
}
However, when I run ./gradlew clean build test publish on the command line, the artifacts are always published before the tests run:
> Task :clean
> Task :processResources NO-SOURCE
> Task :processTestResources
> Task :generatePomFileFor«project-name»Publication
> Task :compileKotlin
> Task :compileJava NO-SOURCE
> Task :classes UP-TO-DATE
> Task :inspectClassesForKotlinIC
> Task :jar
> Task :assemble
> Task :generateMetadataFileFor«project-name»Publication
> Task :publish«project-name»PublicationTo«repository-name»Repository
> Task :compileTestKotlin
> Task :compileTestJava NO-SOURCE
> Task :testClasses
> Task :test
Is there something I'm missing?
Took me long enough. Apparently, the publish task is not the one that actually publishes the jar, but the publish«project-name»PublicationTo«repository-name»Repository. However, this task is not available until after configuration, so the best I could do is:
afterEvaluate {
tasks.getByName("publish«project-name»PublicationTo«repository-name»Repository").dependsOn(tasks.test)
}

Grails 4 - Gradle 'assemble' task not generating .war.original archive

I just upgraded my Grails web-app from version 3.2 to 4.0.
I have a provided dependency in build.gradle (fairly common configuration):
dependencies {
...
compile "org.springframework.boot:spring-boot-starter-logging"
compile "org.springframework.boot:spring-boot-autoconfigure"
compile "org.springframework.boot:spring-boot-starter-actuator"
provided "org.springframework.boot:spring-boot-starter-tomcat"
...
}
I’ve just noticed that by executing Gradle assemble task I don’t get the myApp.war.original archive anymore (which used to be build without all provided libs), but only the one including provided dependencies.
Am I missing something here? I'd really like to spare those ~4MB of jars in lib-provided folder.
Thanks in advance!
Update 1
Following #ck1's advice i changed provided dependency to providedCompile, but the result is the same.
Though I already use the war plugin, I noticed that the sequence of tasks initiated by assemble task is:
> Task :assetCompile
Finished Precompiling Assets
> Task :compileJava
> Task :compileGroovy
> Task :buildProperties
> Task :processResources
> Task :classes
> Task :compileWebappGroovyPages NO-SOURCE
> Task :compileGroovyPages
> Task :compileGsonViews
> Task :findMainClass
> Task :bootWar
> Task :war SKIPPED
> Task :assemble
So the war task is skipped in favor of the new bootWar task (not available in Gradle 3, used by Grails 3).
Any way to force it? Or is it something the plugin should already support?
Update 2
After some research, I added to build.gradle
war {
enabled = true
}
and was able to get the war task to execute:
> Task :assetCompile
Finished Precompiling Assets
> Task :compileJava
> Task :compileGroovy
> Task :buildProperties
> Task :processResources
> Task :classes
> Task :compileWebappGroovyPages NO-SOURCE
> Task :compileGroovyPages
> Task :compileGsonViews
> Task :findMainClass
> Task :bootWar
> Task :war // not skipped
> Task :assemble
I basically got to where I wanted to, i.e. get a .war archive without all the provided dependencies; differently from before though, not a pair of .war archives (myApp.war and myApp.war.original) but a single one named myApp.war not including the unneeded stuff.
But I'm still pretty much confused, as
Spring Boot's Gradle plugin documentation states bootWar is an extension of war.
The bootRepackage task has been replaced with bootJar and bootWar tasks for building executable jars and wars respectively. Both tasks extend their equivalent standard Gradle jar or war task, giving you access to all of the usual configuration options and behaviour.
But then Spring Boot 2.0 Migration Guide states war task is expected to be skipped:
The bootRepackage task has been replaced with bootJar and bootWar tasks for building executable jars and wars respectively. The jar and war tasks are no longer involved.
So again, what am I missing out?
You should replace provided with either the providedCompile or providedRuntime dependency configuration from the war plugin.
These two configurations have the same scope as the respective compile
and runtime configurations, except that they are not added to the WAR
archive.
Reference:
https://docs.gradle.org/4.10.2/userguide/war_plugin.html

Gradle : Copy different properties file depending on the environment and create jar

I am evaluating gradle for my spring boot project. Everything seems to work but here is where I am stuck. I have 2 properties file. One for prod i.e.:
application_prod.properties
and another for qa i.e.:
application_qa.properties
My requirement is such that while I build (create jar file) the project from gradle, I've to rename the properties file to
application.properties
and then build the jar file. As far as I know, gradle has a default build task. So here I've to override it such that it considers only the required properties file and rename it and then build depending on the environment.
How can I achieve this?
What you need to do is to override processResources configuration:
processResources {
def profile = (project.hasProperty('profile') ? project.profile : 'qa').toLowerCase()
include "**/application_${profile}.properties"
rename {
'application.properties'
}
}
With the following piece of code changed you will get the output below:
$ ./gradlew run -Pprofile=PROD
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:run
LOL
Profile: PROD
BUILD SUCCESSFUL
Total time: 3.63 secs
$ ./gradlew run -Pprofile=QA
:compileJava UP-TO-DATE
:processResources
:classes
:run
LOL
Profile: QA
BUILD SUCCESSFUL
Total time: 3.686 secs
$ ./gradlew run
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:run
LOL
Profile: QA
BUILD SUCCESSFUL
Total time: 3.701 secs
Demo is here.

Resources