Gradle7 Version Catalog: How to use it with buildSrc? - gradle

I am very excited about the incubating Gradle's version catalogs and have been experimenting with it. I’ve found that the information in my gradle/libs.versions.toml is accessible in the build.gradle.kts scripts for my app and utility-lib projects.
However, I am unable to use the content of the toml file for buildSrc/build.gradle.kts or the convention files.
The only way that I could build was to hard-code the dependencies into those files, as I did before the version catalog feature.
In the buildSrc folder, I created a settings.gradle.kts file and inserted the dependencyResolutionManagement code for versionCatalogs, which is pointing to the same file as for my app and utility-lib projects.
Based on the Gradle7 docs, it seems that sharing a version catalog with buildSrc and modules is possible… I’d appreciate a nudge into getting it to work with buildSrc, if possible.
Here is a simple sample project, which I created via gradle init: my-version-catalog
Thank you for your time and help,
Mike

With Gradle 7.3.3, it is possible. Note version catalogs are GA since Gradle 7.4
The code snippet assumes Gradle is at least 7.4, but if you need them prior that version, insert enableFeaturePreview("VERSION_CATALOGS") at the beginning of each settings.gradle.kts.
Using buildSrc
buildSrc/settings.gradle.kts
dependencyResolutionManagement {
versionCatalogs {
create("libs") {
from(files("../gradle/libs.versions.toml"))
}
}
}
buildSrc/build.gradle.kts
dependencies {
implementation(libs.gradleplugin.intellij) // <- the lib reference
}
You can even use the version catalog for plugins
gradle/libs.versions.toml
...
[plugins]
kotlin-jvm = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin" }
jetbrains-changelog = { id = "org.jetbrains.changelog", version.ref = "changelog-plugin" }
jetbrains-intellij = { id = "org.jetbrains.intellij", version.ref = "intellij-plugin" }
hierynomus-license = { id = "com.github.hierynomus.license", version.ref = "license-plugin" }
nebula-integtest = { id = "nebula.integtest", version.ref = "nebula-integtest-plugin" }
build.gradle.kts
plugins {
id("java")
alias(libs.plugins.kotlin.jvm)
alias(libs.plugins.nebula.integtest)
alias(libs.plugins.jetbrains.intellij)
alias(libs.plugins.jetbrains.changelog)
alias(libs.plugins.hierynomus.license)
}
Note for accessing the catalog within scripts, please refer to the below section, the trick is the same.
Using convention plugins and included build
In the main project include a the Gradle project that holds the convention plugins.
build.gradle.kts
includeBuild("convention-plugins") // here it's a subfolder
convention-plugins/settings.gradle.kts
dependencyResolutionManagement {
repositories {
gradlePluginPortal()
}
versionCatalogs {
create("libs") {
from(files("../gradle/libs.versions.toml"))
}
}
}
rootProject.name = "convention-plugins"
The trick to enable convention plugins to access the version catalog is split in two part, add an ugly implementation dependency that locate where the version catalog generated classes are located.
libs.javaClass.superclass.protectionDomain.codeSource.location
Then in the convention plugin refer to the libs extension via Project::the.
val libs = the<LibrariesForLibs>()
This is tracked by gradle/gradle#15383.
convention-plugins/build.gradle.kts
plugins {
`kotlin-dsl`
}
dependencies {
implementation(libs.gradleplugin.kotlin.jvm)
// https://github.com/gradle/gradle/issues/15383
implementation(files(libs.javaClass.superclass.protectionDomain.codeSource.location))
}
And in the actual convention plugin
import org.gradle.accessors.dm.LibrariesForLibs
plugins {
id("org.jetbrains.kotlin.jvm")
}
// https://github.com/gradle/gradle/issues/15383
val libs = the<LibrariesForLibs>()
dependencies {
detektPlugins(libs.bundles.kotlinStuff) // access catalog entries
}
The org.gradle.accessors.dm.LibrariesForLibs class is generated by gradle is somewhere in local gradle folder ./gradle/<version>/dependency-accessors/<hash>/classes
Quick note that older IntelliJ IDEA currently (2022.3) reports alias(libs.gradleplugin.thePlugin) as an error in the editor,
although the dependencies are correctly resolved.
This tracked by KTIJ-19369, the ticket indicates this is actually a bug in Gradle Kotlin DSL gradle/gradle#22797, and someone made a simple IntelliJ IDEA plugin to hide this error until resolved.

Brice, it looks like a can of worms to go down that path, particularly for my situation, where I'm trying to use a libs.version.toml file from an android project, but the custom plugin is of course from a java/kotlin project. I tried creating the libs file by hardwiring the path to the toml file in the custom plugin. It might work if both were java projects, but I never tried that since that's not what I'm after. The ideal solution would be for the plugin to use the libs file from the project it is applied to, but it looks like the version catalog needs to be created in the settings file, before you even have access to "Project", so that's why you would have to hardwire the path.
Short answer. No, but there are other techniques for a custom plugin to get project version data from the project it is applied to.

Related

Publishing test fixtures with submodules, gradle

I have been looking around for a way to include test fixtures in my gradle publications.
https://developer.android.com/studio/publish-library/configure-test-fixtures#kts suggests that it should work automatically so long as the project name is set correctly, which I have done in the settings.gradle file. This seems to solve the issue in the case of https://github.com/slackhq/EitherNet/issues/44.
For context, my project is built with several sub modules and I have defined a custom publication for each (I suspect this is the clue to the issue) as shown here:
subprojects {
// ... some repos and unimportant plugin applications
tasks {
register("prepareKotlinBuildScriptModel") {}
withType<BootJar> {
enabled = false // this is enabled in the jar I wish to be bootable
}
withType<Test> {
useJUnitPlatform()
}
getByName<Jar>("jar") {
enabled = true
archiveClassifier.set("")
}
}
publishing {
publications {
create<MavenPublication>(project.name) {
version = projectVersion
artifactId = tasks.jar.get().archiveBaseName.get()
groupId = "${projectGroup}.${rootProject.name}"
from(components["kotlin"])
}
}
}
For ref, this is currently what my module structure and build.gradle looks like for the module in question:
module structure
plugins {
id("java-test-fixtures")
id("java-library")
}
dependencies {
testFixturesApi(project(":model"))
... unrelated stuff
The test fixtures work fine as internal dependencies in the project itself, but they do not get published so that they can be used in external projects.
So my question is if there is a way to bake the test fixtures into my submodule jars so they can be used in external projects?
Any input would be highly appreciated.
Tried, expected, result:
Publishing to local repo, expected the test fixtures to be bundled, they were not.

How to create multi project fat jar as bundle of some libraries with buildSrc

First of all, sorry for my poor english.
Goal
I want create multi project containing some custom libraries as subproject with gradle.
For centralized dependency version control, using buildSrc and setting versions (spring-boot, detekt, ktlint etc.)
my-core-project(root)
+buildSrc
+-src/main/kotlin
+--int-test-convention.gradle.kts
+--library-convention.gradle.kts
+library-A
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+library-B
+-src
+--main/kotlin
+--test/kotlin
+-build.gradle.kts
+build.gradle.kts
+setting.gradle.kts
buildSrc contains common tasks for libraries(integration test, detekt, etc.)
library-A and library-B are custom libraries based on spring boot.
There is no application module or any main method.
my goal is using method of library-A and/or library-B with another separated project with adding my-core-project to dependency.
Problem
./gradlew build created 3 jar files
my-core-project
+build/libs
+-my-core-project.jar
+library-A
+-build/libs
+--library-A.jar
+library-B
+-build/libs
+--library-B.jar
copied 3 jar files to libs directory under project which actually using these library,
tried adding dependency created jar
with implementation(files("libs/library-A.jar")), class and methods are resolved well.
but with implementation(files("libs/my-core-project.jar")),
class and methods are not unresolved.
when check my-core-project.jar, recognized that any information of sub projects contained.
Here is my setting.gradle.kts and build.gradle.kts of root directory.
# setting.gradle.kts
pluginManagement {
repositories {
mavenCentral()
gradlePluginPortal()
}
}
rootProject.name = "my-core-project"
include(
"library-A",
"library-B"
)
# build.gradle.kts
plugins {
id("java-library")
id("io.spring.dependency-management")
}
group = "com.demo"
version = "0.0.1-SNAPSHOT"
dependencies {
api(project(":library-A"))
api(project(":library-B"))
}
repositories {
mavenCentral()
}
Tried things
In my opinion, my-core-project.jar should be fatJar(uberJar),
so i added FatJar task like this
val fatJar = task("fatJar", type = Jar::class) {
archiveBaseName.set("${project.name}-fat")
from(configurations.runtimeClasspath.get().map { if (it.isDirectory) it else zipTree(it) })
with(tasks.jar.get() as CopySpec)
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
tasks {
"build" {
dependsOn(fatJar)
}
}
but cannot resolve class and method,
additionally occurs version conflict with other dependency of projects using this package, due to library-A created as fatJar too.
Question
Is there a simple way packaging/bundling sub-modules into one jar file?
if there are tasks like this already in gradle, prefer to use that.
Modifying fatJar task like "add jar of submodules, not contained dependencies" can solve this problem?(even couldn't try completely newbie to gradle and kts.)
if so, can you show me how to modify task?
tried shadowJar already. that solved version-conflict problem with relocate option. but still couldn't resolve package in library-A
If structure has problem, is there a good practice/example for "bundle of library"?
thanks for reading.
TL;DR
If someone faced this problem, try set archive name shorter than current one.
For someone who faced same problem, sharing some informations.
as result, resolved this problem.(maybe even not problem)
current shadowJar configure is as following
tasks.named<ShadowJar>("shadowJar").configure {
archiveBaseName.set("shorten-name")
archiveClassifier.set("")
exclude("**/*.kotlin_metadata")
exclude("**/*.kotlin_builtins")
}
exclude kotlin_metadata, kotlin_builtins
set shorten name(original project name was 30 long characters)
I have no idea but shorten jar file name has no problem.
Interesting one is, upload in artifactory package with original(long) name worked well.
I don't know Gradle declaring dependency with files has length constraints.
implementation(files("path/to/package"))
And now it works well with original name with local jar package file.

Gradle 7.2 Custom Plugin: How to consume a version catalog within the apply(project: Project)?

I have a Gradle 7.2 custom plugin that is working properly. I want to configure it to use the new Gradle Version Catalog for dependency information. I know how to configure and use the version catalog, and have generated a shared jar for our shared dependencies. It is being read by a few other builds without any issues.
However, I cannot seem to find the correct incantation to consume the version catalog jar by the plugin when it is setting dependencies. I keep getting "Extension of type 'VersionCatalogsExtension' does not exist".
Here are two snippets showing what I have done to access the version catalog in the apply method:
override fun apply(project: Project) {
val libs = project
.extensions
.getByType(VersionCatalogsExtension::class.java)
.named("libs")
addBuildDependencies(libs)
.
.
.
private fun Project.addBuildDependencies(libs: VersionCatalog) {
dependencies.apply {
// BOMs
add(JavaPlugin.IMPLEMENTATION_CONFIGURATION_NAME, platform(libs.findDependency("arrow.bom").get()))
add(JavaPlugin.IMPLEMENTATION_CONFIGURATION_NAME, platform(libs.findDependency("detekt.bom").get()))
I'd appreciate any snippets or redirects.
Thanks,
Mike
How are you applying the plugin?
I noticed that if you're trying to do it from an allProjects or subprojects block it doesn't work, it fails with this error:
Extension of type 'VersionCatalogsExtension' does not exist. Currently registered extension types: ...
I assume this the same reason why we can't use libs directly inside those blocks, see this comment for more details.
A solution is to reference the custom plugin directly in the projects that will use it and avoid cross-configuration.
Please make sure you add this block of code in your settings.gradle.kts file
enableFeaturePreview("VERSION_CATALOGS")
// == Define locations for components ==
dependencyResolutionManagement {
repositories {
google()
mavenCentral()
gradlePluginPortal()
}
versionCatalogs {
create("libs") {
from(files("PATH/libs.versions.toml"))
}
}
}

How to centralize Gradle build settings?

Say I'm using the palantir/gradle-git-version Gradle plugin, and have the following code in build.gradle.kts to determine the project version:
// If release branch, return after incrementing patch version.
// Else, return $lastTag-SNAPSHOT.
val projectVersion: String by lazy {
val versionDetails: groovy.lang.Closure<VersionDetails> by extra
with(versionDetails()) {
if (!lastTag.matches("^(?:(?:\\d+\\.){2}\\d+)\$".toRegex())) {
throw GradleException("Tag '$lastTag' doesn't match 'MAJOR.MINOR.PATCH' format")
}
// If it detached state, get branch name from GitLab CI env var
val branch = branchName ?: System.getenv("CI_COMMIT_REF_NAME")
if (branch?.startsWith("release/") == true) {
val tokens = lastTag.split('.')
"${tokens[0]}.${tokens[1]}.${tokens[2].toInt() + commitDistance}"
} else "$lastTag-SNAPSHOT"
}
}
This works, but the code is duplicated across all the projects, which is difficult to maintain except for a very small number of projects.
This is just one example, the same applies for other Gradle tasks that assume certain conventions within the company/team, like creating a Dockerfile.
What is a good way to centralize such code so that all projects can use them? Note that code like this don't usually stand on their own, but rely on Gradle plugins.
What is a good way to centralize such code so that all projects can use them?
You'll want to create a custom Gradle plugin to hold your project's conventions.
If you have Gradle installed locally, you can use the Build Init Plugin to create a skeleton plugin project. With Gradle installed locally, simple run gradle init in a new project directory and follow the prompts to create the plugin project.
As a concrete example (assuming you generated a plugin project as mentioned earlier), to apply your versioning conventions, a plugin could be:
// Plugin's build.gradle.kts
dependencies {
// Add dependency for plugin, GAV can be found on the plugins page:
// https://plugins.gradle.org/plugin/com.palantir.git-version
implementation("com.palantir.gradle.gitversion:gradle-git-version:0.12.3")
}
Then a versioning conventions plugin could be:
import com.palantir.gradle.gitversion.VersionDetails
import groovy.lang.Closure
import org.gradle.api.GradleException
import org.gradle.api.Plugin
import org.gradle.api.Project
class VersioningConventionsPlugin : Plugin<Project> {
override fun apply(project: Project) {
// Apply plugin to project as you would in the main Gradle build file.
project.pluginManager.apply("com.palantir.git-version")
// Configure version conventions
val projectVersion: String by lazy {
// Gradle generates some Kotlin DSL code on the fly, in a plugin implementation we don't have that.
// So we must convert the DSL to the Gradle API.
val versionDetails: Closure<VersionDetails> = project.extensions.extraProperties.get("versionDetails") as Closure<VersionDetails>
with(versionDetails.call()) {
if (!lastTag.matches("^(?:(?:\\d+\\.){2}\\d+)\$".toRegex())) {
throw GradleException("Tag '$lastTag' doesn't match 'MAJOR.MINOR.PATCH' format")
}
val branch = branchName ?: System.getenv("CI_COMMIT_REF_NAME")
if (branch?.startsWith("release/") == true) {
val tokens = lastTag.split('.')
"${tokens[0]}.${tokens[1]}.${tokens[2].toInt() + commitDistance}"
} else "$lastTag-SNAPSHOT"
}
}
// Set the version as an extra property on the project
// Accessible via extra["projectVersion"]
project.extensions.extraProperties["projectVersion"] = projectVersion
}
}
I gave a Kotlin example since your sample used the Kotlin DSL. Once you've finished development work of your conventions plugin, then you would publish to a repository such as the Gradle Plugins repository. If it's an internal company plugin, then publish it to an internal Nexus Repository or similar.
Follow the docs for the maven-publish plugin for more details on publishing. Gradle plugins can be published like any other artifact/JAR.

Gradle: assemble output of a specific configuration

For a project, I have a custom configuration which should simply extend the default one, both in terms of java sources and in terms of dependencies.
This is what it looks like at the moment:
configurations {
// Tools inherits everything from default and adds on
toolsCompile.extendsFrom(compile)
}
sourceSets {
main {
java {
srcDirs = ['src']
}
}
tools {
java {
// Tools extends on the core sources
srcDirs = sourceSets.main.java.srcDirs + ['src-tools']
}
}
}
dependencies {
compile libs.a
compile libs.b
compile libs.c
// Tools adds on the dependencies of the default configuration
toolsCompile libs.d
toolsCompile libs.e
}
I know I could have also used sub-projects for this. I gave up on it, after trying, because I can't get it work properly together with the Eclipse integration plugin (it works fine when used from command line).
I have a couple of questions on the solution above:
Is the way I extend tools.java.srcDirs correct? Is there a more elegant way?
EDIT: Apparently it is not correct, as gradle eclipse generates a .classpath with a duplicate entry for src. Help please?
After I created my tools configuration, I know I can for example use it as a dependency from another project, as in compile project(path: ':myproject', configuration: 'tools'). What do I need to add if I instead want to get the output of the assemble task for my tools configuration? Do I have to make an explicit task for that? The task toolsClasses is created automatically, but not toolsAssemble or toolsBuild.

Resources