How to efficiently populate extra properties in the Gradle Kotlin DSL? - gradle

I'm migrating Gradle build scripts from Groovy to Kotlin DSL and one of the things which is not really documented is how to populate extra properties.
In Groovy, I can write:
ext {
comp = 'node_exporter'
compVersion = '0.16.0'
compProject = 'prometheus'
arch = 'linux-amd64'
tarball = "v${compVersion}/${comp}-${compVersion}.${arch}.tar.gz"
downloadSrc = "https://github.com/${compProject}/${comp}/releases/download/${tarball}"
unzipDir = "${comp}-${compVersion}.${arch}"
}
I figured out that in the Kotlin DSL, I can achive the same functionality with:
val comp by extra { "filebeat" }
val compVersion by extra { "6.4.0" }
val arch by extra { "linux-x86_64" }
val tarball by extra { "${comp}-${compVersion}-${arch}.tar.gz" }
val downloadSrc by extra { "https://artifacts.elastic.co/downloads/beats/${comp}/${tarball}" }
val unzipDir by extra { "${comp}-${compVersion}-${arch}" }
which looks pretty repetitive.
Implementation of ExtraPropertiesExtension in Kotlin is a little bit complex, but at the end, it holds just plain old Map<String, Object>.
So, my question: is it possible to populate extra object with multiple properties more easily, than just repeating val myProp by extra { "myValue"}?

According the current (5.2.1) documentation:
All enhanced objects in Gradle’s domain model can hold extra user-defined properties. This includes, but is not limited to, projects, tasks, and source sets.
Extra properties can be added, read and set via the owning object’s extra property. Alternatively, they can be addressed via Kotlin delegated properties using by extra.
Here is an example of using extra properties on the Project and Task objects:
val kotlinVersion by extra { "1.3.21" }
val kotlinDslVersion by extra("1.1.3")
extra["isKotlinDsl"] = true
tasks.register("printExtProps") {
extra["kotlinPositive"] = true
doLast {
// Extra properties defined on the Project object
println("Kotlin version: $kotlinVersion")
println("Kotlin DSL version: $kotlinDslVersion")
println("Is Kotlin DSL: ${project.extra["isKotlinDsl"]}")
// Extra properties defined on the Task object
// this means the current Task
println("Kotlin positive: ${this.extra["kotlinPositive"]}")
}
}

You don't have to use delegation, just write extra.set("propertyName", "propertyValue"). You can do this with an apply block if you want:
extra.apply {
set("propertyName", "propertyValue")
set("propertyName2", "propertyValue2")
}

Related

In gradle kotlin dsl, how to call a dynamic test extension?

I'm trying to add a new configuration option when using the gradle ScalaTest plugin:
https://github.com/maiflai/gradle-scalatest
In its source code, the config was injected into the Test class as a dynamic extension:
static void configure(Test test) {
...
Map<String, ?> config = [:]
test.extensions.add(ScalaTestAction.CONFIG, config)
test.extensions.add("config", { String name, value -> config.put(name, value) })
test.extensions.add("configMap", { Map<String, ?> c -> config.putAll(c) })
...
}
If using groovy as the dsl, calling this property is easy:
test {
configMap([
'db.name': 'testdb'
'server': '192.168.1.188'
])
}
unfortunately the kotlin dsl can't use this method due to static typing, when being invoked as a test plugin, it is clearly visible within the test scope, e.g. when using extensions.getByName:
tasks {
test {
val map = extensions.getByName("configMap")
println(map)
}
}
It yields the following output:
...
> Configure project :
com.github.maiflai.ScalaTestPlugin$_configure_closure6#45c21cac
But there is no way to retrieve or assert its type in compile time, and it ends up being useless (unless reflection is used, which is against the design philosophy of kotlin dsl). Is there a easy way for kotlin dsl to achieve the same?
I saw in the Scala test gradle plugin that the dynamic extension is defined like this:
test.extensions.add("configMap", { Map<String, ?> c -> config.putAll(c) })
The com.github.maiflai.ScalaTestPlugin$_configure_closure6#45c21cac you saw should be a closure of type (Map<String, Any>) -> Unit, which means you can do that. We'll have to change the map values so let's assume that it's also mutable.
extensions.getByName("configMap").closureOf<MutableMap<String, Any?>> {
this["db.name"] = "testdb"
this["server"] = "192.168.1.188"
}
This builds fine but I don't have Scala installed and never used Scala test. I have no idea if it actually works, so please tell me.

Quarkus Gradle plugin: overriding duplicate file entries coming from dependency libraries

Can I tell the Quarkus Gradle plugin (gradle quarkusDev or gradlew quarkusBuild -Dquarkus.package.uber-jar=true), to use resources provided by myself instead of choosing resources from dependency jars when they are duplicate?
I get these messages when building an uber-jar:
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.metadata-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.ner-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
These DKPro / uimaFIT libraries are NLP libraries that bring all their own META-INF/org.apache.uima.fit/types.txt file. You are supposed to merge these files yourself and adding your own types, and then only include this newly merged file in your uber-jar, or as first one in your classpath.
There is an option quarkus.package.user-configured-ignored-entries in application.properties, but it also removes my own provided files. So that's not what I want (see also https://github.com/quarkusio/quarkus/blob/master/core/deployment/src/main/java/io/quarkus/deployment/pkg/steps/JarResultBuildStep.java#L186 ). I haven't checked the sources of gradle quarkusDev, but it results in the same runtime exceptions.
For reference for other people using uimaFIT, this incorrect META-INF/org.apache.uima.fit/types.txt file results in an error like
org.apache.uima.analysis_engine.AnalysisEngineProcessException: JCas type "org.apache.uima.conceptMapper.support.tokenizer.TokenAnnotation" used in Java code, but was not declared in the XML type descriptor..
So my question is, how do I tell Gradle or Quarkus to use this file provided by myself instead of randomly choosing a file from a dependency jar?
The example Gradle script written in Kotlin DSL. The task generateNlpFiles and the function joinResources automatically generate Java source files from XML files in src/main/typesystem into build/generated/sources/jcasgen/main/, as required by uimaFIT, and joins the duplicate resources like META-INF/org.apache.uima.fit/types.txt into /generated/resources/uimafit/. You don't need to look at them too hard.
import java.io.FileOutputStream
import java.net.URLClassLoader
import org.apache.commons.io.IOUtils
plugins {
id("java")
id("io.quarkus")
id("eclipse")
}
repositories {
jcenter()
// required for downloading OpenNLP models
maven("https://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-releases/")
}
group = "com.example"
version = "0.0.0-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_11
java.targetCompatibility = JavaVersion.VERSION_11
dependencies {
val quarkusPlatformGroupId: String by project
val quarkusPlatformArtifactId: String by project
val quarkusPlatformVersion: String by project
// Quarkus dependencies
implementation(enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}"))
implementation("io.quarkus:quarkus-jaxb")
implementation("io.quarkus:quarkus-jackson")
implementation("io.quarkus:quarkus-resteasy")
implementation("io.quarkus:quarkus-jdbc-mariadb")
implementation("io.quarkus:quarkus-resteasy-jsonb")
implementation("io.quarkus:quarkus-smallrye-openapi")
implementation("io.quarkus:quarkus-container-image-docker")
// UIMA
implementation("org.apache.uima:uimaj-core:2.10.3")
implementation("org.apache.uima:ConceptMapper:2.10.2")
implementation("org.apache.uima:uimafit-core:2.4.0")
// DKPro
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.io.xmi-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.metadata-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.langdetect-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.icu-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-tagger-de-maxent:20120616.1")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-tagger-en-maxent:20120616.1")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-de-nemgp:20141024.1")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-en-location:20100907.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-en-organization:20100907.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-en-person:20130624.1")
// tests
testImplementation("io.quarkus:quarkus-junit5")
testImplementation("io.rest-assured:rest-assured")
// for generating NLP type system during compile time
compileOnly("org.apache.uima:uimaj-tools:2.10.4")
}
// joins resource files from classpath into single file
fun joinResources(classLoader: URLClassLoader, inputResourceName: String, outputFile: File) {
val outputStream = FileOutputStream(outputFile)
val resources = classLoader.findResources(inputResourceName).toList()
resources.forEach {
val inputStream = it.openStream()
IOUtils.copy(inputStream, outputStream)
outputStream.write('\n'.toInt());
inputStream.close()
}
outputStream.close()
}
// generate NLP type system from XML files and join uimaFIT files
val generateNlpFiles = task("generateNlpFiles") {
inputs.files(fileTree("src/main/typesystem"))
inputs.files(fileTree("src/main/resources"))
outputs.dir("${buildDir}/generated/sources/jcasgen/main/")
outputs.dir("${buildDir}/generated/resources/uimafit/")
val compileClasspath = project.sourceSets.main.get().compileClasspath
val runtimeClasspath = project.sourceSets.main.get().runtimeClasspath
val compileClassLoader = URLClassLoader(compileClasspath.map{ it.toURI().toURL() }.toTypedArray())
val runtimeClassLoader = URLClassLoader(runtimeClasspath.map{ it.toURI().toURL() }.toTypedArray())
// from XML files in src/main/typesystem/ generate Java sources into build/generated/sources/jcasgen/main/
val jCasGen = compileClassLoader.loadClass("org.apache.uima.tools.jcasgen.Jg").newInstance()
fileTree("src/main/typesystem").forEach() { typeSystemFile ->
doFirst {
// see https://github.com/Dictanova/gradle-jcasgen-plugin/blob/master/src/main/groovy/com/dictanova/jcasgen/gradle/JCasGenTask.groovy#L45
val jcasgeninput = "${typeSystemFile}"
val jcasgenoutput = "${buildDir}/generated/sources/jcasgen/main/"
val jcasgenclasspath = "${runtimeClasspath.asPath}"
val arguments: Array<String> = arrayOf("-jcasgeninput", jcasgeninput, "-jcasgenoutput", jcasgenoutput, "-jcasgenclasspath", jcasgenclasspath)
val main1 = jCasGen.javaClass.getMethod("main1", arguments.javaClass)
main1.invoke(jCasGen, arguments)
}
}
// collect types.txt and components.txt from classpath and join them in build/generated/resources/uimafit/META-INF/org.apache.uima.fit/
val uimafitDir = "${buildDir}/generated/resources/uimafit/META-INF/org.apache.uima.fit"
mkdir(uimafitDir)
joinResources(runtimeClassLoader, "META-INF/org.apache.uima.fit/types.txt", File("${uimafitDir}/types.txt"))
joinResources(runtimeClassLoader, "META-INF/org.apache.uima.fit/components.txt", File("${uimafitDir}/components.txt"))
}
eclipse {
project {
natures(
"org.eclipse.wst.common.project.facet.core.nature",
"org.eclipse.buildship.core.gradleprojectnature"
)
}
classpath {
file.withXml {
val attributes = mapOf("kind" to "src", "path" to "build/generated/sources/jcasgen/main")
this.asNode().appendNode("classpathentry", attributes)
}
}
}
tasks {
compileJava {
options.encoding = "UTF-8"
options.compilerArgs.add("-parameters") // was in original Quarkus Gradle file, not sure what this does
dependsOn(generateNlpFiles)
// add generated sources to source sets
sourceSets["main"].java.srcDir(file("${buildDir}/generated/sources/jcasgen/main/"))
sourceSets["main"].resources.srcDir(file("${buildDir}/generated/resources/uimafit/"))
}
compileTestJava {
options.encoding = "UTF-8"
}
"eclipse" {
dependsOn(generateNlpFiles)
}
}
One workaround would be using gradlew quarkusBuild -Dquarkus.package.uber-jar=true with entries in quarkus.package.user-configured-ignored-entries and adding my own files manually to the resulting jar, but that wouldn't work with gradle quarkusDev.
I am using Quarkus 1.3.2, as Quarkus 1.4.1 cannot handle multiple resource directories (see also https://github.com/quarkusio/quarkus/blob/master/devtools/gradle/src/main/java/io/quarkus/gradle/tasks/QuarkusDev.java#L391 ), as needed by my project.
I also tried to exclude files with some Gradle JarJar plugins, like https://github.com/shevek/jarjar , but couldn't get them running.
Right now, you can't, it will just take one from the jars providing it.
Could you create a feature request in our tracker: https://github.com/quarkusio/quarkus/issues/new?assignees=&labels=kind%2Fenhancement&template=feature_request.md&title= .
Sounds like something useful.
Thanks!

GString lazy evaluation in Kotlin DSL using gradle-git-properties plugin

I'm using Gradle 6.2.2 with this plugin: com.gorylenko.gradle-git-properties (version 2.2.2). I'm trying to "translate" the following snippet into Kotlin DSL:
gitProperties {
extProperty = "gitProps" // git properties will be put in a map at project.ext.gitProps
}
shadowJar {
manifest {
attributes(
"Build-Revision": "${ -> project.ext.gitProps["git.commit.id"]}" // Uses GString lazy evaluation to delay until git properties are populated
)
}
}
...but this what I've come up with so far:
gitProperties {
extProperty = "gitProps"
keys = listOf("git.branch", "git.build.host", "git.build.version", "git.commit.id", "git.commit.id.abbrev",
"git.commit.time", "git.remote.origin.url", "git.tags", "git.total.commit.count")
}
tasks {
withType<ShadowJar> {
manifest.attributes.apply {
put("Build-Revision", "${project.ext.properties["git.commit.id"]}")
}
}
}
I can't figure out to make the "GString lazy evaluation" part working in Kotlin DSL, nor how the gitProps map fits on here; eventually that approach (which I know it's partially wrong) is returning null. Any ideas?
The below Kotlin syntax worked for me:
put("Build-Revision", object {
override fun toString():String = (project.extra["gitProps"] as Map<String, String>)["git.commit.id"]!!
})
I think you have some confusion over where and how the data is being stored, and in particular when it's available.
I just got hold of this plugin and had a look at it: it supplies a project extension, which you're configuring to specify why extras property to populate, and a task: "generateGitProperties". This task is added as a dependency for the "classes" task, so it's already run once you get to "shadowJar"
The issue is that figuring out the git properties and populating the extra properties only happens when that task is executed, so they're not available when the build is configured, hence the need for the lazy GString shenanigans to pass a lazy value down into the shadowJar configuration that will only be evaluated once shadowJar executes.
You can get hold of the extra properties like this:
tasks.register("example") {
dependsOn("generateGitProperties")
doFirst {
val gitProps: Map<String, String> by project.ext
for ((name, value) in gitProps) {
println("GIT: $name -> $value")
}
}
}
That works because it's in a "doFirst" block, so it's happening at task execution time, not configuration time. So essentially, you could emulate the "lazy GString" stuff. Something like this:
withType<Jar>().configureEach {
val lazyCommitId = object {
override fun toString(): String {
val gitProps: Map<String, String> by project.ext
return gitProps["git.commit.id"] ?: ""
}
}
manifest {
attributes["Git-Commit-Id"] = lazyCommitId
}
}
I did this just for "jar", but "shadowJar" is just a subtype of a Jar task anyway.

How to configure gradle rule base model plugin in gradle kotlin DSL?

I'm using play-application plugin, which in turn use gradle rule based model configuration. build.gradle.kts looks like this:
plugins {
`play-application`
}
/* the snippet does not work
model {
components {
play {
platform play: playVersion, scala: scalaVersion, java: javaVersion
injectedRoutesGenerator = true
}
}
}
*/
// this works instead
apply(from = "play_setup.gradle")
val setup: groovy.lang.Closure<Any?> by extra
setup(project, jVersion, scalaVersion, playVersion)
where play_setup.gradle is:
ext.setup = { project, javaVersion, scalaVersion, playVersion ->
model {
components {
play {
platform play: playVersion, scala: scalaVersion, java: javaVersion
injectedRoutesGenerator = true
}
}
}
}
Is there a way to stop using groovy for plugins made with rule based model and configure them directly via kotlin-DSL?
Not according to the limitations listed in the Gradle Kotlin DSL Primer.
The Kotlin DSL will not support the model {} block, which is part of
the discontinued Gradle Software Model. However, you can apply model
rules from scripts — see the model rules sample for more information
The link to the model rules sample is broken in the documentation, but I fixed it above.

How to access variant.outputFileName in Kotlin

We've been using a snippet like this one to rename the APK file generated by our Gradle build:
android.applicationVariants.all { variant ->
variant.outputs.all {
outputFileName = "${variant.name}-${variant.versionName}.apk"
}
}
Source: https://developer.android.com/studio/build/gradle-plugin-3-0-0-migration#variant_output
I am now in the process of converting my build.gradle to build.gradle.kts, i. e. to the Gradle Kotlin DSL. This is one of the last missing pieces: I can't figure out how to access outputFileName.
According to the API docs it does not even seem to exist:
BaseVariant.getOutputs() returns a DomainObjectCollection<BaseVariantOutput> which provides the all method used in the snippet.
BaseVariantOutput extends OutputFile which extends VariantOutput but none of these has an outputFileName or any getters or setters of a matching name.
So, I suspect there is some advanced Groovy magic at work to make this work - but how do I get there in Kotlin?
A little simplified version of #david.mihola answer:
android {
/**
* Notes Impl: Use DomainObjectCollection#all
*/
applicationVariants.all {
val variant = this
variant.outputs
.map { it as com.android.build.gradle.internal.api.BaseVariantOutputImpl }
.forEach { output ->
val outputFileName = "YourAppName - ${variant.baseName} - ${variant.versionName} ${variant.versionCode}.apk"
println("OutputFileName: $outputFileName")
output.outputFileName = outputFileName
}
}
}
Browsing through the source code of the Android Gradle plugin, I think I found the answer - here we go:
We are actually dealing with objects of type BaseVariantOutputImpl and this class does have both these methods:
public String getOutputFileName() {
return apkData.getOutputFileName();
}
public void setOutputFileName(String outputFileName) {
if (new File(outputFileName).isAbsolute()) {
throw new GradleException("Absolute path are not supported when setting " +
"an output file name");
}
apkData.setOutputFileName(outputFileName);
}
Using this knowledge we can now:
import com.android.build.gradle.internal.api.BaseVariantOutputImpl
and then cast our target objects like so:
applicationVariants.all(object : Action<ApplicationVariant> {
override fun execute(variant: ApplicationVariant) {
println("variant: ${variant}")
variant.outputs.all(object : Action<BaseVariantOutput> {
override fun execute(output: BaseVariantOutput) {
val outputImpl = output as BaseVariantOutputImpl
val fileName = output.outputFileName
.replace("-release", "-release-v${defaultConfig.versionName}-vc${defaultConfig.versionCode}-$gitHash")
.replace("-debug", "-debug-v${defaultConfig.versionName}-vc${defaultConfig.versionCode}-$gitHash")
println("output file name: ${fileName}")
outputImpl.outputFileName = fileName
}
})
}
})
So, I guess: Yes, there is some Groovy magic at work, namely that Groovy's dynamic type system allows you to just access getOutputFileName and setOutputFileName (by way of the abbreviated outputImpl.outputFileName syntax, as in Kotlin) from your code, hoping they will be there at runtime, even if the compile time interfaces that you know about don't have them.
Shorter version using lambdas:
applicationVariants.all{
outputs.all {
if(name.contains("release"))
(this as BaseVariantOutputImpl).outputFileName = "../../apk/$name-$versionName.apk"
}
}
This will place APK into app/apk folder with name made of variant name and version code.
You can change the format of filename as you wish.
Important: it must be done only on release builds, because ".." in path corrupts debug build process with strange errors.
For libraryVariants it is possible to change output file name without accessing internal api:
libraryVariants.all {
outputs.all {
packageLibraryProvider {
archiveFileName.set("yourlibrary-${buildType.name}.aar")
}
}
}
For Kotlin KTS.
NOTE: This is considered a temporal solución, until a proper way to do it in KTS is released by Android team.
Working in AGP v7.1.2 it might work also in lower versions of AGP.
:app build.gradle
android {
// ...
this.buildOutputs.all {
val variantOutputImpl = this as com.android.build.gradle.internal.api.BaseVariantOutputImpl
val variantName: String = variantOutputImpl.name
val outputFileName = "custom-name-${variantName}.apk"
variantOutputImpl.outputFileName = outputFileName
}
}

Resources