Extend build.gradle - gradle

I want to add some extra properties to a build.gradle I can't change because it is symlinked to another repo which I can't change. I also can't change gradle.properties.
I want to add these properties to build.gradle:
ext {
androidxAppCompatVersion = '1.3.1'
grpcVersion = '1.41.0'
protobufVersion = '3.17.3'
}
Is there a way to make a build.gradle of your own and import the build.gradle I can't modify into it, then add these extra properties to it?

yes, in your build-override.gradle you have as below
ext {
androidxAppCompatVersion = '1.3.1'
grpcVersion = '1.41.0'
protobufVersion = '3.17.3'
}
apply from: ./build.gradle //orig build gradle

Related

Gradle version catalogue - share a version between multiple toml files

Currently I do something like following:
Toml Files
// tools.versions.toml
[versions]
kotlin = "1.7.10"
gradle = "7.2.1"
maven = "2.0"
// libs.versions.toml
[versions]
kotlin = "1.7.10"
[libraries]
kotlin = { module = "org.jetbrains.kotlin:kotlin-stdlib-jdk8", version.ref = "kotlin" }
Version Catalogue
versionCatalogs {
// would be nice to be able to define a version here and use them inside the toml files...
// val kotlin = "1.7.10"
create("tools") {
from(files("gradle/tools.versions.toml"))
}
create("libs") {
from(files("gradle/libs.versions.toml"))
}
// others...
}
Question
Is there a way to share a version definition in my setup and still keep the separate files? Doing all inside the versionCatalogs blog allows me to do this but only if I define everything there instead of the files...
I don't mind changing the toml files to gradle.kts files, but I could not find a way to merge multiple gradle.kts files - but maybe there's a way in this direction?
Note
I'm not looking for a different solution like buildSrc or composite builds, I'd like to use the gradle version catalogue.

Download Gradle dependencies with ".modules" files

I currently have an offline environment where I have all my dependencies (jar, aar & pom files). The thing is that I want to use Coil (image library) as a dependency in my Android project. This library requires kotlinx-coroutines-core to be also downloaded in my environment. I have been able to download all the required files except for .module file (Gradle Module Metadata), which is necessary because there are multiple variants of kotlinx-coroutines-core (versions for the JVM, JS and Native).
In other words, my code downloads kotlinx-coroutines-core-jvm-1.3.9.jar (with its POM) and kotlinx-coroutines-core-1.3.9.pom which is great but kotlinx-coroutines-core-1.3.9.module file is still missing and not sure how can I download it.
Here is my code based on this gist:
task copyDependencies() {
def name = "default"
def configuration = configurations.getByName(name)
copyJars(configuration)
copyPoms(configuration)
}
private void copyJars(Configuration configuration) {
File repoDir = new File(project.buildDir, 'repository')
configuration.resolvedConfiguration.resolvedArtifacts.each { artifact ->
def moduleVersionId = artifact.moduleVersion.id
File moduleDir = new File(repoDir, "${moduleVersionId.group.replace('.', '/')}/${moduleVersionId.name}/${moduleVersionId.version}")
GFileUtils.mkdirs(moduleDir)
GFileUtils.copyFile(artifact.file, new File(moduleDir, artifact.file.name))
}
}
private void copyPoms(Configuration configuration) {
def componentIds = configuration.incoming.resolutionResult.allDependencies.collect { it.selected.id }
def result = project.dependencies.createArtifactResolutionQuery()
.forComponents(componentIds)
.withArtifacts(MavenModule, MavenPomArtifact)
.execute()
for (component in result.resolvedComponents) {
def componentId = component.id
if (componentId instanceof ModuleComponentIdentifier) {
File repoDir = new File(project.buildDir, 'repository')
File moduleDir = new File(repoDir, "${componentId.group.replace('.', '/')}/${componentId.module}/${componentId.version}")
GFileUtils.mkdirs(moduleDir)
File pomFile = component.getArtifacts(MavenPomArtifact)[0].file
GFileUtils.copyFile(pomFile, new File(moduleDir, pomFile.name))
}
}
}
So my question is: How can I download .module files when downloading my dependencies? An example would be downloading this file.
To copy a configuration's files to a directory, you can do this:
task copyLibs(type: Copy) {
from configurations.runtimeClasspath
into file("$buildDir/repository")
}
This will copy everything including transitive dependencies.
Not need to write all that code yourself. Why do you want POMs and all that in your application's classpath? Just let Gradle sort it out for you.

Quarkus Gradle plugin: overriding duplicate file entries coming from dependency libraries

Can I tell the Quarkus Gradle plugin (gradle quarkusDev or gradlew quarkusBuild -Dquarkus.package.uber-jar=true), to use resources provided by myself instead of choosing resources from dependency jars when they are duplicate?
I get these messages when building an uber-jar:
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.metadata-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.ner-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
These DKPro / uimaFIT libraries are NLP libraries that bring all their own META-INF/org.apache.uima.fit/types.txt file. You are supposed to merge these files yourself and adding your own types, and then only include this newly merged file in your uber-jar, or as first one in your classpath.
There is an option quarkus.package.user-configured-ignored-entries in application.properties, but it also removes my own provided files. So that's not what I want (see also https://github.com/quarkusio/quarkus/blob/master/core/deployment/src/main/java/io/quarkus/deployment/pkg/steps/JarResultBuildStep.java#L186 ). I haven't checked the sources of gradle quarkusDev, but it results in the same runtime exceptions.
For reference for other people using uimaFIT, this incorrect META-INF/org.apache.uima.fit/types.txt file results in an error like
org.apache.uima.analysis_engine.AnalysisEngineProcessException: JCas type "org.apache.uima.conceptMapper.support.tokenizer.TokenAnnotation" used in Java code, but was not declared in the XML type descriptor..
So my question is, how do I tell Gradle or Quarkus to use this file provided by myself instead of randomly choosing a file from a dependency jar?
The example Gradle script written in Kotlin DSL. The task generateNlpFiles and the function joinResources automatically generate Java source files from XML files in src/main/typesystem into build/generated/sources/jcasgen/main/, as required by uimaFIT, and joins the duplicate resources like META-INF/org.apache.uima.fit/types.txt into /generated/resources/uimafit/. You don't need to look at them too hard.
import java.io.FileOutputStream
import java.net.URLClassLoader
import org.apache.commons.io.IOUtils
plugins {
id("java")
id("io.quarkus")
id("eclipse")
}
repositories {
jcenter()
// required for downloading OpenNLP models
maven("https://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-releases/")
}
group = "com.example"
version = "0.0.0-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_11
java.targetCompatibility = JavaVersion.VERSION_11
dependencies {
val quarkusPlatformGroupId: String by project
val quarkusPlatformArtifactId: String by project
val quarkusPlatformVersion: String by project
// Quarkus dependencies
implementation(enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}"))
implementation("io.quarkus:quarkus-jaxb")
implementation("io.quarkus:quarkus-jackson")
implementation("io.quarkus:quarkus-resteasy")
implementation("io.quarkus:quarkus-jdbc-mariadb")
implementation("io.quarkus:quarkus-resteasy-jsonb")
implementation("io.quarkus:quarkus-smallrye-openapi")
implementation("io.quarkus:quarkus-container-image-docker")
// UIMA
implementation("org.apache.uima:uimaj-core:2.10.3")
implementation("org.apache.uima:ConceptMapper:2.10.2")
implementation("org.apache.uima:uimafit-core:2.4.0")
// DKPro
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.io.xmi-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.metadata-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.langdetect-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.icu-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-tagger-de-maxent:20120616.1")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-tagger-en-maxent:20120616.1")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-de-nemgp:20141024.1")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-en-location:20100907.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-en-organization:20100907.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-en-person:20130624.1")
// tests
testImplementation("io.quarkus:quarkus-junit5")
testImplementation("io.rest-assured:rest-assured")
// for generating NLP type system during compile time
compileOnly("org.apache.uima:uimaj-tools:2.10.4")
}
// joins resource files from classpath into single file
fun joinResources(classLoader: URLClassLoader, inputResourceName: String, outputFile: File) {
val outputStream = FileOutputStream(outputFile)
val resources = classLoader.findResources(inputResourceName).toList()
resources.forEach {
val inputStream = it.openStream()
IOUtils.copy(inputStream, outputStream)
outputStream.write('\n'.toInt());
inputStream.close()
}
outputStream.close()
}
// generate NLP type system from XML files and join uimaFIT files
val generateNlpFiles = task("generateNlpFiles") {
inputs.files(fileTree("src/main/typesystem"))
inputs.files(fileTree("src/main/resources"))
outputs.dir("${buildDir}/generated/sources/jcasgen/main/")
outputs.dir("${buildDir}/generated/resources/uimafit/")
val compileClasspath = project.sourceSets.main.get().compileClasspath
val runtimeClasspath = project.sourceSets.main.get().runtimeClasspath
val compileClassLoader = URLClassLoader(compileClasspath.map{ it.toURI().toURL() }.toTypedArray())
val runtimeClassLoader = URLClassLoader(runtimeClasspath.map{ it.toURI().toURL() }.toTypedArray())
// from XML files in src/main/typesystem/ generate Java sources into build/generated/sources/jcasgen/main/
val jCasGen = compileClassLoader.loadClass("org.apache.uima.tools.jcasgen.Jg").newInstance()
fileTree("src/main/typesystem").forEach() { typeSystemFile ->
doFirst {
// see https://github.com/Dictanova/gradle-jcasgen-plugin/blob/master/src/main/groovy/com/dictanova/jcasgen/gradle/JCasGenTask.groovy#L45
val jcasgeninput = "${typeSystemFile}"
val jcasgenoutput = "${buildDir}/generated/sources/jcasgen/main/"
val jcasgenclasspath = "${runtimeClasspath.asPath}"
val arguments: Array<String> = arrayOf("-jcasgeninput", jcasgeninput, "-jcasgenoutput", jcasgenoutput, "-jcasgenclasspath", jcasgenclasspath)
val main1 = jCasGen.javaClass.getMethod("main1", arguments.javaClass)
main1.invoke(jCasGen, arguments)
}
}
// collect types.txt and components.txt from classpath and join them in build/generated/resources/uimafit/META-INF/org.apache.uima.fit/
val uimafitDir = "${buildDir}/generated/resources/uimafit/META-INF/org.apache.uima.fit"
mkdir(uimafitDir)
joinResources(runtimeClassLoader, "META-INF/org.apache.uima.fit/types.txt", File("${uimafitDir}/types.txt"))
joinResources(runtimeClassLoader, "META-INF/org.apache.uima.fit/components.txt", File("${uimafitDir}/components.txt"))
}
eclipse {
project {
natures(
"org.eclipse.wst.common.project.facet.core.nature",
"org.eclipse.buildship.core.gradleprojectnature"
)
}
classpath {
file.withXml {
val attributes = mapOf("kind" to "src", "path" to "build/generated/sources/jcasgen/main")
this.asNode().appendNode("classpathentry", attributes)
}
}
}
tasks {
compileJava {
options.encoding = "UTF-8"
options.compilerArgs.add("-parameters") // was in original Quarkus Gradle file, not sure what this does
dependsOn(generateNlpFiles)
// add generated sources to source sets
sourceSets["main"].java.srcDir(file("${buildDir}/generated/sources/jcasgen/main/"))
sourceSets["main"].resources.srcDir(file("${buildDir}/generated/resources/uimafit/"))
}
compileTestJava {
options.encoding = "UTF-8"
}
"eclipse" {
dependsOn(generateNlpFiles)
}
}
One workaround would be using gradlew quarkusBuild -Dquarkus.package.uber-jar=true with entries in quarkus.package.user-configured-ignored-entries and adding my own files manually to the resulting jar, but that wouldn't work with gradle quarkusDev.
I am using Quarkus 1.3.2, as Quarkus 1.4.1 cannot handle multiple resource directories (see also https://github.com/quarkusio/quarkus/blob/master/devtools/gradle/src/main/java/io/quarkus/gradle/tasks/QuarkusDev.java#L391 ), as needed by my project.
I also tried to exclude files with some Gradle JarJar plugins, like https://github.com/shevek/jarjar , but couldn't get them running.
Right now, you can't, it will just take one from the jars providing it.
Could you create a feature request in our tracker: https://github.com/quarkusio/quarkus/issues/new?assignees=&labels=kind%2Fenhancement&template=feature_request.md&title= .
Sounds like something useful.
Thanks!

In a custom task in my buildSrc folder: How do I determine the file path of the subproject it is called in?

If I define a custom gradle task in buildSrc: How do I find out the relative path to the project from which the task is called?
In my buildSrc folder, I have a custom task that creates a Enum our of my message.properties file:
open class GenerateEnumTask : DefaultTask() {
#get:Input
open var inputFolder: String = "src/main/resources"
#get:Input
open val targetFilePath: String = "src/generated/kotlin/MessageCode.kt"
#get:OutputFile
val enumFile = File(targetFilePath)
#TaskAction
fun generateEnum() {
...
}
#Internal
override fun getDescription() = "This task uses downloaded property files and creates an enum kotlin file"
}
I then want to make sure the enum is generated before code compilation.
So I put this in the subproject "core", where I need the Enum.
build.gradle.kts:
tasks {
val generateEnumTask by registering(GenerateEnumTask::class)
withType<KotlinCompile> {
kotlinOptions.jvmTarget = Versions.jvmTarget
dependsOn(generateEnumTask)
dependsOn(formatKotlin)
doFirst{
println("compile kotlin in core project")
}
}
}
This does indeed work if I run gradle compileKotlin directly from the subfolder of the core project.
However, if I run the same command from the root project, the code searches for a src folder in the root directory.
Ah, the answer was simple: DefaultTask inherits from AbstractTask, which has a reference to the project that the task was called in (getProject)
This works nicely:
open var targetFolder: String = this.project.file("src/main/resources").absolutePath

Is there a way to find direct dependencies defined for a gradle configuration

I want to extract the dependencies defined in a particular gradle configuration; My code is like :
project.configurations.myConfig.files.each { src ->
logger.debug "Extracting ${src.absolutePath} to ${to}"
project.copy {
eachFile { fileCopyDetails ->
logger.debug("Extracting file : ${fileCopyDetails.file.path}")
}
from project.zipTree(src)
into to
}
But this is extracting the ALL the files including dependencies defined in the pom files. My requirement is to just extract the first level dependencies as defined in dependencies{ myConfig ... }
Solution 1
I tried with setting transitive = false and it works but that breaks the build because we are removing the dependent libraries from classpath.
Solution 2
Tried with creating a new configuration which is copy of myConfig but set transitive = false; And it works
I'm looking for any better solution where I do not have to copy the configuration.
You can make a copy of a configuration, and then set the copy to non-transitive. This way you only have to specify the dependencies once:
// This will include dependencies from superconfigurations. If that is not
// what you want, use "copy()" instead.
def nonTransitiveMyConfig = configurations.myConfig.copyRecursive()
nonTransitiveMyConfig.transitive = false
nonTransitiveMyConfig.files.each { src ->
// ...
}
See:
copyRecursive()
copy()

Resources