I'm writing some multi-module Android app (actually in Kotlin) and I'd like to have some project-wide flags. I was thinking about setting those flags in Gradle and accessing them from my code. But I couldn't find how to do that. Is it possible?
You could generate a property file in Gradle which is added to your runtime classpath. Please adapt the following groovy/java to kotlin/android
ext {
someProperty = 'xxx'
}
dependencies {
compile files("$buildDir/generated/resources")
}
task generateResources {
// configure task inputs/outputs for up-to-date checking/caching
inputs.property("someProperty", someProperty)
outputs.dir "$buildDir/generated/resources"
doLast {
file("$buildDir/generated/resources/myprops.properties").text = "someProperty=$someProperty"
}
}
// important: wire the task into the DAG
compileJava.dependsOn generateResources
Then in your java code
InputStream in = MyClass.getClassLoader().getResourceAsStream("myprops.properties");
Properties props = new Properties()
props.load(in)
String someProperty = props getProperty("someProperty"); // xxx
Related
Can I tell the Quarkus Gradle plugin (gradle quarkusDev or gradlew quarkusBuild -Dquarkus.package.uber-jar=true), to use resources provided by myself instead of choosing resources from dependency jars when they are duplicate?
I get these messages when building an uber-jar:
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.metadata-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
Duplicate entry META-INF/org.apache.uima.fit/types.txt entry from de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.ner-asl::jar:1.10.0(runtime) will be ignored. Existing file was provided by de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.syntax-asl::jar:1.10.0(runtime)
These DKPro / uimaFIT libraries are NLP libraries that bring all their own META-INF/org.apache.uima.fit/types.txt file. You are supposed to merge these files yourself and adding your own types, and then only include this newly merged file in your uber-jar, or as first one in your classpath.
There is an option quarkus.package.user-configured-ignored-entries in application.properties, but it also removes my own provided files. So that's not what I want (see also https://github.com/quarkusio/quarkus/blob/master/core/deployment/src/main/java/io/quarkus/deployment/pkg/steps/JarResultBuildStep.java#L186 ). I haven't checked the sources of gradle quarkusDev, but it results in the same runtime exceptions.
For reference for other people using uimaFIT, this incorrect META-INF/org.apache.uima.fit/types.txt file results in an error like
org.apache.uima.analysis_engine.AnalysisEngineProcessException: JCas type "org.apache.uima.conceptMapper.support.tokenizer.TokenAnnotation" used in Java code, but was not declared in the XML type descriptor..
So my question is, how do I tell Gradle or Quarkus to use this file provided by myself instead of randomly choosing a file from a dependency jar?
The example Gradle script written in Kotlin DSL. The task generateNlpFiles and the function joinResources automatically generate Java source files from XML files in src/main/typesystem into build/generated/sources/jcasgen/main/, as required by uimaFIT, and joins the duplicate resources like META-INF/org.apache.uima.fit/types.txt into /generated/resources/uimafit/. You don't need to look at them too hard.
import java.io.FileOutputStream
import java.net.URLClassLoader
import org.apache.commons.io.IOUtils
plugins {
id("java")
id("io.quarkus")
id("eclipse")
}
repositories {
jcenter()
// required for downloading OpenNLP models
maven("https://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-releases/")
}
group = "com.example"
version = "0.0.0-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_11
java.targetCompatibility = JavaVersion.VERSION_11
dependencies {
val quarkusPlatformGroupId: String by project
val quarkusPlatformArtifactId: String by project
val quarkusPlatformVersion: String by project
// Quarkus dependencies
implementation(enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}"))
implementation("io.quarkus:quarkus-jaxb")
implementation("io.quarkus:quarkus-jackson")
implementation("io.quarkus:quarkus-resteasy")
implementation("io.quarkus:quarkus-jdbc-mariadb")
implementation("io.quarkus:quarkus-resteasy-jsonb")
implementation("io.quarkus:quarkus-smallrye-openapi")
implementation("io.quarkus:quarkus-container-image-docker")
// UIMA
implementation("org.apache.uima:uimaj-core:2.10.3")
implementation("org.apache.uima:ConceptMapper:2.10.2")
implementation("org.apache.uima:uimafit-core:2.4.0")
// DKPro
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.io.xmi-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.api.metadata-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.langdetect-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.icu-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-tagger-de-maxent:20120616.1")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-tagger-en-maxent:20120616.1")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-asl:1.10.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-de-nemgp:20141024.1")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-en-location:20100907.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-en-organization:20100907.0")
implementation("de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-model-ner-en-person:20130624.1")
// tests
testImplementation("io.quarkus:quarkus-junit5")
testImplementation("io.rest-assured:rest-assured")
// for generating NLP type system during compile time
compileOnly("org.apache.uima:uimaj-tools:2.10.4")
}
// joins resource files from classpath into single file
fun joinResources(classLoader: URLClassLoader, inputResourceName: String, outputFile: File) {
val outputStream = FileOutputStream(outputFile)
val resources = classLoader.findResources(inputResourceName).toList()
resources.forEach {
val inputStream = it.openStream()
IOUtils.copy(inputStream, outputStream)
outputStream.write('\n'.toInt());
inputStream.close()
}
outputStream.close()
}
// generate NLP type system from XML files and join uimaFIT files
val generateNlpFiles = task("generateNlpFiles") {
inputs.files(fileTree("src/main/typesystem"))
inputs.files(fileTree("src/main/resources"))
outputs.dir("${buildDir}/generated/sources/jcasgen/main/")
outputs.dir("${buildDir}/generated/resources/uimafit/")
val compileClasspath = project.sourceSets.main.get().compileClasspath
val runtimeClasspath = project.sourceSets.main.get().runtimeClasspath
val compileClassLoader = URLClassLoader(compileClasspath.map{ it.toURI().toURL() }.toTypedArray())
val runtimeClassLoader = URLClassLoader(runtimeClasspath.map{ it.toURI().toURL() }.toTypedArray())
// from XML files in src/main/typesystem/ generate Java sources into build/generated/sources/jcasgen/main/
val jCasGen = compileClassLoader.loadClass("org.apache.uima.tools.jcasgen.Jg").newInstance()
fileTree("src/main/typesystem").forEach() { typeSystemFile ->
doFirst {
// see https://github.com/Dictanova/gradle-jcasgen-plugin/blob/master/src/main/groovy/com/dictanova/jcasgen/gradle/JCasGenTask.groovy#L45
val jcasgeninput = "${typeSystemFile}"
val jcasgenoutput = "${buildDir}/generated/sources/jcasgen/main/"
val jcasgenclasspath = "${runtimeClasspath.asPath}"
val arguments: Array<String> = arrayOf("-jcasgeninput", jcasgeninput, "-jcasgenoutput", jcasgenoutput, "-jcasgenclasspath", jcasgenclasspath)
val main1 = jCasGen.javaClass.getMethod("main1", arguments.javaClass)
main1.invoke(jCasGen, arguments)
}
}
// collect types.txt and components.txt from classpath and join them in build/generated/resources/uimafit/META-INF/org.apache.uima.fit/
val uimafitDir = "${buildDir}/generated/resources/uimafit/META-INF/org.apache.uima.fit"
mkdir(uimafitDir)
joinResources(runtimeClassLoader, "META-INF/org.apache.uima.fit/types.txt", File("${uimafitDir}/types.txt"))
joinResources(runtimeClassLoader, "META-INF/org.apache.uima.fit/components.txt", File("${uimafitDir}/components.txt"))
}
eclipse {
project {
natures(
"org.eclipse.wst.common.project.facet.core.nature",
"org.eclipse.buildship.core.gradleprojectnature"
)
}
classpath {
file.withXml {
val attributes = mapOf("kind" to "src", "path" to "build/generated/sources/jcasgen/main")
this.asNode().appendNode("classpathentry", attributes)
}
}
}
tasks {
compileJava {
options.encoding = "UTF-8"
options.compilerArgs.add("-parameters") // was in original Quarkus Gradle file, not sure what this does
dependsOn(generateNlpFiles)
// add generated sources to source sets
sourceSets["main"].java.srcDir(file("${buildDir}/generated/sources/jcasgen/main/"))
sourceSets["main"].resources.srcDir(file("${buildDir}/generated/resources/uimafit/"))
}
compileTestJava {
options.encoding = "UTF-8"
}
"eclipse" {
dependsOn(generateNlpFiles)
}
}
One workaround would be using gradlew quarkusBuild -Dquarkus.package.uber-jar=true with entries in quarkus.package.user-configured-ignored-entries and adding my own files manually to the resulting jar, but that wouldn't work with gradle quarkusDev.
I am using Quarkus 1.3.2, as Quarkus 1.4.1 cannot handle multiple resource directories (see also https://github.com/quarkusio/quarkus/blob/master/devtools/gradle/src/main/java/io/quarkus/gradle/tasks/QuarkusDev.java#L391 ), as needed by my project.
I also tried to exclude files with some Gradle JarJar plugins, like https://github.com/shevek/jarjar , but couldn't get them running.
Right now, you can't, it will just take one from the jars providing it.
Could you create a feature request in our tracker: https://github.com/quarkusio/quarkus/issues/new?assignees=&labels=kind%2Fenhancement&template=feature_request.md&title= .
Sounds like something useful.
Thanks!
I'm using Gradle 6.2.2 with this plugin: com.gorylenko.gradle-git-properties (version 2.2.2). I'm trying to "translate" the following snippet into Kotlin DSL:
gitProperties {
extProperty = "gitProps" // git properties will be put in a map at project.ext.gitProps
}
shadowJar {
manifest {
attributes(
"Build-Revision": "${ -> project.ext.gitProps["git.commit.id"]}" // Uses GString lazy evaluation to delay until git properties are populated
)
}
}
...but this what I've come up with so far:
gitProperties {
extProperty = "gitProps"
keys = listOf("git.branch", "git.build.host", "git.build.version", "git.commit.id", "git.commit.id.abbrev",
"git.commit.time", "git.remote.origin.url", "git.tags", "git.total.commit.count")
}
tasks {
withType<ShadowJar> {
manifest.attributes.apply {
put("Build-Revision", "${project.ext.properties["git.commit.id"]}")
}
}
}
I can't figure out to make the "GString lazy evaluation" part working in Kotlin DSL, nor how the gitProps map fits on here; eventually that approach (which I know it's partially wrong) is returning null. Any ideas?
The below Kotlin syntax worked for me:
put("Build-Revision", object {
override fun toString():String = (project.extra["gitProps"] as Map<String, String>)["git.commit.id"]!!
})
I think you have some confusion over where and how the data is being stored, and in particular when it's available.
I just got hold of this plugin and had a look at it: it supplies a project extension, which you're configuring to specify why extras property to populate, and a task: "generateGitProperties". This task is added as a dependency for the "classes" task, so it's already run once you get to "shadowJar"
The issue is that figuring out the git properties and populating the extra properties only happens when that task is executed, so they're not available when the build is configured, hence the need for the lazy GString shenanigans to pass a lazy value down into the shadowJar configuration that will only be evaluated once shadowJar executes.
You can get hold of the extra properties like this:
tasks.register("example") {
dependsOn("generateGitProperties")
doFirst {
val gitProps: Map<String, String> by project.ext
for ((name, value) in gitProps) {
println("GIT: $name -> $value")
}
}
}
That works because it's in a "doFirst" block, so it's happening at task execution time, not configuration time. So essentially, you could emulate the "lazy GString" stuff. Something like this:
withType<Jar>().configureEach {
val lazyCommitId = object {
override fun toString(): String {
val gitProps: Map<String, String> by project.ext
return gitProps["git.commit.id"] ?: ""
}
}
manifest {
attributes["Git-Commit-Id"] = lazyCommitId
}
}
I did this just for "jar", but "shadowJar" is just a subtype of a Jar task anyway.
If I define a custom gradle task in buildSrc: How do I find out the relative path to the project from which the task is called?
In my buildSrc folder, I have a custom task that creates a Enum our of my message.properties file:
open class GenerateEnumTask : DefaultTask() {
#get:Input
open var inputFolder: String = "src/main/resources"
#get:Input
open val targetFilePath: String = "src/generated/kotlin/MessageCode.kt"
#get:OutputFile
val enumFile = File(targetFilePath)
#TaskAction
fun generateEnum() {
...
}
#Internal
override fun getDescription() = "This task uses downloaded property files and creates an enum kotlin file"
}
I then want to make sure the enum is generated before code compilation.
So I put this in the subproject "core", where I need the Enum.
build.gradle.kts:
tasks {
val generateEnumTask by registering(GenerateEnumTask::class)
withType<KotlinCompile> {
kotlinOptions.jvmTarget = Versions.jvmTarget
dependsOn(generateEnumTask)
dependsOn(formatKotlin)
doFirst{
println("compile kotlin in core project")
}
}
}
This does indeed work if I run gradle compileKotlin directly from the subfolder of the core project.
However, if I run the same command from the root project, the code searches for a src folder in the root directory.
Ah, the answer was simple: DefaultTask inherits from AbstractTask, which has a reference to the project that the task was called in (getProject)
This works nicely:
open var targetFolder: String = this.project.file("src/main/resources").absolutePath
I am creating basic custom tasks in Gradle and learning how to extend them to do more complicated actions (Learning from here: https://docs.gradle.org/current/userguide/tutorial_using_tasks.html).
One of my reference projects, which I am extending to learn Gradle looks something like this
// pmd config
pmd {
ignoreFailures = false
reportsDir = file("$globalOutputDir/pmd")
toolVersion = toolVersions.pmdVersion
}
repositories {
mavenCentral()
}
task listSubProjects{
doLast{
println 'Searching in root dir `'
}
}
My question is around the pmd and repositories sections and why they don't have a clear qualifier like "task" on them but my listSubProjects requires a task qualifier? Are these inherited tasks from plugins and don't need a task qualifier?
The blocks that you see are task extensions, also discussed here.
A plugin creator can define extensions to allow users to configure a plugin:
// plugin code
class GreetingPluginExtension {
// default value
String message = 'Hello from GreetingPlugin'
}
// plugin code
class GreetingPlugin implements Plugin<Project> {
void apply(Project project) {
// Add the 'greeting' extension object
def extension = project.extensions.create('greeting', GreetingPluginExtension)
// Add a task that uses configuration from the extension object
...
}
}
In project.extensions.create('greeting',... the greeting block to be used later in build.gradle files is defined.
Then in user build.gradle files
apply plugin: GreetingPlugin
// Configure the extension
greeting.message = 'Hi from Gradle'
// Same effect as previous lines but with different syntax
greeting {
message = 'Hi from Gradle'
}
Often the name of the extension is chosen to be the same as the plugin and/or the task, which can make things confusing.
I use vendor library to generate Java sources from an Xml. This source xml imports other xml which exists in some jar file. I know the coordinates of this Jar file. The vendor library is a blackbox to me but I know that it uses ThreadContextClassLoader to load the imports from a jar. However, it fails because it cannot find the imported xmls from the classpath/jars.
What is the gradle way of accomplishing this?
// body of gradle task
#TaskInput
void execute(IncrementalTaskInputs inputs) {
inputs.outOfDate { changes ->
// CodeGenerator is the vendor library
CodeGenerator generator = new CodeGenerator();
// call some setter methods to set the inputs.
//
generators.setXml(file("<path/to/the-file"))
generator.generate();
}
}
From my other answer we have ascertained that there's no option to set the classloader for the CodeGenerator. So, the only option is to have the jar with the xml files loaded by the same classloader as the CodeGenerator`.
Option 1: Add the jar to the buildscript classpath in a buildscript { ... } block
buildscript {
dependencies {
classpath 'com.group:jar-with-xmls:1.0'
}
}
Option 2: Add the jar to the buildscript classpath via buildSrc/build.gradle
dependencies {
compile 'com.vendor:code-generator:1.0'
runtime 'com.group:jar-with-xmls:1.0'
}
Does the CodeGenerator have a setClassloader(Classloader) method? You could likely use a Configuration and a URLClassloader. Eg:
configurations {
codeGenerator
}
dependencies {
codeGenerator 'foo:bar:1.0'
codeGenerator 'baz:biff:2.0'
}
task generateCode {
inputs.files configurations.codeGenerator
outputs.dir "$buildDir/codeGenerator"
doLast {
URL[] urls = configurations.codeGenerator.files.collect { it.toUri().toUrl() }
Classloader cl = new URLClassLoader(urls, null)
CodeGenerator generator = new CodeGenerator()
generator.setClassLoader(cl)
...
generator.generateTo("$buildDir/codeGenerator")
}
}
See a similar concept here
jar files are zip archives. You can open them as zipfiles using java.util.zip classes. In gradle, you might want to use ziptree to extract, as explained here:
http://mrhaki.blogspot.jp/2012/06/gradle-goodness-unpacking-archive.html