How to set gradle subproject artifact as task input? - gradle

My gradle build has a subproject with a task that produces a file
$ ./gradlew :strings:tokenizeStrings # creates strings/string_tokens.csv
then in my root project I have a task which consumes that file
tasks.generateLocalizationFiles {
inputTokensCsvFile.set(layout.projectDirectory.file("strings/string_tokens.csv"))
}
this works, but since gradle doesn't know about the dependency, it only works if I run the two tasks manually in the right order
$ ./gradlew :strings:tokenizeStrings
$ ./gradlew :generateLocalizationFiles
I want to add the proper dependency to gradle so that I can run just :generateLocalizationFiles and it will go into the subproject and do whatever it needs to. But I can't figure out the right way to do it.
What I've tried:
Following Simple sharing of artifacts between projects, I tried adding a consumable configuration to the suproject build script
val localizationData by configurations.creating {
isCanBeConsumed = true
isCanBeResolved = false
}
tasks.tokenizeStrings {
artifacts {
add("localizationData", outputTokensCsvFile) {
builtBy(this)
}
}
}
and then a resolvable configuration plus the dependency to the root project build script
val localizedStringData by configurations.creating {
isCanBeConsumed = false
isCanBeResolved = true
}
// hook up our resolvable configuration to the strings' consumable configuration
dependencies {
localizedStringData(project(mapOf(
"path" to ":strings",
"configuration" to "localizationData")
))
}
tasks.generateLocalizationFiles {
dependsOn(localizedStringData)
inputTokensCsvFile.set(localizedStringData.singleFile)
}
but that fails, seemingly because the consumable configuration is not populated?
Caused by: java.lang.IllegalStateException: Expected configuration ':localizedStringData' to contain exactly one file, however, it contains no files.

You need to add the outgoing artifact directly in the subproject build script, not inside the task configuration (which is only run lazily). You also don't need builtBy if you're using a RegularFileProperty for the artifact.
val localizationData by configurations.creating {
isCanBeConsumed = true
isCanBeResolved = false
}
artifacts {
add("localizationData", tasks.tokenizeStrings.flatMap { it.outputTokensCsvFile })
}
The trick is to use flatMap to lazily access the task. You should similarly use map when passing it to the task resolving the data. That allows for lazy task creation and implicitly tells gradle about the dependency between the two:
tasks.generateLocalizationFiles {
inputTokensCsvFile.set(localizedStringData.elements.map { it.first().asFile })
}
This still feels somewhat hacky, since it would be very clumsy if you wanted to repeat this for many artifacts, but it does seem to be the idiomatic way of doing it in gradle since it doesn't require any explicit dependency creation via builtBy/dependsOn.

Related

Print sorted list of dependencies for all projects in Gradle build

I want to create a complete list of dependencies for all projects in my Gradle build to track changes over time.
For this purpose, both the list of projects and the list of all dependencies (direct and transitive) must be sorted. Without both, the diff between releases is useless.
My first attempt was
fun listDependencies(project: Project, configName: String) {
val config = project.configurations.findByName(configName) ?: return
println()
println("$configName of module ${project.name}:")
config.allDependencies.map {
" ${it.group}:${it.name}:${it.version}"
}.sorted().forEach {
println(it)
}
}
fun listDependencies(project: Project) {
listDependencies(project, "compileClasspath")
listDependencies(project, "testCompileClasspath")
}
/** Create sorted list of dependencies per module and only for the compileClasspath and testCompileClasspath */
task("listDependencies") {
doLast {
println("Dependency list per project")
allprojects.sortedBy { it.name }
.forEach {
listDependencies(it)
}
}
}
In the output, most versions are null and all transitive dependencies are missing.
I had to force dependency resolution by changing the config.allDependencies above to
config.resolvedConfiguration.resolvedArtifacts.map {
" ${it.moduleVersion}"
}.sorted().forEach {
println(it)
}
This works but gives me a warning that I shouldn't call resolvedConfiguration (Resolving unsafe configuration resolution errors).
What do I have to change to run this task once after the configuration phase only when I ask for it by running Gradle with ./gradlew listDependencies?
We don't need this task in every build.

gradle: how to remove task dependency

Gradle makes it easy to add task dependencies at runtime.
Is it also possible to remove a task dependency after it has been added?
As a use case, let's say we want to run checkstyle in a separate pipeline step, not as part of the main build. So, we apply the checkstyle plugin, but that adds some tasks that depend on check. I would like to break that dependency, so that checkstyle only runs when explicitly run, not as part of build (which depends on check, which depends on checkstyle*).
To accomplish the above, I could run the main pipeline step as build -x checkstyleMain -x checkstyleTest.
Another workaround would be to set the enabled property of the checkstyle tasks to false.
But for this question, I am interested in a generic way to remove a task dependency so that the task graph is missing the edge between 2 tasks, like that dependency was never added.
The problem is not an easy one because:
removing a dependency before it was added won't have any effect
trying to remove a dependency on gradle.taskGraph.whenReady will probably fail (since it's no longer allowed to change the task graph.
A good answer would also be that this is for sure not possible, in which case I could raise a feature request with the Gradle team.
Update 1
I tried to convert #Chriki's groovy to kotlin, like this:
project.afterEvaluate {
tasks.check {
dependsOn -= tasks.find {
it.name.startsWith("checkstyle")
}
}
}
But this doesn't work. I get an error: "Removing a task dependency from a task instance is not supported."
Executing ./gradlew bar on the following self-contained build, will not run the foo task, even though it was declared as a dependency of bar (tested with Gradle 7.1):
task foo {
doFirst {
println 'foo'
}
}
task bar {
dependsOn foo
doFirst {
println 'bar'
}
}
project.afterEvaluate {
bar.dependsOn -= foo
}
In your case, you could hence add something like the following to your build.gradle to unconditionally remove the undesired Checkstyle task dependencies from the check task:
project.afterEvaluate {
check.dependsOn -= checkstyleMain
check.dependsOn -= checkstyleTest
}
Using the Kotlin DSL, the following works for my build script above:
project.afterEvaluate {
val bar = tasks.findByPath("bar")
if (bar != null) {
bar.setDependsOn(bar.dependsOn - listOf(tasks.findByPath("foo"), "foo"))
}
}
It’s important not to use the -= operator.
Please excuse the verbosity. I’m sure this can be written more concisely but I’m not fluent enough in Kotlin.
Thanks to great #Chriki answer, I've got mine as well. Leaving it here for others to use it.
I'm using quarkus and annotation processing. When upgraded to newer version, I had circular dependencies:
Circular dependency between the following tasks:
:kspKotlin
+--- :quarkusGenerateCode
| \--- :processResources
| \--- :kspKotlin (*)
\--- :quarkusGenerateCodeDev
\--- :processResources (*)
In order to break that, I used the following code in my build.gradle.kts:
project.afterEvaluate {
tasks.findByPath("application:quarkusGenerateCode")?.let {
it.setDependsOn(it.dependsOn.map { it as Provider<*> }.filter { it.get() !is ProcessResources })
}
tasks.findByPath("application:quarkusGenerateCodeDev")?.let {
it.setDependsOn(it.dependsOn.map { it as Provider<*> }.filter { it.get() !is ProcessResources })
}
}
If you have more than one module that has circular dependency, you may want to use this snippet, as it searches through all modules:
project.afterEvaluate {
getTasksByName("quarkusGenerateCode", true).forEach { task ->
task.setDependsOn(task.dependsOn.map { it as Provider<Task> }.filter { it.get().name != "processResources" })
}
getTasksByName("quarkusGenerateCodeDev", true).forEach { task ->
task.setDependsOn(task.dependsOn.map { it as Provider<Task> }.filter { it.get().name != "processResources" })
}
}
It assumes that provider is a type Task and searches the breaking dependency by name.

How to run JBoss TattleTale from inside Gradle build

I am in love with JBoss TattleTale. Typically, in my Ant builds, I follow the docs to define the Tattletale tasks and then run them like so:
<taskdef name="report"
classname="org.jboss.tattletale.ant.ReportTask"
classpathref="tattletale.lib.path.id"/>
...
<tattletale:report source="${src.dir]" destination="${dest.dir}"/>
I am now converting my builds over to Gradle and am struggling to figure out how to get Tattletale running in Gradle. There doesn't appear to be a Gradle-Tattletale plugin, and I'm not experienced enough with Gradle to contribute one. But I also know that Gradle can run any Ant plugin and can also executing stuff from the system shell; I'm just not sure how to do this in Gradle because there aren't any docs on this (yet).
So I ask: How do I run the Tattletale ReportTask from inside a Gradle build?
Update
Here is what the Gradle/Ant docs show as an example:
task loadfile << {
def files = file('../antLoadfileResources').listFiles().sort()
files.each { File file ->
if (file.isFile()) {
ant.loadfile(srcFile: file, property: file.name)
println " *** $file.name ***"
println "${ant.properties[file.name]}"
}
}
}
However, no where in here do I see how/where to customize this for Tattletale and its ReportTask.
The following is adapted from https://github.com/roguePanda/tycho-gen/blob/master/build.gradle
It bypasses ant and directly invokes the Tattletale Java class.
It was changed to process a WAR, and mandates a newer javassist in order to handle Java 8 features such as lambdas.
configurations {
tattletale
}
configurations.tattletale {
resolutionStrategy {
force 'org.javassist:javassist:3.20.0-GA'
}
}
dependencies {
// other dependencies here...
tattletale "org.jboss.tattletale:tattletale:1.2.0.Beta2"
}
task createTattletaleProperties {
ext.props = [reports:"*", enableDot:"true"]
ext.destFile = new File(buildDir, "tattletale.properties")
inputs.properties props
outputs.file destFile
doLast {
def properties = new Properties()
properties.putAll(props)
destFile.withOutputStream { os ->
properties.store(os, null)
}
}
}
task tattletale(type: JavaExec, dependsOn: [createTattletaleProperties, war]) {
ext.outputDir = new File(buildDir, "reports/tattletale")
outputs.dir outputDir
inputs.files configurations.runtime.files
inputs.file war.archivePath
doFirst {
outputDir.mkdirs()
}
main = "org.jboss.tattletale.Main"
classpath = configurations.tattletale
systemProperties "jboss-tattletale.properties": createTattletaleProperties.destFile
args([configurations.runtime.files, war.archivePath].flatten().join("#"))
args outputDir
}
The previous answers either are incomplete or excessively complicated. What I did was use the ant task from gradle which works fine. Let's assume your tattletale jars are beneath rootDir/tools/...
ant.taskdef(name: "tattleTaleTask", classname: "org.jboss.tattletale.ant.ReportTask", classpath: "${rootDir}/tools/tattletale-1.1.2.Final/tattletale-ant.jar:${rootDir}/tools/tattletale-1.1.2.Final/tattletale.jar:${rootDir}/tools/tattletale-1.1.2.Final/javassist.jar")
sources = "./src:./src2:./etcetera"
ant.tattleTaleTask(
source: sources,
destination: "tattleTaleReport",
classloader: "org.jboss.tattletale.reporting.classloader.NoopClassLoaderStructure",
profiles: "java5, java6",
reports: "*",
excludes: "notthisjar.jar,notthisjareither.jar,etcetera.jar"
){
}
So the above code will generate the report beneath ./tattleTaleReport. It's that simple. The annoyance is that the source variable only accepts directories so if there are jars present in those directories you do not wish to scan you need to add them to the excludes parameter.

Grails gradle "a task with that name already exists"

I'm trying to create a test task rule using the example provided in the grails gradle doc but I keep getting "a task with that name already exists" error.
My build script is as follows:
import org.grails.gradle.plugin.tasks.* //Added import here else fails with "Could not find property GrailsTestTask"
buildscript {
repositories {
jcenter()
}
dependencies {
classpath "org.grails:grails-gradle-plugin:2.0.0"
}
}
version "0.1"
group "example"
apply plugin: "grails"
repositories {
grails.central() //creates a maven repo for the Grails Central repository (Core libraries and plugins)
}
grails {
grailsVersion = '2.3.5'
groovyVersion = '2.1.9'
springLoadedVersion '1.1.3'
}
dependencies {
bootstrap "org.grails.plugins:tomcat:7.0.50" // No container is deployed by default, so add this
compile 'org.grails.plugins:resources:1.2' // Just an example of adding a Grails plugin
}
project.tasks.addRule('Pattern: grails-test-app-<phase>') { String taskName ->
println tasks //shows grails-test-app-xxxxx task. Why?
//if (taskName.startsWith('grails-test-app') && taskName != 'grails-test-app') {
// task(taskName, type: GrailsTestTask) {
// String testPhase = (taskName - 'grails-test-app').toLowerCase()
// phases = [testPhase]
// }
//}
}
Running $gradle grails-test-integration
or in fact anything of the form $gradle grails-test-app-xxxxxxxx yields the error "Cannot add task 'gradle grails-test-app-xxxxxxxx as a task with that name already exists".
Can someone please advise how I can resolve this error? Thanks.
If you don't mind overriding the task created by the plugin, you might want to try
task(taskName, type: GrailsTestTask, overwrite: true)
In general, when using task rules that can be called multiple times (for instance if you have multiple tasks depending on a task eventually added by your rules), I use the following test before actually creating the task:
if (tasks.findByPath(taskName) == null) {tasks.create(taskName)}
This will call the task() constructor only if this task name does not exists.

Extract specific JARs from dependencies

I am new to gradle but learning quickly. I need to get some specific JARs from logback into a new directory in my release task. The dependencies are resolving OK, but I can't figure out how, in the release task, to extract just logback-core-1.0.6.jar and logback-access-1.0.6.jar into a directory called 'lib/ext'. Here are the relevant excerpts from my build.gradle.
dependencies {
...
compile 'org.slf4j:slf4j-api:1.6.4'
compile 'ch.qos.logback:logback-core:1.0.6'
compile 'ch.qos.logback:logback-classic:1.0.6'
runtime 'ch.qos.logback:logback-access:1.0.6'
...
}
...
task release(type: Tar, dependsOn: war) {
extension = "tar.gz"
classifier = project.classifier
compression = Compression.GZIP
into('lib') {
from configurations.release.files
from configurations.providedCompile.files
}
into('lib/ext') {
// TODO: Right here I want to extract just logback-core-1.0.6.jar and logback-access-1.0.6.jar
}
...
}
How do I iterated over the dependencies to locate those specific files and drop them in the lib/ext directory created by into('lib/ext')?
Configurations are just (lazy) collections. You can iterate over them, filter them, etc. Note that you typically only want to do this in the execution phase of the build, not in the configuration phase. The code below achieves this by using the lazy FileCollection.filter() method. Another approach would have been to pass a closure to the Tar.from() method.
task release(type: Tar, dependsOn: war) {
...
into('lib/ext') {
from findJar('logback-core')
from findJar('logback-access')
}
}
def findJar(prefix) {
configurations.runtime.filter { it.name.startsWith(prefix) }
}
It is worth nothing that the accepted answer filters the Configuration as a FileCollection so within the collection you can only access the attributes of a file. If you want to filter on the dependency itself (on group, name, or version) rather than its filename in the cache then you can use something like:
task copyToLib(type: Copy) {
from findJarsByGroup(configurations.compile, 'org.apache.avro')
into "$buildSrc/lib"
}
def findJarsByGroup(Configuration config, groupName) {
configurations.compile.files { it.group.equals(groupName) }
}
files takes a dependencySpecClosure which is just a filter function on a Dependency, see: https://gradle.org/docs/current/javadoc/org/gradle/api/artifacts/Dependency.html

Resources