How can I make a sub-project copy a file that is produced by a sibling sub-project? All this with proper dependency management, and without assuming that any language-specific plugins (like the JavaPlugin) are used.
I have looked at the updated Gradle 6 draft Sharing artifacts between projects but it does not really answer that question.
My multi-project structure is something like:
top/
build.gradle
settings.gradle
producer/
build.gradle
myFile_template.txt
consumer/
build.gradle
I want a Copy-task in producer/build.gradle to copy+transform myFile_template.txt into $buildDir/target/myFile.txt and another Copy-task in consumer/build.gradle should further copy+transform that myFile.txt to a finalFile.txt.
Presumably a proper solution would be able to use task outputs.files or some such so that consumer/build.gradle does not need to explicitly mention the location of $buildDir/target/myFile.txt.
(I'm completely new to Gradle).
Gradle gives you lots of freedom but I prefer that projects only "share" with each other by Configurations and/or Artifacts. I feel that one project should never concern itself with another project's tasks and feel that the tasks are private to each project.
With this principle in mind you could do something like
project(':producer') {
configurations {
transformed
}
task transformTemplate(type: Copy) {
from 'src/main/template'
into "$buildDir/transformed"
filter(...) // transformation goes here
}
dependencies {
// file collection derived from a task.
// Any task which uses this as a task input will depend on the transformTemplate task
transformed files(transformTemplate)
}
}
project(':consumer') {
configurations {
producerTransformed
}
dependencies {
producerTransformed project(path: ':producer', configuration: 'transformed')
}
task transformProducer(type:Copy) {
from configurations.producerTransformed // this will create a task dependency
into ...
filter ...
}
}
Related
I want to run a specific task after EVERY build of my subprojects. I can go into each of my subprojects build.gradle.kts file and add the following
tasks.build {
finalizedBy("afterbuildtask")
}
However, this should be possible to do in my root project build.gradle.kts file right? I tried it by doing the following:
subprojects {
this.tasks.findByName("build")?.dependsOn("afterbuildtask")
}
But nothing happens. How can I achieve this?
You can't programatically execute tasks from other tasks in newer versions of Gradle.
Instead, you are supposed to declare task dependencies and Gradle will ensure they get executed in the correct order. But I think it's not what you want
Alternatively, you could move your logic into the doLast block in the build task. eg:
build {
doLast {
println("Copying...")
copy {
from 'source'
into 'target'
include '*.war'
}
println("completed!")
}
}
good coding! ¯_(ツ)_/¯
So let's say I have the following settings.gradle:
include 'math-server'
project(':math-server').projectDir = file('math/server')
include 'math-client-gen'
project(':math-client-gen').projectDir = file('math/client')
include 'ai'
Now I'd like to not commit the any of the files in math-client-gen (including the build.gradle) since those are generated by a build job in math-server:
// math/server/build.gradle
task generateClient(type:Exec) {
workingDir '.'
inputs.dir('./app')
inputs.dir('.')
outputs.dir('../client')
commandLine './client-generator/generate.sh'
}
The generate.sh leverages the openapi client generator for kotlin.
Now the ai project relies on the math-client-gen:
// ai/build.gradle
dependencies {
compile project(':math-client-gen')
}
Now I have currently found two suboptimal ways to make this work.
Option 1 is to run ./gradlew :math-server:generateClient before I'm able to run ./gradlew :ai:build. This sucks, since you cannot build ai on its own anymore.
Option 2 is to commit the files, which of course also isn't the way it should be.
I'm sure there is a better way to do it with gradle, but I just didn't manage to find it yet. As a compromise, I'd be willing to commit the generated math-client-gen/build.gradle if it doesn't work without that.
What's the best solution to this problem?
Note: I also saw something like:
implementation files(layout.buildDirectory.dir('classes')) {
builtBy 'compile'
}
in the docs, that looks promising, but i'd like to have it for an entire subproject and not just some source files if possible.
// ai/build.gradle
afterEvaluate {
tasks.build.dependsOn(project(":math-client").tasks["generateClient"])
}
To automate your first option.
I ended up committing the build.gradle of the math-client-gen and have this line there:
// math/client/build.gradle
compileKotlin {
dependsOn ':math-server:buildMathClient'
}
This ensures that the client is always generated when this project is listed as a dependency somewhere.
Then you can simply add the math-client-gen as a dependency in other projects, no additional steps required:
// ai/build.gradle
dependencies {
compile project(':math-client-gen')
}
I have two projects in a multi project build. One looks like this:
// project-a build.gradle
task generateJar(type: Exec) {
commandLine "command", "to", "generate", "jar"
}
task generateArtifact(type: Zip, dependsOn: generateJar) {
outputs.file("/path/to/generated/jar")
}
artifacts { archives generateArtifact }
And the other simply depends on it like so:
// project-b build.gradle
dependencies {
implementation project(':project-a')
}
My expectation is that when I run a build on project b, it will first try to run generateArtifact + generateJar from project a, then get the outputted jar for use in project b, however I just get a "package does not exist" error in :project-b:compileJava. Based on the output, it looks like it is not trying to run any tasks in project-a, almost like it doesn't recognize the dependency exists
It looks like you're mixing up project and task dependencies. While you correctly made project-b depend on project-a, that does not automatically mean your custom generateJar and generateArtifact tasks get executed. You still need to mark your JAR in project-a as an "outgoing" artifact or add it to the output of the main source set which you can depend on in the consuming project like so:
sourceSets {
main {
output.file(generateArtifact.outputFile, builtBy: generateArtifact)
}
}
With 'compile' split into 'implementation' and 'api' configuration the behaviour might have changed. Can you just try with 'compile'(its deprecated though) once, to validate this. If it works then it means as suggsted by #sschuberth you may have to handle to explicitly call the generateArtifact of :projecta when you build :projectb.
I have a multi-module project in Gradle
root
-- ProjectA
-- ProjectB
Both ProjectA and ProjectB use the application plugin to create a zip in "ProjectA/build/distributions" and "ProjectB/build/distributions" respectively.
Now I want to copy the two zip files into "root/build/distributions".
I have tried various approaches, e.g. adding this in the build.gradle of the root project:
subprojects {
task copyFiles(type: Copy) {
includeEmptyDirs = true
from "$buildDir/distributions"
include '*.zip'
into "$parent.buildDir/distributions"
}
copyFiles.dependsOn(build)
}
or just adding a task to the root project:
task copyFiles(type: Copy) {
from "ProjectA/build/distributions"
from "ProjectB/build/distributions"
include "*.zip"
into "$buildDir/distributions"
}
build.dependsOn(copyFiles)
However, in both cases, nothing happens. No file gets copied.
What am I doing wrong?
I can see two things you are doing wrong:
You have relative paths to the subprojects. This is discouraged as it means you will always have to invoke Gradle from the root folder. And if a Gradle daemon was started from somehere else, it will fail. You could fix it by using the rootDir property (e.g. `from "$rootDir/ProjectA/...") but there is a better way...
The other problem is that you have no dependencies from your copyFiles task in your root project to the required distZip tasks in the sub-projects. So if the distributions have not already been built previously, there are no guarantees that it will work (which it apparently doesn't).
To fix it, you can have a look at the question "Referencing the outputs of a task in another project in Gradle", which covers the more general use case of what you ask. There are currently two answers, both of which are good.
So in your case, you can probably do either this:
task copyFiles(type: Copy) {
from tasks.getByPath(":ProjectA:distZip").outputs
from tasks.getByPath(":ProjectB:distZip").outputs
into "$buildDir/distributions"
}
or this:
task copyFiles(type: Copy) {
subprojects {
from(tasks.withType(Zip)) {
include "*.zip"
}
}
into "$buildDir/distributions"
}
Gradle will implicitly make the copy task depend on the other tasks automatically, so you don't need to do that yourself.
Also note that the currently accepted answer to the question I referenced is about configuration variants, and this is probably the most correct way to model the relationships (see here for more documentation on the topic). But I prefer the simplicity of the direct access to the tasks over the verbose and arguably more complex way to model it through configurations. Let's hope it gets simpler in a later release.
I have a task that generates java sources and a set of jars from these sources (say, project a). I would like to export these jars to dependent projects (say, project b). So here's roughly what I have right now:
//a.gradle
configurations{
generatedJars
}
task generateJars(type: JavaExec) {
//generate jars ...
outputs.files += //append generated jars here
}
dependencies{
generatedJars generateJars.outputs.files
}
//b.gradle
dependencies{
project(path: ':a', configuration: 'generatedJars')
}
It works OK, except that adding generateJars.outputs.files as a dependency does not tell gradle that it has to run generateJars task when there are no jars generated yet. I have tried adding the task itself as a dependency hoping that it would work in the same way as it does when you add a jar/zip task to an artifact configuration (e.g. artifacts{ myJarTask }), but it throws an error telling me that I cannot do that. Of course I can inject the generateJars task somewhere in the build process before :b starts evaluating, but that's clumsy and brittle, so I would like to avoid it.
I feel like I should be adding the generated jars to artifacts{ ... } of the project, but I am not sure how to make them then visible to dependent projects. Is there a better way of achieving this?
Dependent projects (project b) will need to do setup IntelliJ IDEA module classpath to point to project a's generated jars. Something rather like this (pseudo-code):
//b.gradle
idea{
module{
scopes.COMPILE.plus += project(path: ':a', configuration: 'generatedJars').files
}
}
So far I have tried simply adding a project dependecy on :a's generatedJars in :b, but Idea plugin simply adds module :a as a module-dependency and assumes that it exports its generated jars (which is probably a correct assumption), therefore not adding the generated jars to :b's classpath.
Any help would be greatly appreciated!
First, do you need a separate configuration? That is, do you have clients of a that should not see the generated Jars? If not, you can add the generated Jars to the archives configuration, which will simplify things.
Second, the correct way to add the generated Jars to the configuration is (instead of the dependencies block):
artifacts {
generatedJars generateJars
}
This should make sure that the generateJars task gets run automatically when needed.
Third, I'd omit the += after outputs.files, although it might not make a difference. You should also add the necessary inputs.
Fourth, why do you need a JavaExec task to generate the Jars? Can you instead add the generated sources to some source set and let Gradle build them?
Fifth, IDEA doesn't have a concept corresponding to Gradle's project configuration dependencies. Either an IDEA module fully depends on another module, or not at all. You have two options: either use a module dependency and make the generated sources a source folder of the depended-on module (preferably both in the Gradle and the IDEA build), or pass the generated Jars as external dependencies to IDEA. In either case, you should probably add a task dependency from ideaModule to the appropriate generation task. If this still doesn't lead to a satisfactory IDEA setup, you could think about moving the generation of the Jars into a separate subproject.
For my use case, I had a C++ project which generated some native libraries which my java project needed to load in order to run.
In the project ':native' build.gradle:
task compile(type: Exec, group: 'build') {
dependsOn ...
outputs.files(fileTree('/some/build/directory') {
include 'mylib/libmy.so'
})
...
}
In project java application build.gradle:
configurations {
nativeDep
}
// Add dependency on the task that produces the library
dependencies {
nativeDep files(project(':native').tasks.findByPath('compile'))
}
// Unfortunately, we also have to do this because gradle will only
// run the ':native:compile' task if we needed the tasks inputs for another
// task
tasks.withType(JavaCompile) {
dependsOn ':native:compile'
}
run {
doFirst {
// Use the configuration to add our library to java.library.path
def libDirs = files(configurations.nativeDep.files.collect {it.parentFile})
systemProperty "java.library.path", libDirs.asPath
}
}