How do you use artifacts to segment build artifacts into custom configurations for distribution - gradle

I have a simple multi-module project and I'd like to create a distribution from it. I roughly would like the distro to look like
foo/
bar/
baz/
For each module, I'd like to declare that their artifacts (and dependencies) go into one of foo, bar, or baz directories in the distro.
The obvious thing doesn't seem to work, and the Gradle docs are vague about how exactly artifacts + configurations work, so I'm not sure what I've done wrong.
I have in the root build.gradle
configurations {
foo
bar
baz
}
Then in moduleA I put for its artifacts:
artifacts {
foo jar
}
And so on, for each module, its artifacts are set to the desired configuration.
These configurations are listed if I print them out ala configurations.each {c -> println(c.name) }, so they're created correctly, but if I try to iterate over the files or allArtifacts of my custom configurations, they're empty.
I'd previously done something like this by creating ancillary modules whose dependencies would be listed like:
dependencies {
foo "com.mypackage.group:module:1.0"
}
And then another module for bar, and another for baz and so on. This seemed to work but it was a hassle to maintain. Using custom configurations along with artifacts seemed like a more maintainable solution for this.

I think you are interested in the getArtifacts()
Here's an example
// generated with gradle init --type java-library
apply plugin: 'java'
repositories {
jcenter()
}
dependencies {
compile 'org.slf4j:slf4j-api:1.7.21'
testCompile 'junit:junit:4.12'
}
configurations {
foo {
extendsFrom compile
}
}
artifacts {
foo jar
}
task showMe << {
println "Files:"
configurations.foo.files.each { println " $it" }
println "Artifacts: "
configurations.foo.artifacts.each { println " $it" }
}
Output:
:showMe
Files:
/home/alpar/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.21/139535a69a4239db087de9bab0bee568bf8e0b70/slf4j-api-1.7.21.jar
Artifacts:
ArchivePublishArtifact_Decorated alpar:jar:jar:
You could potentially simplify it even more by referring to the projects tasks directly ( considering that foo bar etc are projects )
task distribution(type: Zip) {
from project(':foo').jar { into "foo" }
}
or
task distribution(type: Zip) {
('foo', 'bar').each { p ->
from project(":$p").jar { into p }
}
}
for short.
It seems like the publish PublishArtifact was not ment to be used with a Zip task, as it's non trivial to connect.
You could also loop trough all sub projects that have some custom parameter defined for the name of the task to pick up to make it more self contained in the sub-projects.
Make sure to also include some dependencies to make your task work even if called after a complete clean, or on a fresh checkout.

Related

How to set gradle subproject artifact as task input?

My gradle build has a subproject with a task that produces a file
$ ./gradlew :strings:tokenizeStrings # creates strings/string_tokens.csv
then in my root project I have a task which consumes that file
tasks.generateLocalizationFiles {
inputTokensCsvFile.set(layout.projectDirectory.file("strings/string_tokens.csv"))
}
this works, but since gradle doesn't know about the dependency, it only works if I run the two tasks manually in the right order
$ ./gradlew :strings:tokenizeStrings
$ ./gradlew :generateLocalizationFiles
I want to add the proper dependency to gradle so that I can run just :generateLocalizationFiles and it will go into the subproject and do whatever it needs to. But I can't figure out the right way to do it.
What I've tried:
Following Simple sharing of artifacts between projects, I tried adding a consumable configuration to the suproject build script
val localizationData by configurations.creating {
isCanBeConsumed = true
isCanBeResolved = false
}
tasks.tokenizeStrings {
artifacts {
add("localizationData", outputTokensCsvFile) {
builtBy(this)
}
}
}
and then a resolvable configuration plus the dependency to the root project build script
val localizedStringData by configurations.creating {
isCanBeConsumed = false
isCanBeResolved = true
}
// hook up our resolvable configuration to the strings' consumable configuration
dependencies {
localizedStringData(project(mapOf(
"path" to ":strings",
"configuration" to "localizationData")
))
}
tasks.generateLocalizationFiles {
dependsOn(localizedStringData)
inputTokensCsvFile.set(localizedStringData.singleFile)
}
but that fails, seemingly because the consumable configuration is not populated?
Caused by: java.lang.IllegalStateException: Expected configuration ':localizedStringData' to contain exactly one file, however, it contains no files.
You need to add the outgoing artifact directly in the subproject build script, not inside the task configuration (which is only run lazily). You also don't need builtBy if you're using a RegularFileProperty for the artifact.
val localizationData by configurations.creating {
isCanBeConsumed = true
isCanBeResolved = false
}
artifacts {
add("localizationData", tasks.tokenizeStrings.flatMap { it.outputTokensCsvFile })
}
The trick is to use flatMap to lazily access the task. You should similarly use map when passing it to the task resolving the data. That allows for lazy task creation and implicitly tells gradle about the dependency between the two:
tasks.generateLocalizationFiles {
inputTokensCsvFile.set(localizedStringData.elements.map { it.first().asFile })
}
This still feels somewhat hacky, since it would be very clumsy if you wanted to repeat this for many artifacts, but it does seem to be the idiomatic way of doing it in gradle since it doesn't require any explicit dependency creation via builtBy/dependsOn.

Execute gradle task on sub projects

I have a MultiModule gradle project that I am trying to configure.
Root
projA
projB
other
projC
projD
projE
...
What I want to be able to do is have a task in the root build.gradle which will execute the buildJar task in each of the projects in the other directory.
I know I can do
configure(subprojects.findAll {it.name != 'tropicalFish'}) {
task hello << { task -> println "$task.project.name"}
}
But this will also get projA and projB, I want to only run the task on c,d,e...
Please let me know the best way to achieve this.
Not entirely sure which of these you're after, but they should cover your bases.
1. Calling the tasks directly
You should just be able to call
gradle :other/projC:hello :other/projD:hello
I tested this with:
# Root/build.gradle
allprojects {
task hello << { task -> println "$task.project.name" }
}
and
# Root/settings.gradle
include 'projA'
include 'projB'
include 'other/projC'
include 'other/projD'
2. Only creating tasks in the sub projects
Or is it that you only want the task created on the other/* projects?
If the latter, then the following works:
# Root/build.gradle
allprojects {
if (project.name.startsWith("other/")) {
task hello << { task -> println "$task.project.name" }
}
}
and it can then be called with:
$ gradle hello
:other/projC:hello
other/projC
:other/projD:hello
other/projD
3. Creating a task that runs tasks in the subprojects only
This version matches my reading of your question meaning there's already a task on the subprojects (buildJar), and creating a task in root that will only call the subprojects other/*:buildJar
allprojects {
task buildJar << { task -> println "$task.project.name" }
if (project.name.startsWith("other/")) {
task runBuildJar(dependsOn: buildJar) {}
}
}
This creates a task "buildJar" on every project, and "runBuildJar" on the other/* projects only, so you can call:
$ gradle runBuildJar
:other/projC:buildJar
other/projC
:other/projC:runBuildJar
:other/projD:buildJar
other/projD
:other/projD:runBuildJar
Your question can be read many ways, hope this covers them all :)
All of the ways mentioned by Mark can be used but all of them have some cons. So I am adding one more option:
4. Switching the current project
gradle -p other hello
This switches the "current project" and then runs all tasks named hello under the current project.
Example 5. Defining common behavior of all projects and subprojects,
allprojects {
task hello {
doLast { task ->
println "I'm $task.project.name"
}
}
}
subprojects {
hello {
doLast {
println "- I depend on water"
}
}
}
From the Gradle documentation,
https://docs.gradle.org/current/userguide/multi_project_builds.html

Can a Gradle plugin modify the list of subprojects in a multi-module project?

I've hacked together combination of build.gradle and settings.gradle below for creating an ad-hoc multi-module project out of several single-module projects (e.g., an application and all of its dependencies, or a shared library and everything that uses that library).
settings.gradle:
// find all subprojects and include them
rootDir.eachFileRecurse {
if (it.name == "build.gradle") {
def projDir = it.parentFile
if (projDir != rootDir) {
include projDir.name
project(":${projDir.name}").projectDir = projDir
}
}
}
build.gradle::
// Make sure we've parsed subproject dependencies
evaluationDependsOnChildren()
// Map of all projects by artifact group and name
def declarationToProject = subprojects.collectEntries { p -> [toDeclaration(p), p] }
// Replace artifact dependencies with subproject dependencies, if possible
subprojects.each { p ->
def changes = [] // defer so we don't get ConcurrentModificationExceptions
p.configurations.each { c ->
c.dependencies.each { d ->
def sub = declarationToProject[[group:d.group, name:d.name]]
if (sub != null) {
changes.add({
c.dependencies.remove(d)
p.dependencies.add(c.name, sub)
})
}
}
}
for (change in changes) {
change()
}
}
This works, but it's hard to share -- if somebody else wants to do something similar they have to copy my *.gradle files or cut and paste.
What I'd like to do is take this functionality and encapsulate it in a plugin. The build.gradle part looks easy enough to do in the plugin apply() method, but it seems like the list of subprojects is already set in stone before the plugin gets a chance at it. Is there any way to get in earlier in the build process, e.g. by applying to something other than Project? Or should I resign myself to giving my plugin a task for overwriting settings.gradle?
Solution: Per Peter Niederweiser's answer, I moved the code above into two plugins, one to be called from settings.gradle and the other to be called from build.gradle. In settings.gradle:
buildscript {
repositories { /* etc... */ }
dependencies { classpath 'my-group:my-plugin-project:1.0-SNAPSHOT' }
}
apply plugin: 'find-subprojects'
And in build.gradle:
buildscript {
repositories { /* etc... */ }
dependencies { classpath 'my-group:my-plugin-project:1.0-SNAPSHOT' }
}
evaluationDependsOnChildren()
apply plugin: 'local-dependencies'
Note that calling the plugin from settings.gradle doesn't work in Gradle 1.11 or 1.12 but does work in Gradle 2.0.
You'd need to apply a plugin in settings.gradle, which I believe is supported in recent versions.

How can I access the source artifacts of my gradle compile dependencies?

I have written the following task, which extracts all the compile dependencies for each of my sub-projects and puts them in a per sub-project directory:
task exportCompileLibs << {
subprojects.each { iSubProject ->
iSubProject.configurations.findAll{it.name == "compile"}.each{ jConfig ->
println "copying compile libs for ${iSubProject.name}..."
copy {
into "${iSubProject.buildDir}/gradle-lib-export"
from jConfig
eachFile {println it.name}
}
}
}
}
I'd like to extend this to also export the source artifacts that Gradle does already know about (I can see the source jars in the cache directory), I just can't figure out how to use the object model to get a handle to them.
The IDEA and Eclipse plugins seem to be able to do this (they point the project files they build directly into the gradle cache), but I can't figure out how to do it - and looking at the IDE plugin source code, it looks... tricky. I'm hoping there's something obvious that I'm missing in the gradle DSL or API.
Anyone got any ideas?
For anyone looking for at least an interim solution to this, the following seems to be doing pretty much exactly what I want at the moment.
You have to apply the IDEA plugin to the build.gradle file for each project you want to export the dependencies for:
apply plugin: 'idea'
And then define this task:
task exportDependencies << {
def deps = project.extensions.getByType(IdeaModel).module.resolveDependencies()
copy {
from deps*.classes.file
into "${buildDir}/gradle-lib-export/libs"
}
copy {
from deps*.sources.file
into "${buildDir}/gradle-lib-export/sources"
}
}
And here's my horrific hack so I don't have to apply the plugin for each sub-project:
task exportDependencies(description: "export project dependency jars") << {
subprojects.each { Project iSubProject ->
String target = "${iSubProject.buildDir}/gradle-lib-export"
IdeaPlugin ideaPlugin = new IdeaPlugin()
ideaPlugin.apply(iSubProject)
Set<Dependency> deps = ideaPlugin.model.module.resolveDependencies()
println "exporting dependencies for $iSubProject.name into $target"
copy {
from deps*.classes.file
into "${target}/libs"
eachFile { println "lib -> $it.name" }
}
copy {
from deps*.sources.file
into "${target}/sources"
eachFile{ println "source -> $it.name" }
}
}
}
+10 points for not cluttering my build task list with stuff I don't want, -several million points for ewwwww.
There isn't currently a simpler way than what the IDE plugins are doing. This will hopefully change in the future.

how can I override source dirs in gradle BUT only for this one annoying subproject

We have one main project and two subprojects. One of the subprojects is the playframework which has a "unique" build structure. How can I override the source directories BUT only for that one subproject such that all other projects are using the standard layout of source directories src/main/java, etc.
I tried the first answer which is not working and for my directory structure
stserver
build.gradle (1)
project1
webserver
build.gradle (2)
The 2nd gradle file is this
sourceSets.main{
java.srcDirs = ['app']
}
task build << {
println "source sets=$sourceSets.main.java.srcDirs"
}
When I run this, it prints out stserver/app as my srcDir instead of stserver/webserver/app???? What am I doing wrong here?
thanks,
Dean
Please have a look at the docs Peter has suggested. I have a ready build.gradle that I have working with Play Framework 2.0~, so I'll share it here in hope you'll find some useful setup tips.
My project structure:
+- master/
+- build.gradle <-- contains common setup,
and applies 'java' plugin to all subprojects
+- ui/ <-- project using Play framework
+- build.gradle <-- excerpt from this file is posted below
The excerpt from build.gradle
repositories{
maven{
//Play dependencies will be downloaded from here
url " http://repo.typesafe.com/typesafe/releases"
}
}
//Optional but useful. Finds 'play' executable from user's PATH
def findPlay20(){
def pathEnvName = ['PATH', 'Path'].find{ System.getenv()[it] != null }
for(path in System.getenv()[pathEnvName].split(File.pathSeparator)){
for(playExec in ['play.bat', 'play', 'play.sh']){
if(new File(path, playExec).exists()){
project.ext.playHome = path
project.ext.playExec = new File(path, playExec)
return
}
}
}
throw new RuntimeException("""'play' command was not found in PATH.
Make sure you have Play Framework 2.0 installed and in your path""")
}
findPlay20()
configurations{
//Configuration to hold all Play dependencies
providedPlay
}
dependencies{
providedPlay "play:play_2.9.1:2.0+"
//Eclipse cannot compile Play template, so you have to tell it to
//look for play-compiled classes
providedPlay files('target/scala-2.9.1/classes_managed')
//other dependencies
}
sourceSets.main{
java.srcDirs = ['app', 'target/scala-2.9.1/src_managed/main']
//Make sure Play libraries are visible during compilation
compileClasspath += configurations.providedPlay
}
//This task will copy your project dependencies (if any) to 'lib'
//folder, which Play automatically includes in its compilation classpath
task copyPlayLibs(type: Copy){
doFirst { delete 'lib' }
from configurations.compile
into 'lib'
}
//Sets up common play tasks to be accessible from gradle.
//Can be useful if you use gradle in a continuous integration
//environment like Jenkins.
//
//'play compile' becomes 'gradle playCompile'
//'play run' becomes 'gradle playRun', and so on.
[ ['run', [copyPlayLibs]],
['compile', [copyPlayLibs]],
['clean', []],
['test', []],
['doc', [copyPlayLibs]],
['stage', [copyPlayLibs]] ].each { cmdSpec ->
def playCommand = cmdSpec[0]
def depTasks = cmdSpec[1]
task "play${playCommand.capitalize()}" (type: Exec,
dependsOn: depTasks,
description: "Execute 'play ${playCommand}'") {
commandLine playExec, playCommand
}
}
//Interate playClean and playCompile task into standard
//gradle build cycle
clean.dependsOn "playClean"
[compileScala, compileJava]*.dependsOn "playCompile"
//Include Play libraries in Eclipse classpath
eclipse {
classpath {
plusConfigurations += configurations.providedPlay
}
}
Note: I have just extracted the above from an existing bigger gradle file, so it might be missing some things, so no guarantees:) Hope it's useful anyway. Good luck.

Resources