We have some "heavy weight" artifacts, and we don't necessarily always need them. If we have some tasks with dependencies to them, they will get downloaded, regardless of the task which we invoke.
It would be nice if gradle only downloaded dependencies when it actually needs them.
Example
configurations {
webresources
}
dependencies {
webresources my:hugeIconCollection:2.0#zip
}
def unpackIcons = tasks.register('unpackIcons', Copy) {
configurations.webresources.resolveConfiguration.resolvedArtifacts.each {
from zipTree(it.file)
into 'resources/path'
}
}
Now simply running gradle help will already download the hugeIconCollection.zip.
The code uses the "Configuration Avoidance API", but it doesn't help. Still it would be desirable that the resource only be downloaded when the unpackIcons task is run (or a task that dependsOn it), and then the file be unzipped into the destination path.
Gradle downloads dependencies just-in-time on their first use. If your build downloads dependencies at configuration time, then because your build scripts/plugins are using them at configuration time. In most cases this indicates a problem with the build scripts/plugins.
Related
I new to Gradle. I'm trying to have all dependencies from my project copied to folder build/lib. Is it possible to do that?
Currently, I have a .zip and a .tar in build/distributions via the distribution plugin. However, I would like to have just the uncompressed libs.
I've seem a possible solution here, however I would like to have this as part of the main build and not as a separated task.
Just add the task as a dependency to build. That way it will be "part of the main build".
task copyDependencies(type: Copy) {
from configurations.runtimeClasspath // And/or:
from configurations.compileClasspath
into "$buildDir/lib"
}
build.dependsOn copyDependencies
You can, of cause, also hook it up to any other task that you use to build or deploy the application.
Just in case you are not aware of it, you can also use the installDist task from the distribution plugin (in combination with the application plugin) to create an exploded version in $buildDir/install.
We have a tool that runs from the command line. One of the commands is -version.
Before we converted to the nebula release plugin, the version was in the gradle.properties file, and as part of the build we copied it from there to a src/main/resources/version.txt file, that was later read by the tool to output the version.
But now the version is never in a file that's checked into git. Instead, it is only known during the nebula release process.
We want to obtain that version during the nebula release process and inject it into the jar that nebula is about to publish. For example, it could be added to the manifest.
We've tried to figure out how to do this, but don't see any examples online, and nothing about it in the documentation.
Simply create a task that caches the version that is dynamically inferred by Nebula.
Since you originally copied/created src/main/resources/version.txt, we'll use that that model our task.
Assuming a simple/standard Java project, using the Kotlin DSL:
val cacheNebulaVersion by tasks.registering {
mustRunAfter(tasks.named("release"))
doLast {
val sourceSets = project.extensions.getByName("sourceSets") as SourceSetContainer
sourceSets.getByName(SourceSet.MAIN_SOURCE_SET_NAME).output.resourcesDir?.let {
// If there are not existing resources in your project then you must create
// the resources dir otherwise a FileNotFoundException will be thrown.
if (!it.exists()) {
it.mkdirs()
}
File(it, "version.txt").printWriter().use { out ->
out.println(project.version)
}
}
}
}
When I invoke ./gradlew clean build snapshot cacheNebulaVersion, the version produced by Nebula is cached/created at src/main/resources/version.txt in the build output. The task above does not bundle it with the jar.
Hopefully that gives you an idea what to do.
I am struggling with the Gradle build lifecycle; specifically with the split between the configuration and execution phases. I have read a number of sections in the Gradle manual and have seen a number of ideas online, but have not found a solution to the following problem:
I want to run a specific task to produce an artifact at the end of my java-library-distribution build that is a flattened version of the runtime configuration jars. That is, I only want to produce the artifact when I run the specific task to create the artifact.
I have created the following task:
task packageSamplerTask(type: Tar, dependsOn: distTar) {
description "Packages the build jars including dependencies as a flattened tar file. Artifact: ${distsDir}/${archivesBaseName}-${version}.tar"
from tarTree("${distsDir}/${archivesBaseName}-${version}.tar").files
classifier = 'dist'
into "${distsDir}/${archivesBaseName}-dist-${version}.tar"
}
Although this task does produce the required artifact, the task runs during gradle's configuration phase. This behavior has the following consequences:
Irrespective of which task I run from the command line, this packageSamplerTask task is always run, often unnecessarily; and
If I clean the project, then the build fails on the next run because $distsDir doesn't exist during the configuration phase (obviously).
It appears that if I extend the Copy task in this manner I'm always going to get this kind of premature behavior.
Is there a way to use the << closure / doLast declarations to get what I want? Or is there something else I'm missing / should be doing?
Update
After further work I have clarified my requirements, and resolved my question as follows (specifically):
"I want to package my code and my code's dependencies as a flat archive of jars that can be deployed as a jMeter plugin. The package can then be installed by unpacking into the jMeter lib/ext directory, as is. The package, therefore, must not include the jMeter jars (and their dependencies) which are used for building and testing"
Because Gradle doesn't appear to support the Maven-like provided dependency management, I created a new configuration for my package which excludes the jMeter jars.
configurations {
jmpackage {
extendsFrom runtime
exclude group: 'org.apache.jmeter', name: 'ApacheJMeter_core', version: '2.11'
exclude group: 'org.apache.jmeter', name: 'ApacheJMeter_java', version: '2.11'
}
}
And then created the following task (using the closure recommendation from Peter Niederwieser):
task packageSamplerTask(type: Tar, dependsOn: assemble) {
from { libsDir }
from { configurations.jmpackage.getAsFileTree() }
classifier = 'dist'
}
This solution appears to work, and it allows me to use just theGradle java plugin, too.
The task declaration is fine, but the flattening needs to be deferred too:
...
from { tarTree("${distsDir}/${archivesBaseName}-${version}.tar").files }
Also, the Tar file should be referred to in a more abstract way. For example:
from { tarTree(distTar.archivePath).files }
First your task isn't executed in the configuration phase but like EVERY task it is configured in that phase. And your closure is just a configuration of your task (a Configuration closure, not an Action closure). That is why your code is "executed" in the configuration phase".
If you want your code to be executed in the execution phase have to write it in a doLastclosure or doFirst. But in your case it is better to keep it in a configuration closure, because you are configuring your task.
To make sure your build doesn't fail because of the missing folder, you can create it with distsDir.mkdirs().
I have a custom compile task.
task compileSpeedTest(type: JavaCompile) {
classpath = files('build')
source = fileTree('src/test/java/speed')
destinationDir = file('bin')
}
Gradle doesn't try to download dependencies before its execution.
I cannot find anywhere a task name which does it to add it on list dependsOn.
Downloading java dependencies is possible, if you actually really need to download them into a folder.
Example:
apply plugin: 'java'
dependencies {
runtime group: 'com.netflix.exhibitor', name: 'exhibitor-standalone', version: '1.5.2'
runtime group: 'org.apache.zookeeper', name: 'zookeeper', version: '3.4.6'
}
repositories { mavenCentral() }
task getDeps(type: Copy) {
from sourceSets.main.runtimeClasspath
into 'runtime/'
}
Download the dependencies (and their dependencies) into the folder runtime when you execute gradle getDeps.
For Intellij go to View > Tool Windows > Gradle > Refresh All Projects (the blue circular arrows at the top of the Gradle window.
A slightly lighter task that doesn't unnecessarily copy files to a dir:
task downloadDependencies(type: Exec) {
configurations.testRuntime.files
commandLine 'echo', 'Downloaded all dependencies'
}
Updated for kotlin & gradle 6.2.0, with buildscript dependency resolution added:
fun Configuration.isDeprecated() = this is DeprecatableConfiguration && resolutionAlternatives != null
fun ConfigurationContainer.resolveAll() = this
.filter { it.isCanBeResolved && !it.isDeprecated() }
.forEach { it.resolve() }
tasks.register("downloadDependencies") {
doLast {
configurations.resolveAll()
buildscript.configurations.resolveAll()
}
}
I have found this answer https://stackoverflow.com/a/47107135/3067148 also very helpful:
gradle dependencies will list the dependencies and download them as a
side-effect.
This version builds on Robert Elliot's, but I'm not 100% sure of its efficacy.
// There are a few dependencies added by one of the Scala plugins that this cannot reach.
task downloadDependencies {
description "Pre-downloads *most* dependencies"
doLast {
configurations.getAsMap().each { name, config ->
println "Retrieving dependencies for $name"
try {
config.files
} catch (e) {
project.logger.info e.message // some cannot be resolved, silentlyish skip them
}
}
}
}
I tried putting it into configuration instead of action (by removing doLast) and it broke zinc. I worked around it, but the end result was the same with or without. So, I left it as an explicit state. It seems to work enough to reduce the dependencies that have to be downloaded later, but not eliminate them in my case. I think one of the Scala plugins adds dependencies later.
You should try this one :
task getDeps(type: Copy) {
from configurations.runtime
into 'runtime/'
}
I was was looking for it some time ago when working on a project in which we had to download all dependencies into current working directory at some point in our provisioning script. I guess you're trying to achieve something similar.
Building on top of Robert Elliot's answer. For whatever reason, if one is interested in downloading the dependencies to Gradle cache then copying to a local repository like maven's (by default ~/.m2/repository):
task downloadDependencies(type: Exec) {
configurations.implementation.files + configurations.runtimeOnly.files
finalizedBy "cacheToMavenLocal"
commandLine "echo", "Downloaded all dependencies and copied to mavenLocal"
}
task cacheToMavenLocal(type: Copy) {
from new File(gradle.gradleUserHomeDir, "caches/modules-2/files-2.1")
into repositories.mavenLocal().url
eachFile {
List<String> parts = it.path.split("/")
it.path = [parts[0].replace(".","/"), parts[1], parts[2], parts[4]].join("/")
}
includeEmptyDirs false
}
The task cacheToMavenLocal was copied and adapted from #Adrodoc55's answer on Gradle forum.
It is hard to figure out exactly what you are trying to do from the question. I'll take a guess and say that you want to add an extra compile task in addition to those provided out of the box by the java plugin.
The easiest way to do this is probably to specify a new sourceSet called 'speedTest'. This will generate a configuration called 'speedTest' which you can use to specify your dependencies within a dependencies block. It will also generate a task called compileSpeedTestJava for you.
For an example, take a look at defining new source sets in the Java plugin documentation
In general it seems that you have some incorrect assumptions about how dependency management works with Gradle. I would echo the advice of the others to read the 'Dependency Management' chapters of the user guide again :)
There is no task to download dependencies; they are downloaded on demand. To learn how to manage dependencies with Gradle, see "Chapter 8. Dependency Management Basics" in the Gradle User Guide.
I have a task that generates java sources and a set of jars from these sources (say, project a). I would like to export these jars to dependent projects (say, project b). So here's roughly what I have right now:
//a.gradle
configurations{
generatedJars
}
task generateJars(type: JavaExec) {
//generate jars ...
outputs.files += //append generated jars here
}
dependencies{
generatedJars generateJars.outputs.files
}
//b.gradle
dependencies{
project(path: ':a', configuration: 'generatedJars')
}
It works OK, except that adding generateJars.outputs.files as a dependency does not tell gradle that it has to run generateJars task when there are no jars generated yet. I have tried adding the task itself as a dependency hoping that it would work in the same way as it does when you add a jar/zip task to an artifact configuration (e.g. artifacts{ myJarTask }), but it throws an error telling me that I cannot do that. Of course I can inject the generateJars task somewhere in the build process before :b starts evaluating, but that's clumsy and brittle, so I would like to avoid it.
I feel like I should be adding the generated jars to artifacts{ ... } of the project, but I am not sure how to make them then visible to dependent projects. Is there a better way of achieving this?
Dependent projects (project b) will need to do setup IntelliJ IDEA module classpath to point to project a's generated jars. Something rather like this (pseudo-code):
//b.gradle
idea{
module{
scopes.COMPILE.plus += project(path: ':a', configuration: 'generatedJars').files
}
}
So far I have tried simply adding a project dependecy on :a's generatedJars in :b, but Idea plugin simply adds module :a as a module-dependency and assumes that it exports its generated jars (which is probably a correct assumption), therefore not adding the generated jars to :b's classpath.
Any help would be greatly appreciated!
First, do you need a separate configuration? That is, do you have clients of a that should not see the generated Jars? If not, you can add the generated Jars to the archives configuration, which will simplify things.
Second, the correct way to add the generated Jars to the configuration is (instead of the dependencies block):
artifacts {
generatedJars generateJars
}
This should make sure that the generateJars task gets run automatically when needed.
Third, I'd omit the += after outputs.files, although it might not make a difference. You should also add the necessary inputs.
Fourth, why do you need a JavaExec task to generate the Jars? Can you instead add the generated sources to some source set and let Gradle build them?
Fifth, IDEA doesn't have a concept corresponding to Gradle's project configuration dependencies. Either an IDEA module fully depends on another module, or not at all. You have two options: either use a module dependency and make the generated sources a source folder of the depended-on module (preferably both in the Gradle and the IDEA build), or pass the generated Jars as external dependencies to IDEA. In either case, you should probably add a task dependency from ideaModule to the appropriate generation task. If this still doesn't lead to a satisfactory IDEA setup, you could think about moving the generation of the Jars into a separate subproject.
For my use case, I had a C++ project which generated some native libraries which my java project needed to load in order to run.
In the project ':native' build.gradle:
task compile(type: Exec, group: 'build') {
dependsOn ...
outputs.files(fileTree('/some/build/directory') {
include 'mylib/libmy.so'
})
...
}
In project java application build.gradle:
configurations {
nativeDep
}
// Add dependency on the task that produces the library
dependencies {
nativeDep files(project(':native').tasks.findByPath('compile'))
}
// Unfortunately, we also have to do this because gradle will only
// run the ':native:compile' task if we needed the tasks inputs for another
// task
tasks.withType(JavaCompile) {
dependsOn ':native:compile'
}
run {
doFirst {
// Use the configuration to add our library to java.library.path
def libDirs = files(configurations.nativeDep.files.collect {it.parentFile})
systemProperty "java.library.path", libDirs.asPath
}
}