Module replacement works well in Gradle, however it only applies when there is a conflict.
Although I understand the reason, it breaks my use-case where there is extension of configurations and the conflict happens in some but not others that I need to consume.
I have two special configurations and some module replacement:
configurations {
lib // what should be bundled
provided // what should not be bundled
implementation.extendsFrom(lib)
implementation.extendsFrom(provided)
}
dependencies {
modules {
module('javax.annotation:javax.annotation-api') {
replacedBy('jakarta.annotation:jakarta.annotation-api', 'Javax to Jakarta')
}
}
}
task collectLibs(type: Copy) {
// bundle everything from lib which is not provided (not even transitively)
from configurations.lib - configurations.provided
into "$buildDir/lib"
}
I also use company BOM, here for example: api platform('org.springframework.boot:spring-boot-dependencies:2.5.4') and so I don't want to specify versions anywhere in my project.
Let's assume these dependencies:
dependencies {
lib 'javax.annotation:javax.annotation-api'
provided 'jakarta.annotation:jakarta.annotation-api'
}
the task dependencies then correctly resolves compileClasspath and runtimeClasspath to jakarta.annotation-api, however the collected files in build/lib contain javax.annotation-api-1.3.2.jar even though it "should have been replaced and subtracted"
If I use module substitution instead, it works:
configurations.all {
resolutionStrategy.dependencySubstitution {
substitute module('javax.annotation:javax.annotation-api') using module('jakarta.annotation:jakarta.annotation-api:1.3.5')
}
}
However there I must specify version. Is there any possibility to force module replacement to always act?
My problem is caused by the subtraction, maybe there is a better way to find all dependencies that come from provided but not lib by looking at runtimeClasspath?
I tried something but it gets too complicated very quickly.
I found a solution. Instead of subtracting provided configuration, I can exclude everything from resolved provided configuration. The tricky part is to exclude not too much and not too little:
platform must remain otherwise resolution of versions will fail
both requested and selected must be excluded
This is not a general solution; it still requires some fiddling with configurations (provided must declare both javax and jakarta) but it works for me.
private static excludeFromConfiguration(Configuration configuration, Configuration toExclude) {
toExclude.incoming.resolutionResult.allDependencies.each { dep ->
if (dep instanceof ResolvedDependencyResult && dep.requested instanceof ModuleComponentSelector) {
def isPlatform = dep.requested.attributes.keySet().any {
// asking for org.gradle.api.attributes.Category.CATEGORY_ATTRIBUTE does not work
def attribute = dep.requested.attributes.getAttribute(it)
return attribute == org.gradle.api.attributes.Category.ENFORCED_PLATFORM ||
attribute == org.gradle.api.attributes.Category.REGULAR_PLATFORM
}
if (!isPlatform) {
// we exclude both - the requested and selected because there could have been some:
// module replacement, dependency substitution, capability matching
configuration.exclude(group: dep.requested.group, module: dep.requested.module)
configuration.exclude(group: dep.selected.moduleVersion.group, module: dep.selected.moduleVersion.name)
}
}
}
}
Related
In a Maven project, it is easy to add extra deps and include extra source codes via defining a new Maven profile.
How to do the following things in a Gradle project.
Includes extra deps
Includes another source codes directory
And for example, use an extra property existence(eg. add to command line) to decide to activate it or not. I am not sure the best way in Gradle world.
I am not recommending your approach.
But it can be done via - project properties from gradle command line and groovy if (condition) { } for dependencies and multiple sourceset defs
on command line
gradle build -PbProfile=extra1
ext.buildFlag = 'default'
if (project.hasProperty('bProfile')) {
ext.buildFlag = property('bProfile')
}
println "running profile - ${buildFlag}"
dependencies {
//common-deps
if ("extra1".equals(buildFlag)) {
//extra deps
}
}
if ("extra1".equals(buildFlag)) {
//custom sourceset def
} // more else if needed
I use conditionally applied sub-configurations. This is done thru the apply from directive:
if (project.hasProperty('browsers')) {
ext.browsers.split(',').each {
def browser = it.trim()
if (browser) {
apply from: "${browser}Deps.gradle"
}
}
}
This block checks for specification of the browsers property (either from gradle.properties or the -P command line argument). If this property is defined, I split the property's value on commas and apply sub-configurations whose names conform to the pattern <browser>Deps.gradle.
The project in which I use this pattern is here
I'd like to separate the dependencies in my project by type, and am considering doing so in the following way:
// Implementation dependencies
dependencies {
implementation("foo:bar:1") {
because("reason 1")
}
implementation("foo:bar:2") {
because("reason 2")
}
implementation("foo:bar:3") {
because("reason 3")
}
}
// Test implementation dependencies
dependencies {
testImplementation("foo:bar:4") {
because("reason 4")
}
testImplementation("foo:bar:5") {
because("reason 5")
}
}
Questions:
I am able to build the project after structuring the build file in this way, but I don't see any authoritative material stating that specifying multiple dependencies blocks is formally supported. Does such material exist?
Is there a more preferable way of separating dependencies by type than this? Preferably, I'd like to have a dependency-configuration (implementation, testImplementation, etc.) per module in order to document the reason for including each module, like the configuration above does.
I don't see any authoritative material stating that specifying multiple dependencies blocks is formally supported. Does such material exist?
There doesn't need to be any material because the Gradle DSL (Groovy or Kotlin) isn't anything special or magical. It's simply sugar over the Gradle API.
Specifying multiple dependencies block is perfectly legal. If you were to de-sugar the Gradle DSL, invoking multiple dependencies blocks is actually just doing:
project.getDependencies().add("implementation", "foo:bar:1")
project.getDependencies().add("testImplementation", "foo:bar:4")
It's no different than simply calling the add(...) method on a List multiple times.
Is there a more preferable way of separating dependencies by type than this?
Create a library (project or subproject) that bundles dependencies together. This is easily accomplished with the Java Library Plugin. For example, for your test library:
dependencies {
api("foo:bar:4") {
because("reason 4")
}
api("foo:bar:5") {
because("reason 5")
}
}
Then simply consume the library in your main project:
dependencies {
testImplementation(project(":my-test-library")) {
because("bundles test libs")
}
}
There is no such support and I don't think is there is need also, but to achieve your requirements we can create an extension function just to differentiate the different dependencies. Anyway many Kotlin DSL is extension functions only so add something like below. just declare this in your buildSrc Dependencies.kts file or anywhere you like but should be accessible global.
// test
fun Project.dependenciesTest(configuration: DependencyHandlerScope.() -> Unit) =
DependencyHandlerScope.of(dependencies).configuration()
//app
fun Project.dependenciesApp(configuration: DependencyHandlerScope.() -> Unit) =
DependencyHandlerScope.of(dependencies).configuration()
now call something like this in the calling site.
dependenciesApp {
implementation(fileTree(mapOf("dir" to "libs", "include" to listOf("*.jar"))))
}
dependenciesTest {
testImplementation(AppDependencies.junit)
}
In app-level build.gradle, I often include the following dependencies
dependencies {
implementation "androidx.appcompat:appcompat:$appCompatVersion"
implementation "androidx.cardview:cardview:$cardVersion"
implementation "com.google.android.material:material:$materialVersion"
implementation "androidx.recyclerview:recyclerview:$recyclerViewVersion"
implementation "androidx.annotation:annotation:$androidXAnnotations"
...
}
It makes that file becomes longer and longer, thus I think about moving all such dependencies into project-level build.gradle, such as (just example):
ext {
includeUnitTestDeps() {
implementation "androidx.appcompat:appcompat:$appCompatVersion"
implementation "androidx.cardview:cardview:$cardVersion"
implementation "com.google.android.material:material:$materialVersion"
implementation "androidx.recyclerview:recyclerview:$recyclerViewVersion"
implementation "androidx.annotation:annotation:$androidXAnnotations"
}
}
(The reason why to project-level but not app-level because we might have multiple modules in project, thus project-level is the best places)
Then in app-level build.gradle we call
dependencies {
ext.includeUnitTestDeps()
...
}
Note: I'm not so familiar with Groovy/Gradle syntax, therefore not sure it works (In fact I tried but it doesn't allow to define such method in ext). But if you know any solution, please help me. Thanks so much.
There are couple of ways that I can think of
1 - Add dependencies to all sub projects (in the parent)
subprojects {
dependencies {
implementation 'com.google.guava:guava:23.0'
testImplementation 'junit:junit:4.12'
}
....
}
2 - Add dependencies to specific sub projects (in the parent)
// this will add dependencies to project a, b, and c
// when You add a new subproject, You have to add it to here also
// If You these need dependencies available in it
configure([project(':a'), project(':b'), project(':c')]) {
dependencies {
implementation 'com.google.guava:guava:23.0'
.....
}
}
3 - Using a method to add dependencies
//in parent
// define a method to add dependencies
// sub projects who need these dependencies will call this method
def addDependencies(subProject) {
subProject.dependencies.add("implementation", "com.google.guava:guava:23.0")
subProject.dependencies.add("implementation", "org.apache.commons:commons-lang3:3.8.1")
// add others
}
// in child
dependencies {
addDependencies(this)
// You can add other dependencies here If this child has any
}
4 - Define dependencies as a list in the parent
// parent
ext.appDependencies = [
[configuration: "implementation", dependency: "org.apache.commons:commons-lang3:3.8.1"],
[configuration: "implementation", dependency: "com.google.guava:guava:23.0"]
]
// child
dependencies {
rootProject.appDependencies.each {
add(it.configuration, it.dependency)
}
}
There is a detailed explanation of this method in the following link, which uses external file to define these dependencies.
https://hackernoon.com/android-how-to-add-gradle-dependencies-using-foreach-c4cbcc070458
You can also combine 3. and 4. methods like define a list that has dependencies, call a function which iterate and add dependencies in that list.
I would use the first or second method If I can. (Also there might be other ways to achieve this.)
Background: Running Android Studio 3.0-beta7 and trying to get a javadoc task to work for an Android library (the fact that this is not available as a ready-made task in the first place is really strange), and I managed to tweak an answer to a different question for my needs, ending up with this code (https://stackoverflow.com/a/46810617/1226020):
task javadoc(type: Javadoc) {
failOnError false
source = android.sourceSets.main.java.srcDirs
// Also add the generated R class to avoid errors...
// TODO: debug is hard-coded
source += "$buildDir/generated/source/r/debug/"
// ... but exclude the R classes from the docs
excludes += "**/R.java"
// TODO: "compile" is deprecated in Gradle 4.1,
// but "implementation" and "api" are not resolvable :(
classpath += configurations.compile
afterEvaluate {
// Wait after evaluation to add the android classpath
// to avoid "buildToolsVersion is not specified" error
classpath += files(android.getBootClasspath())
// Process AAR dependencies
def aarDependencies = classpath.filter { it.name.endsWith('.aar') }
classpath -= aarDependencies
aarDependencies.each { aar ->
System.out.println("Adding classpath for aar: " + aar.name)
// Extract classes.jar from the AAR dependency, and add it to the javadoc classpath
def outputPath = "$buildDir/tmp/exploded-aar/${aar.name.replace('.aar', '.jar')}"
classpath += files(outputPath)
// Use a task so the actual extraction only happens before the javadoc task is run
dependsOn task(name: "extract ${aar.name}").doLast {
extractEntry(aar, 'classes.jar', outputPath)
}
}
}
}
// Utility method to extract only one entry in a zip file
private def extractEntry(archive, entryPath, outputPath) {
if (!archive.exists()) {
throw new GradleException("archive $archive not found")
}
def zip = new java.util.zip.ZipFile(archive)
zip.entries().each {
if (it.name == entryPath) {
def path = new File(outputPath)
if (!path.exists()) {
path.getParentFile().mkdirs()
// Surely there's a simpler is->os utility except
// the one in java.nio.Files? Ah well...
def buf = new byte[1024]
def is = zip.getInputStream(it)
def os = new FileOutputStream(path)
def len
while ((len = is.read(buf)) != -1) {
os.write(buf, 0, len)
}
os.close()
}
}
}
zip.close()
}
This code tries to find all dependency AAR:s, loops through them and extracts classes.jar from them, and puts them in a temp folder that is added to the classpath during javadoc generation. Basically trying to reproduce what the really old android gradle plugin used to do with "exploded-aar".
However, the code relies on using compile dependencies. Using api or implementation that are recommended with Gradle 4.1 will not work, since these are not resolvable from a Gradle task.
Question: how can I get a list of dependencies using the api or implementation directives when e.g. configuration.api renders a "not resolvable" error?
Bonus question: is there a new, better way to create javadocs for a library with Android Studio 3.0 that doesn't involve 100 lines of workarounds?
You can wait for this to be merged:
https://issues.apache.org/jira/browse/MJAVADOC-450
Basically, the current Maven Javadoc plugin ignores classifiers such as AAR.
I ran in to the same problem when trying your answer to this question when this error message kept me from resolving the implementation dependencies:
Resolving configuration 'implementation' directly is not allowed
Then I discovered that this answer has a solution that makes resolving of the implementation and api configurations possible:
configurations.implementation.setCanBeResolved(true)
I'm not sure how dirty this workaround is, but it seems to do the trick for the javadocJar task situation.
For each sub-project in our build, we have a structure like this:
apply from: '../dependencies.gradle'
dependencies {
... omitting other dependencies ...
compile libraries.poi
}
These libraries are defined in dependencies.gradle, which looks like this:
ext.libraries = [
... omitting other libraries ...
poi: [
'poi:poi:3.9.custom.1',
'poi:poi-ooxml:3.9.custom.1',
'poi:poi-ooxml-schemas:3.9.custom.0',
'poi:poi-scratchpad:3.9.custom.0',
],
... omitting other libraries ...
]
A few days ago I wanted to try something against a nightly build of POI. Nightly builds don't go into their repository, so I'm forced to try and get it to work with local files.
Looking in the docs, you're supposed to use files(...) for this, so I tried this:
poi: [
files('/path/to/poi-3.14-beta1/poi-3.14-beta1-20151027.jar'),
files('/path/to/poi-3.14-beta1/poi-3.14-ooxml-20151027.jar'),
files('/path/to/poi-3.14-beta1/poi-3.14-ooxml-schemas-20151027.jar'),
files('/path/to/poi-3.14-beta1/poi-3.14-scratchpad-20151027.jar'),
],
When I run this, I get an error:
* What went wrong:
A problem occurred evaluating root project 'product'.
> Cannot convert the provided notation to an object of type ModuleVersionSelector: file collection.
The following types/formats are supported:
- Instances of ModuleVersionSelector.
- String or CharSequence values, for example 'org.gradle:gradle-core:1.0'.
- Maps, for example [group: 'org.gradle', name:'gradle-core', version: '1.0'].
- Collections or arrays of any other supported format. Nested collections/arrays will be flattened.
So really it seems like files() does not actually work, as it doesn't return one of the things listed here.
What is the correct way to do it? (Assuming it's even possible!)
Edit: More information
Now that I updated to Gradle 2.8, I get a line number pointing at the problem. It points at some custom build code which we put in to work around Gradle sucking at dependency resolution:
resolutionStrategy {
libraries.each {
libraryName, libraryList ->
libraryList.each {
library -> force library // 👈 this line
}
}
failOnVersionConflict()
}
So I take it the problem is that force doesn't support all the same things that other methods support?
My crap workaround for a workaround is to filter out elements of type FileCollection:
resolutionStrategy {
libraries.each { libraryName, libraryList ->
[libraryList].flatten()
.findAll { library ->
!(library instanceof FileCollection)
}
.each { library -> force library }
}
failOnVersionConflict()
}
Maybe there is a better way.