How do I stop gradle from upgrading transitive dependencies? - gradle

Why is Gradle changing my library's transitive dependency to a newer version? How do I get it to stop?
Details
I am working on an internal plugin library for my company that uses Spring Security. The plugin explicitly declares a dependency on the latest version of Spring Security 4:
compile ('org.springframework.security:spring-security-core:4.2.13.RELEASE') {
force = true
}
When I include the plugin in a client project, Gradle is upgrading me from spring security 4 to 5, which breaks the plugin.
compile 'com.mycompany:my-security-plugin:0.3.0-SNAPSHOT'
Here is the output from dependencyInsight in the client project:
> Task :dependencyInsight
org.springframework.security:spring-security-core:5.1.6.RELEASE (selected by rule)
variant "compile" [
org.gradle.status = release (not requested)
org.gradle.usage = java-api
org.gradle.component.category = library (not requested)
]
org.springframework.security:spring-security-core:5.1.6.RELEASE
+--- org.springframework.security:spring-security-config:5.1.6.RELEASE
| \--- com.mycompany:my-security-plugin:0.3.0-SNAPSHOT:20200122.162056-4 (requested org.springframework.security:spring-security-config:4.2.13.RELEASE)
| \--- compileClasspath
\--- org.springframework.security:spring-security-web:5.1.6.RELEASE
\--- com.mycompany:my-security-plugin:0.3.0-SNAPSHOT:20200122.162056-4 (requested org.springframework.security:spring-security-web:4.2.13.RELEASE) (*)
org.springframework.security:spring-security-core:4.2.13.RELEASE -> 5.1.6.RELEASE
\--- com.mycompany:my-security-plugin:0.3.0-SNAPSHOT:20200122.162056-4
\--- compileClasspath
It looks to me like in all cases, I am requesting spring security 4 in my config. What am I doing wrong?
I am using Gradle 5.1.1.
Update
As a workaround, it is possible to have the client app declare a direct dependency on spring security, using a specific version. I'm trying to avoid this, if possible.
Update 2
Output from gradlew dependencyInsight --dependency org.springframework.security:spring-security-web:
> Task :dependencyInsight
org.springframework.security:spring-security-web:5.1.6.RELEASE (selected by rule)
variant "compile" [
org.gradle.status = release (not requested)
org.gradle.usage = java-api
org.gradle.component.category = library (not requested)
]
org.springframework.security:spring-security-web:4.2.13.RELEASE -> 5.1.6.RELEASE
\--- com.mycompany:my-security-plugin:0.3.0-SNAPSHOT:20200122.162056-4
\--- compileClasspath
Update 3
The buildEnvironment includes the following, via grails:
+--- org.springframework.boot:spring-boot-gradle-plugin:2.1.9.RELEASE
| | +--- org.springframework.boot:spring-boot-loader-tools:2.1.9.RELEASE (*)
| | +--- io.spring.gradle:dependency-management-plugin:1.0.8.RELEASE

Am I correct in assuming that you use the Spring Dependency Management Plugin (io.spring.dependency-management) and maybe also the Spring Boot Gradle Plugin (org.springframework.boot)? Then it’s very likely that this combination will also manage the eventual version of spring-security-core for you, irrespective of what other pieces of your build setup request.
Minimal Setup Demonstrating the Problem
I have the following two Gradle builds sitting next to each other (omitting the Gradle 5.1.1 Wrapper files for brevity):
my-security-plugin
└── build.gradle
my-client
├── build.gradle
└── settings.gradle
Running the following Gradle command from within my-client gives me (roughly) the same output as in the OP’s question:
./gradlew dependencyInsight --configuration compile --dependency org.springframework.security:spring-security-core
my-security-plugin/build.gradle
plugins {
id 'java'
}
group = 'com.mycompany'
version = '0.3.0-SNAPSHOT'
repositories {
jcenter()
}
dependencies {
compile ('org.springframework.security:spring-security-core:4.2.13.RELEASE') {
force = true
}
compile 'org.springframework.security:spring-security-config:4.2.13.RELEASE'
compile 'org.springframework.security:spring-security-web:4.2.13.RELEASE'
}
my-client/build.gradle
plugins {
id 'java'
id 'org.springframework.boot' version '2.1.7.RELEASE'
id 'io.spring.dependency-management' version '1.0.9.RELEASE'
}
group = 'com.mycompany'
version = '1.0.0'
repositories {
jcenter()
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-security'
compile 'com.mycompany:my-security-plugin:0.3.0-SNAPSHOT'
}
my-client/settings.gradle
includeBuild '../my-security-plugin'
This file is only here to define a Gradle composite build; includeBuild won’t be there in the production build, you will rather publish the my-security-plugin to some repository from where my-client downloads it.
Possible Solutions to the Problem
As mentioned in the introduction, the combination of Spring Dependency Management Plugin and Spring Boot Gradle Plugin defines the version of spring-security-core for your build. Aside from your own workaround, there are three possibilities to get to the version you want:
Use the Spring Boot version that selects the same version of spring-security-core that you need: id 'org.springframework.boot' version '1.5.22.RELEASE'
Customize the managed versions by setting an extra property in your build: ext['spring-security.version'] = '4.2.13.RELEASE'
Don’t let Spring manage your dependencies at all …
All these solutions obviously have drawbacks:
solution 1 forces you to use an old Spring Boot version
solution 2 forces you to request the required version of a transitive dependency in your client app (which is similar to your own workaround that you wanted to avoid)
solution 3 may not be worth the extra work that manually managing all Spring dependencies would incur
Now you only have to choose the lesser evil ;-)

Related

Gradle java-library dependency issues

I have developed a Java library using the java-library Gradle plugin. This library has e dependency on protobuf-java which I need to expose as a transitive dependency to users of the library.
I have the following snippets in my library's build.gradle
plugins {
id 'java-library'
id "maven-publish"
id "com.google.protobuf" version "0.8.16"
}
...
dependencies {
api ("com.google.protobuf:protobuf-java:3.17.3")
testImplementation("com.google.protobuf:protobuf-java-util:3.17.3")
testImplementation("org.junit.jupiter:junit-jupiter-api:5.7.2")
testImplementation("org.junit.jupiter:junit-jupiter-engine:5.7.2")
}
Running gradlew dependencies gives me the following as expected
api - API dependencies for source set 'main'. (n)
\--- com.google.protobuf:protobuf-java:3.17.3 (n)
But when I add the library as a dependency to in my main app, I don't get any transitive dependencies. Doing a gradlew dependencies in my main app gives me:
compileClasspath - Compile classpath for source set 'main'.
+--- my-library:my-library:1.0.21
+--- javax.jms:javax.jms-api -> 2.0.1
....
What could be the cause for the dependency not showing up in the main app?
The problem was with the maven-publish plugin.
I had to add the following to my publishing section to get it to work.
publications {
mavenJava(MavenPublication) { from project.components.java }
}

Specify Gradle version constraints for multiple configurations

I have a multi-module Gradle 6.5 project which also includes some test fixtures code. To avoid dependency issues I'd like to set (and maintain) the versions in one place, and just refer to the version-less dependencies in the individual modules:
subprojects {
apply plugin: 'java'
dependencies {
implementation 'com.google.guava:guava'
constraints {
implementation 'com.google.guava:guava:20.0'
compileOnly 'com.google.code.findbugs:jsr305:3.0.2'
}
}
}
Now, if a module includes compilyOnly 'com.google.code.findbugs:jsr305' the version 3.0.2 is taken based on the constraints above. This works fine if I know the configuration (implementation, compileOnly, ...).
The question now is: How can I specify versions that apply to all configuration? What if a module decides to use JSR305 annotations for test fixtures code? testFixtures 'com.google.code.findbugs:jsr305' fails, as the version is not specified anywhere. I also think it's a bad idea to repeat the version specification for all (possible) configuration:
implementation 'com.google.code.findbugs:jsr305:3.0.2'
testFixturesImplementation 'com.google.code.findbugs:jsr305:3.0.2'
compileOnly 'com.google.code.findbugs:jsr305:3.0.2'
Is there a shortcut to this?
If fixed this using the java-platform plugin.
Platform:
plugins {
id 'java-platform'
}
dependencies {
constraints {
api 'com.google.guava:guava:20.0'
api 'com.google.code.findbugs:jsr305:3.0.2'
}
}
Regular module with test fixtures:
plugins {
id "java-test-fixtures"
}
dependencies {
testFixturesImplementation platform(project(':platform-module'))
testFixturesCompileOnly 'com.google.code.findbugs:jsr305'
}
./gradlew -q module-with-test-fixtures:dependencies gives:
[...]
testFixturesCompileClasspath - Compile classpath for source set 'test fixtures'.
+--- project :module-with-test-fixtures (*)
+--- com.google.code.findbugs:jsr305 -> 3.0.2
\--- project :platform-module
\--- com.google.code.findbugs:jsr305:3.0.2 (c)
[...]
(c) - dependency constraint

Gradle plugins' transitive dependencies interfering and breaking the build

I am using Gradle 6.1 in a multimodule project. I am also using two plugins: kotlin("jvm") and id("com.google.cloud.tools.jib"), and they are loaded in the following modules:
root/
build.gradle.kts loads kotlin("jvm")
services/
my-service/
rest/
build.gradle.kts loads id("com.google.cloud.tools.jib")
(There are more modules, files etc. but these are the relevant ones.)
The build fails:
$ ./gradlew clean jibDockerBuild
...
* What went wrong:
Execution failed for task ':services:driver:rest:jibDockerBuild'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: 'org.apache.http.client.config.RequestConfig$Builder
org.apache.http.client.config.RequestConfig$Builder.setNormalizeUri(boolean)'
I identified the issue: both the Kotlin and JIB plugins have a transitive dependency on org.apache.httpcomponents:httpclient: Kotlin requires 4.5.3 and JIB 4.5.10. The problem is, in this project setup only 4.5.3 is loaded, and JIB fails as the new method is not available. This can be checked with ./gradlew buildEnv.
I've found a workaround, I need to load both plugins at the root level (which one is first seems to be irrelevant) in the main Gradle file; now ./gradlew buildEnv shows that the higher dependency version is used, also for Kotlin (output shortened and incomplete):
classpath
+--- org.jetbrains.kotlin.jvm:org.jetbrains.kotlin.jvm.gradle.plugin:1.3.61
| \--- org.jetbrains.kotlin:kotlin-gradle-plugin:1.3.61
| +--- de.undercouch:gradle-download-task:3.4.3
| | \--- org.apache.httpcomponents:httpclient:4.5.3 -> 4.5.10
It works in this case, but it could be that the new library version breaks the Kotlin plugin. The problem is that the plugins and their dependencies are on the classpath without separation, something that was normal on Java before Jigsaw etc. Is there any way for Gradle to be able to separate the dependencies so that each plugin uses exactly the version it declares? I am building on Java 11, so the module system could be utilized, but does Gradle have an option to turn it on?
EDIT: updating to Kotlin 1.3.70 also fixes the issue as it doesn't depend on the library any longer. The general question is still valid, though.
Is there any way for Gradle to be able to separate the dependencies so that each plugin uses exactly the version it declares
No.
All plugins share the same build script configuration: classpath
It follows the same dependency resolution that application dependencies follow. So you can enforce that for this particular dependency only use a specific version always:
buildscript {
configurations {
classpath {
resolutionStrategy {
force("org.apache.httpcomponents:httpclient:4.5.10")
}
}
}
}
That's just one of many ways you can take control of dependency resolution for build script dependencies. You could also use a platform to advise on the dependency versions:
buildscript {
dependencies {
classpath(enforcedPlatform("org.springframework.boot:spring-boot-dependencies:2.2.5.RELEASE"))
}
}
Refer to the docs for more info:
https://docs.gradle.org/current/userguide/resolution_rules.html
https://docs.gradle.org/current/userguide/platforms.html

Gradle is selecting wrong dependency version

I am seeing some weirdness in my Gradle build. I have a Spring Boot app (which uses Gradle for its build) and am trying to pull in both the Hibernate Validator as well as Hibernate Core. Here's the dependencies declaration in my build.gradle file:
dependencies {
compile('org.springframework.boot:spring-boot-starter-web') {
exclude module: 'spring-boot-starter-tomcat'
}
compile(
'org.codehaus.groovy:groovy-all:2.4.12'
,'com.google.inject:guice:4.1.0'
,'ch.qos.logback:logback-classic:1.2.3'
,'org.slf4j:jul-to-slf4j:1.7.25'
,'org.apache.logging.log4j:log4j-to-slf4j:2.9.1'
,'commons-cli:commons-cli:1.4'
,'org.apache.commons:commons-lang3:3.7'
,'io.dropwizard.metrics:metrics-core:3.2.5'
,'io.dropwizard.metrics:metrics-json:3.2.5'
,'org.springframework.security:spring-security-jwt:1.0.9.RELEASE'
,'org.springframework.security.oauth:spring-security-oauth2:2.2.1.RELEASE'
,'io.jsonwebtoken:jjwt:0.9.0'
,'org.hibernate:hibernate-validator:6.0.7.Final'
,'mysql:mysql-connector-java:6.0.6'
,'org.hibernate:hibernate-core:5.2.12.Final'
,'com.h2database:h2:1.4.196'
,'org.springframework.boot:spring-boot-starter-jetty'
,'org.springframework.boot:spring-boot-starter-actuator'
,'org.springframework.boot:spring-boot-starter-security'
,'org.springframework.boot:spring-boot-starter-data-rest'
,'org.springframework.boot:spring-boot-starter-data-jpa'
)
dev('org.springframework.boot:spring-boot-devtools')
testCompile(
'org.spockframework:spock-core:1.0-groovy-2.4'
,'junit:junit:4.12'
)
}
When I run ./gradlew dependencies I get a huge output, but from the compile dependencies tree I see the following:
| +--- org.springframework.boot:spring-boot-starter:1.5.8.RELEASE
| +--- org.hibernate:hibernate-validator:5.3.5.Final -> 6.0.7.Final
| | \--- org.hibernate.validator:hibernate-validator:6.0.7.Final
| | +--- javax.validation:validation-api:2.0.1.Final -> 1.1.0.Final
| | +--- org.jboss.logging:jboss-logging:3.3.0.Final -> 3.3.1.Final
| | \--- com.fasterxml:classmate:1.3.1 -> 1.3.4
So to me it looks like spring-boot-starter:1.5.8.RELEASE is pulling in validation-api:2.0.1.Final but for some reason Gradle is selecting validation-api:1.1.0.Final for me...am I reading that correctly? In my IDE compile classpath I only see validation-api:1.1.0.Final, not 2.0.1.Final.
Why is Gradle selecting 1.1.0.Final instead of 2.0.1.Final? I ask because Hibernate Validator 5.x is not compatible with Validation API 1.x and when my app runs I get all sorts of Hibernate Validation-related errors.
Update
Some more output:
gradle -q dependencyInsight --configuration compile --dependency validation-api
javax.validation:validation-api:1.1.0.Final (selected by rule)
javax.validation:validation-api:2.0.1.Final -> 1.1.0.Final
\--- org.hibernate.validator:hibernate-validator:6.0.7.Final
\--- org.hibernate:hibernate-validator:6.0.7.Final
+--- compile
\--- org.springframework.boot:spring-boot-starter-web:1.5.8.RELEASE
+--- compile
\--- org.springframework.boot:spring-boot-starter-data-rest:1.5.8.RELEASE
\--- compile
The full compile configuration output can be found here.
The version is enforced by Spring Boot.
See the POM for the Spring Boot dependencies: http://search.maven.org/remotecontent?filepath=org/springframework/boot/spring-boot-dependencies/1.5.8.RELEASE/spring-boot-dependencies-1.5.8.RELEASE.pom and look for "javax-validation.version".
See https://docs.spring.io/platform/docs/Brussels-SR4/reference/html/getting-started-overriding-versions.html for more information on how to override Spring Boot versions.
I would recommend overriding directly "javax-validation.version" and "hibernate-validator.version" instead of redefining the dependencies.
There is some conflict with another dependency that is pulling the older 1.1.0 in the compile classpath.
This means that some other library which has higher priority in gradle build order is dependent the older 1.1.0 version.
You can see here more info on how to specify the gradle build order.
I met similar problems, then I found it caused by using the Dependency management in gradle:
plugins {
...
id 'io.spring.dependency-management' version '1.0.10.RELEASE'
}
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:Hoxton.SR8"
}
}
this dependency management impact the transitive dependency version solution. after comment it out. all the version is correct.

How do I debug why Gradle is using a different version of the library I specified in a dependent sub-project?

I have a Gradle build that splits up my JOOQ generated code into a separate jooq subproject that my api-svc project then depends on.
When I upgrade the jooq subproject to 3.10.1, for some reason Gradle decides to use 3.9.5 to build the api-svc instead. I have no idea why, and I have to override by adding an explicit dependency in my api-svc project to work around it.
How can I debug what's going on to see why Gradle is overriding the version?
My JOOQ subproject's definition:
buildscript {
repositories {
mavenCentral()
jcenter()
}
dependencies {
...
classpath 'org.jooq:jooq-codegen:3.10.1'
...
}
}
...
dependencies {
compile 'org.jooq:jooq:3.10.1'
}
...
Here's the full version of the jooq subproject: https://bitbucket.org/snippets/shorn/64RnL5
And the inclusion in the api-svc project:
dependencies {
compile project(":idl")
compile project(":api-svc:jooq")
...
various other compile dependencies, spring-boot, etc.
When I do ./gradlew :api-svc:jooq:dependencies, it says:
------------------------------------------------------------
Project :api-svc:jooq
------------------------------------------------------------
...
compile - Dependencies for source set 'main' (deprecated, use 'implementation ' instead).
\--- org.jooq:jooq:3.10.1
...
But, when I do ./gradlew :api-svc:dependencies, it shows:
------------------------------------------------------------
Project :api-svc
------------------------------------------------------------
...
compile - Dependencies for source set 'main' (deprecated, use 'implementation ' instead).
+--- project :idl
| +--- org.apache.commons:commons-lang3:3.4
| \--- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.8.8
| +--- com.fasterxml.jackson.core:jackson-core:2.8.8 -> 2.8.10
| \--- com.fasterxml.jackson.core:jackson-databind:2.8.8 -> 2.8.10
| +--- com.fasterxml.jackson.core:jackson-annotations:2.8.0
| \--- com.fasterxml.jackson.core:jackson-core:2.8.10
+--- project :api-svc:jooq
| \--- org.jooq:jooq:3.10.1 -> 3.9.5
...
Full output here, if it helps: https://bitbucket.org/snippets/shorn/4x8eaG
So, you can see Gradle is choosing to use the 3.9.5 version of JOOQ instead of what I specified.
How do I debug why Gradle is doing that?
I can workaround this by adding jooq:3.10.1 as a direct compile dependency of the api-svc project - but that's redundant and as far as I understand, I shouldn't have to do that.
EDIT: Lukas Eder has pointed out in the comments that the root cause of the problem is the Spring has a dependency on JOOQ 3.9.5 and Gradle is using that. But the question is - how do I figure that out for myself without trawling through every dependency in my project, just in case they happen to have the reference that's messing up my build?
I made a post over on the Gradle forms to try and get an answer to this: https://discuss.gradle.org/t/how-do-i-debug-why-a-dependency-was-overridden/24572
There was some other discussion in private messages, but my conclusion is that Gradle has no way to debug these kind of dependency overrides.
If you found this question while trying to figure out a problem with your own build - the only advice seems to be that you need to dig through all your dependencies looking for the source of the conflict.
Alternatively, try a StackOverflow question or posting over on the Gradle forums.

Resources