In gradle dependencies, why API configurations discards all transitive dependencies? - gradle

I have a gradle project that contains 2 subprojects: common & demo.
The common project depends on a published library:
dependencies {
implementation("eu.timepit:singleton-ops_${vs.scalaBinaryV}:0.5.0")
...
The demo project depends on common as part of its API:
dependencies {
api(project(":common")) {
isTransitive =true
}
}
When I compile both, I observe the correct dependency in common:
compileClasspath - Compile classpath for source set 'main'.
+--- org.scala-lang:scala-compiler:2.12.11
| +--- org.scala-lang:scala-library:2.12.11
| +--- org.scala-lang:scala-reflect:2.12.11
| | \--- org.scala-lang:scala-library:2.12.11
| \--- org.scala-lang.modules:scala-xml_2.12:1.0.6
+--- org.scala-lang:scala-library:2.12.11
+--- org.scala-lang:scala-reflect:2.12.11 (*)
\--- eu.timepit:singleton-ops_2.12:0.5.0
+--- org.scala-lang:scala-compiler:2.12.8 -> 2.12.11 (*)
+--- org.scala-lang:scala-library:2.12.8 -> 2.12.11
\--- com.chuusai:shapeless_2.12:2.3.3
+--- org.scala-lang:scala-library:2.12.4 -> 2.12.11
\--- org.typelevel:macro-compat_2.12:1.1.1
\--- org.scala-lang:scala-library:2.12.0 -> 2.12.11
but in demo, the transitive dependecies under common are empty!
compileClasspath - Compile classpath for source set 'main'.
+--- project :common
+--- org.scala-lang:scala-compiler:2.12.11
...
This leads to the very common classpath missing errors, like:
[Error] /xxx/DoubleVectorDemo.scala:9: Symbol 'type shapeless.ProductArgs' is missing from the classpath.
This symbol is required by 'object edu.umontreal.kotlingrad.shapesafe.tensor.DoubleVector'.
Make sure that type ProductArgs is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'DoubleVector.class' was compiled against an incompatible version of shapeless.
So what's the point of dropping libraries for something that is part of your API? And how to override this behaviour in all projects?

From the gradle java-plugin doc,
The api configuration should be used to declare dependencies which are exported by the library API, whereas the implementation configuration should be used to declare dependencies which are internal to the component. Dependencies appearing in the api configurations will be transitively exposed to consumers of the library, and as such will appear on the compile classpath of consumers. Dependencies found in the implementation configuration will, on the other hand, not be exposed to consumers, and therefore not leak into the consumers' compile classpath
Let's say of you want to expose eu.timepit:singleton-ops_${vs.scalaBinaryV}:0.5.0 to all of the common library then you need to add this as api dependency in common module build.gradle.kts.
dependencies {
api("eu.timepit:singleton-ops_${vs.scalaBinaryV}:0.5.0")

Related

Enforce the highest version from among several conflicting transitive dependencies in gradle

I work on a large project with multiple services and libraries, mostly in grails, with gradle builder. I'm trying to update a library (say logback) for security reasons.
I already updated it in one of our libraries (say our-logger), like so:
#our-logger/build.gradle
...
dependencies {
...
compile 'ch.qos.logback:logback-classic:1.2.3'
...
}
when I update a service (say our-service) to use the new version of our-logger i get logback included from other libraries, and gradle chooses the lower version coming through cobertura and some other dependencies, instead of the higher version coming through our-logger.
#our-service/build.gradle
...
apply plugin: 'cobertura'
...
dependencies {
...
compile 'our-logger:9.99' # safe now with logback-1.2.3
...
}
~/our-service $ ./gradlew dependencies
...
cobertura
\--- net.sourceforge.cobertura:cobertura:2.1.1
+--- ch.qos.logback:logback-classic:1.0.13 -> 1.1.11
| \--- ch.qos.logback:logback-core:1.1.11
...
compile - Dependencies for source set 'main' (deprecated, use 'implementation ' instead).
+--- org.grails:grails-dependencies:3.3.8
| +--- org.springframework.boot:spring-boot-starter-logging:1.4.2.RELEASE -> 1.5.15.RELEASE
| | +--- ch.qos.logback:logback-classic:1.1.11
| | | +--- ch.qos.logback:logback-core:1.1.11
...
+--- our-logger:9.99
| +--- ch.qos.logback:logback-classic:1.2.3 -> 1.1.11 (*)
How do I enforce logback-1.2.3 without explicitly declaring it in all services?
The gradle docs file this under Advanced Dependency Management. You should be able to satisfy your goal using excludes. There are other ways too Gradle Docs
compile(“some:other:dependency”) {
exclude group: 'ch.qos.logback', module: 'logback-classic'
}

Gradle is selecting wrong dependency version

I am seeing some weirdness in my Gradle build. I have a Spring Boot app (which uses Gradle for its build) and am trying to pull in both the Hibernate Validator as well as Hibernate Core. Here's the dependencies declaration in my build.gradle file:
dependencies {
compile('org.springframework.boot:spring-boot-starter-web') {
exclude module: 'spring-boot-starter-tomcat'
}
compile(
'org.codehaus.groovy:groovy-all:2.4.12'
,'com.google.inject:guice:4.1.0'
,'ch.qos.logback:logback-classic:1.2.3'
,'org.slf4j:jul-to-slf4j:1.7.25'
,'org.apache.logging.log4j:log4j-to-slf4j:2.9.1'
,'commons-cli:commons-cli:1.4'
,'org.apache.commons:commons-lang3:3.7'
,'io.dropwizard.metrics:metrics-core:3.2.5'
,'io.dropwizard.metrics:metrics-json:3.2.5'
,'org.springframework.security:spring-security-jwt:1.0.9.RELEASE'
,'org.springframework.security.oauth:spring-security-oauth2:2.2.1.RELEASE'
,'io.jsonwebtoken:jjwt:0.9.0'
,'org.hibernate:hibernate-validator:6.0.7.Final'
,'mysql:mysql-connector-java:6.0.6'
,'org.hibernate:hibernate-core:5.2.12.Final'
,'com.h2database:h2:1.4.196'
,'org.springframework.boot:spring-boot-starter-jetty'
,'org.springframework.boot:spring-boot-starter-actuator'
,'org.springframework.boot:spring-boot-starter-security'
,'org.springframework.boot:spring-boot-starter-data-rest'
,'org.springframework.boot:spring-boot-starter-data-jpa'
)
dev('org.springframework.boot:spring-boot-devtools')
testCompile(
'org.spockframework:spock-core:1.0-groovy-2.4'
,'junit:junit:4.12'
)
}
When I run ./gradlew dependencies I get a huge output, but from the compile dependencies tree I see the following:
| +--- org.springframework.boot:spring-boot-starter:1.5.8.RELEASE
| +--- org.hibernate:hibernate-validator:5.3.5.Final -> 6.0.7.Final
| | \--- org.hibernate.validator:hibernate-validator:6.0.7.Final
| | +--- javax.validation:validation-api:2.0.1.Final -> 1.1.0.Final
| | +--- org.jboss.logging:jboss-logging:3.3.0.Final -> 3.3.1.Final
| | \--- com.fasterxml:classmate:1.3.1 -> 1.3.4
So to me it looks like spring-boot-starter:1.5.8.RELEASE is pulling in validation-api:2.0.1.Final but for some reason Gradle is selecting validation-api:1.1.0.Final for me...am I reading that correctly? In my IDE compile classpath I only see validation-api:1.1.0.Final, not 2.0.1.Final.
Why is Gradle selecting 1.1.0.Final instead of 2.0.1.Final? I ask because Hibernate Validator 5.x is not compatible with Validation API 1.x and when my app runs I get all sorts of Hibernate Validation-related errors.
Update
Some more output:
gradle -q dependencyInsight --configuration compile --dependency validation-api
javax.validation:validation-api:1.1.0.Final (selected by rule)
javax.validation:validation-api:2.0.1.Final -> 1.1.0.Final
\--- org.hibernate.validator:hibernate-validator:6.0.7.Final
\--- org.hibernate:hibernate-validator:6.0.7.Final
+--- compile
\--- org.springframework.boot:spring-boot-starter-web:1.5.8.RELEASE
+--- compile
\--- org.springframework.boot:spring-boot-starter-data-rest:1.5.8.RELEASE
\--- compile
The full compile configuration output can be found here.
The version is enforced by Spring Boot.
See the POM for the Spring Boot dependencies: http://search.maven.org/remotecontent?filepath=org/springframework/boot/spring-boot-dependencies/1.5.8.RELEASE/spring-boot-dependencies-1.5.8.RELEASE.pom and look for "javax-validation.version".
See https://docs.spring.io/platform/docs/Brussels-SR4/reference/html/getting-started-overriding-versions.html for more information on how to override Spring Boot versions.
I would recommend overriding directly "javax-validation.version" and "hibernate-validator.version" instead of redefining the dependencies.
There is some conflict with another dependency that is pulling the older 1.1.0 in the compile classpath.
This means that some other library which has higher priority in gradle build order is dependent the older 1.1.0 version.
You can see here more info on how to specify the gradle build order.
I met similar problems, then I found it caused by using the Dependency management in gradle:
plugins {
...
id 'io.spring.dependency-management' version '1.0.10.RELEASE'
}
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:Hoxton.SR8"
}
}
this dependency management impact the transitive dependency version solution. after comment it out. all the version is correct.

Determine source of dependency version override in Gradle

I have declared a dependency for org.spockframework:spock-spring:1.1.d91bf785-groovy-2.4, but in runtime scope it is overridden to 1.0-groovy-2.4. Is there a way to figure out which dependency is overriding it?
1.0-groovy-2.4 is not present in any gradle file locally, and it is not visible when I run gradle dependencies in the shared-config project.
Here's gradle dependencies from the main-web project:
testCompile
[...]
+--- project :shared-config
| [...]
| +--- org.spockframework:spock-spring:1.1.d91bf785-groovy-2.4
[...]
runtime
[...]
+--- project :shared-config
| [...]
| +--- org.spockframework:spock-spring:1.1.d91bf785-groovy-2.4 -> 1.0-groovy-2.4
| | +--- org.spockframework:spock-core:1.0-groovy-2.4 -> 1.1.d91bf785-groovy-2.4
| | \--- org.codehaus.groovy:groovy-all:2.4.1 -> 2.4.12
dependencyInsight gives some insight:
$ gradle dependencyInsight --dependency org.spockframework:spock-spring:1.0-groovy-2.4 --configuration runtime
> Task :main-web:dependencyInsight
org.spockframework:spock-spring:1.0-groovy-2.4 (selected by rule)
org.spockframework:spock-spring:1.1.d91bf785-groovy-2.4 -> 1.0-groovy-2.4
\--- project :shared-config
\--- runtime
But the same command in the shared-config project doesn't yield any results:
No dependencies matching given input were found in configuration ':shared-config:runtime'
I have tried overriding the version without success:
configurations.all {
resolutionStrategy {
force "org.spockframework:spock-spring:1.1.d91bf785-groovy-2.4"
}
}
Seems like you are using spring boot as a parent or spring boot dependencies as a bom (dependency manager)
by default in latest spring boot versions 1.5 + spock.version is set to 1.0-groovy-2.4 that's why spock-core still have old version.
In order to fix this in gradle you need to override property spock.version in your gradle app. By adding spock.version = 1.1-groovy-2.4 to gradle.properties file.
Visit Spring doc for overriding dep properties to find more.

How do I debug why Gradle is using a different version of the library I specified in a dependent sub-project?

I have a Gradle build that splits up my JOOQ generated code into a separate jooq subproject that my api-svc project then depends on.
When I upgrade the jooq subproject to 3.10.1, for some reason Gradle decides to use 3.9.5 to build the api-svc instead. I have no idea why, and I have to override by adding an explicit dependency in my api-svc project to work around it.
How can I debug what's going on to see why Gradle is overriding the version?
My JOOQ subproject's definition:
buildscript {
repositories {
mavenCentral()
jcenter()
}
dependencies {
...
classpath 'org.jooq:jooq-codegen:3.10.1'
...
}
}
...
dependencies {
compile 'org.jooq:jooq:3.10.1'
}
...
Here's the full version of the jooq subproject: https://bitbucket.org/snippets/shorn/64RnL5
And the inclusion in the api-svc project:
dependencies {
compile project(":idl")
compile project(":api-svc:jooq")
...
various other compile dependencies, spring-boot, etc.
When I do ./gradlew :api-svc:jooq:dependencies, it says:
------------------------------------------------------------
Project :api-svc:jooq
------------------------------------------------------------
...
compile - Dependencies for source set 'main' (deprecated, use 'implementation ' instead).
\--- org.jooq:jooq:3.10.1
...
But, when I do ./gradlew :api-svc:dependencies, it shows:
------------------------------------------------------------
Project :api-svc
------------------------------------------------------------
...
compile - Dependencies for source set 'main' (deprecated, use 'implementation ' instead).
+--- project :idl
| +--- org.apache.commons:commons-lang3:3.4
| \--- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.8.8
| +--- com.fasterxml.jackson.core:jackson-core:2.8.8 -> 2.8.10
| \--- com.fasterxml.jackson.core:jackson-databind:2.8.8 -> 2.8.10
| +--- com.fasterxml.jackson.core:jackson-annotations:2.8.0
| \--- com.fasterxml.jackson.core:jackson-core:2.8.10
+--- project :api-svc:jooq
| \--- org.jooq:jooq:3.10.1 -> 3.9.5
...
Full output here, if it helps: https://bitbucket.org/snippets/shorn/4x8eaG
So, you can see Gradle is choosing to use the 3.9.5 version of JOOQ instead of what I specified.
How do I debug why Gradle is doing that?
I can workaround this by adding jooq:3.10.1 as a direct compile dependency of the api-svc project - but that's redundant and as far as I understand, I shouldn't have to do that.
EDIT: Lukas Eder has pointed out in the comments that the root cause of the problem is the Spring has a dependency on JOOQ 3.9.5 and Gradle is using that. But the question is - how do I figure that out for myself without trawling through every dependency in my project, just in case they happen to have the reference that's messing up my build?
I made a post over on the Gradle forms to try and get an answer to this: https://discuss.gradle.org/t/how-do-i-debug-why-a-dependency-was-overridden/24572
There was some other discussion in private messages, but my conclusion is that Gradle has no way to debug these kind of dependency overrides.
If you found this question while trying to figure out a problem with your own build - the only advice seems to be that you need to dig through all your dependencies looking for the source of the conflict.
Alternatively, try a StackOverflow question or posting over on the Gradle forums.

Finding unwanted code dependencies on transitive dependencies

I want to find all my Java code dependencies on libraries that I have not included as top level dependencies in Gradle.
My first though as to how to accomplish this is to turn off all transitive dependencies in Gradle and see what compilation errors I get.
From my research the way to do this seems to be:
configurations.all { transitive = false }
Is there a better way, or does this do it?
I'm not sure I understand the question, but the command line "gradle dependencies" might help.
For example, consider this (from this modest project):
dependencies {
groovy 'org.codehaus.groovy:groovy-all:1.6.4'
groovy 'com.google.guava:guava-collections:r03'
releaseJars 'org.codehaus.groovy:groovy-all:1.6.4'
releaseJars 'com.google.guava:guava-collections:r03'
}
Using gradle dependencies gives output such as:
compile - Classpath for compiling the main sources.
+--- org.codehaus.groovy:groovy-all:1.6.4
| +--- junit:junit:3.8.2
| +--- org.apache.ant:ant:1.7.1
| | \--- org.apache.ant:ant-launcher:1.7.1
| +--- org.apache.ant:ant-launcher:1.7.1
| \--- jline:jline:0.9.94
| \--- junit:junit:3.8.1 -> 3.8.2
\--- com.google.guava:guava-collections:r03
+--- com.google.guava:guava-annotations:r03
\--- com.google.guava:guava-primitives:r03
....

Resources