I was studying Maven's build system and it adds a lot of transitive dependencies because of its transitive dependency system (maven itself does not add dependencies but transitive dependency system does). I see issues with it like major version conflicts and unknown dependencies coming in.
I was thinking why is the system designed this way and why not take direct dependencies. My library does not need to depend on something which my dependency is using but not my library (I mean I understand why it needs to be included in the build list, my dependencies need to build using those, but why does it needs to cause major version conflict?). Am I missing something fundamental here? One thing that I can think of is that my library's build dependency list can grow to be very big because of all the direct dependencies I will need to take, but that does seem to be as big of a problem as problems with transitive dependency system.
I am new to build systems, so please don't be too harsh. I also tried to google this question but didn't find useful answers but please feel free to comment anything that I might have missed.
Thanks
If you need library A to run, and library A needs library B to run, and this needs C to run, it is very tedious to figure this out and add all the relevant dependencies to your project.
Before Maven and Gradle, many people worked that way and found out, that it is much easier to let a build tool figure out the transitive dependencies.
My library does not need to depend on something which my dependency is using but not my library [...]
This is your major misconception. There are two possibilities:
The direct dependency of your library exposes types from the transitive dependency in its public API. To use this public API you need to access these types, so you need the transitive dependencies during compile time.
The direct dependency of your library only uses its own dependency internally, but not in its public API. In this case, your library does not need to depend on the transitive dependency during compile time. But as soon as your library code runs (even in a test), it may use some functionality of its direct dependency that internally uses functionality of the transitive dependency, causing your library code to fail.
[...] I mean I understand why it needs to be included in the build list, my dependencies need to build using those [...]
There is no actual build list (or order) for external dependencies, because they are used when they are already built (the downloaded .jar files contain compiled .class files). But as I mentioned above, you will need the transitive dependencies either during compile time or during runtime (e.g. tests), so your build system (Maven or Gradle) will fetch them for you.
[...] but why does it needs to cause major version conflict?
#khmarbaise already explained in his comment, why and how version conflict between transitive dependencies may occur:
You are using two libs X and Y. Both of them using another lib (A) So X is using A in version 1.0.0 but Y is using A in version 2.0.0. In the end you can't have both on the classpath there must be done a decision for one version. So depending on how X,Y are implemented either X can fail while using A in V1.0.0 or Y can fail in using A in V1.0.0 or with V2.0.0 the same... This can happen if X or Y are being updated. This is also true for different version combinations like A in 1.0.0 and 1.1.0 (if compatibility is not 100%)
Related
I am creating a Java library in Maven that has a dependency on a third party library.
In case any consumer applications already use that third party library, I want their build to use theirs. However in case they don't already have it, I want them to use the one I packaged in my archive.
Is there a Maven way to have your cake and eat it too, so that I can tell consumer applications to use theirs if they have one, else use mine?
When multiple versions of an artifact are found maven will attempt to select the version. It uses the nearest definition from the dependency tree.
https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
http://techidiocy.com/maven-dependency-version-conflict-problem-and-resolution/
This means that if the project consuming your library specifies a different version that will be used. If the common dependency is specified by a transitive dependency the version closest to the root of the tree will be used.
You can also use version ranges to specify many versions that work for you library.
https://docs.oracle.com/middleware/1212/core/MAVEN/maven_version.htm#MAVEN402
I have a project that uses gradle and mavenCentral() (plus mavenLocal()). It has enough dependencies that I can't go through them one by one.
Given the name of a .jar file in build/install/x/lib, how do I find out the chain of transitive dependencies that caused it to be included?
update: I discovered gradle dependencies. The output shows:
org.apache.commons:commons-jexl:2.1.1
\---- commons-logging:commons-logging:1.1.1 -> 1.1.3
What does this mean? 1.1.1 is the version I expect, and 1.1.3 is the version I seem to actually end up using. Looking at the pom for commons-jexl it looks like it does indeed list logging:1.1.1 as a requirement. What's going on? Is there a way for me to tell it to avoid certain versions, or force it to use the version it was set to?
The problem in my case is that it's including a -SNAPSHOT version and I'd rather it didn't. In fact I probably want it to just use the version numbers I'm asking for instead of the most recent it can find.
Dependencies of gradle-managed project have their own dependencies (they're called transitive). It may happen (and happens quite often) that two different dependencies has the same (group and module) dependency but in the different version). This is the case with commons-logging:commons-logging. In this case there are two transitive dependencies one versioned with 1.1.1 and the second one with 1.1.3. If both of the libraries will be included in the final artifact it may result in a conflict and exception. To prevent such situation gradle tries to resolve mentioned version resolution problems by picking (by default) the latest version. It's indicated with the right arrow -> see here. You can exclude transitive dependencies from a particular dependency. This chapter of manual might be useful.
I have the problem that Gradle is pulling in two versions of a library, which is causing runtime issues. These versions are both necessary, and are dependencies of two of the dependencies I have. Library A needs Library C version x, and Library B needs Library C version y. At runtime, the incorrect version of Library C is used, causing a NoSuchFieldError. Is this something I can resolve in Gradle? Or is it more of an issue for my IDE/JVM options?
In general Gradle will do the right thing by choosing the latest version of a library when there are dependency conflicts. However for various reasons this may not always work correctly. To get around this you can tell Gradle explicitly to not include a certain transitive dependency during its resolution. Here is an example:
compile (group:'com.project', name:'library', version:'1.0') {
// These lines will exclude these other libraries from being included
exclude module: 'groovy-all'
exclude module: 'log4j'
exclude module: 'commons-lang'
}
You can make this more fine grained if you need to, but I've found excluding modules seems to work well for me.
A problem that relates to basic maven concepts:
Once released I would like to have a guarantee that the project build is fully reproducible. So all project and plugin dependencies, including transitive one, should be always resolved the same way.
Unfortunately it is not the case, if dependencies are expressed in terms of version ranges. It can happen that even though direct dependencies of a project are set (using versions:use-releases), the transitive dependencies can still be resolved in some other way in the future.
How to address the problem? Is there a known solution?
I was thinking (just an idea), about creating a plugin, which on release time would dump all dependencies of the project to a separate file, and then once building in the future, the dependencies read from the file would take precedence over the standard way maven uses to resolve dependencies. But I'm afraid that there is no plugin api for that. So it would require some hacking, which I would like to avoid. Is there another way?
Thanks,
Lukasz
Freeze artifacts versions using <dependencyManagement>. Even if you don't use version ranges (as you said), but rather 3rd party libs (your dependencies) do, your <dependencyManagement> will have higher priority in specifying version of any artifacts.
The simple solution is: Do not use version-ranges. This is bad practice cause it will result in the described problems.
I have a module X that is dependent on a Third party library which in turn depends on apache-commons-collections 2.1.
In module X, I want to use the latest apache-commons-collections 3.0 which has some additional methods than 2.1. If I add a dependency to 3.0, I'm guessing this will create a problem since the class loader just picks up the first class it sees in the classpath. Is there a good way to get around this problem?
Thanks,
S
IMHO there is no really good way without additional solution for modularity (like Java EE's EAR or OSGi). I guess however that you're asking about just simple web (or not) module that directly use this 3rd party lib. I'm afraid you have to resolve this conflict manually. If fact, Maven won't provide 2 versions of commons-collections and depend on classloader's resolution, but rather resolve dependencies graph and pick the version it guess it's better with your POMs' declarations in mind. That means, if you declare in module X dependency on commons-collections version 3.0, that version will be used since this declaration is more important than some 3rd party lib's dependencies.
That's a serious problem of Java Platform itself, cousing such problems like the famous JAR hell. Unfortunately, it is your problem to choose and declare commons-collections version that satisfy both you and your 3rd party lib.
Just add the dependency to 3.0 to your project and it will prefer it over the older version of 2.1. To be explicit you can add an exclusion. In any case use the dependency plugin and the analyze and tree goals to see what is happening.
Long story short... this happens all the time and will be fine and in any case you can control what happens.