Maven has the lame dependency conflict resolving rule of 'the closer the better': if 2 versions exists for a library, only the one closest to the trunk of the transitive dependency tree will be kept, the others will be dropped from classpath. This has caused pervasive instability in all projects build by maven.
I would like to address this problem from 2 sides:
use conflict detection analyzer like JHades to scan everything in the classpath, and report version conflict from within the classloader
use a fancy classloader to situationally customise the classpath, such that the correct version will be used at the right moment. This is also the method adopted by OSGi
To achieve either of them I need to override maven such that it append the jars of the further versions into the classpath in various lifecycles ('test' in particular). How do I achieve this?
UPDATE: I realise maven is an old product and wasn't design with megaproject that has thousands of dependency in mind. So if you have a solution in Gradle, Pants, Buck, sbt, or any esoteric build tools, I will still accept it as a valid answer.
Related
I have a maven war project with submodules. One module uses google-api-client, another use google-cloud-storage. I draw some of their dependencies below
A
|-google-api-client:jar:1.33.1
|-google-http-client-gson:jar:1.41.1
B
|-google-cloud-storage:jar:2.4.4
|-google-api-client:jar:1.33.1
|-google-http-client-gson:jar:1.41.2
When packaging wars, both gson 1.41.1 and 1.41.2 will be packaged. I know maven has a nearest rule to determine which jar to use when compiling. But when the webserver loads my project I have no control to which jar will be loaded first. So I want to keep only a newer version for each jar.
I know that I can add <exclusion> tags to the dependencies and add a new dependency to tell maven to use a specific version of jars. However, I am not sure if that is the best practice because it requires me to go through the dependencies of third-party libraries. There are just too many of them.
Any suggestions on how to handle the multiple versions of jars properly?
A good practice I recommend is to use enforcer Plugin with dependency convergence goal. This way you are forced to decide which version will be on the class path. Of course it might be additional effort because you have to handle conflicts (also by setting exclusions), but in the end it's well defined, which versions you get.
need to add the Dependencies of spring web jar 3.1.3 and 3.2.2 in the same pom.xml.
Does maven support the same jar of different versions?
In addition to this,
Maven does that in a different way. If there are two dependency declarations with different versions, the higher version takes the precedence.
EDIT1 :
This statement is for the direct dependency declaration in the same pom. Forcing the required version to use is the strategy here.
Maven uses dependency mediation to resolve the version conflicts.
AFAIK, as the higher versions always have backward compatibility, preferred is to use the higher version.
Unless there is a very strong reason behind adding a specific version, it all goes to designing the modules and their hierarchy.
This gives you complete insight of how to manage the dependencies using modules.
This describes the conflict resolution scenarios.
EDIT2:
For the transitive dependencies, yes the nearest definition is the strategy followed. As in example.
I have to add a new dependency to a maven project. This dependency has four transitive dependencies(according to http://mvnrepository.com/) and between them, there is spring-data-jpa jar.
The maven project I am working in has many dependencies configured in the pom so I understand there could be a big possibility that there is already a spring-data-jpa dependency in the project(transitive or not).
When you work in a large project with many dependencies and you have to add a new one, how to check if there is already the same transitive dependency of a different version? I have to check manually the transitive dependencies for each direct dependency configured? Has maven a warning for this situation?
How maven works in this situation? I mean, there could be two spring-data-jpa jars of different versions(this would be a problem) or maven resolves this in another way?
The simple answer is that the dependency plugin can tell you. The longer answer is that there are a number of different situations to consider about transitive dependency management, and how the plugin helps and what to do about it differs for each one.
Maven automatically chooses which dependency to include if two dependencies have the same coordinates (groupId, artifactId) with different versions. Broadly speaking, it picks the version that's highest in the tree - effectively overriding dependencies defined in downstream transitive dependency poms. So, if you have two different versions of exactly the same dependency then you will still only find one version of the dependency on the relevant classpath.
The dependency plugin can help you identify this situation by highlighting points where its made a decision, but you probably want to use the dependencyManagement section of your top-level pom to ensure that the dependencies which you bring in are the ones you expect.
Separate difficulties can arise when a dependency changes its groupId or artifactId. Then you can get two dependencies on the classpath - one with an old version on the old coordinates and one with the new version on the new coordinates. As examples, Spring, Hibernate and Apache commons have all found themselves doing this at some point or another. In this case all you can do is use to the dependency plugin to identify duplicated dependencies and then use exclusions tags to explicitly exclude them as transitive dependencies from the dependencies which are pull them in.
It's important to note that all of this dependency management can cause unintended breakage. If the thing that your application depends on really does depend on some specific version of a package as a transitive dependency then you can break it by overriding that version. So testing the features that you use is essential.
Have you tried the Maven Dependency Plugin? There's some useful goals you can run, such as mvn dependency:tree etc.
I have a Maven project that depends on numerous other projects, which often have several conflicting dependencies.
Maven will automatically resolve dependency conflicts using its nearest-wins strategy, in which case it will list the chosen version on the result mvn dependency:tree:
[INFO] | - (commons-collections:commons-collections:jar:2.1:compile - omitted for conflict with 2.0)
In this example, should commons-collections:2.1 be chosen, I would have an alternate dependency tree, potentially containing multiple other dependencies.
What I need to know is how this alternate tree would look like should the other version be chosen.
The way I'm currently doing is identifying the top level dependency that has conflicts with other dependencies, and running dependency tree just for it, i.e. by effectively creating multiple new poms with a single dependency just for the purpose of obtain their trees. This works, and I can write a script for that, but it's a manual process. I'm looking for Maven goal or other approach, which would act on my pom making things straightforward.
I've looked at other goals of the Maven dependency plugin such as analyse, but they do not appear to be helpful in this case.
There is no plugin that does exactly that, but there is a plugin that can help deal with the nearest-win maven dependency resolution strategy, and protect against it's occasionally unintended consequences.
It's the maven enforcer plugin, that with it's dependencyConvergence rule basically turns off the maven nearest-win strategy. With that rule applied to a build, if there is a dependency on version 1 of a transitive library, but also a dependency on version 2 of the same library, the build will fail.
This is a fail early mechanism that will allow to detect when the maven nearest dependency mechanism kicks in and silently makes a choice for a library version that we would prefer to do ourselves based on some code/library analysis.
What you can do with this plugin is to fix the current build by choosing the versions you want one by one, and then turn on the dependencyConvergence for future builds. This will ensure that you won't have to do the same analysis and fix in a near future, whe someone else changes the poms and the problem occurs again.
When there is a conflict in the dependency tree (same artifact but different versions) then, AFAIK, Maven will resolve the conflict by selecting the highest version of the dependency and will omit the 'old' ones.
However, when the newer version is a SNAPSHOT then apparently it will choose the older stable version over the SNAPSHOT.
In my case: some-artifact: 0.5.0-SNAPSHOTS (omitted for conflict with 0.4.0) => version 0.4.0 is picked over the wanted 0.5.0-SNAPSHOT.
I assume this functions as designed but I don't understand the reason why. Next to that, any idea if there is a way to tell Maven to take the SNAPSHOT over the stable version?
Your assumption about Maven's always selecting the highest version isn't accurate. Artifacts are chosen based on a number of factors including depth of the dependency in the tree, order in the tree, whether the dependency is a snapshot or a release, and dependency management, which pretty much overrides everything else.
Unfortunately, I don't know of any one, definitive source of information on Maven's dependency resolution algorithms. You'll find bits and pieces of it scattered all over. A few handy references:
Introduction to the Dependency Mechanism gives an overview of the topic with a good, if short, section on Transitive Dependencies and how they're selected from a dependency tree.
The Sonatype Maven book has a more thorough section on Project Dependencies in general that will add a lot to your knowledge about the subject.
An earlier section of that same book discusses Project Versions, which is strongly related to this problem and has a good section on SNAPSHOT versions, though not as much as I could wish on how they play into dependency resolution.
Project Relationships talks about the coordinate system and how project inheritance affects what dependencies get included.
Finally, the POM Reference is a good jumping-off point for almost anything to do with the pom. There's at least a brief description of every pom element that can help you understand enough to be able to begin searching for more info effectively.
As for some practical advice, the output of mvn dependency:tree is highly useful in discovering why a particular version of a dependency was chosen. It'll often even tell you something like "foo:bar:1.2 (was 1.1)". Once you figure out where the errant version is coming from, there are a number of ways to ensure a specific dependency version is used for a project:
Exclude wrongly-versioned dependencies from other dependencies that are causing them to be included in the build.
Add an explicit top-level dependency to your pom instead of relying on a transitive dependency.
List the dependency in the dependencyManagement section of your pom (scroll down a bit from this link) to force the dependency to have the specified characteristics, regardless of what level of transitive dependency it is. Use this option with care, as dependencyManagement is viral, in that other projects depending on your project will be "infected" with your dependency management. There's also a good section on dependency management in the pom reference.
If the 0.4.0 version is being pulled in as a transitive dependency via another dependency in your POM, then you should be able to exclude it. The dependency:tree goal should help you see if this is what's happening.
Maven is designed to favor release versions over snapshot versions. I'm not sure why you would have two dependencies in the same POM and not be able to resolve a conflict by removing one, so I will assume that one of the dependencies is inherited from a parent pom. In this case you can set the inherited dependency as <optional>true</optional> and I THINK it should allow the child POMs to override it, even with a lower version.
bad/hacky solution for if that doesn't work - edit your local repository in such a way that it doesn't realize the 0.5.0 version is a snapshot (or even edit your private nexus repo if you have the ability)