Our team has multiple projects; most projects are just libraries. Let's assume for simplicity that the libraries don't depend on each other and there is a single project that uses them, e.g.:
Project Main:
Project Lib-A:
X (3rd-party library)
Project Lib-B:
X (3rd-party library)
To avoid surprises in 'Main', we'd like to make sure that all our own projects are using the same versions of the 3rd-party libraries, so that e.g. both 'Lib-A' and 'Lib-B' are built and tested with the same version of library X.
To achieve this we use a parent pom with <dependencyManagement> section detailing the versions of all relevant 3rd-party libraries and also their transitive dependencies. This parent pom is inherited by all the projects, i.e. 'Main', 'Lib-A', and 'Lib-B' from the above example. Then each child pom would only use <dependency> without specifying any version. We also have maven enforcer plugin's dependencyConvergence rule to make sure we have not missed any library conflict in any of our projects.
The problem: increasing the version of X: a developer of 'Lib-A' increases a version of X from 1.0 to 2.0. So he changes X's version in the parent pom, increases the version of parent, releases the parent pom, and notifies guys from 'Main' that they should now use a new parent. The situation becomes like this:
Main - inherits from Parent:2.0 and depends on:
Lib-A:2.0 - inherits from Parent 2.0 and depends on X:2.0
Lib-B:1.0 - inherits from Parent 1.0 and depends on X:1.0
X:2.0 (taken from Parent:2.0 <dependencyManagement> section)
Everything, including 'Main', builds fine, 'maven enforcer plugin' does not detect any conflict because the version of X is clearly specified in the Parent:2.0 from which 'Main' inherits. So we release 'Main'.
Ooops.... Lib-B has never been built with X:2.0. It has great unit tests that would uncover the problem, but we never tried this. We forgot to update Lib-B, try it with X:2.0 and release it. Still 'Main' has been built without problems and maven enforcer plugin has never complained.
Question: we need maven to detect that there are dependencies that inherit from the same artifact but different major versions and fail the build.
In our case the build had to fail since 'Main' and 'Lib-A' inherit from Parent:2.0, but 'Lib-B' inherits from Parent:1.0.
My solution so far (a hack): in addition to inheriting, add an explicit dependency on the parent pom to all out projects (i.e. 'Main', 'Lib-A', and 'Lib-B'):
<dependency>
<artifactId>Parent</artifactId>
<type>pom</type>
<version>${project.parent.version}</version>
</dependency>
Then use <bannedDependencies> rule of maven enforcer plugin to ban other major Parent versions (we could also use its <dependencyConvergence/> rule if we want to fail even on minor Parent version conflicts).
Is there a less hacky and cumbersome way to fail on conflicting major versions of parent pom?May be our entire approach to managing maven dependencies is wrong, what is the recommended way then?
Update:
Tried writing my own rule for maven-enforcer-plugin as suggested by #JF Mayer and described here, before giving up. Reasons:
First, the parent pom information is not available from the dependencies, at least not from the nodes built by maven's DependencyGraphBuilder
OK, I've added my parent poms as explicit dependencies to the children and tried to use this DependencyGraphBuilder to detect dependencies on the parent with different major versions. No way! As could be seen with mvn dependency:tree that also uses this class, DependencyGraphBuilder does not provide all the dependencies so it can't be used to detect dependencies conflicts. That's why the <dependencyConvergence> maven enforcer rule is using a super-deprecated DefaultDependencyTreeBuilder that has been even deleted from the GitHub and everywhere else - not a good choice for a trouble-free custom solution.
For completeness, my own poor-man's solution:
Add an explicit dependency of type pom to the parent to every project so that maven-enforcer-plugin's <dependencyConvergence> rule would detect conflicting parent versions. No big deal with this one as we only add this section once and forget about it:
<dependency>
<groupId>${project.parent.groupId}</groupId>
<artifactId>${project.parent.artifactId}</artifactId>
<version>${project.parent.version}</version>
<type>pom</type>
</dependency>
<dependencyConvergence> will fail a build on 'Main' even on 'increment' differences in the parent versions, e.g. 1.0.1 and 1.0.2. In this case, the developer of 'Main' can decide that it's OK to build it despite the parent version conflict, because it's insignificant (this was my original question). So he builds 'Main' with some special profile that excludes the dependency on the parent: mvn -P I-know-what-I-am-doing deploy.
I am not very happy with this solution because of the step 2 that requires the developers of 'Main' to build it with a special profile in case of a parent versions conflict. I'd prefer a solution that always fails on major parent version conflicts but ignores insignificant differences in the parent pom versions automatically, I just don't know how to achieve this.
I think what you are looking for is Reactor Module Convergence enforcer:
https://maven.apache.org/enforcer/enforcer-rules/reactorModuleConvergence.html
Its not the same as the dependency convergence rule.
Related
We have a custom plugin that runs during the initialize phase of our Maven build, and sets certain properties to the project. We also have a dependency, whose version relies on those properties. Our build is failing because the dependency version is invalid, because the project attempts to resolve the dependency before the plugin runs. Is there any way around this? Potential solutions (that I don't know how to implement):
Somehow allow the plugin to run before the project really even kicks off/attempts to resolve dependencies
delay the dependency resolution until later in the lifecycle
Are either of these possible?
Edit: Following a couple questions on context I'll provide some.
For a couple of different reasons, maintaining versions in a large parent (aggregator) pom file was becoming a big problem for us, so it actually started to make sense for us to define them in an external service. The plugin reads in the current versions off of the service, and sets them to properties. A couple of these properties would be used as dependency versions.
Anticipating the comments to be -> "you should just go back to using the large aggregator", we can.... but it is definitely as a last resort.
Edit 2: relevant part of POM per request:
<dependencies>
<dependency>
<groupId>com.myCompany</groupId>
<artifactId>myDep</artifactId>
<version>${myDep.major.version}.${myDep.minor.version}</version>
<type>ear</type>
</dependency>
</dependencies>
Traditionally myDep.major.version and myDep.minor.version we're defined in our parent pom.xml, but now we want to move them to the service we're using.
Let's say you have 4 projects depending on each other in chain:
A depends on B, B depends on C, C depends on D.
All projects are built using maven.
What's the best practice for keeping their versions in sync so that when someone updates project D or C it doesn't break the build of project A?
The idea behind Maven was to include modularity in different projects. This makes teams working on these projects independent and they can work on their respective modules without worrying about other teams.
But we may run into situations where a new version of dependent project has been published and we are not aware of it. If you are running Maven 2 to resolve this issue, you can use <version>LATEST</version> command in the pom.xml for that particular dependency. So it would be like -
<dependency>
<groupId>javax.portlet</groupId>
<artifactId>portlet-api</artifactId>
<version>LATEST</version>
</dependency>
But this was discontinued from Maven 3. As per official note from Maven -
Plugin Metaversion Resolution Internally, Maven 2.x used the special
version markers RELEASE and LATEST to support automatic plugin version
resolution. These metaversions were also recognized in the
element for a declaration. For the sake of reproducible
builds, Maven 3.x no longer supports usage of these metaversions in
the POM. As a result, users will need to replace occurrences of these
metaversions with a concrete version.
So it is advisable to always use concrete versions.
However, if you still want the latest versions of dependencies to be used, you can use below command in Maven command prompt -
mvn versions:use-latest-versions
We have a big and deep module structure with three "real" layers of inheritance and three to four layers of module aggregation.
I am fine with the real inheritance: a company wide super-pom, a product-wide parent and a customer-wide parent for customizations of the product.
But I observe that also the "empty" aggregating modules are defined as parents of their submodules.
If the whole concept is the same as in OO, this makes no sense to me if the aggregating modules (they are empty besides their submodules) add no specific configuration to the pom.
Is there any other reason (perhaps operational) why this could be useful?
Sidenote: Introduction to the pom is not clear in this respect: term "parent" is not clear: it can mean super-pom (=parent in inheritance tree) or aggregating pom (=parent in filesystem...)
There is a great problem in the definition of these concepts in the Maven community. You are right with stating that a parent-pom is inheritance and module-pom is composition. But unfortunately the separation of these two concepts have no history in Maven. The Maven documentation says clearly that it is better to separate them. This would result in this ideal structure
app-api/
app-impl/
app-war/
app-parent/
pom.xml
Where pom.xml is the module pom.
But almost every project you will see in the open source sector does not distinguish between them.
This has a reason: Many plugins do not distiguish between them as well. And even Maven itself assumes another setting: If <relativePath> is not set Maven assumes that the parent is at ... So every project has to point to <relativePath>../app-parent</relativePath> as parent.
The most popular plugin that has huge problems with the mentioned structure is the maven-release-plugin! You will face strange problems when you do not follow the assumption that the module-pom is the parent-pom.
The worst bug is that the release-plugin cannot replace version-properties in your parent and fails because of SNAPSHOT-Dependencies. Your parent perhaps will contain something like this
<properties>
<app-api.version>1.1-SNAPSHOT</app-api.version>
<app-impl.version>1.2-SNAPSHOT</app-impl.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>myorg</groupId>
<artifactId>app-api</artifactId>
<version>${app-api.version}</version>
</dependency>
<dependency>
<groupId>myorg</groupId>
<artifactId>app-impl</artifactId>
<version>${app-impl.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
Normally while releasing these properties would automatically be set to their release versions 1.1 and 1.2. But the release-plugin (tested up to version 2.2.2) fails with a message that the module cannot be released with snapshot dependencies.
If you only "skip" some module-poms this might not be a problem when you define <relativePath> correctly. But you have to try and expect some bugs in Maven plugins. Besides these bugs you are totally right to seperate the poms and I would give it a try if you have the time to make a sandboxed release.
The short answer IIUC is, you don't inherit from your aggregation-only poms. You do when your aggregator is the same as your parent, as in the canonical maven structure. But if you separate aggregator from parent, don't inherit from the aggregator
The inheritance of a parent pom is usefull if you have properties that are valid for several projects. From what I understood you already got that part right ;-)
What you call aggregating modules has a valid practical use too. You can group several submodules into a single pom. If you want to build your project you mostly don't want to build every module sperately but join them into one single application. This can be done by such a pom that references all the submodules as such. It can't be done with just the concept of a parent pom as it knows nothing of the other modules.
Adding an eclipse perspective here :
when you create maven module within a maven project using eclipse IDE's wizard, by default, the new module will created as child of top level project and also top level project will be an aggregation of child.
When there is a conflict in the dependency tree (same artifact but different versions) then, AFAIK, Maven will resolve the conflict by selecting the highest version of the dependency and will omit the 'old' ones.
However, when the newer version is a SNAPSHOT then apparently it will choose the older stable version over the SNAPSHOT.
In my case: some-artifact: 0.5.0-SNAPSHOTS (omitted for conflict with 0.4.0) => version 0.4.0 is picked over the wanted 0.5.0-SNAPSHOT.
I assume this functions as designed but I don't understand the reason why. Next to that, any idea if there is a way to tell Maven to take the SNAPSHOT over the stable version?
Your assumption about Maven's always selecting the highest version isn't accurate. Artifacts are chosen based on a number of factors including depth of the dependency in the tree, order in the tree, whether the dependency is a snapshot or a release, and dependency management, which pretty much overrides everything else.
Unfortunately, I don't know of any one, definitive source of information on Maven's dependency resolution algorithms. You'll find bits and pieces of it scattered all over. A few handy references:
Introduction to the Dependency Mechanism gives an overview of the topic with a good, if short, section on Transitive Dependencies and how they're selected from a dependency tree.
The Sonatype Maven book has a more thorough section on Project Dependencies in general that will add a lot to your knowledge about the subject.
An earlier section of that same book discusses Project Versions, which is strongly related to this problem and has a good section on SNAPSHOT versions, though not as much as I could wish on how they play into dependency resolution.
Project Relationships talks about the coordinate system and how project inheritance affects what dependencies get included.
Finally, the POM Reference is a good jumping-off point for almost anything to do with the pom. There's at least a brief description of every pom element that can help you understand enough to be able to begin searching for more info effectively.
As for some practical advice, the output of mvn dependency:tree is highly useful in discovering why a particular version of a dependency was chosen. It'll often even tell you something like "foo:bar:1.2 (was 1.1)". Once you figure out where the errant version is coming from, there are a number of ways to ensure a specific dependency version is used for a project:
Exclude wrongly-versioned dependencies from other dependencies that are causing them to be included in the build.
Add an explicit top-level dependency to your pom instead of relying on a transitive dependency.
List the dependency in the dependencyManagement section of your pom (scroll down a bit from this link) to force the dependency to have the specified characteristics, regardless of what level of transitive dependency it is. Use this option with care, as dependencyManagement is viral, in that other projects depending on your project will be "infected" with your dependency management. There's also a good section on dependency management in the pom reference.
If the 0.4.0 version is being pulled in as a transitive dependency via another dependency in your POM, then you should be able to exclude it. The dependency:tree goal should help you see if this is what's happening.
Maven is designed to favor release versions over snapshot versions. I'm not sure why you would have two dependencies in the same POM and not be able to resolve a conflict by removing one, so I will assume that one of the dependencies is inherited from a parent pom. In this case you can set the inherited dependency as <optional>true</optional> and I THINK it should allow the child POMs to override it, even with a lower version.
bad/hacky solution for if that doesn't work - edit your local repository in such a way that it doesn't realize the 0.5.0 version is a snapshot (or even edit your private nexus repo if you have the ability)
We have a maven module that is set up as so:
a (parent)
-> b (submodule)
-> c (submodule)
-> d (submodule)
This list of submodules is set to grow as time goes on (to a list of 20 or so). We have another module that will include as dependencies all submodules of a. Is there a neat way of doing this, rather than having to manually keep the submodule list in sync with the dependencies list. I.e. is there any way of including a and all submodules as a dependency?
You have a few choices here:
No change. List all dependencies in each pom. However, if they have a common parent, you can use dependencyManagement in the parent pom to set the versions for the different dependencies. In the child poms, you do not need to list the version. See this section from Maven By Example. A downside of this approach is that you have to re-list the same dependencies over and over.
Create a parent pom which lists all shared dependencies. See an example here. A downside here is you are restricting all projects that want to take advantage of this to use a parent project when they may need to use another parent project for some reason.
Wait for Maven mixins, which last I heard were still not ready.
Rethink your design. Does it make sense for projects to depend on so many different modules and interfaces? To reduce coupling, you may want to create one interface that these new projects can use. Several open source multi-module projects, such as Apache Axis2, follow this pattern. One module contains your 20 dependencies and exposes an interface which the new modules can call. The new modules can just list that one main module as a dependency and all of the 20 dependencies are pulled in as transitive dependencies.
I think choice #4 is probably right, but I am not familiar enough with your situation.
Design considerations aside, this is easily done - simply include <type>pom</type> in your dependency pointing at the parent pom. E.g:
<dependency>
<groupId>my.group.id</groupId>
<artifactId>a</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>pom</type>
</dependency>
The only way dependencies get "automatically" included is via the transitive dependency mechanism, and it only works by pulling in dependencies of dependencies, so unless you have a pom that depends on all the submodules, no, you won't get them without listing them all out. A need to do this points to a flaw in your project design. If the modules are "all or none", then they're probably not separate modules.