Let's say you have 4 projects depending on each other in chain:
A depends on B, B depends on C, C depends on D.
All projects are built using maven.
What's the best practice for keeping their versions in sync so that when someone updates project D or C it doesn't break the build of project A?
The idea behind Maven was to include modularity in different projects. This makes teams working on these projects independent and they can work on their respective modules without worrying about other teams.
But we may run into situations where a new version of dependent project has been published and we are not aware of it. If you are running Maven 2 to resolve this issue, you can use <version>LATEST</version> command in the pom.xml for that particular dependency. So it would be like -
<dependency>
<groupId>javax.portlet</groupId>
<artifactId>portlet-api</artifactId>
<version>LATEST</version>
</dependency>
But this was discontinued from Maven 3. As per official note from Maven -
Plugin Metaversion Resolution Internally, Maven 2.x used the special
version markers RELEASE and LATEST to support automatic plugin version
resolution. These metaversions were also recognized in the
element for a declaration. For the sake of reproducible
builds, Maven 3.x no longer supports usage of these metaversions in
the POM. As a result, users will need to replace occurrences of these
metaversions with a concrete version.
So it is advisable to always use concrete versions.
However, if you still want the latest versions of dependencies to be used, you can use below command in Maven command prompt -
mvn versions:use-latest-versions
Related
Our team has multiple projects; most projects are just libraries. Let's assume for simplicity that the libraries don't depend on each other and there is a single project that uses them, e.g.:
Project Main:
Project Lib-A:
X (3rd-party library)
Project Lib-B:
X (3rd-party library)
To avoid surprises in 'Main', we'd like to make sure that all our own projects are using the same versions of the 3rd-party libraries, so that e.g. both 'Lib-A' and 'Lib-B' are built and tested with the same version of library X.
To achieve this we use a parent pom with <dependencyManagement> section detailing the versions of all relevant 3rd-party libraries and also their transitive dependencies. This parent pom is inherited by all the projects, i.e. 'Main', 'Lib-A', and 'Lib-B' from the above example. Then each child pom would only use <dependency> without specifying any version. We also have maven enforcer plugin's dependencyConvergence rule to make sure we have not missed any library conflict in any of our projects.
The problem: increasing the version of X: a developer of 'Lib-A' increases a version of X from 1.0 to 2.0. So he changes X's version in the parent pom, increases the version of parent, releases the parent pom, and notifies guys from 'Main' that they should now use a new parent. The situation becomes like this:
Main - inherits from Parent:2.0 and depends on:
Lib-A:2.0 - inherits from Parent 2.0 and depends on X:2.0
Lib-B:1.0 - inherits from Parent 1.0 and depends on X:1.0
X:2.0 (taken from Parent:2.0 <dependencyManagement> section)
Everything, including 'Main', builds fine, 'maven enforcer plugin' does not detect any conflict because the version of X is clearly specified in the Parent:2.0 from which 'Main' inherits. So we release 'Main'.
Ooops.... Lib-B has never been built with X:2.0. It has great unit tests that would uncover the problem, but we never tried this. We forgot to update Lib-B, try it with X:2.0 and release it. Still 'Main' has been built without problems and maven enforcer plugin has never complained.
Question: we need maven to detect that there are dependencies that inherit from the same artifact but different major versions and fail the build.
In our case the build had to fail since 'Main' and 'Lib-A' inherit from Parent:2.0, but 'Lib-B' inherits from Parent:1.0.
My solution so far (a hack): in addition to inheriting, add an explicit dependency on the parent pom to all out projects (i.e. 'Main', 'Lib-A', and 'Lib-B'):
<dependency>
<artifactId>Parent</artifactId>
<type>pom</type>
<version>${project.parent.version}</version>
</dependency>
Then use <bannedDependencies> rule of maven enforcer plugin to ban other major Parent versions (we could also use its <dependencyConvergence/> rule if we want to fail even on minor Parent version conflicts).
Is there a less hacky and cumbersome way to fail on conflicting major versions of parent pom?May be our entire approach to managing maven dependencies is wrong, what is the recommended way then?
Update:
Tried writing my own rule for maven-enforcer-plugin as suggested by #JF Mayer and described here, before giving up. Reasons:
First, the parent pom information is not available from the dependencies, at least not from the nodes built by maven's DependencyGraphBuilder
OK, I've added my parent poms as explicit dependencies to the children and tried to use this DependencyGraphBuilder to detect dependencies on the parent with different major versions. No way! As could be seen with mvn dependency:tree that also uses this class, DependencyGraphBuilder does not provide all the dependencies so it can't be used to detect dependencies conflicts. That's why the <dependencyConvergence> maven enforcer rule is using a super-deprecated DefaultDependencyTreeBuilder that has been even deleted from the GitHub and everywhere else - not a good choice for a trouble-free custom solution.
For completeness, my own poor-man's solution:
Add an explicit dependency of type pom to the parent to every project so that maven-enforcer-plugin's <dependencyConvergence> rule would detect conflicting parent versions. No big deal with this one as we only add this section once and forget about it:
<dependency>
<groupId>${project.parent.groupId}</groupId>
<artifactId>${project.parent.artifactId}</artifactId>
<version>${project.parent.version}</version>
<type>pom</type>
</dependency>
<dependencyConvergence> will fail a build on 'Main' even on 'increment' differences in the parent versions, e.g. 1.0.1 and 1.0.2. In this case, the developer of 'Main' can decide that it's OK to build it despite the parent version conflict, because it's insignificant (this was my original question). So he builds 'Main' with some special profile that excludes the dependency on the parent: mvn -P I-know-what-I-am-doing deploy.
I am not very happy with this solution because of the step 2 that requires the developers of 'Main' to build it with a special profile in case of a parent versions conflict. I'd prefer a solution that always fails on major parent version conflicts but ignores insignificant differences in the parent pom versions automatically, I just don't know how to achieve this.
I think what you are looking for is Reactor Module Convergence enforcer:
https://maven.apache.org/enforcer/enforcer-rules/reactorModuleConvergence.html
Its not the same as the dependency convergence rule.
I'm rather new to Liferay, Maven and Java so it may be more of a general question about dependencies. I am maintaining a Liferay portlet that has been migrated from 6.2 to 7.1 and there are a number of Liferay maven dependencies with version numbers (eg. com.liferay.portal.kernel).
How does one know which versions of these dependencies one is to use for the version of the product that they are using?
Is this a typical case where one should always be trying to use the most recent version of dependencies even if the version of the product is a minor release behind?
Probably the easiest thing to do to make sure you compile agains JAR version that is in your target environment is to use the respective BOM (bill of material).
You can have a look at this code sample's POM for Liferay Portal 7.2 for example. Note the dependencyManagement that indicates which BOM to use:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.liferay.portal</groupId>
<artifactId>release.portal.bom</artifactId>
<version>${portal.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Then note how the actual dependency for com.liferay.portal.kernel JAR does not have a version specified.
<dependency>
<groupId>com.liferay.portal</groupId>
<artifactId>com.liferay.portal.kernel</artifactId>
</dependency>
The JAR version will be taken from the BOM which ensures it will match with the one the given version of Liferay Portal contains.
For comparison, here is the exact same POM but for Liferay Portal 7.1. As you can see the only difference is the portal.version property.
how does one know which versions of these dependencies one is to use
for the version of the product that they are using?
There are several ways to know which dependency versions you are using. The simplest one is to open up your bundle's jar files and look at the manifest file.
Of course, opening manifests is pretty tedious. You can use your system's Gogo shell to obtain information about the bundles on a running system.
Or you could look for the dependency on git. Use the Liferay tag that corresponds to your system, and subtract 1 from the minor portion of the version you see in the bnd file.
Finally, the logs can help you telling when a dependency is missing or a package version exists with a mismatch in version numbers.
Personally, I would say that the Gogo shell and App manager option is the easiest way.. but sometimes you are on git already..
is this a typical case where one should always be trying to use the
most recent version of dependencies even if the version of the product
is a minor release behind?
No, this is not a good thing for you. Although minors portions of the version scheme usually indicate that things are not likely to break, they do. If you use a method that was added in a minor version, on your running system that method will not be available and debugging can be confusing as you will clearly see that you IDE is even auto completing nonexistent stuff.
Additionally, there is no real advantage on using the latest version to compile your modules as the one running on the system is not updated, the one that will be running is the one that comes with your product (if you have not changed it, installed or embedded it even inside your on module... but if you did adjustments on your bundle, then it is up to you to track...).
You can use version ranges like 3.1.+ to build your modules, assuming that modules compiled with that dependency will work with all dot versions of it in a running system. If the dependency is known to be compatible with the older versions of itself, you can use older versions to build while the system will run a newer one. Liferay does this all the time in their own code (sometimes hidden by the word default). Of course, if you do this you will not be able to enjoy the latest features and IDE provided autocomplete and verification.
You also need to be aware that in OSGi based systems, several versions of the same library is possible. Sometimes, only one runs (singletons) but sometimes several will be available on runtime...so, you can pick the latest of the available...
So, in short: do not use the latest version to build if your system will not be running it. Maybe a range works, but you need to check if that dependency actually cares about being compatible inside that range, according to its versioning scheme.
Useful links:
lib/portal/dependencies.properties
modules/modules.properties
It's a long story. My current place uses Ant for builds. They create about 20 different foundation class jars that are used in multiple projects. Originally, the projects would check in the particular versions of the various jars they needed and then never updated them. That meant that each application had incompatible jars with other projects and our servers. Hilarity ensued.
To handle this, I immediately brought in a Maven repository, and integrated Ivy into our Ant builds. No more checking in jars. Instead, you fetch the correct version from the Maven repository. Originally, I expected the developers to keep the version numbers in the ivy.xml up to date, but they never did. Instead, the Ivy integration and setup depends upon an external Subversion project. This allowed me to integrate Ivy with minimal changes to the old build.xml files. I added a ivy.version.properties file to that external ivy project and maintain the version numbers of various jars in there. There's a corporate wide version number.
The various projects use the ${corporate.version} property for our foundation jars version numbers. When I update that ivy.version.properties file, all projects get updated with the right version number for our foundation classes. We use <ivy:makepom> to generate a pom.xml for our projects and use this to deploy our jars and wars into our Maven repository.
The result: I no longer have to worry about the developers keeping the version numbers of their projects in sync. I, as the release engineer handle that by updating that one ivy.version.properties file. All projects are in sync.
Now, we're moving to Maven, and I want to be able to do the same thing. I doubt developers will remember to update their pom.xml with the correct version numbers, so I want to read that in from another file, and use that.
There are two issues: One is that Maven first reads in the version number of a project before it executes any goal. No version number in the pom.xml, no version number for the generated jar.
Even if I manage to get by that first step, we have the fact that the pom.xml has no version number in it for foundation classes. When Maven pulls down the pom.xml to get the dependencies, it can't figure out which revision.
Yes, I could put that into a corporate pom.xml and have that as a parent project for all of the other projects. We already have a parent project to help set up various aspect of all projects. I could put a foundation class version number in there. However, that means that the developers now have to update the parent project's version number with each release. If developers can't be trusted to update the version number of their project with each release, what makes you think they'll do that with the parent's version for each release?
I know other people must have run into a similar issue. How do you handle this?
I could have an Ant script that generates the pom.xml from a template pom.xml, but that seems a bit silly.
I was wondering if is it possible for Maven to generate a pom.xml on the fly and then to use that to continue the executing the right goal. For example, I type in mvn package, and Maven will take a template.pom.xml file, fill in the missing version numbers to generate a generated.pom.xml file, then execute mvn package on that generated pom.
Or, is there a better way to solve this issue? Basically, I need to control the version number of our releases across all projects. This way, all projects are using the same version of our foundation classes. Also, I control some other versions of other jars (like log4j) this way. I've done this with Ant and Ivy, and now I want to move to Maven.
I think the best option is to create a pom.xml with all the dependencies to your packages in its and then import it in your developer project paren pom.xml using import
So, in the project parent POM:
<properties>
<corporate.version>...</corporate.version>
<properties>
...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.mycompany.libs</groupId>
<artifactId>foundation<artifactId>
<version>${corporate.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
...
</dependencies>
</dependencyManagement>
So for new set of foundation libraries, you deploy a new pom.xml with all the versions in it and update the ${corporate.version} in the relevant parent pom.xml file. You can even have when the versions are not yet fixed define a -SNAPSHOT artifact with these version numbers.
for more information, see: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Importing_Dependencies
We actually have the same kind of problem and I know that there is trade-off between "lazy programmers" who never update their poms and the stability aspect: Changing the versions of the foundation jars "suddenly" which may cause unexpected build fails.
If you really want the behaviour you describe, I would put the desired dependencies into a SNAPSHOT pom and use it either as a parent pom or import it as a BOM. In this way, you can change the content and everybody pulls in the new content with the next update.
I've decided that this a stupid idea and should never have been done. It's the developers' responsibility to update their own damn POM and make sure they're pulling the correct version. This is the way it really should be and developers should simply learn to do it (what's the technical term? ...oh yeah) the Correct Way.
I originally implemented this because the company had something like this (which wasn't working) and I had problems getting the developers to do what should be their job. And, the developers liked it because I was now officially responsible when something goes wrong rather than them. It's what you get when you do someone else's job.
I've convinced the company to stop trying to keep all our core jars (the jars used in our other projects) and our releases at the same version number. Instead, every project will have its own versioning. Jars will version only when there's an actual change and not when we force an upgrade to the version. And, the developers must know what versions of the various jars they depend upon.
Use the maven-release-plugin. This plugin can stamp a version number on all pom.xml files. The plugin also understands version control and knows how to tag a maven project for release (or branching).
I have a maven project with multiple modules. When I release it I just change the versions of the modules from SNAPSHOT to release's version and its ok. This can be done with Maven Release Plugin.
The problem arises because some of the dependencies I have are actually the artifacts, developed by other groups of our programmers. Thus their versions may often change, which is a behaviour opposite to other dependencies, for example hibernate's artifact versions. At the moment of release I would like to use some available versions of that rapidly changing libraries. Probably the last one. May be they will release a new version of their library specially for my release.
Note that their library is a separate Maven project with separate version controlled by them.
All I can do now is just to check manually which version of the that dependency is the last and put it down manually into my POM. Its not that convenient. May be there is a better way to organize it with Maven and TeamCity? Can I update the versions of that other group's artifacts too? Their version should be derived from their Snapshot version, or from the last release they have deployed into the Nexus.
You can use versions-maven-plugin to automate updating external dependencies.
As mentioned, you can use Versions Maven Plugin, and more specifically you need versions:update-properties. As you can read the manual, it
Sets properties to the latest versions of specific artifacts.
The condition is that you work with repository manager (e.g. Artifactory). Maven knows to search this repo for the most updated aritfacts.
Before you run the maven-release-plugin, you run the versions-plugin that updates your dependencies. For example, you run versions:update-properties with the relevant parameters.
If you would like to print latest versions of artifacts, the same versions-plugin is your friend. Have a look and read the link I've sent you above; the relevant command is versions:display-dependency-updates.
If you would like to print selectively, only your artifacts latest versions, you can set their version using a property. for example, if you have dependency JAR X, write in the main pom something like this:
<dependency>
<groupId>myGroup</groupId>
<artifactId>X</artifactId>
<version>${x.version}</version>
</dependency>
<properties>
<x.version>3.1.0.RELEASE</x.version>
</properties>
Then you use versions:display-property-updates -DincludeProperties="x.version"
My team has a common parent project with a module containing several reporting plugin configurations (e.g. checkstyle and findbugs, similar to Checkstyle's multimodule configuration, but in a separate project). I'm going to call the common parent project "common" and the reporting module "build-tools".
I'm trying to find a way to, when the common project is released, to have the common project reference the correct version of the build-tools module without doing a manual release.
Here are a couple of the things I've tried:
Use ${project.version} for the build-tools version number. This uses the version number specified in the projects using the common as a parent.
Use regular version numbers. These are not updated in the common project.
Use a property. Again, the property value isn't updated on release.
Thanks!
Which version of the maven-release-plugin are you using? Try 2.1. That should properly handle the replacement of version properties.
The only way I know to do this is to make common and build-tools the same version and use -DautoVersionSubmodules when you release:prepare. Since common aggregates build-tools, if both modules have the same SNAPSHOT version when you do a release, the release plugin will release and uprev both of them.
Edit: To keep the dependency version correct, your first option should work. In common:
<dependency>
<groupId>com.foo</groupId>
<artifactId>build-tools</artifactId>
<version>${project.version}</version>
</dependency>
That will make your common project always have a dependency on build-tools with the same version as common itself. If they always uprev in lock-step, that should do what you want. Is there a problem with it?