It's a long story. My current place uses Ant for builds. They create about 20 different foundation class jars that are used in multiple projects. Originally, the projects would check in the particular versions of the various jars they needed and then never updated them. That meant that each application had incompatible jars with other projects and our servers. Hilarity ensued.
To handle this, I immediately brought in a Maven repository, and integrated Ivy into our Ant builds. No more checking in jars. Instead, you fetch the correct version from the Maven repository. Originally, I expected the developers to keep the version numbers in the ivy.xml up to date, but they never did. Instead, the Ivy integration and setup depends upon an external Subversion project. This allowed me to integrate Ivy with minimal changes to the old build.xml files. I added a ivy.version.properties file to that external ivy project and maintain the version numbers of various jars in there. There's a corporate wide version number.
The various projects use the ${corporate.version} property for our foundation jars version numbers. When I update that ivy.version.properties file, all projects get updated with the right version number for our foundation classes. We use <ivy:makepom> to generate a pom.xml for our projects and use this to deploy our jars and wars into our Maven repository.
The result: I no longer have to worry about the developers keeping the version numbers of their projects in sync. I, as the release engineer handle that by updating that one ivy.version.properties file. All projects are in sync.
Now, we're moving to Maven, and I want to be able to do the same thing. I doubt developers will remember to update their pom.xml with the correct version numbers, so I want to read that in from another file, and use that.
There are two issues: One is that Maven first reads in the version number of a project before it executes any goal. No version number in the pom.xml, no version number for the generated jar.
Even if I manage to get by that first step, we have the fact that the pom.xml has no version number in it for foundation classes. When Maven pulls down the pom.xml to get the dependencies, it can't figure out which revision.
Yes, I could put that into a corporate pom.xml and have that as a parent project for all of the other projects. We already have a parent project to help set up various aspect of all projects. I could put a foundation class version number in there. However, that means that the developers now have to update the parent project's version number with each release. If developers can't be trusted to update the version number of their project with each release, what makes you think they'll do that with the parent's version for each release?
I know other people must have run into a similar issue. How do you handle this?
I could have an Ant script that generates the pom.xml from a template pom.xml, but that seems a bit silly.
I was wondering if is it possible for Maven to generate a pom.xml on the fly and then to use that to continue the executing the right goal. For example, I type in mvn package, and Maven will take a template.pom.xml file, fill in the missing version numbers to generate a generated.pom.xml file, then execute mvn package on that generated pom.
Or, is there a better way to solve this issue? Basically, I need to control the version number of our releases across all projects. This way, all projects are using the same version of our foundation classes. Also, I control some other versions of other jars (like log4j) this way. I've done this with Ant and Ivy, and now I want to move to Maven.
I think the best option is to create a pom.xml with all the dependencies to your packages in its and then import it in your developer project paren pom.xml using import
So, in the project parent POM:
<properties>
<corporate.version>...</corporate.version>
<properties>
...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.mycompany.libs</groupId>
<artifactId>foundation<artifactId>
<version>${corporate.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
...
</dependencies>
</dependencyManagement>
So for new set of foundation libraries, you deploy a new pom.xml with all the versions in it and update the ${corporate.version} in the relevant parent pom.xml file. You can even have when the versions are not yet fixed define a -SNAPSHOT artifact with these version numbers.
for more information, see: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Importing_Dependencies
We actually have the same kind of problem and I know that there is trade-off between "lazy programmers" who never update their poms and the stability aspect: Changing the versions of the foundation jars "suddenly" which may cause unexpected build fails.
If you really want the behaviour you describe, I would put the desired dependencies into a SNAPSHOT pom and use it either as a parent pom or import it as a BOM. In this way, you can change the content and everybody pulls in the new content with the next update.
I've decided that this a stupid idea and should never have been done. It's the developers' responsibility to update their own damn POM and make sure they're pulling the correct version. This is the way it really should be and developers should simply learn to do it (what's the technical term? ...oh yeah) the Correct Way.
I originally implemented this because the company had something like this (which wasn't working) and I had problems getting the developers to do what should be their job. And, the developers liked it because I was now officially responsible when something goes wrong rather than them. It's what you get when you do someone else's job.
I've convinced the company to stop trying to keep all our core jars (the jars used in our other projects) and our releases at the same version number. Instead, every project will have its own versioning. Jars will version only when there's an actual change and not when we force an upgrade to the version. And, the developers must know what versions of the various jars they depend upon.
Use the maven-release-plugin. This plugin can stamp a version number on all pom.xml files. The plugin also understands version control and knows how to tag a maven project for release (or branching).
Related
Consider a project that is used in other projects, and this project has version 1.0 and is present in the Nexus.
After that, some changes are done in this project.
Now there are two solutions:
Increment the version of the project to 2.0, and delete the version 1.0 from Nexus. When the developers try to get the dependencies from the Nexus with version 1.0 they will get an error that this version does not exist and need to change the version to 2.0.
Change the functionalities of this project and inform the crew that some changes are done, but this is not the practice at all.
Is there any functionality in Maven and Nexus to simplify this task and make this all happen in the backend so the developers can't do anything, or is this not possible?
If you have an old version of an artifact that must not be used anymore because it has some dangerous bug, or it does not work with the new database structure or something like this, it may be advisable to move it to some non-public Nexus repository (and also delete it from the local repository of the build server), so that nobody can use it for release builds (people can use it for local builds, but this is usually not dangerous).
If you want to manage standard versions throughout your company, it is a good idea to have a parent pom or some boms which collect versions in a <dependenyManagment> section and can be included by the developers. This way, you only need to inform them to change one version number (namely the one of the parent pom or bom) instead of many.
Still, you are left with the problem that people do not read company newsletters. I know the problem that many developers of jars compile and test their source code against very old versions of their dependencies while the war/ear (that includes the jar) uses new versions.
This is probably supported, so I apologize if the answer is trivial, but I can't find an answer.
I have a multimodule (aggregate) pom containing a number of modules with interdependencies. Let's say something like:
infrastructure
framework
business-module1
business-module2
At a certain moment, infrastructure is stable, so I can release it. The version is updated to 1.0 and all dependencies to it are updated to that version as well.
When doing a local install, the module is still build. I know want to avoid building the 1.0 version locally, since it is released and costs build time. The question could be rephrased to, never build a released module.
I have quite a lot of modules in a multimodule setup, and only want to build snapshots. I know that I can skip modules using command line options, but don't want to repeat all these modules. Besides, from the version declarations, it should be clear to maven which modules to build.
I have a maven project with multiple modules. When I release it I just change the versions of the modules from SNAPSHOT to release's version and its ok. This can be done with Maven Release Plugin.
The problem arises because some of the dependencies I have are actually the artifacts, developed by other groups of our programmers. Thus their versions may often change, which is a behaviour opposite to other dependencies, for example hibernate's artifact versions. At the moment of release I would like to use some available versions of that rapidly changing libraries. Probably the last one. May be they will release a new version of their library specially for my release.
Note that their library is a separate Maven project with separate version controlled by them.
All I can do now is just to check manually which version of the that dependency is the last and put it down manually into my POM. Its not that convenient. May be there is a better way to organize it with Maven and TeamCity? Can I update the versions of that other group's artifacts too? Their version should be derived from their Snapshot version, or from the last release they have deployed into the Nexus.
You can use versions-maven-plugin to automate updating external dependencies.
As mentioned, you can use Versions Maven Plugin, and more specifically you need versions:update-properties. As you can read the manual, it
Sets properties to the latest versions of specific artifacts.
The condition is that you work with repository manager (e.g. Artifactory). Maven knows to search this repo for the most updated aritfacts.
Before you run the maven-release-plugin, you run the versions-plugin that updates your dependencies. For example, you run versions:update-properties with the relevant parameters.
If you would like to print latest versions of artifacts, the same versions-plugin is your friend. Have a look and read the link I've sent you above; the relevant command is versions:display-dependency-updates.
If you would like to print selectively, only your artifacts latest versions, you can set their version using a property. for example, if you have dependency JAR X, write in the main pom something like this:
<dependency>
<groupId>myGroup</groupId>
<artifactId>X</artifactId>
<version>${x.version}</version>
</dependency>
<properties>
<x.version>3.1.0.RELEASE</x.version>
</properties>
Then you use versions:display-property-updates -DincludeProperties="x.version"
I am attempting to create a way in which hermetic builds can be achieved while still relying on SNAPSHOT dependencies in your project.
For the purposes of example, say I have a project which has a dependency structure like this:
┌ other-1.2-SNAPSHOT
mine-1.2.3 ──┤
└ thing-3.1-SNAPSHOT ── gizmo-6.1.3-SNAPSHOT
What I would like to do is resolve all the SNAPSHOT dependencies locally to something which is related to my current version and then deploy those as releases to my Nexus' release repository. Not all of these dependencies are internal so I cannot simply just make a release on each.
So, in this example, other-1.2-SNAPSHOT would become something like other-1.2-mine-1.2.3 and thing-3.1-SNAPSHOT would become thing-3.1-mine-1.2.3. This is relatively trivial in about 60 lines of python.
The problem, however, is in resolving transitive SNAPSHOTs to concrete versions. So I also need to convert gizmo-6.1.3-SNAPSHOT to gizmo-6.1.3-mine.1.2.3 and have thing-3.1-mine-1.2.3 depend on it.
This is only an example of one way in which to achieve what I want. The goal is that in a year or two down the road I can checkout my release branch for version 1.2.3 and be able to run mvn clean package or the like without having to worry about resolving long-since-gone SNAPSHOT dependencies.
It's important that this branch be compilable and not just retain all dependencies using something like the jar-and-dependencies functionality of the assembly plugin. I'd like to potentially be able to modify the source files and make another release build (e.g., applying a hotfix).
So,
Is there anything like this available that will be able to convert SNAPSHOT dependencies in a recursive fashion to be concrete?
Are there any plugins which manage this kind of thing for you? The release plugin had promise with some configuration options on its branch goal but it doesn't resolve external deps to the degree that I want.
Are other techniques available for creating hermetic Maven builds?
This is not a widely used technique, but you can always check your specific SNAPSHOT dependencies into your project as a "project" repository, as described in this blog post: Maven is to Ant as a Nail Gun is to a Hammer
In short, use the Dependencies Plugin to create repository located in your project directory. The below is copied from the linked blog post (which you should read):
1) Run mvn -Dmdep.useRepositoryLayout=true -Dmdep.copyPom=true dependency:copy-dependencies
"This creates /target/dependencies with a repo-like layout of all your projects dependencies"
2) Copy target/dependencies/ to something like libs/
3) Add a repository declaration like the following to your POM:
<repositories>
<repository>
<releases />
<id>snapshots-I-need-forever</id>
<name>snapshots-I-need-forever</name>
<url>file:///${basedir}/libs</url>
</repository>
</repositories>
You make this an automated part of your build/release process: step 1 by configuring the Dependencies plugin to a lifecycle phasephase, and step 2 using AntRun Plugin to move the downloaded dependencies to the right place..
Hope this works for you. I have to go take a shower now...
The maven versions plugin will do most of what you want.
http://mojo.codehaus.org/versions-maven-plugin/
However you will almost certianly need to run it in a pre-build step in which you resolve all the dependencies and update the pom file accordingly. Then re-run maven (which re-reads the pom) to run the real build. You might be able to configure everything within the pom itself triggered with a separate goal thus avoiding a separate script.
This works better if you use particular versions instead of SNAPSHOT dependencies and let the pre-build step upgrade them if necessary. The only real difference for dependency resolution is that maven will always re-download -SNAPSHOT dependencies whereas it will only download normal dependencies if there is a new version available. However many plugins (including the versions plugin) treat -SNAPSHOT dependencies differently causing problems. Since every CI build has a new version number I never use -SNAPSHOT, prefering a different tag like -DEV with more predictable behaviour for things like developer local builds etc.
I've spent a lot of time getting maven to do things similar to this. Most maven projects I know have some kind of pre-build step in order to set version numbers or get around other limitations such as this. Trying to do all this in one step usually fails because maven only reads the pom once, string substitution doesn't work in a few places and the deployed/installed pom doesn't generally doesn't contain the results of string substituion or changes made during the build.
What are the best practices for software versioning and multimodules projects with Maven?
I mean, when I create a multimodules project with Maven, what is the best approach for the versioning? To use a single version for all the modules (defined in the top project)? To use a version for each module (defined in the POM of each module)? Is there another approach that I'm missing? What are the pros and cons of each approach?
In general, are the different modules released together (possibly sharing the same version number)?
Thanks
Honestly it depends on what you would like to do. Multimodule projects are created for multiple reasons, one of them being you only need to deploy what has changed instead of all modules.
Think about it this way: if you had a non-multi-module project and you only had to change one line in the services layer, you have to rebuild the entire project and deploy all of the code again...even though only your services layer will change.
With multi-module projects, you can regenerate your project and deploy only what changed...your services. This reduces risk and you're assured that only your services module changed.
You also have a multitude of benefits to using multi-module projects that I'm not listing here but there is certainly a huge benefit to NOT keeping your version numbers of your modules in sync.
When you build your project, consider deploying it to a repository that will hold all compatible jars together for builds (each build creates a new folder with the parent-most pom version number). That way, you don't need to keep documentation about which jars are compatible...they're all just deployed together with a build number.
I was looking for a solution for this exact problem myself and versions-maven-plugin was exactly what I needed. I don't like the release plugin communicating with the SCM system. The versions plugin does just what we need: it sets a new version number in all poms of the project:
mvn versions:set -DnewVersion=2.0.0
Then I can proceed with commits, tags and an official build server build...
EDIT:
The versions plugin depends on how a maven multi-module project has been organised: as a result, it often does not update all POM files in a complex multi-module project.
I've found that sed and find do the job much more reliably:
sed -i 's/1.0.0-SNAPSHOT/1.0.0-RC1/g' `find . -name 'pom.xml'`
Typically you create a multi-module project because you have deemed that the various modules are parts of a single whole. Maybe the client-piece, the controller-piece and the services-piece. Or maybe the UI with services.
In any case, it makes sense to have the version numbers for the various modules to move in lock-step. However Maven does not enforce that as a rule.
As to your question
are the different modules released together (possibly sharing the same
version number)
I would think so. That is one of the reasons for having it a multi-module project. Otherwise you could have the modules as independent projects.
Of course this is the kind of stuff that is rife with edge cases and exceptions ;-)
I had the same problem with a project I`m working on. I also decided to use separate versions and even the dependency to the parent pom only has to be updated if some of the managed dependencies change. (so mostly as #vinnybad describes it)
Two additions
exists-maven-plugin
With the usage of the "org.honton.chas.exists-maven-plugin" only the modules will be deployed to the repository that have actually changed, which is really great, because also the according docker-images will only be published if something has changed on one of the service. This avoids "polluting" the image repository with different but unchanged versions.
versioning
One main downside of the "separated versions" approach are the questions regarding versioning:
What's the current version of my project?
Which module versions work with each other? (even thought they don't directly depend on each other, one does rely on what another does, e.g. they share the database schema)
To solve that I put all module versions into the dependency management part of the parent pom, even if no other module depends on them. A "integration-test" module could solve that by depending on all of the modules - and of course testing them together.
This way I would be "forced" to update the parent pom with every change, since it's referring the released module versions. This way the parent pom would have the "leading" version and at the dependency-management block state the versions of all modules that are compatible with each other (which will be ensured by the integration test).