Maven branch update projects version automating release workflow - maven

I have a difficult case to solve regarding the Maven automated projects versioning, I hope I'll find a proper solution based on your experience and advices.
The problem is like that:
We have a huge mavenized java product which comprise of ~200 very interdependent different projects.
We agreed that each project should be developed independently, so that each of them should have it's own lifecycle.
It's all working fine in the development stage, there is no problem. The problem comes when we are preparing the release for these projects: Because the are so many projects the manual changes are a pain so we decided to find an automated solution to solve the release process.
The prequisites are these:
We all agreed that the release policy from SVN perspective should be like that:
- all development should be performed on SVN trunk, releases should be created and maintained on branches. Each performed release should automatically create a tag.
The policy from MAVEN perspective is like that:
- before releasing a project, we first copy the trunk to a branch in order to have control over projects maintainance on the branched code. The versioning system we choose is: Major.Minor.BuildNumber-SNAPSHOT (e.g. 1.0.0-SNAPSHOT). When branching the code, we want to change the project version number by incrementing the MinorVersion (e.g trunk-1.0.0-SNAPSHOT will become 1.1.0-SNAPSHOT, and 1.0.0-SNAPSHOT will be copied and released on the new created branch)
- when we decide that the project is mature enough in order to be released we are releasing it by using maven-release-plugin (mvn release:clean release:prepare release:perform) so that our project version will be transformed from Major.Minor.BuildVersion-SNAPSHOT (e.g. 1.0.0-SNAPSHOT) to Major.Minor.BuildVersion (e.g. 1.0.0), then will be prepared for the next development iteration like: Major.Minor.BuildVersion+1-SNAPSHOT (e.g. 1.0.1-SNAPSHOT)
The problems we are facing are related to projects versioning.
So, during the development phase on the trunk, all the projects are using the latest SNAPSHOT versions of their dependencies (mvn versions:use-latest-versions -DallowSnapshots=true -DupdateDependencies=true), but when we consider it's time to start the release procedure and prepare to branch the code, there problems start:
we are start branching
parent-pom
(mvn -B release:clean release:branch -DbranchName=${project.artifactId}_${project.version} -Dusername=${username} -Dpassword=${passwd} -Dproject.rel.${groupId}:${projectId}=1.0.0-SNAPSHOT
-Dproject.dev.${groupId}:${projectId}=1.1.0-SNAPSHOT)
copy project from trunk to new created branch, transform pom version on trunk from 1.0.0-SNAPSHOT to 1.1.0-SNAPSHOT
non-dependent projects
(mvn -B release:clean release:branch -DbranchName=${project.artifactId}_${project.version} -Dusername=${username} -Dpassword=${passwd} -Dproject.rel.${groupId}:${projectId}=1.0.0-SNAPSHOT
-Dproject.dev.${groupId}:${projectId}=1.1.0-SNAPSHOT versions:update-parent
-DallowSnapshots=true)
copy project from trunk to new created branch,
transform the trunk pom version 1.0.0-SNAPSHOT become 1.0.1-SNAPSHOT
update parent-pom.version: 1.0.0-SNAPSHOT become 1.1.0-SNAPSHOT
dependent projects:
(mvn -B release:clean release:branch -DbranchName=${project.artifactId}_${project.version} -Dusername=${username} -Dpassword=${passwd} -Dproject.rel.${groupId}:${projectId}=1.0.0-SNAPSHOT
-Dproject.dev.${groupId}:${projectId}=1.1.0-SNAPSHOT versions:update-parent
-DallowSnapshots=true versions:use-latest-versions -DallowSnapshots=true
-DupdateDependencies=true)
copy project from trunk to new created branch
transform pom version on trunk from 1.0.0-SNAPSHOT to 1.1.0-SNAPSHOT
update parent-pom on trunk from 1.0.0-SNAPSHOT to 1.1.0-SNAPSHOT
update the already branched dependency projects from 1.0.0-SNAPSHOT to 1.1.0-SNAPSHOT
The first problem here is that there is no way yet to have an argument to increase the MinorVersion when branch the project, the maven-release-plugin 2.2.2 does not increment the pom MinorVersion on trunk when branching, so that's why we need to use -Dproject.rel.${groupId}:${projectId}=1.0.0-SNAPSHOT
-Dproject.dev.${groupId}:${projectId}=1.1.0-SNAPSHOT arguments and change them manually for every project, so 200 times every time when we preapare for a new release.
I'm wondering if it's not a way to make all described procedure somehow in an automated fashion and to not need to perform all these changes manually all the time.
We've taking account even to modularize this product so that to colapse those 200 project in 100 probably, but this is not acceptable since the idea is to have a fine grained projects versioning and have all the projects with it's own lifecycle, so an aggregator (I mean a classic one) is out of discussion here.
We are using SVN as VCS, Maven as build tool (probably you already figured out about that :) ), and Bamboo as a CI server (actually, instead of "Maven Dependency Processor" feature, Bamboo is not helping me to much regarding the versioning problem).
Do you guys have any idea in order to find a proper solution for this problem, maybe another plugin which would help (versions-maven-plugin does not change versions automatically when branch), maybe another point of view for this, I don't know..., any help or sugestion are welcome.
Thanks!

I try to avoid keeping internal projects versions in the <properties> section of the parent-pom, since every time when I release a project, the <version>${project.version}</version> variable will be switched with the explicit version of the project like:
<properties>
<project_A.version>1.0.0-SNAPSHOT</projectA.version>
</properties>
stored in parent pom, translated in the projects pom like: ${project.version}, will become 1.0.1-SNAPSHOT when release that project, and will overwrite the <project_A.version> version from parent-pom <properties>. So this trick of keeping the projects versions as properties in a centralized location like the parent-pom, is valid as long as you don't release the projects.
This is the question strictly related to the branching question.
Now, if you want I can tell you more about releasing the projects using this <properties> substitution:
Lets's say you want to release your project_A, what will you do with the <projectA.version> stored in the parent pom <properties> which is a -SNAPSHOT version when you start to release the parent pom, right? I mean, in order to release a project, you'll release all its dependencies and its related parent-pom first right? otherwise your project release will fail since is pointing to a -SNAPSHOT version of the parent pom.
Now, you are releasing the parent-pom keeping the -SNAPSHOT version of project_A inside of it's <properties>, next when you'll release the project_A, your project will refer to the new created release version of your parent pom, which parent pom released version is refering to the -SNAPSHOT version of your project_A, still not a problem since your parent pom is keeping the true version of your project_A, until your projectA released version (lets say 1.0.0) will refer to the same parent pom released version which is containing 1.0.0-SNAPSHOT version of your project_A, and now you have a problem already because your parent pom <properties> is keeping a wrong info.
Ofcourse you can hack the parent pom when release it and store the release version of project_A, but this against the rules and I would not agree with that at all, plus that are other scenarios when this "hack" will cause more problems than help.
I'm sorry if this sounds pretty complicated and detailed but I just want to explain as much as possible the real life situation not just theoretical, plus that I also need to keep in mind that I have 200+ projects I need to keep somehow aligned.

Related

Need to control Maven Version Numbers for multiple projects

It's a long story. My current place uses Ant for builds. They create about 20 different foundation class jars that are used in multiple projects. Originally, the projects would check in the particular versions of the various jars they needed and then never updated them. That meant that each application had incompatible jars with other projects and our servers. Hilarity ensued.
To handle this, I immediately brought in a Maven repository, and integrated Ivy into our Ant builds. No more checking in jars. Instead, you fetch the correct version from the Maven repository. Originally, I expected the developers to keep the version numbers in the ivy.xml up to date, but they never did. Instead, the Ivy integration and setup depends upon an external Subversion project. This allowed me to integrate Ivy with minimal changes to the old build.xml files. I added a ivy.version.properties file to that external ivy project and maintain the version numbers of various jars in there. There's a corporate wide version number.
The various projects use the ${corporate.version} property for our foundation jars version numbers. When I update that ivy.version.properties file, all projects get updated with the right version number for our foundation classes. We use <ivy:makepom> to generate a pom.xml for our projects and use this to deploy our jars and wars into our Maven repository.
The result: I no longer have to worry about the developers keeping the version numbers of their projects in sync. I, as the release engineer handle that by updating that one ivy.version.properties file. All projects are in sync.
Now, we're moving to Maven, and I want to be able to do the same thing. I doubt developers will remember to update their pom.xml with the correct version numbers, so I want to read that in from another file, and use that.
There are two issues: One is that Maven first reads in the version number of a project before it executes any goal. No version number in the pom.xml, no version number for the generated jar.
Even if I manage to get by that first step, we have the fact that the pom.xml has no version number in it for foundation classes. When Maven pulls down the pom.xml to get the dependencies, it can't figure out which revision.
Yes, I could put that into a corporate pom.xml and have that as a parent project for all of the other projects. We already have a parent project to help set up various aspect of all projects. I could put a foundation class version number in there. However, that means that the developers now have to update the parent project's version number with each release. If developers can't be trusted to update the version number of their project with each release, what makes you think they'll do that with the parent's version for each release?
I know other people must have run into a similar issue. How do you handle this?
I could have an Ant script that generates the pom.xml from a template pom.xml, but that seems a bit silly.
I was wondering if is it possible for Maven to generate a pom.xml on the fly and then to use that to continue the executing the right goal. For example, I type in mvn package, and Maven will take a template.pom.xml file, fill in the missing version numbers to generate a generated.pom.xml file, then execute mvn package on that generated pom.
Or, is there a better way to solve this issue? Basically, I need to control the version number of our releases across all projects. This way, all projects are using the same version of our foundation classes. Also, I control some other versions of other jars (like log4j) this way. I've done this with Ant and Ivy, and now I want to move to Maven.
I think the best option is to create a pom.xml with all the dependencies to your packages in its and then import it in your developer project paren pom.xml using import
So, in the project parent POM:
<properties>
<corporate.version>...</corporate.version>
<properties>
...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.mycompany.libs</groupId>
<artifactId>foundation<artifactId>
<version>${corporate.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
...
</dependencies>
</dependencyManagement>
So for new set of foundation libraries, you deploy a new pom.xml with all the versions in it and update the ${corporate.version} in the relevant parent pom.xml file. You can even have when the versions are not yet fixed define a -SNAPSHOT artifact with these version numbers.
for more information, see: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Importing_Dependencies
We actually have the same kind of problem and I know that there is trade-off between "lazy programmers" who never update their poms and the stability aspect: Changing the versions of the foundation jars "suddenly" which may cause unexpected build fails.
If you really want the behaviour you describe, I would put the desired dependencies into a SNAPSHOT pom and use it either as a parent pom or import it as a BOM. In this way, you can change the content and everybody pulls in the new content with the next update.
I've decided that this a stupid idea and should never have been done. It's the developers' responsibility to update their own damn POM and make sure they're pulling the correct version. This is the way it really should be and developers should simply learn to do it (what's the technical term? ...oh yeah) the Correct Way.
I originally implemented this because the company had something like this (which wasn't working) and I had problems getting the developers to do what should be their job. And, the developers liked it because I was now officially responsible when something goes wrong rather than them. It's what you get when you do someone else's job.
I've convinced the company to stop trying to keep all our core jars (the jars used in our other projects) and our releases at the same version number. Instead, every project will have its own versioning. Jars will version only when there's an actual change and not when we force an upgrade to the version. And, the developers must know what versions of the various jars they depend upon.
Use the maven-release-plugin. This plugin can stamp a version number on all pom.xml files. The plugin also understands version control and knows how to tag a maven project for release (or branching).

Releasing the project with Maven: different release versions of artifacts

I have a maven project with multiple modules. When I release it I just change the versions of the modules from SNAPSHOT to release's version and its ok. This can be done with Maven Release Plugin.
The problem arises because some of the dependencies I have are actually the artifacts, developed by other groups of our programmers. Thus their versions may often change, which is a behaviour opposite to other dependencies, for example hibernate's artifact versions. At the moment of release I would like to use some available versions of that rapidly changing libraries. Probably the last one. May be they will release a new version of their library specially for my release.
Note that their library is a separate Maven project with separate version controlled by them.
All I can do now is just to check manually which version of the that dependency is the last and put it down manually into my POM. Its not that convenient. May be there is a better way to organize it with Maven and TeamCity? Can I update the versions of that other group's artifacts too? Their version should be derived from their Snapshot version, or from the last release they have deployed into the Nexus.
You can use versions-maven-plugin to automate updating external dependencies.
As mentioned, you can use Versions Maven Plugin, and more specifically you need versions:update-properties. As you can read the manual, it
Sets properties to the latest versions of specific artifacts.
The condition is that you work with repository manager (e.g. Artifactory). Maven knows to search this repo for the most updated aritfacts.
Before you run the maven-release-plugin, you run the versions-plugin that updates your dependencies. For example, you run versions:update-properties with the relevant parameters.
If you would like to print latest versions of artifacts, the same versions-plugin is your friend. Have a look and read the link I've sent you above; the relevant command is versions:display-dependency-updates.
If you would like to print selectively, only your artifacts latest versions, you can set their version using a property. for example, if you have dependency JAR X, write in the main pom something like this:
<dependency>
<groupId>myGroup</groupId>
<artifactId>X</artifactId>
<version>${x.version}</version>
</dependency>
<properties>
<x.version>3.1.0.RELEASE</x.version>
</properties>
Then you use versions:display-property-updates -DincludeProperties="x.version"

Distinguishing maven releases/snapshots from trunk than the ones from branches when getting latest dependencies

I am working on project to automate maven releases for large set of inter-dependent modules. First the task involves getting the latest versions for internal dependencies. Than order the builds in a way every project is built before it’s needed. Hopefully that combination will get me to my goal of one click bulk release.
However my biggest problem is:
We branch projects from time to time and we do releases/snapshots to Nexus repo manager from that branch. Obviously because the groupId and ArtifactId are the same as the trunk- Snapshots/Releases from trunk and one from branches end up in the same place in nexus.
In a scenario where we have projectA with a trunk version 1.1.x
And projectB → projectA version 1.1.x
So if I use the version-plugin to get the latest dependencies for B, I will get the latest. (perfect)
However if I later branch ProjectA with a version 1.2.0 and add changes that I don’t want projectB to pick up or could potentially break it.
Now next time I run projectB and I use the version plugin to get the latest , The plugin will get the branch version (1.2.0) for ProjectA, as it’s numerically the latest. And there you go, projectB build fails.
I have so far tried adding a classifier to the artifact to distinguish between the two but I later realised the plugin does not act upon the classifier.
This give me so much frustration and will appreciate any hints or advice
Simple use proper/diffrent artifact names for branches. You can automatically create proper artifact names Maven Release plugin
mvn --batch-mode release:branch -DbranchName=my-branch-1.2 -Dproject.rel.org.myCompany:projectA=1.2 -Dproject.dev.org.myCompany:projectA=2.0-SNAPSHOT
The typical approach to solve this problem is to use branch names as version classifiers and that approach works for all scenarios I have seen so far. What problem did you have with this?

How do I automate the update of SNAPSHOT dependencies to the latest released versions when using maven

I am trying to define a release process for our project.
I was able to use the maven release plugin to accomplish the release but I had to manually update all of our internal SNAPSHOT dependencies to release versions prior to the prepare and then again back to SNAPSHOT versions after the release.
I have used the maven versions plugin and it detects my corporate dependencies that are SNAPSHOT builds and lists the correct release build to use.
I tried the maven release plugin, prepare goal, and complains about the SNAPSHOT versions in my dependencies.
Is there a way to do a release that updates the SNAPSHOT depenedencies to latest released versions and then back to SNAPSHOT versions after the release? Or maybe this is just not the way you are supposed to be using maven with releases and SNAPSHOTS.
When you create a realease with maven: it must be definitive. i.e. once it is created it cannot change anymore (i.e. the sources cannot change and the dependencies cannot change).
So when you create a release of moduleA and moduleA having a dependency on moduleB. moduleB must be released before moduleA is released, and moduleA MUST depends on a released version of moduleB.
One important thing is that you shouldn't go back to SNAPSHOT version. At least as I understand it means :
create moduleB-1.0.0 release
changing dependency to moduleB-1.0.0
create moduleA-1.0.0 release
then change the dependency back to moduleB-1.0.0-SNAPSHOT
It must be clear that once moduleB-1.0.0 exists (i.e. is released): the artifact moduleB-1.0.0-SNAPSHOT shouldn't be used anymore.
Instead of go back to SNAPSHOT should update dependency to next SNAPSHOT version (for instance moduleB-SNAPSHOT-1.0.1)
That being said, releasing a module depending on many SNAPSHOT artifacts is not an easy process, since every dependency must be released prior to releasing your main artifact.
What we have, most of the time, is a main artifact depending on many other artifacts (let's call them the corporate-modules) having the same versionning strategy. And so you can define a property corporate-module-version holding the version used by many dependencies in one place.
The release process is the following:
release every snapshot dependency with version number 1.0.0 (using the maven-release-plugin: after this step all pom.xml of your corporate-module in SCM are 1.0.1-SNAPSHOT) (see important remark at the end of this post to facilitate this step)
manually changing the corporate-module-version property to "1.0.0" in the main artifact (so that all SNAPSHOT dependencies are replaced by just released versions)
commit the modified pom.xml holding the corporate-module-version
with maven-release-plugin: releasing the main artifact (and after that, the new version in the SCM will be something like 1.0.1-SNAPSHOT)
manually changing the corporate-module-version property to "1.0.1-SNAPSHOT" in the main artifact (so that all dependencies are replaced by latest snapshot versions)
Important remark: having a multi-module parent project holding all your corporate modules is a must to perform at once the release of of all corporate modules. To not having too much trouble with the maven-release-plugin and the multi-module parent project be sure to put your parent pom.xml one directory upper all your child pom.xml (it's the designed recommend by maven, but unfortunately, sometimes, eclipse users don't follow it because eclipse don't like hierarchical projects)

Maven and snapshots?

I'm working on a rapid project, of which, I'm directly working on a module that is continuously changing. Others have a direct dependency on the module, and as such, I stubbed out the public interface and deployed it to our local Nexus repository as 0.0-SNAPSHOT for use.
Now that I've almost completed my first iteration of the module, I've attempted to redeploy the updated artifact. Reading about snapshots, others claim that a snapshot should represent the current head trunk. Is this true?
Maven automatically increments my snapshot version upon redeploy - so, going from 0.0-SNAPSHOT, I'm still at 0.0-SNAPSHOT, only, it's iteration 3 or 4 of the same snapshot. When should I roll over to 0.1-SNAPSHOT? Is there a plugin I can use to automate the version change, instead of manually editing my pom?
After integration testing and the deployment of our first system release, 1.0-RELEASE, how should my module progress? Should I move my module to 1.0-SNAPSHOT and continue thereon? Is there a methodology one should follow, or is it left to the discretion of the developer?
The X.Y.Z-SNAPSHOT notation identifies temporary versions leading up to release X.Y.Z, so you usually do not move from X.Y.Z-SNAPSHOT to X.Y.Z+1-SNAPSHOT unless you release X.Y.Z. If you adhere to this convention the maven-release-plugin may help you with the full release process.
Note that the most common Maven convention uses 3-number release identifiers without any suffix (i.e. no -RELEASE). Suffixes are usually used to distinguish variants of the same release.
Maven is all about conventions, so there's little chance you'll go very far without reading about it: This book is a good starting point.
For updating your POM version without manually updating, you may take a look at Maven Release plugin (Although I wrote myself a little script to do the POM update as I find Release plugin don't fit that good in my work flow)
It then come to your version number issue. It is more a release procedure issue. Normally a planned release is denoted by increment of Major or Minor version in the version number. SNAPSHOT version denote that certain release is in progress. For example, I will prefer doing something like this for your case:
Assume I am planning to release first iteration as 0.1, then I will make my head trunk in SCM (e.g. trunk in SVN) with 0.1-SNAPSHOT as version. Which denotes that all development is in fact contributing to release of version 0.1. Upon finish, I'll update the POM version from 0.1-SNAPSHOT to 0.1, perform an actual release of version 0.1 (including release branching, tagging, deploying the artifact), and then change the POM version to SNAPSHOT of next planned release (for example, 0.2-SNAPSHOT).
Similarly, after releasing 1.0 (or 1.0-RELEASE in your example), POM version in head trunk should then be updated to snapshot of your next release version, for example, 1.1-SNAPSHOT.
Just bear in mind that there should no longer be SNAPSHOTs of certain version, if that version is actually released.
Understand the way maven interprets SNAPSHOTS is to clear any doubt you can have.
Extract from http://books.sonatype.com/mvnref-book/reference/pom-relationships-sect-pom-syntax.html
For example, if your project has a version of “1.0-SNAPSHOT” and you deploy this project’s artifacts to a Maven repository, Maven would expand this version to “1.0-20080207-230803-1” if you were to deploy a release at 11:08 PM on February 7th, 2008 UTC
First, Let me suggest to stick on maven conventions and change your version to 0.1 as maven archetype:generate propose.
So SNAPSHOT helps others to stay up-to-date easily with your active project. On every compilation, their projects will check new releases of SNAPSHOTS dependencies (based on that pseudo-datetime-version they have on its .m2 directory).
When you finish work on 0.1-SNAPSHOT you deploy an 0.1 and start a 0.2-SNAPSHOT or 1.0-SNAPSHOT

Resources