Consider a project that is used in other projects, and this project has version 1.0 and is present in the Nexus.
After that, some changes are done in this project.
Now there are two solutions:
Increment the version of the project to 2.0, and delete the version 1.0 from Nexus. When the developers try to get the dependencies from the Nexus with version 1.0 they will get an error that this version does not exist and need to change the version to 2.0.
Change the functionalities of this project and inform the crew that some changes are done, but this is not the practice at all.
Is there any functionality in Maven and Nexus to simplify this task and make this all happen in the backend so the developers can't do anything, or is this not possible?
If you have an old version of an artifact that must not be used anymore because it has some dangerous bug, or it does not work with the new database structure or something like this, it may be advisable to move it to some non-public Nexus repository (and also delete it from the local repository of the build server), so that nobody can use it for release builds (people can use it for local builds, but this is usually not dangerous).
If you want to manage standard versions throughout your company, it is a good idea to have a parent pom or some boms which collect versions in a <dependenyManagment> section and can be included by the developers. This way, you only need to inform them to change one version number (namely the one of the parent pom or bom) instead of many.
Still, you are left with the problem that people do not read company newsletters. I know the problem that many developers of jars compile and test their source code against very old versions of their dependencies while the war/ear (that includes the jar) uses new versions.
Related
I've got multiple projects using a certain Dependency of Version XX, if I release a new version, I have to touch every project to change it to version XX.Y.
I've came across an approach to edit my m2 settings <version>${my.version}</version>, to add a parameter and bind it into my POM.xml, but this implicit means, everyuser has to manage their m2 settings when I do a new release.
Is there a way to central (user independant) manage the versions as in SVN, so none has to change anything and it always uses the up2date version, if I release a new version?
In general, the maintainer of every project should decide for themselves if they update the version or not. Updating the version might break things, so they may choose to stay on the older version. Particularly, it is important that the maintainers notice that something has changed, so that tests are run.
For development, though, there are Snapshot versions. A Snapshot dependency always references the newest version, but the -SNAPSHOT indicates this to the maintainer of the project. Snapshot versions should not go to production - the builds are not reproducible.
If artifacts are so tightly coupled that they are build together, think about using multi module projects.
You can use a pattern called "Bill of Material".
I think your question is somehow similar to this question (but not exactly a duplicate), and my answer applies here aswell:
You create a new maven project (the bill of material) that only consists of a pom with dependency management block. Here you declare all your dependencies and their versions. The packaging should be set to pom.
This bill of material (bom) project is now used as parent of all other projects. When using a dependency, only group id and artifact id is specified, the version tag is ommited. In that way, the version will be taken from the bom and you have one central place to manage the versions of the dependencies.
More details with examples are here (in the lower part of the page) or here.
In development, I can reference the latest version of an artifact as 1.2.3-SNAPSHOT. Now I need the same behaviour for release candidates, i.e. I would like to be able to depend on the latest release candidate (there should also be a procedure for the developer to declare development versions as release candidates).
I am not sure how to implement this behaviour properly. Should I use an additional repository for release candidates and move development versions to this repository if the developer requests it? Or can I somehow define a "second snapshot list", like 1.2.3-RC?
You can get there be re-configure a few things:
use a version range for the dependency
change the updatePolicy for the repository you store the release candidates. see https://maven.apache.org/settings.html (updatePolicy). If you store the artifacts in a maven proxy usually you need to allow to overwrite releases.
Remember in a multi module build that they might upload modules before detecting a failed build (due to mvn deploy being a phase not a goal). You need to verify the complete build is ok before starting to upload artifacts in the maven repository. Or stage them somehow.
Remember this will most certainly prevent your builds being reproducible. Since an RC dependency might change between builds. You would need to change the version range - which is not always an issue. Ranges may work for you.
I've better experience to let developers stay on snapshots but have the CI server set an explicit version (for example using the versions plugin) prior to deployment / releasing for the dependency the artifact uses.
I am in the midst of an initiative to convert our build.gradle files to use semantic versions. In addition to using Gradle, we also use Git and are following the Gitflow Workflow. Jenkins is used to build the projects.
Versions for released artifacts follow a MAJOR.MINOR.PATCH format. When declaring dependencies in the build.gradle file, we use dynamic versions, such as 10.0.+ (i.e. take the latest 10.0.PATCH version).
We promote our artifacts from a Release Candidates repository to a Releases repository in Nexus. The repository has its policy set to "Releases". Because of the complexity of the product (200+ projects, with many upstream and downstream dependencies), a lot of the promotion plugins available for Jenkins appear to fall short. We were thinking of having Jenkins build the master branch as a way to rename artifacts (10.0.0-rc.1-abcdefg becomes 10.0.0) and upload them to the correct Nexus repository.
I am unsure of how to handle a situation where an upstream dependency has a patch version incremented. The downstream project - a WAR - is re-built by Jenkins and bundles the new JAR, but the version of the downstream project doesn't change. When an attempt is made to upload to Nexus, it fails because only one artifact can have the same version.
Here is an example:
The Releases Nexus repository has upstream-api versioned at 10.0.0, and downstream-project versioned at 10.0.0
downstream-project depends on 10.0.+ of upstream-api
upstream-api.jar is bundled into the downstream-project.war file
The two artifacts are deployed as part of Release X of the product
When a hotfix branch has been merged into master, the upstream-api version has changed to 10.0.1
The fix means that when deployed, the product is now Release X'
downstream-project stays at 10.0.0, but is re-built because of the change in the upstream dependency
Jenkins fails to upload downstream-project-10.0.0.war to Nexus because it already exists
I could have the old artifact replaced with the new artifact, but then that means that Release X can no longer be deployed from artifacts in Nexus (eg. in the case of a rollback, or needing to replicate an issue on an older release).
How is this typically handled?
How is this typically handled?
I don’t have a universal answer here. I would assume that these are the most “common” possibilities:
Don’t distribute your dependencies with the release and continue to use dependency version declarations such as 10.0.+. The assumption is then that the software will indeed work with any 10.0.x version – at least as far as your users will tolerate it. That usually happens for free software which is distributed in source or in a package system of a Linux distribution. The dependency version declaration is only updated when there is a required improvement in the dependency, i.e., when the change is so important that your users won’t tolerate any earlier version.
Distribute your dependencies with the release and either:
Use a build number in addition to the main/semantic version number of the original code – for example 1.3.4-b3. If I’m not mistaken, then this is often being done for proprietary Windows software.
Increment the main/semantic version number when a dependency changes and make the dependency requirement explicit.
Some more General Thoughts on the Issue
I think the core issue is the dynamic dependency declaration – the 10.0.+ version declaration. What you state with this declaration is that your release will work equally well with any 10.0.x version.
If that is really the case, i.e., the bugs that are fixed by a patch in the dependency are guaranteed to never affect the release, then your release should probably simply not be rebuilt, since its functionality wouldn’t change anyway. The version of the dependency wouldn’t matter, your release could stay with the older dependency version.
More likely, though, the upstream bugfixes will also make a difference in your downstream project, i.e., they will affect the functionality of the release. In that case you should make the “new” dependency explicit in your build.gradle. Since that’s a change to your release artifact, a new release version is due.
It's a long story. My current place uses Ant for builds. They create about 20 different foundation class jars that are used in multiple projects. Originally, the projects would check in the particular versions of the various jars they needed and then never updated them. That meant that each application had incompatible jars with other projects and our servers. Hilarity ensued.
To handle this, I immediately brought in a Maven repository, and integrated Ivy into our Ant builds. No more checking in jars. Instead, you fetch the correct version from the Maven repository. Originally, I expected the developers to keep the version numbers in the ivy.xml up to date, but they never did. Instead, the Ivy integration and setup depends upon an external Subversion project. This allowed me to integrate Ivy with minimal changes to the old build.xml files. I added a ivy.version.properties file to that external ivy project and maintain the version numbers of various jars in there. There's a corporate wide version number.
The various projects use the ${corporate.version} property for our foundation jars version numbers. When I update that ivy.version.properties file, all projects get updated with the right version number for our foundation classes. We use <ivy:makepom> to generate a pom.xml for our projects and use this to deploy our jars and wars into our Maven repository.
The result: I no longer have to worry about the developers keeping the version numbers of their projects in sync. I, as the release engineer handle that by updating that one ivy.version.properties file. All projects are in sync.
Now, we're moving to Maven, and I want to be able to do the same thing. I doubt developers will remember to update their pom.xml with the correct version numbers, so I want to read that in from another file, and use that.
There are two issues: One is that Maven first reads in the version number of a project before it executes any goal. No version number in the pom.xml, no version number for the generated jar.
Even if I manage to get by that first step, we have the fact that the pom.xml has no version number in it for foundation classes. When Maven pulls down the pom.xml to get the dependencies, it can't figure out which revision.
Yes, I could put that into a corporate pom.xml and have that as a parent project for all of the other projects. We already have a parent project to help set up various aspect of all projects. I could put a foundation class version number in there. However, that means that the developers now have to update the parent project's version number with each release. If developers can't be trusted to update the version number of their project with each release, what makes you think they'll do that with the parent's version for each release?
I know other people must have run into a similar issue. How do you handle this?
I could have an Ant script that generates the pom.xml from a template pom.xml, but that seems a bit silly.
I was wondering if is it possible for Maven to generate a pom.xml on the fly and then to use that to continue the executing the right goal. For example, I type in mvn package, and Maven will take a template.pom.xml file, fill in the missing version numbers to generate a generated.pom.xml file, then execute mvn package on that generated pom.
Or, is there a better way to solve this issue? Basically, I need to control the version number of our releases across all projects. This way, all projects are using the same version of our foundation classes. Also, I control some other versions of other jars (like log4j) this way. I've done this with Ant and Ivy, and now I want to move to Maven.
I think the best option is to create a pom.xml with all the dependencies to your packages in its and then import it in your developer project paren pom.xml using import
So, in the project parent POM:
<properties>
<corporate.version>...</corporate.version>
<properties>
...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.mycompany.libs</groupId>
<artifactId>foundation<artifactId>
<version>${corporate.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
...
</dependencies>
</dependencyManagement>
So for new set of foundation libraries, you deploy a new pom.xml with all the versions in it and update the ${corporate.version} in the relevant parent pom.xml file. You can even have when the versions are not yet fixed define a -SNAPSHOT artifact with these version numbers.
for more information, see: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Importing_Dependencies
We actually have the same kind of problem and I know that there is trade-off between "lazy programmers" who never update their poms and the stability aspect: Changing the versions of the foundation jars "suddenly" which may cause unexpected build fails.
If you really want the behaviour you describe, I would put the desired dependencies into a SNAPSHOT pom and use it either as a parent pom or import it as a BOM. In this way, you can change the content and everybody pulls in the new content with the next update.
I've decided that this a stupid idea and should never have been done. It's the developers' responsibility to update their own damn POM and make sure they're pulling the correct version. This is the way it really should be and developers should simply learn to do it (what's the technical term? ...oh yeah) the Correct Way.
I originally implemented this because the company had something like this (which wasn't working) and I had problems getting the developers to do what should be their job. And, the developers liked it because I was now officially responsible when something goes wrong rather than them. It's what you get when you do someone else's job.
I've convinced the company to stop trying to keep all our core jars (the jars used in our other projects) and our releases at the same version number. Instead, every project will have its own versioning. Jars will version only when there's an actual change and not when we force an upgrade to the version. And, the developers must know what versions of the various jars they depend upon.
Use the maven-release-plugin. This plugin can stamp a version number on all pom.xml files. The plugin also understands version control and knows how to tag a maven project for release (or branching).
What are the best practices for software versioning and multimodules projects with Maven?
I mean, when I create a multimodules project with Maven, what is the best approach for the versioning? To use a single version for all the modules (defined in the top project)? To use a version for each module (defined in the POM of each module)? Is there another approach that I'm missing? What are the pros and cons of each approach?
In general, are the different modules released together (possibly sharing the same version number)?
Thanks
Honestly it depends on what you would like to do. Multimodule projects are created for multiple reasons, one of them being you only need to deploy what has changed instead of all modules.
Think about it this way: if you had a non-multi-module project and you only had to change one line in the services layer, you have to rebuild the entire project and deploy all of the code again...even though only your services layer will change.
With multi-module projects, you can regenerate your project and deploy only what changed...your services. This reduces risk and you're assured that only your services module changed.
You also have a multitude of benefits to using multi-module projects that I'm not listing here but there is certainly a huge benefit to NOT keeping your version numbers of your modules in sync.
When you build your project, consider deploying it to a repository that will hold all compatible jars together for builds (each build creates a new folder with the parent-most pom version number). That way, you don't need to keep documentation about which jars are compatible...they're all just deployed together with a build number.
I was looking for a solution for this exact problem myself and versions-maven-plugin was exactly what I needed. I don't like the release plugin communicating with the SCM system. The versions plugin does just what we need: it sets a new version number in all poms of the project:
mvn versions:set -DnewVersion=2.0.0
Then I can proceed with commits, tags and an official build server build...
EDIT:
The versions plugin depends on how a maven multi-module project has been organised: as a result, it often does not update all POM files in a complex multi-module project.
I've found that sed and find do the job much more reliably:
sed -i 's/1.0.0-SNAPSHOT/1.0.0-RC1/g' `find . -name 'pom.xml'`
Typically you create a multi-module project because you have deemed that the various modules are parts of a single whole. Maybe the client-piece, the controller-piece and the services-piece. Or maybe the UI with services.
In any case, it makes sense to have the version numbers for the various modules to move in lock-step. However Maven does not enforce that as a rule.
As to your question
are the different modules released together (possibly sharing the same
version number)
I would think so. That is one of the reasons for having it a multi-module project. Otherwise you could have the modules as independent projects.
Of course this is the kind of stuff that is rife with edge cases and exceptions ;-)
I had the same problem with a project I`m working on. I also decided to use separate versions and even the dependency to the parent pom only has to be updated if some of the managed dependencies change. (so mostly as #vinnybad describes it)
Two additions
exists-maven-plugin
With the usage of the "org.honton.chas.exists-maven-plugin" only the modules will be deployed to the repository that have actually changed, which is really great, because also the according docker-images will only be published if something has changed on one of the service. This avoids "polluting" the image repository with different but unchanged versions.
versioning
One main downside of the "separated versions" approach are the questions regarding versioning:
What's the current version of my project?
Which module versions work with each other? (even thought they don't directly depend on each other, one does rely on what another does, e.g. they share the database schema)
To solve that I put all module versions into the dependency management part of the parent pom, even if no other module depends on them. A "integration-test" module could solve that by depending on all of the modules - and of course testing them together.
This way I would be "forced" to update the parent pom with every change, since it's referring the released module versions. This way the parent pom would have the "leading" version and at the dependency-management block state the versions of all modules that are compatible with each other (which will be ensured by the integration test).