I am in the midst of an initiative to convert our build.gradle files to use semantic versions. In addition to using Gradle, we also use Git and are following the Gitflow Workflow. Jenkins is used to build the projects.
Versions for released artifacts follow a MAJOR.MINOR.PATCH format. When declaring dependencies in the build.gradle file, we use dynamic versions, such as 10.0.+ (i.e. take the latest 10.0.PATCH version).
We promote our artifacts from a Release Candidates repository to a Releases repository in Nexus. The repository has its policy set to "Releases". Because of the complexity of the product (200+ projects, with many upstream and downstream dependencies), a lot of the promotion plugins available for Jenkins appear to fall short. We were thinking of having Jenkins build the master branch as a way to rename artifacts (10.0.0-rc.1-abcdefg becomes 10.0.0) and upload them to the correct Nexus repository.
I am unsure of how to handle a situation where an upstream dependency has a patch version incremented. The downstream project - a WAR - is re-built by Jenkins and bundles the new JAR, but the version of the downstream project doesn't change. When an attempt is made to upload to Nexus, it fails because only one artifact can have the same version.
Here is an example:
The Releases Nexus repository has upstream-api versioned at 10.0.0, and downstream-project versioned at 10.0.0
downstream-project depends on 10.0.+ of upstream-api
upstream-api.jar is bundled into the downstream-project.war file
The two artifacts are deployed as part of Release X of the product
When a hotfix branch has been merged into master, the upstream-api version has changed to 10.0.1
The fix means that when deployed, the product is now Release X'
downstream-project stays at 10.0.0, but is re-built because of the change in the upstream dependency
Jenkins fails to upload downstream-project-10.0.0.war to Nexus because it already exists
I could have the old artifact replaced with the new artifact, but then that means that Release X can no longer be deployed from artifacts in Nexus (eg. in the case of a rollback, or needing to replicate an issue on an older release).
How is this typically handled?
How is this typically handled?
I don’t have a universal answer here. I would assume that these are the most “common” possibilities:
Don’t distribute your dependencies with the release and continue to use dependency version declarations such as 10.0.+. The assumption is then that the software will indeed work with any 10.0.x version – at least as far as your users will tolerate it. That usually happens for free software which is distributed in source or in a package system of a Linux distribution. The dependency version declaration is only updated when there is a required improvement in the dependency, i.e., when the change is so important that your users won’t tolerate any earlier version.
Distribute your dependencies with the release and either:
Use a build number in addition to the main/semantic version number of the original code – for example 1.3.4-b3. If I’m not mistaken, then this is often being done for proprietary Windows software.
Increment the main/semantic version number when a dependency changes and make the dependency requirement explicit.
Some more General Thoughts on the Issue
I think the core issue is the dynamic dependency declaration – the 10.0.+ version declaration. What you state with this declaration is that your release will work equally well with any 10.0.x version.
If that is really the case, i.e., the bugs that are fixed by a patch in the dependency are guaranteed to never affect the release, then your release should probably simply not be rebuilt, since its functionality wouldn’t change anyway. The version of the dependency wouldn’t matter, your release could stay with the older dependency version.
More likely, though, the upstream bugfixes will also make a difference in your downstream project, i.e., they will affect the functionality of the release. In that case you should make the “new” dependency explicit in your build.gradle. Since that’s a change to your release artifact, a new release version is due.
Related
I maintain a few Java library projects on GitLab, which I currently build with a GitLab CI workflow and deploy to a GitLab Maven repository. Now I would like to make them available via JitPack while keeping the GitLab Maven repo for a while.
GitLab’s Maven repo, like most of the others out there, uses the contents of the <version> tag in pom.xml for versioning. JitPack, on the other hand, needs a Git ref to work with (a branch name, a tag or a plain hash). I am looking for a way to use the same versioning regardless of the repo, so that version FOO will fetch a jar based on the same code, regardless of whether it is taken from GitLab or JitPack.
For a released, stable version that could be solved by tagging each release with its version number. That is, when I release version 3.7.0, I would ensure the <version> tag in my pom.xml reads 3.7.0, and tag the commit with 3.7.0 as well. I would need to enforce a match between the two (e.g. by teaching CI to bar{k|f} upon detecting a discrepancy), but with some homework, both repos would carry the same version of my code under the same version number.
Things get tricky when it comes to unstable versions. If I am working on the upcoming 3.7.2 release and would already like to make it available for testing, I understand common practice is to set the version tag to 3.7.2-SNAPSHOT, telling everyone that this is not a stable version. The GitLab Maven repo would serve the jar under that version.
For JitPack, versions ending in -SNAPSHOT indicate that the code could have changed since the last build and the jar should be rebuilt from source. A version named 3.7.2-SNAPSHOT would cause JitPack to look for a ref named 3.7.2 (tag or branch) and build it from scratch.
So I would have to ensure that the version name of the upcoming version resolves to the latest commit for the upcoming version. I could do that by developing the upcoming version in a branch named like the version, but that would presumably result in a naming conflict as I release it, as I would then introduce a tag with the same identifier.
Is there a general recommendation for addressing this, i.e. serving unstable versions of a jar via JitPack and a conventional Maven repo under the same version number?
We are using release plugin to convert snapshots to releases, but snapshots were already added as dependencies in other projects. Do we need to update every project pom, to use the new releases.
Eg: ArtifactA is included in ArtifactB,
ArtifactA is included in ArtifactC
ArtifactB and ArtifactC is included in Artifact D
Is there an alternate way to push dependent releases in place of snapshots
[We use Jenkins to build and push to Nexus]
Our build server runs versions:use-releases followed by a commit to the SVN before the build. This replaces all SNAPSHOT versions by their respective release versions if they exist.
A less cumbersome way of dealing with releases - keep using snapshots. Technically there's nothing wrong with snapshots and there's no need to change them to release versions. Just don't use versions with SNAPSHOT suffix - use resolved snapshots (with timestamp and build number).
As for the dependencies - usually there's no big need to change the version immediately after the release. It's only when clients need the updates - then they should change to newer versions manually.
If we're talking about remote API and its client lib as a dependency - such remote API needs to keep backward compatibility to give some time for others to upgrade.
I've got multiple projects using a certain Dependency of Version XX, if I release a new version, I have to touch every project to change it to version XX.Y.
I've came across an approach to edit my m2 settings <version>${my.version}</version>, to add a parameter and bind it into my POM.xml, but this implicit means, everyuser has to manage their m2 settings when I do a new release.
Is there a way to central (user independant) manage the versions as in SVN, so none has to change anything and it always uses the up2date version, if I release a new version?
In general, the maintainer of every project should decide for themselves if they update the version or not. Updating the version might break things, so they may choose to stay on the older version. Particularly, it is important that the maintainers notice that something has changed, so that tests are run.
For development, though, there are Snapshot versions. A Snapshot dependency always references the newest version, but the -SNAPSHOT indicates this to the maintainer of the project. Snapshot versions should not go to production - the builds are not reproducible.
If artifacts are so tightly coupled that they are build together, think about using multi module projects.
You can use a pattern called "Bill of Material".
I think your question is somehow similar to this question (but not exactly a duplicate), and my answer applies here aswell:
You create a new maven project (the bill of material) that only consists of a pom with dependency management block. Here you declare all your dependencies and their versions. The packaging should be set to pom.
This bill of material (bom) project is now used as parent of all other projects. When using a dependency, only group id and artifact id is specified, the version tag is ommited. In that way, the version will be taken from the bom and you have one central place to manage the versions of the dependencies.
More details with examples are here (in the lower part of the page) or here.
Consider a project that is used in other projects, and this project has version 1.0 and is present in the Nexus.
After that, some changes are done in this project.
Now there are two solutions:
Increment the version of the project to 2.0, and delete the version 1.0 from Nexus. When the developers try to get the dependencies from the Nexus with version 1.0 they will get an error that this version does not exist and need to change the version to 2.0.
Change the functionalities of this project and inform the crew that some changes are done, but this is not the practice at all.
Is there any functionality in Maven and Nexus to simplify this task and make this all happen in the backend so the developers can't do anything, or is this not possible?
If you have an old version of an artifact that must not be used anymore because it has some dangerous bug, or it does not work with the new database structure or something like this, it may be advisable to move it to some non-public Nexus repository (and also delete it from the local repository of the build server), so that nobody can use it for release builds (people can use it for local builds, but this is usually not dangerous).
If you want to manage standard versions throughout your company, it is a good idea to have a parent pom or some boms which collect versions in a <dependenyManagment> section and can be included by the developers. This way, you only need to inform them to change one version number (namely the one of the parent pom or bom) instead of many.
Still, you are left with the problem that people do not read company newsletters. I know the problem that many developers of jars compile and test their source code against very old versions of their dependencies while the war/ear (that includes the jar) uses new versions.
In development, I can reference the latest version of an artifact as 1.2.3-SNAPSHOT. Now I need the same behaviour for release candidates, i.e. I would like to be able to depend on the latest release candidate (there should also be a procedure for the developer to declare development versions as release candidates).
I am not sure how to implement this behaviour properly. Should I use an additional repository for release candidates and move development versions to this repository if the developer requests it? Or can I somehow define a "second snapshot list", like 1.2.3-RC?
You can get there be re-configure a few things:
use a version range for the dependency
change the updatePolicy for the repository you store the release candidates. see https://maven.apache.org/settings.html (updatePolicy). If you store the artifacts in a maven proxy usually you need to allow to overwrite releases.
Remember in a multi module build that they might upload modules before detecting a failed build (due to mvn deploy being a phase not a goal). You need to verify the complete build is ok before starting to upload artifacts in the maven repository. Or stage them somehow.
Remember this will most certainly prevent your builds being reproducible. Since an RC dependency might change between builds. You would need to change the version range - which is not always an issue. Ranges may work for you.
I've better experience to let developers stay on snapshots but have the CI server set an explicit version (for example using the versions plugin) prior to deployment / releasing for the dependency the artifact uses.