I want some "Maven with Mercurial release" knowledgeable opinion.
I already saw this thread and a few others.
We want to be agile and fast. We want to build once. Not one build for testing and one for release.
We have setup a continuous release process where we do the following:
Jenkins checks for SCM changes on "server" repository (every 30 minutes). Also can be triggered manually
Jenkins clones "server" repository
Jenkins increments version in pom (example: 1.0.1 -> 1.0.2). NO Snapshots
Start build, package, automatic deployments and tests
If all is good, commit changed pom, tag and push back to "server" repository (merge if needed)
Release good version to a "releases" system, where it's available for QA for further testing
Developers will pull back changes from "server" repository and get new versiond poms
If a hot fix is needed for a good release
A developer will clone the relevant tag from "server" repository
Do the fix, push back to a "server+fix" repository
Jenkins will build as before
If good, the fix will also be pushed to the main "server" repository
We found this process to be very quick and clean.
I want your opinion on the flow, with ideas to improve. Remember - Fast, Continuous and Build once.
Thanks in advance!
+1 to the no snapshots rule. I think its all good.
Release good version to a "releases" system, where it's available for
QA for further testing
What is the "releases" system. Is it a maven repository? Does a new build automatically get deployed to the QA environment? How does it pull from the respository? You could pull a jar from a maven repo using dependency:get. But if want to make use of infrastructure automation tools like Chef or Puppet for your deployment side, you'd be better off publishing rpms to your maven repository (publish using the maven-rpm-plugin and enable pulls with the nexus-yum-plugin for example) or simply setting up your own yum repository.
Related
I'm on a project that uses Mulesofts Anypoint studio for API development. Anypoint studio uses Maven to build its projects and handle versioning. Currently we are using a Git-flow branching model without release branches (feature/development/master) and the CI tool is Jenkins. We want to migrate to Azure-DevOps with Trunk-based branching.
The current git-flow CI set-up works as follows:
Merge feature with Dev
Jenkins triggers on Dev and builds a package with Maven
Jenkins deploys the package to D
Test your API on the D, if its good manually trigger Jenkins release pipeline
Jenkins will merge dev with master
Maven creates a release; creating a package (e.g. v0.1) and prepare the next development iteration (v0.2-SNAPSHOT) in master
Jenkins Deploys the package on A and P
Merge master back into Dev
So we have 2 separate pipelines here: dev and release. I'm hoping trunk-based development makes this process simpler by reducing it to one pipeline that is triggered by merges into master. But I don't know how to handle the versioning with Maven here.
This is what I have in mind for the Trunk-based (feature/master) pipeline:
Merge feature with Master
Azure pipelines triggers Maven release
(!) Maven takes the 0.1-SNAPSHOT version and creates a 0.1 release version and puts it..where? in a release branch?
Maven then creates the next working version 0.2-SNAPSHOT on the master branch
(!) this would trigger Azure pipelines again, creating a loop
Anyone have experience with Trunk Based development with Maven solve this? I would actually prefer to do it without a release branch because the released packages are already providing the release-branch functionality.
At first glance this answer seemingly solves my issue, but it just ignores the issue by going around it and just using a hash.
How can I copy the maven artifacts from one repository to another?
I create a release and I do not want to publish this release immediately. I want to leave it for some days as "internal" to test it. Later on, when all tests pass, I want to move these artifacts into a public release repository.
maven-stage-plugin is no longer maintained.
The release plugin would build a new release, but I want to use the same artifact, that has already been tested.
it looks to me that you need CI environment which supports different pipelines.
Every commit/push should trigger a deployment on your DEV environment where you can test your application.
Upon tag the CI environment would trigger a different pipeline stage which can 'release' application/library.
The tag effectively snapshots your version and creates a distribution.
I have Jenkins version 2.7.1 running on a Windows 7 machine. It is successfully pulling code from a subversion repository and running tests. I have the test jobs set up for the development branch of each project only.
We periodically make stable releases of the projects in jar files with version numbers. I would like to have Jenkins be the repository manager for those stable releases. These are made by hand - There is no Jenkins job making or testing stable releases. The projects do use Maven.
Each stable build is tagged in the subversion repository, so it could be made again on demand if needed.
I downloaded the Maven repository server hoping to make this fit the purpose. I read the documentation that's provided, but it's pretty terse. As I understand it and have it configured now, this appears to have a couple of issues:
If I go to jenkins-ip/plugin/repository/project, it has made directories there that expose the names of all of my projects, which seems undesirable. (Here jenkins-ip is the IP where I access Jenkins on my local network.)
On the other hand, there's nothing but empty directories under these projects, so they're currently useless.
These projects all correspond to the continuous testing of the development branch. There's no apparent way to get the stable builds into the hierarchy. (It doesn't seem efficient to create a job for each stable release...)
Is there anyway to get Jenkins (with this plugin or through another method) to be the repository manager just for the stable builds? I know that I can start a different repository manager like archiva, but it would be ideal to use Jenkins since it's already running and it seems to claim capability for this function now.
To use Maven repository server you have to build the project on Jenkins.
Then the plugin will expose all archived artifacts as maven repo.
Note you need to use a "Maven project" type for it to work (freestyle is not supported)
There are several plugins that will help you manage building from multiple tags, however not all of them work with "Maven project" type.
You could also try Jenkins pipeline (previously "Workflow") or the Job-DSL plugin.
A simplest solution would be to have a build parameter specify the tag name (then checkout e.g. ^/tags/projectname/${tagParam}), but you have to figure out how to trigger the job then.
I understand that during development build artifacts are placed in the snapshot repository.
When a product needs to go to QA for testing, do teams pull from the snapshot repository? Or do they do a full build, deploy to the release repository, and then give it to QA from there?
Also, if my snapshots repository holds all the build artifacts from each build, how is this commonly cleaned up? I could see keeping the last 5 builds from the build server, but not every one. I'm using Artifactory if it helps.
Opinions differ, here'e my approach:
Snapshots are for Dev
Typically used for "throwaway" builds. I publish them from my CI server, triggered by changes committed to the source code. The purpose of the snapshot build is to share the latest tested artifact from a particular team. This is important as teams might be sharing jars between each other.
This approach is much more stable than our previous approach of sharing the source code (Constant fire-fighting problems, when another team commits something that fails their build.... and by extension everyone elses).
Cleaning up snapshots
I use Nexus to manage my repository, it has a scheduled task that can be configured to periodically purge the snapshot repository. I'd imagine Artifactory has similar functionality.
Release candidates are for QA
I treat QA like a full-blown release to the customer. That's why I prefer the term "Release Candidate".
The key difference between a release candidate build and a snapshot is that the source code is "tagged" or "labelled" (dependent on your SCM system's terminology). What you're doing is drawing a line in the sand from which you can conveniently re-create the binary on demand.
Nexus professional has a staging suite, which enables development to cut a new release and hold it on a temporary "staging repository". Obviously some releases will fail testing in which case they're dropped. others are either promoted to the next group in the chain, or released to the public area.
There are several methods of implementing this "promotional model" of release management.
How release revisions are managed
I use the following numbering convention for my releases.
<major number>.<minor number>.<patch number>.<build number>
Example:
1.0.0.24
(For smaller/simpler projects I might alter the convention and drop the patch number).
Ivy has a wonderfully useful buildnumber task to manage the incrementing build number, based on what has already been published to your repository.
<property name="release.candidate" value="1.0.0"/>
..
<ivy:buildnumber organisation="${ivy.organisation}" module="${ivy.module}" revision="${release.candidate}"/>
..
<echo message="Release revision: ${ivy.new.revision}"/>
The release.candidate property is typically stored in a properties file, under version control. What I really like about this solution is that it enables parallel branch management (See answer to this question).
What would be a good way to get 'mvn deploy' for snapshot deployment to create a snapshot tag(x.y.x-20110304.mmhhss-1) in git using the snapshot version(x.y.x-20110304.mmhhss-1) identifier created by maven, release plugin creates release tags. I would like a tag for snapshots.
Thanks
-am
This feature is not supported by Maven (Snapshot) repositories. Snapshots are designed to be throw-away or development builds. Repository managers like Nexus often have background jobs which purge old snapshots.
The purpose of tagging is to be able to reproduce that binary again. If you really care, perhaps you should release more often?
Finally, some version control systems impose a rather stupid overhead on tagging operations (updating each file in the repository.... Clearcase, CVS, etc). This has traditionally discouraged tagging during a CI build (which would typically generate the snapshot release).