I understand that during development build artifacts are placed in the snapshot repository.
When a product needs to go to QA for testing, do teams pull from the snapshot repository? Or do they do a full build, deploy to the release repository, and then give it to QA from there?
Also, if my snapshots repository holds all the build artifacts from each build, how is this commonly cleaned up? I could see keeping the last 5 builds from the build server, but not every one. I'm using Artifactory if it helps.
Opinions differ, here'e my approach:
Snapshots are for Dev
Typically used for "throwaway" builds. I publish them from my CI server, triggered by changes committed to the source code. The purpose of the snapshot build is to share the latest tested artifact from a particular team. This is important as teams might be sharing jars between each other.
This approach is much more stable than our previous approach of sharing the source code (Constant fire-fighting problems, when another team commits something that fails their build.... and by extension everyone elses).
Cleaning up snapshots
I use Nexus to manage my repository, it has a scheduled task that can be configured to periodically purge the snapshot repository. I'd imagine Artifactory has similar functionality.
Release candidates are for QA
I treat QA like a full-blown release to the customer. That's why I prefer the term "Release Candidate".
The key difference between a release candidate build and a snapshot is that the source code is "tagged" or "labelled" (dependent on your SCM system's terminology). What you're doing is drawing a line in the sand from which you can conveniently re-create the binary on demand.
Nexus professional has a staging suite, which enables development to cut a new release and hold it on a temporary "staging repository". Obviously some releases will fail testing in which case they're dropped. others are either promoted to the next group in the chain, or released to the public area.
There are several methods of implementing this "promotional model" of release management.
How release revisions are managed
I use the following numbering convention for my releases.
<major number>.<minor number>.<patch number>.<build number>
Example:
1.0.0.24
(For smaller/simpler projects I might alter the convention and drop the patch number).
Ivy has a wonderfully useful buildnumber task to manage the incrementing build number, based on what has already been published to your repository.
<property name="release.candidate" value="1.0.0"/>
..
<ivy:buildnumber organisation="${ivy.organisation}" module="${ivy.module}" revision="${release.candidate}"/>
..
<echo message="Release revision: ${ivy.new.revision}"/>
The release.candidate property is typically stored in a properties file, under version control. What I really like about this solution is that it enables parallel branch management (See answer to this question).
Related
The issue is known to many maven based projects using feature branches. In maven, there is no concept of a branch. Now in a git-flow environment there are basically 2 ways to deal with the build artifacts:
modify the version number to something like "featureX-1.0.0-SNAPSHOT".
Separate the SNAPSHOT repositories to get an isolated build and dev environment. Use a maven extension to determine the branch and select the proper repositories early in the process.
Variants of the two seem to exist, but both have its drawbacks. We would like to go in our team to option 2 and share the pipelines to publish maven artifacts to separate package feeds.
Is there any way to automate the creation of package feeds on the event of a git branch creation?
Wondering how people manage their project artefacts through an environment lifecycle of say DEV - AQA - CQA - RELEASE and if there's some best practices to follow.
I use a Jenkins build server to build my projects (code checkout then maven build). My artefacts all have version 1.0.0-SNAPSHOT and are published to a local .m2 repo on the build server. There are also Jenkins jobs that rebuild the DEV system (on the same server) using those artefacts. The project build is automated whenever someone checks in code. The DEV build is automated on a nightly basis.
At some point, my lead developer determines that our project is fit to go to AQA (the first level of testing environment on a different server).
For this I need to mark the artefacts as version 1.0.0-1 and publish to a remote AQA repository (it's actually a Nexus repo).
The Maven deploy plugin sounds like the right approach, but how do I change the version number to be effectively 1.0.0-$release (where $release is just an incrementing number starting from 1)? Would Maven/Nexus be able to manage the value of $release, or would I need a simple properties file in my project to store/update the last used $release.
Furthermore, someone tests AQA and determines its fit to move on to CQA (second testing env). This is 'promote to AQA'. So my requirement is to copy the artefact from the AQA Nexus repo and publish to the CQA Nexus repo.
Likewise, after CQA, there'd be a 'promote to RELEASE' job too.
I think the version value remains unchanged during the 'promote' phases. I'd expect the AQA repo to see all versions 1-50, but CQA only 25 and 50, then RELEASE only 50, for example.
I can find loads of info about Maven plugins/goals/phases, but very little about a prescriptive method on how or where to use outside of the immediate development environment.
Any suggestions gratefully received.
Staging/promoting is out of scope for Maven. Once deployed/uploaded to a remote repository, that system is responsible for the completion of the release cycle. Read this chapter about staging: http://books.sonatype.com/nexus-book/reference/staging.html if you use Nexus.
Build numbers are just that build numbers. They are not promotion / staging numbers.
You should come up with another means of tracking your promotions, because otherwise one might get confused in "knowing" that build 10.1.4-2 is the same as 10.1.4-6. Certainly, all of the Maven related software will see those two builds as different builds.
In addition, if a person "grabs" the wrong copy of the build, the way you are managing staging within your build number will increase confusion. As if you don't kill all of the 10.1.4-2 builds, then someone might get a copy of that not realizing that the build has been promoted to 10.1.4-6. This means that for the "last" staging number to be the most likely one to be grabbed, you must do two things (which are impossible in combination)
Remove all the old staging numbers, updating them to the new ones.
Ensure that no copy of an old staging number escaped the update.
Since people generally can copy files without being tracked, or said files might not be reachable at time up "update", or timing between reaching all the files cannot be simultaneous, such a system is doomed to fail.
Instead, I recommend (if you must track by file), placing the same file in different "staging directories". This defines release gateways by whether the file exists in a certain directory, and makes it clear that it is the same file that is going through the entire process. In addition, it becomes easy to have various stages of verification poll their respective directories (and you can write Jenkins tasks to promote from one directory to another, if you really wish).
My project has snapshot dependencies for which no releases are available. If I fix the version, such as <version>0.0.1-20140219.100706-347</version> instead of <version>0.0.1-SNAPSHOT</version>, do I now enjoy the benefits in speed of using releases or am I still subject to automatic updates slowing my build down just by using a dependency that resides in a snapshot repository? Are there any benefits of releases then other than having kind of a tag to a specific version?
do I now enjoy the benefits in speed of using releases or am I still subject to automatic updates slowing my build down
Yes; a timestamped SNAPSHOT version refers to a unique artifact, so Maven won't check again. You could also consider setting an update policy to reduce the frequency of checks (How does the updatePolicy in maven really work?).
Are there any benefits of releases then other than having kind of a tag to a specific version?
As a general practice, SNAPSHOT builds (even timestamped ones) aren't intended to stick around. Because you'll have one for every build it's normal to prune them (e.g. How to limit number of deployed snapshots artifacts in Nexus?). At some point you'll want to pick a specific version that will be kept permanently, and which can be used for reproducible builds: that's what final release versions are for.
I want some "Maven with Mercurial release" knowledgeable opinion.
I already saw this thread and a few others.
We want to be agile and fast. We want to build once. Not one build for testing and one for release.
We have setup a continuous release process where we do the following:
Jenkins checks for SCM changes on "server" repository (every 30 minutes). Also can be triggered manually
Jenkins clones "server" repository
Jenkins increments version in pom (example: 1.0.1 -> 1.0.2). NO Snapshots
Start build, package, automatic deployments and tests
If all is good, commit changed pom, tag and push back to "server" repository (merge if needed)
Release good version to a "releases" system, where it's available for QA for further testing
Developers will pull back changes from "server" repository and get new versiond poms
If a hot fix is needed for a good release
A developer will clone the relevant tag from "server" repository
Do the fix, push back to a "server+fix" repository
Jenkins will build as before
If good, the fix will also be pushed to the main "server" repository
We found this process to be very quick and clean.
I want your opinion on the flow, with ideas to improve. Remember - Fast, Continuous and Build once.
Thanks in advance!
+1 to the no snapshots rule. I think its all good.
Release good version to a "releases" system, where it's available for
QA for further testing
What is the "releases" system. Is it a maven repository? Does a new build automatically get deployed to the QA environment? How does it pull from the respository? You could pull a jar from a maven repo using dependency:get. But if want to make use of infrastructure automation tools like Chef or Puppet for your deployment side, you'd be better off publishing rpms to your maven repository (publish using the maven-rpm-plugin and enable pulls with the nexus-yum-plugin for example) or simply setting up your own yum repository.
What would be a good way to get 'mvn deploy' for snapshot deployment to create a snapshot tag(x.y.x-20110304.mmhhss-1) in git using the snapshot version(x.y.x-20110304.mmhhss-1) identifier created by maven, release plugin creates release tags. I would like a tag for snapshots.
Thanks
-am
This feature is not supported by Maven (Snapshot) repositories. Snapshots are designed to be throw-away or development builds. Repository managers like Nexus often have background jobs which purge old snapshots.
The purpose of tagging is to be able to reproduce that binary again. If you really care, perhaps you should release more often?
Finally, some version control systems impose a rather stupid overhead on tagging operations (updating each file in the repository.... Clearcase, CVS, etc). This has traditionally discouraged tagging during a CI build (which would typically generate the snapshot release).