The issue is known to many maven based projects using feature branches. In maven, there is no concept of a branch. Now in a git-flow environment there are basically 2 ways to deal with the build artifacts:
modify the version number to something like "featureX-1.0.0-SNAPSHOT".
Separate the SNAPSHOT repositories to get an isolated build and dev environment. Use a maven extension to determine the branch and select the proper repositories early in the process.
Variants of the two seem to exist, but both have its drawbacks. We would like to go in our team to option 2 and share the pipelines to publish maven artifacts to separate package feeds.
Is there any way to automate the creation of package feeds on the event of a git branch creation?
Related
I have Jenkins version 2.7.1 running on a Windows 7 machine. It is successfully pulling code from a subversion repository and running tests. I have the test jobs set up for the development branch of each project only.
We periodically make stable releases of the projects in jar files with version numbers. I would like to have Jenkins be the repository manager for those stable releases. These are made by hand - There is no Jenkins job making or testing stable releases. The projects do use Maven.
Each stable build is tagged in the subversion repository, so it could be made again on demand if needed.
I downloaded the Maven repository server hoping to make this fit the purpose. I read the documentation that's provided, but it's pretty terse. As I understand it and have it configured now, this appears to have a couple of issues:
If I go to jenkins-ip/plugin/repository/project, it has made directories there that expose the names of all of my projects, which seems undesirable. (Here jenkins-ip is the IP where I access Jenkins on my local network.)
On the other hand, there's nothing but empty directories under these projects, so they're currently useless.
These projects all correspond to the continuous testing of the development branch. There's no apparent way to get the stable builds into the hierarchy. (It doesn't seem efficient to create a job for each stable release...)
Is there anyway to get Jenkins (with this plugin or through another method) to be the repository manager just for the stable builds? I know that I can start a different repository manager like archiva, but it would be ideal to use Jenkins since it's already running and it seems to claim capability for this function now.
To use Maven repository server you have to build the project on Jenkins.
Then the plugin will expose all archived artifacts as maven repo.
Note you need to use a "Maven project" type for it to work (freestyle is not supported)
There are several plugins that will help you manage building from multiple tags, however not all of them work with "Maven project" type.
You could also try Jenkins pipeline (previously "Workflow") or the Job-DSL plugin.
A simplest solution would be to have a build parameter specify the tag name (then checkout e.g. ^/tags/projectname/${tagParam}), but you have to figure out how to trigger the job then.
Our company currently uses TFS for source control and build server. Most of our projects are written in C/C++, but we also have some .NET projects and wouldn't want to be limited if we need to use other languages in the future.
We'd like to use Git for our source control and we're trying to understand what would be the best choice for a build server. We have started looking into TeamCity, but there are some issues we're having trouble with which will probably be relevant regardless of our choice of build server:
Build dependencies - We'd like to be able to control the build dependencies for each <project, branch>. For example, have <MyProj, feature_branch> depend on <InfraProj1, feature_branch> and <InfraProj2, master>.
From what we’ve seen, to do that we might need to use Gradle or something similar to build our projects instead of plain MSBuild. Is this correct? Are there simpler ways of achieving this?
Local builds - Obviously we'd like to be able to build projects locally as well. This becomes somewhat of a problem when project dependencies are introduced, as we need a way to reference these resources or copy them locally for the build to succeed. How is this usually solved?
I'd appreciate any input, but a sample setup which covers these issues will also be a great help.
IMHO both issues you mention fall really in the config management category, thus, as you say, unrelated to the build server choice.
A workspace for a project build (doesn't matter if centralized or local) should really contain all necessary resources for the build.
How can you achieve that? Have a project "metadata" git repo with a "content" file containing all your project components and their dependencies (each with its own git/other repo) and their exact versions - effectively tying them together coherently (you may find it useful to store other metadata in this component down the road as well, like component specific SCM info if using a mix of SCMs across the workspace).
A workspace pull wrapper script would first pull this metadata git repo, parse the content file and then pull all the other project components and their dependencies according with the content file info. Any build in such workspace would have all the parts it needs.
When time comes to modify either the code in a project component or the version of one of the dependencies you'll need to also update this content file in the metadata git repo to reflect the update and commit it - this is how your project makes progress coherently, as a whole.
Of course, actually managing dependencies is another matter. Tons of opinions out there, some even conflicting.
We have 3 projects and a 4th one that is shared among them.
A multiple project in gradle requires one to create a wrapper project and include all the sub projects in it.
Each of our sub projects are being worked on by different team members and we use git as an svn.
Our concern before going into gradle is the fact that we will only have 1 git repository that consists of the wrapper project with all sub projects instead of 4 different repos, each for each sub project.
1) Are we missing something?
2) Is it possible to create 4 repos on a multi project?
3) One of our requirements is to deploy a single war (for example only webapp #1 out of the 4)--does using the multi project template make it possible?
ad 1) You have some choices:
Use a single Git repo.
Use multiple Git repos and exchange artifacts via a binary repository (e.g. Artifactory), with artifacts being produced on a regular basis by CI jobs.
Use something like Github submodules to create a "virtual" overall Git repo (wouldn't recommend this one).
ad 2) Gradle doesn't really care how many Git repos the build is comprised of, as long as everything that settings.gradle points to (at least all build scripts) exists on disk when the build starts. Of course it may be inconvenient for developers (and CI admins) to juggle multiple Git repositories and put them in the right (relative) locations.
ad 3) Yes.
I stumbled across this question on the same quest, and only later somebody pointed out another option to me (it's really a variant of the third choice listed under ad 1 in https://stackoverflow.com/a/20678444/1016514), which seems like an elegant hack:
You keep the subprojects in separate Git repos, and from within your root project point to them by setting the respective rootDir properties. This requires a fixed checkout layout, obviously, but that would usually be the case anyway.
The approach is described here: https://medium.com/codequest/gradle-multi-project-build-substituting-jar-dependencies-with-local-projects-4a5323f8680b
Update with Gradle 7.3+: this will now yield a warning, and likely not work after some point in the future:
Subproject ':xyz' has location '...\xyz' which is outside of the project root.
This behaviour has been deprecated and is scheduled to be removed in Gradle 8.0.
Consult the upgrading guide for further information:
https://docs.gradle.org/7.3.1/userguide/upgrading_version_7.html#deprecated_flat_project_structure
I am working with Team City for .net and use it for continuous integration - works well. I have it running off my main branch.
I now have a release branch - how I can I configure to set up a release branch in team city. What is the best way to do this?
What I've done before is to copy the build configuration of my trunk build and then just create a new VCS root pointing at the other branch and use that in the new configuration.
Your can track several branches using Branch Specification field of VCS root. Specify wildcard for you branches like
+:refs/heads/release_* (for release branches) or
+:refs/heads/* (for all branches)
More details in docs.
Some notes:
Run build button runs it for default branch. Click ellipsis -> Changes tab to select specific branch.
Now you cannot use artifact dependencies for specific branch. Such dependencies will always use the default branch.
Regarding issues with artifact dependencies, it's not easy to create deployment configurations from branch specific artifacts. In this case I'd go using separate configurations for each branch. Otherwise you should rely on API and/or some artifact path name parsing logic.
If you don't need per branch deployments, it's completely ok to just use branch specs approach.
I understand that during development build artifacts are placed in the snapshot repository.
When a product needs to go to QA for testing, do teams pull from the snapshot repository? Or do they do a full build, deploy to the release repository, and then give it to QA from there?
Also, if my snapshots repository holds all the build artifacts from each build, how is this commonly cleaned up? I could see keeping the last 5 builds from the build server, but not every one. I'm using Artifactory if it helps.
Opinions differ, here'e my approach:
Snapshots are for Dev
Typically used for "throwaway" builds. I publish them from my CI server, triggered by changes committed to the source code. The purpose of the snapshot build is to share the latest tested artifact from a particular team. This is important as teams might be sharing jars between each other.
This approach is much more stable than our previous approach of sharing the source code (Constant fire-fighting problems, when another team commits something that fails their build.... and by extension everyone elses).
Cleaning up snapshots
I use Nexus to manage my repository, it has a scheduled task that can be configured to periodically purge the snapshot repository. I'd imagine Artifactory has similar functionality.
Release candidates are for QA
I treat QA like a full-blown release to the customer. That's why I prefer the term "Release Candidate".
The key difference between a release candidate build and a snapshot is that the source code is "tagged" or "labelled" (dependent on your SCM system's terminology). What you're doing is drawing a line in the sand from which you can conveniently re-create the binary on demand.
Nexus professional has a staging suite, which enables development to cut a new release and hold it on a temporary "staging repository". Obviously some releases will fail testing in which case they're dropped. others are either promoted to the next group in the chain, or released to the public area.
There are several methods of implementing this "promotional model" of release management.
How release revisions are managed
I use the following numbering convention for my releases.
<major number>.<minor number>.<patch number>.<build number>
Example:
1.0.0.24
(For smaller/simpler projects I might alter the convention and drop the patch number).
Ivy has a wonderfully useful buildnumber task to manage the incrementing build number, based on what has already been published to your repository.
<property name="release.candidate" value="1.0.0"/>
..
<ivy:buildnumber organisation="${ivy.organisation}" module="${ivy.module}" revision="${release.candidate}"/>
..
<echo message="Release revision: ${ivy.new.revision}"/>
The release.candidate property is typically stored in a properties file, under version control. What I really like about this solution is that it enables parallel branch management (See answer to this question).