I'm fairly new to TeamCity (but not to CI systems) and I've been trying to figure out how to work this configuration out:
I have latest TeamCity Professional Version 9.1.3 (3 build agents, 20 configs) installed
Here's my TeamCity Project layout:
Project A
-- Build Product X (WIN)
-- Build Product Y (WIN)
-- Build Product Z (Linux)
I have dedicated 3 agents to build the above build configurations accordingly - 2 on Windows and 1 agent on Linux
WIN products are built using a mix of batch, powershell and msbuild scripts
Linux product is built using shell script
Triggering these 3 builds (under Project A), manually, works just fine. However, this is not feasible as we have many feature branches and all of them would have similar build configurations - 3 clicks for each build + setting Build Parameters on each Build configuration is expensive.
So, Here are my questions:
Is there a way to trigger the entire Project to build with a Single click? doing so, should run these builds in parallel
If 1 is possible, then how do I set the same build number (build params) across these 3 build configurations?
Is it possible to set up a VCS trigger that would poll for changes on any of the repos that build these and trigger the entire project (provided 1 is possible)
Please note I tried configuring both snapshot and artifact dependencies to make this work, but creating dependencies only pauses the other build configurations to wait until the dependent project is complete but that's not feasible for us - they need to run in parallel. (our builds take approx 45mins to complete) - Yes, we have a huge product to package.
I'll be grateful for any pointers in the right direction
Thanks
Yes, one-click triggering of parallel runs is achievable: create a 4th (or 0th) build configuration that does nothing (no build steps). Let's call this config "Zero". Your three build configs will each have a "Finished build" trigger on Zero (trigger when Zero finishes) as well as a snapshot dependency on Zero.
The best bit: you only need to define the common/shared parameters in Zero and the other three configs can reuse these. For example, if you define %foo% in Zero, the other three can all use %dep.MyProject_Zero.foo%. This also means you can get at Zero's build number: %dep.MyProject_Zero.build.number%. In each of your three build configs, switch to the "General Settings" and set your "Build number format" to the above.
For VCS triggering, just set Zero to span all the three VCS areas. You're suggesting they are each in a different repo... I have no experience with that but assume that Zero can span all three repos.
Last, if you use feature branches, and your VCS is git, mercurial, or Perforce Streams, make sure you've read about TeamCity feature branches support; it can save you a lot of time!
Related
BACKGROUND :
We are working on a few C projects, and the sources are in ClearCase. Each project have his own project ClearCase. My mission consists in putting all of them into Jenkins. There is no Continuous integration so far on this projects.
Some details, but very important :
The projects MUST be build with Windows AND Linux. Clients wishes.
The projects are developed on Eclipse, but no Maven, Ant... Only manual build with shell and batch. Already got some makefile to execute, using MinGW and GCC.
Same for the futures tests, they will be shell and batch scripts. They use real time, so, one test could be 3 minutes long. Average of 15 tests for each projects).
Here we are.
I'm thinking using a free-style project, with 2 slaves (Windows and Unix) for the build. The duration of the test are, now, not really a problem, they should be execute the night. So, maybe using slaves, don't know yet.
What do you guys think ? Got some advice before I get my hand dirty ?
Also, is it possible to have only one Jenkins project, with multiple sub-project with distinct ClearCase sources (and job and sub jub on each of this projects) ? (was looking to MultiJobs, matrix project, multi-SCM Jenkins: best way to build a project with sub projects, but couldn't understand or find the right method)
You do not exactly "put a project into Jenkins"
You define Jenkins jobs (start with one per project, you will worry about pipeline later). Those jobs must access the sources of your project in order to populate a Jenkins build workspace and execute whatever you define as a job (in a freestyle job, that can be any script in any language).
You will need the ClearCase plugin (or ClearCase UCM plugin of your projects are using UCM): that will allow your job to get the sources of the project.
Start by testing a job directly on the master (that node will be its own slave).
Once it is working, depending on the number of jobs and their frequency, you will scale by adding more slaves.
we're trying to improve our Jenkins setup. So far we have two directories: /plugins and /tests.
Our project is a multi-module project of Eclipse Plugins. The test plugins in the /tests folder are fragment projects dependent on their corresponding productive code plugins in /plugins.
Until now, we had just one Jenkins job which checked out both /plugins and /tests, built all of them and produced the Surefire results etc.
We're now thinking about splitting the project into smaller jobs corresponding to features we provide. It seems that the way we tried to do it is suboptimal.
We tried the following:
We created a job for the core feature. This job checks out the whole /plugins and /tests directories and only builds the plugins the feature is comprised of. This job has a separate pom.xml which defines the core artifact and tells about the modules contained in the feature.
We created a separate job for the tests that should be run on the feature plugins. This job uses the cloned workspace from the core job. This job is to be run after the core feature is built.
I somehow think this is less than optimal.
For instance, only the core job can update the checked out files. If only the tests are updated, the core feature does not need to be built again, but it will be.
As soon as I have a feature which is dependent on the core feature, this feature would either need to use a clone of the core feature workspace or check out its own copy of /plugins and /tests, which would lead to bloat.
Using a cloned workspace, I can't update my sources. So when I have a feature depending on another feature, I can only do the job if the core feature is updated and built.
I think I'm missing some basic stuff here. Can someone help? There definitely is an easier way for this.
EDIT: I'll try to formulate what I think would ideally happen if everything works:
check if the feature components have changed (i.e. an update on them is possible)
if changed, build the feature
Build the dependent features, if necessary (i.e. check ob corresponding job)
Build the feature itself
if build successful, start feature test job
let me see the results of the test job in the feature job
Finally, the project job should
do a nightly build
check out all sources from /plugins and /tests
build all, test all, send results to Sonar
Additionally, it would be neat if the nightly build was unnecessary because the builds and test results of the projects' features would be combined in the project job results.
Is something like this possible?
Starting from the end of the question. I would keep a separate nightly job that does a clean check-out (gets rid of any generated stuff before check-out), builds everything from scratch, and runs all tests. If you aren't doing a clean build, you can't guarantee that what is checked into your repository really builds.
check if the feature components have changed (i.e. an update on them is possible)
if changed, build the feature
Build the dependent features, if necessary (i.e. check ob corresponding job)
Build the feature itself
if build successful, start feature test job
let me see the results of the test job in the feature job
[I am assuming that by "dependent features" in 1 you mean the things needed by the "feature" in 2.]
To do this, I would say that you have multiple jobs.
a job for every individual feature and every dependent feature that simply builds that feature. The jobs should be started by SCM changes for the (dependent) feature.
I wouldn't keep separate test jobs from compile jobs. It allows the possibility that successfully compiled code is never tested. Instead, I would rely on the fact that wen a build step fails in Jenkins, it normally aborts further build steps.
The trick is going to be in how you thread all of these together.
Let's say we have a feature and it's build job called F1 that is built on 2 dependent features DF1.1 and DF1.2, each with their own build jobs.
Both DF1.1 and DF1.2 should be configured to trigger the build of F1.
F1 should be configured to get the artifacts it needs from the latest successful DF1.1 and DF1.2 builds. Unfortunately, the very nice "Clone SCM" plugin is not going to be of much help here as it only pulls from one previous job. Perhaps one of the artifact publisher plugins might be useful, or you may need to add some custom build steps to put/get artifacts.
We are using bamboo as our integration server. During each build it produces binary packs of our products. Some of the built artifacts then goes in to QA.
Is there a way to retain build artifacts of a certain build job number, irrespective of the global build expire configuration. For an example, at some point we identify one of the built artifacts as release candidate.
The QA should be able to download that specific pack even after one weeks time. Right now we are copying the build artifacts from CI server to some other machine. It is a script, but still it is a manual process.
In Hudson there is an option called 'keep this build forever'.
Depending on your version this is possible by applying a label to it. Under a plan's configuration on the "Miscellaneous" tab, you can set a label which can then be used to prevent a build from expiring.
For example, our system has builds that can get labelled "SaveBuild" which then prevents their expiry.
I would like to use some build parameters from Project 1 in Project 2. I know that I can make Project 1 a dependency of Project 2 and then access its build parameters as described in Dependencies Properties, but I do not want Project 1 to be built in response to a build of Project 2. For example, suppose I want Project 2 to be built nightly, while I only want Project 1 built monthly.
Is there any way Project 2 can access Project 1's build parameters under these conditions?
I would use a build configuration template that is shared between the two projects.
This means you can share properties between the projects, but also override certain ones in each individual project.
We use this for hourly builds that are not tagged and nightly ones that are tagged.
Then use a different build trigger to set one off nightly and the other monthly.
EDIT
I'll just expand slightly as a result of your comment.
In TeamCity we have 2 build configuration for the same project. One that builds on every check-in to give developers quick feedback on their contribution (build within 15 minutes). It does the following:
Builds the project in Debug
Runs all unit tests
Checks results of build into Subversion
The other configuration runs every night at midnight; it build everything and as a result takes a long time (around 45 minutes). It does the following:
Build the project in Debug and Release
Runs all unit tests
Builds Sandcastle documentaion
Checks results of build into Subversion
Grabs the Sandcastle output at an artefact so developers can easily download it.
As you pointed out this isn't as straightforward as one would like; however you can use the following to achieve it:
We use the Autoincrementer to share build numbers between the two configurations (they both increment the same build number when built).
We have a property on template that defines what artefacts to collect and is referenced from the artefacts field. The property is overridden on the second build config to define the sandcastle output to grab.
Sharing VCS Roots is mentioned on the documentation. Both our builds get the source from the same place, and tag the results to the same place. One VCS is most definitely all we need.
Bit of a long edit but I think it goes exactly on the lines of what you're trying to achieve. I appreciate I should have included this in the original answer.
HTH
Dependency is different from Build Triggering in TeamCity. If you make one project dependent on another ( artifact dependency ), it does not mean that the the latter will trigger the former.
Even when one project has been defined as dependent on another ( and also, even if not ) you have to specify explicitly the build trigger ( in this case a Finish build trigger ) for the dependent project to be triggered.
We've just started to use TeamCity as a continuous integration server. There's a problem we're trying to solve now:
We have a 'release' configuration, it has build versions set like this: 1.0.0.{0}
We also have a 'nightly build' configuration, which build number is: 1.0.0.0.{build.vcs.number.1}
So the first 2 digits are OK, Major+Minor versions. Third one is supposed to be set up manually as well according to our process (rarely though).
But as you can see, the last one increases with every 'release'.
The question is - how do I make TC copy the current 4th digit (or all of them) of 'release' to that of 'nightly build'?
One way to do it in 4.5 is to
Make nightly have a snap shot dependency on release.
find the internal build id of release.
use a build number format in nightly that looks like
%dep.releaseid.system.build.number%.{build.vcs.number.1}
where releaseid is the id you found in step 2. This will replace the whole %...% thing with the build number from release.
The TeamCity 4.5 Docs for dependency properties explains this and shows you how to find the internal id.
If you can know it in your build script, you can send it to team city during the build. Here are the instructions. I can think of a couple of strategies for getting the build number from the release build, none of them elegant. The most obvious would be to publish an artifact from the release build that is basically a text file with the build number in it, and retrieve that from your build script.