We have many projects that are treated, built and deployed the same way and we want to have a unified TeamCity build chain for all of them.
Our build chain should contain:
Compilation and testing for pull requests and develop branch.
SonarQube analysis for pull requests.
NPM publish and autolabel for merged pull requests into develop.
All of the projects are NPM packages that comply with the following:
required scripts: install, test, clean, build
package.json and sonar-project.properties in the root
We had the idea of using common parametrized builds for SonarQube and NPM publish, since they are agnostic to the project itself, but it looks like TeamCity does not support anything like this unless we replicate chain builds for each project.
Ideally it should look something like this:
The problem here is that if I add triggers for build finish and add dependencies to SonarQube and NPM publish for all of the projects it will require building every project and not only the one that has just been built.
TeamCity provides something like this by being able to duplicate build configurations:
I do not want to have many duplicated builds in the same way I do not fancy duplicated code. Is there a way to create the common build chain or should I move forward and look for a scripted way of generating the latter?
Related
We have set of different web application projects, which is delivered by Teamcity to different environments. At this moment, we are doing all-in-one builds: compile, package and deploy at once; all based on a templates.
Now I am investigating a way to separate concenrs: one build tests and produces package, another - delivers. Naturally, both builds are having own templates. Is there a way to template this build chain - so, when I choose some meta-template - both builds will be created with present Artifact dependencies?
Sadly not. What we've had to do is clone the build chains. We've put in place a mechanism which makes sure the set-ups of the different chains do not diverge.
Another option, admittedly ugly, is to set up a single build chain and let each of your projects pretend that it's a separate VCS branch. In this case there's a single set-up (for the single build chain) and to view the history of a given project you filter by that project "branch". Needless to say, that's not how branches should be used and you may run into some issue down the line.
Our company currently uses TFS for source control and build server. Most of our projects are written in C/C++, but we also have some .NET projects and wouldn't want to be limited if we need to use other languages in the future.
We'd like to use Git for our source control and we're trying to understand what would be the best choice for a build server. We have started looking into TeamCity, but there are some issues we're having trouble with which will probably be relevant regardless of our choice of build server:
Build dependencies - We'd like to be able to control the build dependencies for each <project, branch>. For example, have <MyProj, feature_branch> depend on <InfraProj1, feature_branch> and <InfraProj2, master>.
From what we’ve seen, to do that we might need to use Gradle or something similar to build our projects instead of plain MSBuild. Is this correct? Are there simpler ways of achieving this?
Local builds - Obviously we'd like to be able to build projects locally as well. This becomes somewhat of a problem when project dependencies are introduced, as we need a way to reference these resources or copy them locally for the build to succeed. How is this usually solved?
I'd appreciate any input, but a sample setup which covers these issues will also be a great help.
IMHO both issues you mention fall really in the config management category, thus, as you say, unrelated to the build server choice.
A workspace for a project build (doesn't matter if centralized or local) should really contain all necessary resources for the build.
How can you achieve that? Have a project "metadata" git repo with a "content" file containing all your project components and their dependencies (each with its own git/other repo) and their exact versions - effectively tying them together coherently (you may find it useful to store other metadata in this component down the road as well, like component specific SCM info if using a mix of SCMs across the workspace).
A workspace pull wrapper script would first pull this metadata git repo, parse the content file and then pull all the other project components and their dependencies according with the content file info. Any build in such workspace would have all the parts it needs.
When time comes to modify either the code in a project component or the version of one of the dependencies you'll need to also update this content file in the metadata git repo to reflect the update and commit it - this is how your project makes progress coherently, as a whole.
Of course, actually managing dependencies is another matter. Tons of opinions out there, some even conflicting.
I am new to Jenkins CI tool and I want to know if it is possible to specify what build to use when there are several projects, on different SVN locations, dependent on one another. For example, if I have the web project on SVN location1 and the backend project on SVN location2 and the web depends on the backend and one of the developers modifies something in backend, when the web developer does a commit, there will be a build failure. Is there the possibility to specify that the build from the web part should take into consideration build x from backend and not the newest build?
Thanks in advance.
yes that can be done. in Jenkins check for the Build Triggers options in your project web-settings and on the line Build after other projects are built you can specify the name of projects you want to build automatically after there has been changes made to the base project.
And similarly, in the Post-build Actions, look for Build other projects, where you can specify that if the base project builds successfully, it will automatically trigger a build on children projects.
Hope this helps.
Your example of building a project against a specific version of another project is a little non-standard, but not impossible.
In your case, I would use Jenkins' ability to execute arbitrary scripts to help. The script would take care of getting the correct version of the project that the one I want to build depends on.
Building on your example of a Web and Backend project, here's how I would do things without using a parametrized build:
Add a file to the repository of the Web project that stores the version of the Backend project to use
Configure a job to build the Web project when the source for the backend project changes in SVN.
The project should check out the latest version of the Web project
The first Build Step for the project would be a script (Execute Shell or Execute Windows Batch Command) that does the following:
Gets the version of the Backend to use from the file containing the version info
Either pulls the appropriate version of the Backend from the Backend's repository; or pulls the source of the appropriate version of the Backend's source
(If you pulled the source only for the Backend, the next Build Step should be to build the Backend next)
Build the Web piece
Do any unit tests
I've started using TeamCity personal builds, via the new Git remote run feature in TeamCity 6.5. Doing a single build works fine; I have a project that compiles from source, and I gave it a Branch Remote Run trigger.
However, it looks like TeamCity only triggers the one project that has the Branch Remote Run trigger applied. I have several unit test projects, set up in a chain with Finish Build triggers, and none of these get run. Furthermore, if I try to start a custom build of one of these unit test projects, I can't use the artifacts from my personal build: I can only pick artifacts from one of the 'official' builds.
Can I get TeamCity personal builds to work with build chains?
With the setup that you have (snapshot dependencies and finish build triggers), you can achieve build chaining by submitted your personal changes to the builds you are looking to trigger. For example, if you have projects A and B where B depends on A - run the remote build against project B and A will be triggered first and B will be added to the queue. Both of these builds will have your personal changes.
If you are using the TeamCity Visual Studio plugin you can select which builds you want to send your changes to and you just need to tick the box for B instead of A.
The finished build trigger won't be fired, but the build chaining means that A must be built first.
More info - http://confluence.jetbrains.net/display/TCD7/Build+Chain
(You have tagged TeamCity 6.5, but 7 has now been released so I have included the documentation for the newer version)
I suppose you should setup your chain not with Finish Build trigger, but with "Snapshot dependencies" feature of TeamCity. And, setup artifacts dependency basing on the snapshots.
Please read about snapshot dependencies in TeamCity here.
I am the Configuration manager for an IT firm. Currently we are using anthill build management server for all our build related purposes. We are looking to implement Continuous Integration in our development life cycle.
Currently the building process is done manually. Suppose there are 5 projects A,B,C,D,E and E is the parent project and the dependency chain does like this:
A->B->C->D->E
What we do is we build A first update project.xml of B to the latest version of A, build B so on and so forth untill all dependent projects get built and finally parent project gets built.
What I am thinking is automating the entire process i.e. automatically finding out dependencies and building them first and then updating the version of parent projects and building them again to a newer version.
Would continuum do this for me? If not is here any other CI tool that does this?
Hudson does this really well, if you're using Maven, it'll even automatically figure out the build dependencies for you automatically after the first build, otherwise you can manually define the build dependencies. I.e., it lets you configure the system to build project B after a successful project A build.
I'm not sure if it matters to you, but Hudson is also open source.
If not is here any other CI tool that does this?
I like TeamCity, which does pretty much everything you'll need. With the latest version (and a plugin from JetBrains), there's even Git support.
On the other hand, any continuous integration system should handle dependencies easily.
We use Zed Builds and Bugs for a setup similar to this. We have a master project that has sub-project dependencies and the build system handles everything in the proper order.
We also have very small, tight builds for the sub-projects so that each of them can be built when the developers commit to source control. The Zed Server is capable of pulling the latest artifacts from these small builds and putting them together into larger builds, but we haven't yet used that feature.
Our check-ins trigger the small CI builds, and then twice per day the entire application is re-built from scratch, following the dependency chain.
I'd agree with OregonGhost, though, any CI system should be able to set up this type of chain.
I don't think you need a CI tool for this. Try to automate this using a buildscript and use Continuum (or any other CI tool) to trigger your preferred buildtool.