We have set of different web application projects, which is delivered by Teamcity to different environments. At this moment, we are doing all-in-one builds: compile, package and deploy at once; all based on a templates.
Now I am investigating a way to separate concenrs: one build tests and produces package, another - delivers. Naturally, both builds are having own templates. Is there a way to template this build chain - so, when I choose some meta-template - both builds will be created with present Artifact dependencies?
Sadly not. What we've had to do is clone the build chains. We've put in place a mechanism which makes sure the set-ups of the different chains do not diverge.
Another option, admittedly ugly, is to set up a single build chain and let each of your projects pretend that it's a separate VCS branch. In this case there's a single set-up (for the single build chain) and to view the history of a given project you filter by that project "branch". Needless to say, that's not how branches should be used and you may run into some issue down the line.
Related
When a private agent build starts in VSTS, it gets assigned a directory, e.g. C:\vstsagent_work\1\s
Is there a way to set this to a different path? On other CI servers, like Jenkins, I can define a custom workspace for a job. I'm dealing with a huge monorepo and have dozens of build definitions around the same repository. It makes sense (to me anyway) to share a single directory on the build agent computer.
The benefit to me is that my builds can use pre-built components from upstream repositories, if they have already been built.
Thanks for any help
VSTS build always creates a working directory per build definition. This leaves you two options:
Create a single build definition and use conditionals on steps to skip certain steps in order to only run what is needed. This allows you to use the standard steps and may require a powershell script to figure out which steps to run and which ones to skip. Set variables from powershell using the special logging commands.
Disable the get sources step and add a step that manually fetches sources. You'll need to clean the working directory, checkout the right commit, basically replicating the actions in the get sources step manually. It may require some fidgeting to get all the behavior correctly for normal build, pull request builds etc. That way you can take full control over the location where sources are checked out.
I'd also recommend you investigate the 2017 project formats that use the new <packageReference> in the project files to fetch packages. The new system supports configuring a version range which can always fetch the latest available version of packages. It's a better long-term solution.
No, it isn’t available in VSTS build system.
You can change working directory of agent (C:\vstsagent_work) (Re-configure it and specify another working folder), but it won’t uses the same source folder for different build definitions, the folder would be 1, 2, 3 ….
I have been working with TeamCity, Jenkins and Bamboo my last 8years. Latest 2years very involved in ContinuousIntegration factory setup and mainteinance on my team with very good results, giving me a lot of habits about how to deal with builds, artifacts and pipelines.
Now, i'm on a new company, new team, new CI, TFS2015, first time for me.
Just one month before I arrive to this new team they were on TFS2012, with XAMLs, so I took the migration to vNext builds.
At first look, I found on vNext builds the classic build definition, i mean adding steps as a single task to the build, instead of monolithic XAML file.
But with the time, i was trying to create more complex builds, like TeamCity build chains, but this is not possible, strike one...
Then, I was trying to deal with multiple branches, one continuous build for each branch (we are on TFSVC), create packages from each branch, and I found I was duplicating my builds just to change repository paths and a few details, so I moved builds to templates trying to reuse build definitions, introducing variables to generates paths(for repos and branches) and versions and expecting to change only in one place the build and having this changes reflected in all builds derived from templates... but that's was not the case... :
variables are not accepted everywhere, like in repo paths.
can't change templates after created them, just replace them, and builds created from templates are not affected after template is changed.
strike two ?.....
I'm wondering if maybe I'm doing things wrong with TFS, maybe this is a different system and I can't do things like in other CIs.
Any advice in how to approach TFS to have a good, dynamic and reusable set of builds ??
here isn’t the feature that the changes of a build definition or template affects other existing build definitions.
If the build steps of build definitions are the same for each branch, you just need a build definition and add filters for each branch (Triggers > Continuous integrations (CI), after that it will uses corresponding source to build for CI build, for example, develop branch changes > Trigger a build automatically with develop branch source.
On the other hand, you can change branch and source version when queue build manually or through REST API.
If the build steps are different for each branch, you need to modify the definitions independently for detail requirements.
Our company currently uses TFS for source control and build server. Most of our projects are written in C/C++, but we also have some .NET projects and wouldn't want to be limited if we need to use other languages in the future.
We'd like to use Git for our source control and we're trying to understand what would be the best choice for a build server. We have started looking into TeamCity, but there are some issues we're having trouble with which will probably be relevant regardless of our choice of build server:
Build dependencies - We'd like to be able to control the build dependencies for each <project, branch>. For example, have <MyProj, feature_branch> depend on <InfraProj1, feature_branch> and <InfraProj2, master>.
From what we’ve seen, to do that we might need to use Gradle or something similar to build our projects instead of plain MSBuild. Is this correct? Are there simpler ways of achieving this?
Local builds - Obviously we'd like to be able to build projects locally as well. This becomes somewhat of a problem when project dependencies are introduced, as we need a way to reference these resources or copy them locally for the build to succeed. How is this usually solved?
I'd appreciate any input, but a sample setup which covers these issues will also be a great help.
IMHO both issues you mention fall really in the config management category, thus, as you say, unrelated to the build server choice.
A workspace for a project build (doesn't matter if centralized or local) should really contain all necessary resources for the build.
How can you achieve that? Have a project "metadata" git repo with a "content" file containing all your project components and their dependencies (each with its own git/other repo) and their exact versions - effectively tying them together coherently (you may find it useful to store other metadata in this component down the road as well, like component specific SCM info if using a mix of SCMs across the workspace).
A workspace pull wrapper script would first pull this metadata git repo, parse the content file and then pull all the other project components and their dependencies according with the content file info. Any build in such workspace would have all the parts it needs.
When time comes to modify either the code in a project component or the version of one of the dependencies you'll need to also update this content file in the metadata git repo to reflect the update and commit it - this is how your project makes progress coherently, as a whole.
Of course, actually managing dependencies is another matter. Tons of opinions out there, some even conflicting.
I'm wondering how teamcity handles the checkout for multiple configurations. They recommend to use the Auto option where the checkout directory is shared between all configurations. Then, what happens if one configuration is currently building, and another is going to checkout? I think I really have to set different directories for these? Also if different configurations have run, their output gets overwritten by each other.
As a side-note: we have one project with three configurations, namely Continuous, Nightly and Weekly. Maybe I have to split them into different projects?! But on the other hand I think different projects are more suitable for different branches or so.
Teamcity won't ever run two builds on one agent at the same time, so you'll never have the case where two builds try to use the same directory at the same time.
But that aside, Teamcity will use a different directory for configurations with different VCS roots; it'll only share if they have the same VCS roots with the same checkout rules. But that shouldn't cause you any problems, if you set things up right.
Also if different configurations have run, their output gets overwritten by each other.
You shouldn't be relying on files being in that directory. If you need access to the files, you should publish them as artifacts, and set the build configurations that need them to have artifact dependencies on the artifacts.
As a side-note: we have one project with three configurations, namely Continuous, Nightly and Weekly. Maybe I have to split them into different projects?!
You shouldn't have to; why would you need to split them? They can run fine in the same project, but you can move them into separate ones if you prefer.
I would like to use some build parameters from Project 1 in Project 2. I know that I can make Project 1 a dependency of Project 2 and then access its build parameters as described in Dependencies Properties, but I do not want Project 1 to be built in response to a build of Project 2. For example, suppose I want Project 2 to be built nightly, while I only want Project 1 built monthly.
Is there any way Project 2 can access Project 1's build parameters under these conditions?
I would use a build configuration template that is shared between the two projects.
This means you can share properties between the projects, but also override certain ones in each individual project.
We use this for hourly builds that are not tagged and nightly ones that are tagged.
Then use a different build trigger to set one off nightly and the other monthly.
EDIT
I'll just expand slightly as a result of your comment.
In TeamCity we have 2 build configuration for the same project. One that builds on every check-in to give developers quick feedback on their contribution (build within 15 minutes). It does the following:
Builds the project in Debug
Runs all unit tests
Checks results of build into Subversion
The other configuration runs every night at midnight; it build everything and as a result takes a long time (around 45 minutes). It does the following:
Build the project in Debug and Release
Runs all unit tests
Builds Sandcastle documentaion
Checks results of build into Subversion
Grabs the Sandcastle output at an artefact so developers can easily download it.
As you pointed out this isn't as straightforward as one would like; however you can use the following to achieve it:
We use the Autoincrementer to share build numbers between the two configurations (they both increment the same build number when built).
We have a property on template that defines what artefacts to collect and is referenced from the artefacts field. The property is overridden on the second build config to define the sandcastle output to grab.
Sharing VCS Roots is mentioned on the documentation. Both our builds get the source from the same place, and tag the results to the same place. One VCS is most definitely all we need.
Bit of a long edit but I think it goes exactly on the lines of what you're trying to achieve. I appreciate I should have included this in the original answer.
HTH
Dependency is different from Build Triggering in TeamCity. If you make one project dependent on another ( artifact dependency ), it does not mean that the the latter will trigger the former.
Even when one project has been defined as dependent on another ( and also, even if not ) you have to specify explicitly the build trigger ( in this case a Finish build trigger ) for the dependent project to be triggered.