Let's say I have two "projects" within TeamCity, which are two websites, that each use a shared library that isn't within the svn path of the website. Here is the svn structure to make it more clear:
Website A: svn://root/web/websitea (uses shared library a)
Website B: svn://root/web/websiteb (uses shared library b)
Shared Library A: svn://root/shared/liba
Shared Library B: svn://root/shared/libb
How would I setup a teamcity project for website a? Right now I have it point to the svn://root but that would make it trigger a build even if website b or shared library b was changed, which is not right. What I really need is a way for it to trigger a build only if there is a change in svn://root/web/websitea OR in svn://root/shared/liba.
I tried setting up two vcs roots within the same project which point to the two svn paths above, however there doesn't seem to be a way to set a working directory for each vcs root. For this reason, it ended up just copying the contents of the two svn paths directly into the root of my build directory instead of putting them in the proper places (C:\Build\Web\WebsiteA & C:\Build\Shared\LibA).
TeamCity's checkout rules functionality is able to support the setup you're describing. My team uses it to do something similar to what you're trying to do.
First, set up a single SVN Root. The URL of this root would be something like this:
svn://root/
Then set up the following checkout rules.
For Project A:
+:web/websitea=>/web/websitea
+:shared/liba=>/shared/liba
For Project B:
+:web/websiteb=>/web/websiteb
+:shared/libb=>/shared/libb
The TeamCity documentation on checkout rules isn't totally clear about this point, but only the particular paths you have included will be used to trigger builds. This should meet your need to have only changes is websitea and liba trigger its build (and the same for B).
In addition to what Eric mentioned, if you need the entire source root to be checked out, but only trigger the build based on particular paths, you can edit the VCS Trigger rules in the Build Triggering section to have something like:
+:web/websitea
+:share/liba
Related
When a private agent build starts in VSTS, it gets assigned a directory, e.g. C:\vstsagent_work\1\s
Is there a way to set this to a different path? On other CI servers, like Jenkins, I can define a custom workspace for a job. I'm dealing with a huge monorepo and have dozens of build definitions around the same repository. It makes sense (to me anyway) to share a single directory on the build agent computer.
The benefit to me is that my builds can use pre-built components from upstream repositories, if they have already been built.
Thanks for any help
VSTS build always creates a working directory per build definition. This leaves you two options:
Create a single build definition and use conditionals on steps to skip certain steps in order to only run what is needed. This allows you to use the standard steps and may require a powershell script to figure out which steps to run and which ones to skip. Set variables from powershell using the special logging commands.
Disable the get sources step and add a step that manually fetches sources. You'll need to clean the working directory, checkout the right commit, basically replicating the actions in the get sources step manually. It may require some fidgeting to get all the behavior correctly for normal build, pull request builds etc. That way you can take full control over the location where sources are checked out.
I'd also recommend you investigate the 2017 project formats that use the new <packageReference> in the project files to fetch packages. The new system supports configuring a version range which can always fetch the latest available version of packages. It's a better long-term solution.
No, it isn’t available in VSTS build system.
You can change working directory of agent (C:\vstsagent_work) (Re-configure it and specify another working folder), but it won’t uses the same source folder for different build definitions, the folder would be 1, 2, 3 ….
I have several projects which use code from a large set of component libraries. These libraries are under source control.
The libraries repository contains all the libraries used by all my projects and contains multiple versions of multiple libraries. Each library/version pair lives in its own folder. Each of my projects identifies the specific library/version pairs it needs through the folder paths of the references in its project file.
For example $(LibraryPath)\SomeLibrary\v1.1.5
Please note that the libraries repository is only ever added to. No changes are made to stuff already in the repository. Ever.
I have been of course been able to configure my build plan to pull the libraries repository to a libraries subfolder of the working directory. So far so good. However, using the auto branch management feature of Bamboo, this setup means that the libraries repository is cloned for each and every branch in all projects.
Not funny. No, really, not funny...
What I would like to do is:
pull the libraries repository in each build plan
but pull it to a fixed location that is the same for all build plans
it doesn't have to be an absolute path
but it does need to be outside the working directory of the current build plan to avoid unnecessary duplication
Unfortunately the Checkout Directory of the Source Code Checkout configuration task in a Bamboo build plan doesn't allow me to specify either an absolute path or a relative one that goes "up" for one or more levels from the working dir. The hint text explicitly states "(Optional) Specify an alternative sub-directory to which the code will be checked out." And indeed, specifying something like ..\Library gets punished with the message "Checkout to parent directory is forbidden".
I have seen information on the "artifact sharing" feature of Bamboo. This will probably work, but it seems like overkill for what I want to achieve.
What would be the easiest and least complicated way to achieve my goal using Atlassian's Bamboo Continuous Integration?
Out-of-the-box alternatives are welcome, but please don't direct me to any products that require intimate CLI use and/or whose documentation assumes (extensive) knowledge of 'nix and/or Java setup. I am on Windows and spoiled rotten by powerful (G)UI's.
I have the same problem - with a repository weighing in at around 2GB.
I'd like to simply "git checkout myBranch" and "git clean -fxd" instead of cloning every time (which should save a lot of time and disk space). However I also like Bamboo's automatic trigger with new branches showing up.
Like the OP, I'd love to be able to put "..\SharedDirectory" in the "CheckoutDirectory" for the
"Source Code Checkout" task but it won't let me go out above the \JOB_KEY\ folder
One possible solution is: replacing the "Source Code Checkout" task with the two git commands above. That way I can specify exact when/where/how to do the checkout. I think there may be problems with the initial checkout in this case - but once that is solved, all subsequent branches would use the same shared folder, and no more pulling down 2GB every time.
I have a lot of projects in my TeamCity server, for building and testing multiple sites. We use Selenium RC to test the sites every night, and I would like to reuse one step or configuration instead of having copies in each project.
I have looked into using Templates, but they only seem to be accessible from the project it is defined in. The same goes for properties, as I tried to put paths and some shared values in system or environment properties and using copied build steps using those shared properties.
But it all fails, as none of these methods seem to be usable across projects. How do you solve this type of issue? It must be possible somehow, right?
As per my comments for your question you can use a single template configuration build across multiple Projects.
See TeamCity documention for information on how to do this.
TeamCity has a new feature (as of 8.0) that supports global configs/parameters/env variables. 8.0 supports the new concept of project organization/nesting, and as part of that feature set all projects now inhert from the "Root" project. And on that root project you can specify a number of things, including parameters, etc. This does not appear to support build steps.
They documented the change in http://youtrack.jetbrains.com/issue/TW-11202.
--
You could achieve a "global property" by creating a dummy build configuration which has your shared properties then for all your builds which need access to the global properties you create a build dependency. When your real build configuration is called it will first call the dependent build configuration (which does nothing meaningful) then it passes the properties from the dependent config (with a dep.bt prefix instead of just the system prefix).
Depending on the logic you need in a shared build step you may be able to actually perform real logic in the dependent build (it won't be able to inherit any properties or VCS details from the "real" build).
Since this is a bit outdated, and the answers above aren't completely clear, thought I would post how I did this with TeamCity 8+.
Basically, a template can be used within the project and its children. If you move a template to the parent, it becomes available to all the siblings of the original project.
Go to the template.
Click the "Move" button (bottom-right pane).
Select a parent (or root).
Note: If the template is attached to a VCS root that is bound to the current project, you'll first have to move the VCS root to the same level to which you're trying to move the template.
I'm new to TeamCity and we have a number of common projects under source control that are then referenced directly within relevant projects i.e.:
Common
branches
tags
trunk
CommonProject
CommonProject.csproj
Common.sln
ProjectX
branches
tags
trunk
ProjectX.sln
As a result, the reference to "CommonProject.csproj" in "ProjectX.sln" is something along the lines of ....\Common\trunk\CommonProject\CommonProject.csproj which is fine within our development enviroments but when it comes to TeamCity it falls over saying it can't find the path "....\Common\trunk\CommonProject\CommonProject.csproj"
What's the best way around this problem? I've tried adding CommonProject to TeamCity as a dependancy but it still doesn't seem to want to play ball...
Thanks
Tim
We address this by using Externals in Subversion which allows you to pull in stuff from a different (bit of the) repository.
Then, when we're building the solutions, we have those common projects grouped into the same folders with project specific solution - i.e. when we check stuff out we have:
Solution1
+---Project1
+---Project2
+---Project3
+---Common1
+---Common2
Then, separately:
Solution2
+---ProjectA
+---ProjectB
+---ProjectC
+---Common1
+---Common2
Because we have the externals and the directory/folder structure set up this way you should, in theory, be able to checkout (or export) a "solution" to an empty directory and have it build successfully from scratch (subject to all the necessary tools being installed) and therefore TeamCity (or whatever your continuous integration server is) should be able to also be build it from scratch. In fact even before we stared using TeamCity I had this as policy but the value is clearer once you start doing continuous integration.
The appropriate bit of the Subversion Red Book is here: Externals Definitions
Suppose I have a project "MyFramework" that has some code, which is used across quite a few solutions. Each solution has its own source control management (SVN).
MyFramework is an internal product and doesn't have a formal release schedule, and same goes for the solutions.
I'd prefer not having to build and copy the DLLs to all 12 projects, i.e. new developers should to be able to just do a svn-checkout, and get to work.
What is the best way to share MyFramework across all these solutions?
Since you mention SVN, you could use externals to "import" the framework project into the working copy of each solution that uses it. This would lead to a layout like this:
C:\Projects
MyFramework
MyFramework.csproj
<MyFramework files>
SolutionA
SolutionA.sln
ProjectA1
<ProjectA1 files>
MyFramework <-- this is a svn:externals definition to "import" MyFramework
MyFramework.csproj
<MyFramework files>
With this solution, you have the source code of MyFramework available in each solution that uses it. The advantage is, that you can change the source code of MyFramework from within each of these solutions (without having to switch to a different project).
BUT: at the same time this is also a huge disadvantage, since it makes it very easy to break MyFramwork for some solutions when modifiying it for another.
For this reason, I have recently dropped that approach and am now treating our framework projects as a completely separate solution/product (with their own release-schedule). All other solutions then include a specific version of the binaries of the framework projects.
This ensures that a change made to the framework libraries does not break any solution that is reusing a library. For each solution, I can now decide when I want to update to a newer version of the framework libraries.
That sounds like a disaster... how do you cope with developers undoing/breaking the work of others...
If I were you, I'd put MyFrameWork in a completely seperate solution. When a developer wants to develop one of the 12 projects, he opens that project solution in one IDE & opens MyFrameWork in a seperate IDE.
If you strong name your MyFramework Assemby & GAC it, and reference it in your other projects, then the "Copying DLLs" won't be an issue.
You just Build MyFrameWork (and a PostBuild event can run GacUtil to put it in the asssembly cache) and then Build your other Project.
The "best way" will depend on your environment. I worked in a TFS-based, continuous integration environment, where the nightly build deployed the binaries to a share. All the dependent projects referred to the share. When this got slow, I built some tools to permit developers to have a local copy of the shared binaries, without changing the project files.
Does work in any of the 12 solutions regularly require changes to the "framework" code?
If so your framework is probably new and just being created, so I'd just include the framework project in all of the solutions. After all, if work dictates that you have to change the framework code, it should be easy to do so.
Since changes in the framework made from one solution will affect all the other solutions, breaks will happen, and you will have to deal with them.
Once you rarely have to change the framework as you work in the solutions (this should be your goal) then I'd include a reference to a framework dll instead, and update the dll in each solution only as needed.
svn:externals will take care of this nicely if you follow a few rules.
First, it's safer if you use relative URIs (starting with a ^ character) for svn:externals definitions and put the projects in the same repository if possible. This way the definitions will remain valid even if the subversion server is moved to a new URL.
Second, make sure you follow the following hint from the SVN book. Use PEG-REVs in your svn:externals definitions to avoid random breakage and unstable tags:
You should seriously consider using
explicit revision numbers in all of
your externals definitions. Doing so
means that you get to decide when to
pull down a different snapshot of
external information, and exactly
which snapshot to pull. Besides
avoiding the surprise of getting
changes to third-party repositories
that you might not have any control
over, using explicit revision numbers
also means that as you backdate your
working copy to a previous revision,
your externals definitions will also
revert to the way they looked in that
previous revision ...
I agree with another poster - that sounds like trouble. But if you can't want to do it the "right way" I can think of two other ways to do it. We used something similar to number 1 below. (for native C++ app)
a script or batch file or other process that is run that does a get and a build of the dependency. (just once) This is built/executed only if there are no changes in the repo. You will need to know what tag/branch/version to get. You can use a bat file as a prebuild step in your project files.
Keep the binaries in the repo (not a good idea). Even in this case the dependent projects have to do a get and have to know about what version to get.
Eventually what we tried to do for our project(s) was mimic how we use and refer to 3rd party libraries.
What you can do is create a release package for the dependency that sets up a path env variable to itself. I would allow multiple versions of it to exist on the machine and then the dependent projects link/reference specific versions.
Something like
$(PROJ_A_ROOT) = c:\mystuff\libraryA
$(PROJ_A_VER_X) = %PROJ_A_ROOT%\VER_X
and then reference the version you want in the dependent solutions either by specific name, or using the version env var.
Not pretty, but it works.
A scalable solution is to do svn-external on the solution directory so that your imported projects appear parallel to your other projects. Reasons for this are given below.
Using a separate sub-directory for "imported" projects, e.g. externals, via svn-external seems like a good idea until you have non-trivial dependencies between projects. For example, suppose project A depends on project on project B, and project B on project C. If you then have a solution S with project A, you'll end up with the following directory structure:
# BAD SOLUTION #
S
+---S.sln
+---A
| \---A.csproj
\---externals
+---B <--- A's dependency
| \---B.csproj
\---externals
\---C <--- B's dependency
\---C.csproj
Using this technique, you may even end up having multiple copies of a single project in your tree. This is clearly not what you want.
Furthermore, if your projects use NuGet dependencies, they normally get loaded within packages top-level directory. This means that NuGet references of projects within externals sub-directory will be broken.
Also, if you use Git in addition to SVN, a recommended way of tracking changes is to have a separate Git repository for each project, and then a separate Git repository for the solution that uses git submodule for the projects within. If a Git submodule is not an immediate sub-directory of the parent module, then Git submodule command will make a clone that is an immediate sub-directory.
Another benefit of having all projects on the same layer is that you can then create a "super-solution", which contains projects from all of your solutions (tracked via Git or svn-external), which in turn allows you to check with a single Solution-rebuild that any change you made to a single project is consistent with all other projects.
# GOOD SOLUTION #
S
+---S.sln
+---A
| \---A.csproj
+---B <--- A's dependency
| \---B.csproj
\---C <--- B's dependency
\---C.csproj