I have a project called SPI with 15 configs. Those configs have dependency on a build config of project A.
I have been asked to run those same 15 SPI configs for project B and C. A, B and C are not dependent on each other. One way to do it is replicate the SPi project for B and C. But maintenance will be a nightmare. DO you have any suggestions on how to address this situation? DO you think I should use meta-runner? It will reduce the maintenance but not remove it. Any input is highly appreciated.
Which version of Teamcity are you using? Teamcity 8 has Build Configuration templates. I'm not sure when they were first introduced but if you have access to them, I'd recommend using them.
You can create build configuration templates from existing ones. So in your case, you can ask to create a build configuration template from each of the configs you have in project A. You can then modify those templates to make them generic and keep the customization specific to project A in project A itself. Then, base projects B and C's build configurations on the templates and simply customize the parts you need to in both those projects.
Here's more information: http://confluence.jetbrains.com/display/TCD8/Build+Configuration+Template
Build configuration templates is how we have set up our build system to build multiple branches of the same source code. When something must change, the changes are made to the templates which means they are automatically inherited by all projects that leverage said templates.
Related
We have a directory structure like so
java
build/build.gradle (This does NOT exist yet, but we want this)
servers
server1/build.gradle
server2/build.gradle
libraries
lib1/build.gradle
lib2/build.gradle
We have 11 servers and 14 libraries with varying uses of dependencies. EACH server is a composite build ONLY depending on libraries (we don’t allow servers to depend on each other). In this way, as our mono-repo grows, opening up server1 does NOT get slower and slower as more and more gradle code is added(ie. gradle only loads server1 and all it’s libraries and none of the other libraries OR servers are loaded keeping things FAST).
Ok, so one problem we are running into is duplication now which is why we need build/build.gradle file AND we want EVERY module in our mono repo to include that somehow for a few goals(each goal may need a different solution)
GOAL 1: To have an ext { … } section containing a Map of Strings to gradle dependencies much like so
deps = [
'web-webserver': "org.webpieces:http-webserver:${webpiecesVersion}",
'web-webserver-test': "org.webpieces:http-webserver-test:${webpiecesVersion}",
'web-devrouter': "org.webpieces:http-router-dev:${webpiecesVersion}"
]
In this way, we want ALL our projects to them import dependencies like so
compile deps['web-webserver']
GOAL 2: We want to 'include' a standard list of plugins so we are versioning all gradle plugins the same across the repo. While the above configures all jars to avoid jar hell in a mono-repo, we would like to do the same with just this section
plugins {
id 'com.github.sherter.google-java-format' version '0.9'
}
Of course, it each project may also want to add a few more plugins OR even not depend on this section(in case of an emergency and trying to just get the job done).
GOAL 3: We want checkstyle configuration (or any plugin config) to be defined the SAME for all projects (eventually!!!). We would like the checkstyle gradle to live in a common area but have all libraries somehow pull it in. Again, it would be nice for it to be optional in that, I can pull the gradle section into my build.gradle OR can create a new one in case of emergencies so I don't have to fix all projects in the monorepo right away.
IDEALLY, perhaps I kind of want configuration injection where when I run server1/build.gradle, it actually runs java/build/build.grade as it’s parent somehow but with overrides (IF I declare 'extends xxx.gradle' maybe) then all libraries it uses also use java/build/build.gradle as their parent. I am not sure this is possible or feasible. I am pretty sure 'extends xxx' doesn't exist in gradle.
Are any of these GOALS possible?
thanks,
Dean
I have been working on a monorepo with the exact same requirement as you, using gradle composite builds as well. The way we have solved this problem is by using pre compiled plugins
You need to do a new gradle project with only the code you want to share. This will create a plugin, that you can just add as a composite build and apply to the other projects.
I'm a bit confused by why you don't just use a "standard" gradle top level build file and compose the others as subprojects.
This solves all 3 of your goals
If you are concerned by build speed, you can target each server individually simply by running
./gradlew :server1:build
But if you are not able to do this for some reason you can use the apply from: syntax as described here
I'm trying to ensure that my entire build configuration in Teamcity is under configuration management.
But, I'm also sharing a build template across different projects (with different VCS roots). The template is used in different sub-projects in Teamcity, where the VCS root is specified.
So, when I enable Versioned Settings for the project, only the specific settings are added, not the contents of the template. And as the template are defined in the root project, I can't enable Versioned Settings at this level.
Any suggestions?
BR
Rasmus
Create or move VCS-roots in Root Project or parent project,
if you change anything in above vcs-roots you can apply it to one or all configuration, there is an option in the last section of vcs-root setting looks like
I have C# project in TFS + TeamCity (TC). Now i have tag 1.0, trunk 2.0 and TC build config for 2.0.
I need to enable build for 1.0 from tag (and for future tag 2.0, 3.0, etc..).
I've created common TC project MyProject with all necessary configurations. Now i added subproject '1.0 release brunch'. And i'm goning to copy all configurations from root project + change VCS roots of that configuration. When i'll have to make tag 2.0 i'll create subproject '2.0 release brunch' and copy all root configurations to this subproject and change VCS root.
Is this normal steps to support previous projects? Should i use configuration templates or something? Each project contains 5+ configurations to build CI and Full Release builds with all changes, E2E testing, etc. So this is not just copy configuration to the subproject and change VCS root. This seems will be a tuning for every project. And i'd like to automate this as much as possible.
One way of handling this situation is to create a single build configuration and use a parameter in the VCS trunk. So instead of using a VCS root of
https://server/svn/MyProject/trunk/
you would instead use
https://server/svn/MyProject/%TagPath%/
Then create a configuration parameter in the TeamCity build configuration called TagPath and simply set it's value to trunk or tags/1.0 or whatever.
The one caveat to be aware of no matter how you do this is that you will have problems if your build process changes in the future and in that case you would probably end up creating separate "old" and "new" build configurations.
You can use Build configuration template for this. You can create the template at the root project level and then use it for sub projects.
Look at the "Redefining settings inherited from template" section in the link below on how to use parameters to change the values at the sub-project level.
http://confluence.jetbrains.com/display/TCD8/Build+Configuration+Template
I'm using teamcity to deploy our web application to several servers using msbuild and webdeploy.
I have a configuration per customer / environment combination - for example:
Internal-Dev
Customer1-Test
Customer1-Pilot
Customer1-Live
Customer2-Test
Customer2-Pilot
Customer2-Live
I only need these configurations so that the deployment process can use web.config transformations to set the appropriate connection strings and other settings for the installation.
The problem that I have is that I have a shared web server controls project which really only needs two configurations - debug and release - but I need to copy all the above configurations into this project in order to get them to work. If I delete Customer2-Pilot from the shared project, I can't deploy to customer 2's pilot server because it is looking for the Customer2-Pilot configuration for the shared project which doesn't exist.
Currently, I copy the application's configuration names into the shared project but that is impractical as we have several applications each going to different customers so the shared project could get dozens of configurations which are all copies of Release or Debug
I would like to be able to configure the build system so that I can say that the Internal-Dev configuration of the web application project uses the shared project's Debug configuration and all the other configurations use the shared project's Release configuration. This is possible from within the IDE, using the configuration manager which can tie a solution configuration to individual project configurations but I'm not sure how to do this from within teamcity.
My current teamcity configuration is using the MSBuild runner with the web application's project file as the build file path and the configuration parameters set to deploy it to the relevant server with the correct configuration after it has been built. The shared project is in a sub-repository of the web application repository so it is fetched and built automatically.
Is there any way that I can change this set up to get what I want? Do I need to change the build file path to be the solution instead of the project so that I can use the configuration manager settings to configure it?
It turns out that this is as simple as changing the build file path parameter to be the solution file instead of the project file. This means that you specify a solution configuration instead of a project configuration. Solution configurations are managed in the configuration manager and you can specify the configuration of each project in the solution configuration so it all works nicely.
I have two projects that need to be published (a web application and a WCF service application) and they depend upon five other class library projects. I have it set up so that the class library projects are built using either the Debug or Release configurations and the two projects that have config files that need to be transformed have one configuration per system.
One thing of note. In the old set up I had to have two build steps - one to build and deploy the web application and one to build and deploy the WCF service application. When I use the solution file as the build file, I don't need two steps any more as they are both part of the solution and so they both get built and deployed.
Suppose that a large project is split into multiple projects, each housed in an individual Mercurial repository (as per What's a good way to organize projects with shared dependencies in Mercurial?).
Suppose also, that a dependency manager is being used internally (we're using NuGet, but the same could apply to Maven) so that:
ProjectA depends on Ninject and MongoDB
ProjectB depends on ProjectA, and log4net
Projects A and B can be built independently; NuGet automatically downloads both OSS and internal dependencies from a NuGet server (ProGet in this case).
Suppose finally, that ProjectB depends on v1.2.3.4-SNAPSHOT of ProjectA, and that a CI server continually updates the ProjectA.1.2.3.4-SNAPSHOT package in the NuGet server. Thereby ProjectB will always be developed against the latest checked in changes of ProjectA.
What if related changes are required in both Project A and B? What neat and clever ways are there to do this properly? Some ideas:
Developer checks out Project A and B. Changes are made to A, built, and checked in. Developer waits for CI server to build and update the NuGet server. Changes are made to B, built, and checked in. (I dislike this as code is being checked in as part of development process.)
Developer checks out Project A and B, and rewires B to use A source as a dependency (instead of NuGet package ProjectA). Changes are done to both A and B. Check in is performed for both A and B together after proper testing, but developer must ensure dependency changes are not checked in.
I'm not particularly good at this, so I think that someone will blow my ideas out of the water with something quite clever.
I don't know about how NuGet does it, but with Maven, your second idea works fine, apart from 'rewires B to use A source as a dependency' being unnecessary. You would just build A locally (using install) and it would be installed to the your local Maven repo. Then when building B, it will pick up the newly built A, rather than the one from the central repo.
I could think of the following with nuget :
In Project A drop the packages at a central location (in this example I am placing it in c:\localpackages)
Build A with
MSBUILD.exe /t:Build,Package A.csproj
With Project B you could add a .nuget\nuget.config that specifies
(you can read more about specifying package folder location here http://docs.nuget.org/docs/release-notes/nuget-2.1)
This should pick the changes made by project A. It might become tricky when you have different versions of nuget package dropped for A
Hope this helps.