Given a solution containing both a Web Api project, an Api Client Project, and an assembly project containing shared Dto objects as follows:
Solution
Web Api Project
Shared Dto Project
Api Client Project
Unit Tests
I have an Azure Devops build pipeline to build and test all of the projects.
Within that build pipeline, I have a step that creates 2 Nuget packages for the Api Client and the Shared Dto projects.
However, I don't need to create new packages all of the time, say I only added a unit test or fixed a bug in the Web Api project, I don't want to create new Nuget packages (or worry about bumping the Package version) if nothing in the Api Client Project (or its dependencies) has changed, but I do still want to regression test it.
I thought that perhaps I could do this by having multiple build pipelines, one for the Web Api project, and others for the Api Client & Shared projects, then use path filters to trigger the appropriate builds.
Are there any other ways to achieve this?
Whats the best practice?
I don't want to have to maintain more build definitions than I need.
I ended up using a slight modification of
this answer
# Checks which files have been updated to determine if new Nuget packages are required
$editedFiles = git diff HEAD HEAD~ --name-only
echo "$($editedFiles.Length) files modified:"
$editedFiles | ForEach-Object {
echo $_
Switch -Wildcard ($_ ) {
'Whds.Configuration.Models/*' {
# If the Models project is updated, we need to create both packages
Write-Output "##vso[task.setvariable variable=CreateModelsPackage]True"
Write-Output "##vso[task.setvariable variable=CreateClientPackage]True"
}
'Whds.Configuration.Standard/*' { Write-Output "##vso[task.setvariable
variable=CreateClientPackage]True" }
# The rest of your path filters
}
}
This script sets variables which are then referenced in custom conditions in the dotnet pack step in the build pipeline:
and(succeeded(), eq(variables['CreateModelsPackage'], 'True'))
If the Dto project is changed, both variables are set in order to create both packages.
If only the client (aka Standard) project is the only thing that has changed, the package for the Dto project will not be created.
Are there any other ways to achieve this? Whats the best practice? I don't want to have to maintain more build definitions than I need.
There are different ways to achieve it, but not sure which one is the best practice, it all depends on your requirements or tastes.
The simple method is similar to your thought. Also need to create a new build pipeline. The difference is that we do not need to maintain this build definition.
Details:
Add a new pipeline without any more task in this pipeline, and use
path filters to trigger the appropriate builds (Api Client and the
Shared Dto projects).
Add a build completion to your original Azure Devops build pipeline:
Add a custom condition for the step that creates 2 Nuget packages based on the Build.Reason, like:
and(succeeded(), eq(variables['Build.Reason'], 'BuildCompletion'))
Now, the steps to create 2 Nuget packages only executed when the file changes come from a specific project. Of course the limitation of this solution is that if you already have a build completion, it will not work.
If the above method is not what you want, we could invoke the REST API commits to get the commit info for each build:
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/commits/{commitId}?changeCount=100&api-version=5.1
We could find the changes/path in the returned body:
"changes": [
{
"item": {
"gitObjectType": "blob",
"path": "/.gitattributes",
"url": "https://dev.azure.com/fabrikam/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249/items/.gitattributes?versionType=Commit"
},
"changeType": "add"
},
{
"item": {
"gitObjectType": "blob",
"path": "/.gitignore",
"url": "https://dev.azure.com/fabrikam/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249/items/.gitignore?versionType=Commit"
},
"changeType": "add"
},
{
"item": {
"gitObjectType": "tree",
"path": "/MyWebSite",
"isFolder": true,
"url": "https://dev.azure.com/fabrikam/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249/items/MyWebSite?versionType=Commit"
},
"changeType": "add"
},
Then we could use the powershell script to traverse these paths to see if they include Api Client and the Shared Dto projects, if yes, we set a variables with a value, add condition based on this value for the steps that creates 2 Nuget packages.
Note: Before use the REST API commits, we need use the Commits - Get Commits to get the latest commit Id:
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/commits?$top=1&api-version=5.1
Hope this helps.
Related
We have an SDK project that includes a test engine. These live under two different csproj's. So we have SDK.csproj and TestEngine.csproj where SDK.csproj is a ProjectReference.
I have a DocFx project set up that builds metadata for these two separately, with docfx.json metadata that looks like this:
{
"src": [
{
"src": "../../sdk",
"files": ["csharp/SDK/**/*.cs"]
}],
"dest": "reference/SDK"
},
{
"src": [
{
"src": "../../sdk",
"files": ["csharp/TestEngine/**/*.cs"]
}],
"dest": "reference/TestEngine"
}
This way I can set up TOC's to put these documentation trees under separate tabs.
However, I cannot use a cref in TestEngine xml docs that refers to a class from SDK. I get an error like this from DocFX build:
Warning:[MetadataCommand.ExtractMetadata]Invalid cref value "!:SDK.SDKClass" found in triple-slash-comments for TestEngineClass, ignored.
I can imagine why this fails - DocFX is generating the metadata for TestEngine alone so it doesn't know about the SDK classes or how to link to them. Is there a way I can change the configuration so that I can keep these two projects separate (under separate TOC's) in the final website but still link from TestEngine to SDK classes?
I realized that using # and/or xref tags as described in the DocFx documentation do resolve to links properly in the generated web pages. So that gets me a lot of what I want. However, it's not a complete solution as other generated references do not resolve to links. For example, if a TestEngine method has a parameter of type SDK.SDKClass, the generated docs won't make a link for SDKClass where it appears on the parameter documentation. So I'm still wondering if there is another solution.
We've recently implemented Team City and I've been tasked with making it work with our existing VSTS environment. We'd ideally like to have Team City build and VSTS release. We've got the Team City generated Artifacts sitting inside VSTS however you can't trigger CI off them as per this article here; https://www.visualstudio.com/en-us/docs/build/concepts/definitions/release/artifacts
What I'd like to do is have Team City trigger the release via the VSTS REST API. I have have followed the documentation here https://www.visualstudio.com/en-us/docs/integrate/api/rm/releases but I'm having issues getting VSTS to use the Team City artifacts. No matter what I try I get the following error: "VS402962: No artifact version id is specified corresponding to artifact source 'TeamCity Build'. Specify a valid value and try again."
This is what my JSON body for the REST call looks like:
{
"definitionId": 13,
"description": "Build from TeamCity",
"artifacts": [
{
"alias": "Build Dev (TeamCity)",
"instanceReference": {
"id": 160,
"name": "160"
}
}
]
}
I have tried this with and without the name as per this question: VS402881: No artifact version is specified corresponding to artifact source 'MyBuild.' Release Management vNext REST API but I'm not having any luck.
I have found the issue. You need to remove (Team City) from the alias as it's not part of the Artifact Alias.
My intent is to store SNAPSHOT and RELEASE firmware binary images in Artifactory.
I have setup a custom repository layout where the artifact path pattern is set to -
[org]/[module]/[baseRev]-[folderItegRev]/[baseRev]-[folderItegRev](-[fileItegRev]).[ext]
Here are screenshots of the configuration -
I created a new repository based on the generic package type and chose the custom repository layout that I created.
I have uploaded release and snapshot artifacts to this repo and this is what it looks like on the web UI -
myrepo
mygroup/myartifactid
1.0.0-RELEASE
1.0.0-RELEASE.bin
2.0.0-RELEASE
2.0.0-RELEASE.bin
3.0.0-SNAPSHOT
3.0.0-SNAPSHOT-20170630161531.bin
3.0.0-SNAPSHOT-20171202161531.bin
3.0.0-SNAPSHOT-20171231161531.bin
Now if I use the REST API to search for the latest artifact based on layout as per the API published here
GET http://artifactory-server:8082/artifactory/api/search/latestVersion?g=mygroup&a=myartifactid&v=3.0.0-SNAPSHOT
I get the below response -
{
"errors": [
{
"status": 404,
"message": "Unable to find artifact versions"
}
]
}
What am I doing wrong? I want to be able to search for the latest release and snapshot versions using the REST API.
Make sure that "Folder Integration Revision RegExp" and "File Integration Revision RegExp" are defined for your custom layout.
In the case of your layout, they should be:
Folder Integration Revision RegExp: SNAPSHOT
File Integration Revision RegExp:(?:[0-9]{14})
To make sure the layout is properly configured, test it against a sample path, for example: mygroup/myartifactid/3.0.0-SNAPSHOT/3.0.0-SNAPSHOT-20170630161531.bin
If everything is configured correctly the test result should show the various parts of the layout: organization, module etc.
In addition, it is better, in terms of performance, to specify which repositories you wish to query, for example:
GET http://localhost:8081/artifactory/api/search/latestVersion?g=mygroup&a=myartifactid&v=3.0.0-SNAPSHOT&repos=myrepo
I am trying to find a list of all branches for a given project. Is it possible to get this information through Teamcity REST API? I found a different answer showing how to get a list of branches for a given build configuration:
Can you use the team city rest api to fetch plan branch names?
But this way I would have to run the query for all build configurations under given project.
However TeamCity has a concept of "active" branches on a given project. I am wondering if it is possible to fetch exactly that.
Actually, it is possible now.
Implemented for 2017.1 as an experimental features:
listing of the branches for a project (merged list of all project's build configuration's branches) via .../app/rest/projects/XXX/branches?locator=policy:XXX
additional branch node fields: "active", "lastActivity" timestamp, "builds" (with locator), available via "fields" parameter of the request
added "branches" into buildType node, available only via "fields" parameter of the request
Source: https://youtrack.jetbrains.com/issue/TW-44148#comment=27-2018515
I'm using exactly this url: http://TCSERVERADDRESS/app/rest/projects/PROJECTNAME/branches and it works good for me.
TeamCtiy REST API does not support showing active branches now. You are welcome to drop a feature request in the tracker
I'm looking for a way to attach some specific build parameter to a scheduled trigger.
The idea is that we are continuously building debug versions of our products. Our nightly build has to be a release build, though. The build configurations for most of our projects is absolutely the same. It even has a configuration parameter, already. So all I would need is a trigger which allows for specifying an override for a single build parameter. That would cut the build configurations to maintain by half.
Is there a way to achieve this?
Not right now, you can follow this issue.
The approach I use is to create a "Deploy :: Dev D1 :: Run all integration tests" build. I then create a build trigger on each integration service build.
I create a parameter called "env:OctopusEnvironment" for integration service build. Set the value to be empty. I like to use prompt and display:
select display='prompt' label='OctopusEnvironment' data_13='Production' data_12='CI' data_11='Local - Hassan' data_10='Local - Mustafa' description='OctopusEnvironment' data_02='Test T1' data_01='Dev D1' data_04='Local - Taliesin' data_03='Continuous Deployment CI 1' data_06='Local - Paulius' data_05='Local - Ravi' data_08='Local - Venkata' data_07='Local - Marko' data_09='Local - Ivan'
In each integration service build I add this powershell step:
$octopusEnvironment = ($env:OctopusEnvironment).Trim()
Write-Host "Octopus environment = '$octopusEnvironment'"
if ($octopusEnvironment.Length -lt 1) {
Write-Host "Auto detecting octopus environment"
$trigger = '%teamcity.build.triggeredBy%' -split '::'
if ($trigger.Length -gt 2){
$environment = $trigger[1].Trim()
Write-Host "##teamcity[setParameter name='env.OctopusEnvironment' value='$environment']"
}
}
So now I can run the integration test via a trigger and when I run it directly it will prompt me on which environment to run integration test against.
I was stuck with the same problem and voted for the issue mentioned by Evgeny. One solution we thought, as mentioned sergiussergius, was to add a final step in the build-steps sequence to trigger manually the next build-configuration by passing custom-build parameters using the REST API. But in this case, we are loosing the build-chain information.
Using TeamCity 9.x, trying some stuff on the REST API, I could implement a solution who makes it possible to retrieve the triggering (ancestor) build and its parameters from the triggered (child) build.
The first thing we do is getting the current build using the environment variables set by TeamCity:
https://<host>/httpAuth/app/rest/builds/number:<env.BUILD_NUMBER>,buildType:(name:<env.TEAMCITY_BUILDCONF_NAME>,project:<env.TEAMCITY_PROJECT_NAME>)
In the response from the REST API, we have a /build/triggered tag which contains information about the trigger. It looks like this
<triggered type="unknown" details="##triggeredByBuildType='<triggering-build-configuration-internalId>' triggeredByBuild='<triggering-build-number>'" date="20160105T190642+0700"/>
The looks like btxxx for us.
From it, we can access the triggering-build (ancestor) using the following request to the REST API:
https://<host>/httpAuth/app/rest/builds/number:<triggering-build-number>'4,buildType:(internalId:<triggering-build-configuration-internalId>1,project:name:<env.TEAMCITY_PROJECT_NAME>)
from the response, we can get the ancestor-build's parameters values, and set it in the current build using:
echo "##teamcity[setParameter name='env.ENV_AAA' value='aaaaaaaaaa']")
Notes:
this post reference TeamCity version 7.X. I did this using TeamCity version 9.X, and could not try it with a previous version. I don't know if the REST API calls mentioned in my post are similar in the previous versions.
In this solution, the ancestor's build-configuration (the one who trigger the build) and the child's build-configuration (the one triggered) are in the same project. I did not do the test using build-configurations in 2 different projects: I would expect the "trigger" tag to provide information about the ancestor's project. It would be nice if someone could do the test.
I hope this solution may help!
This is not a general solution, but in certain cases (for example if you want to determine whether the build was started by a schedule trigger or some other method), a workaround is to examine the predefined parameter teamcity.build.triggeredBy.
This parameter is set to the same string that is shown on the build's overview page next to the label "Triggered by:". For example, "Schedule Trigger", "Git", or a user's full name. (There is also a teamcity.build.triggeredBy.username parameter, but it is only set in the latter case).
The limitation of this approach is that you cannot, for example, distinguish between two separate schedule triggers defined for the same build configuration. But in that case you could resort to examining the current time as well.
I added request to last build step
curl -i -u "%login%:%pass%" -H "Content-type: text/plain" -X PUT -d "v1" http://tc.server/httpAuth/app/rest/buildTypes/id:%buildConfigurationId%/parameters/env.%SOME_PARAMETER%
http://confluence.jetbrains.com/display/TCD8/REST+API