In our Azure DevOps Server 2019 we want to trigger a build pipeline on completion of another build pipeline. The triggered build should use the same source branch the triggering build used.
According to the documentation this does not work with classic builds or the classic trigger definition, but in the YAML definition for the triggered build:
build.yaml:
# define triggering build as resource
resources:
pipelines:
- pipeline: ResourceName
source: TriggeringBuildPipelineName
trigger:
branches:
- '*'
# another ci build trigger
trigger:
branches:
include:
- '*'
paths:
include:
- SubFoldder
pool:
name: Default
When creating the pipeline like this, the trigger element under the pipeline resource gets underlined and the editor states that trigger is not expected inside a pipeline.
When saving the definition and trying to run it, it fails with this error:
/SubFolder/build.yaml (Line: 6, Col: 7): Unexpected value 'trigger'
(where "line 6" is the trigger line in the resources definition).
So my question is: How to correctly declare a trigger that starts a build pipeline on completion of another build pipeline, using the same source branch? Since the linked documentation actually explains this, the question rather is: what did I miss, why is trigger unexpected at this point?
Update: I just found this. So it seems one of the main features that they promised to have and documented as working, one of the main features we switched to DevOps for, is not even implemented, yet. :(
Updates are made every few weeks to the cloud-hosted version, Azure DevOps Services. These updates are then rolled up and made available through quarterly updates to the on-premises Azure DevOps Server and TFS.So, all features are release in Azure DevOps Service first.
A timeline of feature released and those planned in the feature release can be found at here-- Azure DevOps Feature Timeline
You could choose to directly use cloud version Azure DevOps Service instead or monitor latest update on above Feature Timeline with Azure DevOps Server. Sorry for any inconvenience.
Related
We are currently running a linter for each of our CI builds within GitHub PRs. This linter, though, requires a dbt manifest.json which we can only build when the full dbt schemas are available in the PR schema.
We currently need to manually re-trigger the action workflow after dbt Cloud Slim CI is finished, since we have the on trigger as follows:
on:
pull_request:
branches:
- main
types: [opened, reopened, ready_for_review, synchronize]
And, as mentioned, the manifest.json is not yet built at the moment when the PR is e.g. opened.
Is there any way that we can trigger the workflow only once the Slim CI job (that is triggered by dbt Cloud) is actually completed, and therefore the manifest.json already built?
I've come across on.check_run and on.status for Events that trigger workflows (see GH docs), but not sure whether these can be used since, as specified in the docs: "Note: This event will only trigger a workflow run if the workflow file is on the default branch."
I am stuck in the following Situation. We got an Azure DevOps Organisation with different Projects. The goal is to store all Pipelines and Releases in the "Operations" Project.
The benefit of this is, to keep the individual teams away from dealing with creating Pipelines or handling Secrets (.netrc, Container Registry, etc.)
However there seems to be no way to trigger a build Validation Pull Request in Project A, that triggers the Pipeline in the Operations Project. Build Validations of certain Branches can only trigger Pipelines inside the Project itself
So in short: PR in Project A should trigger Pipeline in Project "Operations"
Is there a workaround for this or is this feature to be implemented in the near future?
This threat looked promising that it might be implemented. But it has been quiet since.
I initially asked if you're using YAML pipelines because I have a similar setup designed for the same purpose, to keep YAML pipelines separate from the codebase.
I've heavily leveraged YAML templates to accomplish this. They're the key to keeping the bulk of the logic out of the source code repos. However, you do still need a very light-weight pipeline within the source code itself.
Here's how I'd recommend setting up your pipelines:
Create a "Pipelines" repo within your "Operations" project. Use this repo to store the stages of your pipelines.
Within the repos containing the source code of the projects you'll be deploying create a YAML file that will extend from a template within your "Operations/Pipelines" repo. It will look something like this:
name: CI-Example-Root-Pipeline-$(Date:yyyyMMdd-HHmmss)
resources:
repositories:
- repository: Templates
type: git
name: Operations/Pipelines
trigger:
- master
extends:
- template: deploy-web-app-1.yml#Templates
parameters:
message: "Hello World"
Create a template within your "Operations/Pipelines" repo that holds all stages for your app deployment. Here's an example:
parameters:
message: ""
stages:
- stage: output_message_stage
displayName: "Output Message Stage"
jobs:
- job: output_message_job
displayName: "Output Message Job"
pool:
vmImage: "ubuntu-latest"
steps:
- powershell: Write-Host "${{ parameters.message }}
With this configuration, you can control all of your pipelines within your Operations/Pipelines repo, outside of the individual projects that will be consuming. You can restrict access to it, so only authorized team members have the capability of creating/modifying pipelines.
Optionally, you can add an environment check that requires certain environments to inherit from your templates, which will stop deployments that have been modified to not use the pipelines you've created:
Microsoft has published a good primer on using YAML templates for security that will elaborate on some of the other strategies you can use as well:
https://learn.microsoft.com/en-us/azure/devops/pipelines/security/templates?view=azure-devops
What is the difference between Individual CI and Batched CI in Azure Pipelines ?
How does it relate to the batch option of trigger in Azure Pipelines YAML ?
trigger:
batch: 'true'
branches:
include:
- main
How does it relate to the batch option of trigger in Azure Pipelines YAML ?
As the document Push trigger stated:
trigger:
batch: boolean # batch changes if true; start a new build for every push if false (default)
Explanation:
Build batching will take multiple commits and build them all at once in one batch instead of queuing each commit as a separate build which would extend the total time to build. If you are building code in Azure Pipelines and often find yourself waiting for queued builds. You may find it useful to enable build batching.
So, we could understand the state in that document now start a new build for every push if false (default). That means that if we set the value of this batch option to false, it will start a new build for every push (commit).
This is equivalent to our Batch changes while a build is in progress option in Classic modeļ¼
I just setup CI/CD for a GitHub repo.
The CI build which validates a pull request is setup up as GitHub Action.
The CD build (which should run after the pull request was merged) is setup using Azure Pipelines as i would like to use the artifacts generated as a trigger for a Release Pipeline using Azure Pipelines as well.
The only thing that's still bugging me is, that the CD Build is also triggering automatically for a pull request and i can't figure out where i can configure those checks.
The checks currently running when a pull request is created are the following:
I want to get rid of the Continous Delivery Build here.
I tried to configure the branch protection rules but this has no effect:
On the Azure Pipeline side i completely disabled the triggers:
But this also has no visible effect to me.
I tested Disable pull request validation in the Triggers of the azure devops pipeline. On my side, it works well, and the build pipeline validation check is not displayed in the github pull request.
You can first check whether the pipeline source repo that you set the "Disable pull request validation" option corresponds to the github repo that created the pull request. Then try a few more times, it is possible that the settings are not applied immediately.
In addition, as workaround you can opt out of pull request validation entirely by specifying pr: none in yaml. Please refer to this official document.
# no PR triggers
pr: none
VSTS allows you to select which branches automatically trigger a CI build by specifying a branch pattern.
However, my unit tests are using a real database which causes a problem when more than one build triggers e.g. master and feature-123 as they will clash on the database tests.
Is there a way of specifying that only one such build should be run at at time; I don't want to go away from executing tests against a real database as there are significant differences between an in-memory database and SQL Azure.
VSTS already serialize builds which are triggered by the same CI build.
Even CI build can be triggered by multiple branches, but for a certain time, only one build is running by default (unless you use pipelines to run builds concurrently).
Such as if both master branch and feature-123 branch are pushed to remote repo at the time time, the CI build definition will trriger two builds serially (not concurrently).
If you are using pipeline and need to run the triggered builds serially, you should make sure only one agent is used for the CI builds. You can use the way below:
In your CI build definition -> Options Tab -> add demands to specify which agent you want to use for the CI build.
Assume in default agent pool, there are three agents with the agent name: default1, default2 and default3.
If you need to specify default2 agent to run the CI build, then you can add the demands as below:
Now even multiple branches have been pushed at the same time, they will be triggered one by one since only one agent is available for the CI build.
If you use a YAML pipeline, you can use a deployment job in stead of a regular job.
With a deployment job you select a named Environment you want to deploy to.
You can configure the Environment in azure devops under Pipelines->Environments and you can choose to add an Exclusive Lock.
Then only one run can use the environment at a time and this serializes your runs.
Unfortunately, if you have multiple runs waiting for the environment (because one run currently has it locked), when the environment becomes unlocked only the latest run will continue. All the others waiting for the lock will be cancelled.
If you want to do it via .yml or .yaml file you can do following
- phase: Build
queue:
name: <Agent pool name>
demands:
- agent.name -equals <agent name from agent pool>
steps:
- task: <taskname>
displayName: 'some display name'
inputs:
value: '<input variable based on type of task'
variableName: '<input variable name>'