I want to be able to reuse the deployment variable on my bitbucket pipeline but this is not allowed. Is there any way to combine the steps?
pipelines:
default:
- step: *build
branches:
master:
- step: *build
- step:
<<: *deploy1
name: Deployment 1
deployment: Production
- step:
<<: *deploy2
name: Deployment 2
deployment: Production
I see that bitbucket has this on the roadmap for bb pipelines. I'm currently trying to do something similar. If I find a solution or something works for me. I'll post my solution.
Related
I have a pipeline which I want to trigger when PR is merged into master. I have tried different things, but this did not work. Furthermore, I am neither getting any error nor pipeline is triggering.
I do not want this to be triggered on PR creation. I want this to be triggered when PR is merged into master. That is the reason I have not added pr in my yml.
What am I missing here?
Approaches:
Enabled "Continuous Integration" option
Following trigger syntax per Microsoft recommendation
trigger:
batch: True
branches:
include:
- master
paths:
include:
- cosmos
Setting up valid YML file
Pipeline:
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
batch: True
branches:
include:
- master
paths:
include:
- cosmos
stages:
- stage: SOME_PATH_dev
displayName: SOME_PATH_dev
jobs:
- deployment: 'DeployToDev'
environment: Dev
cancelTimeoutInMinutes: 1
pool:
vmImage: ubuntu-latest
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: AzureResourceGroupDeployment#2
displayName: Azure Deployment:Create Or Update Resource Group action on SOME_PATH_dev
inputs:
ConnectedServiceName: SOME_KEY
resourceGroupName: SOME_PATH_dev
location: West US
csmFile: cosmos/deploy.json
csmParametersFile: cosmos/parameters-dev.json
deploymentName: SOME_PATH.Cosmos.DEV
Repo Structure:
References:
"Configuring the trigger failed, edit and save the pipeline again" with no noticeable error and no further details
Azure Devops build pipeline: CI triggers not working on PR merge to master when PR set to none
https://medium.com/#aksharsri/add-approval-gates-in-azure-devops-yaml-based-pipelines-a06d5b16b7f4
https://erwinstaal.nl/posts/manual-approval-in-an-azure-devops-yaml-pipeline/
I was changing YML (pipeline) files instead of files inside cosmos folder and expecting trigger.
I have self-hosted agents on multiple environments that I am trying to run the same build/deploy processes on. I would like to be able to deploy the same code from a single repo to multiple systems concurrently. Thus, I have created an "overhead" pipeline, and several "processes" pipeline templates. Everything seems to be going very well, except for when I try to perform checkouts of the same repo twice in the same pipeline execution. I get the following error:
An error occurred while loading the YAML build pipeline. An item with the same key has already been added.
I would really like to be able to just click ONE button to trigger a main pipeline that calls all the templates requires and gives the parameters needed to get all my jobs done at once. I could of course define this "overhead" pipeline and then queue up as many instances as I need of it per systems that I need to deploy to, but I'm lazy, hence why I'm using pipelines!
As soon as I remove the checkout from Common.yml, the validation succeeds without any issues. If I keep the checkout in there but only call the Common.yml once for the entire Overhead pipeline, then it succeeds without any issues as well. But the problem is: I need to pull the contents of the repo to EACH of my agents that are running on completely separate environments that are in no way ever able to talk to each other (can't pull the information to one agent and have it do some sort of a "copy" to all the other agent locations.....).
Any assistance is very much welcomed, thank you!
The following is my "overhead" pipeline:
# azure-pipelines.yml
trigger:
none
parameters:
- name: vLAN
type: string
default: 851
values:
- 851
- 1105
stages:
- stage: vLAN851
condition: eq('${{ parameters.vLAN }}', '851')
pool:
name: xxxxx
demands:
- vLAN -equals 851
jobs:
- job: Common_851
steps:
- template: Procedures/Common.yml
- job: Export_851
dependsOn: Common_851
steps:
- template: Procedures/Export.yml
parameters:
Server: ABTS-01
- stage: vLAN1105
condition: eq('${{ parameters.vLAN }}', '1105')
pool:
name: xxxxx
demands:
- vLAN -equals 1105
jobs:
- job: Common_1105
steps:
- template: Procedures/Common.yml
- job: Export_1105
dependsOn: Common_1105
steps:
- template: Procedures/Export.yml
parameters:
Server: OTS-01
And here is the "Procedures/Common.yml":
steps:
- checkout: git://xxxxx/yyyyy#$(Build.SourceBranchName)
clean: true
enabled: true
timeoutInMinutes: 1
- task: UsePythonVersion#0
enabled: true
timeoutInMinutes: 1
displayName: Select correct version of Python
inputs:
versionSpec: '3.8'
addToPath: true
architecture: 'x64'
- task: CmdLine#2
enabled: true
timeoutInMinutes: 5
displayName: Ensure Python Requirements Installed
inputs:
script: |
python -m pip install GitPython
And here is the "Procedures/Export.yml":
parameters:
- name: Server
type: string
steps:
- task: PythonScript#0
enabled: true
timeoutInMinutes: 3
displayName: xxxxx
inputs:
arguments: --name "xxxxx" --mode True --Server ${{ parameters.Server }}
scriptSource: 'filePath'
scriptPath: 'xxxxx/main.py'
I managed to make checkout work with variable branch names by using template expression variables ${{ ... }} instead of macro syntax $(...) variables.
The difference is that, template expressions are processed at compile time while macros are processed at runtime.
So in my case I have something like:
- checkout: git://xxx/yyy#${{ variables.BRANCH_NAME }}
For more information about variables syntax :
Understand variable syntax
I couldn't get it to work with expressions but I was able to get it to work using repository resources following the documentation at: https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops
resources:
repositories:
- repository: MyGitHubRepo # The name used to reference this repository in the checkout step
type: git
name: MyAzureProjectName/MyGitRepo
ref: $(Build.SourceBranch)
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
#some job
steps:
- checkout: MyGitHubRepo
#some other job
steps:
- checkout: MyGitHubRepo
- script: dir $(Build.SourcesDirectory)
I have a base template that accepts a stageList parameter. I don't do anything with the jobs in those stages:
parameters:
- name: stages
type: stageList
default: []
stages:
- ${{ parameters.stages }}
I'm passing into that a stage that contains a deployment job. I have hardcoded the environment for testing purposes, but even so it inserts the key "name: environment" under environment:
resources:
repositories:
- repository: templates
type: git
name: basePipelineTemplatesHost/basePipelineTemplatesHost
extends:
template: templateExtendedByDeployment/template.yml#templates
parameters:
stages:
- stage: buildStage1
jobs:
- deployment:
displayName: Deploy to demo environment
environment: DTL-Demo-Env
strategy:
runOnce:
deploy:
steps:
- script: echo test
Resulting in the following rendered yaml:
environment: {
name: DTL-Demo-Env
}
This causes the job to run on a hosted vm instead of my on-prem environment agent. Is this a bug?
Just a suggestion, you should add resourceType under environment.
jobs:
- deployment:
displayName: Deploy to demo environment
environment:
name: DTL-Demo-Env
resourceType: VirtualMachine
strategy:
runOnce:
deploy:
steps:
- script: echo test
If not, the new created environment will always be created under hosted agent when you use your private agent. You should add it to let the environment variable under your private agent.
Currently have a config.yml file with the following workflow jobs:
- build-test-staging:
name: COM Staging Build
filters:
branches:
only: /^release-.*/
context: COM Deploy Settings
- deploy-staging:
name: COM Staging Deploy
requires:
- COM Staging Build
filters:
branches:
only: /^release-.*/
context: COM Deploy Settings
- build-test-staging:
name: UK Staging Build
filters:
branches:
only: /^release-.*/
context: UK Deploy Settings
- deploy-staging:
name: UK Staging Deploy
requires:
- UK Staging Build
filters:
branches:
only: /^release-.*/
context: UK Deploy Settings
There will be more of these, as well as a production version that has the same setup, but with different names.
As you can see, they all follow the same pattern: a name, a branch to run on (release for staging, master for production), and a context to bring in some env variables.
Without constant copy and pasting, is there a way to generate these jobs dynamically with something? I'm not too good at yml.
I have a .gitlab-ci.yml file which has a couple of sections for deploying to staging servers - in our small dev team of two, we each have a staging server to test on. The portion of the file looks like this:
...
.deploy: &deploy
image: docker:stable
stage: deploy
script:
- ./deploy.sh
deploy_to_staging_sf:
<<: *deploy
only:
- staging_sf
tags:
- staging_sf
deploy_to_staging_ay:
<<: *deploy
only:
- staging_ay
tags:
- staging_ay
...
The different sections are there to match the different tags for gitlab's CI runners.
If we were to add another developer (or another platform; I was testing deployment to a Raspberry Pi once), I would need to replicate another deploy_to_... section.
I'm just wondering if there's a Gitlab or YAML way to refactor this and make it generic enough so I can add another deployment platform without modifying the file.
Thanks