I wanted to add a custom condition in yaml for the below code. Both the tasks are in a single job, I wanted to run the task 2 only if the first task fails in yaml. if the task 1 passes to skip task2
- script: |
grype sbom:./xxx.xxxx.json --only-fixed -o table --fail-on high
enabled: true
condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
displayName: 'Run Task1'
continueOnError: true
- task: AzureCLI#2
displayName: Run Task 2
condition: failed()
inputs:
azureSubscription: ${{ parameters.ServiceConnection }}
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
$imageTag = az acr repository show-tags -n $(ACRServerName) --repository kubealert --output tsv | Where {$_ -eq "${{ parameters.imageTag }}"}
write-host 'Image '$imageTag
if($imageTag)
{
write-host 'Deleting image '$imageTag
az acr repository delete --name $(ACRServerName) --image kubealert:${{ parameters.imageTag }} --yes
}
Related
I am trying to increment the version number for my builds and then update the pom.xml file with the new version. My Powershell to edit and save the pom.xml does not seem to work, I don't get it to reach the xml.project.version i.e. < version > tag and make changes to it.
Do you have any suggestion how to get it to find the < version > tag and save the updated document?
For additional info, the DevOps pipeline runs in windows-latest at the moment.
trigger:
branches:
include:
- localdev
variables:
- name: azureSubscription
value: 'xxxx'
- name: webAppName
value: 'xxx'
- name: environmentName
value: 'xxx'
- name: vmImageName
value: 'ubuntu-latest'
- name: version.MajorMinor
value: '1.0'
- name: version.Patch
value: $[counter(variables['version.MajorMinor'], 0)]
- name: stableVersionNumber
value: '$(version.MajorMinor).$(version.Patch)'
- name: prereleaseVersionNumber
value: 'Set dynamically below in a task'
- name: versionNumber
value: 'Set dynamically below in a task'
- name: isMainBranch
value: $[eq(variables['Build.SourceBranch'], 'refs/heads/main')]
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Fixversionnumber
displayName: Fix Version number
pool:
vmImage: $(vmImageName)
steps:
- task: PowerShell#2
displayName: Set the prereleaseVersionNumber variable value
inputs:
targetType: 'inline'
script: |
[string] $prereleaseVersionNumber = "$(stableVersionNumber)"
Write-Host "Setting the prerelease version number variable to '$prereleaseVersionNumber'."
Write-Host "##vso[task.setvariable variable=prereleaseVersionNumber]$prereleaseVersionNumber"
- task: PowerShell#2
displayName: Set the versionNumber to the stable or prerelease version number based on if the 'main' branch is being built or not
inputs:
targetType: 'inline'
script: |
[bool] $isMainBranch = $$(isMainBranch)
[string] $versionNumber = "$(prereleaseVersionNumber)"
if ($isMainBranch)
{
$versionNumber = "$(stableVersionNumber)"
}
Write-Host "Setting the version number to use to '$versionNumber'."
Write-Host "##vso[task.setvariable variable=versionNumber]$versionNumber"
- task: PowerShell#2
displayName: Set the name of the build (i.e. the Build.BuildNumber)
inputs:
targetType: 'inline'
script: |
[string] $buildName = "$(versionNumber)_$(Build.SourceBranchName)"
Write-Host "Setting the name of the build to '$buildName'."
Write-Host "##vso[build.updatebuildnumber]$buildName"
- task: PowerShell#2
displayName: Set the name of the build (i.e. the Build.BuildNumber)
inputs:
targetType: 'inline'
script: |
#Get version
$versionNum = "$(versionNumber)"
# Specify the file path
#$xmlFileName= "pom.xml"
# Read the existing file
[xml]$xml = Get-Content "pom.xml"
#[xml]$xmlDoc = Get-Content pom.xml
# If it was one specific element you can just do like so:
$xml.project.version = "$versionNum"
#Remove the old pom.xml
#Remove-Item $xmlFileName
# Then you can save that back to the xml file
$xml.Save("$(System.DefaultWorkingDirectory)\pom.xml")
# Print new file content
Write-Host "#########################################'$versionNum'.#########################################"
Write-Host "Setting the version number to use to '$versionNum'."
Write-Host "######################################### '$versionNum'.#########################################"
gc $(System.DefaultWorkingDirectory)\pom.xml
- task: Maven#3
displayName: 'Maven Package'
inputs:
mavenPomFile: 'pom.xml'
- task: CopyFiles#2
displayName: 'Copy Files to artifact staging directory'
inputs:
SourceFolder: '$(System.DefaultWorkingDirectory)'
Contents: '**/target/*.?(war|jar)'
TargetFolder: $(Build.ArtifactStagingDirectory)
- task: buildDropFile
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
artifactName: drop
You can use a third party extension Replace Tokens. It can search files to find what to replace by a parameter tokenPattern and replaces values with variables in the pipeline. Click "Git if free" and download it to your organization.
In your pom.xml file, please replace your version number to a unique value, like #{version}#.
<version>#{versionNumber}#</version>
Please notice the #{...}#, this will help task to find which content to replace. The versionNumber should be the variable that you want to replace.
Then, in your pipeline, search and add a replace tokens task. Here is an example:
steps:
- task: replacetokens#5
inputs:
targetFiles: '**/pom.xml'
encoding: 'auto'
tokenPattern: 'default' # The defult token pattern is #{...}#
writeBOM: true
actionOnMissing: 'continue'
keepToken: false
actionOnNoFiles: 'continue'
enableTransforms: false
enableRecursion: false
useLegacyPattern: false
enableTelemetry: true
How can i read json key-value using inline script in git actions and pass them outside the bash tas a variable, below is the file to create a resource group and parse the json from the repo and pass the vaues as variable to the parameter
on: [push]
name: sp_keyvault
jobs:
build-and-deploy:
runs-on: ubuntu-latest
env:
ResourceGroupName: sp_keyvault-rg
ResourceGroupLocation: "eastus"
steps:
- uses: actions/checkout#master
- uses: azure/login#v1
with:
creds: ${{ secrets.Azure_cred }}
- uses: Azure/CLI#v1
with:
inlineScript: |
#!/bin/bash
if $(az group exists --name ${{ env.ResourceGroupName }}) ; then
echo "Azure resource group already exists, skipping creation..."
else
az group create --name ${{ env.ResourceGroupName }} --location ${{ env.ResourceGroupLocation }}
echo "Azure resource group created"
fi
inlineScript: |
#!/bin/bash
#need to access json from the repo and pass it to parameter variable
- uses: azure/arm-deploy#v1
with:
resourceGroupName: ${{ env.ResourceGroupName }}
template: ./sp_repo-main/KeyVaultSetup.json
parameters: "need variable from bash script"
Is there a better/cleaner/more idiomatic way to exit a .yaml-based Azure Pipeline than say having a script throw an error?
e.g., this works but it feels clunky:
- task: PowerShell#2
displayName: "Exit"
inputs:
targetType: 'inline'
script: |
throw 'Exiting';
- powershell: |
write-host "##vso[task.complete result=Failed;]The reason you quit"
Would be neater, but would still fail the job.
There is no equivalent to skip the rest of the job, unless you work with conditions to skip all future tasks based on a variable value:
variables:
skiprest: false
- powershell: |
write-host "##vso[task.setvariable variable=skiprest]true"
- powershell:
condition: and(succeeded(), eq(skiprest, 'false'))
- powershell:
condition: and(succeeded(), eq(skiprest, 'false'))
- powershell:
condition: and(succeeded(), eq(skiprest, 'false'))
- powershell:
condition: and(succeeded(), eq(skiprest, 'false'))
You could use a YAML iterative insertion from a template to apply that condition to all tasks in a job I suppose. I don't have a working sample at hand, but the docs show how to inject a dependsOn:, the trick would be very similar I suppose:
# job.yml
parameters:
- name: 'jobs'
type: jobList
default: []
jobs:
- job: SomeSpecialTool # Run your special tool in its own job first
steps:
- task: RunSpecialTool#1
- ${{ each job in parameters.jobs }}: # Then do each job
- ${{ each pair in job }}: # Insert all properties other than "dependsOn"
${{ if ne(pair.key, 'dependsOn') }}:
${{ pair.key }}: ${{ pair.value }}
dependsOn: # Inject dependency
- SomeSpecialTool
- ${{ if job.dependsOn }}:
- ${{ job.dependsOn }}
I have a few places, where I need to define a set of K8s secrets during deployment at various stages, so I want to extract the recurring script into a template:
parameters:
- name: secretName
type: string
default: ""
- name: secrets
type: object
default:
Foo: Bar
steps:
- task: Bash#3
displayName: Create generic secret ${{ parameters.secretName }}
inputs:
targetType: inline
script: |
echo "Creating generic secret ${{ parameters.secretName }}"
microk8s kubectl delete secret ${{ parameters.secretName }}
microk8s kubectl create secret generic ${{ parameters.secretName }} ${{ each secret in parameters.secrets }}: --from-literal=${{ secretKey }}="${{ secret.value }}"
I want to call it like this multiple times, to create all neccessary secrets for the deployment to each stage
- job: CreateSecrets
pool:
name: $(poolName)
steps:
- template: "Templates/template-create-secret.yml"
parameters:
secretName: "testSecret"
secrets:
username: $(staging-user)
password: $(staging-password)
foo: $(bar)
And it should simply execute a scriptn similar to this one:
kubectl create secret generic secretName \
-- from-literal=username=user1 \
-- from-literal=password=pass1 \
-- ...etc
With my current approach I am receiving the error:
/Code/BuildScripts/Templates/template-create-secret.yml (Line: 18,
Col: 15): The directive 'each' is not allowed in this context.
Directives are not supported for expressions that are embedded within
a string. Directives are only supported when the entire value is an
expression.
How is it possible to iterate over a parameter of type object and use its key and value to build a string for bash? The alternative would be to simply use a single key-value-pair per secret and create multiple secrets, which I'd like to avoid
If you concatenate the command arguments into a variable you can use that in a future step/ task. This example will concatenate all secrets within the key vaults indicated in the keyVaultSecretSources parameter into one command. It shouldn't be too hard to adjust so you can specify which secrets you'd like to include/ exclude:
- name: environment
type: string
- name: namespace
type: string
- name: releaseName
type: string
# contains an array of Azure key vault names
- name: keyVaultSecretSources
type: object
stages:
- stage: MountSecrets${{ parameters.releaseName }}
pool: [Your k8s Pool]
displayName: Mount Key Vault Secrets ${{ parameters.releaseName }}
# Then key vault arguments will be concatenated into the stage variable secretArgs
variables:
secretArgs: ""
jobs:
- deployment: [Your Job Deployment Name]
displayName: [Your Job Display Name]
strategy:
runOnce:
deploy:
steps:
# skip artifacts download for this stage
- download: none
- ${{ each keyVault in parameters.keyVaultSecretSources }}:
# 1. obtain all secrets from keyVault.name key vault
# 2. remove all Json formatting, left with each secret name on one line
# 3. assign to local variable secretNameArray as an array
# 4. loop through secretNameArray and assign the secret to local variable kvs
# 5. construct the argument --from-literal and append to local variable mountCommand
# 6. append mountCommand to the stage variable secretArgs
- task: AzureCLI#2
displayName: 'Concatenate Keyvault Secrets'
inputs:
azureSubscription: [Your subscription]
scriptType: 'bash'
failOnStandardError: true
scriptLocation: 'inlineScript'
inlineScript: |
secretNameArray=($(az keyvault secret list --vault-name ${{ keyVault.name }} --query "[].name" | tr -d '[:space:][]"' | sed -r 's/,+/ /g'));
for i in "${secretNameArray[#]}"; do kvs="$(az keyvault secret show --vault-name ${{ keyVault.name }} --name "$i" --query "value" -o tsv)"; mountCommand="$mountCommand --from-literal=$(echo -e "$i" | sed -r 's/-/_/g')='$kvs'"; done;
echo "##vso[task.setvariable variable=secretArgs;issecret=true]$(secretArgs)$mountCommand"
- task: Kubernetes#1
displayName: 'Kubectl Login'
inputs:
kubernetesServiceEndpoint: [Your Service Connection Name]
command: login
namespace: ${{ parameters.namespace }}
- task: AzureCLI#2
displayName: 'Delete Secrets'
inputs:
azureSubscription: [Your subscription]
scriptType: 'bash'
failOnStandardError: false
scriptLocation: 'inlineScript'
inlineScript: |
kubectl delete secret ${{ parameters.releaseName }}-keyvault -n '${{ parameters.namespace}}'
exit 0
- task: AzureCLI#2
displayName: 'Mount Secrets'
inputs:
azureSubscription: [Your subscription]
scriptType: 'bash'
failOnStandardError: false
scriptLocation: 'inlineScript'
inlineScript: |
kubectl create secret generic ${{ parameters.releaseName }}-keyvault$(secretArgs) -n '${{ parameters.namespace}}'
exit 0
- task: Kubernetes#1
displayName: 'Kubectl Logout'
inputs:
command: logout
According the doc: Parameter data types, we can know that we need to use the 'each' key words before the script and here is the doc to help you know more about the Runtime parameters. Here is the demo script:
parameters:
- name: listOfStrings
type: object
default:
- one
- two
steps:
- ${{ each value in parameters.listOfStrings }}:
- script: echo ${{ value }}
Here is the docker stage:
- template: ../docker-template.yml
parameters:
buildArgs:
- name: Arg1
value: $(arg1-value)
- name: Arg2
value: $(arg2-value)
docker-template.yml
parameters:
buildArgs: []
stages:
- stage: Docker
displayName: Docker Stage
jobs:
- job: Docker
steps:
# - ${{ each arg in parameters.buildArgs }}:
# - bash: |
# buildArgString+=' --build-arg ${{arg.name}}=${{arg.value}}'
# displayName: "Getting Params"
- bash: |
${{ each arg in parameters.buildArgs }}:
buildArgString+=' --build-arg ${{arg.name}}=${{arg.value}}'
echo 'buildstring=$buildArgString'
displayName: 'build string'
- bash: |
cd $(sourceDirectory)
docker build \
-t $(registryName)/$(imageName):$(imageTag) \
$buildArgString \
.
failOnStderr: true
displayName: 'docker build'
Here I want to construct buildArgString based on the parameters passed and pass it to docker build command as shown. When I use this I get error
The directive 'each' is not allowed in this context. Directives are not supported for expressions that are embedded within a string. Directives are only supported when the entire value is an expression.
Any suggestions?
As the message mentioned, you are using each in a wrong place. Try with below script sample:
azure-pipelines.yml:
extends:
template: docker-template.yml
parameters:
buildArgs:
Arg1 : $(arg1-value)
Arg2 : $(arg2-value)
docker-template.yml:
parameters:
- name: buildArgs
type: object
default: []
stages:
- stage: Docker
displayName: Docker Stage
jobs:
- job: Docker
steps:
- ${{ each arg in parameters.buildArgs }}:
- bash: |
echo ${{ arg.key }}
echo ${{ arg.value }}
echo "##vso[task.setvariable variable=buildArgStringOther]--build-arg ${{ arg.key }}=${{ arg.value }}"
displayName: ${{ arg.key }}
- bash: |
echo "buildstring=$(buildArgStringOther)"
displayName: ECHO-${{ arg.key }}