Azure pipeline with custom variables as condition - yaml

Question

We would like to optimize our pipelines to avoid running steps that are not needed for the particular case. I have created the variable shouldTriggerAnyBuild but it always seems to be true (or ignored), as the specified steps always runs, even though the none of the steps later, where the conditions are combined from, are run.
What is the problem with the script, or how can I debug it?
trigger:
- master
- stage
- release/*
pool:
vmImage: 'macOS-latest'
variables:
shouldTriggerAnyBuild: $[ or(and(eq(variables['Build.SourceBranch'], 'refs/heads/master'), not(startsWith(variables['Build.SourceVersionMessage'], 'release:'))), eq(variables['Build.SourceBranch'], 'refs/heads/stage'), startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')) ]
steps:
- task: UseRubyVersion#0
inputs:
versionSpec: '~> 2.6'
- script: echo "y" | $ANDROID_HOME/tools/bin/sdkmanager --install 'system-images;android-29;google_apis;x86'
displayName: 'install/setup android sdkmanager'
condition: variables.shouldTriggerAnyBuild
- script: gem install bundler
displayName: 'gem install bundler'
condition: variables.shouldTriggerAnyBuild
- script: bundle install
displayName: 'bundle install'
condition: variables.shouldTriggerAnyBuild
- script: bundle exec fastlane ciBuildDev
displayName: 'build dev'
condition: and(eq(variables['Build.SourceBranch'], 'refs/heads/master'), not(startsWith(variables['Build.SourceVersionMessage'], 'release:')))
- script: bundle exec fastlane ciDeployToTest
displayName: 'build stage'
condition: eq(variables['Build.SourceBranch'], 'refs/heads/stage')
- script: bundle exec fastlane ciDeployToGooglePlay
displayName: 'build release'
condition: startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')
- task: PublishBuildArtifacts#1
displayName: "Publish artifacts .apk"
condition: startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')
inputs:
PathtoPublish: ./app/build/outputs/apk/prod/app-prod-unsigned.apk
ArtifactName: Prod_app_apk
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops

Answer 1

What is the problem with the script?
Conditions are written as expressions. condition: variables.shouldTriggerAnyBuild won't take effect.
Instead you can use condition: eq(variables['SHOULDTRIGGERANYBUILD'], 'True') for conditional step. I think it's the direct cause of your issue. Also, feel free to use condition: eq(variables['shouldTriggerAnyBuild'], 'True') if you want, it also works.
How to debug it?
Here's a quick way to debug the value when necessary:
1.Change the second value to a impossible variable, then you can check the real value for your custom variable when the step is skipped:
2.If you want to make all thing clear, then you can do something like:
- task: CmdLine#2
inputs:
script: |
echo Hello world
condition: eq(and(eq(variables['Build.SourceBranch'], 'refs/heads/master'), not(startsWith(variables['Build.SourceVersionMessage'], 'release:'))),'True')
- task: CmdLine#2
inputs:
script: |
echo Hello world
condition: eq(eq(variables['Build.SourceBranch'], 'refs/heads/stage'),'True')
- task: CmdLine#2
inputs:
script: |
echo Hello world
condition: eq(startsWith(variables['Build.SourceBranch'], 'refs/heads/release/'),'True')
Since your variable is the combination of and(eq(variables['Build.SourceBranch'], 'refs/heads/master'), not(startsWith(variables['Build.SourceVersionMessage'], 'release:'))), eq(variables['Build.SourceBranch'], 'refs/heads/stage') and startsWith(variables['Build.SourceBranch'], 'refs/heads/release/') via Or function, you can divide them to debug how the shouldTriggerAnyBuild is expanded as always True.
In this way, you can debug them easily to find how the variable is expanded:
Note:
1.Or function: Evaluates True if any parameter is true.
2.Most of the time we choose to skip current task if one of previous tasks have failed, so you can consider combining succeeded() and your variable shouldTriggerAnyBuild. Examples here.

Answer 2

After a lot of trial and error we found that there was Build.SourceVersionMessage that meant we could not use it as a variable, so we ended up with a hack as shown below:
trigger:
- master
- stage
- release/*
pool:
vmImage: "macOS-latest"
variables:
# we can't use Build.SourceVersionMessage up here because it's not defined when the variables is set here
isMaster: ${{eq(variables['Build.SourceBranch'], 'refs/heads/master')}}
isStage: ${{eq(variables['Build.SourceBranch'], 'refs/heads/stage')}}
isRelease: ${{startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')}}
releaseMessagePrefix: "release:"
jobs:
- job:
steps:
- task: UseRubyVersion#0
displayName: "set ruby version"
inputs:
versionSpec: "~> 2.6"
- task: Bash#3
displayName: "set commitMessage variable"
inputs:
targetType: inline
script: echo '##vso[task.setvariable variable=commitMessage]$(Build.SourceVersionMessage)'
- script: echo "y" | $ANDROID_HOME/tools/bin/sdkmanager --install 'system-images;android-29;google_apis;x86'
displayName: "install/setup android sdkmanager"
condition: or(and(variables.isMaster, not(startsWith(variables.commitMessage, variables.releaseMessagePrefix))), ${{variables.isStage}}, ${{variables.isRelease}})
- script: gem install bundler
displayName: "gem install bundler"
condition: or(and(variables.isMaster, not(startsWith(variables.commitMessage, variables.releaseMessagePrefix))), ${{variables.isStage}}, ${{variables.isRelease}})
- script: bundle install
displayName: "bundle install"
condition: or(and(variables.isMaster, not(startsWith(variables.commitMessage, variables.releaseMessagePrefix))), ${{variables.isStage}}, ${{variables.isRelease}})
- script: bundle exec fastlane ciBuildDev
displayName: "Build: dev"
condition: and(variables.isMaster, not(startsWith(variables.commitMessage, variables.releaseMessagePrefix)))
- script: bundle exec fastlane ciDeployToTest
displayName: "Build: stage"
condition: ${{variables.isStage}}
- script: bundle exec fastlane ciDeployToGooglePlay
displayName: "Build: release"
condition: ${{variables.isRelease}}
- task: PublishBuildArtifacts#1
displayName: "Publish: .apk"
condition: ${{variables.isRelease}}
inputs:
PathtoPublish: ./app/build/outputs/apk/prod/app-prod-unsigned.apk
ArtifactName: Prod_app_apk