When translating existing Azure DevOps YAML pipelines to GitHub Actions YAML, I noticed some of my Azure pipelines were just templates calling other YAML files.
trigger:
- master
resources:
repositories:
- repository: templates
type: git
name: 'Cloud Integration\PipelineTemplates'
name: $(Build.SourceBranchName)_$(Build.Reason)_$(rev:r)
variables:
- group: var-lc-integration-emailservice
- name: logicapp_workflows
value: false
- name: base_resources
value: false
- name: functionapp_resources
value: false
- name: functionapp
value: false
- name: ia_resources
value: false
- name: ia_configs
value: false
- name: apim_resources
value: true
stages:
- template: yml-templates\master.yml#templates
parameters:
logicapp_workflows: ${{ variables.logicapp_workflows }}
base_resources: ${{ variables.base_resources }}
functionapp_resources: ${{ variables.functionapp_resources }}
functionapp: ${{ variables.functionapp }}
ia_resources: ${{ variables.ia_resources }}
ia_configs: ${{ variables.ia_configs }}
apim_resources: ${{ variables.apim_resources }}
While writing a GitHub workflow for the above Azure pipeline, Can we have a "dummy job" or no job at all within a workflow to solve this issue?
IIUC yes, see reusing GitHub Actions workflows.
It allows you to call another workflow from your repository and provide inputs/secrets as necessary.
Related
I have no knowledge in the yaml language, I am a designer and not a programmer, but I followed a step by step to create an atomization that would create an automatic release, but it is returning an Error: Input required and not supplied: tag_name
YAML code:
name: Release
on:
push:
branches:
- master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
FILE_NAME: "Dark-Everywhere"
FILE_EXTENSION: ".zip"
BRANCHES: "1.19.3,1.19,1.18"
PACKAGE_NAME: "assets,pack.mcmeta,pack.png"
TAG_NAME: "1.0.0"
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Create a release
uses: actions/create-release#v1
env:
GITHUB_TOKEN: ${{ env.GITHUB_TOKEN }}
with:
tag_name: "v${{ env.TAG_NAME }}"
release_name: "Dark Everywhere ${{ env.TAG_NAME }}🌙"
draft: true
prerelease: false
- name: Generate a zip archive for each branch
run: |
for branch in $(echo $BRANCHES | tr "," "\\n"); do
zip -r "$FILE_NAME-$branch$FILE_EXTENSION" $PACKAGE_NAME
done
- name: Upload files
uses: actions/upload-artifact#v2
with:
name: "$FILE_NAME-$branch$FILE_EXTENSION"
path: "$FILE_NAME-$branch$FILE_EXTENSION"
- name: Update release
uses: actions/update-release#v1
env:
GITHUB_TOKEN: ${{ env.GITHUB_TOKEN }}
with:
release_id: ${{ env.RELEASE_ID }}
body: "Build description here"
I already tried to add TAG_NAME: "1.0.0" inside env in the code but the error persists
I am trying to deploy a Lambda function with the GO runtime. The AWS console indicates that the function was indeed updated when the pipeline runs (the Last Modified date is updated in the Lambda console), but Github Actions indicates a failure (this following error is shown twice):
2022/04/04 00:08:30 ResourceConflictException: The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:us-east-2:***:function:get_all_products_go
{
RespMetadata: {
StatusCode: 409,
RequestID: "cd2b5f3f-c245-4c55-9440-9bdc078bb2b9"
},
Message_: "The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:us-east-2:***:function:get_all_products_go",
Type: "User"
}
Here is my pipeline configuration yaml file:
name: Deploy Lambda function
on:
push:
branches:
- main
jobs:
deploy-lambda:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v1
- name: Install Go
uses: actions/setup-go#v2
with:
go-version: '1.17.4'
- name: Install dependencies
run: |
go version
go mod init storefront-lambdas
go mod tidy
- name: Build binary
run: |
GOOS=linux GOARCH=amd64 go build get-all-products.go && zip deployment.zip get-all-products
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-2
- name: default deploy
uses: appleboy/lambda-action#master
with:
aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws_region: us-east-2
function_name: get_all_products_go
zip_file: deployment.zip
memory_size: 128
timeout: 10
handler: get-all-products
role: arn:aws:iam::xxxxxxxxxx:role/lambda-basic-execution
runtime: go1.x
Here is a piece of code:
parameters:
- name: Scenario1
type: object
default: ['Test1','Test2','Test3','Test4']
- name: Scenario2
type: object
default: ['Test5','Test6']
jobs:
- job: Test_Run
pool:
vmImage: 'ubuntu-latest'
steps:
- template: Test.yml
parameters:
tests: ${{ parameters['Scenario1'] }}
For now the part:
tests: ${{ parameters['Scenario1'] }} is hardcoded. I would like to have something like this to be able to pick a scenario:
parameters:
- name: Scenario1
type: object
default: ['Test1','Test2','Test3','Test4']
- name: Scenario2
type: object
default: ['Test5','Test6']
jobs:
- job: Test_Run
pool:
vmImage: 'ubuntu-latest'
steps:
- template: Test.yml
parameters:
tests: ${{ parameters[$(Scenario)] }}
I would like to pass a $(Scenario) variable from Azure pipeline, but I do not know how to insert a variable inside ${{xxx}}. :|
You are looking for using object types. Would be something like
- name: Scenarios
type: object
default:
- ScenarioName: 'Scenario1'
Tests: ['Test1','Test2','Test3','Test4']
- ScenarioName: 'Scenario2'
Tests: ['Test5','Test6']
Then it would be called like:
- ${{ each Scenario in parameters.Scenarios}} :
- template: test.yml
parameters:
ScenarioName: ${{ Scenario.ScenarioName}}
ScenarioTests: ${{ Scenario.Tests}}
I read this document https://learn.microsoft.com/zh-cn/azure/devops/pipelines/process/expressions?view=azure-devops#conditional-insertion
but not like what demoed in the document, I need add three variables with same condition as below:
name: arm_temp
resources:
repositories:
- repository: self
type: git
variables:
- ${{ if in(lower(coalesce(variables['ENV'], variables['Build.SourceBranchName'])), 'release', 'prod') }}:
- newEnv: 'Prod'
- account: '$(ACCOUNT)'
- password: '$(PASSWORD)'
- ${{ if eq(lower(coalesce(variables['ENV'], variables['Build.SourceBranchName'])), 'qa') }}:
- newEnv: 'QA'
- account: '$(ACCOUNT)'
- password: '$(PASSWORD)'
- resGroupName: ${{ format('RESGROUP-{0}', variables['newEnv']) }}
ACCOUNT, PASSWORD and ENV are variables defined in azure build pipeline
but I always get error before run the build pipeline.
and error notification is about line under the if conditiona.
From your Yaml sample, it seems that the Yaml format has some issues.
You could refer to the following YAML Sample:
variables:
${{ if in(lower(coalesce(variables['ENV'], variables['Build.SourceBranchName'])), 'release', 'prod') }}:
newEnv: 'Prod'
account: $(myaccount)
password: $(mypassword)
${{ if eq(lower(coalesce(variables['ENV'], variables['Build.SourceBranchName'])), 'qa') }}:
newEnv: 'QA'
account: $(myaccount)
password: $(mypassword)
resGroupName: ${{ format('RESGROUP-{0}', variables['newEnv']) }}
pool:
vmimage: windows-latest
steps:
- script: |
echo $(newEnv)
echo $(account)
echo $(password)
Variable:
Result:
Note: You need to change the variable name $(ACCOUNT) $(PASSWORD). They cannot have the same name as the variable defined in yaml($(account), $(password)). Or the variable couldn't pass successfully.
In GitHub Actions, I can write a matrix job like so:
jobs:
test:
name: Test-${{matrix.template}}-${{matrix.os}}
runs-on: ${{matrix.os}}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
template: ['API', 'GraphQL', 'Orleans', 'NuGet']
steps:
#...
This will run every combination of os and template. In Azure Pipelines, you have to specify each combination manually like so:
stages:
- stage: Test
jobs:
- job: Test
strategy:
matrix:
Linux:
os: ubuntu-latest
template: API
Mac:
os: macos-latest
template: API
Windows:
os: windows-latest
template: API
# ...continued
pool:
vmImage: $(os)
timeoutInMinutes: 20
steps:
#...
Is it possible to create a data driven matrix strategy similar to GitHub Actions?
Is it possible to create a data driven matrix strategy similar to GitHub Actions?
The answer is yes. This is a known issue that has already been reported on github:
Add cross-product matrix strategy
In addition, there is workaround that mentioned this issue in the official documentation:
Note
The matrix syntax doesn't support automatic job scaling but you can
implement similar functionality using the each keyword. For an
example, see nedrebo/parameterized-azure-jobs.
jobs:
- template: azure-pipelines-linux.yml
parameters:
images: [ 'archlinux/base', 'ubuntu:16.04', 'ubuntu:18.04', 'fedora:31' ]
pythonVersions: [ '3.5', '3.6', '3.7' ]
swVersions: [ '1.0.0', '1.1.0', '1.2.0', '1.3.0' ]
- template: azure-pipelines-windows.yml
parameters:
images: [ 'vs2017-win2016', 'windows-2019' ]
pythonVersions: [ '3.5', '3.6', '3.7' ]
swVersions: [ '1.0.0', '1.1.0', '1.2.0', '1.3.0' ]
azure-pipelines-windows.yml:
jobs:
- ${{ each image in parameters.images }}:
- ${{ each pythonVersion in parameters.pythonVersions }}:
- ${{ each swVersion in parameters.swVersions }}:
- job:
displayName: ${{ format('OS:{0} PY:{1} SW:{2}', image, pythonVersion, swVersion) }}
pool:
vmImage: ${{ image }}
steps:
- script: echo OS version &&
wmic os get version &&
echo Lets test SW ${{ swVersion }} on Python ${{ pythonVersion }}
Not an ideal solution, but for now, you can loop over parameters. Write a template like the following, and pass your data to it.
# jobs loop template
parameters:
jobs: []
jobs:
- ${{ each job in parameters.jobs }}: # Each job
- ${{ each pair in job }}: # Insert all properties other than "steps"
${{ if ne(pair.key, 'steps') }}:
${{ pair.key }}: ${{ pair.value }}
steps: # Wrap the steps
- task: SetupMyBuildTools#1 # Pre steps
- ${{ job.steps }} # Users steps
- task: PublishMyTelemetry#1 # Post steps
condition: always()
See here for more examples: https://github.com/Microsoft/azure-pipelines-yaml/blob/master/design/each-expression.md#scenario-wrap-jobs