I want to introduce a for loop in a workflow that consists of 2 individual tasks. The second will be dependent on the first. Each one should use different templates. The second should iterate with {{item}}. For each iteration I want to know if the default is to execute only the second task or it will re-execute the whole flow?
To repeat the second step only, use withItems/withParameter (there is no withArtifact, though you can get the same behavior with data). These loops repeat the specific step they are mentioned in for the specified items/parameter only.
- name: create-resources
inputs:
paramet`enter code here`ers:
- name: env
- name: giturl
- name: resources
- name: awssecret
dag:
tasks:
- name: resource
template: resource-create
arguments:
parameters:
- name: env
value: "{{inputs.parameters.env}}"
- name: giturl
value: "{{inputs.parameters.giturl}}"
- name: resource
value: "{{item}}"
- name: awssecret
value: "{{inputs.parameters.awssecret}}"
withParam: "{{inputs.parameters.resources}}"
############# For parallel execution use steps ##############
steps:
- - name: resource
template: resource-create
arguments:
parameters:
- name: env
value: "{{inputs.parameters.env}}"
- name: giturl
value: "{{inputs.parameters.giturl}}"
- name: resource
value: "{{item}}"
- name: awssecret
value: "{{inputs.parameters.awssecret}}"
withParam: "{{inputs.parameters.resources}}"
Related
I have the following YAML pipeline that I cannot seem to get working based on the conditional values.
parameters:
- name: "workloads"
type: object
default:
wkld001: update
wkld002: "delete"
wkld003: "update"
wkld004: "update"
wkld005: "update"
wkld006: "update"
- name: "environment"
type: string
values:
- prd
- dev
- name: "landing_zone"
type: string
values:
- private
- integration
stages:
- stage:
jobs:
- job: create_params
steps:
- powershell: |
write-host "test"
- ${{each item in parameters.workloads}}:
- ${{ if eq(parameters.workloads[item.key].value, 'update') }}:
- job: "${{item.key}}"
dependsOn: create_params
steps:
- powershell: |
write-host "testing loop - ${{item.value}}"
What i want to do is have a specific command run based on the value set for the workload map.
When I run the above no conditions are met so only the pre-job runs.
The expected behaviour is the loop runs, and the right job is spawned based on the map conditions.
The example only shows the "update" condition only; I plan to have a few more.
I got there in the end with this.
pool:
name: "UOLUKSSHPOOL01"
parameters:
- name: "workloads"
type: object
default:
wkld001: update
wkld002: "delete"
wkld003: "update"
wkld004: "update"
wkld005: "update"
wkld006: "update"
- name: "environment"
type: string
values:
- prd
- dev
- name: "landing_zone"
type: string
values:
- private
- integration
stages:
- stage:
jobs:
- job: create_params
steps:
- powershell: |
write-host "test"
- ${{each item in parameters.workloads}}:
- ${{ if eq(item.value, 'update') }}:
- job: "${{item.key}}"
condition:
dependsOn: create_params
steps:
- powershell: |
write-host "testing loop - ${{item.value}}"
I have two yaml files. One is azure-pipeline.yml
name: test-resources
trigger: none
resources:
repositories:
- repository: pipeline
type: git
name: test-templates
parameters:
- name: whetherYesOrNo
type: string
default: Yes
values:
- Yes
- No
extends:
template: pipelines/ansible-playbook-deploy.yml#pipeline
parameters:
folderName: test-3scale
As for this file, when I run the pipeline, I could choose Yes or No as options before running it.
The other one is the playbook.yml for Ansible
- hosts: localhost
connection: local
become: true
vars_files:
- test_service.yml
- "vars/test.yml"
collections:
- test_collection
tasks:
- name: Find out playbooks pwd
shell: pwd
register: playbook_path_output
no_log: false
- debug: var=playbook_path_output.stdout
- name: echo something
shell: echo 'test this out'
register: playbook_ls_content_output
no_log: false
- debug: var=playbook_ls_content_output.stdout
I wish to add a condition in the playbook.yml task, so that
When I choose "Yes" when running the pipeline, task named "echo something" will run, but if I choose "No", this task will be skipped. I am really new in yaml syntax and logic. Could someone help? Many thanks!
These runs successfully on my side(I can judge the condition with no problem, at compile time it will be expanded.):
azure-pipeline.yml
trigger: none
parameters:
- name: whetherYesOrNo
type: string
default: Yes
values:
- Yes
- No
extends:
template: pipelines/ansible-playbook-deploy.yml
parameters:
whetherYesOrNo: ${{parameters.whetherYesOrNo}}
ansible-playbook-deploy.yml
parameters:
- name: whetherYesOrNo
type: string
default: No
steps:
- ${{ if eq(parameters.whetherYesOrNo, 'Yes') }}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Hello World"
Repository structure on my side:
If Yes:
If No:
I'm working on a concourse pipeline and I need to duplicate a lot of code in my YAML so I'm trying to refactor it so it is easily maintainable and I don't end up with thousands of duplicates lines/blocks.
I have achieve the following yaml file after what seems to be the way to go but it doesn't fullfill all my needs.
add-rotm-points: &add-rotm-points
task: add-rotm-points
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ((registre))/polygone/concourse/cf-cli-python3
tag: 0.0.1
insecure_registries: [ ((registre)) ]
run:
path: source-pipeline/commun/rotm/trigger-rotm.sh
args: [ "source-pipeline", "source-code-x" ]
inputs:
- name: source-pipeline
- name: source-code-x
jobs:
- name: test-a
plan:
- in_parallel:
- get: source-pipeline
- get: source-code-a
trigger: true
- <<: *add-rotm-points
- name: test-b
plan:
- in_parallel:
- get: source-pipeline
- get: source-code-b
trigger: true
- <<: *add-rotm-points
My problem is that both my jobs uses the generic task defined at the top. But in the generic task I need to change source-code-x to the -a or -b version I use in my jobs.
I cannot find a way to achieve this without duplicating my anchor in every jobs and that seems to be counter productive. But i may not have full understood yaml anchors/merges.
All you need to do is map inputs on individual tasks, like this:
add-rotm-points: &add-rotm-points
task: add-rotm-points
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ((registre))/polygone/concourse/cf-cli-python3
tag: 0.0.1
insecure_registries: [ ((registre)) ]
run:
path: source-pipeline/commun/rotm/trigger-rotm.sh
args: [ "source-pipeline", "source-code-x" ]
inputs:
- name: source-pipeline
- name: source-code-x
jobs:
- name: test-a
plan:
- in_parallel:
- get: source-pipeline
- get: source-code-a
trigger: true
- <<: *add-rotm-points
input_mapping:
source-code-x: source-code-a
- name: test-b
plan:
- in_parallel:
- get: source-pipeline
- get: source-code-b
trigger: true
- <<: *add-rotm-points
input_mapping:
source-code-x: source-code-b
See Example Three in this blog: https://blog.concourse-ci.org/introduction-to-task-inputs-and-outputs/
Here is a piece of code:
parameters:
- name: Scenario1
type: object
default: ['Test1','Test2','Test3','Test4']
- name: Scenario2
type: object
default: ['Test5','Test6']
jobs:
- job: Test_Run
pool:
vmImage: 'ubuntu-latest'
steps:
- template: Test.yml
parameters:
tests: ${{ parameters['Scenario1'] }}
For now the part:
tests: ${{ parameters['Scenario1'] }} is hardcoded. I would like to have something like this to be able to pick a scenario:
parameters:
- name: Scenario1
type: object
default: ['Test1','Test2','Test3','Test4']
- name: Scenario2
type: object
default: ['Test5','Test6']
jobs:
- job: Test_Run
pool:
vmImage: 'ubuntu-latest'
steps:
- template: Test.yml
parameters:
tests: ${{ parameters[$(Scenario)] }}
I would like to pass a $(Scenario) variable from Azure pipeline, but I do not know how to insert a variable inside ${{xxx}}. :|
You are looking for using object types. Would be something like
- name: Scenarios
type: object
default:
- ScenarioName: 'Scenario1'
Tests: ['Test1','Test2','Test3','Test4']
- ScenarioName: 'Scenario2'
Tests: ['Test5','Test6']
Then it would be called like:
- ${{ each Scenario in parameters.Scenarios}} :
- template: test.yml
parameters:
ScenarioName: ${{ Scenario.ScenarioName}}
ScenarioTests: ${{ Scenario.Tests}}
Team,
I have 10 tasks and I want to run 2-10 only if condition in task1 is met.
- name: 1Check if the node needs to be processed
stat: /tmp/fscache-cleaned-up1.log
register: dc_status
- name: 2Check if the node needs to be processed
stat: /tmp/fscache-cleaned-up2.log
register: dc_status
failed_when: dc_status.stat.exists
..
..
..
You have to use "path" in your stat module call and "block" for combining dependant tasks, like this:
- name: Check if the node needs to be processed
stat:
path: /tmp/fscache-cleaned-up1.log
register: dc_status
- name: Run this only when the log file exists
block:
- name: Install something
yum:
name:
- somepackage
state: present
- name: Apply a config template
template:
src: templates/src.j2
dest: /etc/foo.conf
- name: Start a service and enable it
service:
name: bar
state: started
enabled: True
when:
- dc_status.stat.exists
- dc_status.stat.is_file
Additional information: ansible stat module, block usage