I'm attempting to loop over a list ( I think) with a conditional which also is a list. According to the documentation the conditional should be re-evaluated with each loop. However, I'm finding this not the case.
Here are my two lists which are being set via a set_fact task:
image_id:
- 4bc0467496b6c7a60543069c570ef0e1be4565d25cb2bc7d524600a5fe0d3b8f
- dbcefaa52e7009f5d9b6179a256e70890d29148add0f77741ca4550ba2e2ffa6
- e911c149c0ca46a11a8b6eb604439e972685ec25abfde07eb1cdb272a9c0d1a9
- eb40451959b6c5f4aebb2b687a589a58370faab9b15faa43c0aea8d711155b9e
container_id:
- dbcefaa52e7009f5d9b6179a256e70890d29148add0f77741ca4550ba2e2ffa6
Then, I'm attempting to loop through the image_id list and remove any images. However, don't remove the items listed in the container_id list.
- name: remove images no longer used
containers.podman.podman_image:
name: "{{ item }}"
state: absent
loop: "{{ image_id }}"
when: container_id != image_id
I'm expecting all images to be remove except for the one listed in the container_id variable. Here is my output from the task above:
TASK [docker-host : remove images no longer used] *********************************************************************************************************************************************************
changed: [dtest05] => (item=4bc0467496b6c7a60543069c570ef0e1be4565d25cb2bc7d524600a5fe0d3b8f)
failed: [dtest05] (item=dbcefaa52e7009f5d9b6179a256e70890d29148add0f77741ca4550ba2e2ffa6) => changed=false
ansible_loop_var: item
item: dbcefaa52e7009f5d9b6179a256e70890d29148add0f77741ca4550ba2e2ffa6
msg: |-
Failed to remove image with id dbcefaa52e7009f5d9b6179a256e70890d29148add0f77741ca4550ba2e2ffa6. Error: Image used by 099e729245fb339753cc7ad5e86480965faf2f638dbeb7f535336a532af34de0: image is in use by a container
changed: [dtest05] => (item=e911c149c0ca46a11a8b6eb604439e972685ec25abfde07eb1cdb272a9c0d1a9)
changed: [dtest05] => (item=eb40451959b6c5f4aebb2b687a589a58370faab9b15faa43c0aea8d711155b9e)
I was expecting the failed task to be skipped. If I re-run the task, I do see the task being skipped.
TASK [docker-host : Setting Facts for Image IDs] **********************************************************************************************************************************************************
ok: [dtest05]
TASK [docker-host : debug image_id] ***********************************************************************************************************************************************************************
ok: [dtest05] =>
image_id:
- dbcefaa52e7009f5d9b6179a256e70890d29148add0f77741ca4550ba2e2ffa6
TASK [docker-host : debug container_id] *******************************************************************************************************************************************************************
ok: [dtest05] =>
container_id:
- dbcefaa52e7009f5d9b6179a256e70890d29148add0f77741ca4550ba2e2ffa6
TASK [docker-host : remove images no longer used] *********************************************************************************************************************************************************
skipping: [dtest05] => (item=dbcefaa52e7009f5d9b6179a256e70890d29148add0f77741ca4550ba2e2ffa6)
Am I not understanding the documentation correctly? I'm thinking that my first run should not fail but have an skipped item.
The expression container_id != image_id doesn't make any sense. You're comparing the same two things for every iteration of the loop, and because the two variables are two unequal lists the comparison will always be false.
It looks like you're trying to check if item is contained in the container_id list, which would look something like:
- name: remove images no longer used
containers.podman.podman_image:
name: "{{ item }}"
state: absent
loop: "{{ image_id }}"
when: item not in container_id
Note that you can achieve the same thing with less work by running podman image prune -af (perhaps via command task).
Use the filter difference. For example, the playbook below
- hosts: localhost
vars:
image_id:
- 4bc046
- dbcefa
- e911c1
- eb4045
container_id:
- dbcefa
tasks:
- debug:
msg: "{{ item }}"
loop: "{{ image_id|difference(container_id) }}"
gives (abridged)
msg: 4bc046
msg: e911c1
msg: eb4045
Related
EDIT-UPDATE:
I found a way to achieve what was trying to do, using the index_of plugin. The following code outputs what I need.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
mac_address: "{{ hostvars[inventory_hostname]['interfaces'][int_idx|int]['mac_address'] }}"
vars:
int_name: 'PCI1.1'
int_idx: "{{ lookup('ansible.utils.index_of', hostvars[inventory_hostname]['interfaces'], 'eq', int_name, 'name') }}"
- debug:
var: mac_address
Output:
PLAY [CASPOSR1BDAT003] ***********************************************************************************************************************************************************************************************
TASK [ansible.builtin.set_fact] **************************************************************************************************************************************************************************************
ok: [CASPOSR1BDAT003]
TASK [debug] *********************************************************************************************************************************************************************************************************
ok: [CASPOSR1BDAT003] =>
mac_address: 20:67:7C:00:36:A0
What I am trying to do:
Use the Netbox dynamic inventory plugin (this works, brings back all the info I need)
Query hostvars for a particular host, and get the value of the MAC address for a particular interface called PCI1.1
What I have tried:
Converting the hostvars to JSON and using json_query: this hasn't worked, and having looked at some issues on GitHub, hostvars isn't a "normal" dictionary. I've logged a couple of issues anyway (https://github.com/ansible/ansible/issues/76289 and https://github.com/ansible-collections/community.general/issues/3706).
Use a sequence loop and conditional "when" to get the value - this sort of works when using the debug module, but still not just returning the value
What works:
I have tried the following, which outputs the mac_address variable as expected. The length of the list is found, and then the conditional matches the name. I do get an warning about using jinja2 templating delimiters but that's not the target of this question.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- debug:
var: hostvars[inventory_hostname]['interfaces'][{{ item }}]['mac_address']
with_sequence: start=0 end="{{ end_at }}"
vars:
- end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
The result is:
TASK [debug] *************************************************************************************************************************************
[WARNING]: conditional statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found:
hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
skipping: [CASPOSR1BDAT003] => (item=0)
skipping: [CASPOSR1BDAT003] => (item=1)
skipping: [CASPOSR1BDAT003] => (item=2)
skipping: [CASPOSR1BDAT003] => (item=3)
skipping: [CASPOSR1BDAT003] => (item=4)
ok: [CASPOSR1BDAT003] => (item=5) =>
ansible_loop_var: item
hostvars[inventory_hostname]['interfaces'][5]['mac_address']: 20:67:7C:00:36:A0
item: '5'
skipping: [CASPOSR1BDAT003] => (item=6)
skipping: [CASPOSR1BDAT003] => (item=7)
skipping: [CASPOSR1BDAT003] => (item=8)
skipping: [CASPOSR1BDAT003] => (item=9)
I'm trying to use set_fact to store this mac_address variable as I need to use it in a couple of different ways. However, I am unable to use set_fact on this (or any other hostvars data, it seems). For example, the following:
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item]['mac_address'] }}"
with_sequence: start=0 end="{{ end_at }}"
vars:
- end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
- debug:
var: interfaces
results in:
fatal: [CASPOSR1BDAT003]: FAILED! =>
msg: |-
The task includes an option with an undefined variable. The error was: 'list object' has no attribute '5'
The error appears to be in '/Users/kivlint/Documents/GitHub/vmware-automation/ansible/prepare-pxe.yml': line 19, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
# when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
- ansible.builtin.set_fact:
^ here
If I hard-code the number 5 in, it works fine:
TASK [ansible.builtin.set_fact] ******************************************************************************************************************
ok: [CASPOSR1BDAT003]
TASK [debug] *************************************************************************************************************************************
ok: [CASPOSR1BDAT003] =>
interfaces: 20:67:7C:00:36:A0
If I use '5' as a var for the task, it also works.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][int_index]['mac_address'] }}"
vars:
- int_index: 5
So I'm wondering, is this a "bug/feature" in how set_fact does or doesn't work with loops (meaning, the same loop worked fine with debug? Or do I need to re-think the approach and consider trying to use set_fact to set a variable with the index of the list (e.g. 5 in the above example)? Or something else?
There's a lot going on in your code, and achieving the result you want is simpler than you've made it.
Firstly, don't use hostvars[inventory_hostname]; plain variables are the ones belonging to the current host, and going through hostvars introduces some exciting opportunities for things to go wrong. hostvars is for accessing variables belonging to other hosts.
Secondly, using Jinja's built-in filtering capabilities avoids the need to worry about the index of the item that you want.
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
vars:
int_name: PCI1.1
mac_address: "{{ interfaces | selectattr('name', 'eq', int_name) | map(attribute='mac_address') | first }}"
tasks:
- debug:
var: mac_address
there is a confusion between the [5] (6th item of a list) and ['5'] (a key named "5") ,
you see in your error: The error was: 'list object' has no attribute '5'.
with the module debug you have not error because [{{item}}] is replaced by [5] and not by ['5']. Its not the same thing with set_fact.
its the reason you have to use filter int to clarify the situation.
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item|int]['mac_address'] }}"
with_sequence: start=0 end="{{ end_at }}"
vars:
end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][item|int]['name'] == "PCI1.1"
so i suggest you to use loop instead with_sequence:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item]['mac_address'] }}"
loop: "{{ range(0, end_at|int, 1)|list }}"
vars:
end_at: "{{ hostvars[inventory_hostname]['interfaces'] | length }}"
when: hostvars[inventory_hostname]['interfaces'][item]['name'] == "PCI1.1"
set_fact works with loops, but not in a way you expect.
This example constructs list with loop from lists of dicts:
- set_fact:
foo: '{{ foo|d([]) + [item.value] }}'
loop:
- value: 1
- value: 2
Basically, each execution of set_fact creates a fact. You may refer to the same fact in jinja expression for set_fact, but you can't expect it to automatically build lists or something like that.
I want to execute an include tasks list until a certain condition is met, I do not have a fixed loop but execution depends upon a condition.
A sample play below
Tasks list playbook
tasks.yml
---
- name: "inc test-var {{ test_var }}"
set_fact:
test_var: "{{ test_var | int + 1 }} "
parent playbook parent.yml
---
- hosts: all
gather_facts: no
tasks:
- set_fact:
test_var: '1'
req_var: '4'
- name: "Test multi run of task"
include_tasks: ./includes/tasks.yml
register: versions_result
until: test_var is version(req_var, '<')
retries: 5
here I am expecting parent.yml tasks to run multiple times but it only run once.
Could some one point out what I am doing wrong and how to run a task multiple times until a condition is met.
Cheers,
One way to include_tasks multiple times is to loop over the range of numbers till it reaches the required number. However as you expect the "parent" playbook will not be run multiple times, the tasks file will be.
Consider the below example:
Through my main playbook parent.yml, I would like to run tasks1.yml multiple times (as defined in set_fact).
tasks:
- set_fact:
num: 1
req_num: 4
- include_tasks: tasks1.yml
loop: "{{ range(num, req_num + 1)|list }}"
And in my tasks1.yml, I have a simple debug message:
- debug:
msg: "Run {{ item }}"
Includes tasks1.yml 4 times and gives below output when I run ansible-playbook parent.yml:
TASK [include_tasks] ******************************************************************************************************************************************************************
included: /home/user/tasks1.yml for localhost
included: /home/user/tasks1.yml for localhost
included: /home/user/tasks1.yml for localhost
included: /home/user/tasks1.yml for localhost
TASK [debug] **************************************************************************************************************************************************************************
ok: [localhost] =>
msg: Run 1
TASK [debug] **************************************************************************************************************************************************************************
ok: [localhost] =>
msg: Run 2
# ...goes till "Run 4"
I am looking for the easiest way to check if all elements from a list are present in another bigger list, in Ansible.
Example checking that ['pkg_mgr', 'python'] are both present in ansible_facts.
when: "{{ ['pkg_mgr', 'python'] | difference(ansible_facts.keys()) | length == 0 }}"
Q: "I am far from being pleased regarding how ugly it looks. I would be more than happy to see cleaner solutions."
A: An empty list evaluates to False in Ansible. It's not necessary to test the length of the list. Ansible condition when expands the expression by default. It's not necessary to close it in braces. The equivalent condition is
when: not ['pkg_mgr', 'python']|difference(ansible_facts.keys())
Python3 returns a dictionary view object instead of a list for methods dict.keys(), dict.values(), and dict.items(). Add list filter to make the code portable. See Dictionary Views.
when: not ['pkg_mgr', 'python']|difference(ansible_facts.keys()|list)
I was able to find a solution that works but I am far from being pleased regarding how ugly it looks.
- when: "{{ ['pkg_mgr', 'python'] | difference(ansible_facts.keys()) | length == 0 }}"
...
I would be more than happy to see cleaner solutions.
How about using is subset?
Test:
- name: "Check lists"
hosts: localhost
connection: local
tasks:
- debug:
msg: "{{ ['pkg_mgr', 'python'] is subset(ansible_facts.keys()) }}"
- debug:
msg: "{{ ['pkg_mgr', 'python', 'foo'] is subset(ansible_facts.keys()) }}"
Output:
PLAY [Check lists] *****************************************************************************************************************************************************************************************
TASK [Gathering Facts] *************************************************************************************************************************************************************************************
ok: [localhost]
TASK [debug] ***********************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": true
}
TASK [debug] ***********************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": false
}
I do have a simple json file, where i need to pull a set of value from EACH array item, but during iteration it fails.
My playbook looks like:
code:
---
- name: direct - this works like charm
set_fact:
bb: "{{ pr_json.json.issues[0].fields.customfield_11756.value }}"
- debug:
var: bb
- name: via array - this is not working since iteration is not happening
set_fact:
dd_branch: "{{ pr_json.json.issues[{{ item }}].fields.customfield_11756.value }}"
register: mass
- debug:
var: mass
Getting output as:
TASK [jira_update : direct - this works like charm] ********************************************************************************************************************
task path: /home/test/ansible_jira/roles/jira_update/tasks/call.yml:3
ok: [localhost] => {
"ansible_facts": {
"bb": "R4.19"
},
"changed": false
}
TASK [jira_update : debug] *********************************************************************************************************************************************
task path: /home/test/ansible_jira/roles/jira_update/tasks/call.yml:7
ok: [localhost] => {
"bb": "R4.19"
}
TASK [jira_update : via array - this is not working since iteratoin is not happening] **********************************************************************************
task path: /home/test/ansible_jira/roles/jira_update/tasks/call.yml:10
fatal: [localhost]: FAILED! => {
"msg": "template error while templating string: expected token ':', got '}'. String: {{ pr_json.json.issues[{{ item }}].fields.customfield_11756.value }}"
}
Please do let us know how can I iterate through an array variable value on every sequence.
tried this too, but can somebody help to iterate the array values, please.
- name: Create PR request in TEMS JIRA
jira:
uri: "{{ tems_jira }}"
username: "{{ user }}"
password: "{{ pass }}"
operation: create
project: PR
issuetype: 'PR-Form'
summary: "{{ pr_json.json| json_query('issues[].fields.summary') }}"
description: "{{ pr_json.json | json_query('issues[].fields.description') }}"
args:
fields:
customfield_10303:
value: "{{ pr_json.json | json_query('issues[].fields.customfield_11756.value') }}"
Youy need to feed your list into a with_items iterator. Thats what sets the item variable for looping purposes.
- name: via array - this is not working since iteration is not happening
set_fact:
dd_branch: "{{ pr_json.json.issues[ item ].fields.customfield_11756.value }}"
register: mass
with_items:
- 0
- 1
That will iterate through all of the list items of pr_json.json.issues which will let you dive deeper into the variable structure like you are looking for. There are a lot of other factors that you can feed into the loop that might interest you that you can find detailed here.
https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html
I want to automate the installation process of our software for our client. Therefore I wrote an Ansible playbook which has a task which should check if all the mandatory variables are set:
- name: Check environment variables.
hosts: all
vars_files:
- required_vars.yml
tasks:
- fail: msg="Variable '{{ item }}' is not defined"
when: item not in hostvars[inventory_hostname]
with_items:
- required_vars
The required_vars.yml looks like this:
required_vars:
- APPHOME: /home/foo/bar
- TMPDIR: /home/foo/bar/tmp
When I execute the playbook via ansible-playbook -i inventory/dev.yml playbook.yml I get the following error:
TASK [Gathering Facts] *************************************************************************************************************************************************************************************************************************ok: [localhost]
TASK [fail] ************************************************************************************************************************************************************************************************************************************failed:
[localhost] (item=required_vars) => {"changed": false, "failed": true, "item": "required_vars", "msg": "Variable 'required_vars' is not defined"}
It is obvious that I am doing something wrong, but I cannot point to the error. Can you help me please?
Edit: the accepted answer helped me out. Thank you.
But I have two more questions:
The executed task says:
TASK [fail]
skipping: [some_ip] => (item=/root)
skipping: [some_ip] => (item=TMPDIR: /home/foo/bar/tmp)
It is getting skipped because all variables are set, correct?
I think I figured out how to print the correct message, if the variable is not set:
- name: Check environment variables.
hosts: all
vars_files:
- required_vars.yml
tasks:
- fail:
msg: "Variable '{{ item }}' is not defined"
with_items: "{{ required_vars }}"
when: item is undefined
Correct? Or is there a better solution?
Two problems here:
You want to iterate over the value of required_vars variable value, so you need to provide it as an argument to with_items: "{{ required_vars }}":
with_items: "{{ required_vars }}"
Currently you are providing a list of a single element with a statically defined string required_vars.
You need to change the data type of the elements in your required_vars list to strings:
required_vars:
- "APPHOME: /home/foo/bar"
- "TMPDIR: /home/foo/bar/tmp"
Currently (because of : followed by space) you defined dictionaries, so for example in the first iteration item will have a value of { "APPHOME": "/home/foo/bar" }, which will then always fail on the when condition.
Bonus problem:
you defined a message in the form "Variable '{{ item }}' is not defined";
Ansible reports Variable 'required_vars' is not defined;
the above is not an error, as you think ("I get the following error"), but a correct result of the fail module with the message you defined yourself.
Since you have only one value for 'with_items' I think it should look like this:
with_items: "{{ required_vars }}"
On one line and with the brackets and quotation marks. Once you have more then one item, you can use the list like you did:
with_items:
- "{{ one }}"
- "{{ two }}"