Line too long for setting up variable - ansible

This is my Ansible task:
- name: get the custom job id
ansible.builtin.set_fact:
custom_job_id: >
"{{ train_custom_image_unmanaged_response.stderr_lines |select('search', 'describe') |list |regex_search('.*/customJobs/(\\d+)', '\\1') |first }}"
when: "(gcs_model_list.stdout is not defined) or ('saved_model.pb' not in gcs_model_list.stdout)"
I am getting "line too long" as Ansible lint error for custom_job_id line.
Any idea how can I break it down in smaller parts?

You can do it using YAML multi lines syntaxes, as you started doing it.
With this syntax, the indentation is what is defining a block, so, as long as you are indented inward of the fact name custom_job_id, all the following code is considered as being the expression that is going to be assigned to that fact.
For example:
- name: get the custom job id
ansible.builtin.set_fact:
custom_job_id: >-
{{
train_custom_image_unmanaged_response.stderr_lines
| select('search', 'describe')
| list
| regex_search('.*/customJobs/(\d+)', '\1')
| first
}}
when: >-
gcs_model_list.stdout is not defined
or 'saved_model.pb' not in gcs_model_list.stdout
Here is a playbook complying with the Ansible linting demonstrating this:
- hosts: localhost
gather_facts: true
tasks:
- name: Get the custom job id
ansible.builtin.set_fact:
custom_job_id: >-
{{
train_custom_image_unmanaged_response.stderr_lines
| select('search', 'describe')
| list
| regex_search('.*/customJobs/(\d+)', '\1')
| first
}}
when: >-
gcs_model_list.stdout is not defined
or 'saved_model.pb' not in gcs_model_list.stdout
vars:
train_custom_image_unmanaged_response:
stderr_lines:
- foo
- bar
- describe - /customJobs/123
- baz
gcs_model_list:
- name: Display `custom_job_id`
ansible.builtin.debug:
var: custom_job_id
Which yields:
PLAY [localhost] **********************************************************
TASK [Get the custom job id] **********************************************
ok: [localhost]
TASK [Display `custom_job_id`] ********************************************
ok: [localhost] =>
custom_job_id: '123'

Related

Unicode error while increment variable value

When I work with static variables it works absolutely fine. But when I try to use dynamic it does not work.
The playbook:
---
- hosts: Swi1
vars:
NewOne: 0
provider:
host: "192.168.0.30"
transport: "cli"
username: "cisco"
password: "cisco"
tasks:
- name: gather facts
register: iosfacts
ios_facts:
provider: "{{ provider }}"
- name: Display the value of the counter
debug:
msg: "NewOne={{ NewOne }} / Data type={{ NewOne | type_debug }}"
- name: interface description
set_fact:
NewOne: " {{ NewOne + 1 }}"
parents: "interface {{ item.key }}"
with_dict: "{{ iosfacts.ansible_facts.ansible_net_interfaces }}"
when: item.value.operstatus == "up"
- debug:
msg: " This is Debug {{ NewOne }}"
Gives the error:
fatal: [Swi1]: FAILED! => {"msg": "Unexpected templating type error
occurred on ({{ NewOne + 1 }}): coercing to Unicode: need string or
buffer, int found"}
If you want to do an increment on a variable, you need to recast it as an int, as set_fact will always make you end up with a string.
As an example, the two tasks:
- set_fact:
NewOne: "{{ NewOne | d(0) + 1 }}"
- debug:
var: NewOne | type_debug
Are giving
TASK [set_fact] ***************************************************************
ok: [localhost]
TASK [debug] ******************************************************************
ok: [localhost] =>
NewOne | type_debug: str
The fix is, then, to use the int filter.
Given:
- set_fact:
NewOne: "{{ NewOne | d(0) | int + 1 }}"
loop: "{{ range(1, 4) }}"
- debug:
var: NewOne
This yields the expected
TASK [set_fact] ***************************************************************
ok: [localhost] => (item=1)
ok: [localhost] => (item=2)
ok: [localhost] => (item=3)
TASK [debug] ******************************************************************
ok: [localhost] =>
NewOne: '3'
But then with your use case, there are more elaborated and shorter way to achieve the same:
- set_fact:
NewOne: >-
{{
iosfacts
.ansible_facts
.ansible_net_interfaces
| selectattr('value.operstatus', '==', 'up')
| length
}}
Given:
- debug:
msg: >-
{{
iosfacts
.ansible_facts
.ansible_net_interfaces
| selectattr('value.operstatus', '==', 'up')
| length
}}
vars:
iosfacts:
ansible_facts:
ansible_net_interfaces:
- value:
operstatus: up
- value:
operstatus: down
- value:
operstatus: up
This yields:
ok: [localhost] =>
msg: '2'
It seems you are trying to implement a loop counter with a programming paradigm, which isn't plain possible in that way since Ansible is not a programming language but a Configuration Management Tool in which you declare a state.
Your current issue is reproducible in the following way:
---
- hosts: localhost
become: false
gather_facts: false
vars:
NewOne: 0
tasks:
- name: Show var
debug:
msg: "{{ NewOne | type_debug }}"
- name: Add value
set_fact:
NewOne: " {{ NewOne + 1 }}"
loop: [1, 2, 3]
- name: Show result
debug:
msg: "{{ NewOne }}
resulting into an output of
TASK [Add value] *************
ok: [localhost] => (item=1)
fatal: [localhost]: FAILED! =>
msg: 'Unexpected templating type error occurred on ( {{ NewOne + 1 }}): coercing to Unicode: need string or buffer, int found'
Possible Solutions
You may have a look into Migrating from with_X to loop and Extended loop variables as an iteration counter is already provided there.
An other approach is given via type casting with filter in the answer of #β.εηοιτ.βε.
There as well if you are just interested in the amount of occurrences of certain status, like interface status up or down.
Further Q&A
Ansible set_fact type cast
Further Documentation
Discovering the data type
Forcing the data type

Ansible compare two list variables

I have to check if a list of mount points are available on the system.
So, I defined a variable with the list of mount points then extracted the available mount points from Ansible facts.
---
- hosts: all
vars:
required_mounts:
- /prom/data
- /prom/logs
tasks:
- name: debug mountpoint
set_fact:
mount_points: "{{ ansible_mounts|json_query('[].mount') }}"
- name: check fs
fail:
msg: 'mount point not found'
when: required_mounts not in mount_points
I am stuck here, I don't know how to compare the variable required_mounts with existing mount points.
If any item in required_mounts is not in the existing mount points the task should fail.
The task check fs always fail, even if the mount points are present.
Do I have to loop one by one? And compare item by item? If so, how can I achieve this?
You can use the set theory for this, since what you are looking for is simply the difference between the required_mounts and the ansible_mounts.
Also, there is no need for a JMESPath query here, this simple requirement can be achieved with a simple map.
So, this can be achieved with the task alone:
- fail:
msg: "Missing mounts: `{{ missing_mounts | join(', ') }}`"
when: missing_mounts | length > 0
vars:
missing_mounts: >-
{{
required_mounts
| difference(
ansible_mounts | map(attribute='mount')
)
}}
Given the playbook:
- hosts: localhost
gather_facts: yes
vars:
required_mounts:
- /etc/hostname
- /etc/hosts
- /tmp/not_an_actual_mount
- /tmp/not_a_mount_either
tasks:
- fail:
msg: "Missing mounts: `{{ missing_mounts | join(', ') }}`"
when: missing_mounts | length > 0
vars:
missing_mounts: >-
{{
required_mounts
| difference(
ansible_mounts | map(attribute='mount')
)
}}
This yields:
TASK [Gathering Facts] *******************************************************
ok: [localhost]
TASK [fail] ******************************************************************
fatal: [localhost]: FAILED! => changed=false
msg: 'Missing mounts: `/tmp/not_an_actual_mount, /tmp/not_a_mount_either`'

In Ansible, how to query hostvars to get a specific value of a key from a list item based on the value of a different key?

EDIT-UPDATE:
I found a way to achieve what was trying to do, using the index_of plugin. The following code outputs what I need.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
mac_address: "{{ hostvars[inventory_hostname]['interfaces'][int_idx|int]['mac_address'] }}"
vars:
int_name: 'PCI1.1'
int_idx: "{{ lookup('ansible.utils.index_of', hostvars[inventory_hostname]['interfaces'], 'eq', int_name, 'name') }}"
- debug:
var: mac_address
Output:
PLAY [CASPOSR1BDAT003] ***********************************************************************************************************************************************************************************************
TASK [ansible.builtin.set_fact] **************************************************************************************************************************************************************************************
ok: [CASPOSR1BDAT003]
TASK [debug] *********************************************************************************************************************************************************************************************************
ok: [CASPOSR1BDAT003] =>
mac_address: 20:67:7C:00:36:A0
What I am trying to do:
Use the Netbox dynamic inventory plugin (this works, brings back all the info I need)
Query hostvars for a particular host, and get the value of the MAC address for a particular interface called PCI1.1
What I have tried:
Converting the hostvars to JSON and using json_query: this hasn't worked, and having looked at some issues on GitHub, hostvars isn't a "normal" dictionary. I've logged a couple of issues anyway (https://github.com/ansible/ansible/issues/76289 and https://github.com/ansible-collections/community.general/issues/3706).
Use a sequence loop and conditional "when" to get the value - this sort of works when using the debug module, but still not just returning the value
What works:
I have tried the following, which outputs the mac_address variable as expected. The length of the list is found, and then the conditional matches the name. I do get an warning about using jinja2 templating delimiters but that's not the target of this question.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- debug:
var: hostvars[inventory_hostname]['interfaces'][{{ item }}]['mac_address']
with_sequence: start=0 end="{{ end_at }}"
vars:
- end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
The result is:
TASK [debug] *************************************************************************************************************************************
[WARNING]: conditional statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found:
hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
skipping: [CASPOSR1BDAT003] => (item=0)
skipping: [CASPOSR1BDAT003] => (item=1)
skipping: [CASPOSR1BDAT003] => (item=2)
skipping: [CASPOSR1BDAT003] => (item=3)
skipping: [CASPOSR1BDAT003] => (item=4)
ok: [CASPOSR1BDAT003] => (item=5) =>
ansible_loop_var: item
hostvars[inventory_hostname]['interfaces'][5]['mac_address']: 20:67:7C:00:36:A0
item: '5'
skipping: [CASPOSR1BDAT003] => (item=6)
skipping: [CASPOSR1BDAT003] => (item=7)
skipping: [CASPOSR1BDAT003] => (item=8)
skipping: [CASPOSR1BDAT003] => (item=9)
I'm trying to use set_fact to store this mac_address variable as I need to use it in a couple of different ways. However, I am unable to use set_fact on this (or any other hostvars data, it seems). For example, the following:
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item]['mac_address'] }}"
with_sequence: start=0 end="{{ end_at }}"
vars:
- end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
- debug:
var: interfaces
results in:
fatal: [CASPOSR1BDAT003]: FAILED! =>
msg: |-
The task includes an option with an undefined variable. The error was: 'list object' has no attribute '5'
The error appears to be in '/Users/kivlint/Documents/GitHub/vmware-automation/ansible/prepare-pxe.yml': line 19, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
# when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
- ansible.builtin.set_fact:
^ here
If I hard-code the number 5 in, it works fine:
TASK [ansible.builtin.set_fact] ******************************************************************************************************************
ok: [CASPOSR1BDAT003]
TASK [debug] *************************************************************************************************************************************
ok: [CASPOSR1BDAT003] =>
interfaces: 20:67:7C:00:36:A0
If I use '5' as a var for the task, it also works.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][int_index]['mac_address'] }}"
vars:
- int_index: 5
So I'm wondering, is this a "bug/feature" in how set_fact does or doesn't work with loops (meaning, the same loop worked fine with debug? Or do I need to re-think the approach and consider trying to use set_fact to set a variable with the index of the list (e.g. 5 in the above example)? Or something else?
There's a lot going on in your code, and achieving the result you want is simpler than you've made it.
Firstly, don't use hostvars[inventory_hostname]; plain variables are the ones belonging to the current host, and going through hostvars introduces some exciting opportunities for things to go wrong. hostvars is for accessing variables belonging to other hosts.
Secondly, using Jinja's built-in filtering capabilities avoids the need to worry about the index of the item that you want.
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
vars:
int_name: PCI1.1
mac_address: "{{ interfaces | selectattr('name', 'eq', int_name) | map(attribute='mac_address') | first }}"
tasks:
- debug:
var: mac_address
there is a confusion between the [5] (6th item of a list) and ['5'] (a key named "5") ,
you see in your error: The error was: 'list object' has no attribute '5'.
with the module debug you have not error because [{{item}}] is replaced by [5] and not by ['5']. Its not the same thing with set_fact.
its the reason you have to use filter int to clarify the situation.
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item|int]['mac_address'] }}"
with_sequence: start=0 end="{{ end_at }}"
vars:
end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][item|int]['name'] == "PCI1.1"
so i suggest you to use loop instead with_sequence:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item]['mac_address'] }}"
loop: "{{ range(0, end_at|int, 1)|list }}"
vars:
end_at: "{{ hostvars[inventory_hostname]['interfaces'] | length }}"
when: hostvars[inventory_hostname]['interfaces'][item]['name'] == "PCI1.1"
set_fact works with loops, but not in a way you expect.
This example constructs list with loop from lists of dicts:
- set_fact:
foo: '{{ foo|d([]) + [item.value] }}'
loop:
- value: 1
- value: 2
Basically, each execution of set_fact creates a fact. You may refer to the same fact in jinja expression for set_fact, but you can't expect it to automatically build lists or something like that.

Issue with using Omit option

I am trying configure a attribute only if my_int_http is defined else I dont want it. So I coded it like below:
profiles:
- "{{ my_int_L4 }}"
- "{{ my_int_http | default(omit) }}"
However my execution fail and when check the arguments passed by the code in actual configuration it shows like below:
"profiles": [
"my_example.internal_tcp",
"__omit_place_holder__ef8c5b99e9707c044ac07fda72fa950565f248a4"
So how to pass absolutely no value where it is passing __omit_place_holder_****?
Q: "How to pass absolutely no value where it is passing omit_place_holder ?"
A1: Some filters also work with omit as expected. For example, the play
- hosts: localhost
vars:
test:
- "{{ var1|default(false) }}"
- "{{ var1|default(omit) }}"
tasks:
- debug:
msg: "{{ {'a': item}|combine({'b': true}) }}"
loop: "{{ test }}"
gives
msg:
a: false
b: true
msg:
b: true
As a sidenote, default(omit) is defined type string
- debug:
msg: "{{ item is defined }}"
loop: "{{ test }}"
- debug:
msg: "{{ item|type_debug }}"
loop: "{{ test }}"
give
TASK [debug] *************************************************************
ok: [localhost] => (item=False) =>
msg: true
ok: [localhost] => (item=__omit_place_holder__6e56f2f992faa6e262507cb77410946ea57dc7ef) =>
msg: true
TASK [debug] *************************************************************
ok: [localhost] => (item=False) =>
msg: bool
ok: [localhost] => (item=__omit_place_holder__6e56f2f992faa6e262507cb77410946ea57dc7ef) =>
msg: str
A2: No value in Ansible is YAML null. Quoting:
This is typically converted into any native null-like value (e.g., undef in Perl, None in Python).
(Given my_int_L4=bob). If the variable my_int_http defaults to null instead of omit
profiles:
- "{{ my_int_L4 }}"
- "{{ my_int_http | default(null) }}"
the list profiles will be undefined
profiles: VARIABLE IS NOT DEFINED!
Use None instead
profiles:
- "{{ my_int_L4 }}"
- "{{ my_int_http | default(None) }}"
The variable my_int_http will default to an empty string
profiles:
- bob
- ''
See also section "YAML tags and Python types" in PyYAML Documentation.
You can try something like this,
profiles:
- "{{ my_int_L4 }}"
- "{{ my_int_http | default(None) }}"
This will give you an empty string. And you can add a check while iterating over the profiles.
Please have a look at this GitHub Issue to get more understanding.

How to include files one by one using dictionary key in ansible?

I am trying to invoke test cases one by one from ansible. Each test case is written in a yml file.
---
- hosts: localhost
tasks:
- set_fact:
k8s_tests:
san-1.yml: "Scale replica to 1"
san-2.yml: "Scale replica to 2"
- name: display local tests
debug: var=k8s_tests
- include: "{{ k8s_test_item }}"
vars:
local_test: "{{ k8s_test_item }}"
with_items: "{{ k8s_tests }}"
loop_control:
loop_var: k8s_test_item
When i execute it , the order of the file is wrong.
The include should be running in the order of san-1.yml first then san-2.yml
but it was opposite.
TASK [include] *********************************************************************************
included: /root/oc310/jenkinsrun/containers/K8S/san-2.yml for localhost
included: /root/oc310/jenkinsrun/containers/K8S/san-1.yml for localhost
So i added dictsort to sort the dictionary.
- include: "{{ k8s_test_item }}"
vars:
local_test: "{{ k8s_test_item }}"
with_items: "{{ k8s_tests | dictsort }}"
loop_control:
loop_var: k8s_test_item
but it tries to include both key and value and fails.
TASK [include] *********************************************************************************
included: /root/oc310/jenkinsrun/containers/K8S/san-1.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 1' on the Ansible Controller."}
included: /root/oc310/jenkinsrun/containers/K8S/san-2.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 2' on the Ansible Controller."}
The values 'Scale replica to 1' and 'Scale replica to 2' are not files and they should not be included. I tried to sort by key but still it tries to include based on value as well and fails.
- include: "{{ k8s_test_item }}"
vars:
local_test: "{{ k8s_test_item }}"
with_items: "{{ k8s_tests | dictsort(false,'key') }}"
loop_control:
loop_var: k8s_test_item
below is the same output
TASK [include] *********************************************************************************
included: /root/oc310/jenkinsrun/containers/K8S/san-1.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 1' on the Ansible Controller."}
included: /root/oc310/jenkinsrun/containers/K8S/san-2.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 2' on the Ansible Controller."}
I need to include the file in order one by one. How to achieve this?
Here is a solution to make sure your file names are always sorted correctly. You need to:
Transform your dictionary to a list with the dict2items filter
Extract only the key attribute for each element in the list with the map filter
sort the resulting list
Below is an implementation with a debug to show each described steps.
Notes:
I had to reverse the order in var declaration to get the same initial result as yours and have a successful demo
loop is the new keyword for defining loops and is equivalent here to with_list. See loop documentation
---
- name: Looping demo
hosts: localhost
gather_facts: false
vars:
k8s_tests:
san-2.yml: "Scale replica to 2"
san-1.yml: "Scale replica to 1"
tasks:
- name: Show initial var
debug:
var: k8s_tests
- name: Show transforming dict to list
debug:
msg: "{{ k8s_tests | dict2items }}"
- name: Show attribute extraction
debug:
msg: "{{ k8s_tests | dict2items | map(attribute='key') | list }}"
- name: Show final sorted result
debug:
msg: "{{ k8s_tests | dict2items | map(attribute='key') | sort }}"
- name: Actually looping over the data
debug:
var: item
loop: "{{ k8s_tests | dict2items | map(attribute='key') | sort }}"

Resources