Locate string/integer not in list - ansible

Im pulling a list of VLANs from our IPAM via an API and I want to be able to locate an unused "vlanId" that isnt in the list. I was expecting that I could use with_items for the JSON content and then use the random function with an until loop and occasionally it will initially generate a number that doesnt exist in the list. Mostly it just gets stuck and doesnt generate a new random number when the one generated already exists.
Playbook:
- uri:
url: "#"
validate_certs: no
headers:
token: "{{ token }}"
method: GET
force_basic_auth: yes
return_content: yes
register: ipam
- set_fact:
value: "{{ 4094 | random(start=1) }}"
until: value not in item.vlanId
with_items: "{{ ipam.json.data }}"
retries: 4093
- debug: msg="{{ value }}"
Relevant Output:
ok: [localhost] => (item={u'domainId': u'3', u'description': u'#', u'editDate': None, u'Customer ID': None, u'number': u'2241', u'vlanId': u'548', u'name': u'2241', u'Customer Name': None, u'custom_fields': None, u'Engineer': None}) => {
"ansible_facts": {
"value": "2727"
},
"ansible_facts_cacheable": false,
"attempts": 1,
"changed": false,
"item": {
"Customer ID": null,
"Customer Name": null,
"Engineer": null,
"custom_fields": null,
"description": "#",
"domainId": "3",
"editDate": null,
"name": "2241",
"number": "2241",
"vlanId": "548"
}
}
ok: [localhost] => (item={u'domainId': u'3', u'description': u'#', u'editDate': None, u'Customer ID': None, u'number': u'2242', u'vlanId': u'549', u'name': u'2242', u'Customer Name': None, u'custom_fields': None, u'Engineer': None}) => {
"ansible_facts": {
"value": "1955"
},
"ansible_facts_cacheable": false,
"attempts": 1,
"changed": false,
"item": {
"Customer ID": null,
"Customer Name": null,
"Engineer": null,
"custom_fields": null,
"description": "#",
"domainId": "3",
"editDate": null,
"name": "2242",
"number": "2242",
"vlanId": "549"
}
}
FAILED - RETRYING: set_fact (4000 retries left).Result was: {
"ansible_facts": {
"value": "50"
},
"ansible_facts_cacheable": false,
"attempts": 1,
"changed": false,
"retries": 4001
}
FAILED - RETRYING: set_fact (3999 retries left).Result was: {
"ansible_facts": {
"value": "50"
},
"ansible_facts_cacheable": false,
"attempts": 2,
"changed": false,
"retries": 4001
}
FAILED - RETRYING: set_fact (3998 retries left).Result was: {
"ansible_facts": {
"value": "50"
},
"ansible_facts_cacheable": false,
"attempts": 3,
"changed": false,
"retries": 4001
Im currently using ansible 2.4.2.0
If this is something that cant/shouldnt be done in Ansible, any guidance would be appreciated.

Here's what you want:
- debug:
msg: "{{ range(1, 4095) | difference(vlanIds) | random }}"
vars:
vlanIds: "{{ ipam.json.data | map(attribute='vlanId') | list }}"
Explanation:
map(attribute=... - create a list of vlan IDs,
range - generate a sequence from 1 to 4094,
difference - select all the elements from the above. that are not on the vlanIds list,
random - select a random element from the above.
For strings (as the title suggests) replace range with a list of possible strings.

Related

Ansible vmware_guest module with_items how to loop the debug msg output

I'm trying to deploy some virtual machines which seems to be working fine, and I can get the output to a debug screen, but what I would like is to be able to filter the output a bit and show hostname and IP address really. But I'm not sure how to loop the output! I've tried:
- debug:
msg: {{ item.instance.ipv4 }}
with_items: _result.results
Here is my playbook:
---
- name: Clone VMs
hosts: localhost
gather_facts: false
vars_files:
- vars.yml
- secrets.yml
tasks:
- name: Clone multiple VMs from template
local_action:
module: vmware_guest
hostname: "{{ vcenter_hostname }}"
username: "{{ username }}"
password: "{{ password }}"
validate_certs: no
folder: "{{ folder }}"
template: "{{ item.template }}"
name: "{{ item.name }}"
esxi_hostname: "{{ esxi_hostname }}"
datacenter: "{{ datacenter }}"
state: "{{ state }}"
# customization_spec: "{{ customization_spec }}"
with_items: "{{ servers }}"
register: _result
- name: Output
debug:
msg: "{{ _result.results }}"
#with_items: _result.results.msg
In my vars I have the following (amongst other things)
servers:
- { name: centos8_01, template: centos8 }
- { name: centos7_01, template: centos7-minimal }
The output in the debug currently is:
ok: [localhost] => {
"msg": [
{
"ansible_loop_var": "item",
"changed": false,
"failed": false,
"instance": {
"annotation": "",
"current_snapshot": null,
"customvalues": {},
"guest_consolidation_needed": false,
"guest_question": null,
"guest_tools_status": "guestToolsRunning",
"guest_tools_version": "10346",
"hw_cluster": "Cluster",
"hw_cores_per_socket": 1,
"hw_datastores": [
"SSD"
],
"hw_esxi_host": "******",
"hw_eth0": {
"addresstype": "assigned",
"ipaddresses": [],
"label": "Network adapter 1",
"macaddress": "00:50:56:a3:4b:ba",
"macaddress_dash": "00-50-56-a3-4b-ba",
"portgroup_key": null,
"portgroup_portkey": null,
"summary": "10.0.0.0"
},
"hw_files": [
"[SSD] centos8_01/centos8_01.vmx",
"[SSD] centos8_01/centos8_01.nvram",
"[SSD] centos8_01/centos8_01.vmsd",
"[SSD] centos8_01/centos8_01.vmdk"
],
"hw_folder": "/Learn/vm/L2L",
"hw_guest_full_name": "CentOS 8 (64-bit)",
"hw_guest_ha_state": null,
"hw_guest_id": "centos8_64Guest",
"hw_interfaces": [
"eth0"
],
"hw_is_template": false,
"hw_memtotal_mb": 4096,
"hw_name": "centos8_01",
"hw_power_status": "poweredOn",
"hw_processor_count": 2,
"hw_product_uuid": "4223465f-32ce-38ab-0d0e-a92f187b7d4d",
"hw_version": "vmx-14",
"instance_uuid": "5023b456-cda0-1fa6-c0a5-67d57ede5707",
"ipv4": null,
"ipv6": null,
"module_hw": true,
"moid": "vm-12173",
"snapshots": [],
"vimref": "vim.VirtualMachine:vm-12173",
"vnc": {}
},
"invocation": {
"module_args": {
"annotation": null,
"cdrom": [],
"cluster": null,
"convert": null,
"customization": {},
"customization_spec": null,
"customvalues": [],
"datacenter": "******",
"datastore": null,
"disk": [],
"esxi_hostname": "******",
"folder": "L2L",
"force": false,
"guest_id": null,
"hardware": {},
"hostname": "******",
"is_template": false,
"linked_clone": false,
"name": "centos8_01",
"name_match": "first",
"networks": [],
"password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"port": 443,
"proxy_host": null,
"proxy_port": null,
"resource_pool": null,
"snapshot_src": null,
"state": "poweredon",
"state_change_timeout": 0,
"template": "centos8",
"use_instance_uuid": false,
"username": "******#vsphere.local",
"uuid": null,
"validate_certs": false,
"vapp_properties": [],
"wait_for_customization": false,
"wait_for_ip_address": false
}
},
"item": {
"name": "centos8_01",
"template": "centos8"
}
},
{
"ansible_loop_var": "item",
"changed": false,
"failed": false,
"instance": {
"annotation": "",
"current_snapshot": null,
"customvalues": {},
"guest_consolidation_needed": false,
"guest_question": null,
"guest_tools_status": "guestToolsRunning",
"guest_tools_version": "10309",
"hw_cluster": "Corby",
"hw_cores_per_socket": 1,
"hw_datastores": [
"SSD"
],
"hw_esxi_host": "******",
"hw_eth0": {
"addresstype": "assigned",
"ipaddresses": [
"10.0.0.137",
"fe80::e1b4:c7c5:8268:4e7b"
],
"label": "Network adapter 1",
"macaddress": "00:50:56:a3:f8:9f",
"macaddress_dash": "00-50-56-a3-f8-9f",
"portgroup_key": null,
"portgroup_portkey": null,
"summary": "VM Network"
},
"hw_files": [
"[SSD] centos7_01/centos7_01.vmx",
"[SSD] centos7_01/centos7_01.nvram",
"[SSD] centos7_01/centos7_01.vmsd",
"[SSD] centos7_01/centos7_01_2.vmdk"
],
"hw_folder": "/Learn/vm/L2L",
"hw_guest_full_name": "CentOS 6 (64-bit)",
"hw_guest_ha_state": null,
"hw_guest_id": "centos6_64Guest",
"hw_interfaces": [
"eth0"
],
"hw_is_template": false,
"hw_memtotal_mb": 2048,
"hw_name": "centos7_01",
"hw_power_status": "poweredOn",
"hw_processor_count": 1,
"hw_product_uuid": "4223d5ef-9363-03d4-8aea-7b1822b21e36",
"hw_version": "vmx-14",
"instance_uuid": "5023eea9-85ac-1c18-48bb-e0b63339c9fa",
"ipv4": "10.0.0.137",
"ipv6": null,
"module_hw": true,
"moid": "vm-12174",
"snapshots": [],
"vimref": "vim.VirtualMachine:vm-12174",
"vnc": {}
},
"invocation": {
"module_args": {
"annotation": null,
"cdrom": [],
"cluster": null,
"convert": null,
"customization": {},
"customization_spec": null,
"customvalues": [],
"datacenter": "******",
"datastore": null,
"disk": [],
"esxi_hostname": "******",
"folder": "******",
"force": false,
"guest_id": null,
"hardware": {},
"hostname": "******",
"is_template": false,
"linked_clone": false,
"name": "centos7_01",
"name_match": "first",
"networks": [],
"password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"port": 443,
"proxy_host": null,
"proxy_port": null,
"resource_pool": null,
"snapshot_src": null,
"state": "poweredon",
"state_change_timeout": 0,
"template": "centos7-minimal",
"use_instance_uuid": false,
"username": "******",
"uuid": null,
"validate_certs": false,
"vapp_properties": [],
"wait_for_customization": false,
"wait_for_ip_address": false
}
},
"item": {
"name": "centos7_01",
"template": "centos7-minimal"
}
}
]
}
There are multiple ways to do this.
A loop is one of them:
- debug:
msg:
hostname: "{{ item.invocation.module_args.hostname }}"
ip: "{{ item.instance.ipv4 }}"
loop: "{{ _result.results }}"
loop_control:
label: "item.invocation.module_args.hostname"
This would yield:
ok: [localhost] => (item=item.invocation.module_args.hostname) =>
msg:
hostname: '******'
ip: ''
ok: [localhost] => (item=item.invocation.module_args.hostname) =>
msg:
hostname: '******'
ip: 10.0.0.137
Another solution is to use JMESPath and the json_query filter:
- debug:
var: >-
_result.results
| json_query('[].{
hostname: invocation.module_args.hostname, ip: instance.ipv4
}')
This would yield:
ok: [localhost] =>
? |-
_result.results | json_query('[].{
hostname: invocation.module_args.hostname, ip: instance.ipv4
}')
: - hostname: '******'
ip: null
- hostname: '******'
ip: 10.0.0.137

create k8s secrets by using nested ansible loops

I am trying to create k8s secrets with the storing username and secret in results module of ansible but i created one loop for the name and namespace section and i am trying to create one more loop from the json results output. but it was taking one secret for all the projects
variables:
project_namespaces:
- projectName: helm
Namespaces:
- default
- core
- projectName: proxy
Namespaces:
- default
- core
robot_result:
ok: [harbor_stg1_dkp1] => {
"robot_result": {
"changed": false,
"msg": "All items completed",
"results": [
{
"ansible_loop_var": "item",
"content_type": "application/json",
"cookies": {
"sid": "557b377ba2bbe3f054d68bd56b0e10ef"
},
"invocation": {
"module_args": {
"attributes": null,
"body": {
"description": "pull artifacts for helm",
"disable": true,
"duration": -1,
"level": "system",
"name": "helm-robot",
"permissions": [
{
"access": [
{
"action": "pull",
"resource": "repository"
}
],
"kind": "project",
"namespace": "helm"
}
]
},
"body_format": "json",
"headers": {
"Accept": "application/json",
"Authorization": "Basic ",
"Content-Type": "application/json"
},
"http_agent": "ansible-httpget",
"status_code": [
201
],
"timeout": 30,
"unix_socket": null,
"validate_certs": false
}
},
"item": {
"name": "helm-robot",
"projectName": "helm"
},
"json": {
"creation_time": "2022-03-21T10:05:49.248Z",
"expires_at": -1,
"id": 67,
"name": "robot#helm-robot",
"secret": "Q8mjthgRJFmscjfmqW1QzXEyKjmLEPQm"
},
"x_envoy_upstream_service_time": "18",
"x_request_id": "496faaa6-bdc4-4e83-890a-2c577576f16b"
},
{
"ansible_loop_var": "item",
"content_type": "application/json",
"cookies": {
"sid": "6275d7bfe74e71db0a3947f4beb1e159"
},
"cookies_string": "sid=6275d7bfe74e71db0a3947f4beb1e159",
"date": "Mon, 21 Mar 2022 10:05:56 GMT",
"elapsed": 1,
"failed": false,
"failed_when_result": false,
"invocation": {
"module_args": {
"attributes": null,
"body": {
"description": "pull artifacts for proxy",
"disable": true,
"duration": -1,
"level": "system",
"name": "proxy-robot",
"permissions": [
{
"access": [
{
"action": "pull",
"resource": "repository"
}
],
"kind": "project",
"namespace": "proxy"
}
]
},
"body_format": "json",
"group": null,
"headers": {
"Accept": "application/json",
"Authorization": "Basic ",
"Content-Type": "application/json"
},
"http_agent": "ansible-httpget",
"method": "POST"
],
"timeout": 30,
"unix_socket": null,
"validate_certs": false
}
},
"item": {
"name": "proxy-robot",
"projectName": "proxy"
},
"json": {
"creation_time": "2022-03-21T10:05:56.807Z",
"expires_at": -1,
"id": 68,
"name": "robot#proxy-robot",
"secret": "slPnm8Zkp0OGBLec6tTcPuPITgOU2PAn"
},
"msg": "OK (144 bytes)",
"x_envoy_upstream_service_time": "15",
"x_request_id": "93478b05-897b-4df9-abb4-e07e03723af0"
}
task.yaml
- name: Create secrets
k8s:
state: present
definition:
apiVersion: v1
kind: Secret
metadata:
name: "{{ item.0.projectName }}"
namespace: "{{ item.1 }}"
stringData:
password: "{{ project.secret }}"
username: "{{ project.name }}"
type: Opaque
vars:
project: "{{ (robot_result.results | json_query('[*].json'))[ansible_loop.index0] }}"
loop: "{{ project_namespaces | subelements('Namespaces') }}"
loop_control:
extended: yes
label: "{{ item.0.projectName }}"
when i executing this task i was getting the output like
ok: [localhost] => (item=helm) => {
"msg": "name: helm, namespace: default, password: Q8mjthgRJFmscjfmqW1QzXEyKjmLEPQm, username: robot#helm-robot"
}
ok: [localhost] => (item=helm) => {
"msg": "name: helm, namespace: core, password: slPnm8Zkp0OGBLec6tTcPuPITgOU2PAn, username: robot#proxy-robot"
}
fatal: [harbor_shiplab_stg1_dkp1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: {{ (robot_result.results | json_query('[*].json'))[ansible_loop.index0] }}: list object has no element 2\n\nThe error appears to be in '/home/ubuntu/konvoy/ansible/roles/harbor-gc/tasks/main.yml': line 47, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Create secrets\n ^ here\n"}
but my requirement would be like this
ok: [localhost] => (item=helm) => {
"msg": "name: helm, namespace: default, password: Q8mjthgRJFmscjfmqW1QzXEyKjmLEPQm, username: robot#helm-robot"
}
ok: [localhost] => (item=helm) => {
"msg": "name: helm, namespace: core, password: Q8mjthgRJFmscjfmqW1QzXEyKjmLEPQm, username: robot#helm-robot"
}
ok: [localhost] => (item=proxy) => {
"msg": "name: proxy, namespace: default, password: slPnm8Zkp0OGBLec6tTcPuPITgOU2PAn, username: robot#proxy-robot"
}
ok: [localhost] => (item=proxy) => {
"msg": "name: proxy, namespace: core, password: slPnm8Zkp0OGBLec6tTcPuPITgOU2PAn, username: robot#proxy-robot"
}
As i am new to this ansible i was not much understanding this loops.Any help or suggestions would be appreciated and Thank you
You have to link secret with projectName:
- name: link projectname and json
set_fact:
dico: "{{ dico | d({}) | combine({item.projectName: project[ansible_loop.index0]}) }}"
vars:
project: "{{ (robot_result.results | json_query('[*].json')) }}"
it: "{{ (robot_result.results | json_query('[*].item')) }}"
loop: "{{ it }}"
loop_control:
extended: yes
- debug:
msg: "name: {{ item.0.projectName }}, namespace: {{ item.1 }}, password: {{ dico[item.0.projectName].secret }}, username: {{ dico[item.0.projectName].username }}" #{{ ansible_loop.index0 }}"
vars:
project: "{{ (robot_result.results | json_query('[*].json')) }}"
loop: "{{ project_namespaces | subelements('Namespaces') }}"
loop_control:
label: "{{ item.0.projectName }}"
result:
ok: [localhost] => (item=helm) => {
"msg": "name: helm, namespace: default, password: YzDDEtJcqYoBL2soZHfTqZxvhIfGKURT, username: robot#helm-robot"
}
ok: [localhost] => (item=helm) => {
"msg": "name: helm, namespace: core, password: YzDDEtJcqYoBL2soZHfTqZxvhIfGKURT, username: robot#helm-robot"
}
ok: [localhost] => (item=proxy) => {
"msg": "name: proxy, namespace: default, password: 7imXCVAGHV91AkeN7LAhWxQHabmYDRmg, username: robot#proxy-robot"
}
ok: [localhost] => (item=proxy) => {
"msg": "name: proxy, namespace: core, password: 7imXCVAGHV91AkeN7LAhWxQHabmYDRmg, username: robot#proxy-robot"
}

Ansible set_fact output

I need the string or destination path as the output and use it in other task but the output displayed is dfifferent.
Here is the register output
ok: [localhost] => {
"msg": {
"changed": true,
"msg": "All items completed",
"results": [
{
"ansible_loop_var": "item",
"changed": true,
"checksum": "b045e5836bbd01d9c6dd2b7426afb5d1c8957b30",
"dest": "/home/ec2-user/bb/Data-1.1.tar.gz",
"failed": false,
"invocation": {
"module_args": {
"_original_basename": null,
"attributes": null,
"backup": false,
"checksum": null,
"content": null,
"delimiter": null,
"dest": "/home/ec2-user/bb/Data-1.1.tar.gz",
"directory_mode": null,
"follow": false,
"force": true,
"group": null,
"local_follow": null,
"mode": null,
"owner": null,
"regexp": null,
"remote_src": true,
"selevel": null,
"serole": null,
"setype": null,
"seuser": null,
"src": "/home/ec2-user/aa/Data-1.1.tar.gz",
"unsafe_writes": null,
"validate": null
}
},
"item": "/home/ec2-user/aa/Data-1.1.tar.gz",
"md5sum": "df9309334454cc3ceac9a6ac8fea8989",
"src": "/home/ec2-user/aa/Data-1.1.tar.gz"
I used the below the task to display the destination path
- set_fact:
filefact: "{{ output.results | map(attribute='item') | string }}"
- debug:
msg: "{{ filefact }}"
The output displayed is
ok: [localhost] => {
"msg": "<generator object do_map at 0x7f59539f01e0>"
i'd write :
- set_fact:
filefact: "{{ filefact | default([]) + [output.results.item] }}"
- debug:
msg: "{{ filefact }}"
the right syntax , following your output should be:
- set_fact:
filefact: "{{ filefact | default([]) + [output.results[0].item] }}"
or
- set_fact:
filefact: "{{ filefact | default([]) + [item.item] }}"
loop: "{{ output.results }}"

Ansible - Find File with a Specific Path and Copy it to a Specific Pre-Set Destination from a Dictionary

I've got a problem that I've been trying to solve for a few days and I'm starting to think I'm just not going to get it. So, here I am.
I have to write an Ansible playbook that will look for a a bunch of files in a bunch of repos and, if it should happen to find that file in those repos, copy it to a specific destination directory which will be pre-set and specific to that file. A catch is that there might be multiple instances of that file but we only want that file if it's under a specific sub-directory in the path (the location of the sub-directory changes from repo to repo and the directory structure of the repos are all pretty different).
So that I don't put you all through our terrible naming conventions, I've abstracted the problem to this kinda silly example.
Let's say that this is our directory structure:
/tmp/example
└── foods
├── breakfast
│   ├── pancakes
│   └── waffles
├── dinner
│   ├── fish
│   └── pasta
└── lunch
├── burger
└── burrito
And this is our dictionary that states the user, the food they want, and the destination path of the food should we find it:
orders:
'Bob':
food: 'burrito'
dst: '/tmp/example-homes/home/bob/plate'
'Jack':
food: 'fish'
dst: '/tmp/example-homes/home/jack/plate'
'Mary':
food: 'fish'
dst: '/tmp/example-homes/mary/plate'
Now, let's say that we want to iterate through the list of people in the orders dictionary and find their food item in /tmp/example, BUT it should only match if their food choice is under the foods/lunch directory. If we happen to find their food in the foods/lunch directory, copy it to that user's specified dst directory (notice that not all of the dst directories are the same). Skip the food if it's found under foods/breakfast or foods/dinner or even something like restaurant/lunch; we only care about foods/lunch. For example, Mary wants fish, and fish does exist, but it's in the foods/dinner directory so we're going to consider it as missing and not copy it.
I've gotten to the point where I can find the food but I'm stuck trying to tie that found food file to the dst field that tells us where the food should go. It's frustrating because I feel that all of the data that I need is actually in the find_food_results dictionary. I just don't know how to act on the results of find so as to perform the logical equivalent of "if food matched, and path contains 'foods/lunch', copy it to item.value.dst".
I also can't help but feel that there's a simpler way to do this and I'm just chasing my tail at this point. Anyhow, thanks a bunch in advance. Please let me know if I can clarify anything.
Here's the code:
---
- name: "directory finder"
hosts: 127.0.0.1
connection: local
vars:
orders:
'Bob':
food: 'burrito'
dst: '/tmp/example-homes/home/bob/plate'
'Jack':
food: 'fish'
dst: '/tmp/example-homes/home/jack/plate'
'Mary':
food: 'fish'
dst: '/tmp/example-homes/mary/plate'
tasks:
- name: "describe orders"
debug:
var: orders
- name: "describe orders | dict2items"
debug:
var: orders | dict2items
- name: "find the food"
find:
paths: "/tmp/example"
recurse: yes
file_type: file
patterns: "{{ item.value.food }}"
with_items:
- "{{ orders | dict2items }}"
register: find_food_results
- name: "describe find_food_results"
debug:
var: find_food_results
- name: "narrow down our findings to paths that contain 'foods/lunch'"
set_fact:
food_lunch_directory_paths: "{{ food_lunch_directory_paths | default([]) }} + [ '{{ item.path }}' ]"
with_items: "{{ find_food_results.results | map(attribute='files') | list }}"
when: "'foods/lunch' in item.path"
- name: "describe our food/lunch paths"
debug:
var: food_lunch_directory_paths
And here's the output:
PLAY [directory finder] *******************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************
ok: [127.0.0.1]
TASK [describe orders] ********************************************************************************************************************************************
ok: [127.0.0.1] => {
"orders": {
"Bob": {
"dst": "/tmp/example-homes/home/bob/plate",
"food": "burrito"
},
"Jack": {
"dst": "/tmp/example-homes/home/jack/plate",
"food": "fish"
},
"Mary": {
"dst": "/tmp/example-homes/mary/plate",
"food": "fish"
}
}
}
TASK [describe orders | dict2items] *******************************************************************************************************************************
ok: [127.0.0.1] => {
"orders | dict2items": [
{
"key": "Bob",
"value": {
"dst": "/tmp/example-homes/home/bob/plate",
"food": "burrito"
}
},
{
"key": "Jack",
"value": {
"dst": "/tmp/example-homes/home/jack/plate",
"food": "fish"
}
},
{
"key": "Mary",
"value": {
"dst": "/tmp/example-homes/mary/plate",
"food": "fish"
}
}
]
}
TASK [find the food] **********************************************************************************************************************************************
ok: [127.0.0.1] => (item={u'key': u'Bob', u'value': {u'food': u'burrito', u'dst': u'/tmp/example-homes/home/bob/plate'}})
ok: [127.0.0.1] => (item={u'key': u'Jack', u'value': {u'food': u'fish', u'dst': u'/tmp/example-homes/home/jack/plate'}})
ok: [127.0.0.1] => (item={u'key': u'Mary', u'value': {u'food': u'fish', u'dst': u'/tmp/example-homes/mary/plate'}})
TASK [describe find_food_results] *********************************************************************************************************************************
ok: [127.0.0.1] => {
"find_food_results": {
"changed": false,
"msg": "All items completed",
"results": [
{
"ansible_loop_var": "item",
"changed": false,
"examined": 10,
"failed": false,
"files": [
{
"atime": 1604184466.0584693,
"ctime": 1604184466.0584693,
"dev": 66305,
"gid": 0,
"gr_name": "root",
"inode": 58724223,
"isblk": false,
"ischr": false,
"isdir": false,
"isfifo": false,
"isgid": false,
"islnk": false,
"isreg": true,
"issock": false,
"isuid": false,
"mode": "0644",
"mtime": 1604184466.0584693,
"nlink": 1,
"path": "/tmp/example/foods/lunch/burrito",
"pw_name": "root",
"rgrp": true,
"roth": true,
"rusr": true,
"size": 0,
"uid": 0,
"wgrp": false,
"woth": false,
"wusr": true,
"xgrp": false,
"xoth": false,
"xusr": false
}
],
"invocation": {
"module_args": {
"age": null,
"age_stamp": "mtime",
"contains": null,
"depth": null,
"excludes": null,
"file_type": "file",
"follow": false,
"get_checksum": false,
"hidden": false,
"paths": [
"/tmp/example"
],
"patterns": [
"burrito"
],
"recurse": true,
"size": null,
"use_regex": false
}
},
"item": {
"key": "Bob",
"value": {
"dst": "/tmp/example-homes/home/bob/plate",
"food": "burrito"
}
},
"matched": 1,
"msg": ""
},
{
"ansible_loop_var": "item",
"changed": false,
"examined": 10,
"failed": false,
"files": [
{
"atime": 1604184480.3713806,
"ctime": 1604184480.3713806,
"dev": 66305,
"gid": 0,
"gr_name": "root",
"inode": 62917805,
"isblk": false,
"ischr": false,
"isdir": false,
"isfifo": false,
"isgid": false,
"islnk": false,
"isreg": true,
"issock": false,
"isuid": false,
"mode": "0644",
"mtime": 1604184480.3713806,
"nlink": 1,
"path": "/tmp/example/foods/dinner/fish",
"pw_name": "root",
"rgrp": true,
"roth": true,
"rusr": true,
"size": 0,
"uid": 0,
"wgrp": false,
"woth": false,
"wusr": true,
"xgrp": false,
"xoth": false,
"xusr": false
}
],
"invocation": {
"module_args": {
"age": null,
"age_stamp": "mtime",
"contains": null,
"depth": null,
"excludes": null,
"file_type": "file",
"follow": false,
"get_checksum": false,
"hidden": false,
"paths": [
"/tmp/example"
],
"patterns": [
"fish"
],
"recurse": true,
"size": null,
"use_regex": false
}
},
"item": {
"key": "Jack",
"value": {
"dst": "/tmp/example-homes/home/jack/plate",
"food": "fish"
}
},
"matched": 1,
"msg": ""
},
{
"ansible_loop_var": "item",
"changed": false,
"examined": 10,
"failed": false,
"files": [
{
"atime": 1604184480.3713806,
"ctime": 1604184480.3713806,
"dev": 66305,
"gid": 0,
"gr_name": "root",
"inode": 62917805,
"isblk": false,
"ischr": false,
"isdir": false,
"isfifo": false,
"isgid": false,
"islnk": false,
"isreg": true,
"issock": false,
"isuid": false,
"mode": "0644",
"mtime": 1604184480.3713806,
"nlink": 1,
"path": "/tmp/example/foods/dinner/fish",
"pw_name": "root",
"rgrp": true,
"roth": true,
"rusr": true,
"size": 0,
"uid": 0,
"wgrp": false,
"woth": false,
"wusr": true,
"xgrp": false,
"xoth": false,
"xusr": false
}
],
"invocation": {
"module_args": {
"age": null,
"age_stamp": "mtime",
"contains": null,
"depth": null,
"excludes": null,
"file_type": "file",
"follow": false,
"get_checksum": false,
"hidden": false,
"paths": [
"/tmp/example"
],
"patterns": [
"fish"
],
"recurse": true,
"size": null,
"use_regex": false
}
},
"item": {
"key": "Mary",
"value": {
"dst": "/tmp/example-homes/mary/plate",
"food": "fish"
}
},
"matched": 1,
"msg": ""
}
]
}
}
TASK [narrow down our findings to paths that contain 'foods/lunch'] ***********************************************************************************************
ok: [127.0.0.1] => (item={u'rusr': True, u'uid': 0, u'rgrp': True, u'xoth': False, u'islnk': False, u'woth': False, u'nlink': 1, u'issock': False, u'mtime': 1604184466.0584693, u'gr_name': u'root', u'path': u'/tmp/example/foods/lunch/burrito', u'xusr': False, u'atime': 1604184466.0584693, u'inode': 58724223, u'isgid': False, u'size': 0, u'isdir': False, u'wgrp': False, u'ctime': 1604184466.0584693, u'isblk': False, u'xgrp': False, u'isuid': False, u'dev': 66305, u'roth': True, u'isreg': True, u'isfifo': False, u'mode': u'0644', u'pw_name': u'root', u'gid': 0, u'ischr': False, u'wusr': True})
skipping: [127.0.0.1] => (item={u'rusr': True, u'uid': 0, u'rgrp': True, u'xoth': False, u'islnk': False, u'woth': False, u'nlink': 1, u'issock': False, u'mtime': 1604184480.3713806, u'gr_name': u'root', u'path': u'/tmp/example/foods/dinner/fish', u'xusr': False, u'atime': 1604184480.3713806, u'inode': 62917805, u'isgid': False, u'size': 0, u'isdir': False, u'wgrp': False, u'ctime': 1604184480.3713806, u'isblk': False, u'xgrp': False, u'isuid': False, u'dev': 66305, u'roth': True, u'isreg': True, u'isfifo': False, u'mode': u'0644', u'pw_name': u'root', u'gid': 0, u'ischr': False, u'wusr': True})
skipping: [127.0.0.1] => (item={u'rusr': True, u'uid': 0, u'rgrp': True, u'xoth': False, u'islnk': False, u'woth': False, u'nlink': 1, u'issock': False, u'mtime': 1604184480.3713806, u'gr_name': u'root', u'path': u'/tmp/example/foods/dinner/fish', u'xusr': False, u'atime': 1604184480.3713806, u'inode': 62917805, u'isgid': False, u'size': 0, u'isdir': False, u'wgrp': False, u'ctime': 1604184480.3713806, u'isblk': False, u'xgrp': False, u'isuid': False, u'dev': 66305, u'roth': True, u'isreg': True, u'isfifo': False, u'mode': u'0644', u'pw_name': u'root', u'gid': 0, u'ischr': False, u'wusr': True})
TASK [describe our food/lunch paths] ******************************************************************************************************************************
ok: [127.0.0.1] => {
"food_lunch_directory_paths": [
"/tmp/example/foods/lunch/burrito"
]
}
PLAY RECAP ********************************************************************************************************************************************************
127.0.0.1 : ok=7 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Let's take into account missing files. For example, given the orders
orders:
'Bob':
food: 'burrito'
dst: '/tmp/example-homes/home/bob/plate'
'Jack':
food: 'fish'
dst: '/tmp/example-homes/home/jack/plate'
'Mary':
food: 'fish'
dst: '/tmp/example-homes/mary/plate'
'Joe':
food: 'steak'
dst: '/tmp/example-homes/joe/plate'
Create the list of the foods
- name: "create list of foods"
set_fact:
foods: "{{ orders|dict2items|
map(attribute='value.food')|unique|sort|list }}"
- debug:
var: foods
gives
foods:
- burrito
- fish
- steak
Find the files and create a dictionary of foods and related files. Then use the dictionary to copy the existing files to destinations
- name: "find the foods"
find:
paths: "/tmp/example"
recurse: yes
file_type: file
patterns: "{{ item }}"
loop: "{{ foods }}"
register: find_food_results
- name: "create dictionary of foods"
set_fact:
foods: "{{ dict(foods|zip(paths)) }}"
vars:
paths: "{{ find_food_results.results|
map(attribute='files')|list }}"
- name: "copy property files to destination"
debug:
msg: "Copy {{ foods[item.value.food][0]['path'] }} to {{ item.value.dst }}"
with_dict: "{{ orders }}"
loop_control:
label: "{{ item.key }}"
when: foods[item.value.food]|length > 0
give
ok: [localhost] => (item=Bob) =>
msg: Copy /tmp/example/foods/lunch/burrito to /tmp/example-homes/home/bob/plate
ok: [localhost] => (item=Jack) =>
msg: Copy /tmp/example/foods/dinner/fish to /tmp/example-homes/home/jack/plate
ok: [localhost] => (item=Mary) =>
msg: Copy /tmp/example/foods/dinner/fish to /tmp/example-homes/mary/plate
skipping: [localhost] => (item=Joe)
Q: "File 'burrito' in the breakfast, lunch, and dinner directories ... How can I iterate over ... find_food_results.files? Almost like a nested loop."
A: Display the dictionary foods with selected paths only. The task
- debug:
msg: "{{ msg.split('\n')[:-1] }}"
vars:
msg: |
{{ item.key }}
{{ item.value|map(attribute='path')|list|to_nice_yaml }}
loop: "{{ foods|dict2items }}"
loop_control:
label: "{{ item.key }}"
gives
TASK [debug] ****
ok: [localhost] => (item=burrito) =>
msg:
- burrito
- '- /tmp/example/foods/breakfast/burrito'
- '- /tmp/example/foods/dinner/burrito'
- '- /tmp/example/foods/lunch/burrito'
ok: [localhost] => (item=fish) =>
msg:
- fish
- '- /tmp/example/foods/dinner/fish'
ok: [localhost] => (item=steak) =>
msg:
- steak
- '[]'
Use subelements if you want to iterate the lists of the files. For example
- name: "List all property files"
debug:
msg: "{{ item.0.key }} {{ item.1.path }}"
with_subelements:
- "{{ foods|dict2items }}"
- value
loop_control:
label: "{{ item.0.key }}"
gives
TASK [List all property files] ****
ok: [localhost] => (item=burrito) =>
msg: burrito /tmp/example/foods/breakfast/burrito
ok: [localhost] => (item=burrito) =>
msg: burrito /tmp/example/foods/dinner/burrito
ok: [localhost] => (item=burrito) =>
msg: burrito /tmp/example/foods/lunch/burrito
ok: [localhost] => (item=fish) =>
msg: fish /tmp/example/foods/dinner/fish
And of course as soon as I ask the question, I find the answer to it :-/
Well, in case anyone else was struggling with this, here's how I fixed it. Though, I'd love to see more efficient solutions, so please do chime in if you have one.
I was able to create a new value in the dictionary called "src" that contained the path of the file found using the find module. Then, in the copy module, I was able to iterate over the dictionary and only copy when the src contained 'foods/lunch'.
Here's the code:
---
- name: "directory finder"
hosts: 127.0.0.1
connection: local
vars:
orders:
'Bob':
food: 'burrito'
dst: '/tmp/example-homes/home/bob/plate'
'Jack':
food: 'fish'
dst: '/tmp/example-homes/home/jack/plate'
'Mary':
food: 'fish'
dst: '/tmp/example-homes/mary/plate'
tasks:
- name: "describe orders"
debug:
var: orders
- name: "describe orders | dict2items"
debug:
var: orders | dict2items
- name: "find the food"
find:
paths: "/tmp/example"
recurse: yes
file_type: file
patterns: "{{ item.value.food }}"
with_items:
- "{{ orders | dict2items }}"
register: find_food_results
- name: "create a src key/value dictionary element"
set_fact:
orders: "{{ orders | combine( new_item, recursive=true ) }}"
vars:
new_item: "{ '{{ item.item.key }}': { 'src': '{{ item.files[0].path }}' } }"
with_items: "{{ find_food_results.results }}"
- name: "copy property files to destination"
copy:
src: "{{ item.value.src }}"
dest: "{{ item.value.dst }}"
owner: root
group: root
mode: 0644
with_dict: "{{ orders }}"
when: "'foods/lunch' in item.value.src"

Ansible: loop our shell command(Linux user group search) result and display the groups

Ansible: loop our shell command(Linux user group search) result and display the groups
Task to check if group exist
- name: "Checking if group doesn't exist"
shell: "grep -i {{ item.group }} /etc/group"
register: presence
loop: "{{ UserAddList.add_users }}"
ignore_errors: true
no_log: true
Json input file:
UserAddList is a json file
{
"add_users": [
{
"name": "test1_123",
"group": "test1_123",
"additional_groups":
[
"test2",
"group1"
],
"password" : "test1_newcdsaf",
"sudo_entry": "ALL=(ALL) NOPASSWD: ALL",
"comment": "test1"
}
],
"delete_users": [
]
}
Task to display the groups doesn't exist
- name: The following groups does't' exist
debug:
msg:
"{{ item._ansible_item_label.group }}"
loop: "{{ presence.results }}"
output:
(item={
'_ansible_parsed': True,
'stderr_lines': [
],
u'changed': True,
u'stdout': u'',
'_ansible_item_result': True,
u'msg': u'non-zero return code',
u'delta': u'0:00:00.008175',
'stdout_lines': [
],
'_ansible_item_label': {
u'comment': u'test1',
u'password': u'test1_newcdsaf',
u'group': u'test1_123',
u'name': u'test1_123',
u'sudo_entry': u'ALL=(ALL) NOPASSWD: ALL',
u'additional_groups': [
u'test2',
u'group1'
]
},
u'end': u'2019-12-10 14:23:15.725676',
'_ansible_no_log': True,
'item': {
u'comment': u'test1',
u'password': u'test1_newcdsaf',
u'group': u'test1_123',
u'name': u'test1_123',
u'sudo_entry': u'ALL=(ALL) NOPASSWD: ALL',
u'additional_groups': [
u'test2',
u'group1'
]
},
u'cmd': u'grep -i test1_123 /etc/group',
u'failed': True,
u'stderr': u'',
u'rc': 1,
u'invocation': {
u'module_args': {
u'warn': True,
u'executable': None,
u'_uses_shell': True,
u'_raw_params': u'grep -i test1_123 /etc/group',
u'removes': None,
u'argv': None,
u'creates': None,
u'chdir': None,
u'stdin': None
}
},
u'start': u'2019-12-10 14:23:15.717501'
})=>{
"changed": false,
"item": {
"changed": true,
"cmd": "grep -i test1_123 /etc/group",
"delta": "0:00:00.008175",
"end": "2019-12-10 14:23:15.725676",
"failed": true,
"invocation": {
"module_args": {
"_raw_params": "grep -i test1_123 /etc/group",
"_uses_shell": true,
"argv": null,
"chdir": null,
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"warn": true
}
},
"item": {
"additional_groups": [
"test2",
"group1"
],
"comment": "test1",
"group": "test1_123",
"name": "test1_123",
"password": "test1_newcdsaf",
"sudo_entry": "ALL=(ALL) NOPASSWD: ALL"
},
"msg": "non-zero return code",
"rc": 1,
"start": "2019-12-10 14:23:15.717501",
"stderr": "",
"stderr_lines": [
],
"stdout": "",
"stdout_lines": [
]
},
"msg": "test1_123"
}
I don't want to display the whole output, I just want to display the groups information.
The debug is printing all the input data as well.
Please any suggestions
The debug is printing all the input data as well.
It's actually not the debug: task that is printing your data, ansible is showing you what it is looping over. However, you are using loop: over the top-level presence.results list, and .results contains not only the output you care about, but also the invocation parameters, success or failure, and the actual returned data that you care about
There are two ways of fixing that problem: tell ansible that you only want it to show something smaller in the loop label, or change the loop: to actually only loop over the deleted users
In the first way, loop_control: will do that (it even cites your exact circumstance in the docs saying When looping over complex data structures, the console output of your task can be enormous. To limit the displayed output, use the label directive with loop_control):
- name: The following groups does't' exist
debug:
msg:
"{{ item._ansible_item_label.group }}"
loop: "{{ presence.results }}"
loop_control:
label: "{{ item.item.name }}"
In the second way, just select out the group you care about and msg: it:
- name: The following groups does't' exist
debug:
msg:
"{{ item }}"
loop: "{{ presence.results | map(attribute='item') | map(attribute='group') | list }}"

Resources