Nested loop with user and folder in Ansible - ansible

I have the following task:
- name: Create required folders.
become: true
ansible.builtin.file:
owner: "{{ item.key }}"
group: ftp
mode: '0755'
path: '/data/{{ item.key }}/in'
state: directory
loop: "{{ query('dict', ftp) | list }}"
when: "'state' not in item.value or item.value.state == 'present'"
And the following host variables with different users:
ftp:
test:
ssh_public_key: "XXXX"
password: "XXX"
home: /data/test
test2:
ssh_public_key: "XXXX"
password: "XXX"
home: /data/test2
What I want is to create two directories for every user :
path: '/data/{{ user }}/in' # item.key, in the code above
path: '/data/{{ user }}/out' # item.key, in the code above
But I already need the loop for iterating over the users itself:
loop: "{{ query('dict', ftp) | list }}"
How can I handle this, for example, with nested loop?

Use a product filter to generate every possible combination of user/folder.
loop: "{{ ftp.keys() | product(['in', 'out']) }}"
Then, respectively,
item.0 contains the users dictionary keys
item.1 contains the folders
It is not fully clear what your condition when does actually, in order to adapt it too, but I guess that you do have an absent or present state in those use dictionaries.
So, the resulting task should be something along the lines of
- name: Create required folders
ansible.builtin.file:
owner: "{{ item.0 }}"
group: ftp
mode: '0755'
path: "/data/{{ item.0 }}/{{ item.1 }}"
state: directory
loop: "{{ ftp.keys() | product(['in', 'out']) }}"
loop_control:
label: "/data/{{ item.0 }}/{{ item.1 }}"
when: "ftp[item.0].state | default('absent') == 'present'"
become: true
Given the task above, when run on those data:
ftp:
test:
state: present
test1:
test2:
state: present
It will yield:
TASK [Create required folders] ***************************************
ok: [localhost] => (item=/data/test/in)
ok: [localhost] => (item=/data/test/out)
skipping: [localhost] => (item=/data/test1/in)
skipping: [localhost] => (item=/data/test1/out)
ok: [localhost] => (item=/data/test2/in)
ok: [localhost] => (item=/data/test2/out)

Test it first, for example
- debug:
msg: "Create /data/{{ item.0.key }}/{{ item.1 }}"
with_nested:
- "{{ ftp|dict2items }}"
- [in, out]
when: item.0.value.state|d('present') == 'present'
gives (abridged)
msg: Create /data/test/in
msg: Create /data/test/out
msg: Create /data/test2/in
msg: Create /data/test2/out
Then try to create the dictionaries
- file:
owner: "{{ item.0.key }}"
group: ftp
mode: '0755'
path: "/data/{{ item.0.key }}/{{ item.1 }}"
state: directory
with_nested:
- "{{ ftp|dict2items }}"
- [in, out]
when: item.0.value.state|d('present') == 'present'
(not tested)

Related

ansible.builtin.file is using user from wrong host

I have this playbook below to set user/group on the user's home directory.
jimbo and bobo here have different UID and GIDs on the different boxes.
Running this script will set the UID/GID ownership of the directories incorrectly.
For example, it will set /home/jimbo on operatorbox1 (1) to be owned by the UID of jimbo from operatorbox2 (2) - which is of course not the correct UID on operatorbox1 (1).
It does this seemingly randomly. If I run this playbook multiple times the ownership of the directories will flip back and forth.
Guessing I have something fundamental missing here. Why is this happening? Thanks!
ansible-playbook v2.9.23
./vars/operators.yml
---
operators:
jimbo: sshekeywhatever
bobo: sshkeywhatever
playbook.yml
---
- name: Setup operators
hosts:
- bastionbox
- operatorbox
become: true
vars_files:
- "./vars/operators.yml"
tasks:
- name: Set home directory permissions
file:
path: "/home/{{ item.key }}"
state: directory
owner: "{{ item.key }}"
group: "{{ item.key }}"
recurse: true
with_dict:
- "{{ operators }}"
I can't reproduce the problem. Below is a playbook for testing
- hosts: bastionbox,operatorbox
gather_facts: false
become: true
vars:
operators: [jimbo, bobo]
tasks:
- name: Create users
user:
name: "{{ item }}"
shell: /usr/sbin/nologin
uid: "{{ range(2500, 2600)|random }}"
loop: "{{ operators }}"
when: create_users|d(false)|bool
- name: List users uid
block:
- getent:
database: passwd
- debug:
msg: "{{ inventory_hostname }} {{ item }} uid: {{ getent_passwd[item].1 }}"
loop: "{{ operators }}"
when: list_users|d(false)|bool
- name: Set home directory owner and group
file:
state: directory
path: "/home/{{ item }}"
owner: "{{ item }}"
group: "{{ item }}"
recurse: true
loop: "{{ operators }}"
when: set_homes|d(false)|bool
- name: List homes
block:
- find:
paths: /home
file_type: directory
patterns: "{{ operators }}"
register: out
- debug:
msg: "{{ inventory_hostname }} {{ item.path }} uid: {{ item.uid }}"
loop: "{{ out.files }}"
loop_control:
label: "{{ inventory_hostname }}"
when: list_homes|d(false)|bool
- name: Delete users
user:
name: "{{ item }}"
state: absent
remove: true
loop: "{{ operators }}"
when: delete_users|d(false)|bool
Create users
shell> ansible-playbook -e create_users=true pb.yml
List users
shell> ansible-playbook -e list_users=true pb.yml
msg: 'bastionbox jimbo uid: 2572'
msg: 'operatorbox jimbo uid: 2537'
msg: 'bastionbox bobo uid: 2505'
msg: 'operatorbox bobo uid: 2557'
List homes
shell> ansible-playbook -e list_homes=true pb.yml
msg: 'bastionbox /home/bobo uid: 2505'
msg: 'operatorbox /home/jimbo uid: 2537'
msg: 'bastionbox /home/jimbo uid: 2572'
msg: 'operatorbox /home/bobo uid: 2557'
Set homes (task is idempotent)
shell> ansible-playbook -e set_homes=true pb.yml
TASK [Set home directory owner and group] *************************
ok: [operatorbox] => (item=jimbo)
ok: [bastionbox] => (item=jimbo)
ok: [operatorbox] => (item=bobo)
ok: [bastionbox] => (item=bobo)

Ansible - Looking for files and compare their hash

Practice:
I have the file files.yml with a list of files and their respective md5_sum hash, like:
files:
- name: /opt/file_compare1.tar
hash: 9cd599a3523898e6a12e13ec787da50a /opt/file_compare1.tar
- name: /opt/file_compare2tar.gz
hash: d41d8cd98f00b204e9800998ecf8427e /opt/file_compare2.tar.gz
I need to create a playbook to check this list of files if the current hash is the same or if it was changed, the playbook should have a debug message like below:
---
- hosts: localhost
connection: local
vars_files:
- files.yml
tasks:
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
with_items:
- "{{ files }}"
- name: Debug files - Different
debug:
msg: |
"Hash changed: {{ item.name }}"
when:
- item.hash != hash_check
with_items:
- "{{ files }}"
- name: Debug files - Equal
debug:
msg: |
"Hash NOT changed: {{ item.name }}"
when:
- item.hash == hash_check
with_items:
- "{{ files }}"
- debug:
msg: |
- "{{ hash_check }} {{ item.name }}"
with_items:
- "{{ files }}"
For example, given the files
files:
- name: /scratch/file_compare1.tar
hash: 4f8805b4b64dcc575547ec1c63793aec /scratch/file_compare1.tar
- name: /scratch/file_compare2.tar.gz
hash: 2dc4f1e9ca4081cc49d25195627982ef /scratch/file_compare2.tar.gz
the tasks below
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
loop: "{{ files }}"
- name: Debug files - Different
debug:
msg: |
Hash NOT changed: {{ item.0.name }}
{{ item.0.hash.split()|first }}
{{ item.1 }}
with_together:
- "{{ files }}"
- "{{ hash_check.results|map(attribute='stat.checksum')|list }}"
when: item.0.hash.split()|first == item.1
give
msg: |-
Hash NOT changed: /scratch/file_compare1.tar
4f8805b4b64dcc575547ec1c63793aec
4f8805b4b64dcc575547ec1c63793aec
msg: |-
Hash NOT changed: /scratch/file_compare2.tar.gz
2dc4f1e9ca4081cc49d25195627982ef
2dc4f1e9ca4081cc49d25195627982ef
A more robust option would be to create a dictionary with the calculated hashes
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
loop: "{{ files }}"
- set_fact:
path_hash: "{{ dict(_path|zip(_hash)) }}"
vars:
_path: "{{ hash_check.results|map(attribute='stat.path')|list }}"
_hash: "{{ hash_check.results|map(attribute='stat.checksum')|list }}"
gives
path_hash:
/scratch/file_compare1.tar: 4f8805b4b64dcc575547ec1c63793aec
/scratch/file_compare2.tar.gz: 2dc4f1e9ca4081cc49d25195627982ef
Then use this dictionary to compare the hashes. For example, the task below gives the same results
- name: Debug files - Different
debug:
msg: |
Hash NOT changed: {{ item.name }}
{{ item.hash.split()|first }}
{{ path_hash[item.name] }}
loop: "{{ files }}"
when: item.hash.split()|first == path_hash[item.name]
The next option is to create a dictionary with the original hashes and both lists of original and calculated hashes
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
loop: "{{ files }}"
- set_fact:
hash_name: "{{ dict(_hash|zip(_name)) }}"
hash_orig: "{{ _hash }}"
hash_stat: "{{ hash_check.results|map(attribute='stat.checksum')|list }}"
vars:
_hash: "{{ files|map(attribute='hash')|map('split')|map('first')|list }}"
_name: "{{ files|map(attribute='name')|list }}"
gives
hash_name:
2dc4f1e9ca4081cc49d25195627982ef: /scratch/file_compare2.tar.gz
4f8805b4b64dcc575547ec1c63793aec: /scratch/file_compare1.tar
hash_orig:
- 4f8805b4b64dcc575547ec1c63793aec
- 2dc4f1e9ca4081cc49d25195627982ef
hash_stat:
- 4f8805b4b64dcc575547ec1c63793aec
- 2dc4f1e9ca4081cc49d25195627982ef
Then calculate the difference of the lists and use it to extract both lists of changed and unchanged files
- set_fact:
files_diff: "{{ _diff|map('extract', hash_name)|list }}"
files_orig: "{{ _orig|map('extract', hash_name)|list }}"
vars:
_diff: "{{ hash_orig|difference(hash_stat) }}"
_orig: "{{ hash_orig|difference(_diff) }}"
- name: Debug files changed
debug:
var: files_diff
- name: Debug files NOT changed
debug:
var: files_orig
gives
files_diff: []
files_orig:
- /scratch/file_compare1.tar
- /scratch/file_compare2.tar.gz
I used your suggestion to complement the playbook, it's working now.
The idea is to get a list of files, read each one and compare with both hash, file, and current hash.
---
- hosts: localhost
connection: local
gather_facts: false
vars_files:
- files3.yml
tasks:
- stat:
path: "{{ item.file }}"
checksum_algorithm: md5
loop: "{{ files }}"
register: stat_results
- name: NOT changed files
debug:
msg: "NOT changed: {{ item.stat.path }}"
when: item.stat.checksum == item.item.checksum.split()|first
loop: "{{ stat_results.results }}"
loop_control:
label: "{{ item.stat.path }}"
- name: Changed files
debug:
msg: "CHANGED: {{ item.stat.path }}"
when: item.stat.checksum != item.item.checksum.split()|first
loop: "{{ stat_results.results }}"
loop_control:
label: "{{ item.stat.path }}"
Result:
>> ansible-playbook playbooks/check-file3.yml
PLAY [localhost] ********************************************************************************************************************************************************************************************************************
TASK [stat] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'file': '/opt/file_compare1.tar', 'checksum': '9cd599a3523898e6a12e13ec787da50a /opt/file_compare1.tar'})
ok: [localhost] => (item={'file': '/opt/file_compare2.tar.gz', 'checksum': 'd41d8cd98f00b204e9800998ecf8427e /opt/file_compare2.tar.gz'})
TASK [NOT changed files] ************************************************************************************************************************************************************************************************************
skipping: [localhost] => (item=/opt/file_compare1.tar)
ok: [localhost] => (item=/opt/file_compare2.tar.gz) => {
"msg": "NOT changed: /opt/file_compare2.tar.gz"
}
TASK [Changed files] ****************************************************************************************************************************************************************************************************************
ok: [localhost] => (item=/opt/file_compare1.tar) => {
"msg": "CHANGED: /opt/file_compare1.tar"
}
skipping: [localhost] => (item=/opt/file_compare2.tar.gz)
PLAY RECAP **************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Ansible when condition registered from csv

I'm using csv file as ingest data for my playbooks, but im having trouble with my when condition. it's either both task will skipped or both task will be ok, my objective is if ansible see the string in when condition it will skipped for the specific instance.
here is my playbook
- name: "Read ingest file from CSV return a list"
community.general.read_csv:
path: sample.csv
register: ingest
- name: debug ingest
debug:
msg: "{{ item.AWS_ACCOUNT }}"
with_items:
- "{{ ingest.list }}"
register: account
- name: debug account
debug:
msg: "{{ account.results | map(attribute='msg') }}"
register: accountlist
- name:
become: yes
become_user: awx
delegate_to: localhost
environment: "{{ proxy_env }}"
block:
- name: "Assume role"
community.aws.sts_assume_role:
role_arn: "{{ item.ROLE_ARN }}"
role_session_name: "pm"
with_items:
- "{{ ingest.list }}"
register: assumed_role
when: "'aws-account-rnd' not in account.results | map(attribute='msg')"
here is the content of sample.csv
HOSTNAME
ENVIRONMENT
AWS_ACCOUNT
ROLE_ARN
test1
dev
aws-account-rnd
arn:aws:iam::XXXX1
test2
uat
aws-account-uat
arn:aws:iam::XXXX2
my objective is to skipped all items in the csv file with aws-acount-rnd
Your condition does not mention item so it will have the same result for all loop items.
Nothing you've shown requires the weird abuse of debug + register that you're doing, and it is in fact getting in your way.
- name: Read CSV file
community.general.read_csv:
path: sample.csv
register: ingest
- name: Assume role
community.aws.sts_assume_role:
role_arn: "{{ item.ROLE_ARN }}"
role_session_name: pm
delegate_to: localhost
become: true
become_user: awx
environment: "{{ proxy_env }}"
loop: "{{ ingest.list }}"
when: item.AWS_ACCOUNT != 'aws-account-rnd'
register: assumed_role
If you'll always only care about one match you can also do this without a loop or condition at all:
- name: Assume role
community.aws.sts_assume_role:
role_arn: "{{ ingest.list | rejectattr('AWS_ACCOUNT', '==', 'aws-account-rnd') | map(attribute='ROLE_ARN') | first }}"
role_session_name: pm
delegate_to: localhost
become: true
become_user: awx
environment: "{{ proxy_env }}"
register: assumed_role
my objective is to skipped all items in the csv file with aws-acount-rnd
The multiple debug you have with register, seems to be a long-winded approach IMHO.
A simple task to debug the Role ARN, only if the account does not match aws-acount-rnd.
- name: show ROLE_ARN when account not equals aws-account-rnd
debug:
var: item['ROLE_ARN']
loop: "{{ ingest.list }}"
when: item['AWS_ACCOUNT'] != 'aws-account-rnd'
This results in:
TASK [show ROLE_ARN when account not equals aws-account-rnd] **********************************************************************************************************************
skipping: [localhost] => (item={'HOSTNAME': 'test1', 'ENVIRONMENT': 'dev', 'AWS_ACCOUNT': 'aws-account-rnd', 'ROLE_ARN': 'arn:aws:iam:XXXX1'})
ok: [localhost] => (item={'HOSTNAME': 'test2', 'ENVIRONMENT': 'uat', 'AWS_ACCOUNT': 'aws-account-uat', 'ROLE_ARN': 'arn:aws:iam:XXXX2'}) => {
"ansible_loop_var": "item",
"item": {
"AWS_ACCOUNT": "aws-account-uat",
"ENVIRONMENT": "uat",
"HOSTNAME": "test2",
"ROLE_ARN": "arn:aws:iam:XXXX2"
},
"item['ROLE_ARN']": "arn:aws:iam:XXXX2"
}
The same logic can be used to pass the item.ROLE_ARN to community.aws.sts_assume_role task.

How do I get the file creation time with ansible on MacOS?

I am trying to list a set of directories with their creation times in an ansible playbook on MacOS.
The problem I have is that the file and stat modules give you the "ctime" of a file, but on MacOS this is the time of last metadata change, not the creation date.
So for example this playbook:
- name: Get dirs and create dates
hosts: localhost
tasks:
- name: make a list of dirs
find:
paths:
- "/Users/me/Work/Customers"
file_type: directory
recurse: false
register: projectdirs
- name: Dump found paths
debug:
msg: "{{ shortname }} {{ date }}"
loop: "{{ projectdirs.files }}"
loop_control:
label: "{{ item.path }}"
vars:
shortname: "{{ item.path | basename }}"
date: "{{ '%Y-%m-%d' | strftime(item.ctime) }}"
This just lists todays date as "creation" date for the directory.
ok: [localhost] => (item=/Users/me/Work/Customers/Base) => {
"msg": "basefarm 2021-08-23 1629699678.021843"
}
ok: [localhost] => (item=/Users/me/Work/Customers/Acme) => {
"msg": "Orange 2021-08-23 1629699678.5415485"
}
ok: [localhost] => (item=/Users/me/Work/Customers/Foo) => {
"msg": "VKB 2021-08-23 1629699679.0579438"
}
ok: [localhost] => (item=/Users/me/Work/Customers/Bar) => {
"msg": "LombardOdier 2021-08-23 1629699679.5856457"
}
...
I found out that the reason is of course that ctime is not the creation time on a mac...
Is there a way around this? I want the real creation date, but Ansible's stat module does not give me this. The creation date is available, as I can get it with ls -ldU for example.
So how do I solve this?
As far as I know, there is no way to do so without using shell command.
- name: make a list of dirs
find:
paths:
- "/Users/me/Work/Customers"
file_type: directory
recurse: false
register: projectdirs
- name: Get creation time
shell: GetFileInfo {{ item.path }} | grep 'created:'
loop: "{{ projectdirs.files }}"
loop_control:
label: "{{ item.path }}"
register: directories
- name: Dump found paths
debug:
msg: "{{ item.item.path }} {{ item.stdout.split('created: ')[1] }}"
loop: "{{ directories.results }}"
loop_control:
label: "{{ item.item.path }}"
Gives me this result :
ok: [localhost] => (item=/Users/jeromeverdoni/stackoverflow/68888321) => {
"msg": "/Users/jeromeverdoni/stackoverflow/68888321 08/23/2021 08:01:44"
}
Hope this is close enough to your expected behaviour.
For reference, GetFileInfo output :
╰─ GetFileInfo /Users/jeromeverdoni/stackoverflow/68888321
directory: "/Users/jeromeverdoni/stackoverflow/68888321"
attributes: avbstclinmedz
created: 08/23/2021 08:01:44
modified: 08/23/2021 08:03:18

`with_items` output is too verbose

I wrote an ansible task to iterate over a list of settings using with_items. Now all my settings are logged when I run ansible. It is very verbose and makes it hard to see what is happening. But, if I disable all the output with no_log, I will have no way to identify specific items when they fail.
How could the output be improved — to show only an identifier for each item?
Example task:
- authorized_key:
user: "{{ item.user }}"
key: "{{ item.key }}"
with_items: "{{ ssh_keys }}"
Example output:
TASK [sshkey-alan-sysop : ssh authorized keys] *********************************
ok: [brick] => (item={u'user': u'alan-sysop', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDAgRe16yLZa8vbzsrxUpT5MdHoEYYd/awAnEWML4g+YoUvLDKr+zwmu78ze/E1NSipoZejXpggUYRVhh8MOiCX6qpUguBDWZFlvSCE/7uXWWg7Oht0f1kDS2xU7YiycPIzMN1dmUEFY9AixnN936Dq6nOtEzgBwjo66I1YC/5jrsQEqF19shx43A4DTFlPUz/PnsqHl2ESrkIk3e8zyidaPN2pRbA5iKzdvPW4E2W2tKw9ll40vqRXzaWIF7v293Ostwi1IPi2erlC777DhjZUhZ1VGXIR7FDAfANzalrMe6c/ZysiXewiUYgMw0I8Dh1LK3QMj9Kuo35S5E0Xj3TB alan-sysop#alan-laptop'})
There's loop_control for that:
- authorized_key:
user: "{{ item.user }}"
key: "{{ item.key }}"
with_items: "{{ ssh_keys }}"
loop_control:
label: "{{ item.user }}"
The identifiers can be used as keys of a dictionary.
- authorized_key:
user: "{{ item }}"
key: "{{ ssh_keys[item] }}"
with_items: "{{ ssh_keys.keys() }}"
Example output:
TASK [sshkey-alan-sysop : ssh authorized keys] *********************************
ok: [brick] => (item=alan-sysop)

Resources