How do I get the file creation time with ansible on MacOS? - macos

I am trying to list a set of directories with their creation times in an ansible playbook on MacOS.
The problem I have is that the file and stat modules give you the "ctime" of a file, but on MacOS this is the time of last metadata change, not the creation date.
So for example this playbook:
- name: Get dirs and create dates
hosts: localhost
tasks:
- name: make a list of dirs
find:
paths:
- "/Users/me/Work/Customers"
file_type: directory
recurse: false
register: projectdirs
- name: Dump found paths
debug:
msg: "{{ shortname }} {{ date }}"
loop: "{{ projectdirs.files }}"
loop_control:
label: "{{ item.path }}"
vars:
shortname: "{{ item.path | basename }}"
date: "{{ '%Y-%m-%d' | strftime(item.ctime) }}"
This just lists todays date as "creation" date for the directory.
ok: [localhost] => (item=/Users/me/Work/Customers/Base) => {
"msg": "basefarm 2021-08-23 1629699678.021843"
}
ok: [localhost] => (item=/Users/me/Work/Customers/Acme) => {
"msg": "Orange 2021-08-23 1629699678.5415485"
}
ok: [localhost] => (item=/Users/me/Work/Customers/Foo) => {
"msg": "VKB 2021-08-23 1629699679.0579438"
}
ok: [localhost] => (item=/Users/me/Work/Customers/Bar) => {
"msg": "LombardOdier 2021-08-23 1629699679.5856457"
}
...
I found out that the reason is of course that ctime is not the creation time on a mac...
Is there a way around this? I want the real creation date, but Ansible's stat module does not give me this. The creation date is available, as I can get it with ls -ldU for example.
So how do I solve this?

As far as I know, there is no way to do so without using shell command.
- name: make a list of dirs
find:
paths:
- "/Users/me/Work/Customers"
file_type: directory
recurse: false
register: projectdirs
- name: Get creation time
shell: GetFileInfo {{ item.path }} | grep 'created:'
loop: "{{ projectdirs.files }}"
loop_control:
label: "{{ item.path }}"
register: directories
- name: Dump found paths
debug:
msg: "{{ item.item.path }} {{ item.stdout.split('created: ')[1] }}"
loop: "{{ directories.results }}"
loop_control:
label: "{{ item.item.path }}"
Gives me this result :
ok: [localhost] => (item=/Users/jeromeverdoni/stackoverflow/68888321) => {
"msg": "/Users/jeromeverdoni/stackoverflow/68888321 08/23/2021 08:01:44"
}
Hope this is close enough to your expected behaviour.
For reference, GetFileInfo output :
╰─ GetFileInfo /Users/jeromeverdoni/stackoverflow/68888321
directory: "/Users/jeromeverdoni/stackoverflow/68888321"
attributes: avbstclinmedz
created: 08/23/2021 08:01:44
modified: 08/23/2021 08:03:18

Related

How to include variables with include_vars with the same name without overwriting previous

I am having this let's call it include.yaml
#- name: "Playing with Ansible and Include files"
- hosts: localhost
connection: local
tasks:
- find: paths="./" recurse=yes patterns="test.yaml"
register: file_to_exclude
- debug: var=file_to_exclude.stdout_lines
- name: shell
shell: "find \"$(pwd)\" -name 'test.yaml'"
register: files_from_dirs
- debug: var=files_from_dirs.stdout_lines
- name: Include variable files
include_vars: "{{ item }}"
with_items:
- "{{ files_from_dirs.stdout_lines }}"
- debug: var=files
and 2 ore more test files
./dir1/test.yaml
that contains
files:
- file1
- file2
./dir2/test.yaml
that contains
files:
- file3
- file4
the result is
TASK [Include variable files] ******************************************************************************************
ok: [localhost] => (item=/mnt/c/Users/GFlorinescu/ansible_scripts/ansible/1st/test.yaml)
ok: [localhost] => (item=/mnt/c/Users/GFlorinescu/ansible_scripts/ansible/2nd/test.yaml)
TASK [debug] ***********************************************************************************************************
ok: [localhost] => {
"files": [
"file3",
"file4"
]
}
How can I get all the values in files, at the moment the last included files variable from last file overrides the files from the previous files? Of course without changing the variables names in files test.yaml?
In other words I want files to be:
ok: [localhost] => {
"files": [
"file1",
"file2",
"file3",
"file4"
]
}
To be more specific, I ask for any kind of solution or module, even not official or some github module, I don't want a specific include_vars module solution.
Put the included variables into the dictionaries with unique names. For example, create the names from the index of the loop. Then, iterate the names and concatenate the lists
- command: "find {{ playbook_dir }} -name test.yaml"
register: files_from_dirs
- include_vars:
file: "{{ item }}"
name: "{{ name }}"
loop: "{{ files_from_dirs.stdout_lines }}"
loop_control:
extended: true
vars:
name: "files_{{ ansible_loop.index }}"
- set_fact:
files: "{{ files|d([]) + lookup('vars', item).files }}"
with_varnames: "files_[0-9]+"
- debug:
var: files
give
files:
- file1
- file2
- file3
- file4
Notes:
You have to provide either a path relative to the home directory or an absolute path. See the example below
- command: "echo $PWD"
register: out
- debug:
var: out.stdout
give
out.stdout: /home/admin
For example, when you want to find the files relative to the directory of the playbook
- command: "find {{ playbook_dir }} -name test.yaml"
register: files_from_dirs
- debug:
var: files_from_dirs.stdout_lines
give
files_from_dirs.stdout_lines:
- /export/scratch/tmp8/test-987/dir1/test.yaml
- /export/scratch/tmp8/test-987/dir2/test.yaml
The same is valid for the module find. For example,
- find:
paths: "{{ playbook_dir }}"
recurse: true
patterns: test.yaml
register: files_from_dirs
- debug:
var: files_from_dirs.files|map(attribute='path')|list
give the same result
files_from_dirs.files|map(attribute='path')|list:
- /export/scratch/tmp8/test-987/dir1/test.yaml
- /export/scratch/tmp8/test-987/dir2/test.yaml
Simplify the code and put the declaration of files into the vars. For example, the below declaration gives the same result
files: "{{ query('varnames', 'files_[0-9]+')|
map('extract', hostvars.localhost, 'files')|
flatten }}"
Example of a complete playbook for testing
- hosts: localhost
vars:
files: "{{ query('varnames', 'files_[0-9]+')|
map('extract', hostvars.localhost, 'files')|
flatten }}"
tasks:
- find:
paths: "{{ playbook_dir }}"
recurse: true
patterns: test.yaml
register: files_from_dirs
- include_vars:
file: "{{ item }}"
name: "{{ name }}"
loop: "{{ files_from_dirs.files|map(attribute='path')|list }}"
loop_control:
extended: true
vars:
name: "files_{{ ansible_loop.index }}"
- debug:
var: files
(maybe off-topic, see comments)
Q: "Is there a way to write the path where it was found?"
A: Yes, it is. See the self-explaining example below. Given the inventory
shell> cat hosts
host_1 file_1=alice
host_2 file_2=bob
host_3
the playbook
- hosts: host_1,host_2,host_3
vars:
file_1_list: "{{ hostvars|json_query('*.file_1') }}"
file_2_list: "{{ hostvars|json_query('*.file_2') }}"
file_1_dict: "{{ dict(hostvars|dict2items|
selectattr('value.file_1', 'defined')|
json_query('[].[key, value.file_1]')) }}"
file_1_lis2: "{{ hostvars|dict2items|
selectattr('value.file_1', 'defined')|
json_query('[].{key: key, file_1: value.file_1}') }}"
tasks:
- debug:
msg: |-
file_1_list: {{ file_1_list }}
file_2_list: {{ file_2_list }}
file_1_dict:
{{ file_1_dict|to_nice_yaml|indent(2) }}
file_1_lis2:
{{ file_1_lis2|to_nice_yaml|indent(2) }}
run_once: true
gives
msg: |-
file_1_list: ['alice']
file_2_list: ['bob']
file_1_dict:
host_1: alice
file_1_lis2:
- file_1: alice
key: host_1

Nested loop with user and folder in Ansible

I have the following task:
- name: Create required folders.
become: true
ansible.builtin.file:
owner: "{{ item.key }}"
group: ftp
mode: '0755'
path: '/data/{{ item.key }}/in'
state: directory
loop: "{{ query('dict', ftp) | list }}"
when: "'state' not in item.value or item.value.state == 'present'"
And the following host variables with different users:
ftp:
test:
ssh_public_key: "XXXX"
password: "XXX"
home: /data/test
test2:
ssh_public_key: "XXXX"
password: "XXX"
home: /data/test2
What I want is to create two directories for every user :
path: '/data/{{ user }}/in' # item.key, in the code above
path: '/data/{{ user }}/out' # item.key, in the code above
But I already need the loop for iterating over the users itself:
loop: "{{ query('dict', ftp) | list }}"
How can I handle this, for example, with nested loop?
Use a product filter to generate every possible combination of user/folder.
loop: "{{ ftp.keys() | product(['in', 'out']) }}"
Then, respectively,
item.0 contains the users dictionary keys
item.1 contains the folders
It is not fully clear what your condition when does actually, in order to adapt it too, but I guess that you do have an absent or present state in those use dictionaries.
So, the resulting task should be something along the lines of
- name: Create required folders
ansible.builtin.file:
owner: "{{ item.0 }}"
group: ftp
mode: '0755'
path: "/data/{{ item.0 }}/{{ item.1 }}"
state: directory
loop: "{{ ftp.keys() | product(['in', 'out']) }}"
loop_control:
label: "/data/{{ item.0 }}/{{ item.1 }}"
when: "ftp[item.0].state | default('absent') == 'present'"
become: true
Given the task above, when run on those data:
ftp:
test:
state: present
test1:
test2:
state: present
It will yield:
TASK [Create required folders] ***************************************
ok: [localhost] => (item=/data/test/in)
ok: [localhost] => (item=/data/test/out)
skipping: [localhost] => (item=/data/test1/in)
skipping: [localhost] => (item=/data/test1/out)
ok: [localhost] => (item=/data/test2/in)
ok: [localhost] => (item=/data/test2/out)
Test it first, for example
- debug:
msg: "Create /data/{{ item.0.key }}/{{ item.1 }}"
with_nested:
- "{{ ftp|dict2items }}"
- [in, out]
when: item.0.value.state|d('present') == 'present'
gives (abridged)
msg: Create /data/test/in
msg: Create /data/test/out
msg: Create /data/test2/in
msg: Create /data/test2/out
Then try to create the dictionaries
- file:
owner: "{{ item.0.key }}"
group: ftp
mode: '0755'
path: "/data/{{ item.0.key }}/{{ item.1 }}"
state: directory
with_nested:
- "{{ ftp|dict2items }}"
- [in, out]
when: item.0.value.state|d('present') == 'present'
(not tested)

Loop through a list of folders to delete old ones

I want to implement some sort of rotation on subdirectories of folders in a list. Say I have dir1 and dir2, I need to go inside each of them and delete all folders older than X days in dir1 and Y days in dir2.
vars:
backups:
dir1:
name: dir1
days: 10d
dir2:
name: dir2
days: 3d
I've tried to create a task like this
- name: find all folders
find:
paths: "/home/user1/{{ item.value.name }}"
age: "{{ item.value.days }}"
file_type: directory
loop: "{{ lookup('dict', backups) }}"
register: dirsOlderThanXd
But, dirsOlderThanXd has a strange format. If there were no loop and just a single directory next step would be something like
- name: remove old folders
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ dirsOlderThanXd.files }}"
Documentation says
When you use register with a loop, the data structure placed in the variable will contain a results attribute that is a list of all responses from the module. This differs from the data structure returned when using register without a loop.
So, it's expected, but, how exactly do I work with this output? Or am I doing it all completely wrong?
You can access the list — results — of list — files — using a map filter, then flatten the resulting list of list:
- name: remove old folders
file:
path: "{{ item.path }}"
state: absent
loop: "{{ dirsOlderThanXd.results | map(attribute='files') | flatten }}"
loop_control:
label: "{{ item.path }}"
Given the two tasks:
- name: find all folders
find:
paths: "/home/user1/{{ item.value.name }}"
age: "{{ item.value.days }}"
file_type: directory
loop: "{{ backups | dict2items }}"
loop_control:
label: "{{ item.key }}"
register: dirsOlderThanXd
vars:
backups:
dir1:
name: dir1
days: 10d
dir2:
name: dir2
days: 3d
- name: remove old folders
file:
path: "{{ item.path }}"
state: absent
loop: "{{ dirsOlderThanXd.results | map(attribute='files') | flatten }}"
loop_control:
label: "{{ item.path }}"
This would yields something along the line of:
TASK [find all folders] **************************************************
ok: [localhost] => (item=dir1)
ok: [localhost] => (item=dir2)
TASK [remove old folders] ************************************************
changed: [localhost] => (item=/home/user1/dir1/foo)
changed: [localhost] => (item=/home/user1/dir1/bar)
changed: [localhost] => (item=/home/user1/dir2/qux)
changed: [localhost] => (item=/home/user1/dir2/baz)

Ansible - Looking for files and compare their hash

Practice:
I have the file files.yml with a list of files and their respective md5_sum hash, like:
files:
- name: /opt/file_compare1.tar
hash: 9cd599a3523898e6a12e13ec787da50a /opt/file_compare1.tar
- name: /opt/file_compare2tar.gz
hash: d41d8cd98f00b204e9800998ecf8427e /opt/file_compare2.tar.gz
I need to create a playbook to check this list of files if the current hash is the same or if it was changed, the playbook should have a debug message like below:
---
- hosts: localhost
connection: local
vars_files:
- files.yml
tasks:
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
with_items:
- "{{ files }}"
- name: Debug files - Different
debug:
msg: |
"Hash changed: {{ item.name }}"
when:
- item.hash != hash_check
with_items:
- "{{ files }}"
- name: Debug files - Equal
debug:
msg: |
"Hash NOT changed: {{ item.name }}"
when:
- item.hash == hash_check
with_items:
- "{{ files }}"
- debug:
msg: |
- "{{ hash_check }} {{ item.name }}"
with_items:
- "{{ files }}"
For example, given the files
files:
- name: /scratch/file_compare1.tar
hash: 4f8805b4b64dcc575547ec1c63793aec /scratch/file_compare1.tar
- name: /scratch/file_compare2.tar.gz
hash: 2dc4f1e9ca4081cc49d25195627982ef /scratch/file_compare2.tar.gz
the tasks below
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
loop: "{{ files }}"
- name: Debug files - Different
debug:
msg: |
Hash NOT changed: {{ item.0.name }}
{{ item.0.hash.split()|first }}
{{ item.1 }}
with_together:
- "{{ files }}"
- "{{ hash_check.results|map(attribute='stat.checksum')|list }}"
when: item.0.hash.split()|first == item.1
give
msg: |-
Hash NOT changed: /scratch/file_compare1.tar
4f8805b4b64dcc575547ec1c63793aec
4f8805b4b64dcc575547ec1c63793aec
msg: |-
Hash NOT changed: /scratch/file_compare2.tar.gz
2dc4f1e9ca4081cc49d25195627982ef
2dc4f1e9ca4081cc49d25195627982ef
A more robust option would be to create a dictionary with the calculated hashes
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
loop: "{{ files }}"
- set_fact:
path_hash: "{{ dict(_path|zip(_hash)) }}"
vars:
_path: "{{ hash_check.results|map(attribute='stat.path')|list }}"
_hash: "{{ hash_check.results|map(attribute='stat.checksum')|list }}"
gives
path_hash:
/scratch/file_compare1.tar: 4f8805b4b64dcc575547ec1c63793aec
/scratch/file_compare2.tar.gz: 2dc4f1e9ca4081cc49d25195627982ef
Then use this dictionary to compare the hashes. For example, the task below gives the same results
- name: Debug files - Different
debug:
msg: |
Hash NOT changed: {{ item.name }}
{{ item.hash.split()|first }}
{{ path_hash[item.name] }}
loop: "{{ files }}"
when: item.hash.split()|first == path_hash[item.name]
The next option is to create a dictionary with the original hashes and both lists of original and calculated hashes
- name: Use md5 to calculate checksum
stat:
path: "{{ item.name }}"
checksum_algorithm: md5
register: hash_check
loop: "{{ files }}"
- set_fact:
hash_name: "{{ dict(_hash|zip(_name)) }}"
hash_orig: "{{ _hash }}"
hash_stat: "{{ hash_check.results|map(attribute='stat.checksum')|list }}"
vars:
_hash: "{{ files|map(attribute='hash')|map('split')|map('first')|list }}"
_name: "{{ files|map(attribute='name')|list }}"
gives
hash_name:
2dc4f1e9ca4081cc49d25195627982ef: /scratch/file_compare2.tar.gz
4f8805b4b64dcc575547ec1c63793aec: /scratch/file_compare1.tar
hash_orig:
- 4f8805b4b64dcc575547ec1c63793aec
- 2dc4f1e9ca4081cc49d25195627982ef
hash_stat:
- 4f8805b4b64dcc575547ec1c63793aec
- 2dc4f1e9ca4081cc49d25195627982ef
Then calculate the difference of the lists and use it to extract both lists of changed and unchanged files
- set_fact:
files_diff: "{{ _diff|map('extract', hash_name)|list }}"
files_orig: "{{ _orig|map('extract', hash_name)|list }}"
vars:
_diff: "{{ hash_orig|difference(hash_stat) }}"
_orig: "{{ hash_orig|difference(_diff) }}"
- name: Debug files changed
debug:
var: files_diff
- name: Debug files NOT changed
debug:
var: files_orig
gives
files_diff: []
files_orig:
- /scratch/file_compare1.tar
- /scratch/file_compare2.tar.gz
I used your suggestion to complement the playbook, it's working now.
The idea is to get a list of files, read each one and compare with both hash, file, and current hash.
---
- hosts: localhost
connection: local
gather_facts: false
vars_files:
- files3.yml
tasks:
- stat:
path: "{{ item.file }}"
checksum_algorithm: md5
loop: "{{ files }}"
register: stat_results
- name: NOT changed files
debug:
msg: "NOT changed: {{ item.stat.path }}"
when: item.stat.checksum == item.item.checksum.split()|first
loop: "{{ stat_results.results }}"
loop_control:
label: "{{ item.stat.path }}"
- name: Changed files
debug:
msg: "CHANGED: {{ item.stat.path }}"
when: item.stat.checksum != item.item.checksum.split()|first
loop: "{{ stat_results.results }}"
loop_control:
label: "{{ item.stat.path }}"
Result:
>> ansible-playbook playbooks/check-file3.yml
PLAY [localhost] ********************************************************************************************************************************************************************************************************************
TASK [stat] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'file': '/opt/file_compare1.tar', 'checksum': '9cd599a3523898e6a12e13ec787da50a /opt/file_compare1.tar'})
ok: [localhost] => (item={'file': '/opt/file_compare2.tar.gz', 'checksum': 'd41d8cd98f00b204e9800998ecf8427e /opt/file_compare2.tar.gz'})
TASK [NOT changed files] ************************************************************************************************************************************************************************************************************
skipping: [localhost] => (item=/opt/file_compare1.tar)
ok: [localhost] => (item=/opt/file_compare2.tar.gz) => {
"msg": "NOT changed: /opt/file_compare2.tar.gz"
}
TASK [Changed files] ****************************************************************************************************************************************************************************************************************
ok: [localhost] => (item=/opt/file_compare1.tar) => {
"msg": "CHANGED: /opt/file_compare1.tar"
}
skipping: [localhost] => (item=/opt/file_compare2.tar.gz)
PLAY RECAP **************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

ansible find a folder in a path that does not contain a certain subfolder fails

I have folders like on my target host:
/home/admin/stream
/home/admin/STB/stream/data_en
/home/admin/STB_1/stream
I search based on stream in /home/admin but need only folders that does not have data_en in it..
What i tried:
Find directories where stream is present with ansbile code
- name: Find all folders with stream subfolder in it
find:
paths: /home/admin/stream
file_type: directory
pattern: 'stream'
use_regex: no
recurse: yes
register: files_to_change
- name: rename folders
command: mv "{{ item['path'] }}" "{{ item['path'] }}.disable"
with_items: "{{ files_to_change.files }}"
when:
- item['path'] is not regex(item['path']/data_en)
I am trying to rename only folder 1 and 3 from the below list, because folder 2 has data_en inside it.
/home/admin/stream
/home/admin/STB/stream
/home/admin/STB_1/stream
I could not figure out a way to skip that..
Fix and test the condition, e.g.
when: item.path is not regex('^(.*)/data_en(.*)$')
You might want to test the code first, e.g.
- debug:
msg: "mv {{ item.path }} {{ item.path }}.disable"
loop: "{{ files_to_change.files }}"
loop_control:
label: "{{ item.path }}"
when: item.path is not regex('^(.*)/data_en(.*)$')
should give
TASK [debug] ********************************************************************
ok: [localhost] => (item=/home/admin/stream) =>
msg: mv /home/admin/stream /home/admin/stream.disable
skipping: [localhost] => (item=/home/admin/STB/stream/data_en)
ok: [localhost] => (item=/home/admin/STB_1/stream) =>
msg: mv /home/admin/STB_1/stream /home/admin/STB_1/stream.disable

Resources