Ansible - Find and Replace - ansible

Question

Use case
List all files in a directory with the format - a1.{{ env }}.js, a2.{{ env }}.js
Find corresponding files in the destination directory with the format - a1.js, a2.js
Copy a1.{{ env }}.js in the directory where a1.js exists, Copy a2.{{ env }}.js in the directory where a2.js exists
Sample code: This code does a direct find and replace
- name: Find files in archive
find:
paths: "archive/"
file_type: file
recurse: yes
register: tmp_file_path
- name: Find files in code matching names in archive
find:
paths: "code/"
file_type: file
recurse: yes
patterns: "{{ tmp_file_path.files | map(attribute='path') | map('basename') | list }}"
register: code_file_path
- set_fact:
code_files: "{{ code_files|default([]) +
[{'path': item, 'name': item|basename}] }}"
loop: "{{ code_file_path.files|map(attribute='path')|list }}"
- name: Copy files from archive to code directory
command: cp "{{ item.0 }}" "{{ item.1.path }}"
when:
- item.0|basename == item.1.path|basename
with_together:
- "{{ tmp_file_path.files|map(attribute='path')|list|sort }}"
- "{{ code_files|sort(attribute='name') }}"
Below listed is the directory structure
├── archive
│   ├── a1.test.js
│   ├── a2.test.js
│   ├── a3.test.js
│   └── a4.test.js
└── code
├── a1.js
├── dir1
│   └── a2.js
└── dir2
├── a4.js
└── dir3
└── a3.js
Copy archive/a1.test.js to code/
Copy archive/a2.test.js to code/dir1/
Copy archive/a3.test.js to code/dir1/dir2/dir3/
Copy archive/a4.test.js to code/dir1/dir2/
Is there a solution to do a direct copy as per the above use case?

Answer 1

Some explanation on the approach:
The entire idea of it is based on the creation of a dictionary instructing the playbook on which file should be added where.
The dictionary, for your use case would look like this:
{
"a1.js": {
"archive": "archive/a1.test.js",
"paths": [
"code"
]
},
"a2.js": {
"archive": "archive/a2.test.js",
"paths": [
"code/dir1"
]
},
"a3.js": {
"archive": "archive/a3.test.js",
"paths": [
"code/dir1/dir2/dir3"
]
},
"a4.js": {
"archive": "archive/a4.test.js",
"paths": [
"code/dir1/dir2"
]
}
}
Where the keys are the files we are searching for under the code folder, the key archive represent the file we aim to copy from the archive folder and paths is an array where the said file should find its destination(s).
Most of the logic is done by the Ansible filter regex_replace, that extract the name of the file to find in the code folder via a quite simple expression: (.*)\..*\.js$
item.path | basename | regex_replace('(.*)\\..*\\.js$', '\\1.js')
Another filter used here that might be interesting to explore is the combine filter, with the parameter recursive=true, that allows to create the paths where the files should find their destination(s).
There is also a Python operation used here: dict.keys(), in order to create the comma separated list of files to search in the code folder out of the keys of the dictionary above.
It is also making use of loop with the subelement filter, to traverse both the dictionary and its sub array paths at the same time.
And to be complete, here are the other, more commonly used filters used in this playbook:
default: to specify a default value when a variable is not defined
basename: to get only the name of a file out of its full path
dirname: to get only the directory from a full path to a file
join: to concatenate the elements of an array, here, based on a separator
Yes, it might be over-engineered compared to your use case, the playbook here-under is able to cope with the fact that a1.js could be in two different folders and will be able to copy a1.test.js in both those folders.
So, here is a solution:
- hosts: localhost
gather_facts: no
tasks:
- find:
paths: archive
file_type: file
recurse: yes
register: archives
- set_fact:
searches: "{{ searches | default({}) | combine({ key: value }) }}"
vars:
key: "{{ item.path | basename | regex_replace('(.*)\\..*\\.js$', '\\1.js') }}"
value: "{{ { 'archive': item.path, 'paths': [] } }}"
loop: "{{ archives.files }}"
loop_control:
label: "{{ item.path }}"
- find:
path: code
file_type: file
recurse: yes
pattern: "{{ searches.keys() | join(',') }}"
register: paths
- set_fact:
searches: "{{ searches | combine({key: value}, recursive=true) }}"
vars:
key: "{{ item.path | basename }}"
value: "{{ { 'paths': [item.path | dirname] + searches[item.path | basename].paths } }}"
loop: "{{ paths.files }}"
loop_control:
label: "{{ item.path }}"
- copy:
src: "{{ item.0.archive }}"
dest: "{{ item.1 ~ '/' ~ item.0.archive | basename }}"
loop: "{{ searches | subelements('paths') }}"
loop_control:
label: "{{ item.0.archive }}"
Situation before:
tree archive code
archive
├── a1.test.js
├── a2.test.js
├── a3.test.js
└── a4.test.js
code
├── a1.js
└── dir1
├── a2.js
└── dir2
├── a4.js
└── dir3
└── a3.js
3 directories, 8 files
Recap of the playbook:
PLAY [localhost] **************************************************************************************************
TASK [find] *******************************************************************************************************
ok: [localhost]
TASK [set_fact] ***************************************************************************************************
ok: [localhost] => (item=archive/a3.test.js)
ok: [localhost] => (item=archive/a2.test.js)
ok: [localhost] => (item=archive/a1.test.js)
ok: [localhost] => (item=archive/a4.test.js)
TASK [find] *******************************************************************************************************
ok: [localhost]
TASK [set_fact] ***************************************************************************************************
ok: [localhost] => (item=code/a1.js)
ok: [localhost] => (item=code/dir1/a2.js)
ok: [localhost] => (item=code/dir1/dir2/a4.js)
ok: [localhost] => (item=code/dir1/dir2/dir3/a3.js)
TASK [copy] *******************************************************************************************************
changed: [localhost] => (item=archive/a3.test.js)
changed: [localhost] => (item=archive/a2.test.js)
changed: [localhost] => (item=archive/a1.test.js)
changed: [localhost] => (item=archive/a4.test.js)
PLAY RECAP ********************************************************************************************************
localhost : ok=5 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Situation after:
tree code
code
├── a1.js
├── a1.test.js
└── dir1
├── a2.js
├── a2.test.js
└── dir2
├── a4.js
├── a4.test.js
└── dir3
├── a3.js
└── a3.test.js
3 directories, 8 files