Ansible - Create JSON output by merging facts - ansible

Question

I am trying to create a JSON output from an Ansible playbook that will be sent to a 3rd party system via file or RabbitMQ. The issue I am having is creating the JSON file/body to send as output.
Environment:
Ansible: 2.10.17
JSON input file: 'Cannot change the format of the input file'
Logical process:
Receive a JSON input file => working
Convert JSON input file to a fact (JSON_input) => working
Search through the a directory and find any folder (bucket) not already
listed (var_client_buckets_diff_state_current)=> working
Create new fact which consist of [input file + var_client_buckets_diff_state_current] (create json file structure for new fact) => not
working
Send the new fact to 3rd party => not attempted yet
What has been attempted:
Tried to use the 'update_fact' with indicies => this seemed not to
work and was intended for updating an existing variable within the
fact
Tried to use the 'set_fact' + [item] => not what I expected
Input_file:
{
"siteIdentifier": "site1",
"configurations": [
{
"tenantName": "patcorp",
"keypairConfigurations": [
{
"accessKey": "key1",
"secretKey": "kjjhgghfdgf765ghdf",
"port": "60008",
"mountPath": "/tmp/mnt/scoutfs/patcorp",
"buckets": [
{
"bucketName": "bucket1",
"settings": {
"status": "OK"
}
},
{
"bucketName": "bucket2",
"settings": {
"status": "PENDING-DELETION"
}
}
]
}
]
},
{
"tenantName": "mattcorp",
"keypairConfigurations": [
{
"accessKey": "key3",
"secretKey": "jhgf785tkjjhg",
"port": "60010",
"mountPath": "/tmp/mnt/scoutfs/mattcorp",
"buckets": [
{
"bucketName": "bucket10",
"settings": {
"status": "OK"
}
},
{
"bucketName": "bucket11",
"settings": {
"status": "PENDING-DELETION"
}
}
]
}
]
}
]
}
Playbook:
---
- name: debug_playbook
hosts:
- localhost
gather_facts: true
vars:
var_client_buckets_diff_state_current: ['/tmp/mnt/scoutfs/mattcorp/bucket12', '/tmp/mnt/scoutfs/mattcorp/bucket22', '/tmp/mnt/scoutfs/patcorp/bucket07']
fact_json_updated: []
tasks:
## Find collections path via ansible.cfg
- name: debug_read_collections_path
ansible.builtin.debug:
msg: "{{ lookup('config', 'COLLECTIONS_PATHS') }}"
run_once: true
register: reg_debug_read_collections_path
## Read JSON file to create stdout
- name: cat_scoutam_client_master_assignments_raw
ansible.builtin.shell:
cmd: cat ./files/scoutam_client_master_assignments_2.json
run_once: true
delegate_to: localhost
register: reg_scoutam_client_master_assignments_raw
## Convert JSON input file into fact
- name: fact_scoutam_client_master_assignments
ansible.builtin.set_fact:
fact_scoutam_client_master_assignments: "{{ reg_scoutam_client_master_assignments_raw.stdout | from_json }}"
run_once: true
delegate_to: localhost
# - name: debug_scoutam_client_tenant_ids
# debug:
# msg: "{{ fact_scoutam_client_master_assignments | json_query(jmesquery) }}"
# vars:
# jmesquery: 'configurations[*].tenantName'
# run_once: true
# delegate_to: localhost
# register: reg_debug_scoutam_client_tenant_ids
## Create duplicate fact file with only diffs found from 'var_client_buckets_diff_state_current'
- name: create json file structure for new fact
ansible.builtin.set_fact:
fact_json_new:
siteIdentifier:
configurations:
- {tenantName: "{{ var_tenantName }}", keypairConfigurations: [{accessKey: , secretKey: , port: , mountPath: , buckets: [{bucketName: "{{ var_bucketName }}", settings: {status: }}] }]}
vars:
var_tenantName: "{{ item.split('/')[4] }}" ## take 'item' and find tenant name
var_bucketName: "{{ item.split('/')[5] }}" ## take 'item' and find bucket name
loop: "{{ var_client_buckets_diff_state_current }}" ## loop through buckets not within input file
register: reg_fact_json_new
## Create new JSON file (input file + any new buckets found )
- name: combine fact
ansible.builtin.set_fact:
fact_json_updated: "{{ fact_json_updated + item }}"
loop: "{{ reg_fact_json_new.results | json_query('[*].ansible_facts.fact_json_new') }}"
- name: debug fact - fact_json_updated
debug:
var: fact_json_updated
## End of playbook message
- name: "debug_playbook_{{ ansible_play_name }}_completion"
ansible.builtin.debug:
msg: "Playbook {{ playbook_dir }}/{{ ansible_play_name }} has finished."
...
Output from [create json file structure for new fact, combine fact]
TASK [combine fact] ************************************************************************************************************************************************************************************************************
task path: /app/ansible/ansible_linux_playbooks/playbooks/TEST_debug_facts_playbook/debug_playbook.yml:70
Thursday 28 October 2021 10:06:50 -0400 (0:00:00.078) 0:00:01.417 ******
ok: [localhost] => (item={'siteIdentifier': None, 'configurations': [{'tenantName': 'mattcorp', 'keypairConfigurations': [{'accessKey': None, 'secretKey': None, 'port': None, 'mountPath': None, 'buckets': [{'bucketName': 'bucket12', 'settings': {'status': None}}]}]}]}) => {
"ansible_facts": {
"fact_json_updated": [
{
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket12",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "mattcorp"
}
],
"siteIdentifier": null
}
]
},
"ansible_loop_var": "item",
"changed": false,
"item": {
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket12",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "mattcorp"
}
],
"siteIdentifier": null
}
}
ok: [localhost] => (item={'siteIdentifier': None, 'configurations': [{'tenantName': 'mattcorp', 'keypairConfigurations': [{'accessKey': None, 'secretKey': None, 'port': None, 'mountPath': None, 'buckets': [{'bucketName': 'bucket22', 'settings': {'status': None}}]}]}]}) => {
"ansible_facts": {
"fact_json_updated": [
{
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket12",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "mattcorp"
}
],
"siteIdentifier": null
},
{
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket22",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "mattcorp"
}
],
"siteIdentifier": null
}
]
},
"ansible_loop_var": "item",
"changed": false,
"item": {
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket22",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "mattcorp"
}
],
"siteIdentifier": null
}
}
ok: [localhost] => (item={'siteIdentifier': None, 'configurations': [{'tenantName': 'patcorp', 'keypairConfigurations': [{'accessKey': None, 'secretKey': None, 'port': None, 'mountPath': None, 'buckets': [{'bucketName': 'bucket07', 'settings': {'status': None}}]}]}]}) => {
"ansible_facts": {
"fact_json_updated": [
{
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket12",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "mattcorp"
}
],
"siteIdentifier": null
},
{
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket22",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "mattcorp"
}
],
"siteIdentifier": null
},
{
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket07",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "patcorp"
}
],
"siteIdentifier": null
}
]
},
"ansible_loop_var": "item",
"changed": false,
"item": {
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket07",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "patcorp"
}
],
"siteIdentifier": null
}
}
TASK [debug fact - reg_fact_json_updated] **************************************************************************************************************************************************************************************
task path: /app/ansible/ansible_linux_playbooks/playbooks/TEST_debug_facts_playbook/debug_playbook.yml:77
Thursday 28 October 2021 10:06:50 -0400 (0:00:00.100) 0:00:01.518 ******
ok: [localhost] => {
"fact_json_updated": [
{
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket12",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "mattcorp"
}
],
"siteIdentifier": null
},
{
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket22",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "mattcorp"
}
],
"siteIdentifier": null
},
{
"configurations": [
{
"keypairConfigurations": [
{
"accessKey": null,
"buckets": [
{
"bucketName": "bucket07",
"settings": {
"status": null
}
}
],
"mountPath": null,
"port": null,
"secretKey": null
}
],
"tenantName": "patcorp"
}
],
"siteIdentifier": null
}
]
}
Expected output JSON:
{
"siteIdentifier": "mas1-sos07",
"configurations": [
{
"tenantName": "patcorp",
"keypairConfigurations": [
{
"accessKey": "key1",
"secretKey": "kjjhgghfdgf765ghdf",
"port": "60008",
"mountPath": "/tmp/mnt/scoutfs/patcorp",
"buckets": [
{
"bucketName": "bucket1",
"settings": {
"status": "OK"
}
},
{
"bucketName": "bucket2",
"settings": {
"status": "PENDING-DELETION"
}
},
{
"bucketName": "bucket07",
"settings": {
"status": null
}
}
]
}
]
},
{
"tenantName": "mattcorp",
"keypairConfigurations": [
{
"accessKey": "key3",
"secretKey": "jhgf785tkjjhg",
"port": "60010",
"mountPath": "/tmp/mnt/scoutfs/mattcorp",
"buckets": [
{
"bucketName": "bucket10",
"settings": {
"status": "OK"
}
},
{
"bucketName": "bucket11",
"settings": {
"status": "PENDING-DELETION"
}
},
{
"bucketName": "bucket12",
"settings": {
"status": null
}
},
{
"bucketName": "bucket22",
"settings": {
"status": null
}
}
]
}
]
}
]
}