Extracting json from ansible playbook - ansible

I am fairly new to Ansible, so apologies if this a naïve question.
When I execute Ansible Playbook using AnsibleRunner, I get output in the following format:
PLAY [MyPlay] ******************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] *********************************************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Get list of volumes] *****************************************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [debug] *******************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"volumes": {
"Array_Software_Version": "2.0.0.0",
"Cluster": [
{
"id": "0",
"name": "111"
}
],
"Volumes": [
{
"id": "003ec9af-7a6d-473b-942f-3418da25d88c",
"name": "server1"
},
...
...
...
and my task
tasks:
- name: Get list of volumes
task: execute
register: volumes
- debug:
var: volumes
I'd like to extract just JSON payload, can I do this using Ansible?

Q: "Retrieve the actual JSON payload from ansible."
A: An option would be to use an intermediate file, for example
shell> cat test-280.yml
- hosts: localhost
become: true
vars:
volumes:
Array_Software_Version: "2.0.0.0"
Cluster:
- id: 0
name: 111
Volumes:
- id: 003ec9af-7a6d-473b-942f-3418da25d88c
name: server1
tasks:
- copy:
dest: /tmp/my_volumes.json
content: |
{{ {'volumes': volumes}|to_nice_json }}
shell> cat test-280.sh
#!/bin/bash
ansible-playbook test-280.yml > /dev/null
my_volumes=`cat /tmp/my_volumes.json`
echo $my_volumes
shell> ./test-280.sh
{ "volumes": { "Array_Software_Version": "2.0.0.0", "Cluster": [ { "id": 0, "name": 111 } ], "Volumes": [ { "id": "003ec9af-7a6d-473b-942f-3418da25d88c", "name": "server1" } ] } }

Related

Ansible - Extracting a variable from a large JSON output

The gcp_sql_instance_info module does not seem to allow you to specify the name of the database you are trying to get info for. All it does is every detail of every database in that project.
What would be the best way to extract the "ipAddress" of a database based on it's name and register that value to a variable?
{
"resources": [
{
"kind": "sql#instance",
"state": "RUNNABLE",
"databaseVersion": "POSTGRES_13",
"settings": {
"authorizedGaeApplications": [],
"tier": "db-custom-1-3840",
"kind": "sql#settings",
"availabilityType": "ZONAL",
"pricingPlan": "PER_USE",
"replicationType": "SYNCHRONOUS",
"activationPolicy": "ALWAYS",
"ipConfiguration": {
"privateNetwork": "example-network",
"authorizedNetworks": [],
"ipv4Enabled": false
},
"locationPreference": {
"zone": "us-west1-b",
"kind": "sql#locationPreference"
},
"dataDiskType": "PD_SSD",
"maintenanceWindow": {
"kind": "sql#maintenanceWindow",
"hour": 0,
"day": 0
},
"backupConfiguration": {
"startTime": "02:00",
"kind": "sql#backupConfiguration",
"backupRetentionSettings": {
"retentionUnit": "COUNT",
"retainedBackups": 7
},
"enabled": false,
"replicationLogArchivingEnabled": false,
"pointInTimeRecoveryEnabled": false,
"transactionLogRetentionDays": 7
},
"settingsVersion": "3",
"storageAutoResizeLimit": "0",
"storageAutoResize": true,
"dataDiskSizeGb": "100"
},
"ipAddresses": [
{
"type": "PRIVATE",
"ipAddress": "10.10.10.1"
}
],
"instanceType": "CLOUD_SQL_INSTANCE",
"backendType": "SECOND_GEN",
"name": "database-1",
"region": "us-west1",
"gceZone": "us-west1-b",
"databaseInstalledVersion": "POSTGRES_13_7",
"createTime": "2022-07-22T20:20:32.274Z"
},
{
"kind": "sql#instance",
"state": "RUNNABLE",
"databaseVersion": "MYSQL_8_0",
"settings": {
"authorizedGaeApplications": [],
"tier": "db-n1-standard-1",
"kind": "sql#settings",
"availabilityType": "ZONAL",
"pricingPlan": "PER_USE",
"replicationType": "SYNCHRONOUS",
"activationPolicy": "ALWAYS",
"ipConfiguration": {
"privateNetwork": "example-network",
"authorizedNetworks": [],
"ipv4Enabled": false
},
"locationPreference": {
"zone": "us-west1-c",
"kind": "sql#locationPreference"
},
"dataDiskType": "PD_SSD",
"backupConfiguration": {
"startTime": "21:00",
"kind": "sql#backupConfiguration",
"backupRetentionSettings": {
"retentionUnit": "COUNT",
"retainedBackups": 7
},
"enabled": false,
"transactionLogRetentionDays": 7
},
"settingsVersion": "1",
"storageAutoResizeLimit": "0",
"storageAutoResize": true,
"dataDiskSizeGb": "10"
},
"ipAddresses": [
{
"type": "PRIVATE",
"ipAddress": "10.10.10.2"
}
],
"instanceType": "CLOUD_SQL_INSTANCE",
"backendType": "SECOND_GEN",
"name": "database-2",
"region": "us-west1",
"gceZone": "us-west1-c",
"databaseInstalledVersion": "MYSQL_8_0_26",
"createTime": "2022-07-27T19:27:59.235Z"
}
]
Q: "Extract the ipAddress of a database based on its name."
A: Crete a dictionary. For example, select the first item from the list ipAddresses
name_ip: "{{ dict(resources|json_query(name_ip_query)) }}"
name_ip_query: '[].[name, ipAddresses[0].ipAddress]'
give
name_ip:
database-1: 10.10.10.1
database-2: 10.10.10.2
Notes
Example of a complete playbook for testing
- hosts: localhost
vars_files:
- resources.yml
vars:
name_ip: "{{ dict(resources|json_query(name_ip_query)) }}"
name_ip_query: '[].[name, ipAddresses[0].ipAddress]'
name_ver: "{{ dict(resources|json_query(name_ver_query)) }}"
name_ver_query: '[].[name, databaseVersion]'
ip_of_db1: "{{ name_ip['database-1'] }}"
tasks:
- debug:
var: name_ip
- debug:
var: name_ver
- debug:
var: ip_of_db1
gives
TASK [debug] *********************************************************************************
ok: [localhost] =>
name_ip:
database-1: 10.10.10.1
database-2: 10.10.10.2
TASK [debug] *********************************************************************************
ok: [localhost] =>
name_ver:
database-1: POSTGRES_13
database-2: MYSQL_8_0
TASK [debug] *********************************************************************************
ok: [localhost] =>
ip_of_db1: 10.10.10.1
There might be more IP addresses. The variable ipAddresses is a list. An option would be to create a list of all IP addresses. For example,
name_ips: "{{ dict(resources|json_query(name_ips_query)) }}"
name_ips_query: '[].[name, ipAddresses[].ipAddress]'
give
name_ips:
database-1: [10.10.10.1]
database-2: [10.10.10.2]
This is a working playbook to achieve the desired output:
- hosts: localhost
vars:
db_instance_name: instance-1
tasks:
- name: get info on an instance
gcp_sql_instance_info:
project: project_id
auth_kind: serviceaccount
service_account_file: "/path/to/sa/example.json"
register: cloud_sql_resources
- set_fact:
query: "resources[?contains(name,'{{ db_instance_name }}')].ipAddresses[].ipAddress"
- set_fact:
ipAddress: "{{ cloud_sql_resources | to_json | from_json | json_query(query) }}"
- debug:
msg: "{{ ipAddress }}"
You can run your playbook like this: ansible-playbook -v main.yaml --extra-vars "db_instance_name=instance-1", that would print:
TASK [debug] ********************************************************************************************************************
ok: [localhost] => {
"msg": [
"35.224.46.127"
]
}

Ansible / jmespath - 1d list to zipped list of dictionaries

I'm attempting to transform a 1d list of AWS EC2 IDs into a list of dictionaries suitable for usage as the targets param to the Ansible elb_target_group module.
Sample input:
TASK [debug]
ok: [localhost] => {
"instance_ids": [
"i-1111",
"i-2222",
"i-3333"
]
}
Desired output:
TASK [debug]
ok: [localhost] => {
"targets": [
{"Id": "i-1111", "Port": 6443},
{"Id": "i-2222", "Port": 6443},
{"Id": "i-3333", "Port": 6443},
]
}
What I've tried: a json_query / JMESPath expression which confusingly returns null values:
---
- name: test
hosts: localhost
connection: local
vars:
instance_ids:
- i-1111
- i-2222
- i-3333
keys_list:
- Id
- Port
tasks:
- debug: var=instance_ids
- debug:
msg: "{{ instance_ids | zip_longest([6443], fillvalue=6443) | list }}" # Halfway there...
- debug:
msg: "{{ instance_ids | zip_longest([6443], fillvalue=6443) | list | map('json_query', '{Id: [0], Port: [1]}') | list }}"
Output:
TASK [debug]
ok: [localhost] => {
"msg": [
[
"i-1111",
6443
],
[
"i-2222",
6443
],
[
"i-3333",
6443
]
]
}
TASK [debug]
ok: [localhost] => {
"msg": [
{
"Id": null,
"Port": null
},
{
"Id": null,
"Port": null
},
{
"Id": null,
"Port": null
}
]
}
How should I adjust the '{Id: [0], Port: [1]}' part ?
Stumbled my way to a working solution with json_query and a JMESPath int literal:
"{{ instance_ids | json_query('[*].{Id: #, Port: `6443`}') }}"
Example:
- name: test
hosts: localhost
connection: local
vars:
instance_ids:
- i-1111
- i-2222
- i-3333
tasks:
- debug:
msg: "{{ instance_ids | json_query('[*].{Id: #, Port: `6443`}') }}"
Output:
TASK [debug]
ok: [localhost] => {
"msg": [
{
"Id": "i-1111",
"Port": 6443
},
{
"Id": "i-2222",
"Port": 6443
},
{
"Id": "i-3333",
"Port": 6443
}
]
}

Ansible nested loop and jinja2 filter

I need to list some directories in my project, I have this kind of tree :
.
└── path
└── to
├── aa
│   └── x9999
├── bb
│   ├── x9997
│   └── x9998
├── cc
└── dd
I have made this task in my playbook to find directories:
---
- name: 'test of find'
hosts: localhost
vars:
loopVar:
- 'aa'
- 'bb'
- 'cc'
- 'dd'
tasks:
- name: 'find'
find:
path: '/path/to/{{item}}'
file_type: directory
patterns: '[^_]*[a-z]+[0-9]{4}[a-z]*$'
use_regex: yes
register: list
loop: '{{loopVar}}'
and the result is :
{
"list": {
"results": [
{
"files": [
{
"path": "/path/to/aa/x9999"
}
],
"item": "aa",
"matched": 1
},
{
"files": [
{
"path": "/path/to/bb/x9997"
},
{
"path": "/path/to/bb/x9998"
}
],
"item": "bb",
"matched": 2
},
{
"files": [],
"item": "cc",
"matched": 0
},
{
"files": [],
"item": "dd",
"matched": 0
}
]
}
}
I want to invoke a include_role with this array but I can't do it by this way because I need to have var at the same level than path.
I want to work on my result to have this kind of value :
{ "final": [
{ "path": "/path/to/aa/x9999", "var": "aa" },
{ "path": "/path/to/bb/x9998", "var": "bb" },
{ "path": "/path/to/bb/x9997", "var": "bb" }
]
}
I Try a lot of things with set_fact but I don't find how to do this kind of work.
Thank a lot for your help :-)
The play below
- find:
paths: "/path/to/{{ item }}"
file_type: directory
recurse: yes
register: list
loop: [ aa, bb, cc, dd]
- set_fact:
pre_final: "{{ list.results|json_query('[].{path: files[].path,
var: item}') }}"
- set_fact:
final: "{{ final|default([]) + [{'var': item.0.var,
'path': item.1}] }}"
loop: "{{ lookup('subelements', pre_final, 'path') }}"
- debug:
var: final
gives (abridged):
"final": [
{
"path": "/path/to/aa/x9999",
"var": "aa"
},
{
"path": "/path/to/bb/x9997",
"var": "bb"
},
{
"path": "/path/to/bb/x9998",
"var": "bb"
}
]
I have found a way to do it. It's not lovelly but it's works.
I have to use the include_tasks module and the json_query given :
playbook.yml
---
- name: 'test of find'
hosts: localhost
vars:
loopVar:
- 'aa'
- 'bb'
- 'cc'
- 'dd'
tasks:
- name: 'find'
find:
path: '/path/to/{{item}}'
file_type: directory
patterns: '[^_]*[a-z]+[0-9]{4}[a-z]*$'
use_regex: yes
register: list
loop: '{{loopVar}}'
- set_fact:
all: '{{ list.results|json_query("[].{path: files[].path, var: item, matched: matched}") }}'
- include_tasks: 'inner.yml'
vars:
paths: '{{item.path}}'
var: '{{item.var}}'
loop: '{{all}}'
when: 'item.matched != 0'
- debug:
var: final
inner.yaml
---
- set_fact:
final: '{{ final|default([]) + [{ "path": path|basename, "var": var }] }}'
loop: '{{paths}}'
loop_control:
loop_var: path
Result:
[root#xxxx path]# ansible-playbook playbook.yml
PLAY [test of find] *********************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ******************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [find] *****************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item=aa)
ok: [localhost] => (item=bb)
ok: [localhost] => (item=cc)
ok: [localhost] => (item=dd)
TASK [set_fact] *************************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [include_tasks] ********************************************************************************************************************************************************************************************************************
skipping: [localhost] => (item={'var': u'cc', 'path': [], 'matched': 0})
skipping: [localhost] => (item={'var': u'dd', 'path': [], 'matched': 0})
included: /mnt/hgfs/Documents/inner.yml for localhost => (item={'var': u'aa', 'path': [u'/path/to/aa/x9999'], 'matched': 1})
included: /mnt/hgfs/Documents/inner.yml for localhost => (item={'var': u'bb', 'path': [u'/path/to/bb/x9997', u'/path/to/bb/x9998'], 'matched': 2})
TASK [set_fact] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item=/path/to/aa/x9999)
TASK [set_fact] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item=/path/to/bb/x9997)
ok: [localhost] => (item=/path/to/bb/x9998)
TASK [debug] ****************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"final": [
{
"path": "x9999",
"var": "aa"
},
{
"path": "x9997",
"var": "bb"
},
{
"path": "x9998",
"var": "bb"
}
]
}
PLAY RECAP ******************************************************************************************************************************************************************************************************************************
localhost : ok=8 changed=0 unreachable=0 failed=0
I don't find a way to do it without use an external playbook.
If someone find a way, I'm happy to learn it :-)
Thank a lot.

set_fact create a list with items

I have task which calls an API and I register the o/p in a varaible;
- name: Get Object storage account ID
uri:
url: 'https://api.softlayer.com/rest/v3.1/SoftLayer_Network_Storage_Hub_Cleversafe_Account/getAllObjects.json?objectFilter={"username":{"operation":"{{ item }}"}}'
method: GET
user: abxc
password: 66c94c447a6ed8a0cf058774fe38
validate_certs: no
register: old_existing_access_keys_sl
with_items: '{{ info["personal"].sl_cos_accounts }}'
old_existing_access_keys_sl holds:
"old_existing_access_keys_sl.results": [
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "xyz-11"
}
]
},
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "abc-12"
}
]
}
I want to make a list of id's for further processing an tried the following task but this did not work:
- name: Create a list of account ids
set_fact:
admin_usernames = "{{ item.json[0].id | list }}"
with_items: old_existing_access_keys_sl.results
I am not sure if that's even possible. I also tried this:
- name: create a list
set_fact:
foo: "{% set foo = [] %}{% for i in old_existing_access_keys_sl.results %}{{ foo.append(i) }}{% endfor %}"
foo always comes as blank and as a string:
TASK [result] *****************************************************************************************************************************************
ok: [localhost] => {
"foo": ""
}
Given your example data, you can extract a list of ids using the json_query filter, like this:
---
- hosts: localhost
gather_facts: false
vars:
old_existing_access_keys_sl:
results:
[
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "xyz-11"
}
]
},
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "abc-12"
}
]
}
]
tasks:
- debug:
var: old_existing_access_keys_sl|json_query('results[*].json[0].id')
This will output:
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => {
"old_existing_access_keys_sl|json_query('results[*].json[0].id')": [
70825621,
70825621
]
}
If you want to store these in a new variable, you can replace that debug task with set_fact:
- set_fact:
admin_ids: "{{ old_existing_access_keys_sl|json_query('results[*].json[0].id') }}"
Update
For a list of dictionaries, just change the json_query expression:
- debug:
var: "old_existing_access_keys_sl|json_query('results[*].json[0].{id: id, username: username}')"
For more information, see the jmespath website for documentation and examples.

Ansible shows error: "One or more undefined variables: 'item' is undefined" when using 'with_items'

I am trying to count the instances inside an elb. This is my Ansible playbook:
- name: Get elb facts
local_action:
module: ec2_elb_facts
name: "{{elb}}"
region: "{{ansible_ec2_placement_region}}"
environment: creds
register: elb_facts
- debug:
var: elb_facts
verbosity: 2
- debug:
msg: "Instance: {{ item.instances }}"
with_items: "{{ elb_facts.elbs }}"
and my output (sensitive data removed):
TASK: [debug ] ****************************************************************
ok: [10.0.0.0] => {
"elb_facts": {
"changed": false,
"elbs": [
{
"availability_zones": [
"ap-southeast-2b",
"ap-southeast-2a"
],
"dns_name": "elbname123.ap-southeast-2.elb.amazonaws.com",
"health_check": {
"healthy_threshold": 2,
"interval": 10,
"target": "TCP:0000",
"timeout": 5,
"unhealthy_threshold": 2
},
"instances": [
{
"id": "i-000000000000000",
"state": null
}
],
"name": "accessgateway",
"scheme": "internal",
"security_groups": [
"sg-00000000"
],
"subnet": [
"subnet-0000000",
"subnet-1111111"
],
"vpc_id": "vpc-000000"
}
],
"invocation": {
"module_args": "",
"module_name": "ec2_elb_facts"
}
}
}
TASK: [debug ] ****************************************************************
fatal: [10.0.0.0] => One or more undefined variables: 'item' is undefined
FATAL: all hosts have already failed -- aborting
So what im trying to do is just loop through and print everything inside the elb_facts, instances variable. From what I can tell it's a hash, containing a list of hashes.
I am using http://docs.ansible.com/ansible/playbooks_loops.html#looping-over-subelements as a reference. I cannot for the life of mine figure out why this is not working.
with_items (and the whole family of with_ loops) is a dictionary key defined in a task, not as a parameter to the action.
Fix the indentation:
- debug:
msg: "Instance: {{ item.instances }}"
with_items: "{{ elb_facts.elbs }}"

Resources