set_fact create a list with items - ansible

I have task which calls an API and I register the o/p in a varaible;
- name: Get Object storage account ID
uri:
url: 'https://api.softlayer.com/rest/v3.1/SoftLayer_Network_Storage_Hub_Cleversafe_Account/getAllObjects.json?objectFilter={"username":{"operation":"{{ item }}"}}'
method: GET
user: abxc
password: 66c94c447a6ed8a0cf058774fe38
validate_certs: no
register: old_existing_access_keys_sl
with_items: '{{ info["personal"].sl_cos_accounts }}'
old_existing_access_keys_sl holds:
"old_existing_access_keys_sl.results": [
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "xyz-11"
}
]
},
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "abc-12"
}
]
}
I want to make a list of id's for further processing an tried the following task but this did not work:
- name: Create a list of account ids
set_fact:
admin_usernames = "{{ item.json[0].id | list }}"
with_items: old_existing_access_keys_sl.results
I am not sure if that's even possible. I also tried this:
- name: create a list
set_fact:
foo: "{% set foo = [] %}{% for i in old_existing_access_keys_sl.results %}{{ foo.append(i) }}{% endfor %}"
foo always comes as blank and as a string:
TASK [result] *****************************************************************************************************************************************
ok: [localhost] => {
"foo": ""
}

Given your example data, you can extract a list of ids using the json_query filter, like this:
---
- hosts: localhost
gather_facts: false
vars:
old_existing_access_keys_sl:
results:
[
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "xyz-11"
}
]
},
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "abc-12"
}
]
}
]
tasks:
- debug:
var: old_existing_access_keys_sl|json_query('results[*].json[0].id')
This will output:
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => {
"old_existing_access_keys_sl|json_query('results[*].json[0].id')": [
70825621,
70825621
]
}
If you want to store these in a new variable, you can replace that debug task with set_fact:
- set_fact:
admin_ids: "{{ old_existing_access_keys_sl|json_query('results[*].json[0].id') }}"
Update
For a list of dictionaries, just change the json_query expression:
- debug:
var: "old_existing_access_keys_sl|json_query('results[*].json[0].{id: id, username: username}')"
For more information, see the jmespath website for documentation and examples.

Related

Ansible: delete one route from route table in AWS

I have a route table in AWS where there is a subnet getting routed to one host for each host. I can setup those routes automatically using this code:
- name: Add route to host container network
ec2_vpc_route_table:
region: region
vpc_id: "vpc-somestring"
purge_subnets: false
purge_routes: false
lookup: id
route_table_id: rtb-somestring
routes:
- dest: "1.2.3.0/24"
instance_id: "i-somestring"
This is fine for creating new hosts automatically. But if I want to remove a host, I want to delete the matching route table entry.
I thought, I could just fetch the route table using ec2_vpc_route_table_info, then take the routes filtered with rejectattr and feed it back to ec2_vpc_route_table, replacing the whole table. But, info gives me this format of routing tables:
"all_routes": [
{
"destination_cidr_block": "1.2.3.0/24",
"gateway_id": null,
"instance_id": "i-somestring",
"instance_owner_id": "1234567890",
"interface_id": "eni-somestring",
"network_interface_id": "eni-somestring",
"origin": "CreateRoute",
"state": "active"
},
{
"destination_cidr_block": "5.5.5.0/21",
"gateway_id": "local",
"instance_id": null,
"interface_id": null,
"network_interface_id": null,
"origin": "CreateRouteTable",
"state": "active"
},
{
"destination_cidr_block": null,
"destination_ipv6_cidr_block": "affe:affe:affe:affe::/56",
"gateway_id": "local",
"instance_id": null,
"interface_id": null,
"network_interface_id": null,
"origin": "CreateRouteTable",
"state": "active"
}
]
However, I can't feed that table to ec2_vpc_route_table, because that module just wants a list looking like this:
[
{
"dest": "1.2.3.0/24",
"instance_id": "i-somestring"
},
{
"dest": "5.5.5.0/21",
"gateway_id": "local
},
{
"dest": "affe:affe:affe:affe::/56",
"gateway_id": "local"
}
]
Why is the output of the info module not in a format that I can feed back to the route_table module? How can I convert the output into a format that I can feed back to the route_table module?
Thanks for any input.
a sample of solution:
- hosts: localhost
gather_facts: false
vars:
all_routes: "{{ lookup('file', 'zson.json') | from_json }}"
tasks:
- name: display json
debug:
var: all_routes
- name: create new json
set_fact:
result: "{{ result | d([]) + [{ 'dest': _block, _key: _gateway }] }}"
vars:
_block: "{{ item.destination_cidr_block if item.destination_cidr_block != None else item.destination_ipv6_cidr_block }}"
_gateway: "{{ item.gateway_id if item.gateway_id != None else item.instance_id }}"
_key: "{{ 'gateway_id' if item.gateway_id != None else 'instance_id' }}"
loop: "{{all_routes }}"
- name: display result
debug:
var: result
result:
ok: [localhost] => {
"result": [
{
"dest": "1.2.3.0/24",
"instance_id": "i-somestring"
},
{
"dest": "5.5.5.0/21",
"gateway_id": "local"
},
{
"dest": "affe:affe:affe:affe::/56",
"gateway_id": "local"
}
]
}

Ansible, how to query deeply nested json keys

I'm using Ansible to make an API call that returns a huge data set, and I need to be able to get a nested value to print to the screen. I tried using json_query but not sure what I'm doing wrong.
My task:
- name: Get certificate by CN name.
uri:
method: GET
url: "https://mayapi/api/1/certificates?filter=cn;{{ inventory_hostname }}"
headers:
Authorization: Bearer {{ login.json.token }}
Content-Type: application/json
validate_certs: no
register: certs
- name: Print certs for application
debug:
msg: "{{ certs.json | json_query(items) }}"
This is a small snippet of the output. I want to be able to print ID, and email.
{
"msg": {
"changed": false,
"connection": "close",
"content_length": "65833",
"content_type": "application/json",
"cookies": {},
"cookies_string": "",
"date": "Mon, 10 May 2021 21:33:29 GMT",
"elapsed": 0,
"failed": false,
"json": {
"items": [
{
"active": true,
"application": [
{
"director": {
"active": true,
"email": "user#domain.com",
"fullname": "John Doe",
"id": 1611,
"manager": "John Doe",
"managerEmail": "johndoe#email.com",
"username": "jdoe"
},
...
...
...
}
I get the following error indicating "certs.items" doesn't exist:
FAILED! => {"msg": "Error in jmespath.search in json_query filter plugin:\n'items' is undefined"}
I was expecting all of the items to get printed to the screen and then if I wanted something below items I would do items.active, items.application, etc... But this is not correct since I keep erroring out.
I also tried looping through cert.json and cert.json.items:
- name: Print certs for application
debug:
msg: "{{ item.application.name }}"
loop: "{{ certs.json}}"
But get this error message:
{"msg": "Invalid data passed to 'loop', it requires a list, got this instead: {u'items': [{u'status': u'Active-Pending Install'...shows all the data of the nested json
Then I tried this:
- name: Print certs for application
debug:
msg: "{{ item.application.name }}"
loop: "{{ certs.json.items}}"
But got this error message:
{"msg": "Invalid data passed to 'loop', it requires a list, got this instead: <built-in method items of dict object at 0x7f0c9ec43050>. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup."}
Made some progress with this:
- name: Print certs for application
debug:
msg: "KEY:::: {{ item.key }}, VALUE:::: {{ item.value.0 }}"
loop: "{{ lookup('dict', certs.json) }}"
when: "'items' in item.key"
ignore_errors: yes
But this only prints items in index 0 of the list:
"msg": "KEY:::: items, VALUE:::: {u'status': u'Active-Pending Install', u'serialHex': u'1111', u'validityStart': u'2021-05-10T21:01:36+00:00', u'cn': u'node2.test.corp.net', u'validityEnd': u'2023-05-10T21:11:36+00:00', u'application': [{u'uuid': u'2222', u'name': u'abc'}], u'certType': u'CertType.INTERNAL', u'id': 2582, u'issuer': u'server1'}"
I'm trying to print the 'cn', 'id', and 'serialHex' values from each list element for the key 'items'.
This is the data set that I'm trying to query with Ansible:
{
"total": 2,
"items": [
{
"application": [
{
"uuid": "111",
"name": "CDE"
}
],
"validityEnd": "2023-05-10T21:11:36+00:00",
"certType": "CertType.INTERNAL",
"issuer": "server1",
"id": 2582,
"validityStart": "2021-05-10T21:01:36+00:00",
"status": "Active-Pending Install",
"serialHex": "aaa",
"cn": "node2.corp.net"
},
{
"application": [
{
"uuid": "222",
"name": "CDE"
}
],
"validityEnd": "2023-05-10T21:05:26+00:00",
"certType": "CertType.INTERNAL",
"issuer": "server1",
"id": 2581,
"validityStart": "2021-05-10T20:55:26+00:00",
"status": "Active-Pending Install",
"serialHex": "bbbb",
"cn": "node1.corp.net"
}
]
}
You are regrettably stepping on a quirk of "objects in ansible are python dicts" in that .items() and .keys() and quite a few other attributes-which-are-methods cannot be referenced using the . notation since jinja2 believes you intend to call that method. Rather, one must use the __getitem__ syntax of ["items"] in order to make it abundantly clear that you mean the dict key, and not the method of the same name
tasks:
- name: use json_query as you were originally asking
debug:
msg: >-
{{ certs.json | json_query('items[*].{c: cn,i: id,s: serialHex}') }}
- name: or a jinja2 for loop as you separately attempted
debug:
msg: >-
[
{%- for i in certs.json["items"] -%}
{{ "" if loop.first else "," }}
{{ [i.cn, i.id, i.serialHex ]}}
{%- endfor -%}
]
produces the output from their respective steps:
TASK [debug] ******************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"c": "node2.corp.net",
"i": 2582,
"s": "aaa"
},
{
"c": "node1.corp.net",
"i": 2581,
"s": "bbbb"
}
]
}
TASK [debug] ******************************************************************************************************************************
ok: [localhost] => {
"msg": [
[
"node2.corp.net",
2582,
"aaa"
],
[
"node1.corp.net",
2581,
"bbbb"
]
]
}

How do I work with Ansible list of dicts?

I'm so confused with this. If I have a file containing:
users:
- name: jconnor
first: john
last: connor
uid: 3003
- name: sconnor
first: sarah
last: connor
uid: 3001
How do I get the details of each user? With this simple playbook:
- name: create users
hosts: localhost
gather_facts: false
tasks:
- name: Include vars
include_vars:
file: user_list.yml
name: users
- name: debug
debug:
msg: "{{ item }}"
with_dict: "{{ users }}"
I get the following which I can't use:
ok: [localhost] => (item={'value': [{u'last': u'connor', u'uid': 3003, u'name': u'jconnor', u'first': u'john'}, {u'last': u'connor', u'uid': 3001, u'name': u'sconnor', u'first': u'sarah'}], 'key': u'users'}) => {
"msg": {
"key": "users",
"value": [
{
"first": "john",
"last": "connor",
"name": "jconnor",
"uid": 3003
},
{
"first": "sarah",
"last": "connor",
"name": "sconnor",
"uid": 3001
}
]
}
}
I want to create user accounts with this but I simply don't understand how to use this structure.
Note that this is part of a larger structure and I can't change it.
Thanks
Since the users variable is a list of dicts, you should loop with loop or with_items. Then we can access the key of each dict with item.key. E.g.: item.name, item.uid, etc.
Note that you are importing the variables from the file with the name users. So this variable now contains the users hash of that file. If you skip name: users in include_var, then you can directly access the users dict while looping.
tasks:
- include_vars:
file: user_list.yml
name: users
- debug:
msg: "Username is {{ item.name }}, full name is {{ item.first }} {{ item.last }}, userid is {{ item.uid }}"
with_items: "{{ users.users }}"
This outputs message (showing 1 item):
ok: [localhost] => (item={u'last': u'connor', u'uid': 3003, u'name': u'jconnor', u'first': u'john'}) => {
"msg": "Username is jconnor, full name is john connor, userid is 3003"
}

Tell Ansible to iterate through a (network camera) configuration json file and update every value via a new api call

I've got a .json file filled with hundreds of configuration values for an Axis network camera. The contents look like this:
{
"Name": "HTTPS.Port",
"Value": "443"
},
{
"Name": "Image.DateFormat",
"Value": "YYYY-MM-DD"
},
{
"Name": "Image.MaxViewers",
"Value": "20"
},
The Axis API, called Vapix, only provides an update function that updates a single value, so I have to circle through the values and trigger a new API call with every iteration:
name: update parameters
local_action:
module: uri
user: x
password: y
url: "{{ axis_snmp_role.server_url }}?action=update&{{ item }}"
with_items:
- "SNMP.V2c=yes"
- "SNMP.Enabled=yes"
- "ImageSource.I0.Sensor.ExposureValue=100"
Now the above example requires me to hardcode hundreds of config values into the loop. Is there a way to tell Ansible to go through the camera configuration json, update every value via a new api call and stop when there are no more values left within the json file?
Given the configuration's values is a list. For example
shell> cat data.yml
config: [
{"Name": "HTTPS.Port", "Value": "443"},
{"Name": "Image.DateFormat", "Value": "YYYY-MM-DD"},
{"Name": "Image.MaxViewers", "Value": "20"}]
The play
- hosts: localhost
tasks:
- include_vars:
file: data.json
- debug:
msg: "?action=update&{{ item.Name }}={{ item.Value }}"
loop: "{{ config }}"
gives
ok: [localhost] => (item={u'Name': u'HTTPS.Port', u'Value': u'443'}) => {
"msg": "?action=update&HTTPS.Port=443"
}
ok: [localhost] => (item={u'Name': u'Image.DateFormat', u'Value': u'YYYY-MM-DD'}) => {
"msg": "?action=update&Image.DateFormat=YYYY-MM-DD"
}
ok: [localhost] => (item={u'Name': u'Image.MaxViewers', u'Value': u'20'}) => {
"msg": "?action=update&Image.MaxViewers=20"
}
If this is what you want loop the uri module. For example
- local_action:
module: uri
user: x
password: y
url: "{{ axis_snmp_role.server_url }}?action=update&{{ item.Name }}={{ item.Value }}"
loop: "{{ config }}"

Inner loop in loop over dict in Ansible

How to loop over dict in this example?
I can loop over list of strings (lorem:), but I can not loop over objects: dict.
I am using module which needs as input list and dict, so I have be able use both cases, but before module call I have to do some workaround with keys from dict so I have to be able reference it.
I am not sure what I am doing wrong. Can anyone show me a proper example?
Thanks
---
- name: test
hosts: localhost
connection: local
vars:
persons:
foo:
name: john
state: us
objects:
phone: samsung
color: black
capacity: 32
lorem:
- 1
- 2
- 3
bar:
name: helmut
state: de
objects:
phone: lg
color: red
capacity: 16
lorem:
- 4
- 5
- 6
tasks:
- name: List of strings is OK
debug:
msg: "{{ item.0.value.name }} and object: {{ item.1 }}"
loop: "{{ persons | dict2items |subelements('value.lorem',{ 'skip_missing': True }) }}"
- name: Dict referencing key:value is not OK
debug:
msg: "Name: {{ item.0.value.name }} and object: {{ item.1.[value] }} with key name: {{ item.1.[key]}}"
loop: "{{ persons | dict2items |subelements('value.objects',{ 'skip_missing': True }) }}"
Produces an error: fatal: [localhost]: FAILED! => {"msg": "the key 'objects' should point to a list, got {u'color': u'black', u'phone': u'samsung', u'capacity': 32}"}
Best I can think of is to do something like this. Using jinja to generate a list that returns what you need.
---
- name: test
hosts: localhost
connection: local
vars:
persons:
foo:
name: john
state: us
objects:
phone: samsung
color: black
capacity: 32
lorem:
- 1
- 2
- 3
bar:
name: helmut
state: de
objects:
phone: lg
color: red
capacity: 16
lorem:
- 4
- 5
- 6
tasks:
- debug:
msg: |
[
{% for p in persons %}
{% for o in persons[p].objects %}
{
"name": "{{ persons[p].name }}",
"key": "{{ o }}",
"value": "{{ persons[p].objects[o] }}"
},
{% endfor %}
{% endfor %}
]
Output
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": [
{
"key": "color",
"name": "john",
"value": "black"
},
{
"key": "phone",
"name": "john",
"value": "samsung"
},
{
"key": "capacity",
"name": "john",
"value": "32"
},
{
"key": "color",
"name": "helmut",
"value": "red"
},
{
"key": "phone",
"name": "helmut",
"value": "lg"
},
{
"key": "capacity",
"name": "helmut",
"value": "16"
}
]
}
Oh, and if you wanted to use that in a loop, just something like this.
- debug:
msg: "{{ item.name }} {{ item.key }} {{ item.value }}"
loop_control:
label: "{{ item.name }} {{ item.key }} {{ item.value }}"
loop: |
[
{% for p in persons %}
{% for o in persons[p].objects %}
{
"name": "{{ persons[p].name }}",
"key": "{{ o }}",
"value": "{{ persons[p].objects[o] }}"
},
{% endfor %}
{% endfor %}
]
Output
TASK [debug] *******************************************************************
ok: [localhost] => (item=john color black) => {
"msg": "john color black"
}
ok: [localhost] => (item=john phone samsung) => {
"msg": "john phone samsung"
}
ok: [localhost] => (item=john capacity 32) => {
"msg": "john capacity 32"
}
ok: [localhost] => (item=helmut color red) => {
"msg": "helmut color red"
}
ok: [localhost] => (item=helmut phone lg) => {
"msg": "helmut phone lg"
}
ok: [localhost] => (item=helmut capacity 16) => {
"msg": "helmut capacity 16"
}
You can use with_items as well.
Please refer below code:
playbook -->
---
- name: test hosts: localhost gather_facts: no tasks:
- include_vars: vars.yml
- name: debug
debug:
msg: "{{ item.vars.persons }}"
with_items:
- "{{ vars }}"
output --->
"msg": {
"bar": {
"lorem": [
4,
5,
6
],
"name": "helmut",
"objects": {
"capacity": 16,
"color": "red",
"phone": "lg"
},
"state": "de"
},
"foo": {
"lorem": [
1,
2,
3
],
"name": "john",
"objects": {
"capacity": 32,
"color": "black",
"phone": "samsung"
},
"state": "us"
}
}
}

Resources