how to filter e filter string from list in ansible - ansible

this is the list which i have filtered from a nested json using the below json_query
Please see the content of tmpdata
{
"content_length": "4883",
"status": 200,
"cookies": {},
"changed": false,
"strict_transport_security": "max-age=15768000",
"x_api_total_time": "0.234s",
"_ansible_item_result": true,
"_ansible_no_log": false,
"json": {
"job_tags": "",
"type": "job",
"scm_revision": "",
"status": "successful",
"credential": null,
"force_handlers": false,
"job_slice_number": 0,
"ask_credential_on_launch": false,
"started": "2021-02-24T06:48:29.884643Z",
"ask_job_type_on_launch": false,
"start_at_task": "",
"event_processing_finished": true,
"elapsed": 27.82,
"finished": "2021-02-24T06:48:57.704791Z",
"ask_variables_on_launch": true,
"ask_limit_on_launch": false,
"job_slice_count": 1,
"ask_skip_tags_on_launch": false,
"extra_vars": "{\"domain\": \"abc-cn-1\", \"mgmt_password\": \"admin123\", \"mgmt_server\": \"192.168.20.30\", \"fingerprint\": \"9125544D272B5AD28F3E9AB7BC8AA3F276E45064\", \"fwcmd\": \"fw sam -v -J src 192.68.10.10\", \"targets\": [\"abc-cn-c1\"]}",
"use_fact_cache": false,
"name": "test-Checkpoint",
"created": "2021-02-24T06:48:29.003796Z",
"url": "/api/v2/jobs/1198/",
"vault_credential": null,
"verbosity": 4,
"job_args": "[\"ansible-playbook\", \"-u\", \"root\", \"-vvvv\", \"-i\", \"/tmp/awx_1198_e4iie_fm/tmpz4bfietf\", \"-e\", \"#/tmp/awx_1198_e4iie_fm/env/extravars\", \"chkpoint.yml\"]",
"modified": "2021-02-24T06:48:29.720325Z",
"unified_job_template": 7,
"project": 6,
"limit": "",
"timeout": 0,
"host_status_counts": {
"ok": 1
},
"result_traceback": "",
"launch_type": "manual"
},
"server": "nginx/1.12.2",
"connection": "close",
"_ansible_parsed": true,
"allow": "GET, DELETE, HEAD, OPTIONS",
"redirected": false,
"cookies_string": "",
"_ansible_ignore_errors": null
}
i am using the below json_query to filter the output from the tempdata
- name: "search for traget fw"
set_fact:
job_status2: "{{ tmpdata | json_query('[json.status, json.extra_vars]') }}"
please see the output from the above facts, you can see dict inside the list and i want to filter only expected output mentioned below
current output:
ok: [localhost] => {
"msg": [
"successful",
"{\"domain\": \"abc-cn-1\", \"mgmt_password\": \"admxxxxxin123\", \"mgmt_server\": \"192.168.10.20\", \"fingerprint\": \"9125544D272B5AD28F3E9AB7BC8AA3F276E45064\", \"fwcmd\": \"fw sam -v -J src 192.68.10.10\", \"targets\": [\"abc-cn-c1\"]}"
]
}
expected output:
["successful", targets": ["abc-cn-c1"]]
please help me id if this is possible

There are (at least) two answers to your question if you want to continue to use json_query, and one that sidesteps that mess entirely. We'll start with the latter because it's less typing for me :-)
- set_fact:
job_status2: '{{ [ tmpdata.json.status, (tmpdata.json.extra_vars|from_json).targets] }}'
the second is a slight variation on that, in that someone needs to resolve that embedded JSON to a dict and (AFAIK) JMESPath has no such ability
- set_fact:
job_status2: "{{ newdata | json_query('[json.status, json.extra_vars.targets]') }}"
vars:
newdata: >-
{{ tmpdata
| combine({"json":{"extra_vars", tmpdata.json.extra_vars|from_json}}, recursive=True)
}}
I mention that one mostly for the case where you are doing other things with json_query and thus it would be advantageous to know how to coerce the structure to be friendlier for JMESPath
and finally, you can cheat and use the JMESPath literal notation to embed the already JSON-ified string right into the query
- set_fact:
job_status2: "{{ newdata | json_query('[json.status, `' + newdata.json.extra_vars + '`.targets]') }}"

Related

Ansible, how to query deeply nested json keys

I'm using Ansible to make an API call that returns a huge data set, and I need to be able to get a nested value to print to the screen. I tried using json_query but not sure what I'm doing wrong.
My task:
- name: Get certificate by CN name.
uri:
method: GET
url: "https://mayapi/api/1/certificates?filter=cn;{{ inventory_hostname }}"
headers:
Authorization: Bearer {{ login.json.token }}
Content-Type: application/json
validate_certs: no
register: certs
- name: Print certs for application
debug:
msg: "{{ certs.json | json_query(items) }}"
This is a small snippet of the output. I want to be able to print ID, and email.
{
"msg": {
"changed": false,
"connection": "close",
"content_length": "65833",
"content_type": "application/json",
"cookies": {},
"cookies_string": "",
"date": "Mon, 10 May 2021 21:33:29 GMT",
"elapsed": 0,
"failed": false,
"json": {
"items": [
{
"active": true,
"application": [
{
"director": {
"active": true,
"email": "user#domain.com",
"fullname": "John Doe",
"id": 1611,
"manager": "John Doe",
"managerEmail": "johndoe#email.com",
"username": "jdoe"
},
...
...
...
}
I get the following error indicating "certs.items" doesn't exist:
FAILED! => {"msg": "Error in jmespath.search in json_query filter plugin:\n'items' is undefined"}
I was expecting all of the items to get printed to the screen and then if I wanted something below items I would do items.active, items.application, etc... But this is not correct since I keep erroring out.
I also tried looping through cert.json and cert.json.items:
- name: Print certs for application
debug:
msg: "{{ item.application.name }}"
loop: "{{ certs.json}}"
But get this error message:
{"msg": "Invalid data passed to 'loop', it requires a list, got this instead: {u'items': [{u'status': u'Active-Pending Install'...shows all the data of the nested json
Then I tried this:
- name: Print certs for application
debug:
msg: "{{ item.application.name }}"
loop: "{{ certs.json.items}}"
But got this error message:
{"msg": "Invalid data passed to 'loop', it requires a list, got this instead: <built-in method items of dict object at 0x7f0c9ec43050>. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup."}
Made some progress with this:
- name: Print certs for application
debug:
msg: "KEY:::: {{ item.key }}, VALUE:::: {{ item.value.0 }}"
loop: "{{ lookup('dict', certs.json) }}"
when: "'items' in item.key"
ignore_errors: yes
But this only prints items in index 0 of the list:
"msg": "KEY:::: items, VALUE:::: {u'status': u'Active-Pending Install', u'serialHex': u'1111', u'validityStart': u'2021-05-10T21:01:36+00:00', u'cn': u'node2.test.corp.net', u'validityEnd': u'2023-05-10T21:11:36+00:00', u'application': [{u'uuid': u'2222', u'name': u'abc'}], u'certType': u'CertType.INTERNAL', u'id': 2582, u'issuer': u'server1'}"
I'm trying to print the 'cn', 'id', and 'serialHex' values from each list element for the key 'items'.
This is the data set that I'm trying to query with Ansible:
{
"total": 2,
"items": [
{
"application": [
{
"uuid": "111",
"name": "CDE"
}
],
"validityEnd": "2023-05-10T21:11:36+00:00",
"certType": "CertType.INTERNAL",
"issuer": "server1",
"id": 2582,
"validityStart": "2021-05-10T21:01:36+00:00",
"status": "Active-Pending Install",
"serialHex": "aaa",
"cn": "node2.corp.net"
},
{
"application": [
{
"uuid": "222",
"name": "CDE"
}
],
"validityEnd": "2023-05-10T21:05:26+00:00",
"certType": "CertType.INTERNAL",
"issuer": "server1",
"id": 2581,
"validityStart": "2021-05-10T20:55:26+00:00",
"status": "Active-Pending Install",
"serialHex": "bbbb",
"cn": "node1.corp.net"
}
]
}
You are regrettably stepping on a quirk of "objects in ansible are python dicts" in that .items() and .keys() and quite a few other attributes-which-are-methods cannot be referenced using the . notation since jinja2 believes you intend to call that method. Rather, one must use the __getitem__ syntax of ["items"] in order to make it abundantly clear that you mean the dict key, and not the method of the same name
tasks:
- name: use json_query as you were originally asking
debug:
msg: >-
{{ certs.json | json_query('items[*].{c: cn,i: id,s: serialHex}') }}
- name: or a jinja2 for loop as you separately attempted
debug:
msg: >-
[
{%- for i in certs.json["items"] -%}
{{ "" if loop.first else "," }}
{{ [i.cn, i.id, i.serialHex ]}}
{%- endfor -%}
]
produces the output from their respective steps:
TASK [debug] ******************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"c": "node2.corp.net",
"i": 2582,
"s": "aaa"
},
{
"c": "node1.corp.net",
"i": 2581,
"s": "bbbb"
}
]
}
TASK [debug] ******************************************************************************************************************************
ok: [localhost] => {
"msg": [
[
"node2.corp.net",
2582,
"aaa"
],
[
"node1.corp.net",
2581,
"bbbb"
]
]
}

ansible problem with single quote, double quote in uri body

I have a problem with the correct syntax to address an API with absolutely don't understands any json arguments.
In the body must be exactly one value in this form '["XXXX"]' so that it works.
- uri:
url: "{{ AssociatePolicies_url }}"
method: POST
return_content: yes
validate_certs: no
body_format: form-urlencoded
body:
apiKey: "{{ apiKey }}"
ASSIGNED_POLICIES: '["XXXX"]'
RESOURCEID: "{{ resourceId }}"
RUNCOMPCHECK_STATUS: true
headers:
Content-Type: "application/x-www-form-urlencoded;charset=UTF-8"
but when I pass the value now as variable i don't get it in the correct form
---
- name: Post to API
hosts: KingKong
connection: local
gather_facts: no
vars:
policygroup:
- XXXX
# - YYYY
# - ZZZZ
tasks:
- uri:
url: "{{ AssociatePolicies_url }}"
method: POST
return_content: yes
validate_certs: no
body_format: form-urlencoded
body:
apiKey: "{{ apiKey }}"
ASSIGNED_POLICIES: "'["{{ item }}"]'"
RESOURCEID: "{{ resourceId }}"
RUNCOMPCHECK_STATUS: true
headers:
Content-Type: "application/x-www-form-urlencoded;charset=UTF-8"
with_items: "{{ policygroup }}"
Is the response: Error while parsing ASSIGNED_POLICIES json data
After various approaches I don't get any closer to the result:
ASSIGNED_POLICIES: "\'[{{ item }}]\'" is the format '[XXXX]
ASSIGNED_POLICIES: \'["{{ item }}"]\' is the format \\'[\"XXXX\"]\\'
ASSIGNED_POLICIES: "\'[\"{{ item }}\"]\'" is the format '[\"XXXX\"]'
ASSIGNED_POLICIES: "\'["{{ item }}"]\'" playbook error
ASSIGNED_POLICIES: "\'[{{ \" }}{{ item }}{{\"}}]\'" is the format '[ }}{{ item }}{{]'"}]
where I get almost closer to the format is:
ASSIGNED_POLICIES: "\'['{{ item }}']\'" is the format "'['XXXX']'"
but with single quote in front anb ack of XXXX but there must be double quote.
Maybe one of you has a creative idea
I have corrected my source code above and post below what I get back as a response.
What i see in see response:
"content": "{\"error\":{\"message\":\"Error while parsing ASSIGNED_POLICIES json data.\",\"errorcode\":\"JSON_PARSE_ERROR\"}}\n"
and the wrong format:
"ASSIGNED_POLICIES": "'[XXXX]'"
It is missing double quote in front and behind XXXX.
this must be the correct post
"ASSIGNED_POLICIES": "'["XXXX"]'"
.
{
"changed": false,
"_ansible_verbose_always": true,
"result_4": {
"msg": "All items completed",
"changed": false,
"results": [
{
"content_length": "104",
"status": 200,
"cookies": {
"opmcsrfcookie": "192d408c-be9b-42f7-985b-56360c06fdd7"
},
"set_cookie": "opmcsrfcookie=192d408c-be9b-42f7-985b-56360c06fdd7;path=/;Secure;priority=high",
"vary": "Accept-Encoding",
"elapsed": 0,
"msg": "OK (104 bytes)",
"content_type": "application/json;charset=UTF-8",
"date": "Wed, 16 Mar 2021 16:22:55 GMT",
"x_frame_options": "deny",
"ansible_loop_var": "item",
"x_xss_protection": "1",
"url": "https://https://AssociatePolicies_url",
"changed": false,
"x_content_type_options": "nosniff",
"connection": "close",
"content": "{\"error\":{\"message\":\"Error while parsing ASSIGNED_POLICIES json data.\",\"errorcode\":\"JSON_PARSE_ERROR\"}}\n",
"failed": false,
"json": {
"error": {
"errorcode": "JSON_PARSE_ERROR",
"message": "Error while parsing ASSIGNED_POLICIES json data."
}
},
"item": 301,
"invocation": {
"module_args": {
"directory_mode": null,
"force": false,
"remote_src": null,
"status_code": [
200
],
"follow": false,
"owner": null,
"body_format": "form-urlencoded",
"client_key": null,
"group": null,
"use_proxy": true,
"unix_socket": null,
"unsafe_writes": null,
"setype": null,
"content": null,
"serole": null,
"follow_redirects": "safe",
"return_content": true,
"client_cert": null,
"body": {
"RUNCOMPCHECK_STATUS": true,
"RESOURCEID": "abcde",
"apiKey": "just_a_Key",
"ASSIGNED_POLICIES": "'[XXXX]'"
},
"url_username": null,
"src": null,
"dest": null,
"selevel": null,
"force_basic_auth": false,
"removes": null,
"http_agent": "ansible-httpget",
"regexp": null,
"url_password": null,
"url": "https://AssociatePolicies_url",
"validate_certs": false,
"seuser": null,
"method": "POST",
"creates": null,
"headers": {
"Content-Type": "application/x-www-form-urlencoded;charset=UTF-8"
},
"delimiter": null,
"mode": null,
"timeout": 30,
"attributes": null,
"backup": null
}
},
"redirected": false,
"cookies_string": "opmcsrfcookie=192d408c-be9b-42f7-985b-56360c06fdd7"
}
]
},
"_ansible_no_log": false
}

Parse / Loop ansible registered variable

Trying to figure out how to filter out the list of UserNames in the output of the following playbook.
- name: Get all users
ome_user_info:
hostname: "{{ dellome_hostname }}"
username: "{{ dellome_username }}"
password: "{{ dellome_password }}"
register: users
Now the output provides the following:
ok: [192.168.1.100] => {
"users": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"deprecations": [
{
"msg": "Distribution Ubuntu 18.04 on host 192.168.1.100 should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.8/reference_appendices/interpreter_discovery.html for more information",
"version": "2.12"
}
],
"failed": false,
"user_info": {
"192.168.1.100": {
"#odata.context": "/api/$metadata#Collection(AccountService.Account)",
"#odata.count": 3,
"value": [
{
"#odata.id": "/api/AccountService/Accounts('10066')",
"#odata.type": "#AccountService.Account",
"Description": "admin",
"DirectoryServiceId": 0,
"Enabled": true,
"Id": "10066",
"IsBuiltin": true,
"Locked": false,
"Name": "admin",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10066')/Permissions",
"RoleId": "10",
"UserName": "admin",
"UserTypeId": 1
},
{
"#odata.id": "/api/AccountService/Accounts('10102')",
"#odata.type": "#AccountService.Account",
"Description": null,
"DirectoryServiceId": 0,
"Enabled": true,
"Id": "10102",
"IsBuiltin": false,
"Locked": false,
"Name": "dell",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10102')/Permissions",
"RoleId": "10",
"UserName": "dell",
"UserTypeId": 1
},
{
"#odata.id": "/api/AccountService/Accounts('10233')",
"#odata.type": "#AccountService.Account",
"Description": null,
"DirectoryServiceId": 10232,
"Enabled": true,
"Id": "10233",
"IsBuiltin": false,
"Locked": false,
"Name": "Domain Users",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10233')/Permissions",
"RoleId": "10",
"UserName": "Domain Users",
"UserTypeId": 2
}
]
}
}
}
}
I was able to determine the following from the output of the registered variable users.
- debug:
var: "{{ users | length }}"
This provides me the length of 5. Which makes sense to an extent. If I start poking into the output I can then determine the following:
- debug:
var: "{{ users.user_info | length }}"
This shows me the length of 1 which makes sense. If I add the var using users.user_info I can then see the output below.
TASK [manage_users : debug] *************************************************************************************************************************************************************
ok: [192.168.1.100] => {
"users.user_info": {
"192.168.1.100": {
"#odata.context": "/api/$metadata#Collection(AccountService.Account)",
"#odata.count": 3,
"value": [
{
"#odata.id": "/api/AccountService/Accounts('10066')",
"#odata.type": "#AccountService.Account",
"Description": "admin",
"DirectoryServiceId": 0,
"Enabled": true,
"Id": "10066",
"IsBuiltin": true,
"Locked": false,
"Name": "admin",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10066')/Permissions",
"RoleId": "10",
"UserName": "admin",
"UserTypeId": 1
},
{
"#odata.id": "/api/AccountService/Accounts('10102')",
"#odata.type": "#AccountService.Account",
"Description": null,
"DirectoryServiceId": 0,
"Enabled": true,
"Id": "10102",
"IsBuiltin": false,
"Locked": false,
"Name": "dell",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10102')/Permissions",
"RoleId": "10",
"UserName": "dell",
"UserTypeId": 1
},
{
"#odata.id": "/api/AccountService/Accounts('10233')",
"#odata.type": "#AccountService.Account",
"Description": null,
"DirectoryServiceId": 10232,
"Enabled": true,
"Id": "10233",
"IsBuiltin": false,
"Locked": false,
"Name": "Domain Users",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10233')/Permissions",
"RoleId": "10",
"UserName": "Domain Users”,
"UserTypeId": 2
}
]
}
}
}
Trying to figure out how I can loop through and get an array of the following: value —> UserName. Essentially I am going to take the following value and loop through and delete users that don’t equal the following. Admin, dell, domain users.
Now one might say you would easily just say while not = to items - then that list would work - I first need to figure out how to search and get the values out. I have tried the following:
- debug:
var: users.user_info().value()
- debug:
var: users.user_info.find('UserName')
- debug:
msg: UserName
loop: users.user_info."192.168.1.100".value
#- debug:
# var: users.user_info."{{ dellome_hostname }}".UserName
#- debug:
# var: "(claims1 | from_json).value"
# msg: "{{ users.user_info.UserName | list }}"
# (output_text.stdout | from_json).ismaster
#- debug:
# msg: "{{ item }}"
#loop: "{{ users.user_info | from_json | list }}"
At the end of this once I understand how to get the data out i can then create a loop to execute the following:
---
- name: Delete a User in Dell OME
ome_user:
hostname: "{{ dellome_hostname }}"
username: "{{ dellome_username }}"
password: "{{ dellome_password }}"
state: "{{ requestedState }}"
name: "{{ requstedUserName }}"
This is where i can then add the loop to eliminate users that don't meet the list of names i provide. Any help would be greatly appreciated.
Here are some of the errors i have run into.
TASK [manage_users : debug] *************************************************************************************************************************************************************
fatal: [192.168.1.100]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{users.user_info().value()}}): 'dict' object is not callable"}
TASK [manage_users : debug] *************************************************************************************************************************************************************
fatal: [192.168.1.100]: FAILED! => {"msg": "template error while templating string: expected name or number. String: {{users.user_info.\"192.168.1.100\".value()}}"}
Or as i am looking at this - if i can figure out a way to create a loop that looks for the roleID and when it is not equal to 10 then delete the user.
Use json_query. The tasks below
- set_fact:
users_rm: "{{ users.user_info|
json_query('*.value[].UserName') }}"
- debug:
var: users_rm
give
users_rm:
- admin
- dell
- Domain Users
You are running the query at the host 192.168.1.100 and the dictionary users comprises the users from this single host only. If there are more hosts in the dictionary the asterisk '*' in the query above would select them all. It would be better to select users for the particular host the query is running at. For example, the task below gives the same result
- set_fact:
users_rm: "{{ users.user_info[inventory_hostname].value|
map(attribute='UserName')|
list }}"

Fetch the values of the variables in ansible

I am trying to list the names whose matches have been found.
- name: search for files containing string
find:
paths: /root/ansible-dir
patterns: "file3.yml"
contains: "{{ item }}"
with_items: "{{ names_list }}"
register: file_match
- name: print file
debug:
msg: "{{ file_match }}"
After the above code is run, the below code gets generated :
"msg": {
"changed": false,
"msg": "All items completed",
"results": [
{
"ansible_loop_var": "item",
"changed": false,
"examined": 14,
"failed": false,
"files": [],
"invocation": {
"module_args": {
"age": null,
"age_stamp": "mtime",
"contains": "node_mem",
"depth": null,
"excludes": null,
"file_type": "file",
"follow": false,
"get_checksum": false,
"hidden": false,
"paths": [
"/root/ansible-dir"
],
"patterns": [
"file3.yml"
],
"recurse": false,
"size": null,
"use_regex": false
}
},
"item": "node_mem",
"matched": 0,
"msg": ""
},
How to fetch the name of the item where the matched attribute is >=1?
Use subelements. The debug below gives you a hint how to iterate the results and fetch files that contain the items
- debug:
msg: "Fetch {{ item.1 }}"
with_subelements:
- "{{ file_match.results }}"
- files
It is not necessary to test the number of files is >=1. If the list of files is empty the iteration will be skipped anyway.

Need to map json to list

I have register variable with below json as value and need to extract name and dn from this JSON to list. tried different options using set_fact without any luck.
{
"nodes": {
"status": -1,
"imdata": [
{
"fabricNode": {
"attributes": {
"status": "",
"dn": "topology/pod-1/node-1",
"name": "NOQCJALAB1"
}
}
},
{
"fabricNode": {
"attributes": {
"status": "",
"dn": "topology/pod-1/node-1",
"name": "NOQCJALAB2"
}
}
}
],
"totalCount": 2,
"changed": false,
"failed": false
},
"changed": false,
"_ansible_verbose_always": true,
"_ansible_no_log": false
}
You need to remove the extraneous calls to map('from_json') because the object is already a dict
- set_fact:
node_names: >-
{{ (nodes.stdout | from_json).data
| map(attribute='fabnode')
| map(attribute='attributes')
| map(attribute='dn')
| list
}}
You would only want those map('from_json') call if your data structure contained embedded JSON, like so:
- set_fact:
inner_value: >-
{{ (example_text | from_json).a_key
| map("from_json")
| map(attribute="inner_key")
| list
}}
vars:
example_text: |
{"a_key": ["{\"inner_key\": \"inner value\"}"]}
It's possible to use json_query. For example the tasks below
- set_fact:
my_list: "{{ nodes.imdata|
json_query('[].{dn: fabricNode.attributes.dn,
name: fabricNode.attributes.name}')
}}"
- debug:
var: my_list
give
"my_list": [
{
"dn": "topology/pod-1/node-1",
"name": "NOQCJALAB1"
},
{
"dn": "topology/pod-1/node-1",
"name": "NOQCJALAB2"
}
]

Resources