Ansible combine two results based on the key - ansible

If someone can suggest a resource to self educate, would be much appreciated. At the moment I am learning by examples based on what works. Sorry if the terminology is incorrect
I have two results in json format (dictionary?) from the commands sent to Netscaler API:
Result1 lists certificate information:
{
"errorcode": 0,
"message": "Done",
"severity": "NONE",
"sslcertkey": [
{
"certkey": "certkey1.pair",
"daystoexpiration": 0,
"status": "Expired",
"subject": "easdm.test.com"
},
{
"certkey": "certkey2.pair",
"daystoexpiration": 0,
"status": "Expired",
"subject": " CN=timer.test.com"
},
Result2 lists which certificate is bound to a virtual server:
{
"errorcode": 0,
"message": "Done",
"severity": "NONE",
"sslcertkey_sslvserver_binding": [
{
"certkey": "certkey1.pair",
"data": "1",
"servername": "easdm_gslb_btfin_pri_lb_vs",
"stateflag": "2",
"version": 1
},
{
"certkey": "certkey2.pair",
"data": "2",
"servername": "timer_gslb_btfin_pri_lb_vs",
"stateflag": "2",
"version": 1
},
I want to combine two results into Result3, so that it will combine info from Result1 with Result2 if "certkey" is matching between the results:
{
"certkey": "certkey1.pair",
"daystoexpiration": 0,
"status": "Expired",
"subject": "easdm.test.com",
"servername": "easdm_gslb_btfin_pri_lb_vs"
},
{
"certkey": "certkey2.pair",
"daystoexpiration": 0,
"status": "Expired",
"subject": " CN=timer.test.com",
"servername": "timer_gslb_btfin_pri_lb_vs"
},
There are 100's of entries in each result and some results in Result1 will not have a match in Result2 as cert is not used anywhere.
I tried using just a simple
- debug:
msg: '{{ result.json.sslcertkey | combine(result2.json.sslcertkey_sslvserver_binding, recursive=True) }}'
but it seem to only show the last match and not all of the matches.

Use filter community.general.lists_mergeby
result3: "{{ result1.sslcertkey|
community.general.lists_mergeby(result2.sslcertkey_sslvserver_binding,
'certkey') }}"
gives
result3:
- certkey: certkey1.pair
data: '1'
daystoexpiration: 0
servername: easdm_gslb_btfin_pri_lb_vs
stateflag: '2'
status: Expired
subject: easdm.test.com
version: 1
- certkey: certkey2.pair
data: '2'
daystoexpiration: 0
servername: timer_gslb_btfin_pri_lb_vs
stateflag: '2'
status: Expired
subject: ' CN=timer.test.com'
version: 1

Related

Getting the values of keys of Ansible JSON output

I have the following JSON data
{
"docker_compose_init_result": {
"changed": true,
"failed": false,
"services": {
"grafana": {
"docker-compose_grafana_1": {
"cmd": [],
"image": "grafana/grafana:8.5.14",
"labels": {
"com.docker.compose.config-hash": "4d0b5dd6e697a8fe5bf5074192770285e54da43ad32cc34ba9c56505cb709431",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "docker-compose",
"com.docker.compose.project.config_files": "/appl/docker-compose/docker-compose-init.yml",
"com.docker.compose.project.working_dir": "/appl/docker-compose",
"com.docker.compose.service": "grafana",
"com.docker.compose.version": "1.29.2"
},
"networks": {
"docker-compose_homeserver-net": {
"IPAddress": "172.20.0.2",
"IPPrefixLen": 16,
"aliases": [
"3d19f54271b2",
"grafana"
],
"globalIPv6": "",
"globalIPv6PrefixLen": 0,
"links": null,
"macAddress": "02:42:ac:14:00:02"
}
},
"state": {
"running": true,
"status": "running"
}
}
},
"node-red": {
"docker-compose_node-red_1": {
"cmd": [],
"image": "nodered/node-red:2.2.2",
"labels": {
"authors": "Dave Conway-Jones, Nick O'Leary, James Thomas, Raymond Mouthaan",
"com.docker.compose.config-hash": "5610863d4b28b11645acb5651e7bab174125743dc86a265969788cc8ac782efe",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "docker-compose",
"com.docker.compose.project.config_files": "/appl/docker-compose/docker-compose-init.yml",
"com.docker.compose.project.working_dir": "/appl/docker-compose",
"com.docker.compose.service": "node-red",
"com.docker.compose.version": "1.29.2",
"org.label-schema.arch": "",
"org.label-schema.build-date": "2022-02-18T21:01:04Z",
"org.label-schema.description": "Low-code programming for event-driven applications.",
"org.label-schema.docker.dockerfile": ".docker/Dockerfile.alpine",
"org.label-schema.license": "Apache-2.0",
"org.label-schema.name": "Node-RED",
"org.label-schema.url": "https://nodered.org",
"org.label-schema.vcs-ref": "",
"org.label-schema.vcs-type": "Git",
"org.label-schema.vcs-url": "https://github.com/node-red/node-red-docker",
"org.label-schema.version": "2.2.2"
},
"networks": {
"docker-compose_homeserver-net": {
"IPAddress": "172.20.0.4",
"IPPrefixLen": 16,
"aliases": [
"fc56e973c98d",
"node-red"
],
"globalIPv6": "",
"globalIPv6PrefixLen": 0,
"links": null,
"macAddress": "02:42:ac:14:00:04"
}
},
"state": {
"running": true,
"status": "running"
}
}
},
"organizr": {
"docker-compose_organizr_1": {
"cmd": [],
"image": "organizr/organizr:linux-amd64",
"labels": {
"base.maintainer": "christronyxyocum,Roxedus",
"base.s6.arch": "amd64",
"base.s6.rel": "2.2.0.3",
"com.docker.compose.config-hash": "430b338b0c0892a25522e1b641a9e3a08eedd255309b1cd275b22a3362dcac58",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "docker-compose",
"com.docker.compose.project.config_files": "/appl/docker-compose/docker-compose-init.yml",
"com.docker.compose.project.working_dir": "/appl/docker-compose",
"com.docker.compose.service": "organizr",
"com.docker.compose.version": "1.29.2",
"maintainer": "christronyxyocum,Roxedus",
"org.label-schema.description": "Baseimage for Organizr",
"org.label-schema.name": "organizr/base",
"org.label-schema.schema-version": "1.0",
"org.label-schema.url": "https://organizr.app/",
"org.label-schema.vcs-url": "https://github.com/organizr/docker-base",
"org.opencontainers.image.created": "2022-05-08_15",
"org.opencontainers.image.source": "https://github.com/Organizr/docker-organizr/tree/master",
"org.opencontainers.image.title": "organizr/base",
"org.opencontainers.image.url": "https://github.com/Organizr/docker-organizr/blob/master/README.md"
},
"networks": {
"docker-compose_homeserver-net": {
"IPAddress": "172.20.0.3",
"IPPrefixLen": 16,
"aliases": [
"organizr",
"f3f61d8938fe"
],
"globalIPv6": "",
"globalIPv6PrefixLen": 0,
"links": null,
"macAddress": "02:42:ac:14:00:03"
}
},
"state": {
"running": true,
"status": "running"
}
}
},
"prometheus": {
"docker-compose_prometheus_1": {
"cmd": [
"--config.file=/etc/prometheus/prometheus.yml",
"--storage.tsdb.path=/prometheus",
"--web.console.libraries=/etc/prometheus/console_libraries",
"--web.console.templates=/etc/prometheus/consoles",
"--web.enable-lifecycle"
],
"image": "prom/prometheus:v2.35.0",
"labels": {
"com.docker.compose.config-hash": "7d2ce7deba1a152ebcf4fe5494384018c514f6703b5e906aef6f2e8820733cb2",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "docker-compose",
"com.docker.compose.project.config_files": "/appl/docker-compose/docker-compose-init.yml",
"com.docker.compose.project.working_dir": "/appl/docker-compose",
"com.docker.compose.service": "prometheus",
"com.docker.compose.version": "1.29.2",
"maintainer": "The Prometheus Authors <prometheus-developers#googlegroups.com>"
},
"networks": {
"docker-compose_homeserver-net": {
"IPAddress": "172.20.0.5",
"IPPrefixLen": 16,
"aliases": [
"04f346e6694f",
"prometheus"
],
"globalIPv6": "",
"globalIPv6PrefixLen": 0,
"links": null,
"macAddress": "02:42:ac:14:00:05"
}
},
"state": {
"running": true,
"status": "running"
}
}
}
}
}
}
And I need an output similar to
- docker-compose_grafana_1
- docker-compose_node-red_1
- docker-compose_organizr_1
- docker-compose_prometheus_1
I can do that with jq easy-peasy:
jq --raw-output '.docker_compose_init_result.services\[\] | keys | .\[\]' jsondata.json
But I am not able to do it with Ansible and especially json_query (and thus JMESPath).
I was able to get one key with
jp -f jsondata.json "keys(docker_compose_init_result.services.grafana)"
[
"docker-compose_grafana_1"
]
But have no idea how to get all four. Also sometimes expressions that worked with jp did not work in Ansible with json_query, which additionally made me mad.
If anyone can give me a solution for this (wether its with json_query or not) in the best case explains how it works, I would be very glad.
Solution using only builtin filters:
docker_compose_list: "{{ docker_compose_init_result.services | dict2items
| map(attribute='value') | map('dict2items')
| flatten | map(attribute='key') }}"
which gives once expanded:
{
"docker_compose_list": [
"docker-compose_grafana_1",
"docker-compose_node-red_1",
"docker-compose_organizr_1",
"docker-compose_prometheus_1"
]
}
In a pure JMESPath way, you query should be:
docker_compose_init_result.services.*.keys(#)[]
Where:
.* is the notation for an object project, that will make you get the values under docker_compose_init_result.services, whatever the key might be.
.keys(#) is the keys() function, in order to get the keys, mapped to the current node — #, which effectively means that keys() is going to be applied to every object which are children of docker_compose_init_result.services.*, (e.g.: on docker_compose_init_result.services.grafana."docker-compose_grafana_1", docker_compose_init_result.services."node-red"."docker-compose_node-red_1", and so on)
[] is the flatten operator, in order to reduce your array of arrays to a single level array
The query below
docker_compose_list: "{{ docker_compose_init_result|
json_query(_query) }}"
_query: 'services.*.keys(#)'
gives
docker_compose_list:
- - docker-compose_grafana_1
- - docker-compose_node-red_1
- - docker-compose_organizr_1
- - docker-compose_prometheus_1
Select first items
docker_compose_list: "{{ docker_compose_init_result|
json_query(_query)|
map('first')|list }}"
or flatten the list
docker_compose_list: "{{ docker_compose_init_result|
json_query(_query)|
flatten }}"
both give
docker_compose_list:
- docker-compose_grafana_1
- docker-compose_node-red_1
- docker-compose_organizr_1
- docker-compose_prometheus_1
Note: Be careful how you select or flatten the list. There might be a reason for the second-level keys. For example, if there are more keys in services.grafana the result might be
docker_compose_list:
- - docker-compose_grafana_1
- docker-compose_grafana_2
- docker-compose_grafana_3
- - docker-compose_node-red_1
- - docker-compose_organizr_1
- - docker-compose_prometheus_1
In this case, taking the first item or flattening the list doesn't necessarily give the result you want.

sputnikdao2 - ChangePolicy - "data did not match any variant of untagged enum VersionedPolicy"

I am trying to change the policy for a deployed sputnikdao2 contract.
I am getting this error:
"ExecutionError":"`Smart contract panicked: panicked at 'Failed to deserialize input from JSON.: Error(\"data did not match any variant of untagged enum VersionedPolicy\", line: 1, column: 423)', src/proposals.rs:384:1`"
},
"transaction_outcome":{
"block_hash":"8aUiGxnJv12BASyKjPKVsYWegEmbH8Lz1LsXu7gGXFwa",
"id":"FTTFLVZzzrK7CT6KCNqWVCs67Hc5oBRHBT9TqCciqjY6",
"outcome":{
"executor_id":"hundred.testnet",
"gas_burnt":2428900339092,
"logs":[
],
"receipt_ids":[
"EuNWubtxcY9YjcbTxSwrrYj59GBVj8u6a8RktQj7tHSh"
],
"status":{
"SuccessReceiptId":"EuNWubtxcY9YjcbTxSwrrYj59GBVj8u6a8RktQj7tHSh"
},
"tokens_burnt":"242890033909200000000"
},
"proof":[
{
"direction":"Left",
"hash":"9eTyjRrHrNP1Bmw4rDgSouGmvxP7Lg3EaoUn15qBQH3h"
},
{
"direction":"Right",
"hash":"4NLf8mPom49oVbXmB2ouujxctjbyZC5FBi5ny1NFcXYj"
}
]
}
}
you can see more information here :
https://gist.github.com/hiba-machfej/3a681d22fc2310966ca7692ec3a189d2
I was trying to send this:
'{"proposal": {"description": "Add New Council", "kind": {"ChangePolicy": { "policy": { "roles": [{ "name": "all", "kind": "Everyone", "permissions": [ "*:AddProposal" ], "vote_policy": "{}"}], "default_vote_policy": { "weight_kind": "RoleWeight", "quorum": "0", "threshold": [ 1, 2 ] }, "proposal_bond": "1000000000000000000000000", "proposal_period": "604800000000000", "bounty_bond": "1000000000000000000000000", "bounty_forgiveness_period": "86400000000000"}}}}}' \
--accountId hundred.testnet \
--amount 1
I re-wrote the objects again and it worked:
'{"proposal": {"description": "Add New Council", "kind": {"ChangePolicy": { "policy": { "roles": [{ "name": "all", "kind": "Everyone", "permissions": ["*:AddProposal", "*:Finalize"], "vote_policy": {}}], "default_vote_policy": { "weight_kind": "RoleWeight", "quorum": "0", "threshold": [ 1, 2 ]}, "proposal_bond": "1000000000000000000000000", "proposal_period": "604800000000000", "bounty_bond": "1000000000000000000000000", "bounty_forgiveness_period": "86400000000000" }}}}}' \
--accountId hundred.testnet \
--amount 1
This is the recipt:
https://explorer.testnet.near.org/transactions/DxXLUUcx2jcLdoCFT2HbhSinWV6zjSREUkNXnN3kkHD4
I think there was an error in json format in the first code I was running.

What the syntax would be to pull the "members" value out of this JSON where the name value is "PROD_poolgroup"

What the syntax would be to pull the "members" value out of this JSON where the name value is "PROD_poolgroup".
I think it should be [?config.name == "PROD_poolgroup"].config.members
I've tried other variations without success also.
"all_members": [
{
"config": {
"_last_modified": "1594434546441212",
"cloud_ref": "https://192.168.86.20/api/cloud/cloud-5a7a41cb-156e-43f4-ad28-031b27dac813",
"implicit_priority_labels": false,
"members": [{
"pool_ref": "https://192.168.86.20/api/pool/pool-cb436bba-000a-4d72-8ff5-7d6cdf1ac2fa",
"ratio": 1
}],
"min_servers": 0,
"name": "PROD_poolgroup",
"tenant_ref": "https://192.168.86.20/api/tenant/admin",
"url": "https://192.168.86.20/api/poolgroup/poolgroup-e08933c0-d142-4787-9990-45f94dfc6b89",
"uuid": "poolgroup-e08933c0-d142-4787-9990-45f94dfc6b89"
},
"uuid": "poolgroup-e08933c0-d142-4787-9990-45f94dfc6b89"},
{
"config": {
"_last_modified": "1594404797173635",
"cloud_ref": "https://192.168.86.20/api/cloud/cloud-5a7a41cb-156e-43f4-ad28-031b27dac813",
"implicit_priority_labels": false,
"members": [{
"pool_ref": "https://192.168.86.20/api/pool/pool-f98ed65c-00b2-4638-83df-8b89df79deec",
"ratio": 1
}],
"min_servers": 0,
"name": "QA_poolgroup",
"tenant_ref": "https://192.168.86.20/api/tenant/admin",
"url": "https://192.168.86.20/api/poolgroup/poolgroup-0b19b515-9ae6-41ef-9111-00e61470e615",
"uuid": "poolgroup-0b19b515-9ae6-41ef-9111-00e61470e615"},
"uuid": "poolgroup-0b19b515-9ae6-41ef-9111-00e61470e615"
}
]
Use either single quote or backquote for using literal expression to compare value of name. Below are two working examples:
Using single quote:
- debug:
msg: "{{ all_members | json_query(query) | list }}"
vars:
query: "[?config.name == 'PROD_poolgroup'].config.members"
Using literal expression:
- debug:
msg: "{{ all_members | json_query(query) | list }}"
vars:
query: '[?config.name == `"PROD_poolgroup"`].config.members'

Ansible select sub list from list of dictionaries with matching regex values

I have a yml list that is similar to the following:
name_ips:
rr: [
{
"name": "db1-137.com",
"ip": "1.1.1.1"
},
{
"name": "core1-137.com",
"ip": "2.2.2.2"
},
{
"name": "core0-139.com",
"ip": "3.3.3.3"
},
{
"name": "db0-147.com",
"ip": "4.4.4.4"
}
]
Using ansible I want to select the items in this list with name values matching the regex:
regexp: ".*137.com"
My end result should look similar to the following, which I want to be able to set using the set_fact module:
[
{
"name": "db1-137.com",
"ip": "1.1.1.1"
},
{
"name": "core1-137.com",
"ip": "2.2.2.2"
}
]
How can I achieve this sub-list select operation?
Update:
I found I can return a single value if I state the exact name:
- debug: msg="{{ name_ips | json_query(\"rr[?name=='core0-139.com']\") }}"`
Returns:
{
"name": "core0-139.com",
"ip": "3.3.3.3"
}
This is a start, but I would still like to use a regex for this search.

Error Parsing JSON from YELP API

I am having trouble parsing JSON from Yelp API. The JSON data looks like this:
{
region: {
span: {
latitude_delta: 0,
longitude_delta: 0
},
center: {
latitude: 38.054117,
longitude: -84.439002
}
},
total: 23,
businesses: [
{
is_claimed: false,
rating: 5,
mobile_url: "http://m.yelp.com/biz/vineyard-community-church-lexington",
rating_img_url: "http://s3-media1.ak.yelpcdn.com/assets/2/www/img/f1def11e4e79/ico/stars/v1/stars_5.png",
review_count: 2,
name: "Vineyard Community Church",
snippet_image_url: "http://s3-media4.ak.yelpcdn.com/photo/VoeMtbk7NRFi6diksSUtOQ/ms.jpg",
rating_img_url_small: "http://s3-media1.ak.yelpcdn.com/assets/2/www/img/c7623205d5cd/ico/stars/v1/stars_small_5.png",
url: "http://www.yelp.com/biz/vineyard-community-church-lexington",
phone: "8592582300",
snippet_text: "I have been a member of Vineyard Community Church since 2004. Here you will find a modern worship service with a full band, witty speakers who teach...",
image_url: "http://s3-media3.ak.yelpcdn.com/bphoto/D71eikniuaHjdOC8DB6ziA/ms.jpg",
categories: [
[
"Churches",
"churches"
]
],
display_phone: "+1-859-258-2300",
rating_img_url_large: "http://s3-media3.ak.yelpcdn.com/assets/2/www/img/22affc4e6c38/ico/stars/v1/stars_large_5.png",
id: "vineyard-community-church-lexington",
is_closed: false,
location: {
city: "Lexington",
display_address: [
"1881 Eastland Pwky",
"Lexington, KY 40505"
],
geo_accuracy: 8,
postal_code: "40505",
country_code: "US",
address: [
"1881 Eastland Pwky"
],
coordinate: {
latitude: 38.054117,
longitude: -84.439002
},
state_code: "KY"
}
}
]
}
The JSON is stored in a ruby string called #stuff
Here is the code I use to try and parse it:
#parsed_stuff = JSON::parse(#stuff)
When i do this and try and display the contents of # parsed_stuff, i get the following error in the browser
Parse error on line 2: { "region"=>{ "span -------------^ Expecting '}', ':', ',', ']'
Any help given on this issue will be highly appreciated.
Use jsonlint for validating JSON. Here you have to give all keys as a string.
Try it
{
"region": {
"span": {
"latitude_delta": 0,
"longitude_delta": 0
},
"center": {
"latitude": 38.054117,
"longitude": -84.439002
}
},
"total": 23,
"businesses": [
{
"is_claimed": false,
"rating": 5,
"mobile_url": "http://m.yelp.com/biz/vineyard-community-church-lexington",
"rating_img_url": "http://s3-media1.ak.yelpcdn.com/assets/2/www/img/f1def11e4e79/ico/stars/v1/stars_5.png",
"review_count": 2,
"name": "Vineyard Community Church",
"snippet_image_url": "http://s3-media4.ak.yelpcdn.com/photo/VoeMtbk7NRFi6diksSUtOQ/ms.jpg",
"rating_img_url_small": "http://s3-media1.ak.yelpcdn.com/assets/2/www/img/c7623205d5cd/ico/stars/v1/stars_small_5.png",
"url": "http://www.yelp.com/biz/vineyard-community-church-lexington",
"phone": "8592582300",
"snippet_text": "I have been a member of Vineyard Community Church since 2004. Here you will find a modern worship service with a full band, witty speakers who teach...",
"image_url": "http://s3-media3.ak.yelpcdn.com/bphoto/D71eikniuaHjdOC8DB6ziA/ms.jpg",
"categories": [
[
"Churches",
"churches"
]
],
"display_phone": "+1-859-258-2300",
"rating_img_url_large": "http://s3-media3.ak.yelpcdn.com/assets/2/www/img/22affc4e6c38/ico/stars/v1/stars_large_5.png",
"id": "vineyard-community-church-lexington",
"is_closed": false,
"location": {
"city": "Lexington",
"display_address": [
"1881 Eastland Pwky",
"Lexington, KY 40505"
],
"geo_accuracy": 8,
"postal_code": "40505",
"country_code": "US",
"address": [
"1881 Eastland Pwky"
],
"coordinate": {
"latitude": 38.054117,
"longitude": -84.439002
},
"state_code": "KY"
}
}
]
}

Resources