Select specific cluster with prefix in VMware cluster from Ansible - ansible

I'm using the below playbook to list all the VMware cluster belonging to my datacenter, but I need to select a specific cluster containing LAB in the cluster name.
- hosts: localhost
vars_files: 1credentials.yml
tasks:
- name: Gather cluster info from given datacenter
community.vmware.vmware_cluster_info:
hostname: '{{ vcenter_hostname }}'
username: '{{ vcenter_username }}'
password: '{{ vcenter_password }}'
datacenter: SDx-CoE-T7
validate_certs: no
delegate_to: localhost
register: cluster_info
- debug:
msg: "{{ cluster_info }}"
- debug:
msg: "{{ cluster_info.clusters.keys() | list | to_yaml }}"
I'm trying to use selectattr(), but, I am not able to get the exact syntax. Can you please suggest the best option to filter the cluster name containing LAB.
Below is the playbook output:
PLAY [localhost] *******************************************************************************************************************************************************
TASK [Gathering Facts] *************************************************************************************************************************************************
ok: [localhost]
TASK [Gather cluster info from given datacenter] ***********************************************************************************************************************
ok: [localhost]
TASK [debug] ***********************************************************************************************************************************************************
ok: [localhost] => {
"msg": {
"changed": false,
"clusters": {
"TEST_LAB": {
"datacenter": "TEST_CoE-T7",
"drs_default_vm_behavior": "fullyAutomated",
"drs_enable_vm_behavior_overrides": true,
"drs_vmotion_rate": 3,
"enable_ha": false,
"enabled_drs": false,
"enabled_vsan": false,
"ha_admission_control_enabled": true,
"ha_failover_level": 1,
"ha_host_monitoring": "enabled",
"ha_restart_priority": [
"medium"
],
"ha_vm_failure_interval": [
30
],
"ha_vm_max_failure_window": [
-1
],
"ha_vm_max_failures": [
3
],
"ha_vm_min_up_time": [
120
],
"ha_vm_monitoring": "vmMonitoringDisabled",
"ha_vm_tools_monitoring": [
"vmMonitoringDisabled"
],
"hosts": [],
"moid": "domain-c28615",
"resource_summary": {
"cpuCapacityMHz": 0,
"cpuUsedMHz": 0,
"memCapacityMB": 0,
"memUsedMB": 0,
"pMemAvailableMB": 0,
"pMemCapacityMB": 0,
"storageCapacityMB": 0,
"storageUsedMB": 0
},
"tags": [],
"vsan_auto_claim_storage": false
},
"TEST_LaaS": {
"datacenter": "TEST_CoE-T7",
"drs_default_vm_behavior": "fullyAutomated",
"drs_enable_vm_behavior_overrides": true,
"drs_vmotion_rate": 3,
"enable_ha": true,
"enabled_drs": true,
"enabled_vsan": false,
"ha_admission_control_enabled": true,
"ha_failover_level": 1,
"ha_host_monitoring": "enabled",
"ha_restart_priority": [
"medium"
],
"ha_vm_failure_interval": [
30
],
"ha_vm_max_failure_window": [
-1
],
"ha_vm_max_failures": [
3
],
"ha_vm_min_up_time": [
120
],
"ha_vm_monitoring": "vmMonitoringDisabled",
"ha_vm_tools_monitoring": [
"vmMonitoringDisabled"
],
"hosts": [
{
"folder": "/TEST_CoE-T7/host/TEST_LaaS",
"name": "172.17.65.84"
},
{
"folder": "/TEST_CoE-T7/host/TEST_LaaS",
"name": "172.17.65.85"
},
{
"folder": "/TEST_CoE-T7/host/TEST_LaaS",
"name": "172.17.168.202"
}
],
"moid": "domain-c861",
"resource_summary": {
"cpuCapacityMHz": 158040,
"cpuUsedMHz": 17494,
"memCapacityMB": 786073,
"memUsedMB": 361592,
"pMemAvailableMB": 0,
"pMemCapacityMB": 0,
"storageCapacityMB": 22707456,
"storageUsedMB": 14326118
},
"tags": [],
"vsan_auto_claim_storage": false
}
},
"failed": false
}
}
TASK [debug] ***********************************************************************************************************************************************************
ok: [localhost] => {
"msg": "[TEST_Automation, TEST_LAB, TEST_LaaS,]\n"
}
If I use ansible version 2.9.27, I'm getting below error:
fatal: [localhost]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{ cluster_info.clusters | dict2items | selectattr('key','contains','Common')| items2dict | list }}): items2dict requires a list, got <class 'generator'> instead."}
I tried with below one removing items2dict:
- debug:
msg: "{{ cluster_info.clusters | dict2items | selectattr('key','contains','TEST') | list }}"
It is giving the below output but I need only the key: TEST_Lab
ok: [localhost] => {
"msg": [
{
"key": "TEST_LAB",
"value": {
"drs_default_vm_behavior": "partiallyAutomated",
"drs_enable_vm_behavior_overrides": true,
"drs_vmotion_rate": 3,
"enable_ha": false,
"enabled_drs": true,
"enabled_vsan": false,
"ha_admission_control_enabled": true,
"ha_failover_level": 1,
"ha_host_monitoring": "enabled",
"ha_restart_priority": [
"medium"
],
"ha_vm_failure_interval": [
30
],
"ha_vm_max_failure_window": [
-1
],
"ha_vm_max_failures": [
3
],
"ha_vm_min_up_time": [
120
],
"ha_vm_monitoring": "vmMonitoringDisabled",
"ha_vm_tools_monitoring": [
"vmMonitoringDisabled"
],
"tags": [],
"vsan_auto_claim_storage": false
}
}
]
}

In order to use selectattr, you need to have a value to filter on, not a key. But transforming a dictionary into a list, containing the key and value is as simple as using the dict2items filter, in Ansible.
Then, to come back to your dictionary, just use the "antonym" filter, items2dict.
So, your debug task ends up being:
- debug:
var: >-
cluster_info.clusters
| dict2items
| selectattr('key','contains','LAB')
| items2dict
Given the task, with a reduced dictionary for readability:
- debug:
var: |-
cluster_info.clusters
| dict2items
| selectattr('key','contains','LAB')
| items2dict
vars:
cluster_info:
clusters:
TEST_LAB:
datacenter: TEST_CoE-T7
drs_default_vm_behavior: fullyAutomated
TEST_LaaS:
datacenter: TEST_CoE-T7
drs_default_vm_behavior: fullyAutomated
This will yield:
ok: [localhost] =>
? |-
cluster_info.clusters
| dict2items
| selectattr('key','contains','LAB')
| items2dict
: TEST_LAB:
datacenter: TEST_CoE-T7
drs_default_vm_behavior: fullyAutomated

Related

Ansible - Extracting a variable from a large JSON output

The gcp_sql_instance_info module does not seem to allow you to specify the name of the database you are trying to get info for. All it does is every detail of every database in that project.
What would be the best way to extract the "ipAddress" of a database based on it's name and register that value to a variable?
{
"resources": [
{
"kind": "sql#instance",
"state": "RUNNABLE",
"databaseVersion": "POSTGRES_13",
"settings": {
"authorizedGaeApplications": [],
"tier": "db-custom-1-3840",
"kind": "sql#settings",
"availabilityType": "ZONAL",
"pricingPlan": "PER_USE",
"replicationType": "SYNCHRONOUS",
"activationPolicy": "ALWAYS",
"ipConfiguration": {
"privateNetwork": "example-network",
"authorizedNetworks": [],
"ipv4Enabled": false
},
"locationPreference": {
"zone": "us-west1-b",
"kind": "sql#locationPreference"
},
"dataDiskType": "PD_SSD",
"maintenanceWindow": {
"kind": "sql#maintenanceWindow",
"hour": 0,
"day": 0
},
"backupConfiguration": {
"startTime": "02:00",
"kind": "sql#backupConfiguration",
"backupRetentionSettings": {
"retentionUnit": "COUNT",
"retainedBackups": 7
},
"enabled": false,
"replicationLogArchivingEnabled": false,
"pointInTimeRecoveryEnabled": false,
"transactionLogRetentionDays": 7
},
"settingsVersion": "3",
"storageAutoResizeLimit": "0",
"storageAutoResize": true,
"dataDiskSizeGb": "100"
},
"ipAddresses": [
{
"type": "PRIVATE",
"ipAddress": "10.10.10.1"
}
],
"instanceType": "CLOUD_SQL_INSTANCE",
"backendType": "SECOND_GEN",
"name": "database-1",
"region": "us-west1",
"gceZone": "us-west1-b",
"databaseInstalledVersion": "POSTGRES_13_7",
"createTime": "2022-07-22T20:20:32.274Z"
},
{
"kind": "sql#instance",
"state": "RUNNABLE",
"databaseVersion": "MYSQL_8_0",
"settings": {
"authorizedGaeApplications": [],
"tier": "db-n1-standard-1",
"kind": "sql#settings",
"availabilityType": "ZONAL",
"pricingPlan": "PER_USE",
"replicationType": "SYNCHRONOUS",
"activationPolicy": "ALWAYS",
"ipConfiguration": {
"privateNetwork": "example-network",
"authorizedNetworks": [],
"ipv4Enabled": false
},
"locationPreference": {
"zone": "us-west1-c",
"kind": "sql#locationPreference"
},
"dataDiskType": "PD_SSD",
"backupConfiguration": {
"startTime": "21:00",
"kind": "sql#backupConfiguration",
"backupRetentionSettings": {
"retentionUnit": "COUNT",
"retainedBackups": 7
},
"enabled": false,
"transactionLogRetentionDays": 7
},
"settingsVersion": "1",
"storageAutoResizeLimit": "0",
"storageAutoResize": true,
"dataDiskSizeGb": "10"
},
"ipAddresses": [
{
"type": "PRIVATE",
"ipAddress": "10.10.10.2"
}
],
"instanceType": "CLOUD_SQL_INSTANCE",
"backendType": "SECOND_GEN",
"name": "database-2",
"region": "us-west1",
"gceZone": "us-west1-c",
"databaseInstalledVersion": "MYSQL_8_0_26",
"createTime": "2022-07-27T19:27:59.235Z"
}
]
Q: "Extract the ipAddress of a database based on its name."
A: Crete a dictionary. For example, select the first item from the list ipAddresses
name_ip: "{{ dict(resources|json_query(name_ip_query)) }}"
name_ip_query: '[].[name, ipAddresses[0].ipAddress]'
give
name_ip:
database-1: 10.10.10.1
database-2: 10.10.10.2
Notes
Example of a complete playbook for testing
- hosts: localhost
vars_files:
- resources.yml
vars:
name_ip: "{{ dict(resources|json_query(name_ip_query)) }}"
name_ip_query: '[].[name, ipAddresses[0].ipAddress]'
name_ver: "{{ dict(resources|json_query(name_ver_query)) }}"
name_ver_query: '[].[name, databaseVersion]'
ip_of_db1: "{{ name_ip['database-1'] }}"
tasks:
- debug:
var: name_ip
- debug:
var: name_ver
- debug:
var: ip_of_db1
gives
TASK [debug] *********************************************************************************
ok: [localhost] =>
name_ip:
database-1: 10.10.10.1
database-2: 10.10.10.2
TASK [debug] *********************************************************************************
ok: [localhost] =>
name_ver:
database-1: POSTGRES_13
database-2: MYSQL_8_0
TASK [debug] *********************************************************************************
ok: [localhost] =>
ip_of_db1: 10.10.10.1
There might be more IP addresses. The variable ipAddresses is a list. An option would be to create a list of all IP addresses. For example,
name_ips: "{{ dict(resources|json_query(name_ips_query)) }}"
name_ips_query: '[].[name, ipAddresses[].ipAddress]'
give
name_ips:
database-1: [10.10.10.1]
database-2: [10.10.10.2]
This is a working playbook to achieve the desired output:
- hosts: localhost
vars:
db_instance_name: instance-1
tasks:
- name: get info on an instance
gcp_sql_instance_info:
project: project_id
auth_kind: serviceaccount
service_account_file: "/path/to/sa/example.json"
register: cloud_sql_resources
- set_fact:
query: "resources[?contains(name,'{{ db_instance_name }}')].ipAddresses[].ipAddress"
- set_fact:
ipAddress: "{{ cloud_sql_resources | to_json | from_json | json_query(query) }}"
- debug:
msg: "{{ ipAddress }}"
You can run your playbook like this: ansible-playbook -v main.yaml --extra-vars "db_instance_name=instance-1", that would print:
TASK [debug] ********************************************************************************************************************
ok: [localhost] => {
"msg": [
"35.224.46.127"
]
}

Ansible / jmespath - 1d list to zipped list of dictionaries

I'm attempting to transform a 1d list of AWS EC2 IDs into a list of dictionaries suitable for usage as the targets param to the Ansible elb_target_group module.
Sample input:
TASK [debug]
ok: [localhost] => {
"instance_ids": [
"i-1111",
"i-2222",
"i-3333"
]
}
Desired output:
TASK [debug]
ok: [localhost] => {
"targets": [
{"Id": "i-1111", "Port": 6443},
{"Id": "i-2222", "Port": 6443},
{"Id": "i-3333", "Port": 6443},
]
}
What I've tried: a json_query / JMESPath expression which confusingly returns null values:
---
- name: test
hosts: localhost
connection: local
vars:
instance_ids:
- i-1111
- i-2222
- i-3333
keys_list:
- Id
- Port
tasks:
- debug: var=instance_ids
- debug:
msg: "{{ instance_ids | zip_longest([6443], fillvalue=6443) | list }}" # Halfway there...
- debug:
msg: "{{ instance_ids | zip_longest([6443], fillvalue=6443) | list | map('json_query', '{Id: [0], Port: [1]}') | list }}"
Output:
TASK [debug]
ok: [localhost] => {
"msg": [
[
"i-1111",
6443
],
[
"i-2222",
6443
],
[
"i-3333",
6443
]
]
}
TASK [debug]
ok: [localhost] => {
"msg": [
{
"Id": null,
"Port": null
},
{
"Id": null,
"Port": null
},
{
"Id": null,
"Port": null
}
]
}
How should I adjust the '{Id: [0], Port: [1]}' part ?
Stumbled my way to a working solution with json_query and a JMESPath int literal:
"{{ instance_ids | json_query('[*].{Id: #, Port: `6443`}') }}"
Example:
- name: test
hosts: localhost
connection: local
vars:
instance_ids:
- i-1111
- i-2222
- i-3333
tasks:
- debug:
msg: "{{ instance_ids | json_query('[*].{Id: #, Port: `6443`}') }}"
Output:
TASK [debug]
ok: [localhost] => {
"msg": [
{
"Id": "i-1111",
"Port": 6443
},
{
"Id": "i-2222",
"Port": 6443
},
{
"Id": "i-3333",
"Port": 6443
}
]
}

Ansible iterate over array and use filter

I'm kinda new to ansible seeking help with below scenario. what I'm trying to do is iterate over the array 'access_key_ids' and run regex_search filter. for the regex_search filter argument is 'item' which is the variable from with_items. It does not work this way below is what I'm trying.
name: Set Fatcs
block:
# extract access key ids from get event response
- set_fact:
event_response_access_key_ids: "{{event_response_access_key_ids}} + [{{event_response.content | regex_search(item)}}]"
with_items: "{{access_key_ids}}"
# check if the response contains access key id for the license
- set_fact:
scwx_output: "{{ (event_response_access_key_ids | length > 0 ) | ternary(event_response, 'License Key does not match with available sensors')}}"
when: event_response.json is undefined
It gives event_response_access_key_ids as empty. but when I hard code a value instead of 'item' it works
Thanks.
I've been testing this solution:
---
- name: Test
hosts: local
gather_facts: False
vars:
event_response:
content: "hi1"
access_key_ids:
- "1"
- "h"
- "3"
tasks:
- name: Fact
set_fact:
event_response_access_key_ids: "{{ event_response_access_key_ids|default([]) + [ event_response.content | regex_search( item ) ] }}"
with_items: "{{ access_key_ids }}"
And It gets the vars properly:
ok: [localhost] => (item=1) => {
"ansible_facts": {
"event_response_access_key_ids": [
"1"
]
},
"changed": false,
"item": "1"
}
ok: [localhost] => (item=h) => {
"ansible_facts": {
"event_response_access_key_ids": [
"1",
"h"
]
},
"changed": false,
"item": "h"
}
ok: [localhost] => (item=3) => {
"ansible_facts": {
"event_response_access_key_ids": [
"1",
"h",
null
]
},
"changed": false,
"item": "3"
}

Ansible Dynamic Variables in Playbook

I'm provisioning a bunch of systems (VMs) on a physical host. I'm up to the step where the VM's are running. Now I need to ssh into the VM's via their DHCP addresses. I can pull the IP addresses from the server but I need a way to set these to host_vars. Here are my groups:
ok: [kvm01] => {
"msg": {
"all": [
"kvm01",
"dcos-master-1",
"dcos-agent-1",
"dcos-agent-2",
"dcos_bootstrap"
],
"dcos": [
"dcos-master-1",
"dcos-agent-1",
"dcos-agent-2",
"dcos_bootstrap"
],
"dcos-agents": [
"dcos-agent-1",
"dcos-agent-2"
],
"dcos-bootstraps": [
"dcos_bootstrap"
],
"dcos-masters": [
"dcos-master-1"
],
"kvm": [
"kvm01"
],
"ungrouped": []
}
}
Here's my command:
- name: Get the IP of the VM (DHCP)
command: "/getip.sh {{ item }}"
register: "result"
with_items: "{{ groups['dcos'] }}"
- name: List the output of the variables
debug: msg="{{item.stdout}}"
with_items: "{{result.results}}"
When I run the playbook I get results but they are the FULL JSON result of the command rather than the stdout. There's probably a way to pull out the stdout and assign it to a fact but it's a complex hash. Here's the last result:
TASK [vm_install : Get the IP of the VM (DHCP)] ***************************************************************************
changed: [kvm01] => (item=dcos-master-1)
changed: [kvm01] => (item=dcos-agent-1)
changed: [kvm01] => (item=dcos-agent-2)
changed: [kvm01] => (item=dcos_bootstrap)
TASK [vm_install : List the output of the variables] **********************************************************************
......
ok: [kvm01] => (item={'_ansible_parsed': True, 'stderr_lines': [u'] => {
"item": {
"changed": false,
"cmd": [
"/getip.sh",
"dcos_bootstrap"
],
"delta": "0:00:00.056193",
"end": "2017-09-18 15:45:45.409693",
"invocation": {
"module_args": {
"_raw_params": "/getip.sh dcos_bootstrap",
"_uses_shell": false,
"chdir": null,
"creates": null,
"executable": null,
"removes": null,
"warn": true
}
},
"item": "dcos_bootstrap",
"rc": 0,
"start": "2017-09-18 15:45:45.353500",
"stderr": " ",
"stdout": "192.168.1.130",
"stdout_lines": [
"192.168.1.130"
]
},
"msg": "192.168.1.130"
}
How can I put the output of the command into an array so that I can use it later in my playbook?
So, like I said in my comment, you've already managed to extract the information you want into an array. You can iterate over those items using with_items as in the follow task that will create an ip_address for each host:
- set_fact:
ip_address: "{{ item.stdout }}"
with_items: "{{ results.results }}"
delegate_to: "{{ item.item }}"
delegate_facts: true
Or you can create a single array containing all of the addresses using Jinja filters:
- set_fact:
all_ip_addresses: "{{ results.results|map(attribute='stdout')|list }}"
Or you could create a dictionary with the same information:
- set_fact:
all_ip_addresses: >
{{ all_ip_addresses
|default({})
|combine({ item.item: item.stdout })}}
with_items: "{{ results.results }}"

Ansible shows error: "One or more undefined variables: 'item' is undefined" when using 'with_items'

I am trying to count the instances inside an elb. This is my Ansible playbook:
- name: Get elb facts
local_action:
module: ec2_elb_facts
name: "{{elb}}"
region: "{{ansible_ec2_placement_region}}"
environment: creds
register: elb_facts
- debug:
var: elb_facts
verbosity: 2
- debug:
msg: "Instance: {{ item.instances }}"
with_items: "{{ elb_facts.elbs }}"
and my output (sensitive data removed):
TASK: [debug ] ****************************************************************
ok: [10.0.0.0] => {
"elb_facts": {
"changed": false,
"elbs": [
{
"availability_zones": [
"ap-southeast-2b",
"ap-southeast-2a"
],
"dns_name": "elbname123.ap-southeast-2.elb.amazonaws.com",
"health_check": {
"healthy_threshold": 2,
"interval": 10,
"target": "TCP:0000",
"timeout": 5,
"unhealthy_threshold": 2
},
"instances": [
{
"id": "i-000000000000000",
"state": null
}
],
"name": "accessgateway",
"scheme": "internal",
"security_groups": [
"sg-00000000"
],
"subnet": [
"subnet-0000000",
"subnet-1111111"
],
"vpc_id": "vpc-000000"
}
],
"invocation": {
"module_args": "",
"module_name": "ec2_elb_facts"
}
}
}
TASK: [debug ] ****************************************************************
fatal: [10.0.0.0] => One or more undefined variables: 'item' is undefined
FATAL: all hosts have already failed -- aborting
So what im trying to do is just loop through and print everything inside the elb_facts, instances variable. From what I can tell it's a hash, containing a list of hashes.
I am using http://docs.ansible.com/ansible/playbooks_loops.html#looping-over-subelements as a reference. I cannot for the life of mine figure out why this is not working.
with_items (and the whole family of with_ loops) is a dictionary key defined in a task, not as a parameter to the action.
Fix the indentation:
- debug:
msg: "Instance: {{ item.instances }}"
with_items: "{{ elb_facts.elbs }}"

Resources