I am trying to deregister EC2 instances from target groups using Automation document in SSM, which I am attempting to write in YAML but I am having major issues with getting my head around YAML lists and arrays.
Here are the relevant parts of the code:
parameters:
DeregisterInstanceId:
type: StringList
description: (Required) Identifies EC2 instances for patching
default: ["i-xxx","i-yyy"]
Further down I am trying to read this DeregisterInstanceId as a list but it's not working - getting various errors regarding expected one type of variable but received another.
name: RemoveLiveInstancesFromTG
action: aws:executeAwsApi
inputs:
Service: elbv2
Api: DeregisterTargets
TargetGroupArn: "{{ TargetGroup }}"
Targets: "{{ DeregisterInstanceId }}"
isEnd: true
What Targets input really needs to look like, is like this:
Targets:
- Id: "i-xxx"
- Id: "i-yyy"
...but I am not sure how to pass my StringList to create the above.
I tried:
Targets:
- Id: "{{ DeregisterInstanceId }}"
and
Targets:
Id: "{{ DeregisterInstanceId }}"
But no go.
I used to have the exact same problem although I created the document in json.
Please checkout the following working script to de-register an instance from a load balancer target group
Automation document v. 74
{
"description": "LoadBalancer deregister targets",
"schemaVersion": "0.3",
"assumeRole": "{{ AutomationAssumeRole }}",
"parameters": {
"TargetGroupArn": {
"type": "String",
"description": "(Required) TargetGroup of LoadBalancer"
},
"Target": {
"type": "String",
"description": "(Required) EC2 Instance(s) to deregister"
},
"AutomationAssumeRole": {
"type": "String",
"description": "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf.",
"default": ""
}
},
"mainSteps": [
{
"name": "DeregisterTarget",
"action": "aws:executeAwsApi",
"inputs": {
"Service": "elbv2",
"Api": "DeregisterTargets",
"TargetGroupArn": "{{ TargetGroupArn }}",
"Targets": [
{
"Id": "{{ Target }}"
}
]
}
}
]
}
Obviously the point of interest is the targets parameter, it needs an json array to work (forget about the cli format, it seems to need json).
It also allows for specifying multiple targets and also allows usage of ports and availability groups, but all I need it for is to choose one instance and pull it out.
Hope it might be of use for someone.
Related
I have a route table in AWS where there is a subnet getting routed to one host for each host. I can setup those routes automatically using this code:
- name: Add route to host container network
ec2_vpc_route_table:
region: region
vpc_id: "vpc-somestring"
purge_subnets: false
purge_routes: false
lookup: id
route_table_id: rtb-somestring
routes:
- dest: "1.2.3.0/24"
instance_id: "i-somestring"
This is fine for creating new hosts automatically. But if I want to remove a host, I want to delete the matching route table entry.
I thought, I could just fetch the route table using ec2_vpc_route_table_info, then take the routes filtered with rejectattr and feed it back to ec2_vpc_route_table, replacing the whole table. But, info gives me this format of routing tables:
"all_routes": [
{
"destination_cidr_block": "1.2.3.0/24",
"gateway_id": null,
"instance_id": "i-somestring",
"instance_owner_id": "1234567890",
"interface_id": "eni-somestring",
"network_interface_id": "eni-somestring",
"origin": "CreateRoute",
"state": "active"
},
{
"destination_cidr_block": "5.5.5.0/21",
"gateway_id": "local",
"instance_id": null,
"interface_id": null,
"network_interface_id": null,
"origin": "CreateRouteTable",
"state": "active"
},
{
"destination_cidr_block": null,
"destination_ipv6_cidr_block": "affe:affe:affe:affe::/56",
"gateway_id": "local",
"instance_id": null,
"interface_id": null,
"network_interface_id": null,
"origin": "CreateRouteTable",
"state": "active"
}
]
However, I can't feed that table to ec2_vpc_route_table, because that module just wants a list looking like this:
[
{
"dest": "1.2.3.0/24",
"instance_id": "i-somestring"
},
{
"dest": "5.5.5.0/21",
"gateway_id": "local
},
{
"dest": "affe:affe:affe:affe::/56",
"gateway_id": "local"
}
]
Why is the output of the info module not in a format that I can feed back to the route_table module? How can I convert the output into a format that I can feed back to the route_table module?
Thanks for any input.
a sample of solution:
- hosts: localhost
gather_facts: false
vars:
all_routes: "{{ lookup('file', 'zson.json') | from_json }}"
tasks:
- name: display json
debug:
var: all_routes
- name: create new json
set_fact:
result: "{{ result | d([]) + [{ 'dest': _block, _key: _gateway }] }}"
vars:
_block: "{{ item.destination_cidr_block if item.destination_cidr_block != None else item.destination_ipv6_cidr_block }}"
_gateway: "{{ item.gateway_id if item.gateway_id != None else item.instance_id }}"
_key: "{{ 'gateway_id' if item.gateway_id != None else 'instance_id' }}"
loop: "{{all_routes }}"
- name: display result
debug:
var: result
result:
ok: [localhost] => {
"result": [
{
"dest": "1.2.3.0/24",
"instance_id": "i-somestring"
},
{
"dest": "5.5.5.0/21",
"gateway_id": "local"
},
{
"dest": "affe:affe:affe:affe::/56",
"gateway_id": "local"
}
]
}
I have a playbook pulling users and groups from windows 2012 using powershell and I convert it to json.
I get the following output
[
{
"Group": "Appsupp",
"Members": [
"WIN-U97DIQUENUM\\appsuport",
"WIN-U97DIQUENUM\\userfirst"
]
},
{
"Group": "DBAdministrators",
"Members": {}
},
{
"Group": "Techsupp",
"Members": {}
},
{
"Group": "Access Control Assistance Operators",
"Members": {}
},
{
"Group": "Administrators",
"Members": "WIN-U97DIQUENUM\\Administrator"
},
{
"Group": "Backup Operators",
"Members": {}
},
{
"Group": "Certificate Service DCOM Access",
"Members": {}
},
{
"Group": "Cryptographic Operators",
"Members": {}
},
{
"Group": "Distributed COM Users",
"Members": {}
},
{
"Group": "Event Log Readers",
"Members": {}
},
{
"Group": "Guests",
"Members": "WIN-U97DIQUENUM\\Guest"
},
{
"Group": "Hyper-V Administrators",
"Members": {}
},
{
"Group": "IIS_IUSRS",
"Members": "NT AUTHORITY\\IUSR"
},
{
"Group": "Network Configuration Operators",
"Members": {}
},
{
"Group": "Performance Log Users",
"Members": {}
},
{
"Group": "Performance Monitor Users",
"Members": {}
},
{
"Group": "Power Users",
"Members": {}
},
{
"Group": "Print Operators",
"Members": {}
},
{
"Group": "RDS Endpoint Servers",
"Members": {}
},
{
"Group": "RDS Management Servers",
"Members": {}
},
{
"Group": "RDS Remote Access Servers",
"Members": {}
},
{
"Group": "Remote Desktop Users",
"Members": {}
},
{
"Group": "Remote Management Users",
"Members": {}
},
{
"Group": "Replicator",
"Members": {}
},
{
"Group": "Users",
"Members": [
"NT AUTHORITY\\Authenticated Users",
"NT AUTHORITY\\INTERACTIVE",
"WIN-U97DIQUENUM\\userfirst",
"WIN-U97DIQUENUM\\usersecond",
"WIN-U97DIQUENUM\\appsupport",
"WIN-U97DIQUENUM\\techsupport",
"WIN-U97DIQUENUM\\sqlserveruser"
]
}
]
I would like to have user compared to that data I pulled from the server. For example, I want to know if appsupport user exists and which group it belongs to.
I have tried the below json_query. FYI, getgroup.stdout is the registered result from the task pulling info from the server.
- debug:
msg: "{{ getgroup.stdout | from_json | json_query('guser') }}"
vars:
guser: "[?Members.contains(#, `appsupport`)].Group"
What I get is an empty result (while this method worked with other json). What am I doing wrong?
I assume that the problem is the Members key in my object as it my contain an empty dictionnary, a single string or a list of strings as recapped in the chosen examples below.
Single string:
{
"Group": "IIS_IUSRS",
"Members": "NT AUTHORITY\\IUSR"
},
List:
{
"Group": "Users",
"Members": [
"NT AUTHORITY\\Authenticated Users",
"NT AUTHORITY\\INTERACTIVE",
"WIN-U97DIQUENUM\\userfirst",
"WIN-U97DIQUENUM\\usersecond",
"WIN-U97DIQUENUM\\appsupport",
"WIN-U97DIQUENUM\\techsupport",
"WIN-U97DIQUENUM\\sqlserveruser"
]
}
Empty dict:
{
"Group": "Replicator",
"Members": {}
}
When I use json_query('[].Members[*]') it only returns the latest member, not all Members.
How can I select the objects from my json result having a particular user in the Members field in this kind of situation?
Without json_query.
For testing, given the output in the file, read it and convert the Members to lists, e.g.
- set_fact:
output: "{{ lookup('file', 'test.data')|from_yaml }}"
- set_fact:
mlist: "{{ mlist|d([]) +
[item|combine({'Members': _Members|from_yaml})] }}"
loop: "{{ output }}"
vars:
_Members: |-
{% if item.Members is mapping %}
{{ item.Members.keys()|list }}
{% elif item.Members is string %}
[{{ item.Members }}]
{% else %}
{{ item.Members }}
{% endif %}
- debug:
var: mlist
gives
mlist:
- Group: Appsupp
Members:
- WIN-U97DIQUENUM\appsuport
- WIN-U97DIQUENUM\userfirst
- Group: DBAdministrators
Members: []
- Group: Techsupp
Members: []
- Group: Access Control Assistance Operators
Members: []
- Group: Administrators
Members:
- WIN-U97DIQUENUM\Administrator
- Group: Backup Operators
Members: []
- Group: Certificate Service DCOM Access
Members: []
- Group: Cryptographic Operators
Members: []
- Group: Distributed COM Users
Members: []
- Group: Event Log Readers
Members: []
- Group: Guests
Members:
- WIN-U97DIQUENUM\Guest
- Group: Hyper-V Administrators
Members: []
- Group: IIS_IUSRS
Members:
- NT AUTHORITY\IUSR
- Group: Network Configuration Operators
Members: []
- Group: Performance Log Users
Members: []
- Group: Performance Monitor Users
Members: []
- Group: Power Users
Members: []
- Group: Print Operators
Members: []
- Group: RDS Endpoint Servers
Members: []
- Group: RDS Management Servers
Members: []
- Group: RDS Remote Access Servers
Members: []
- Group: Remote Desktop Users
Members: []
- Group: Remote Management Users
Members: []
- Group: Replicator
Members: []
- Group: Users
Members:
- NT AUTHORITY\Authenticated Users
- NT AUTHORITY\INTERACTIVE
- WIN-U97DIQUENUM\userfirst
- WIN-U97DIQUENUM\usersecond
- WIN-U97DIQUENUM\appsupport
- WIN-U97DIQUENUM\techsupport
- WIN-U97DIQUENUM\sqlserveruser
Now, you can search for the members, e.g.
- debug:
msg: "{{ mlist|selectattr('Members', 'contains', my_user) }}"
vars:
my_user: 'WIN-U97DIQUENUM\Administrator'
gives
msg:
- Group: Administrators
Members:
- WIN-U97DIQUENUM\Administrator
Next, create a dictionary of all members and the groups they belong to
- set_fact:
members: "{{ members|d({})|combine({item: _groups}) }}"
loop: "{{ mlist|map(attribute='Members')|flatten|unique }}"
vars:
_groups: "{{ mlist|selectattr('Members', 'contains', item)|
map(attribute='Group')|
list }}"
- debug:
var: members
gives
members:
NT AUTHORITY\Authenticated Users:
- Users
NT AUTHORITY\INTERACTIVE:
- Users
NT AUTHORITY\IUSR:
- IIS_IUSRS
WIN-U97DIQUENUM\Administrator:
- Administrators
WIN-U97DIQUENUM\Guest:
- Guests
WIN-U97DIQUENUM\appsuport:
- Appsupp
WIN-U97DIQUENUM\appsupport:
- Users
WIN-U97DIQUENUM\sqlserveruser:
- Users
WIN-U97DIQUENUM\techsupport:
- Users
WIN-U97DIQUENUM\userfirst:
- Appsupp
- Users
WIN-U97DIQUENUM\usersecond:
- Users
Then, the searching is trivial, e.g.
- debug:
msg: "{{ my_user }} is member of the group(s): {{ members[my_user] }}"
vars:
my_user: 'WIN-U97DIQUENUM\Administrator'
gives
msg: 'WIN-U97DIQUENUM\Administrator is member of the group(s): [''Administrators'']'
It's possible to search among the members, e.g.
- debug:
msg: "{{ my_user }} is member of the group(s): {{ _groups }}"
vars:
my_user: appsuport
_keys: "{{ members.keys()|list|select('search', my_user)|list }}"
_groups: "{{ _keys|map('extract', members)|flatten }}"
gives
msg: 'appsuport is member of the group(s): [''Appsupp'']'
I see multiple problems here:
Problem 1:
Your data has three different types for Members, depending on what is in there:
An empty dict ({}) if there are no members
A string, if there is exactly one member
A list of strings, if there are multiple members
As far as I can see, this will not break, but it is not particularly good style. You should always return a list. An empty one, if there is no member, or a list with one element if there is exactly one member or a list of multiple strings, as you already do, if there are multiple members.
Problem 2:
In group Appsupp, the user is misspelled: WIN-U97DIQUENUM\\appsuport (there is a 'p' missing in 'support'), so this will not match.
Problem 3:
This is what actually breaks here. Your query has several problems, it should look like this: [?Members[?contains(#, 'appsupport')]].Group
The result is
[
"Users"
]
Appsupp is not in there due to the spelling mistake mentioned earlier.
You can test your json-path on this website.
I'm getting below wired error when trying to add disk to RHVM. I have checked in the documentation. all parameters seems legit to me. I need extra eye to valiate this case. thank you in-advanced
error msg as follows
"msg": "Unsupported parameters for (ovirt_disk) module: activate Supported parameters include: auth, bootable, description, download_image_path, fetch_nested, force, format, id, image_provider, interface, logical_unit, name, nested_attributes, openstack_volume_type, poll_interval, profile, quota_id, shareable, size, sparse, sparsify, state, storage_domain, storage_domains, timeout, upload_image_path, vm_id, vm_name, wait"
}
Parameters in the role as follows
"module_args": {
"vm_name": "Jxyxyxyxy01",
"activate": true,
"storage_domain": "Data-xxx-Txxx",
"description": "Created using Jira ticket CR-329",
"format": "cow",
"auth": {
"timeout": 0,
"url": "https://xxxxxx.com/ovirt-engine/api",
"insecure": true,
"kerberos": false,
"compress": true,
"headers": null,
"token": "xxcddsvsdvdsvsdvdEFl0910KES84qL8Ff5NReA",
"ca_file": null
},
"state": "present",
"sparse": true,
"interface": "virtio_scsi",
"wait": true,
"size": "20GiB",
"name": "Jxyxyxyxy01_123"
}
Playbook as follows.
- name: Create New Disk size of {{ disk_size }} on {{ hostname }} using storage domain {{ vm_storage_domain }}
ovirt_disk:
auth: "{{ ovirt_auth }}"
description: "Created using Jira ticket {{ issueKey }}"
storage_domain: "{{ vm_storage_domain }}"
name: "{{ hostname }}_123" # name of the disk
vm_name: "{{ hostname }}" #name of the virtual machine
interface: "virtio_scsi"
size: "{{ disk_size }}GiB"
sparse: yes
format: cow
activate: yes
wait: yes
state: present
register: vm_disk_results
The activate parameter was added in ansible 2.8.
Upgrade your ansible installation or drop that parameter.
Source data is JSON dictionary where top-level key names are not predictable. I need the name of that TL key when nested data inside matches a string I have.
I am attempting this using a the json_query filter inside Ansible 2.9.
I have tried various JMESPath queries and searched through their documentation, but everything I have found appears to require knowing the names of those keys ahead of time.
Here is sample data - which is output from ansible ios_facts module against interfaces:
{
"GigabitEthernet0/9": {
"description": "server",
"ipv4": [],
"macaddress": "b8zz.xxxx.7709",
"type": "Gigabit Ethernet"
},
"Vlan13": {
"description": null,
"ipv4": [
{
"address": "10.20.30.19",
"subnet": "24"
}
],
"macaddress": "b8zz.xxx.yy41",
"type": "EtherSVI"
}
}
Here is playbook I am working with:
---
- name: Cisco IOS - Mgmt Int
connection: network_cli
gather_facts: false
hosts: switch01
vars:
netdb: "{{ hostvars[inventory_hostname]['ansible_net_interfaces'] }}"
tasks:
- name: get ios facts
ios_facts:
gather_subset: interfaces
- name: json_query
set_fact:
mgmtinf: "{{ netdb | json_query(query) }}"
vars:
# this gets me true/false
query: "[Vlan13.ipv4[0].address.contains(#, '10.20.30.19')]"
- name: output json_query
debug:
var: mgmtinf
The above works to return true/false if the result matches. The problem is that I have manually defined the interface to look at 'Vlan13'. So I need a way to use an iterator for that TL key, and then return value of that iterator at the part in the loop where match occurs.
You didn't give all your requirements to select the matching key name but the following should put you on track.
The key points:
Use the dict2tems filter to transform the top level dict to a list of {key: X, value: Y} objects
Use jmespath to:
Filter out the elements corresponding to vlans (I used "Vlan*" in my example)
Filter out the list of ips for each ipv4 elements to match only your desired IP.
Note: The to_json | from_json in my expression is necessary to overcome a current bug in the communication between ansible and jmespath (string types mapping).
Here is a sample task illustrating the above scenario.
- name: Filter out specific ip elements on my vlans
vars:
vals: {
"GigabitEthernet0/9": {
"description": "server",
"ipv4": [],
"macaddress": "b8zz.xxxx.7709",
"type": "Gigabit Ethernet"
},
"Vlan13": {
"description": null,
"ipv4": [
{
"address": "10.20.30.19",
"subnet": "24"
}
],
"macaddress": "b8zz.xxx.yy41",
"type": "EtherSVI"
}
}
query: >-
[?starts_with(key, 'Vlan')].value.ipv4[] | [?contains(address, '10.20.30.19')]
debug:
msg: >-
{{ vals | dict2items | to_json | from_json | json_query(query) }}
And the result I get with your current values:
TASK [Filter out specific ip elements on my vlans] *************************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"address": "10.20.30.19",
"subnet": "24"
}
]
}
I've been doing ansible for a while, but have recently started doing some more advanced things such as pulling data to drive actions from outside sources. This has resulted in me having to dig a little deeper into how ansible allows logic and parsing of variables, requiring me to dig a little into jinja2.
In my playbook I am attempting to pull data in from etcd, allowing me to construct authorized sudo spec files, which I then pass onto a role to add to the appropriate systems.
My datasource, in addition to storing data that is needed to construct the specs has metadata used for auditing and logging purposes.
A key aspect of my datasource is that no metadata, i.e. that user XYZ had password less sudo for a period of 10 days, should be deleted when access is removed. So many aspects have a state field that may be active, inactive or in the case of sudo specs grant or revoke.
I have successfully constructed a lookup that pulls back a dictionary similar to below - which I then parse using the subsequent ansible statements. I am able to succcessfully process and extract all data except for groups/users that have their specs in a grant state.
When specs are in a "grant" state, I need to extract the linuxName field, to be passed onto the role that configures sudo.
I have tried a number of variations of filters, most of which end in me getting a reject or similar message, or a NULL value instead of the list of values that I want.
Anyone have an idea on how to accomplish this?
Thanks in advance.
Sample Data
ok: [serverName] => {
"sudoInfraSpecs": [
{
"infra_admins": {
"addedBy": "someUser",
"commands": "FULL_SUDO",
"comment": "platform support admins",
"dateAdded": "20180720",
"defaults": "!requiretty",
"hosts": "SERVERS",
"name": "infra_admins",
"operators": "ROOT",
"state": "active",
"tags": "PASSWD",
"users": {
"admingroup1": {
"addedBy": "someUser",
"dateAdded": "20180719",
"linuxName": "%admingroup1",
"name": "admingroup1",
"state": "grant"
},
"admingroup2": {
"addedBy": "someUser",
"dateAdded": "20180719",
"linuxName": "%admingroup2",
"name": "admingroup2",
"state": "grant"
}
}
},
"ucp_service_account": {
"addedBy": "someUser",
"commands": "FULL_SUDO",
"comment": "platform service account",
"dateAdded": "20180720",
"defaults": "!requiretty",
"hosts": "SERVERS",
"name": "platform_service_account",
"operators": "ROOT",
"state": "active",
"tags": "NOPASSWD,LOG_OUTPUT",
"users": {
"platformUser": {
"addedBy": "someUser",
"dateAdded": "20180719",
"linuxName": "platformUser",
"name": "platformUser",
"state": "grant"
}
}
}
}
]
}
Ansible snippet
- name: Translate infraAdmins sudoers specs from etcd into a list for processing [1]
set_fact:
tempInfraSpecs:
name: "{{ item.value.name}}"
comment: "{{ item.value.comment }}"
users: "{{ item.value.users | list }}"
hosts: "{{ item.value.hosts.split(',') }}"
operators: "{{ item.value.operators.split(',') }}"
tags: "{{ item.value.tags.split(',') }}"
commands: "{{ item.value.commands.split(',') }}"
defaults: "{{ item.value.defaults.split(',') }}"
with_dict: "{{ sudoInfraSpecs }}"
when: item.value.state == 'active'
register: tempsudoInfraSpecs
- name: Translate infraAdmins sudoers specs from etcd into a list for processing [2]
set_fact:
sudoInfraSpecs_fact: "{{ tempsudoInfraSpecs.results | selectattr('ansible_facts','defined')| map(attribute='ansible_facts.tempInfraSpecs') | list }}"
Rough Desired output dictionary:
sudoInfraSpecs:
- infra_admins:
addedBy: someUser
commands: FULL_SUDO
comment: platform support admins
dateAdded: '20180720'
defaults: "!requiretty"
hosts: SERVERS
name: infra_admins
operators: ROOT
state: active
tags: PASSWD
users:
"%admingroup1"
"%admingroup2"
- ucp_service_account:
addedBy: someUser
commands: FULL_SUDO
comment: platform service account
dateAdded: '20180720'
defaults: "!requiretty"
hosts: SERVERS
name: platform_service_account
operators: ROOT
state: active
tags: NOPASSWD,LOG_OUTPUT
users:
"platformUser"
I ended up doing this by creating a custom filter for use in my playbook that parsed the nested dictionary that makes up users:
#!/usr/bin/python
def getSpecActiveMembers(my_dict):
thisSpecActiveMembers = []
for i, value in my_dict.iteritems():
if value['state'] == 'grant':
thisSpecActiveMembers.append(value['linuxName'])
return thisSpecActiveMembers
class FilterModule(object):
def filters(self):
return {
'getSpecActiveMembers': getSpecActiveMembers
}
This ends up flattening the users from the source listed above, to the desired output.