I've been doing ansible for a while, but have recently started doing some more advanced things such as pulling data to drive actions from outside sources. This has resulted in me having to dig a little deeper into how ansible allows logic and parsing of variables, requiring me to dig a little into jinja2.
In my playbook I am attempting to pull data in from etcd, allowing me to construct authorized sudo spec files, which I then pass onto a role to add to the appropriate systems.
My datasource, in addition to storing data that is needed to construct the specs has metadata used for auditing and logging purposes.
A key aspect of my datasource is that no metadata, i.e. that user XYZ had password less sudo for a period of 10 days, should be deleted when access is removed. So many aspects have a state field that may be active, inactive or in the case of sudo specs grant or revoke.
I have successfully constructed a lookup that pulls back a dictionary similar to below - which I then parse using the subsequent ansible statements. I am able to succcessfully process and extract all data except for groups/users that have their specs in a grant state.
When specs are in a "grant" state, I need to extract the linuxName field, to be passed onto the role that configures sudo.
I have tried a number of variations of filters, most of which end in me getting a reject or similar message, or a NULL value instead of the list of values that I want.
Anyone have an idea on how to accomplish this?
Thanks in advance.
Sample Data
ok: [serverName] => {
"sudoInfraSpecs": [
{
"infra_admins": {
"addedBy": "someUser",
"commands": "FULL_SUDO",
"comment": "platform support admins",
"dateAdded": "20180720",
"defaults": "!requiretty",
"hosts": "SERVERS",
"name": "infra_admins",
"operators": "ROOT",
"state": "active",
"tags": "PASSWD",
"users": {
"admingroup1": {
"addedBy": "someUser",
"dateAdded": "20180719",
"linuxName": "%admingroup1",
"name": "admingroup1",
"state": "grant"
},
"admingroup2": {
"addedBy": "someUser",
"dateAdded": "20180719",
"linuxName": "%admingroup2",
"name": "admingroup2",
"state": "grant"
}
}
},
"ucp_service_account": {
"addedBy": "someUser",
"commands": "FULL_SUDO",
"comment": "platform service account",
"dateAdded": "20180720",
"defaults": "!requiretty",
"hosts": "SERVERS",
"name": "platform_service_account",
"operators": "ROOT",
"state": "active",
"tags": "NOPASSWD,LOG_OUTPUT",
"users": {
"platformUser": {
"addedBy": "someUser",
"dateAdded": "20180719",
"linuxName": "platformUser",
"name": "platformUser",
"state": "grant"
}
}
}
}
]
}
Ansible snippet
- name: Translate infraAdmins sudoers specs from etcd into a list for processing [1]
set_fact:
tempInfraSpecs:
name: "{{ item.value.name}}"
comment: "{{ item.value.comment }}"
users: "{{ item.value.users | list }}"
hosts: "{{ item.value.hosts.split(',') }}"
operators: "{{ item.value.operators.split(',') }}"
tags: "{{ item.value.tags.split(',') }}"
commands: "{{ item.value.commands.split(',') }}"
defaults: "{{ item.value.defaults.split(',') }}"
with_dict: "{{ sudoInfraSpecs }}"
when: item.value.state == 'active'
register: tempsudoInfraSpecs
- name: Translate infraAdmins sudoers specs from etcd into a list for processing [2]
set_fact:
sudoInfraSpecs_fact: "{{ tempsudoInfraSpecs.results | selectattr('ansible_facts','defined')| map(attribute='ansible_facts.tempInfraSpecs') | list }}"
Rough Desired output dictionary:
sudoInfraSpecs:
- infra_admins:
addedBy: someUser
commands: FULL_SUDO
comment: platform support admins
dateAdded: '20180720'
defaults: "!requiretty"
hosts: SERVERS
name: infra_admins
operators: ROOT
state: active
tags: PASSWD
users:
"%admingroup1"
"%admingroup2"
- ucp_service_account:
addedBy: someUser
commands: FULL_SUDO
comment: platform service account
dateAdded: '20180720'
defaults: "!requiretty"
hosts: SERVERS
name: platform_service_account
operators: ROOT
state: active
tags: NOPASSWD,LOG_OUTPUT
users:
"platformUser"
I ended up doing this by creating a custom filter for use in my playbook that parsed the nested dictionary that makes up users:
#!/usr/bin/python
def getSpecActiveMembers(my_dict):
thisSpecActiveMembers = []
for i, value in my_dict.iteritems():
if value['state'] == 'grant':
thisSpecActiveMembers.append(value['linuxName'])
return thisSpecActiveMembers
class FilterModule(object):
def filters(self):
return {
'getSpecActiveMembers': getSpecActiveMembers
}
This ends up flattening the users from the source listed above, to the desired output.
Related
I have a playbook pulling users and groups from windows 2012 using powershell and I convert it to json.
I get the following output
[
{
"Group": "Appsupp",
"Members": [
"WIN-U97DIQUENUM\\appsuport",
"WIN-U97DIQUENUM\\userfirst"
]
},
{
"Group": "DBAdministrators",
"Members": {}
},
{
"Group": "Techsupp",
"Members": {}
},
{
"Group": "Access Control Assistance Operators",
"Members": {}
},
{
"Group": "Administrators",
"Members": "WIN-U97DIQUENUM\\Administrator"
},
{
"Group": "Backup Operators",
"Members": {}
},
{
"Group": "Certificate Service DCOM Access",
"Members": {}
},
{
"Group": "Cryptographic Operators",
"Members": {}
},
{
"Group": "Distributed COM Users",
"Members": {}
},
{
"Group": "Event Log Readers",
"Members": {}
},
{
"Group": "Guests",
"Members": "WIN-U97DIQUENUM\\Guest"
},
{
"Group": "Hyper-V Administrators",
"Members": {}
},
{
"Group": "IIS_IUSRS",
"Members": "NT AUTHORITY\\IUSR"
},
{
"Group": "Network Configuration Operators",
"Members": {}
},
{
"Group": "Performance Log Users",
"Members": {}
},
{
"Group": "Performance Monitor Users",
"Members": {}
},
{
"Group": "Power Users",
"Members": {}
},
{
"Group": "Print Operators",
"Members": {}
},
{
"Group": "RDS Endpoint Servers",
"Members": {}
},
{
"Group": "RDS Management Servers",
"Members": {}
},
{
"Group": "RDS Remote Access Servers",
"Members": {}
},
{
"Group": "Remote Desktop Users",
"Members": {}
},
{
"Group": "Remote Management Users",
"Members": {}
},
{
"Group": "Replicator",
"Members": {}
},
{
"Group": "Users",
"Members": [
"NT AUTHORITY\\Authenticated Users",
"NT AUTHORITY\\INTERACTIVE",
"WIN-U97DIQUENUM\\userfirst",
"WIN-U97DIQUENUM\\usersecond",
"WIN-U97DIQUENUM\\appsupport",
"WIN-U97DIQUENUM\\techsupport",
"WIN-U97DIQUENUM\\sqlserveruser"
]
}
]
I would like to have user compared to that data I pulled from the server. For example, I want to know if appsupport user exists and which group it belongs to.
I have tried the below json_query. FYI, getgroup.stdout is the registered result from the task pulling info from the server.
- debug:
msg: "{{ getgroup.stdout | from_json | json_query('guser') }}"
vars:
guser: "[?Members.contains(#, `appsupport`)].Group"
What I get is an empty result (while this method worked with other json). What am I doing wrong?
I assume that the problem is the Members key in my object as it my contain an empty dictionnary, a single string or a list of strings as recapped in the chosen examples below.
Single string:
{
"Group": "IIS_IUSRS",
"Members": "NT AUTHORITY\\IUSR"
},
List:
{
"Group": "Users",
"Members": [
"NT AUTHORITY\\Authenticated Users",
"NT AUTHORITY\\INTERACTIVE",
"WIN-U97DIQUENUM\\userfirst",
"WIN-U97DIQUENUM\\usersecond",
"WIN-U97DIQUENUM\\appsupport",
"WIN-U97DIQUENUM\\techsupport",
"WIN-U97DIQUENUM\\sqlserveruser"
]
}
Empty dict:
{
"Group": "Replicator",
"Members": {}
}
When I use json_query('[].Members[*]') it only returns the latest member, not all Members.
How can I select the objects from my json result having a particular user in the Members field in this kind of situation?
Without json_query.
For testing, given the output in the file, read it and convert the Members to lists, e.g.
- set_fact:
output: "{{ lookup('file', 'test.data')|from_yaml }}"
- set_fact:
mlist: "{{ mlist|d([]) +
[item|combine({'Members': _Members|from_yaml})] }}"
loop: "{{ output }}"
vars:
_Members: |-
{% if item.Members is mapping %}
{{ item.Members.keys()|list }}
{% elif item.Members is string %}
[{{ item.Members }}]
{% else %}
{{ item.Members }}
{% endif %}
- debug:
var: mlist
gives
mlist:
- Group: Appsupp
Members:
- WIN-U97DIQUENUM\appsuport
- WIN-U97DIQUENUM\userfirst
- Group: DBAdministrators
Members: []
- Group: Techsupp
Members: []
- Group: Access Control Assistance Operators
Members: []
- Group: Administrators
Members:
- WIN-U97DIQUENUM\Administrator
- Group: Backup Operators
Members: []
- Group: Certificate Service DCOM Access
Members: []
- Group: Cryptographic Operators
Members: []
- Group: Distributed COM Users
Members: []
- Group: Event Log Readers
Members: []
- Group: Guests
Members:
- WIN-U97DIQUENUM\Guest
- Group: Hyper-V Administrators
Members: []
- Group: IIS_IUSRS
Members:
- NT AUTHORITY\IUSR
- Group: Network Configuration Operators
Members: []
- Group: Performance Log Users
Members: []
- Group: Performance Monitor Users
Members: []
- Group: Power Users
Members: []
- Group: Print Operators
Members: []
- Group: RDS Endpoint Servers
Members: []
- Group: RDS Management Servers
Members: []
- Group: RDS Remote Access Servers
Members: []
- Group: Remote Desktop Users
Members: []
- Group: Remote Management Users
Members: []
- Group: Replicator
Members: []
- Group: Users
Members:
- NT AUTHORITY\Authenticated Users
- NT AUTHORITY\INTERACTIVE
- WIN-U97DIQUENUM\userfirst
- WIN-U97DIQUENUM\usersecond
- WIN-U97DIQUENUM\appsupport
- WIN-U97DIQUENUM\techsupport
- WIN-U97DIQUENUM\sqlserveruser
Now, you can search for the members, e.g.
- debug:
msg: "{{ mlist|selectattr('Members', 'contains', my_user) }}"
vars:
my_user: 'WIN-U97DIQUENUM\Administrator'
gives
msg:
- Group: Administrators
Members:
- WIN-U97DIQUENUM\Administrator
Next, create a dictionary of all members and the groups they belong to
- set_fact:
members: "{{ members|d({})|combine({item: _groups}) }}"
loop: "{{ mlist|map(attribute='Members')|flatten|unique }}"
vars:
_groups: "{{ mlist|selectattr('Members', 'contains', item)|
map(attribute='Group')|
list }}"
- debug:
var: members
gives
members:
NT AUTHORITY\Authenticated Users:
- Users
NT AUTHORITY\INTERACTIVE:
- Users
NT AUTHORITY\IUSR:
- IIS_IUSRS
WIN-U97DIQUENUM\Administrator:
- Administrators
WIN-U97DIQUENUM\Guest:
- Guests
WIN-U97DIQUENUM\appsuport:
- Appsupp
WIN-U97DIQUENUM\appsupport:
- Users
WIN-U97DIQUENUM\sqlserveruser:
- Users
WIN-U97DIQUENUM\techsupport:
- Users
WIN-U97DIQUENUM\userfirst:
- Appsupp
- Users
WIN-U97DIQUENUM\usersecond:
- Users
Then, the searching is trivial, e.g.
- debug:
msg: "{{ my_user }} is member of the group(s): {{ members[my_user] }}"
vars:
my_user: 'WIN-U97DIQUENUM\Administrator'
gives
msg: 'WIN-U97DIQUENUM\Administrator is member of the group(s): [''Administrators'']'
It's possible to search among the members, e.g.
- debug:
msg: "{{ my_user }} is member of the group(s): {{ _groups }}"
vars:
my_user: appsuport
_keys: "{{ members.keys()|list|select('search', my_user)|list }}"
_groups: "{{ _keys|map('extract', members)|flatten }}"
gives
msg: 'appsuport is member of the group(s): [''Appsupp'']'
I see multiple problems here:
Problem 1:
Your data has three different types for Members, depending on what is in there:
An empty dict ({}) if there are no members
A string, if there is exactly one member
A list of strings, if there are multiple members
As far as I can see, this will not break, but it is not particularly good style. You should always return a list. An empty one, if there is no member, or a list with one element if there is exactly one member or a list of multiple strings, as you already do, if there are multiple members.
Problem 2:
In group Appsupp, the user is misspelled: WIN-U97DIQUENUM\\appsuport (there is a 'p' missing in 'support'), so this will not match.
Problem 3:
This is what actually breaks here. Your query has several problems, it should look like this: [?Members[?contains(#, 'appsupport')]].Group
The result is
[
"Users"
]
Appsupp is not in there due to the spelling mistake mentioned earlier.
You can test your json-path on this website.
I'm getting below wired error when trying to add disk to RHVM. I have checked in the documentation. all parameters seems legit to me. I need extra eye to valiate this case. thank you in-advanced
error msg as follows
"msg": "Unsupported parameters for (ovirt_disk) module: activate Supported parameters include: auth, bootable, description, download_image_path, fetch_nested, force, format, id, image_provider, interface, logical_unit, name, nested_attributes, openstack_volume_type, poll_interval, profile, quota_id, shareable, size, sparse, sparsify, state, storage_domain, storage_domains, timeout, upload_image_path, vm_id, vm_name, wait"
}
Parameters in the role as follows
"module_args": {
"vm_name": "Jxyxyxyxy01",
"activate": true,
"storage_domain": "Data-xxx-Txxx",
"description": "Created using Jira ticket CR-329",
"format": "cow",
"auth": {
"timeout": 0,
"url": "https://xxxxxx.com/ovirt-engine/api",
"insecure": true,
"kerberos": false,
"compress": true,
"headers": null,
"token": "xxcddsvsdvdsvsdvdEFl0910KES84qL8Ff5NReA",
"ca_file": null
},
"state": "present",
"sparse": true,
"interface": "virtio_scsi",
"wait": true,
"size": "20GiB",
"name": "Jxyxyxyxy01_123"
}
Playbook as follows.
- name: Create New Disk size of {{ disk_size }} on {{ hostname }} using storage domain {{ vm_storage_domain }}
ovirt_disk:
auth: "{{ ovirt_auth }}"
description: "Created using Jira ticket {{ issueKey }}"
storage_domain: "{{ vm_storage_domain }}"
name: "{{ hostname }}_123" # name of the disk
vm_name: "{{ hostname }}" #name of the virtual machine
interface: "virtio_scsi"
size: "{{ disk_size }}GiB"
sparse: yes
format: cow
activate: yes
wait: yes
state: present
register: vm_disk_results
The activate parameter was added in ansible 2.8.
Upgrade your ansible installation or drop that parameter.
Source data is JSON dictionary where top-level key names are not predictable. I need the name of that TL key when nested data inside matches a string I have.
I am attempting this using a the json_query filter inside Ansible 2.9.
I have tried various JMESPath queries and searched through their documentation, but everything I have found appears to require knowing the names of those keys ahead of time.
Here is sample data - which is output from ansible ios_facts module against interfaces:
{
"GigabitEthernet0/9": {
"description": "server",
"ipv4": [],
"macaddress": "b8zz.xxxx.7709",
"type": "Gigabit Ethernet"
},
"Vlan13": {
"description": null,
"ipv4": [
{
"address": "10.20.30.19",
"subnet": "24"
}
],
"macaddress": "b8zz.xxx.yy41",
"type": "EtherSVI"
}
}
Here is playbook I am working with:
---
- name: Cisco IOS - Mgmt Int
connection: network_cli
gather_facts: false
hosts: switch01
vars:
netdb: "{{ hostvars[inventory_hostname]['ansible_net_interfaces'] }}"
tasks:
- name: get ios facts
ios_facts:
gather_subset: interfaces
- name: json_query
set_fact:
mgmtinf: "{{ netdb | json_query(query) }}"
vars:
# this gets me true/false
query: "[Vlan13.ipv4[0].address.contains(#, '10.20.30.19')]"
- name: output json_query
debug:
var: mgmtinf
The above works to return true/false if the result matches. The problem is that I have manually defined the interface to look at 'Vlan13'. So I need a way to use an iterator for that TL key, and then return value of that iterator at the part in the loop where match occurs.
You didn't give all your requirements to select the matching key name but the following should put you on track.
The key points:
Use the dict2tems filter to transform the top level dict to a list of {key: X, value: Y} objects
Use jmespath to:
Filter out the elements corresponding to vlans (I used "Vlan*" in my example)
Filter out the list of ips for each ipv4 elements to match only your desired IP.
Note: The to_json | from_json in my expression is necessary to overcome a current bug in the communication between ansible and jmespath (string types mapping).
Here is a sample task illustrating the above scenario.
- name: Filter out specific ip elements on my vlans
vars:
vals: {
"GigabitEthernet0/9": {
"description": "server",
"ipv4": [],
"macaddress": "b8zz.xxxx.7709",
"type": "Gigabit Ethernet"
},
"Vlan13": {
"description": null,
"ipv4": [
{
"address": "10.20.30.19",
"subnet": "24"
}
],
"macaddress": "b8zz.xxx.yy41",
"type": "EtherSVI"
}
}
query: >-
[?starts_with(key, 'Vlan')].value.ipv4[] | [?contains(address, '10.20.30.19')]
debug:
msg: >-
{{ vals | dict2items | to_json | from_json | json_query(query) }}
And the result I get with your current values:
TASK [Filter out specific ip elements on my vlans] *************************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"address": "10.20.30.19",
"subnet": "24"
}
]
}
I am trying to deregister EC2 instances from target groups using Automation document in SSM, which I am attempting to write in YAML but I am having major issues with getting my head around YAML lists and arrays.
Here are the relevant parts of the code:
parameters:
DeregisterInstanceId:
type: StringList
description: (Required) Identifies EC2 instances for patching
default: ["i-xxx","i-yyy"]
Further down I am trying to read this DeregisterInstanceId as a list but it's not working - getting various errors regarding expected one type of variable but received another.
name: RemoveLiveInstancesFromTG
action: aws:executeAwsApi
inputs:
Service: elbv2
Api: DeregisterTargets
TargetGroupArn: "{{ TargetGroup }}"
Targets: "{{ DeregisterInstanceId }}"
isEnd: true
What Targets input really needs to look like, is like this:
Targets:
- Id: "i-xxx"
- Id: "i-yyy"
...but I am not sure how to pass my StringList to create the above.
I tried:
Targets:
- Id: "{{ DeregisterInstanceId }}"
and
Targets:
Id: "{{ DeregisterInstanceId }}"
But no go.
I used to have the exact same problem although I created the document in json.
Please checkout the following working script to de-register an instance from a load balancer target group
Automation document v. 74
{
"description": "LoadBalancer deregister targets",
"schemaVersion": "0.3",
"assumeRole": "{{ AutomationAssumeRole }}",
"parameters": {
"TargetGroupArn": {
"type": "String",
"description": "(Required) TargetGroup of LoadBalancer"
},
"Target": {
"type": "String",
"description": "(Required) EC2 Instance(s) to deregister"
},
"AutomationAssumeRole": {
"type": "String",
"description": "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf.",
"default": ""
}
},
"mainSteps": [
{
"name": "DeregisterTarget",
"action": "aws:executeAwsApi",
"inputs": {
"Service": "elbv2",
"Api": "DeregisterTargets",
"TargetGroupArn": "{{ TargetGroupArn }}",
"Targets": [
{
"Id": "{{ Target }}"
}
]
}
}
]
}
Obviously the point of interest is the targets parameter, it needs an json array to work (forget about the cli format, it seems to need json).
It also allows for specifying multiple targets and also allows usage of ports and availability groups, but all I need it for is to choose one instance and pull it out.
Hope it might be of use for someone.
I would like to be able to get list of volume groups available and then use the attributes of the volume group to conditionally create the filesystem on volume which meets the Criteria
Criteria
1: Volume must not have name root in it
2: Volume must be at-least 2 GB
I have this logic
- name: list volume groups
debug: msg="echo volume groups is {{ item }}"
with_items: "{{ ansible_lvm.vgs }}
But am struggling to get that attribute of the vgs.
Here is snippet of ansible_lvm
"ansible_lvm": {
"lvs": {
"lvud01": {
"size_g": "0.50",
"vg": "vgud01"
},
"root": {
"size_g": "79.01",
"vg": "precise32"
},
"swap_1": {
"size_g": "0.75",
"vg": "precise32"
}
},
"vgs": {
"precise32": {
"free_g": "0",
"num_lvs": "2",
"num_pvs": "1",
"size_g": "79.76"
},
"vgud01": {
"free_g": "0.50",
"num_lvs": "1",
"num_pvs": "1",
"size_g": "1.00"
}
}
Just for everyones benefit and thanks #fishi
- name: list volume groups
debug: msg="echo volume groups is {{ item.key }} has {{ item.value.free_g }}"
with_dict: "{{ ansible_lvm.vgs }}"
when: not item.key == "root"
For those like me taken here by Google and wondering where this ansible_lvm dict comes from :
it is part of the facts gathered by the setup module
it is collected only when executed with super-user privileges
In other words, to get ansible_lvm :
with an ad-hoc command (the ✨magic✨ lies in the -b) :
ansible myManagedNode -m setup -kK -b
within a playbook :
- hosts: myManagedNode
become: yes
tasks:
- debug:
var: ansible_lvm