Appending directory with a list in ansible - ansible

I'm trying to get a list of grants based on database role to re-use in one of the modules. For that the result needs to be a list
name: adding permissions
module:
role: database_role
permissions:
- "schema:USAGE/table1:SELECT/table2:SELECT,UPDATE"
- "another_schema:USAGE/ALL:ALL"
My permissions are defined as variables as follows:
db_roles:
- name: role1
grants:
- schema: schema
permissions:
- table1:SELECT
- table2:SELECT,UPDATE
- schema: another_schema:
permissions:
- ALL:ALL
I have more roles defined as well. This definition of roles means I can add a new permission per row, making it more readable.
Now I'm trying to format this variable to receive something like this:
permissions:
- role1:
- "schema:USAGE/table1:SELECT/table2:SELECT,UPDATE"
- "another_schema:USAGE/ALL:ALL"
- role2:
- "schema:USAGE/ALL:ALL"
But I have no idea how to get to this result.
What I've tried
So far the furthest I got is this but I'm not sure if it's possible to retrieve the values from the dictionary somehow
ok: [localhost] => {
"permissions": {
"role1": {
"schema": "schema:USAGE/table1:SELECT/table2:SELECT",
"another_schema": "another_schema:USAGE/ALL:ALL"
}
}
}
The code that got me there is:
- name: Create privs for users
set_fact:
permissions: "{{ permissions|default( {item.0.name:{}} ) | combine( {item.0.name:{item.1.schema: item.1.schema ~ ':USAGE/' ~ item.1.permissions | join('/')}}, recursive=True) }}"
with_subelements:
- "{{ db_roles }}"
- grants

tough one :) hope i got it right.
picking up where you left off, i added this task, to further process the variable you prepared:
- name: Create privs for users - step 2
set_fact:
permissions_final: "{{ permissions_final|default([]) + [{ item : permissions[item] | dict2items | map(attribute='value') | list }] }}"
with_items:
- "{{ permissions.keys() | list }}"
full code and sample output:
---
- hosts: localhost
gather_facts: false
vars:
db_roles:
- name: role1
grants:
- schema: schema
permissions:
- table1:SELECT
- table2:SELECT,UPDATE
- schema: another_schema
permissions:
- ALL:ALL
- name: role2
grants:
- schema: schema3
permissions:
- table1:SELECT
- table2:SELECT,UPDATE
- schema: another_schema4
permissions:
- ALL:ALL
tasks:
- name: Create privs for users
set_fact:
permissions: "{{ permissions|default( {item.0.name:{}} ) | combine( {item.0.name:{item.1.schema: item.1.schema ~ ':USAGE/' ~ item.1.permissions | join('/')}}, recursive=True) }}"
with_subelements:
- "{{ db_roles }}"
- grants
# - name: print results
# debug:
# var: permissions
- name: Create privs for users - step 2
set_fact:
permissions_final: "{{ permissions_final|default([]) + [{ item : permissions[item] | dict2items | map(attribute='value') | list }] }}"
with_items:
- "{{ permissions.keys() | list }}"
- name: print results
debug:
var: permissions_final
variable produced:
TASK [print results] ***************************************************************************************************************************************************************************************************
ok: [localhost] => {
"permissions_final": [
{
"role1": [
"schema:USAGE/table1:SELECT/table2:SELECT,UPDATE",
"another_schema:USAGE/ALL:ALL"
]
},
{
"role2": [
"schema3:USAGE/table1:SELECT/table2:SELECT,UPDATE",
"another_schema4:USAGE/ALL:ALL"
]
}
]
}
hope this helps!

Related

Ansible: Extract json objects from file based on a list

I have a json file like so
{
"service_accounts": {
"sql-automation": "Automation Service account for Cloud SQL Instances",
"gke-proxy": "GKE proxy account",
"gke-node": "Default GKE nodes service account",
"syncdata": "Data sync between environments",
}
}
Then I have another json file with the default access the accounts need (this is a template file)
{
"key_gen_auths": {
"worker": [{
"role": "roles/iam.serviceAccountKeyAdmin",
"members": [
"serviceAccount:dataprovisioning#{{ project_id }}.iam.gserviceaccount.com"
]
}],
"gke-node": [{
"role": "roles/iam.serviceAccountUser",
"members": [
"serviceAccount:deployment-automation#{{ project_id }}.iam.gserviceaccount.com"
]
}],
"sql-automation": [{
"role": "roles/iam.serviceAccountUser",
"members": [
"serviceAccount:deployment-automation#{{ project_id }}.iam.gserviceaccount.com"
]
}]
}
}
Then in the playbook, i would like to say:
If the service account exists in the service_accounts file, extract that object from key_gen_auths if it exists there (if it doesnt exist in key_gen_auths ignore). Ive looked at several options, like selectattr, combine, difference, but im no further than when i started really, and wonder if im just better with a when statement if i can find the right syntax, any help would be appreciated.
- name: Play for reading json
hosts: localhost
vars:
project_id: 123
tasks:
- name: acct_list
set_fact:
acct_list: "{{ lookup('template', 'acct_list.json') }}"
- name: key_auth_default
set_fact:
keyauth_default: "{{ lookup('template', 'key_auths.json') }}"
- name:
set_fact:
access_file: "{{ item }}"
with_items: "{{ keyauth_default.key_gen_auths }}"
when: item in acct_list.service_accounts
- debug:
var: access_file
You can read the files without lookup
vars_files:
- acct_list.json
- key_auths.json
vars:
project_id: 123
give
service_accounts:
gke-node: Default GKE nodes service account
gke-proxy: GKE proxy account
sql-automation: Automation Service account for Cloud SQL Instances
syncdata: Data sync between environments
key_gen_auths:
gke-node:
- members:
- serviceAccount:deployment-automation#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountUser
sql-automation:
- members:
- serviceAccount:deployment-automation#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountUser
worker:
- members:
- serviceAccount:dataprovisioning#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountKeyAdmin
Q: "If the service account exists in service_accounts extract that object from key_gen_auths."
A: Get the lists of the keys from the dictionaries. Then the intersect of these lists gives the existing service accounts
service_accounts_list: "{{ service_accounts.keys()|list }}"
key_gen_auths_list: "{{ key_gen_auths.keys()|list }}"
service_account_exists: "{{ service_accounts_list|intersect(key_gen_auths_list) }}"
give
service_accounts_list: [sql-automation, gke-proxy, gke-node, syncdata]
key_gen_auths_list: [worker, gke-node, sql-automation]
service_account_exists: [sql-automation, gke-node]
Use the list of existing accounts to extract the objects and create a dictionary
access_file: "{{ dict(service_account_exists|
zip(service_account_exists|map('extract', key_gen_auths))) }}"
give
access_file:
gke-node:
- members:
- serviceAccount:deployment-automation#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountUser
sql-automation:
- members:
- serviceAccount:deployment-automation#123.iam.gserviceaccount.com
role: roles/iam.serviceAccountUser
Example of the complete playbook for testing
- hosts: localhost
vars_files:
- acct_list.json
- key_auths.json
vars:
project_id: 123
service_accounts_list: "{{ service_accounts.keys()|list }}"
key_gen_auths_list: "{{ key_gen_auths.keys()|list }}"
service_account_exists: "{{ service_accounts_list|intersect(key_gen_auths_list) }}"
access_file: "{{ dict(service_account_exists|
zip(service_account_exists|map('extract', key_gen_auths))) }}"
tasks:
- debug:
var: service_accounts
- debug:
var: key_gen_auths
# If the service account exists in service_accounts
# extract those objects from key_gen_auths
- debug:
var: service_accounts_list|to_yaml
- debug:
var: key_gen_auths_list|to_yaml
- debug:
var: service_account_exists|to_yaml
- debug:
var: access_file
Well, I think this shows the data you want, in debug, and you can use that with set_fact as you see fit:
- name: Play for reading json
hosts: localhost
vars_files:
- acct_list.json
- key_auths.json
vars:
project_id: 123
tasks:
- debug:
msg: "{{ key_gen_auths[item] | default('Not Found') }}"
loop: "{{ service_accounts.keys() }}"
ignore_errors: yes

Ansible when condition registered from csv

I'm using csv file as ingest data for my playbooks, but im having trouble with my when condition. it's either both task will skipped or both task will be ok, my objective is if ansible see the string in when condition it will skipped for the specific instance.
here is my playbook
- name: "Read ingest file from CSV return a list"
community.general.read_csv:
path: sample.csv
register: ingest
- name: debug ingest
debug:
msg: "{{ item.AWS_ACCOUNT }}"
with_items:
- "{{ ingest.list }}"
register: account
- name: debug account
debug:
msg: "{{ account.results | map(attribute='msg') }}"
register: accountlist
- name:
become: yes
become_user: awx
delegate_to: localhost
environment: "{{ proxy_env }}"
block:
- name: "Assume role"
community.aws.sts_assume_role:
role_arn: "{{ item.ROLE_ARN }}"
role_session_name: "pm"
with_items:
- "{{ ingest.list }}"
register: assumed_role
when: "'aws-account-rnd' not in account.results | map(attribute='msg')"
here is the content of sample.csv
HOSTNAME
ENVIRONMENT
AWS_ACCOUNT
ROLE_ARN
test1
dev
aws-account-rnd
arn:aws:iam::XXXX1
test2
uat
aws-account-uat
arn:aws:iam::XXXX2
my objective is to skipped all items in the csv file with aws-acount-rnd
Your condition does not mention item so it will have the same result for all loop items.
Nothing you've shown requires the weird abuse of debug + register that you're doing, and it is in fact getting in your way.
- name: Read CSV file
community.general.read_csv:
path: sample.csv
register: ingest
- name: Assume role
community.aws.sts_assume_role:
role_arn: "{{ item.ROLE_ARN }}"
role_session_name: pm
delegate_to: localhost
become: true
become_user: awx
environment: "{{ proxy_env }}"
loop: "{{ ingest.list }}"
when: item.AWS_ACCOUNT != 'aws-account-rnd'
register: assumed_role
If you'll always only care about one match you can also do this without a loop or condition at all:
- name: Assume role
community.aws.sts_assume_role:
role_arn: "{{ ingest.list | rejectattr('AWS_ACCOUNT', '==', 'aws-account-rnd') | map(attribute='ROLE_ARN') | first }}"
role_session_name: pm
delegate_to: localhost
become: true
become_user: awx
environment: "{{ proxy_env }}"
register: assumed_role
my objective is to skipped all items in the csv file with aws-acount-rnd
The multiple debug you have with register, seems to be a long-winded approach IMHO.
A simple task to debug the Role ARN, only if the account does not match aws-acount-rnd.
- name: show ROLE_ARN when account not equals aws-account-rnd
debug:
var: item['ROLE_ARN']
loop: "{{ ingest.list }}"
when: item['AWS_ACCOUNT'] != 'aws-account-rnd'
This results in:
TASK [show ROLE_ARN when account not equals aws-account-rnd] **********************************************************************************************************************
skipping: [localhost] => (item={'HOSTNAME': 'test1', 'ENVIRONMENT': 'dev', 'AWS_ACCOUNT': 'aws-account-rnd', 'ROLE_ARN': 'arn:aws:iam:XXXX1'})
ok: [localhost] => (item={'HOSTNAME': 'test2', 'ENVIRONMENT': 'uat', 'AWS_ACCOUNT': 'aws-account-uat', 'ROLE_ARN': 'arn:aws:iam:XXXX2'}) => {
"ansible_loop_var": "item",
"item": {
"AWS_ACCOUNT": "aws-account-uat",
"ENVIRONMENT": "uat",
"HOSTNAME": "test2",
"ROLE_ARN": "arn:aws:iam:XXXX2"
},
"item['ROLE_ARN']": "arn:aws:iam:XXXX2"
}
The same logic can be used to pass the item.ROLE_ARN to community.aws.sts_assume_role task.

How to generate a list with a range and save it in a var of a playbook?

How can I create a list like this in an Ansible playbook:
['user1', 'user2', 'user3', '...', 'user20']
and save it to a var in that same playbook?
I tried Using set_facts and with_items together in Ansible and modified the answer of #serge a little bit.
My code looks like the following:
- name: preparation for list
set_fact:
userItem: userItem= "{{ item }}"
with_sequence: start=1 end=20 format=user%d
register: userResult
- name: make a list
set_fact: users= "{{ userResult.results | map(attribute= 'ansible_facts.userItem) | list }}"
- name resulted list
debug: var=users
But I get the following error:
TASK [make a list] *****************************************************************
fatal: [10.0.2.11]: FAILED! => {"msg": "template error while templating string: unexpected end of template, expected ',' .. String: \"{{ userResult.results | map(attribute"}
I want to use that list in a task where I change a line in a configuration file like this:
- name: allow users to ssh into jail
lineinfile:
path: /etc/ssh/shd_config
regexp: '^AllowUsers '
line: 'AllowUsers ansible {{ users }}'
notify: "restart ssh"
where {{ users }} should be replaced with the desired list.
Is there a way I can achieve getting the desired list saved in some kind of variable (not a file) so I can use it in other tasks?
You can create a list in a set_fact right away, there is no need for any intermediary task:
- set_fact:
users: "{{ users | default([]) + ['user%s' | format(item)] }}"
loop: "{{ range(1, 21) | list }}"
Which gives
{
"users": [
"user1",
"user2",
"user3",
"user4",
"user5",
"user6",
"user7",
"user8",
"user9",
"user10",
"user11",
"user12",
"user13",
"user14",
"user15",
"user16",
"user17",
"user18",
"user19",
"user20"
]
}
This is actually really close to the example found in the documentation: https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html#with-sequence
This said you won't be able to fit a list into your lineinfile module, because you do need a string.
So you can either create a string right away:
- set_fact:
users: "{{ users | default('') + ' user%s' | format(item) }}"
loop: "{{ range(1, 21) | list }}"
- lineinfile:
path: /etc/ssh/shd_config
regexp: '^AllowUsers '
line: 'AllowUsers ansible {{ users }}'
notify: "restart ssh"
So, instead of creating a list we are creating a string:
{
"users": " user1 user2 user3 user4 user5 user6 user7 user8 user9 user10 user11 user12 user13 user14 user15 user16 user17 user18 user19 user20"
}
Or use the join filter when using the users list:
- set_fact:
users: "{{ users | default([]) + ['user%s' | format(item)] }}"
loop: "{{ range(1, 21) | list }}"
- lineinfile:
path: /etc/ssh/shd_config
regexp: '^AllowUsers '
line: 'AllowUsers ansible {{ users | join(' ') }}'
notify: "restart ssh"
For example
- set_fact:
users: "{{ ['user']|product(range(start,end))|map('join')|list }}"
vars:
start: 1
end: 5
gives
users:
- user1
- user2
- user3
- user4
The next option gives the same result
- set_fact:
users: "{{ query('sequence', user_range) }}"
vars:
start: 1
end: 4
user_range: "start={{ start }} end={{ end }} format=user%d"

How to create a list of dictionary of users and list of folders for each user?

I would like to create a dictionary of users which has list of folders in it.
users:
name: user1
folders:
- user1-1
- user1-2
name: user2
folders:
- user2-1
- user2-2
name: userN
folders:
- userN-1
- userN-2
I am trying to create this user dictionary list dynamically.
- hosts: localhost
gather_facts: no
vars:
userCount: 5
folderCount: 5
tasks:
- set_fact:
user_lists: []
- name: creating list of users
set_fact:
user_lists: "{{ user_lists + [ 'user-' ~ item ] }}"
with_sequence: count="{{userCount|int}}"
- debug:
var: user_lists
- set_fact:
usersObj: []
userDict: {}
- name: add each user to dict
set_fact:
usersObj: "{{ usersObj + [userDict| combine({'name': item})] }}"
with_items: "{{user_lists}}"
- debug: var=usersObj
I have created 5 users in dictionary. Now each user should have folder list created based on folderCount. How to resolve this? Can Jinja2 templates be used to simplify this?
The task below does the job
- set_fact:
user_lists: []
- name: creating dictionary of users with lists of folders
set_fact:
user_lists: "{{ user_lists|combine({key: val}) }}"
with_sequence: start=1 end="{{ userCount }}"
vars:
folder_range: "{{ range(1, folderCount|int + 1)|list }}"
key: "{{ 'user-' ~ item }}"
val: "{{ [key]|
product(folder_range)|
map('join', '-')|
list }}"
- debug:
var: user_lists
gives (abridged)
"user_lists": {
"user-1": [
"user-1-1",
"user-1-2",
"user-1-3",
"user-1-4",
"user-1-5"
],
"user-2": [
"user-2-1",
"user-2-2",
"user-2-3",
"user-2-4",
"user-2-5"
],
"user-3": [
"user-3-1",
"user-3-2",
"user-3-3",
"user-3-4",
"user-3-5"
],
"user-4": [
"user-4-1",
"user-4-2",
"user-4-3",
"user-4-4",
"user-4-5"
],
"user-5": [
"user-5-1",
"user-5-2",
"user-5-3",
"user-5-4",
"user-5-5"
]
}

Using Ansible loop to create multiple users: Undefined error

I am using the following ansible code to create multiple unix user accounts
---
- hosts: test
become: true
tasks:
- name: more complex items to add several users
user:
name: "{{ item.name }}"
uid: "{{ item.uid }}"
groups: "{{ item.groups }}"
state: present
with_items: "{{ user_details }}"
I am storing the user information by using a separate a variable file as below
`cat /etc/ansible/vars.yml
---
user_details:
- { name: testuser1, uid: 1002, groups: "admin, logs" }
- { name: testuser2, uid: 1003, groups: logs: }`
To execute above playbook , I tried with both the commands below
sudo ansible-playbook /etc/ansible/userloop.yml -e /etc/ansible/vars.yml
sudo ansible-playbook /etc/ansible/userloop.yml
but both commands are failing with below error
fatal: [host-003]: FAILED! => {"msg": "'user_details' is undefined"}
fatal: [host-004]: FAILED! => {"msg": "'user_details' is undefined"}
How to resolve the issue ? I want to maintain a separate variable file to store the user information rather then putting them in the same playbook file .
You can also refer the multiple variable files in playbooks like below
- hosts: all
become: true
vars_files:
- /etc/ansible/vars.yml
tasks:
- name: more complex items to add several users
user:
name: "{{ item.name }}"
uid: "{{ item.uid }}"
groups: "{{ item.groups }}"
state: present
with_items: "{{ user_details }}"
The type of variables is in the column "Parameter" of the module user. Try the structure of the data below
user_details:
- {name: 'testuser1', uid: 1002, groups: ['admin', 'logs']}
- {name: 'testuser2', uid: 1003, groups: ['logs']}
You are missing # while passing the vars.yml. Hence, the ansible is not reading the file. Try the below command. It works for me.
sudo ansible-playbook /etc/ansible/userloop.yml -e #/etc/ansible/vars.yml

Resources