Ansible - using {{ item }} with csvfile lookups - ansible

I'm trying to work out a way to cut down on lines of code for various modules, repeating stanzas seems pointless. I'd like to use csvfile lookups to help fill in blanks. Take for example the following CSV:
# groups.csv
# name, gid [optional - leave blank], state [present|absent], system [yes|no]
accounts,502,present,no
engineering,504,present,no
So, I have all my group definitions in csv format. The problem is, processing it, no matter what I try I cannot get lookups to work inside the groups module.
So initially, I wanted to do this:
---
- hosts: localhost
become: True
become_user: root
tasks:
- name: get groups
command: /usr/bin/awk -F',' '!/^#/ && !/^$/ { print $1 }' groups.csv
register: groups_out
- debug: var=groups_out.stdout_lines
- name: Process groups
group: >
name="{{ lookup('csvfile', 'item file=groups.csv col=0') }}"
gid="{{ lookup('csvfile', 'item file=groups.csv col=1') }}"
state="{{ lookup('csvfile', 'item file=groups.csv col=2') }}"
system="{{ lookup('csvfile', 'item file=groups.csv col=3') }}"
# with_lines: "/usr/bin/awk -F',' '!/^#/ && !/^$/ { print $1 }' groups.csv"
# with_items: "{{ groups_out.stdout_lines }}"
with_lines: "{{ groups_out.stdout_lines }}"
The result of which is this:
TASK [Process groups] **********************************************************
/bin/sh: accounts: command not found
fatal: [localhost]: FAILED! => {"failed": true, "msg": "lookup_plugin.lines(accounts) returned 127"}
As you can see from the code, I've also tried using with_items and with_lines using the awk command directly, however it appears the groups module doesn't like me doing this.
Ansible 2.1.1.0 on Centos 7.
Python 2.7.5
Jinja 2.8
Any ideas how I might achieve this?
Thanks in advance,
R

Answer below. Thanks to Jon and Kai on the ansible-project googlegroup for their assistance.
---
- hosts: localhost
become: True
become_user: root
tasks:
- name: get groups
command: /usr/bin/awk -F',' '!/^#/ && !/^$/ { print $1 }' /var/tmp/groups.csv
register: groups_out
- name: Process groups one
group: >
name={{ lookup('csvfile', item + ' file=groups.csv col=0 delimiter=,') }}
gid={{ lookup('csvfile', item + ' file=groups.csv col=1 delimiter=,') }}
state={{ lookup('csvfile', item + ' file=groups.csv col=2 delimiter=,') }}
system={{ lookup('csvfile', item + ' file=groups.csv col=3 delimiter=,') }}
with_items: "{{ groups_out.stdout_lines }}"
ignore_errors: True
- name: Process groups two
group: >
name={{ lookup('csvfile', item + ' file=groups.csv col=0 delimiter=,') }}
gid={{ lookup('csvfile', item + ' file=groups.csv col=1 delimiter=,') }}
state={{ lookup('csvfile', item + ' file=groups.csv col=2 delimiter=,') }}
system={{ lookup('csvfile', item + ' file=groups.csv col=3 delimiter=,') }}
with_lines: /usr/bin/awk -F',' '!/^#/ && !/^$/ { print $1 }' /var/tmp/groups.csv

Related

Ansible playbook loop from site yaml or template?

I'm trying to use my Ansible playbook to call upon a site YAML reference to create a filename that increment for multiple switches. What am I doing wrong? I believe the playbook is pulling from the host YAML?
Format: <switch>-<site>-<floor><stackid>.txt
e.g.: with two switches:
swi-lon-101.txt
swi-lon-202.txt
host_vars/host.yaml
project_name: test
device_name: swi
site_abbrev: lon
device_type: switch
switch_stacks:
- id: 01
installation_floor: 1
- id: 02
installation_floor: 2
templates/switch-template.j2
{% for stack in switch_stacks %}
set system host-name {{ device_name }}-{{ site_abbrev }}-{{ stack.installation_floor }}{{ stack.id }}
{% endfor %}
The playbook, in which the problem lies, how do I get the hostname to create correctly for each of the 2 switches?
My playbook:
- name: Create Folder Structure
hosts: junos
gather_facts: false
tasks:
- name: Create Site Specific Folder
file:
path: /home/usr/complete_config/{{ project_name }}
state: directory
mode: 0755
- name: Set Destination Directory & Filename for Switch Configurations
set_fact:
dest_dir: /home/usr/complete_config/{{ project_name }}
full_device_name: "{{ device_name|lower }}-{{ site_abbrev|lower }}-{{ switch_stacks.installation_floor }}{{ switch_stacks.id }}.txt"
when: device_type == 'switch'
Ansible error, running:
ansible-playbook playbooks/site-playbook.yaml
TASK [Set Destination Directory & Filename for Switch Configurations] **************************************************
fatal: [site-switch]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'list object' has no attribute 'installation_floor'\n\nThe error appears to be in '/home/usr/playbooks/switch-playbook.yaml': line 19, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Set Destination Directory & Filename for Switch Configurations\n ^ here\n"}
So, you do need a loop in order to set this fact, otherwise, you are trying to access a installation_floor on a list, which cannot be.
You will also face an issue with the id of your items in switch_stacks, as 01 is an int and will end up displayed as 1, simply. So you either need to declare those as string, or to pad them with a format filter.
So, you end up with this task:
- set_fact:
full_device_name: >-
{{
full_device_name
| default([])
+ [
device_name | lower ~ '-' ~
site_abbrev | lower ~ '-' ~
item.installation_floor ~
"%02d" | format(item.id) ~ '.txt'
]
}}
loop: "{{ switch_stacks }}"
when: device_type == 'switch'
Which will create a list:
full_device_name:
- swi-lon-101.txt
- swi-lon-202.txt
Given the playbook:
- hosts: localhost
gather_facts: false
tasks:
- set_fact:
full_device_name: >-
{{
full_device_name
| default([])
+ [
device_name | lower ~ '-' ~
site_abbrev | lower ~ '-' ~
item.installation_floor ~
"%02d" | format(item.id) ~ '.txt'
]
}}
loop: "{{ switch_stacks }}"
when: device_type == 'switch'
vars:
device_name: swi
site_abbrev: lon
device_type: switch
switch_stacks:
- id: 01
installation_floor: 1
- id: 02
installation_floor: 2
- debug:
var: full_device_name
This yields:
TASK [set_fact] ************************************************************
ok: [localhost] => (item={'id': 1, 'installation_floor': 1})
ok: [localhost] => (item={'id': 2, 'installation_floor': 2})
TASK [debug] ***************************************************************
ok: [localhost] =>
full_device_name:
- swi-lon-101.txt
- swi-lon-202.txt
Let's keep the attribute id as a string
switch_stacks:
- id: '01'
installation_floor: 1
- id: '02'
installation_floor: 2
The task
- set_fact:
full_device_name: "{{ [prefix]|product(sw_fl_id)|map('join')|
product([postfix])|map('join')|list }}"
vars:
prefix: "{{ device_name }}-{{ site_abbrev }}-"
postfix: ".txt"
sw_id: "{{ switch_stacks|map(attribute='id')|list }}"
sw_fl: "{{ switch_stacks|map(attribute='installation_floor')|list }}"
sw_fl_id: "{{ sw_fl|zip(sw_id)|map('join')|list }}"
gives you the list
full_device_name:
- swi-lon-101.txt
- swi-lon-202.txt
#β.εηοιτ.βε
Thanks for your reply, it worked.
I have however slightly tweeked it as you will see below.
Once again thanks for showing the way
name: Set Destination Directory & Filename for Switch Configurations
set_fact:
dest_dir: /home/usr/complete_config/{{ project_name }}
full_device_name: >-
{{
device_name | lower ~ '-' ~
site_abbrev | lower ~ '-' ~
item.installation_floor ~
"%02d" | format(item.id)
}}
loop: "{{ switch_stacks }}"
when: device_type == 'switch'

Error with splitting Arguments due to bad quotes Ansible

I am writing a script, which should alter the Users of all Tenants of the Database. When i print the Command and execute it from hand on the Server, it works fine. With the Command Module i get this weird Error:
ERROR! failed at splitting arguments, either an unbalanced jinja2 block or quotes: {{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a "{{ item.stdout | replace('"', '')}}"
The error appears to be in '/tmp/awx_313202_z7e4l568/project/ansible_labor/scripts/update_users.yml': line 37, column 5, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Execute Querys on Tenants
^ here
I think it has something to do with the Quotes in the Command Module.
This is the Script:
---
- name: Update all Users in DB
hosts: '*'
become: true
vars:
_User: "{{ lookup('env', 'USER') | upper }}_M"
_pattern: ^\"[a-zA-Z0-9]*\"
_tenants: []
_query: select 'alter user ' || user_name || ' IDENTIFIED EXTERNALLY AS ''' || user_name ||'#INTERN'';' as todo from users where EXTERNAL_IDENTITY LIKE '%COMPANY.DE';
_hdbclient: "/hana/shared/{{ SID }}/hdbclient/hdbsql -i {{ Instanz }} -n {{ inventory_hostname }}"
tasks:
- name: "Tenants der DB auslesen"
command: "{{ _hdbclient }} -U BASIS_{{ SID }}_SYSTEMDB -x -a 'SELECT * FROM M_DATABASES'"
register: tenants
- debug:
msg: "{{ tenants }}"
- set_fact:
_tenants: "{{ _tenants + [ item | regex_search(_pattern) | replace('\"', '') ] }}"
with_items: "{{ tenants.stdout_lines }}"
- name: Print extracted Tenants
debug:
msg: "{{ _tenants }}"
- name: Create Query on Tenant
command: '{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item }} -x -a "{{ _query }}"'
register: query_tenants
with_items: "{{ _tenants }}"
- name: Print All Queries
debug:
msg: "{{ query_tenants.results[1] }}"
- name: Execute Querys on Tenants
command: "{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a \"{{ item.stdout | replace('\"', '')}}\" "
with_items: "{{ query_tenants.results }}"
This is an escape issue inside your replace filter. You need to replace double quotes which translates as a string to \". Since that string is itself inside a double quoted string you have to escape the escape => \\\n
So keeping your initial quoting this gives:
command: "{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a \"{{ item.stdout | replace('\\\"', '') }}\""
Meanwhile there are a few (non exhaustive list) alternatives you might want to consider:
command: '{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a "{{ item.stdout | replace("\"", "") }}"'
command: >-
{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a "{{ item.stdout | replace("\"", "") }}"

How to generate a list with a range and save it in a var of a playbook?

How can I create a list like this in an Ansible playbook:
['user1', 'user2', 'user3', '...', 'user20']
and save it to a var in that same playbook?
I tried Using set_facts and with_items together in Ansible and modified the answer of #serge a little bit.
My code looks like the following:
- name: preparation for list
set_fact:
userItem: userItem= "{{ item }}"
with_sequence: start=1 end=20 format=user%d
register: userResult
- name: make a list
set_fact: users= "{{ userResult.results | map(attribute= 'ansible_facts.userItem) | list }}"
- name resulted list
debug: var=users
But I get the following error:
TASK [make a list] *****************************************************************
fatal: [10.0.2.11]: FAILED! => {"msg": "template error while templating string: unexpected end of template, expected ',' .. String: \"{{ userResult.results | map(attribute"}
I want to use that list in a task where I change a line in a configuration file like this:
- name: allow users to ssh into jail
lineinfile:
path: /etc/ssh/shd_config
regexp: '^AllowUsers '
line: 'AllowUsers ansible {{ users }}'
notify: "restart ssh"
where {{ users }} should be replaced with the desired list.
Is there a way I can achieve getting the desired list saved in some kind of variable (not a file) so I can use it in other tasks?
You can create a list in a set_fact right away, there is no need for any intermediary task:
- set_fact:
users: "{{ users | default([]) + ['user%s' | format(item)] }}"
loop: "{{ range(1, 21) | list }}"
Which gives
{
"users": [
"user1",
"user2",
"user3",
"user4",
"user5",
"user6",
"user7",
"user8",
"user9",
"user10",
"user11",
"user12",
"user13",
"user14",
"user15",
"user16",
"user17",
"user18",
"user19",
"user20"
]
}
This is actually really close to the example found in the documentation: https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html#with-sequence
This said you won't be able to fit a list into your lineinfile module, because you do need a string.
So you can either create a string right away:
- set_fact:
users: "{{ users | default('') + ' user%s' | format(item) }}"
loop: "{{ range(1, 21) | list }}"
- lineinfile:
path: /etc/ssh/shd_config
regexp: '^AllowUsers '
line: 'AllowUsers ansible {{ users }}'
notify: "restart ssh"
So, instead of creating a list we are creating a string:
{
"users": " user1 user2 user3 user4 user5 user6 user7 user8 user9 user10 user11 user12 user13 user14 user15 user16 user17 user18 user19 user20"
}
Or use the join filter when using the users list:
- set_fact:
users: "{{ users | default([]) + ['user%s' | format(item)] }}"
loop: "{{ range(1, 21) | list }}"
- lineinfile:
path: /etc/ssh/shd_config
regexp: '^AllowUsers '
line: 'AllowUsers ansible {{ users | join(' ') }}'
notify: "restart ssh"
For example
- set_fact:
users: "{{ ['user']|product(range(start,end))|map('join')|list }}"
vars:
start: 1
end: 5
gives
users:
- user1
- user2
- user3
- user4
The next option gives the same result
- set_fact:
users: "{{ query('sequence', user_range) }}"
vars:
start: 1
end: 4
user_range: "start={{ start }} end={{ end }} format=user%d"

Ansible - Iterate Thru Folders Containing Files. Output to file

Hoping someone might be able to provide me an answer with this. I currently have a folder structure like so.
/BASE_DIR
/FOLDER_A
- file1.txt
- file2.txt
/FOLDER_B
/FOLDER_C
- file3.txt
Im trying to create a playbook that could tell me which folders contain files. My end goal is to have a flat file with:
FOLDER_A, file1.txt
FOLDER_A, file2.txt
FOLDER_C, file3.txt
This is my playbook currently:
- name: get files from all folders
shell: cd /BASE_DIR/{{ item.name }} && ls -p | grep -v / |grep .txt |cat
with_items:
- name: "FOLDER_A"
- name: "FOLDER_B"
- name: "FOLDER_C"
register: "fileitems"
- name: combine to have folder name as key, filenames as values
set_fact:
folders_with_files: "{{ folders_with_files|default({}) | combine( { item.item.name: item.stdout_lines } ) }}"
with_items: "{{ fileitems.results }}"
when: "{{ item.stdout_lines|length }} > 0"
- debug:
var: folders_with_files
I thought I could iterate through each folder looking for *.txt and then use a combine, it would be an easy way to iterate.
ok: [localhost] => {
"folders_with_files": {
"FOLDER_A": [
"file1.txt",
"file2.txt"
],
"FOLDER_C": [
"file3.txt"
]
}
}
But even with this output, I don't think I can properly parse it the way I need to. I thought maybe a nested loop could help, but that would mean I would need to know the name of the keys beforehand.
Any help would be appreciated!
Thanks,
T
Go figure as soon as I post the question, I find my own answer...
I decided to remove the combine and just append to an empty list.
- set_fact:
folders_with_files: []
- name: get all sql from each adapter
shell: cd /tmp/{{ item.name }} && ls -p | grep -v / |grep .txt |cat
with_items:
- name: "FOLDER_A"
- name: "FOLDER_B"
- name: "FOLDER_C"
register: "fileitems"
- name: combine to display which adapters have files
set_fact:
folders_with_files: "{{ folders_with_files + [{ 'name': item.item.name, 'files': item.stdout_lines }] }}"
with_items: "{{ fileitems.results }}"
when: "{{ item.stdout_lines|length }} > 0"
- debug:
var: folders_with_files
My output then became:
ok: [localhost] => {
"folders_with_files": [
{
"files": [
"file1.txt",
"file2.txt"
],
"name": "FOLDER_A"
},
{
"files": [
"file3.txt"
],
"name": "FOLDER_C"
}
]
}
I could then use a with_subelements:
- name: echo
shell: echo "{{ item.0.name }}, {{ item.1}}" >> /tmp/output.txt
with_subelements:
- "{{ folders_with_files }}"
- files

How to conditionally execute group of tasks on ansible?

I want to conditionally execute a set of tasks. Is there any syntax available that would let me execute a group of tasks, where the condition is evaluated once per whole group (like in a if statement in programming languages)?
Take a look at the code snippets below. I know the difference is small, but the first code better expresses my intention without polluting the namespace with additional variables (user_home_result2).
Pseudocode of what I want to do:
- name: Capturing user's home directory
shell: "getent passwd {{ user }} | awk -F: '{ print $6 }'"
register: user_home_result
- set_fact: user_home={{ user_home_result.stdout }}
- when: user_home != ''
- name: Setting up user {{ user }}
user: >
generate_ssh_key=yes
name="{{ user }}"
- name: Capturing user's home directory
shell: "getent passwd {{ user }} | awk -F: '{ print $6 }'"
register: user_home_result
- set_fact: user_home={{ user_home_result.stdout }}
Walkaround:
- name: Capturing user's home directory
shell: "getent passwd {{ user }} | awk -F: '{ print $6 }'"
register: user_home_result
- set_fact: user_home={{ user_home_result.stdout }}
- name: Setting up user {{ user }}
user: >
generate_ssh_key=yes
name="{{ user }}"
when: user_home != ''
- name: Capturing user's home directory
shell: "getent passwd {{ user }} | awk -F: '{ print $6 }'"
register: user_home_result2
when: user_home != ''
- set_fact: user_home={{ user_home_result2.stdout }}
when: user_home != ''
You can put the tasks in a new yml file and use a conditional include:
# subtasks.yml
---
- name: Setting up user {{ user }}
user: >
generate_ssh_key=yes
name="{{ user }}"
- name: Capturing user's home directory
shell: "getent passwd {{ user }} | awk -F: '{ print $6 }'"
register: user_home_result
- set_fact: user_home={{ user_home_result.stdout }}
And in the playbook:
- name: Capturing user's home directory
shell: "getent passwd {{ user }} | awk -F: '{ print $6 }'"
register: user_home_result
- set_fact: user_home={{ user_home_result.stdout }}
- include: subtask.yml
when: user_home != ''
As of version 2.1, ansible has blocks for logical task grouping. Blocks allow you to specify common things for a few tasks only once, including the when conditionals. Ex:
- block:
- name: put a file somewhere
copy: src=asdf dest=asdf
- name: put another file somewhere
template: src=asdf.j2 dest=asdf
when: bool_is_true
The above is equivalent to attaching a when: bool_is_true to both of the tasks inside the block.
More information at https://docs.ansible.com/ansible/latest/user_guide/playbooks_blocks.html

Resources