Load Nested Files With host_group_vars Plugin From Custom Locations - ansible

I am trying to create ansible role to generate kubernetes deployment with ansible. I want all customization to be stored in var files and be loaded automatically during group_vars and host_vars loading. I end up with following nested structure of files:
group_vars/my_group.yml:
applications:
application_foo: "{{ lookup('template', inventory_dir + '/applications/application_foo.yml') | from_yaml }}"
applications/application_foo.yml:
deployments:
application_foo:
deployment:
{{ lookup('template', inventory_dir + '/deployment_vars/deployment_foo.yml') | to_nice_yaml | indent(6)}}
replicas: 1
deployment_vars/deployment_foo.yml:
containers:
conteiner_bar:
container:
{{ lookup('template',inventory_dir + '/container_vars/container_bar.yml') | to_nice_yaml | indent(6) }}
container_baz:
container:
{{ lookup('template',inventory_dir + '/container_vars/container_baz.yml') | to_nice_yaml | indent(6) }}
container_vars/container_bar.yml:
container_vars/container_baz.yml:
image_url: "https://example.com/image_bar"
cpu_requests: "1"
memory_requests: "512Mi"
cpu_limit: "2"
memory_limit: "1024Mi"
readinessProbe:
exec:
command:
- "sh"
- "-c"
- "true"
playbooks/test-playbook.yml:
- hosts: my_group
gather_facts: False
tasks:
- debug:
msg: "{{ applications['application_foo']['deployments']['application_foo']['deployment']['containers']['container_bar']['cpu_requests'] }}"
delegate_to: localhost
When I run playbook it complains that deployment var is of str type and therefore can't have children objects:
fatal: [localhost]: FAILED! => {
"msg": "The task includes an option with an undefined variable. The error was: 'str object' has no attribute 'containers'\n\nThe error appears to have been in '/tmp/so_question/playbooks/test-playbook.yml': line 4, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n - debug:\n ^ here\n"
}
I want layers to be reusable for cases where container or deployment are required by another application so don't want to put everything in one file.
Any suggestions how to fix this issue or how to achieve same nested loading in other way?
UPD: when I cut string in playbook to end on deployment (like this msg: "{{ applications['application_foo']['deployments']['application_foo']['deployment'] }}") it gives following line which after newline substitution does not look like valid yaml:
containers:
conteiner_bar:
container:
\"image_url: \\\"https://example.com/image_bar\\\"\
cpu_requests: \\\"1\\\"\
memory_requests:\\
\\ \\\"512Mi\\\"\
cpu_limit: \\\"2\\\"\
memory_limit: \\\"1024Mi\\\"\
readinessProbe:\
exec:\
\\
\\ command:\
- \\\"sh\\\"\
- \\\"-c\\\"\
- \\\"true\\\"\
\"
container_baz:
container:
\"image_url: \\\"https://example.com/image_bar\\\"\
cpu_requests: \\\"1\\\"\
memory_requests:\\
\\ \\\"512Mi\\\"\
cpu_limit: \\\"2\\\"\
memory_limit: \\\"1024Mi\\\"\
readinessProbe:\
exec:\
\\
\\ command:\
- \\\"sh\\\"\
- \\\"-c\\\"\
- \\\"true\\\"\
\"
I wondering why indent filter does not indents all lines. Am I using it correctly?

Ended up with following workaround: Moved all files containing defaults to group_vars/all to make them loaded automatically and used vars lookup for nesting:
$ find group_vars/
group_vars/
group_vars/all
group_vars/all/10_container_bar.yml
group_vars/all/20_deployment_foo.yml
group_vars/all/30_application_foo.yml
group_vars/all/10_container_baz.yml
group_vars/my_group.yml
group_vars/my_group.yml:
applications:
application_foo: "{{ lookup('vars', 'defaults_application_application_foo') }}"
group_vars/all/30_application_foo.yml:
defaults_application_application_foo:
deployments:
application_foo:
deployment: "{{ lookup('vars', 'defaults_deployment_deployment_foo') }}"
replicas: 1
And so on...
Still not clear why template thing didn't work. Leaving this question open.

Related

Ansible- How to include /import playbook based on when condition result

I have folder in which i have many sub folder and those sub folder will have yaml file, by using the below code , I have got all yaml file and register those file,
tasks:
- name: Find .yml files
find:
paths: /temp/
patterns: '*.yml,*.yaml'
recurse: yes
register: yaml_path_files
One of the yaml file content is below,
apiVersion: v1
kind: ConfigMap
metadata:
name: special-config
namespace: default
data:
special.how: very
---
apiVersion: v1
kind: ConfigMap
metadata:
name: env-config
namespace: default
data:
log_level: INFO
---
apiVersion: v1
kind: ConfigMap
metadata:
name: env-config
namespace: default
data:
log_level: INFO
i want to check if the yaml contains kind is ConfigMap and that yaml file needs to be passed to role/another playbook as parmeter,i tried with below code, it is not working,
- debug:
var: item.path
when: lookup('file',item.path ) | from_yaml_all | list | selectattr('kind', '==', 'ConfigMap') | list | length > 0
with_items: "{{ yaml_path_files.files }}"
i am facing the below error,
fatal: [localhost]: FAILED! => {"msg": "The conditional check 'lookup('file',item.path ) | from_yaml | list | selectattr('kind', '==', 'ConfigMap') | list | length > 0' failed. The error was: error while evaluating conditional (lookup('file',{{ filename }} ) | from_yaml | list | selectattr('kind', '==', 'ConfigMap') | list | length > 0): 'dict object' has no attribute 'kind'\n\nThe error appears to be in '/cygdrive/c/senthil/Ansible/Ansibleusecase4/EKSartifactvalidationV2.yml': line 14, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n # with_items: "{{ yaml_path_files.files }}"\n - debug:\n ^ here\n"}
kindly help me to achive it in ansible
Preliminary note: Your question is missing a complete playbook to be 100% sure of the problem. Edit your question with more info, possibly with an MCVE, if this is not your exact issue.
Your find tasks looks for files on your remote target. The file lookup, as all other lookups (see notes in the lookup plugins documentation), reads files locally on your ansible controller. In other words, you're looking for a file which does not exists.
You have to fetch or slurp the files from the remote target to get their content.
Here is a quickly written untested sample to put you on track. Please note on the last task that item.item.path is not a typo. item.item references the preceding item loop when registering the new var yaml_files_content (i.e. each yaml_path_files.files[*] from the preceding task, debug the full item to see for yourself)
- hosts: all
tasks:
- name: Find .yml files
find:
paths: /temp/
patterns: '*.yml,*.yaml'
recurse: yes
register: yaml_path_files
- name: Slurp file contents from remote
slurp:
src: "{{ item.path }}"
loop: "{{ yaml_path_files.files }}"
register: yaml_files_contents
- name: Show path for files containing ConfigMaps
debug:
var: item.item.path
loop: "{{ yaml_files_contents.results }}"
when: item.content | b64decode | from_yaml_all | list | selectattr('kind', '==', 'ConfigMap') | list | length > 0

Ansible - set_fact syntax issue

In the below playbook, can someone please tell me what is wrong with the syntax of the final set_fact statement. If i explicitly specify the hostname R9 (as per the first set_fact), then it works without issue. But if I assign a variable to have a value of R9, and then use this in the final set_fact statement, I get the error shown below. Argh.
---
- hosts: all
gather_facts: no
vars:
ansible_network_os: ios
ansible_connection: network_cli
tasks:
- ios_command:
commands: show ip arp
register: results
- set_fact:
some_var: "{{ hostvars['R9'].results }}"
- set_fact:
hostname: "R9"
- set_fact:
another_var: "{{ hostvars['{{hostname}}'].results }}"
- debug:
var=some_var
- debug:
var=another_var
Error:
fatal: [R7]: FAILED! =>
msg: |-
The task includes an option with an undefined variable. The error was: "hostvars['{{hostname}}']" is undefined
The error appears to be in '/root/ansible/Rapid/testsetfact.yml': line 20, column 6, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- set_fact:
^ here
fatal: [R9]: FAILED! =>
msg: |-
The task includes an option with an undefined variable. The error was: "hostvars['{{hostname}}']" is undefined
The error appears to be in '/root/ansible/Rapid/testsetfact.yml': line 20, column 6, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- set_fact:
^ here
you can use the inventory_hostname for your purpose:
- set_fact:
another_var: "{{ hostvars[inventory_hostname].results }}"

Syntax to pass dynamic variables to include_tasks along with with_item in Ansible playbook

Executing parent.yml which in turn calls child.yml playbook for execution with dynamic variables.
Variables from parent.yml aren`t interpolated inside child.yml playbook. Correct me if I am using correct syntax?
Parent.yml
- name: Main playbook to call MySQL backup
hosts: localhost
gather_facts: no
tasks:
- include_task: child.yml
vars:
var1: "{{ item.name }}"
var2: "{{ item.db_name }}"
with_items:
- { name: '10.10.10.01', db_name: 'prod1' }
- { name: '10.10.10.02', db_name: 'prod2' }
child.yml (Takes mysqldump from managed DB)
- name: MySQL dump
hosts: localhost
#gather_facts: no
#vars:
# v1: "{{ var1 }}"
# v2: "{{ var2 }}"
tasks:
- name: Executing the shell script
shell: 'mysqldump -h "{{ var1 }}" -u"ansi" -p"*****" "{{ var2 }}"| gzip > /tmp/mysql_dump/"{{ var2 }}"_`date +%Y%m%d-%H%M`.gz'
fatal: [127.0.0.1]: FAILED! => {"reason": "no action detected in task. This often indicates a misspelled module name, or incorrect module path.\n\nThe error appears to be in '/home/ansible/playbooks/DBpatch/Linux/child.yml': line 1, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: MySQL dump\n ^ here\n"}
include_task expects a list of tasks but you give it a complete playbook.
Child.yml should only contain what is currently below the line "tasks:".
See also https://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse_includes.html

How to register with_items and act on conditional check result for each item

I'd like to register the contents of bashrc for two users and edit as/if required. My play is as follows.
- name: Check bashrc
shell: cat {{ item }}/.bashrc
register: bashrc
with_items:
- "{{ nodepool_home }}"
- "{{ zuul_home }}"
- name: Configure bashrc
shell:
cmd: |
cat >> {{ item }}/.bashrc <<EOF
STUFF
EOF
with_items:
- "{{ nodepool_home }}"
- "{{ zuul_home }}"
when: '"STUFF" not in bashrc.stdout'
It fails as follows:
fatal: [ca-o3lscizuul]: FAILED! => {"failed": true, "msg": "The conditional check '\"STUFF\" not in bashrc.stdout' failed. The error was: error while evaluating conditional (\"STUFF\" not in bashrc.stdout): Unable to look up a name or access an attribute in template string ({% if \"STUFF\" not in bashrc.stdout %} True {% else %} False {% endif %}).\nMake sure your variable name does not contain invalid characters like '-': argument of type 'StrictUndefined' is not iterable\n\nThe error appears to have been in '/root/openstack-ci/infrastructure-setup/staging/zuul/create-user.yml': line 35, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Configure bashrc\n ^ here\n"}
I think, if I understand your requirement correctly, you can use the 'lineinfile' or 'blockinfile' modules and save yourself the hassle of testing for the existence of the content:
- name: Noddy example data
set_fact:
single_line: "STUFF"
multi_line: |
STUFF
STUFF
profile_dirs:
- "{{ nodepool_home }}"
- "{{ zuul_home }}"
- name: Ensure STUFF exists in file
lineinfile:
path: "{{ item }}/.bashrc"
line: "{{ single_line }}"
loop: "{{ profile_dirs }}"
- name: Ensure block of STUFF exists in file
blockinfile:
path: "{{ item }}/.bashrc"
block: "{{ multi_line }}"
loop: "{{ profile_dirs }}"
Both modules give a lot more control and you can find their docs here: lineinfile | blockinfile

Ansible Script Module Not Interpreting Variable

I am having an issue with the Ansible script module interpreting a with_items variable.
vsa_deploy_config/tasks/main.yml:
- name: Create VSA scripts for center
template:
src: vsa_deploy.ps1.j2
dest: "/opt/ansible/roles/vsa_deploy_config/files/{{ item.vsa_hostname }}.ps1"
when: target == "local"
with_items:
- "{{ vsa_center }}"
- name: Deploy VSAs on Center
script: "files/{{ item.vsa_hostname }}.ps1"
register: out
when: target == "win_center"
- debug: var=out
with_items:
- "{{ vsa_center }}"
vsa_deploy_config/vars/main.yml:
---
vsa_center:
- vcsa_hostname: 10.10.10.74
vcsa_username: administrator#vsphere.local
vcsa_password: password
vcsa_datacenter: DataCenter1
vsa_rdm_lun: 02000000006006bf1d58d25a1020d292f8fcfb22b3554353432d4d
vsa_hostname: sm01-ct01
vsa_mgmt_ip: 10.10.10.75
vsa_mgmt_netmask: 255.255.255.192
vsa_mgmt_gw: 10.10.10.65
vsa_mgmt_ns: 10.10.10.92
vsa_mgmt_pg: SC-MGMT
vsa_mgmt_moref: Network:network-13
vsa_iscsi_ip: 192.168.2.1
vsa_iscsi_netmask: 255.255.255.0
vsa_iscsi_pg: ISCSI
vsa_iscsi_moref: Network:network-22
vsa_mirror_ip: 192.168.5.1
vsa_mirror_netmask: 255.255.255.0
vsa_mirror_pg: Mirror
vsa_mirror_moref: Network:network-23
esxi_hostname: 10.10.10.72
esxi_datastore: DS-01
- vcsa_hostname: 10.10.10.74
vcsa_username: administrator#vsphere.local
vcsa_password: password
vcsa_datacenter: DataCenter1
vsa_rdm_lun: 02000000006006bf1d58d25dd0210bb356a78344e5554353432d4d
vsa_hostname: sm02-ct01
vsa_mgmt_ip: 10.10.10.76
vsa_mgmt_netmask: 255.255.255.192
vsa_mgmt_gw: 10.10.10.65
vsa_mgmt_ns: 10.10.10.92
vsa_mgmt_pg: SC-MGMT
vsa_mgmt_moref: Network:network-13
vsa_iscsi_ip: 192.168.2.2
vsa_iscsi_netmask: 255.255.255.0
vsa_iscsi_pg: ISCSI
vsa_iscsi_moref: Network:network-22
vsa_mirror_ip: 192.168.5.2
vsa_mirror_netmask: 255.255.255.0
vsa_mirror_pg: Mirror
vsa_mirror_moref: Network:network-23
esxi_hostname: 10.2.120.73
esxi_datastore: DS-02
When I run the playbook I get the following error:
TASK [vsa_deploy_config : Deploy VSAs on Center] *******************************************************************************
fatal: [auto-win1.lab.com]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'item' is undefined\n\nThe error appears to have been in '/opt/ansible/roles/vsa_deploy_config/tasks/main.yml': line 10, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Deploy VSAs on Center\n ^ here\n"}
to retry, use: --limit #/opt/ansible/powershell.retry
The first task using the template module interprets the item.vsa_hostname variable correctly, but the script module does not. Is the script module not capable of using with_items?
There is no with_items for your script task:
- name: Deploy VSAs on Center # -\
script: "files/{{ item.vsa_hostname }}.ps1" # \
register: out # / This is task1
when: target == "win_center" # -/
- debug: var=out # -\
with_items: # > This is task2
- "{{ vsa_center }}" # -/
I guess you'd want to move debug to the very bottom:
- name: Deploy VSAs on Center
script: "files/{{ item.vsa_hostname }}.ps1"
register: out
when: target == "win_center"
with_items: "{{ vsa_center }}"
- debug: var=out
P.S. also there is no need to feed unnecessary nested list into with_items.
just move the line - debug: var=out to the end of the file and it will work

Resources