attempting to dynamically reference jinja templates in ansible playbook - ansible

I am attempting to dynamically reference j2 templates pulled from artifactory.
The following code works:
- name: Confiure the sqoop templates
template: src="{{ item }}" dest="{{ local_root }}/{{ FEED_NAME }}/{{ FEED_SCHEMA }}/sqoopgen_jobs/{{ item|replace(\"/tmp/tardis/\", \"\") }}" mode=0744
with_fileglob:
- "/tmp/{{ FEED_NAME }}/*.j2"
The following does not:
- name: Confiure the sqoop templates
template: src="{{ item }}" dest="{{ local_root }}/{{ FEED_NAME }}/{{ FEED_SCHEMA }}/sqoopgen_jobs/{{ item|replace(\"/tmp/{{ FEED_NAME }}/\", \"\") }}" mode=0744
with_fileglob:
- "/tmp/{{ FEED_NAME }}/*.j2"
Failing with error:
failed: [host_ip] => (item=/tmp/tardis/forecast.j2) => {"changed": true, "failed": true, "invocation": {"module_args": {"backup": false, "content": null, "delimiter": null, "dest": "/path/tardis/tardis/sqoopgen_jobs//tmp/tardis/forecast.j2", "directory_mode": null, "follow": true, "force": true, "group": null, "mode": "0744", "original_basename": "forecast.j2", "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/home/jenkins/.ansible/tmp/ansible-tmp-1457719702.84-279730849155325/source", "validate": null}}, "item": "/tmp/tardis/forecast.j2", "msg": "Destination directory /path/tardis/tardis/sqoopgen_jobs//tmp/tardis does not exist"}
I'm reading this as a case of ansible not expanding a variable within a variable: such as here
What is the best approach to take in this situation? I want to keep the playbook as generic as possible.
ansible 2.0.1.0
thanks

You can not nest Jinja expressions like so:
{{ item|replace("/tmp/{{ FEED_NAME }}/", "") }}
What you want is to concatenate strings.
{{ item|replace("/tmp/" ~ FEED_NAME ~ "/", "") }}
See "Other Operators" in the Jinja2 docs:
~
Converts all operands into strings and concatenates them.
{{ "Hello " ~ name ~ "!" }} would return (assuming name is set to 'John') Hello John!.
- name: Configure the sqoop templates
template:
src: "{{ item }}"
dest: "{{ local_root }}/{{ FEED_NAME }}/{{ FEED_SCHEMA }}/sqoopgen_jobs/{{ item|replace('/tmp/' ~ FEED_NAME ~ '/', '') }}"
mode: 0744
with_fileglob:
- "/tmp/{{ FEED_NAME }}/*.j2"

Related

Output of Ansibble task

I am using command hcloud to create cloud server in Hetzner. I get an output like this:
changed: [localhost] => (item={'name': 'TEST-VARIABLES', 'server_type': 'cx11', 'os_image': 'ubuntu-20.04', 'server_labels': 'Name=test-server', 'server_location': 'hel1'}) => {
"ansible_loop_var": "item",
"changed": true,
"hcloud_server": {
"backup_window": "None",
"datacenter": "hel1-dc2",
"delete_protection": false,
"id": "19461514",
"image": "ubuntu-20.04",
"ipv4_address": "11.111.111.111",
"ipv6": "1a71:7f9:c011:0b09::/64",
"labels": {
"Name": "test-server"
},
"location": "hel1",
"name": "TEST-VARIABLES",
"placement_group": null,
"rebuild_protection": false,
"rescue_enabled": false,
"server_type": "cx11",
"status": "running"
},
"invocation": {
"module_args": {
"allow_deprecated_image": false,
"api_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"backups": null,
"datacenter": null,
"delete_protection": null,
"endpoint": "https://api.SERVER.cloud/v1",
"firewalls": null,
"force": false,
"force_upgrade": false,
"id": null,
"image": "ubuntu-20.04",
"labels": {
"Name": "test-server"
},
"location": "hel1",
"name": "TEST-VARIABLES",
"placement_group": null,
"rebuild_protection": null,
"rescue_mode": null,
"server_type": "cx11",
"ssh_keys": null,
"state": "present",
"upgrade_disk": false,
"user_data": null,
"volumes": null
}
},
"item": {
"name": "TEST-VARIABLES",
"os_image": "ubuntu-20.04",
"server_labels": "Name=test-server",
"server_location": "hel1",
"server_type": "cx11"
},
"root_password": "DFLDJFLDFDLFKJDLFKJ"
}
I try to get line with "ipv4_address": "11.111.111.111", and "root_password": "DFLDJFLDFDLFKJDLFKJ", but when I use task:
---
- name: Create a basic server
hcloud_server:
api_token: "{{ token }}"
name: "{{ item.name }}"
server_type: "{{ item.server_type }}"
image: "{{ item.os_image }}"
labels: "{{ item.server_labels }}"
location: "{{ item.server_location }}"
state: present
register: server_info
with_items: "{{ server }}"
- name: Here IP
debug:
var: server_info.root_password
I got error:
TASK [/etc/ansible/roles/CREATE-server : Here IP.] *********************************************************************************************************
ok: [localhost] => {
"server_info.root_password": "VARIABLE IS NOT DEFINED!"
}
Could you please help, how I can get IP line and password line, to use them in the next task (for example to send via email). Thank you!
you register the content of loop, so your result is a list (results):
- name: display
debug:
msg: "ip: {{ item.hcloud_server.ipv4_address }} has password: {{ item.root_password }}"
loop: "{{ server_info.results }}"
result: here you have just one server declared, so just one item in the list results
"msg": "ip: 11.111.111.111 has password: DFDFDFDFDFDFDFDF"
if you want to limit the output of record, you could add loop_control parameter to loop with argument label:
loop: "{{ server_info.results }}"
loop_control:
label: "{{ item.hcloud_server.ipv4_address }}"
you could put another comment if you want with label or even empty string:
loop: "{{ server_info.results }}"
loop_control:
label: "result"

Ansible unarchive module fails with Unexpected error when accessing exploded file: [Errno 2] No such file or directory:

My zip file contains a folder UAT-04022021_01 which has several files.
ls output on my local ansible server zip achieve file and its contents below:
-rwxrwxr-x 1 user1 user1 171910544 Feb 4 07:02 /web
/playbooks/automation/deployments/tmpfiles/04/UAT-04022021_01.zip
$ unzip -l "/web/playbooks/automation/deployments/tmpfiles/04/UAT-04022021_01.zip"
Archive: /web/playbooks/automation/deployments/tmpfiles/04/UAT-04022021_01.zip
Length Date Time Name
--------- ---------- ----- ----
0 02-04-2021 17:31 UAT-04022021_01/
144241065 02-04-2021 15:58 UAT-04022021_01/canv.ear
8342584 02-04-2021 16:00 UAT-04022021_01/canpizza.ear
10522141 02-04-2021 16:01 UAT-04022021_01/cantity.ear
8778258 02-04-2021 15:59 UAT-04022021_01/canipc.ear
--------- -------
171884048 5 files
[user1#linuxlocalhost deployments]$
Below is my code to extract the zip to all remote destination hosts
---
- name: "Play 1"
host: "{{ dest_host }}"
user: "{{ USER }}"
- name: Give better permissions to the zip file
tags: validate
file:
path: "{{ playbook_dir }}/tmpfiles/{{ Latest_Build_Number }}/{{ depfile | basename }}"
mode: 0775
delegate_to: localhost
run_once: true
- name: "Untar artifacts in CURRENT Folder"
tags: validate
unarchive:
src: "{{ playbook_dir }}/tmpfiles/{{ Latest_Build_Number }}/{{ depfile | basename }}"
dest: "/web/bea_apps/applications/{{ domain_home }}/{{ item }}/CURRENT/"
with_items: "{{ CLUSTER.split(',') }}"
Output:
#############################################\n\n')
The full traceback is:
File "/tmp/ansible_MCNgND/ansible_module_unarchive.py", line 873, in main
res_args['changed'] = module.set_fs_attributes_if_different(file_args, res_args['changed'], expand=False)
File "/tmp/ansible_MCNgND/ansible_modlib.zip/ansible/module_utils/basic.py", line 1473, in set_fs_attributes_if_different
file_args['path'], file_args['mode'], changed, diff, expand
File "/tmp/ansible_MCNgND/ansible_modlib.zip/ansible/module_utils/basic.py", line 1204, in set_mode_if_different
path_stat = os.lstat(b_path)
failed: [myserver2] (item=VL_BATCH) => {
"changed": false,
"dest": "/web/bea_apps/applications/mydom1/VL_BATCH/CURRENT/",
"gid": 64332,
"group": "wladmin",
"handler": "ZipArchive",
"invocation": {
"module_args": {
"attributes": null,
"backup": null,
"content": null,
"creates": null,
"delimiter": null,
"dest": "/web/bea_apps/applications/mydom1/VL_BATCH/CURRENT/",
"directory_mode": null,
"exclude": [],
"extra_opts": [],
"follow": false,
"force": null,
"group": null,
"keep_newer": false,
"list_files": false,
"mode": null,
"original_basename": "UAT-04022021_01.zip",
"owner": null,
"regexp": null,
"remote_src": false,
"selevel": null,
"serole": null,
"setype": null,
"seuser": null,
"src": "/home/wladmin/.ansible/tmp/ansible-tmp-1612477399.28-110140157282432/source",
"unsafe_writes": null,
"validate_certs": true
}
},
"item": "VL_BATCH",
"mode": "0755",
"msg": "Unexpected error when accessing exploded file: [Errno 2] No such file or directory: '/web/bea_apps/applications/mydom1/VL_BATCH/CURRENT/UAT-04022021_01/'",
"owner": "wladmin",
"size": 2,
"src": "/home/wladmin/.ansible/tmp/ansible-tmp-1612477399.28-110140157282432/source",
"state": "directory",
"uid": 600000008
}
/web/bea_apps/applications/mydom1/VL_BATCH/CURRENT/ is empty and has permission for the user to write to.
I checked on the destination and it does contain this folder /web/bea_apps/applications/mydom1/VL_BATCH/CURRENT/ but not /web/bea_apps/applications/mydom1/VL_BATCH/CURRENT/UAT-04022021_01/
It was supposed to create this folder UAT-04022021_01 if it unzips under the CURRENT folder.
After researching a bit more I tried this suggested fix of environment but it does not help either.
- name: "Untar artifacts in CURRENT Folder"
tags: validate
unarchive:
src: "{{ playbook_dir }}/tmpfiles/{{ Latest_Build_Number }}/{{ depfile | basename }}"
dest: "/web/bea_apps/applications/{{ domain_home }}/{{ item }}/CURRENT/"
with_items: "{{ CLUSTER.split(',') }}"
environment:
LANG: C
LC_ALL: C
LC_MESSAGES: C
My source is Linux while destination is Solaris.
Can you please suggest what could be wrong?

Undefined variable when passing dictionary to a submodule-role

I'm having a really weird problem on my ansible role. I send two dictionaries to a submodule like that:
import_role:
name: .submodules/monitoring-plugins
vars:
monitoring_plugins:
check_content:
command: "files/icinga/commands/check_content"
dest: "{{ icinga_server_plugin_directory }}"
group: "root"
owner: "root"
mode: "0755"
package: "curl"
src: "files/plugins/server/check_content"
check_http_response_time:
command: "files/icinga/commands/check_http_response_time"
dest: "{{ icinga_server_plugin_directory }}"
group: "root"
owner: "root"
mode: "0775"
src: "files/plugins/server/check_http_response_time"
check_https_response_time:
command: "files/icinga/commands/check_https_response_time"
dest: "{{ icinga_server_plugin_directory }}"
group: "root"
owner: "root"
mode: "0775"
src: "files/plugins/server/check_https_response_time"
check_port:
command: "files/icinga/commands/check_port"
dest: "{{ icinga_server_plugin_directory }}"
group: "root"
owner: "root"
mode: "0775"
src: "files/plugins/server/check_port"
check_ssl_cert:
command: "files/icinga/commands/check_ssl_cert"
dest: "{{ icinga_server_plugin_directory }}"
group: "root"
owner: "root"
mode: "0775"
src: "files/plugins/server/check_ssl_cert"
custom_services:
content:
service-preamble: 'apply Service "content"'
configuration: |
{{ common_service_header }}
assign where host.vars.ansible.system == "Linux"
definition:
"{{ lookup('template', 'config/services/content.conf') }}"
http_response_time:
service-preamble: 'apply Service "http_response_time"'
configuration: |
{{ common_service_header }}
assign where host.vars.ansible.system == "Linux"
definition:
"{{ lookup('template', 'config/services/http_response_time.conf') }}"
https_response_time:
service-preamble: 'apply Service "https-response-time"'
configuration: |
{{ common_service_header }}
assign where host.vars.ansible.system == "Linux"
definition:
"{{ lookup('template', 'config/services/https_response_time.conf') }}"
http_port:
service-preamble: 'apply Service "http-port"'
configuration: |
{{ common_service_header }}
assign where host.vars.ansible.system == "Linux"
definition:
"{{ lookup('template', 'config/services/http_port.conf') }}"
https_port:
service-preamble: 'apply Service "https-port"'
configuration: |
{{ common_service_header }}
assign where host.vars.ansible.system == "Linux"
definition:
"{{ lookup('template', 'config/services/https_port.conf') }}"
ssl_cert:
service-preamble: 'apply Service "ssl-cert"'
configuration: |
{{ common_service_header }}
assign where host.vars.ansible.system == "Linux"
definition:
"{{ lookup('template', 'config/services/ssl_cert.conf') }}"
on the submodule I created two debug tasks:
- debug:
var: monitoring_plugins
- debug:
var: custom_services
output:
ok: [server.test] => {
"monitoring_plugins": {
"check_content": {
"command": "files/icinga/commands/check_content",
"dest": "/usr/lib/nagios/plugins",
"group": "root",
"mode": "0755",
"owner": "root",
"package": "curl",
"src": "files/plugins/server/check_content"
},
"check_http_response_time": {
"command": "files/icinga/commands/check_http_response_time",
"dest": "/usr/lib/nagios/plugins",
"group": "root",
"mode": "0775",
"owner": "root",
"src": "files/plugins/server/check_http_response_time"
},
"check_https_response_time": {
"command": "files/icinga/commands/check_https_response_time",
"dest": "/usr/lib/nagios/plugins",
"group": "root",
"mode": "0775",
"owner": "root",
"src": "files/plugins/server/check_https_response_time"
},
"check_port": {
"command": "files/icinga/commands/check_port",
"dest": "/usr/lib/nagios/plugins",
"group": "root",
"mode": "0775",
"owner": "root",
"src": "files/plugins/server/check_port"
},
"check_ssl_cert": {
"command": "files/icinga/commands/check_ssl_cert",
"dest": "/usr/lib/nagios/plugins",
"group": "root",
"mode": "0775",
"owner": "root",
"src": "files/plugins/server/check_ssl_cert"
}
}
}
TASK [.submodules/monitoring-plugins : debug] ************************************
ok: [server.test] => {
"custom_services": "VARIABLE IS NOT DEFINED!"
}
Does anybody have any idea what could possibly go wrong? I sent both of them from the very same task but yet one of them looks just fine and the other don't. I even tried to remove the submodule and add it again but it still doesn't work. Any other suggestions would be very appreciated!
The background
A var containing other vars expansion being undefined cannot be interpreted and is undefined.
$ ansible localhost -e undef="{{ i_dont_exist }}" -m debug -a var=undef
localhost | SUCCESS => {
"undef": "VARIABLE IS NOT DEFINED!"
}
To see what is going wrong, you need to actually expand the var at debug time
$ ansible localhost -e undef="{{ i_dont_exist }}" -m debug -a msg="{{ undef }}"
localhost | FAILED! => {
"msg": "The task includes an option with an undefined variable. The error was: 'i_dont_exist' is undefined"
}
Your specific case
The root cause is that common_service_header is undefined. Since it is used inside your variable definition for the role include, ansible tries to interpret it, fails, and the var stays undefined.
You will actually see the same result for monitoring_plugins if you undefine icinga_server_plugin_directory.
To have some more info on what actually happens, the "trick" is to force ansible to interpret the variable in your debug. There are two possible ways to do this:
- name: Use a simple jinja2 expansion
debug:
msg: "{{ custom_services }}"
- name: Use a the 'vars' lookup
debug:
msg: "{{ lookup('vars', 'custom_services') }}"
Which then gives a more meaningful info (reformated for legibility....):
TASK [.submodules/monitoring-plugins : debug] *****************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an
undefined variable. The error was: 'common_service_header' is undefined\n\n
The error appears to be in
'/some/path/.submodules/monitoring-plugins/tasks/main.yml': line 4, column 3,
but may\nbe elsewhere in the file depending on the exact syntax problem.\n\n
The offending line appears to be:\n\n
msg: \"{{ lookup('vars', 'monitoring_plugins') }}\"\n- debug:\n ^ here\n"}

Error when running this ansible play "ERROR! 'unicode object' hasno attribute 'comment'

Here is my playbook
- name: Add multiple users
user:
name: "{{ item[0].name }}"
comment: "{{ item[0].comment }}"
uid: "{{ item[0].uid }}"
groups: "{{ item[0].groups}}"
shell: /bin/bash
with_nested:
- "{{ name }}"
- "{{ comment }}"
- "{{ uid }}"
- "{{ groups }}"
Here is my vars file
---
name:
- test1
- test2
comment:
- "comment1"
- "comment2"
uid:
- 150
- 151
groups: "sudo, admin"
I'm not sure what is causing this, any ideas? I believe I may need to use with subelement instead of with nested? Am I on the right track there?
UPDATE:
Changed my code but am now experiencing the following. Updated code and error message
- name: Add new group if it doesn't exist already
group:
name: "{{ group }}"
when: group is defined
- name: Add multiple users
user:
name: "{{ item.0 }}"
comment: "{{item.1 }}"
uid: "{{ item.2 }}"
group: "{{ group }}"
groups: "{{ groups }}"
append: yes
with_together:
- "{{ name }}"
- "{{ comment }}"
- "{{ uid }}"
- "{{ group }}"
And variable file:
name:
- test1
- test2
comment:
- "comment1"
- "comment2"
uid:
- 150
- 151
group: sudo
groups:
- admin
- test
However, now I am receiving this error.
failed: [127.0.0.1] => (item=[u'test1', u'comment1', 150, u'sudo']) => {"failed": true, "invocation": {"module_args": {"append": true, "comment": "comment1", "createhome": true, "expires": null, "force": false, "generate_ssh_key": null, "group": "sudo", "groups": "{'ungrouped': ['127.0.0.1'], 'all': ['127.0.0.1']}", "home": null, "login_class": null, "move_home": false, "name": "test1", "non_unique": false, "password": null, "remove": false, "shell": null, "skeleton": null, "ssh_key_bits": "2048", "ssh_key_comment": "ansible-generated on ubuntu-512mb-sfo1-01", "ssh_key_file": null, "ssh_key_passphrase": null, "ssh_key_type": "rsa", "state": "present", "system": false, "uid": "150", "update_password": "always"}, "module_name": "user"}, "item": ["test1", "comment1", 150, "sudo"], "msg": "Group 'all': ['127.0.0.1']} does not exist"}
failed: [127.0.0.1] => (item=[u'test2', u'comment2', 151, None]) => {"failed": true, "invocation": {"module_args": {"append": true, "comment": "comment2", "createhome": true, "expires": null, "force": false, "generate_ssh_key": null, "group": "sudo", "groups": "{'ungrouped': ['127.0.0.1'], 'all': ['127.0.0.1']}", "home": null, "login_class": null, "move_home": false, "name": "test2", "non_unique": false, "password": null, "remove": false, "shell": null, "skeleton": null, "ssh_key_bits": "2048", "ssh_key_comment": "ansible-generated on ubuntu-512mb-sfo1-01", "ssh_key_file": null, "ssh_key_passphrase": null, "ssh_key_type": "rsa", "state": "present", "system": false, "uid": "151", "update_password": "always"}, "module_name": "user"}, "item": ["test2", "comment2", 151, null], "msg": "Group 'all': ['127.0.0.1']} does not exist"}
The problem is conflicting variable names. groups is a reserved variable and holds the groups from the inventory. And all is a automatically generated group which holds all the hosts of your inventory.
From the docs:
Even if you didn’t define them yourself, Ansible provides a few variables for you automatically. The most important of these are hostvars, group_names, and groups. Users should not use these names themselves as they are reserved. environment is also reserved.
and
groups is a list of all the groups (and hosts) in the inventory. This can be used to enumerate all hosts within a group.
Simply rename your variable and it should work. In general it's a good idea to prefix all variables of a role with the role name. This gets more important if you use 3rd party roles, e.g. from Ansible Galaxy, just to avoid conflicts. So instead of groups you could use myrole_groups and can be quite sure there never will be conflicts.

Ansible 1.8.2 nested loop, trying to output item[1] in item[0] string, but its being escaped

Here is my example, not sure if this can be done, but I would like to output the value from the sites array, specifically item[1].site, in item[0].dest, but it looks like it is escaping {{ item[1].site }}, to {# item[1].site #}. Is there a way to prevent it from escaping the string?
- name: Put files into docker directory
template: src={{ item[0].src }} dest={{ item[0].src }}
with_nested:
- [
{ src: 'Dockerfile.j2', dest: "/opt/docker-apache2-fpm/{{ item[1].site }}/Dockerfile" },
]
- sites
Here is the output:
failed: [192.168.200.87] => (item=[{'dest': u'/opt/docker-apache2-fpm/{# item[1].site #}/Dockerfile', 'src': 'Dockerfile.j2'}, {'site': 'admin.mysite.com', 'user': 'mysite', 'uid': 11004}]) => {"failed": true, "item": [{"dest": "/opt/docker-apache2-fpm/{# item[1].site #}/Dockerfile", "src": "Dockerfile.j2"}, {"site": "admin.mysite.com", "uid": 11004, "user": "mysite"}]}
msg: Destination directory does not exist
That's not working, because Ansible is not re-evaluating the elements. But you can work with the replace filter to archive your goal.
- name: Put files into docker directory
template: src={{ item[0].src }} dest={{ item[0].dest | replace("%site%", item[1].site) }}
with_nested:
- [
{ src: 'Dockerfile.j2', dest: "/opt/docker-apache2-fpm/%site%/Dockerfile" },
]
- sites
In that example I assumed you meant item[0].dest and not item[0].src in your dest assignment.

Resources