I'm seeing an error due to extra '\' during the ansible run, the task is failing with syntax error due to that.
Here is my .yml file for that particular task
- name: Install HPC Head Node (Legacy)
win_package:
path: '{{ hpc_install_path }}\setup.exe'
product_id: '{{ lookup("vars", "hpc_pack_" + hpc_version + "_product_id") }}'
arguments: >
-Quiet
-Unattend
-HeadNode
-HeadNodeList:{{ groups["head_nodes"] | map("extract", hostvars, "node_name") | join(",") }}
-ClusterName:{{ environment_name }}-HPC
-SSLPfxFilePath:{{ sft_path }}\hpc.pfx
-SSLPfxFilePassword:{{ ansible_password }}
-MgmtDbConStr:"Data Source={{ sql_server }};Initial Catalog=HPCManagement;Integrated Security=True;"
-SchdDbConStr:"Data Source={{ sql_server }};Initial Catalog=HPCScheduler;Integrated Security=True;"
-RptDbConStr:"Data Source={{ sql_server }};Initial Catalog=HPCReporting;Integrated Security=True;"
-DiagDbConStr:"Data Source={{ sql_server }};Initial Catalog=HPCDiagnostics;Integrated Security=True;"
-MonDbConStr:"Data Source={{ sql_server }};Initial Catalog=HPCMonitoring;Integrated Security=True;"
-RuntimeShare:\\{{ node_name }}\HPCNetShares\HPCRunTimeData
-SpoolDirShare:\\{{ node_name }}\HPCNetShares\SpoolOutput
-DiagnosticsShare:\\{{ node_name }}\HPCNetShares\DiagnosticsTestOutput
-ServiceRegistrationShare:\\{{ node_name }}\HPCNetShares\SvcRegConfigurationFiles
-InstallShare:\\{{ node_name }}\HPCNetShares\Installation
During the ansible run it adds an extra '\' before the Data and after True. Can someone suggest how can I avoid it?
unexpected rc from ‘C:\\Software\\HPCPack2016\\setup.exe -Quiet -Unattend -HeadNode -HeadNodeList:HPC-*********,HPC-********* -ClusterName:********-HPC -SSLPfxFilePath:c:\\Software\\****.pfx -SSLPfxFilePassword:************ -MgmtDbConStr:\“Data Source=****************.us-east-1.rds.amazonaws.com;Initial Catalog=HPCManagement;Integrated Security=True;\” -SchdDbConStr:\“Data Source=***************.us-east-1.rds.amazonaws.com;Initial Catalog=HPCScheduler;Integrated Security=True;\” -RptDbConStr:\“Data Source=*******************.us-east-1.rds.amazonaws.com;Initial Catalog=HPCReporting;Integrated Security=True;\” -DiagDbConStr:\“Data Source=******************.us-east-1.rds.amazonaws.com;Initial Catalog=HPCDiagnostics;Integrated Security=True;\” -MonDbConStr:\“Data Source=********************.us-east-1.rds.amazonaws.com;Initial Catalog=HPCMonitoring;Integrated Security=True;\” -RuntimeShare:\\\\HPC-********\\HPCNetShares\\HPCRunTimeData -SpoolDirShare:\\\\HPC-***********\\HPCNetShares\\SpoolOutput -DiagnosticsShare:\\\\HPC-**********\\HPCNetShares\\DiagnosticsTestOutput -ServiceRegistrationShare:\\\\HPC-***********\\HPCNetShares\\SvcRegConfigurationFiles -InstallShare:\\\\HPC-***********\\HPCNetShares\\Installation\n’: see rc, stdout, and stderr for more details”,
Related
I want to deploy an application via ansible that has a configurable number of application nodes running via a systemd service. I have a role that manages these services for me but expects the name of the services along with e.g. the priority.
In my playbook the user defines something like this
application_nodes:
- name: "node1"
api_port: 3900
- name: "node2"
api_port: 3910
Therefore I need to append the list that this the systemd role manages. I tried the following
service_list: "{% for node in application_nodes %}{'name': 'application_{{ node.name }}.service', 'priority': 1000}{% endfor %}"
devture_systemd_service_manager_services_list_auto: |
{{
([{{ service_list }}])
}}
This results in the following error
FAILED! => {"msg": "An unhandled exception occurred while templating '{{\n ([{{ service_list }}])\n}}\n'. Error was a <class 'ansible.errors.AnsibleError'>, original message: template error while templating string: expected token ':', got '}'. String: {{\n ([{{ service_list }}])\n}}\n. expected token ':', got '}'"}
I cannot figure out what I did wrong here.
I the end it should work like this (except automatically for every node)
devture_systemd_service_manager_services_list_auto: |
{{
([{'name': 'application_node1.service', 'priority': 1000}])
+
([{'name': 'application_node2.service', 'priority': 1000}])
}}
I also considered something like this
- name: Append services list by nodes
set_fact:
devture_systemd_service_manager_services_list_auto: "{{ devture_systemd_service_manager_services_list_auto + [{'name': 'application_{{ item.name }}.service', 'priority': 1000, 'groups': ['applicationname']}] }}"
loop: "{{ application_nodes }}"
- name: "Print services"
debug:
msg: "services {{ devture_systemd_service_manager_services_list_auto }}"
Where the error is similar to this issue but the solution does not work for me as I want to access a specific key of the dictionary
TASK [custom/base : Print services B] ********************************************************************************************************************************************************************************************************************
fatal: [mydomain]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: [{'name': 'application_node2.service', 'priority': 1000, 'groups': ['applicationname']}, {'name': 'application_{{ item.name }}.service', 'priority': 1000, 'groups': ['applicationname']}]: 'item' is undefined. 'item' is undefined. [{'name': 'application_node2.service', 'priority': 1000, 'groups': ['applicationname']}, {'name': 'application_{{ item.name }}.service', 'priority': 1000, 'groups': ['applicationname']}]: 'item' is undefined. 'item' is undefined\n\nThe error appears to be in '/.../roles/custom/base/tasks/add_services.yml': line 11, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: \"Print services B\"\n ^ here\n"}
To start with, the var definition in the role your are using is overly complicated for something as simple as a list of dicts. This is a source of confusion. So to be clear the following expression:
devture_systemd_service_manager_services_list_auto: |
{{
([{'name': 'application_node1.service', 'priority': 1000}])
+
([{'name': 'application_node2.service', 'priority': 1000}])
}}
is actually the exact equivalent of:
devture_systemd_service_manager_services_list_auto:
- name: application_node1.service
priority: 1000
- name: application_node2.service
priority: 1000
The second source of confusion IMO is that you used the term append (which is in Ansible the fact to add more elements to a list) rather than combine (i.e. adding/updating keys in one dict from an other dict).
Those precisions being made, we can fulfill your requirement by:
Extracting the name attribute from each element in the original list
Transform that name so that it matches your expected service name
Append a dict containing that name and the expected priority in relevant keys to a result list
Note that having to transform the name makes it difficult to avoid looping inside a template for the var definition (which we generally tend to avoid in Ansible). But this can still be done in one single expression as demonstrated in the following test playbook:
---
- hosts: localhost
gather_facts: false
vars:
application_nodes:
- name: "node1"
api_port: 3900
- name: "node2"
api_port: 3910
devture_systemd_service_manager_services_list_auto: >-
{%- set result = [] -%}
{%- for node in application_nodes -%}
{%- set service_name = node.name | regex_replace('^(.*)$', 'application_\\1.service') -%}
{{ result.append({'name': service_name, 'priority': 1000}) }}
{%- endfor -%}
{{ result }}
tasks:
- name: Show our list
ansible.builtin.debug:
var: devture_systemd_service_manager_services_list_auto
which gives (relevant task output only)
TASK [Show our list] *********************************************************
ok: [localhost] => {
"devture_systemd_service_manager_services_list_auto": [
{
"name": "application_node1.service",
"priority": 1000
},
{
"name": "application_node2.service",
"priority": 1000
}
]
}
You almost had it in your first try. If you add a few things to your service_list it ends up being a list of dicts -- which is ultimately what you want to append to devture_systemd_service_manager_services_list_auto.
- hosts: all
vars:
application_nodes:
- name: "node1"
api_port: 3900
- name: "node2"
api_port: 3910
tasks:
- set_fact:
# The square brackets before and after the loop make this an actual
# list.
# To make sure that the list items are separated from each other, we
# have to add a comma after each item. Luckily Ansible / Python isn't
# too picky about a comma without a final item. ;-)
service_list: >
[
{% for node in application_nodes %}
{'name': 'application_{{ node.name }}.service', 'priority': 1000},
{% endfor %}"
]
- set_fact:
devture_systemd_service_manager_services_list_auto: >
{{ devture_systemd_service_manager_services_list_auto|default([]) +
service_list }}
- debug:
var: devture_systemd_service_manager_services_list_auto
I am trying to insert a string in docker.conf file and it is failing on ansible may be due to my syntax. Works when I do it manually.
vars:
memlock_value: "16777216:16777216"
docker_options_file: "/etc/systemd/system/docker.service.d/docker-options.conf"
The task is
- name: "Set in flag for ulimit in docker conf"
shell: |
- "sed '/^Environment=/ s/\"$/ --default-ulimit memlock={{ memlock_value }}\"/' {{ docker_options_file }} -i"
output:
task path: /ansible-managed/jenkins-slave/slave1/workspace/teamd/run_ansible_playbook/k8s/baremetal/roles/team-node-config/tasks/main.yml
23:27:53 Tuesday 07 December 2021 07:27:53 +0000 (0:00:11.047) 0:00:19.487 ******
fatal: [node1]: FAILED! => {"changed": true, "cmd": "- \"sed '/^Environment=/ s/\\\"$/ --default-ulimit memlock=16777216:16777216\\\"/' /etc/systemd/system/docker.service.d/docker-options.conf -i\"\n \n", "delta": "0:00:00.004384", "end": "2021-12-07 07:27:53.586179", "msg": "non-zero return code", "rc": 2, "start": "2021-12-07 07:27:53.581795", "stderr": "/bin/sh: 0: Illegal option - ", "stderr_lines": ["/bin/sh: 0: Illegal option - "], "stdout": "", "stdout_lines": []}
The manual command works:
sed '/^Environment=/ s/"$/ --default-ulimit memlock=16777216:16777216"/' /etc/systemd/system/docker.service.d/docker-options.conf -i
I also tried lineinefile but it replaces the whole line instead of just inserting at end
- name: "Set in flag for ulimit in docker conf"
lineinfile:
path: "{{ docker_options_file_dummy }}"
regexp: '^Environment='
insertafter: '^Environment=.+$'
line: "--default-ulimit memlock={{ memlock_value }}"
ignore_errors: yes
output
=> {"backup": "", "changed": true, "msg": "line replaced"}
[Service]
--default-ulimit memlock=16777216:16777216
Finally was able to figure out here it is but am still trying to figure out how to make it idempotent. multiple runs is keeping on adding the content at end of line.
- name: "add ulimit in tests file to end of line in {{ dummy_file }}"
lineinfile:
dest: "{{ dummy_file }}"
state: present
regexp: '^(Environment="[^"]*)'
backrefs: yes
line: '\1 {{ ulimit_memlock_flag }}"'
file before task
[Service]
Environment="DOCKER_OPTS= --data-root=/var/lib/docker --log-opt max-size=50m --log-opt max-file=5 --iptables=false"
file after task
[Service]
Environment="DOCKER_OPTS= --data-root=/var/lib/docker --log-opt max-size=50m --log-opt max-file=5 --iptables=false --default-ulimit memlock=16777216:16777216"
Finally was able to figure out here it is with idempotency. multiple runs does not add the content at end of line again.
- name: "with idempotency add ulimit in tests file to end of line in {{ dummy_file }}"
lineinfile:
dest: "{{ dummy_file }}"
state: present
#regexp: '^(DOCKER_OPTS="[^"]*)'
regexp: (.*"DOCKER_OPTS=.*iptables=false)
backrefs: yes
line: '\1 {{ ulimit_memlock_flag }}"'
file before task
[Service]
Environment="DOCKER_OPTS= --data-root=/var/lib/docker --log-opt max-size=50m --log-opt max-file=5 --iptables=false"
file after task run 1
[Service]
Environment="DOCKER_OPTS= --data-root=/var/lib/docker --log-opt max-size=50m --log-opt max-file=5 --iptables=false --default-ulimit memlock=16777216:16777216"
file after task run 2
[Service]
Environment="DOCKER_OPTS= --data-root=/var/lib/docker --log-opt max-size=50m --log-opt max-file=5 --iptables=false --default-ulimit memlock=16777216:16777216"
The literal text inside | block is not an array. Also, remove global " quotes - a command is not a single command but is split on spaces, so spaces have to be there.
Try:
shell: |
sed "'/^Environment=/ s/\"$/ --default-ulimit memlock={{ memlock_value }}\"/'" "{{ docker_options_file }}" -i
It's odd that -i option is on the end. I think It would be nicer to:
shell: <
sed -i -e
"'/^Environment=/ s/\"$/ --default-ulimit memlock={{ memlock_value }}\"/'
"{{ docker_options_file }}"
You should prefer command over shell
command:
argv:
- sed
- -i
- -e
- '/^Environment=/ s/"$/ --default-ulimit memlock={{ memlock_value }}"/'
- "{{ docker_options_file }}"
And, you should prefer lininfile over sed. Something along:
lineinfile:
regexp: '^(Environment=[^"]*)"'
backrefs: yes
line: '\1 --default-ulimit memlock={{ memlock_value }}"'
path: "{{ docker_options_file }}"
I have written jinja2 template in ansible. What i am trying to achieve is that if the service_name is not mentioned and if service_name already exists on the remote machine, ansible should not change the service_name with default name mentioned in the template. However, when the service_name is not defined, ansible replaces service name with "abc" on remote machine even service_name exists. Any help would be appreciated.
active={{ active_status}}
instrument={{ instrument_status }}
{% if service_name is defined %}
service_name={{ service_name }}
{% else %}
service_name=abc
{% endif %}
Thanks
Following my above comment, here is a possible example implementation to meet your requirements. test_template.j2 is the exact copy of your current template. You can pass the service name as an extra variable to test (-e service_name=my_service)
Basically, if service_name is not defined, we:
Check if the remote file already exists and slurp its content into a var
Look for the relevant line in the file. Note: the regex_replace('None', '') is here to make sure we get an empty string if previous search/matches did not return anything.
Set the service name only if something relevant was found in the prior tasks
Once this check/setting is done correctly, you simply have to copy your template, what ever the case is.
---
- name: Conditional writing of template
hosts: localhost
gather_facts: false
vars:
my_dest: /tmp/test_file.txt
active_status: some active value
instrument_status: some instrument value
tasks:
- name: Try to read service name from existing file when it is not defined
when: service_name is not defined
block:
- name: Check if file exists
stat:
path: "{{ my_dest }}"
register: my_file
- name: Try to read target file if exists
slurp:
src: "{{ my_dest }}"
when: my_file.stat.exists
register: my_file_slurp
- name: Look for service name if there
set_fact:
looked_service: >-
{{
my_file_slurp.content
| b64decode
| regex_search('^service_name=.*$', multiline=true)
| regex_replace('^service_name=(.*)$', '\1')
| regex_replace('None', '')
}}
when: my_file.stat.exists
- name: Update service name if found
set_fact:
service_name: "{{ looked_service }}"
when: looked_service | default('') | length > 0
- name: Copy template file to destination
template:
src: test_template.j2
dest: "{{ my_dest }}"
I want my filedet.yaml to look like
10.9.75.78: /app/tmp/tmp.log, /vars/tmp/test.out
10.9.55.74: /app/tmp/tmp1.log, /vars/tmp/admin.out
The below works fine and logs the data correctly but when i add ': ' the syntax breaks and I get error
- name: Logging the deployment's file details to a Ansible variable file
local_action: lineinfile line={{ inventory_hostname }}': '{{ vars['fdetails_' + Layer].results|map(attribute='stdout')|list }} path={{ playbook_dir }}/vars/filedets.yaml
Output Error:
The offending line appears to be:
local_action: lineinfile line={{ inventory_hostname }}': '{{ > vars['fdetails_' + Layer].results|map(attribute='stdout')|list > }} path={{ playbook_dir }}/vars/filedets.yaml
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
I also tried this code but it too fails with syntax error:
line="{{ inventory_hostname }}': '{{ vars['fdetails_' + Layer].results|map(attribute='stdout')|list }}" path="{{ playbook_dir }}/vars/filedets.yaml"
Can you please suggest how can I inject the colons and space ': ' between the the variable in line ?
Just wrap the strings you want to insert between the variables in {{ }}
line="{{ inventory_hostname }}{{': '}}{{ vars['fdetails_' + Layer].results|map(attribute='stdout')|list }}" path="{{ playbook_dir }}/vars/filedets.yaml"
If the : colon is a problem you can mask it by using:
line="{{ inventory_hostname }}{{'%c '%58}}{{ vars['fdetails_' + Layer].results|map(attribute='stdout')|list }}" path="{{ playbook_dir }}/vars/filedets.yaml"
58 is the ASCII Code of :.
I am trying to create ansible role to generate kubernetes deployment with ansible. I want all customization to be stored in var files and be loaded automatically during group_vars and host_vars loading. I end up with following nested structure of files:
group_vars/my_group.yml:
applications:
application_foo: "{{ lookup('template', inventory_dir + '/applications/application_foo.yml') | from_yaml }}"
applications/application_foo.yml:
deployments:
application_foo:
deployment:
{{ lookup('template', inventory_dir + '/deployment_vars/deployment_foo.yml') | to_nice_yaml | indent(6)}}
replicas: 1
deployment_vars/deployment_foo.yml:
containers:
conteiner_bar:
container:
{{ lookup('template',inventory_dir + '/container_vars/container_bar.yml') | to_nice_yaml | indent(6) }}
container_baz:
container:
{{ lookup('template',inventory_dir + '/container_vars/container_baz.yml') | to_nice_yaml | indent(6) }}
container_vars/container_bar.yml:
container_vars/container_baz.yml:
image_url: "https://example.com/image_bar"
cpu_requests: "1"
memory_requests: "512Mi"
cpu_limit: "2"
memory_limit: "1024Mi"
readinessProbe:
exec:
command:
- "sh"
- "-c"
- "true"
playbooks/test-playbook.yml:
- hosts: my_group
gather_facts: False
tasks:
- debug:
msg: "{{ applications['application_foo']['deployments']['application_foo']['deployment']['containers']['container_bar']['cpu_requests'] }}"
delegate_to: localhost
When I run playbook it complains that deployment var is of str type and therefore can't have children objects:
fatal: [localhost]: FAILED! => {
"msg": "The task includes an option with an undefined variable. The error was: 'str object' has no attribute 'containers'\n\nThe error appears to have been in '/tmp/so_question/playbooks/test-playbook.yml': line 4, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n - debug:\n ^ here\n"
}
I want layers to be reusable for cases where container or deployment are required by another application so don't want to put everything in one file.
Any suggestions how to fix this issue or how to achieve same nested loading in other way?
UPD: when I cut string in playbook to end on deployment (like this msg: "{{ applications['application_foo']['deployments']['application_foo']['deployment'] }}") it gives following line which after newline substitution does not look like valid yaml:
containers:
conteiner_bar:
container:
\"image_url: \\\"https://example.com/image_bar\\\"\
cpu_requests: \\\"1\\\"\
memory_requests:\\
\\ \\\"512Mi\\\"\
cpu_limit: \\\"2\\\"\
memory_limit: \\\"1024Mi\\\"\
readinessProbe:\
exec:\
\\
\\ command:\
- \\\"sh\\\"\
- \\\"-c\\\"\
- \\\"true\\\"\
\"
container_baz:
container:
\"image_url: \\\"https://example.com/image_bar\\\"\
cpu_requests: \\\"1\\\"\
memory_requests:\\
\\ \\\"512Mi\\\"\
cpu_limit: \\\"2\\\"\
memory_limit: \\\"1024Mi\\\"\
readinessProbe:\
exec:\
\\
\\ command:\
- \\\"sh\\\"\
- \\\"-c\\\"\
- \\\"true\\\"\
\"
I wondering why indent filter does not indents all lines. Am I using it correctly?
Ended up with following workaround: Moved all files containing defaults to group_vars/all to make them loaded automatically and used vars lookup for nesting:
$ find group_vars/
group_vars/
group_vars/all
group_vars/all/10_container_bar.yml
group_vars/all/20_deployment_foo.yml
group_vars/all/30_application_foo.yml
group_vars/all/10_container_baz.yml
group_vars/my_group.yml
group_vars/my_group.yml:
applications:
application_foo: "{{ lookup('vars', 'defaults_application_application_foo') }}"
group_vars/all/30_application_foo.yml:
defaults_application_application_foo:
deployments:
application_foo:
deployment: "{{ lookup('vars', 'defaults_deployment_deployment_foo') }}"
replicas: 1
And so on...
Still not clear why template thing didn't work. Leaving this question open.