Ansible 2.9.2
Tower 3.6.1
There are two Bitbucket repositories viz. bitbucketconfigurator and sonarconfigurator which contain playbooks to configure Bitbucket and Sonarqube projects, respectively. Here, showing just the bitbucketConfigurator:
In the above project, there is a group_vars/all yaml file as follows(note the entry bitbucket_token: "{{ PAT_admin_admin }}" ):
# Defaults used for branch access and restrictions
# Global vars used for rest api
bitbucket_base_url: 'https://git.net'
bitbucket_rest_base_endpoint: 'rest/api/1.0/projects'
bitbucket_rest_url: "{{ bitbucket_base_url }}/{{ bitbucket_rest_base_endpoint }}"
bitbucket_branch_rest_endpoint: "rest/branch-permissions/2.0/projects"
bitbucket_branching_url: "{{ bitbucket_base_url }}/rest/branch-utils/1.0/projects"
sonar_for_bitbucket_base_endpoint: 'rest/sonar4stash/1.0/projects'
bitbucket_token: "{{ PAT_admin_admin }}"
bitbucket_rest_api_latest: 'rest/api/latest'
bitbucket_project_name_lc: "{{ bitbucket.project.name | lower }}"
bitbucket_repository_name_lc: "{{ bitbucket.project.repository.name | lower }}"
The list of job templates, an example is shown further, below:
As an example, let's consider the bb_configure job template I am using 'Extra Variables' to pass a yml file/dictionary e.g:
Here is the list of all the workflow templates, I will summarize their usage subsequently:
I tested the bitbucketconfigurator playbooks in a single workflow viz. wt-bitbucket. It runs successfully. Note that it uses no-arg versions of the individual playbooks i.e the config yaml file is passed to the wt-bitbucket and NOT the individual playbooks.
In the above workflow template, the extra vars are used:
Now, I am using the wk_onboarding_pipeline as a 'master' workflow template which invokes the no-arg versions of the workflow templates:
This 'master' template now has the config yaml file, also, note the entry PAT_admin_admin::
Issue:
When I run this 'master' template, the first playbook of the first workflow template fails as it is unable to access the 'PAT_admin_admin' passed in the extra vars.
Am I making a mistake in the invocation or have I incorrectly configured the workflow templates within a workflow template?
Related
I have a file in my Ansible role vars/sonarqube.yml with content
---
lvm_roles:
sonarqube:
size: '10g'
path: '{{ sonar_home }}'
And a file group_vars/all/lvm.yml with content
lvm_roles:
sonarqube:
size: '20g'
In ansible.cfg I have a line
hash_behaviour = merge
Without merge the resulting fact will be
lvm_roles:
sonarqube:
size: '20g'
With other words I loose the path var.
With merge the result is
lvm_roles:
sonarqube:
size: '10g'
path: '/opt/sonarqube'
The result I want and expected is however
lvm_roles:
sonarqube:
size: '20g'
path: '/opt/sonarqube'
So the desired behavior is that
Ansible merges vars
config in group_vars takes precedence to config in my role.
Can I configure this behavior in Ansible? How?
Precedence is not configurable in Ansible, you cannot give configuration in group_vars a higher precedence than configuration in the vars directory in a role.
What you want are conditional default vars which Ansible does not support . As shown for example here Set Ansible role defaults conditionally and here use conditionals in vars ansible.
This is an area where Ansible is definitely lacking as an IaC tool. It is for example a very commonly used feature with Chef. Snippet from attributes/default.rb from the Chef Apache cookbook demonstrates this common pattern
....
case node['platform_family']
when 'rhel', 'fedora', 'amazon'
if node['platform'] == 'amazon'
default['apache']['package'] = 'httpd24'
default['apache']['devel_package'] = 'httpd24-devel'
else
default['apache']['package'] = 'httpd'
default['apache']['devel_package'] = 'httpd-devel'
end
default['apache']['service_name'] = 'httpd'
default['apache']['perl_pkg'] = 'perl'
default['apache']['apachectl'] = '/usr/sbin/apachectl'
default['apache']['dir'] = '/etc/httpd'
default['apache']['log_dir'] = '/var/log/httpd'
Configuration in the vars directory can be compared to "override" attributes in Chef. Configuration in a Chef cookbook has a low precedence but you can use "override" attributes to give a very high precedence. The use of "override" attributes in cookbooks is however very uncommon. They are of very limited practical use. Vice versa the Ansible vars directory for its intended use of creating high precedence configuration that overrides almost all other configuration, is of very limited practical use.
If you disagree please share examples of roles where we absolutely need high precedence configuration in a role. You can for example share a link to a Ansible role that demonstrates practical use.
The vars role directory is useful but not for its intended use. In practice the directory is used to store conditional configuration. The fact that configuration gets a high precedence is a more a problem than a desired or intended result.
This is demonstrated by the geerlingguy.posttgresql role. In this role geerlingguy uses "pseudo variables" to work around the fact that Ansible does not have conditional defaults vars.
For example in vars/Debian-7.yml a variable __postgresql_data_dir is introduced. This variable gets a high precedence.
__postgresql_data_dir: "/var/lib/postgresql/{{ __postgresql_version }}/main"
It is of no practical use other than that it can be used to mimic a conditional default var postgresql_data_dir as show in tasks/variables.yml
- name: Define postgresql_data_dir.
set_fact:
postgresql_data_dir: "{{ __postgresql_data_dir }}"
when: postgresql_data_dir is not defined
It would make sense if precedence rules could be configured because the vars directory in a Ansible role is typically of limited practical use because of its high precedence. To make practical use of the vars directory trickery is needed as demonstrated by the postgresql_data_dir in the geerlingguy.posttgresql role to lower precedence of configuration in this directory.
If you don't like this trickery you can alternatively use workaround set_fact as described in Set Ansible role defaults conditionally or unholy inline coding as described in use conditionals in vars ansible.
The Ansible community would be well advised to change the intended use of the vars directory from "override" to "conditional" configuration. Giving high precedence to configuration in a role is very uncommon requirement. Conditional configuration is however very very common.
Q: "The desired behavior is that"
1) Ansible merges vars
2) config in group_vars takes precedence over config in my role.
"Can I configure this behavior in Ansible?"
A: No. The precedence can't be changed. "role vars"(15) override "group_vars"(7). Use "group_vars" to override "role defaults"(2).
Q: "What you want are conditional default vars which Ansible does not support."
A: If Ansible would not support conditional default it would not be possible to write roles for multiple systems. It is possible to create conditional defaults, e.g.
# Defaults variables
- name: "os_vars_playbook_dir: Vars from {{ playbook_dir }}/vars/defaults"
include_vars: "{{ item }}"
with_first_found:
- files:
- "{{ ansible_distribution }}-{{ ansible_distribution_release }}.yml"
- "{{ ansible_distribution }}.yml"
- "{{ ansible_os_family }}.yml"
- "default.yml"
- "defaults.yml"
paths: "{{ playbook_dir }}/vars/defaults"
# Custom variables
- name: "os_vars_playbook_dir: Vars from {{ playbook_dir }}/vars"
include_vars: "{{ item }}"
with_first_found:
- files:
- "{{ ansible_distribution }}-{{ ansible_distribution_release }}.yml"
- "{{ ansible_distribution }}.yml"
- "{{ ansible_os_family }}.yml"
- "default.yml"
- "defaults.yml"
paths: "{{ playbook_dir }}/vars"
(available at GitHub)
Notes
OS-specific variables can be included after a play setup found out what system is running on the host.
Role's vars/defaults have very high precedence "include_vars"(18) (because of 1.). But this directory should comprise system variables only. The user does not want to change these variables under standard circumstances.
If necessary, the role may be customized in vars/. Custom configuration files in vars/ will survive the role's update while vars/defaults may be updated.
I am asking for help with a problem of deploying multiple versions (different variables) of the app with the same role run from one playbook.
We have an app with multiple product families, which are different code versions. Each version has separate uWSGI vassal config and Nginx virtualhost config(/api/v2, /api/v3, ...).
The desired state would be to run playbook and configure the server with all versions specified.
Sadly, ansible's import_role/import_tasks can't be used with with_items, so include_role/include_tasks must be used (pitty because they do not honor role tags).
The include_role method would not be the biggest problem, but we use handlers to notify uWSGI touch to reload - on a code change, link change, virtualenv change, app_config change, ...).
But when using loop (with_items), the variables passed from the loop does not correctly propagate to handlers.
I tried this scenarios
playbook.yml - with_items loop inside the playbook
PROBLEM: Handler is run only for the first iteration of the loop.
#!/usr/bin/env ansible-playbook
# HAndler is run only once, from first notifier
- hosts: localhost
gather_facts: no
vars:
app_root: "/tmp/test_ansible"
app_versions:
- app_product_family: 1
app_release: "v1.0.2"
- app_product_family: 3
app_release: "v4.0.7"
tasks:
- name: Deploy multiple versions of app
include_role:
name: app
with_items: "{{ app_versions }}"
loop_control:
loop_var: app_version
vars:
app_product_family: "{{ app_version.app_product_family }}"
app_release: "{{ app_version.app_release }}"
tags:
- app
- app_debug
playbook_v2.yml - with_items loop inside role task
PROBLEM: Handler is run with the default value from "Defaults"
#!/usr/bin/env ansible-playbook
- hosts: localhost
gather_facts: no
roles:
- app_v2
vars:
app_v2_root: "/tmp/test_ansible_v2"
app_v2_versions:
- app_v2_product_family: 1
app_v2_release: "v1.0.2"
- app_v2_product_family: 3
app_v2_release: "v4.0.7"
Tasks roles/app_v2/main.yml
---
# Workaround because import_tasks can't be run with_items
- include_tasks: deploy.yml
when: app_v2_versions
with_items: "{{ app_v2_versions }}"
loop_control:
loop_var: app_v2
vars:
app_v2_product_family: "{{ app_v2.app_v2_product_family }}"
app_v2_release: "{{ app_v2.app_v2_release }}"
tags:
- app_v2
- app_v2_deploy
...
One idea was about writing a separate role for each product family, but they share nginx and uWSGI, so it will be lots of copy-pasting and sharing tasks (so tags would not work properly).
For now, I solved it with shell script wrapper, but this is not an ideal solution and does not work from Ansible tower.
Sample repo with tasks to reproduce problem (tested with ansible 2.4, 2.5, 2.6)
Any ideas & recommendations are very welcome.
The order of overrides for variables is broken for includes in Ansible. F.e. even set_fact in the included role will be shadowed by role defaults.
See this bug: https://github.com/ansible/ansible/issues/22025
It's closed but not fixed. My advice: use include and variables really carefully.
In practice I never use role includes with loop. If you need loop, include a tasklist in this loop (and that tasklist, in turn, may import_role).
Ok, It is a bug as #George Shuklin posted.
I will use my shell wrapper, which reads group_vars yaml and then runs the playbook multiple times according to the variable list length.
Sadly I hit multiple annoying bugs in ansible in last few weeks, kinda losing my trust in it ):
And probably everybody is using microservices and kubernetes, so need to speed up our migration (:
I'm trying to update a cloudformation stack with just a param value that I need to change via Ansible. The stack is previously created and has about 20 input params, but I just need to update the value for one. I tried the following:
- name: update
cloudformation:
stack_name: "{{ stack_name }}"
state: present
region: "{{ region }}"
disable_rollback: false
args:
template_parameters:
CreateAlarms: "{{ create_alarms }}"
When I run it, the play throws an error stating that it expects values for the other template params. From the ansible documentation here http://docs.ansible.com/ansible/latest/cloudformation_module.html, it says "If state is present, the stack does exist, and neither template nor template_url is specified, the previous template will be reused." How do I tell the cloudformation module to use previous values as well? I know that aws cli supports it via the usePreviousValue flag, but how I do it with Ansible cloudformation?
Thanks in advance.
author/maintainer of the current Ansible cloudformation module here. There isn't a method to reuse previous values, you must specify the parameters every time. Usually that's fine because you have your params stored in your Ansible playbook anyhow.
If you're nervous, the values are listed in the cloudformation console, and you can also use changesets in Ansible to make sure only the expected are changing.
I have some trouble getting my playbook to include another. I use a playbook to roll out a clean VM. After that, I'd like to run another playbook, to configure the VM in a certain way. I've added the new host to the inventory and have ssh access to it.
Our team has set up a project per servertype. I've retrieved the right path to the project in an early stage (running against localhost) and used set_fact to put it in "servertype_project".
I expect this to work (at the playbook-level, running against the new VM):
- name: "Run servertype playbook ({{ project }}) on VM"
vars:
project: "{{ hostvars['localhost']['servertype_project'] }}"
include: "{{ project }}/ansible/playbook.yml"
But it fails the syntax check with this message:
ERROR! {{ hostvars['localhost']['servertype_project'] }}: 'hostvars' is undefined
I can retrieve the right string if I refer to {{ hostvars['localhost']['servertype_project'] }} from within a task, but not from the level in which I can include another playbook.
Since the value is determined at runtime, I think set_fact is the correct way to store the variable, but that one is host-specific. Is there any way I can pass it along to this host as well? Or did I miss some global-var-like-option?
You are not missing anything. This is expected. Facts are bound to hosts and hostvars is not accessible in playbook scope.
There is no way to define a variable that would be accessible in the include declaration, except for passing it as an extra-vars argument in CLI. But if you use an in-memory inventory (which is a reasonable assumption), then calling another ansible-playbook instance is out of question.
I think you should rethink your flow and instead of including a dynamically-defined play, have statically-defined play(s) running against your in-memory inventory and include the roles conditionally.
Does Ansible duplicate role variables for role dependencies that have enabled allow_duplicates?
For example, given a playbook that includes more than once role application-environment that allows duplicates, will Ansible create multiple copies of its variables?
meta/main.yml:
---
allow_duplicates: yes
dependencies:
- src: git+http://javasource/git/ansible/roles/organization
version: 1.1.0
vars/main.yml:
---
application_directory: "{{ organization.directory }}/{{ application_name }}"
application_component_directory: "{{ application_directory }}/{{ application_component_name }}"
If Ansible does not create multiple copies of these variables, how could I rework the role so that it can support multiple variables?
You may find some helpful information here:
About vars:
Anything in the vars directory of the role overrides previous versions of that variable in namespace.
About defaults:
Tasks in each role will see their own role’s defaults. Tasks defined outside of a role will see the last role’s defaults.