I needed to be able to invert variables stored in a JSON file that is passed to the playbook from the command line.
These are the tasks that I set up (they are identical except for vars), this is a fragment of a playbook:
- name: Prepare a .sql file
delegate_to: 127.0.0.1
mysql_db:
name: "{{ source['database']['db_name'] }}"
state: dump
login_host: "{{ source['database']['host'] }}"
login_user: "{{ source['database']['user'] }}"
login_password: "{{ source['database']['password'] }}"
target: test_db.sql
when: invert is not defined
- name: Prepare a .sql file (inverted)
delegate_to: 127.0.0.1
mysql_db:
name: "{{ target['database']['db_name'] }}"
state: dump
login_host: "{{ target['database']['host'] }}"
login_user: "{{ target['database']['user'] }}"
login_password: "{{ target['database']['password'] }}"
target: test_db.sql
when: invert is defined
So consequently when I execute
ansible-playbook -i hosts playbook.yml --extra-vars "#dynamic_vars.json"
the first task is executed. If I execute
ansible-playbook -i hosts playbook.yml --extra-vars "#dynamic_vars.json" --extra-vars "invert-yes"
the second task is executed that takes the same hash as parameters, but only swaps source for target (which essentially becomes a source in my playbook).
As you can see, this is a very simplistic approach, there is a lot of unnecessary duplication, I just do not like it. However, I cannot think of a better way to be able to revert variables at the command line without building some more complex include logic.
Perhaps you can advice me on how I can do it better? Thanks!
I'm a big fan of YAMLs anchors and references when it comes to the topic of avoiding repetition. Since the content is dynamic, you could take advantage of with_items, which can be used to pass a parameter like so:
- &sqldump
name: Prepare a .sql file
delegate_to: 127.0.0.1
mysql_db:
name: "{{ item['database']['db_name'] }}"
state: dump
login_host: "{{ item['database']['host'] }}"
login_user: "{{ item['database']['user'] }}"
login_password: "{{ item['database']['password'] }}"
target: test_db.sql
when: invert is not defined
with_items:
- source
- <<: *sqldump
name: Prepare a .sql file (inverted)
when: invert is defined
with_items:
- target
The 2nd task is a perfect clone of the first one, you then override the name, condition and the loop with_items to pass the target instead of the source.
After reading your answer to #ydaetskcoR it sounds like you have quite some cases where you need to use the data from one or the other dict. Maybe in that case it then would make sense to just define the var globally depending on the invert parameter. Your vars file could look like that:
---
source:
database: ...
db_name: ...
target:
database: ...
db_name: ...
data: "{{ target if invert is defined else source }}"
You then simply can use data in all your tasks without dealing with conditions any further.
- name: Prepare a .sql file
delegate_to: 127.0.0.1
mysql_db:
name: "{{ data['database']['db_name'] }}"
state: dump
login_host: "{{ data['database']['host'] }}"
login_user: "{{ data['database']['user'] }}"
login_password: "{{ data['database']['password'] }}"
target: test_db.sql
Of course, this way you have a fixed task name which does not change with the param you pass.
If you are attempting to do the same thing but just want to specify different variables depending on the host/group then a better approach may be to simply set these as host/group vars and run it as a single task.
If we set up our inventory file a bit like this:
[source_and_target-nodes:children]
source-nodes
target-nodes
[source-nodes]
source database_name='source_db' database_login_user='source_user' database_login_pass='source_pass'
[target-nodes]
target database_name='target_db' database_login_user='target_user' database_login_pass='target_pass'
Then we can target the task at the source_and_target-nodes like so:
- name: Prepare a .sql file
hosts: source_and_target-nodes
mysql_db:
name: "{{ database_name }}"
state: dump
login_host: "{{ inventory_hostname }}"
login_user: "{{ database_login_user }}"
login_password: "{{ database_login_pass }}"
target: test_db.sql
You won't be able to access the host vars of a different host this easily if you need to use delegate_to as you are in your question but if you are simply needing to run the play locally you can instead set ansible_connection to local in your host/group vars or setting connection: local in the play.
Related
I have a task to create a one-off cleanup playbook which is using variables from a role, but i don't need to execute that role. Is there a way to provide a role name to get everything from it's defaults and vars, without hardcoding paths to it? I also want to use vars defined in group_vars or host_vars with higher precedence than the ones included from role.
Example task:
- name: stop kafka and zookeeper services if they exist
service:
name: "{{ item }}"
state: stopped
with_items:
- "{{ kafka_service_name }}"
- "{{ zookeeper_service_name }}"
ignore_errors: true
where kafka_service_name and zookeeper_service_name are contained in role kafka, but may also be present in i.e. group_vars.
I came up with a fairly hacky solution, which looks like this:
- name: save old host_vars
set_fact:
old_host_vars: "{{ hostvars[inventory_hostname] }}"
- name: load kafka role variables
include_vars:
dir: "{{ item.root }}/{{ item.path }}"
vars:
params:
files:
- kafka
paths: "{{ ['roles'] + lookup('config', 'DEFAULT_ROLES_PATH') }}"
with_filetree: "{{ lookup('first_found', params) }}"
when: item.state == 'directory' and item.path in ['defaults', 'vars']
- name: stop kafka and zookeeper services if they exist
service:
name: "{{ item }}"
state: stopped
with_items:
- "{{ old_host_vars['kafka_service_name'] | default(kafka_service_name) }}"
- "{{ old_host_vars['zookeeper_service_name'] | default(zookeeper_service_name) }}"
include_vars task finds the first kafka role folder in ./roles and in default role locations, then includes files from directories defaults and vars, in correct order.
I had to save old hostvars due to include_vars having higher priority than anything but extra vars as per ansible doc, and then using included var only if old_host_vars returned nothing.
If you don't have a requirement to load group_vars - include vars works quite nice in one task and looks way better.
UPD: Here is the regexp that i used to replace vars with old_host_vars hack.
This was tested in vscode search/replace, but can be adjusted for any other editor
Search for vars that start with kafka_:
\{\{ (kafka_\w*) \}\}
Replace with:
{{ old_host_vars['$1'] | default($1) }}
I am wondering is it possible to pass in the --extra-vars when running an ansible-playbook in order to inject the variables into an inventory file, which I am using to run my playbook.
sample playbook
- name: "Create CI pipeline"
hosts: all
tasks:
- name: "Create PreCodeReview jobs"
tags:
- jenkins
- jenkins-jobs
when: jenkins is defined
local_action:
module: jenkins_job
url: "{{ jenkins.url }}"
user: "{{ jenkins.username }}"
token: "{{ jenkins.access_token }}"
name: "{{ jenkins.component.name }}_PreCodeReview"
config: "{{ lookup('template', '../templates/jenkins/add-pre-code-config.xml') }}"
- name: "Create Release jobs"
tags:
- jenkins
- jenkins-jobs
when: jenkins is defined
local_action:
module: jenkins_job
url: "{{ jenkins.url }}"
user: "{{ jenkins.username }}"
token: "{{ jenkins.access_token }}"
name: "{{ jenkins.component.name }}_Release"
config: "{{ lookup('template', '../templates/jenkins/add-release-config.xml') }}"
I am looking to pass in jenkins.component.name at run time, I have attempted this with the following jenkins.component.name=<name> and "{'jenkins':{'component':{'name':<name>}}}"
This didn't work.
Here is the inventory I am using to run the playbook
sample inventory
all:
hosts:
local:
ansible_host: 127.0.0.1
ansible_connection: local
project_name: magic_proj
jenkins:
url: https://my/jenkins
username: admin
access_token: f96hjfg54354b3e8512d491fb471fd
keep_builds: 20
components:
- name: <repo_name>
repository: <repo_url>
I am looking to pass in jenkins.component.name at run time, I have attempted this with the following jenkins.component.name=<name> and "{'jenkins':{'component':{'name':<name>}}}"
You were very close: the --extra-vars wants either key=value pairs, JSON, YAML, or #./some/file, as specified in the fine manual
Regrettably, what you provided was Python syntax, and not JSON syntax; if you change your command line to --extra-vars '{"jenkins":{"component":{"name":<name>}}}'
update:
However, even that has a problem: it appears that for dict structures, ansible does not merge inventory dicts and extra-var dicts, so you will need to either choose a "flat" extra-var name (such as almost what you also attempted: --extra-vars '{"jenkins_component_name": ""}') or manually merge the structures together in your playbook (perhaps via pre_tasks: or similar)
I have a code to backup config using the ios_config module. I used ios_facts to get the hostname of devices and I want to use it to put the backup file in a similarly named folder and also use it in the file name itself.
In the last task of my code, I need to loop through two items - the sequence from 0 to 1(or how many items are in my inventory) as I need to access the hostname in the results and use it in the backup options, and also loop through my inventory of devices which I extracted from a csv file. I am aware of the rule of double curly braces but I do not know how to get around it.
---
- hosts: localhost
gather_facts: false
tasks:
- name: Block
block:
- name: Use CSV
csv_to_facts:
src: '{{playbook_dir}}/NEW/Inventory.csv'
vsheets:
- INFO:
- IP
- OS
- debug:
msg: '{{item.IP}}'
loop: '{{INFO}}'
- name: Create Inventory
add_host:
hostname: '{{item.IP}}'
ansible_network_os: '{{item.OS}}'
ansible_user: cisco
ansible_ssh_pass: cisco
ansible_connection: network_cli
ansible_become: yes
ansible_become_method: enable
groups: group_01
loop: '{{INFO}}'
- name: Gather Facts (IOS)
ios_facts:
register: ios_facts_loop
delegate_to: '{{item}}'
loop: "{{groups['group_01']}}"
- name: Backup Switch (IOS)
ios_config:
backup: yes
backup_options:
dir_path: "tmp/backups/{{ ios_facts_loop.results.{{item[0]}}.ansible_facts.ansible_net_hostname }}"
filename: "{{ios_facts_loop.results.item{{[0]}}.ansible_facts.ansible_net_hostname}} {{ lookup('pipe','date +%Y-%m-%d#%H:%M:%S')}}"
register: backup_ios_location
delegate_to: '{{item[1]}}'
loop:
- with_sequence: "0-{{output|length - 3}}"
- "{{groups['group_01']}}"
TLDR; for vars notation
You cannot add double curly braces inside double curly braces like in your above code. You current var reference:
ios_facts_loop.results.{{item[0]}}.ansible_facts.ansible_net_hostname
should be turned to
ios_facts_loop.results[item[0]].ansible_facts.ansible_net_hostname
# or equivalent
ios_facts_loop.results[item.0].ansible_facts.ansible_net_hostname
Meanwhile, this will only fix your current syntax error (that you didn't share in your question) as the first element in your loop is a string 'with_sequence: "0-X"' which therefore has no index 0.
Attempt to fix the logic
If I understand correctly, for your last task, you just need to loop over the results of your ios_facts register and delegate the task to the server it was taken from. Luckilly, you should already have all the info you need in ios_facts_loop.results
It is a list so you can directly loop over it
Each element should contain an item key with the actual item that was used in the previous run at time of register (i.e. one of your groups['group_01'] element).
So I would try to write your last task like this. Disclaimer this is a pure guess as I didn't see your exact datastructure.
- name: Backup Switch (IOS)
ios_config:
backup: yes
backup_options:
dir_path: "tmp/backups/{{ item.ansible_facts.ansible_net_hostname }}"
filename: "{{ item.ansible_facts.ansible_net_hostname}}{{ lookup('pipe','date +%Y-%m-%d#%H:%M:%S')}}"
register: backup_ios_location
delegate_to: '{{item.item}}'
loop: "{{ ios_facts_loop.results }}"
Going further.
I'm not really familiar with the ios_* modules but they should be really close to other stuff I use daily and I think you could really simplify your playbook taking advantage of more ansible feature (e.g. multiple plays in a playbook). I believe the following should actually do the job:
---
- name: Construct inventory from CSV
hosts: localhost
gather_facts: false
tasks:
- name: Use CSV
csv_to_facts:
src: '{{playbook_dir}}/NEW/Inventory.csv'
vsheets:
- INFO:
- IP
- OS
- name: Create Inventory
add_host:
hostname: '{{item.IP}}'
ansible_network_os: '{{item.OS}}'
ansible_user: cisco
ansible_ssh_pass: cisco
ansible_connection: network_cli
ansible_become: yes
ansible_become_method: enable
groups: group_01
loop: '{{INFO}}'
- name: Backup switches from created inventory
hosts: group_01
gather_facts: false
tasks:
- name: Get facts from network os
ios_facts:
gather_subset: all
- name: Backup Switch (IOS)
ios_config:
backup: yes
backup_options:
dir_path: "tmp/backups/{{ ansible_net_hostname }}"
filename: "{{ ansible_net_hostname }}{{ lookup('pipe','date +%Y-%m-%d#%H:%M:%S') }}"
More background on dot and brackets notation for vars
You can basically navigate a yaml datastructure with two notation which are equivalent.
the dot notation
a_list_var.index_number
a_hasmap_var.keyname
the brackets notation
a_list_var[index_number]
a_hashmap_var['key_name']
If we take the following example:
my_servers:
hostA:
ips:
- x.x.x.x
- y.y.y.y
env:
shell: bash
home: somewhere
hostB:
ips:
- a.a.a.a
- b.b.b.b
env:
shell: sh
home: elsewhere
The following notation are all strictly equivalent:
# all vars of hostA
hostA_vars: "{{ my_servers.hostA }}"
hostA_vars: "{{ my_server['hostA'] }}"
# first IP of hostB
hostB_ip: "{{ my_servers.hostB.0 }}"
hostB_ip: "{{ my_servers.hostB[0] }}"
hostB_ip: "{{ my_servers['hostB'].0 }}"
hostB_ip: "{{ my_servers['hostB'][0] }}"
As you can see, the dot notation tends to be less verbose and more readable. Meanwhile, you cannot use a variable identifier with the dot notation. So If you want to ave the home env of a variable server you would have to use:
# set a var for server
server: hostA
# all equivalent again
server_home: "{{ my_servers[server].env.home }}"
server_home: "{{ my_servers[server]['env'].home }}"
server_home: "{{ my_servers[server].env['home'] }}"
server_home: "{{ my_servers[server]['env']['home'] }}"
I am currently taking two hosts and dynamically adding them to a group, followed by a synchronize task using with_together to use 3 lists of 2 elements in parallel to copy the specified files between two remote servers.
Here's an example based on the idea:
---
- name: Configure Hosts for Copying
hosts: localhost
gather_facts: no
tasks:
- name: Adding given hosts to new group...
add_host:
name: "{{ item }}"
groups: copy_group
with_items:
- ["remoteDest1", "remoteDest2"]
- name: Copy Files between servers
hosts: copy_group
gather_facts: no
tasks:
- name: Copying files...
synchronize:
src: "{{ item[1] }}"
dest: "{{ item[2] }}"
with_together:
- ["remoteSrc1", "remoteSrc2"]
- ["/tmp/remote/source/one/", "/tmp/remote/source/two/"]
- ["/tmp/remote/dest/one/", "/tmp/remote/dest/two/"]
delegate_to: "{{ item[0] }}"
Currently, it does both operations for both servers, resulting in 4 operations.
I need it to synchronize like so:
copy /tmp/remote/source/one/ from remoteSrc1 to /tmp/remote/dest/one/ on remoteDest1
copy /tmp/remote/source/two/ from remoteSrc2 to /tmp/remote/dest/two/ on remoteDest2
Which would mean it's a 1:1 ratio; essentially acting on the hosts in the same manner as with_together does for the lists.
The hosts are obtained dynamically, so I can't just make a different play for each host.
Since synchronize is essentially simplified version of rsync, then if there's a simple solution for this using rsync directly, then it would be much appreciated.
There isn't native functionality for this, so this is how I solved it:
Given the original task, add the following two lines:
- "{{ groups['copy_group'] }}"
when: inventory_hostname == item[3]
To get:
- name: Copying files...
synchronize:
src: "{{ item[1] }}"
dest: "{{ item[2] }}"
with_together:
- ["remoteSrc1", "remoteSrc2"]
- ["/tmp/remote/source/one/", "/tmp/remote/source/two/"]
- ["/tmp/remote/dest/one/", "/tmp/remote/dest/two/"]
- "{{ groups['copy_group'] }}"
delegate_to: "{{ item[0] }}"
when: inventory_hostname == item[3]
Essentially, by adding the hosts as a list, they can be used in the when statement to execute the task only when the current host (inventory_hostname) matches the host currently being indexed in the list.
The result is that the play only runs against each host once in a serial manner with the other list items that have the same index.
How can an Ansible playbook register in a variable the result of including another playbook?
For example, would the following register the result of executing tasks/foo.yml in result_of_foo?
tasks:
- include: tasks/foo.yml
- register: result_of_foo
How else can Ansible record the result of a task sequence?
The short answer is that this can't be done.
The register statement is used to store the output of a single task into a variable. The exact contents of the registered variable can vary widely depending on the type of task (for example a shell task will include stdout & stderr output from the command you run in the registered variable, while the stat task will provide details of the file that is passed to the task).
If you have an include file with an arbitrary number of tasks within it then Ansible would have no way of knowing what to store in the variable in your example.
Each individual task within your include file can register variables, and you can reference those variables elsewhere, so there's really no need to even do something like this.
I was able to do this by passing a variable name as a variable to be used in the task. I included my main.yaml and included cgw.yaml files below.
main.yaml:
- name: Create App A CGW
include: cgw.yaml
vars:
bgp_asn: "{{ asn_spoke }}"
ip_address: "{{ eip_app_a.public_ip }}"
name: cgw-app-a
region: "{{ aws_region }}"
aws_access_key: "{{ ec2_access_key }}"
aws_secret_key: "{{ ec2_secret_key }}"
register: cgw_app_a
cgw.yaml:
- name: "{{ name }}"
ec2_customer_gateway:
bgp_asn: "{{ bgp_asn }}"
ip_address: "{{ ip_address }}"
name: "{{ name }}"
region: "{{ region }}"
aws_access_key: "{{ aws_access_key }}"
aws_secret_key: "{{ aws_secret_key }}"
register: "{{ register }}"