Ansible debug loop output as a single list? - debugging

I have the following debug task, which lists the image and version attributes of {{image_names}}.
---
- name: "List Images"
debug:
msg: "{{ item.image }}:{{ item.version }}"
loop: "{{ image_names }}"
The looped variable looks something like this
image_names:
- { service_name: "xxxxxxx", registry: "xxxxxxx", image: "xxxxxxx", version: "xxxxxxx" }
- { service_name: "xxxxxxx", registry: "xxxxxxx", image: "xxxxxxx", version: "xxxxxxx" }
...
The output I receive on ansible tower is
ok: [host] => (item={u'service_name': u'xxxxxxx', u'image': u'xxxxxxx', u'version': u'xxxxxxx', u'registry': u'xxxxxxx'}) => {
"msg": "xxxxxxx:xxxxxxx"
ok: [host] => (item={u'service_name': u'xxxxxxx', u'image': u'xxxxxxx', u'version': u'xxxxxxx', u'registry': u'xxxxxxx'}) => {
"msg": "xxxxxxx:xxxxxxx"
ok: [host] => (item={u'service_name': u'xxxxxxx', u'image': u'xxxxxxx', u'version': u'xxxxxxx', u'registry': u'xxxxxxx'}) => {
"msg": "xxxxxxx:xxxxxxx"
Is there a way to join all of the "msg:" outputs into one list so it looks like
msg:
- "xxxxxxx:xxxxxxx"
- "xxxxxxx:xxxxxxx"
- "xxxxxxx:xxxxxxx"
...

For example
- debug:
msg: "{{ image_names|
json_query('[].[image, version]')|
map('join', ':') }}"
vars:
image_names:
- {service_name: x1, registry: x2, image: x3, version: x4}
- {service_name: y1, registry: y2, image: y3, version: y4}
- {service_name: z1, registry: z2, image: z3, version: z4}
gives
msg:
- x3:x4
- y3:y4
- z3:z4

Another possible option as of Ansible 2.5 is using yaml for your callback instead of the default skippy. Change the following two options in your ansible.cfg:
# Use the YAML callback plugin.
stdout_callback = yaml
# Use the stdout_callback when running ad-hoc commands.
bin_ansible_callbacks = True
This changes stdout for debug messages from a giant single line:
fatal: [vulnmanager]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 99, "stderr": "Shared connection to <FQDN>.\r\n", "stderr_lines": ["Shared connection to <FQDN> closed."], "stdout": "\r\nRHSA-2021:1989 Important/Sec. bind-export-libs-32:9.11.26-4.el8_4.x86_64\r\nRHSA-2021:1989 Important/Sec. bind-libs-32:9.11.26-4.el8_4.x86_64\r\nRHSA-2021:1989 Important/Sec. bind-libs-lite-32:9.11.26-4.el8_4.x86_64\r\nRHSA-2021:1989 Important/Sec. bind-license-32:9.11.26-4.el8_4.noarch\r\nRHSA-2021:1989 Important/Sec. bind-utils-32:9.11.26-4.el8_4.x86_64\r\nRHSA-2021:2168 Important/Sec. bpftool-4.18.0-305.3.1.el8_4.x86_64\r\nRHSA-2021:1578 There are 44 available for installation on <FQDN>
to a yaml-based, much more readable version:
fatal: [vulnmanager]: FAILED! => changed=true
msg: non-zero return code
rc: 99
stderr: |-
Shared connection to <FQDN> closed.
stderr_lines: <omitted>
stdout: |2-
RHSA-2021:1989 Important/Sec. bind-export-libs-32:9.11.26-4.el8_4.x86_64
RHSA-2021:1989 Important/Sec. bind-libs-32:9.11.26-4.el8_4.x86_64
RHSA-2021:1989 Important/Sec. bind-libs-lite-32:9.11.26-4.el8_4.x86_64
RHSA-2021:1989 Important/Sec. bind-license-32:9.11.26-4.el8_4.noarch
RHSA-2021:1989 Important/Sec. bind-utils-32:9.11.26-4.el8_4.x86_64
RHSA-2021:2168 Important/Sec. bpftool-4.18.0-305.3.1.el8_4.x86_64
RHSA-2021:1578 Important/Sec. bpftool-4.18.0-305.el8.x86_64
There are 44 available for installation on <FQDN>

Related

How do I find which register attribute to use in Ansible?

I am testing out the results of the register commands and it yields different attribute for various tasks: failures, msg, stderr, err..etc
- yum:
name: packagenotfound
state: present
ignore_errors: yes
register: command_result
- debug:
msg: "{{ command_result }}"
ok: [ansible] => {
"msg": {
"changed": false,
"failed": true,
"failures": [
"No package packagenotfound available."
],
"msg": "Failed to install some of the specified packages",
"rc": 1,
"results": []
}}
And
- lvg:
pvs: /dev/sddnotfound
vg: vgdata
ignore_errors: yes
register: command_result
- debug:
msg: "{{ command_result }}"
ok: [ansible] => {
"msg": {
"changed": false,
"failed": true,
"msg": "Device /dev/sddnotfound not found."
}
And
- shell: thiscommandwontwork
ignore_errors: yes
register: command_result
- debug:
msg: "{{ command_result }}"
ok: [ansible] => {
"msg": {
"changed": true,
"cmd": "thiscommandwontwork",
"delta": "0:00:00.002560",
"end": "2020-02-05 04:24:35.297556",
"failed": true,
"msg": "non-zero return code",
"rc": 127,
"start": "2020-02-05 04:24:35.294996",
"stderr": "/bin/sh: thiscommandwontwork: command not found",
"stderr_lines": [
"/bin/sh: thiscommandwontwork: command not found"
],
"stdout": "",
"stdout_lines": []
}
And
- lvol:
lv: lvdata
vg: vgroup
size: 2000M
ignore_errors: yes
register: command_result
- debug:
msg: "{{ command_result }}"
ok: [ansible] => {
"msg": {
"changed": false,
"err": " Volume group \"vgroup\" not found\n Cannot process volume group vgroup\n",
"failed": true,
"msg": "Volume group vgroup does not exist.",
"rc": 5
}
Now if I tried to use when: '"xxx" in command_result.err' with yum task for example, it will result in dict_object not found error.
Is there a way to find out which attribute to use without testing?
Testing is definitely the easiest and fastest way to have a look at the content of your registered var in several situation and to take decisions on how to use it in your playbook.
Meanwhile, there are ways to have a global knowledge of what is returned in your registered var from the documenations:
There is a page on Common modules return values
Modules returning specific values usually document them on each relevant doc page. Here is an example for the stat module
You should also be aware that the global register structure is changed when using a loop by addind a top level results list, as explained in registering variables
Knowing what could be in your register does not mean it will be. Your example mentions the (undocumented...) err attribute for the lvol module, which will only be available for an lvol task in error. You can work around such cases by using tests (like my_register is failed) or defaulting values with the default filter.

Ansible: How do I execute tasks in ansible playbook in parallel with some time gap

Let me try and explain my need:
As part of the regular deployment of our application, we have a SQL script(which would alter tables, add tables or update, etc) which needs to be run on 3 schemas in a region and 5 schemas in another for example. The application is in AWS and the database is Arora db(RDS)- MySQL. This schema can take anywhere between 30 minutes to 3 hours.
This SQL script needs to be run in parallel and with a delay of 2 minutes between each schema run.
This is what I have achieved till now:
A file having DB details- dbdata.yml
---
conn_details:
- { host: localhost, user: root, password: "Password1!" }
- { host: localhost, user: root, password: "Password1!" }
The playbook:
- hosts: localhost
vars:
script_file: "{{ path }}"
vars_files:
- dbdata.yml
tasks:
- name: shell command to execute script in parallel
shell: |
sleep 30s
"mysql -h {{ item.host }} -u {{ item.user }} -p{{ item.password }} < {{ script_file }} >> /usr/local/testscript.log"
with_items: "{{ conn_details }}"
register: sql_query_output
async: 600
poll: 0
- name: Wait for sql execution to finish
async_status:
jid: "{{ item.ansible_job_id }}"
register: _jobs
until: _jobs.finished
delay: 20 # Check every 20 seconds. Adjust as you like.
retries: 10
with_items: "{{ sql_query_output.results }}"
1st part- executes the script in parallel and this also includes a time gap of 30 seconds before each execution.
2nd part- picks the ansible job id from the registered output and checks if the job is completed or not.
Please note: before including the 30 seconds sleep, this playbook was working fine.
We have following erroneous output upon execution:
ansible-playbook parallel_local.yml --extra-vars "path=RDS_script.sql"
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
PLAY [localhost] **************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [localhost]
TASK [sample command- ansible-playbook my_sqldeploy.yml --extra-vars "path=/home/NICEONDEMAND/bsahu/RDS_create_user1.sql"] ****************************************************************************************
changed: [localhost] => (item={u'host': u'localhost', u'password': u'Password1!', u'user': u'root'})
changed: [localhost] => (item={u'host': u'localhost', u'password': u'Password1!', u'user': u'root'})
TASK [Wait for creation to finish] ********************************************************************************************************************************************************************************
FAILED - RETRYING: Wait for creation to finish (10 retries left).
FAILED - RETRYING: Wait for creation to finish (9 retries left).
failed: [localhost] (item={'ansible_loop_var': u'item', u'ansible_job_id': u'591787538842.77844', 'item': {u'host': u'localhost', u'password': u'Password1!', u'user': u'root'}, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/root/.ansible_async/591787538842.77844'}) => {"ansible_job_id": "591787538842.77844", "ansible_loop_var": "item", "attempts": 3, "changed": true, "cmd": "sleep 30s\n\"mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log\"\n", "delta": "0:00:30.073191", "end": "2019-11-28 17:01:57.632285", "finished": 1, "item": {"ansible_job_id": "591787538842.77844", "ansible_loop_var": "item", "changed": true, "failed": false, "finished": 0, "item": {"host": "localhost", "password": "Password1!", "user": "root"}, "results_file": "/root/.ansible_async/591787538842.77844", "started": 1}, "msg": "non-zero return code", "rc": 127, "start": "2019-11-28 17:01:27.559094", "stderr": "/bin/sh: line 1: mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log: No such file or directory", "stderr_lines": ["/bin/sh: line 1: mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log: No such file or directory"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item={'ansible_loop_var': u'item', u'ansible_job_id': u'999397686792.77873', 'item': {u'host': u'localhost', u'password': u'Password1!', u'user': u'root'}, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/root/.ansible_async/999397686792.77873'}) => {"ansible_job_id": "999397686792.77873", "ansible_loop_var": "item", "attempts": 1, "changed": true, "cmd": "sleep 30s\n\"mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log\"\n", "delta": "0:00:30.120136", "end": "2019-11-28 17:01:58.694713", "finished": 1, "item": {"ansible_job_id": "999397686792.77873", "ansible_loop_var": "item", "changed": true, "failed": false, "finished": 0, "item": {"host": "localhost", "password": "Password1!", "user": "root"}, "results_file": "/root/.ansible_async/999397686792.77873", "started": 1}, "msg": "non-zero return code", "rc": 127, "start": "2019-11-28 17:01:28.574577", "stderr": "/bin/sh: line 1: mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log: No such file or directory", "stderr_lines": ["/bin/sh: line 1: mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log: No such file or directory"], "stdout": "", "stdout_lines": []}
PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
Any suggestions how to overcome this. Thanks in advance for all the help.
My bad. I had a silly mistake which was creating trouble. I needed to remove "" from the line executing the sql file. Reproducing the correct yaml file below
- hosts: localhost
vars:
script_file: "{{ path }}"
vars_files:
- dbdata.yml
tasks:
- name: sample command- ansible-playbook my_sqldeploy.yml --extra-vars "path=/home/NICEONDEMAND/bsahu/RDS_create_user1.sql"
shell: |
sleep 30s
mysql -h {{ item.host }} -u {{ item.user }} -p{{ item.password }} < {{ script_file }} >> /usr/local/testscript.log
with_items: "{{ conn_details }}"
register: sql_query_output
async: 600
poll: 0
- name: Wait for creation to finish
async_status:
jid: "{{ item.ansible_job_id }}"
register: _jobs
until: _jobs.finished
delay: 20 # Check every 5 seconds. Adjust as you like.
retries: 10
with_items: "{{ sql_query_output.results }}"
Thanks all for the help.

Register Ansible variable property

Using Ansible I'm having a problem registering a variable the way I want. Using the implementation below I will always have to call .stdout on the variable - is there a way I can do better?
My playbook:
Note the unwanted use of .stdout - I just want to be able to use the variable directly without calling a propery...?
---
- name: prepare for new deployment
hosts: all
user: ser85
tasks:
- name: init deploy dir
shell: echo ansible-deploy-$(date +%Y%m%d-%H%M%S-%N)
# http://docs.ansible.com/ansible/playbooks_variables.html
register: deploy_dir
- debug: var=deploy_dir
- debug: var=deploy_dir.stdout
- name: init scripts dir
shell: echo {{ deploy_dir.stdout }}/scripts
register: scripts_dir
- debug: var=scripts_dir.stdout
The output when I execute the playbook:
TASK [init deploy dir] *********************************************************
changed: [123.123.123.123]
TASK [debug] *******************************************************************
ok: [123.123.123.123] => {
"deploy_dir": {
"changed": true,
"cmd": "echo ansible-deploy-$(date +%Y%m%d-%H%M%S-%N)",
"delta": "0:00:00.002898",
"end": "2016-05-27 10:53:38.122217",
"rc": 0,
"start": "2016-05-27 10:53:38.119319",
"stderr": "",
"stdout": "ansible-deploy-20160527-105338-121888719",
"stdout_lines": [
"ansible-deploy-20160527-105338-121888719"
],
"warnings": []
}
}
TASK [debug] *******************************************************************
ok: [123.123.123.123] => {
"deploy_dir.stdout": "ansible-deploy-20160527-105338-121888719"
}
TASK [init scripts dir] ********************************************************
changed: [123.123.123.123]
TASK [debug] *******************************************************************
ok: [123.123.123.123] => {
"scripts_dir.stdout": "ansible-deploy-20160527-105338-121888719/scripts"
}
Any help or insights appreciated - thank you :)
If I understood it right you want to assign deploy_dir.stdout to a variable that you can use without stdout key. It can be done with set_fact module:
tasks:
- name: init deploy dir
shell: echo ansible-deploy-$(date +%Y%m%d-%H%M%S-%N)
# http://docs.ansible.com/ansible/playbooks_variables.html
register: deploy_dir
- set_fact: my_deploy_dir="{{ deploy_dir.stdout }}"
- debug: var=my_deploy_dir

Ansible Registered Variable not persisting in task

I am trying to use ansible to check to see if there are any aerospike migrations happening, and then run a task when migrations reach 0. To do that, I am using the ansible shell module to output the total number of migrations to stdout, register that output with ansible, and have ansible test on it.
Ansible seems to be recording the output correctly, but it is constantly displaying the stdout as "Hello World"
Here is my test playbook:
---
- hosts:
- foo
- bar
serial: 1
gather_facts: no
tasks:
- name: check for migrates
shell: "echo 10"
register: as_migrates
- debug: var=as_migrates
- debug: msg = "{{ as_migrates.stdout }}"
- debug: msg = "{{ as_migrates.stdout_lines }}"
- debug: msg = "{{ as_migrates }}"
Here is the output:
PLAY [foo;bar] ******************************************************
TASK: [check for migrates] ****************************************************
changed: [foo-10]
TASK: [debug var=as_migrates] *************************************************
ok: [foo-10] => {
"var": {
"as_migrates": {
"changed": true,
"cmd": "echo 10",
"delta": "0:00:00.001367",
"end": "2016-01-26 23:19:20.586245",
"invocation": {
"module_args": "echo 10",
"module_complex_args": {},
"module_name": "shell"
},
"rc": 0,
"start": "2016-01-26 23:19:20.584878",
"stderr": "",
"stdout": "10",
"stdout_lines": [
"10"
],
"warnings": []
}
}
}
TASK: [debug msg = "{{ as_migrates.stdout }}"] ********************************
ok: [foo-10] => {
"msg": "Hello world!"
}
TASK: [debug msg = "{{ as_migrates.stdout_lines }}"] **************************
ok: [foo-10] => {
"msg": "Hello world!"
}
TASK: [debug msg = "{{ as_migrates }}"] ***************************************
ok: [foo-10] => {
"msg": "Hello world!"
}
My question is: Why is does the debug var clearly show the correct stdout and the as_migrats.stdout display "Hello World"??
I know that "Hello World" is the default message for the message module. So is the register not persisting from one task to another?? I feel like I am missing something obvious. I don't have another variable named "as_migrates" in my ansible environment.
Ansible is "space" sensitive :-) You cannot have a space after msg
Try this:
- debug: msg="{{ as_migrates.stdout }}"
- debug: msg="{{ as_migrates.stdout_lines }}"
- debug: msg="{{ as_migrates }}"

Ansible register result of multiple commands

I was given a task to verify some routing entries for all Linux server and here is how I did it using an Ansible playbook
---
- hosts: Linux
serial: 1
tasks:
- name: Check first
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result
changed_when: false
- debug: msg="{{result.stdout}}"
- name: Check second
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result
changed_when: false
- debug: msg="{{result.stdout}}"
You can see I have to repeat same task for each routing entry and I believe I should be able to avoid this. I tried use with_items loop but got following error message
One or more undefined variables: 'dict object' has no attribute 'stdout'
is there a way to register variable for each command and loop over them one by one ?
Starting in Ansible 1.6.1, the results registered with multiple items are stored in result.results as an array. So you can use result.results[0].stdout and so on.
Testing playbook:
---
- hosts: localhost
gather_facts: no
tasks:
- command: "echo {{item}}"
register: result
with_items: [1, 2]
- debug:
var: result
Result:
$ ansible-playbook -i localhost, test.yml
PLAY [localhost] **************************************************************
TASK: [command echo {{item}}] *************************************************
changed: [localhost] => (item=1)
changed: [localhost] => (item=2)
TASK: [debug ] ****************************************************************
ok: [localhost] => {
"var": {
"result": {
"changed": true,
"msg": "All items completed",
"results": [
{
"changed": true,
"cmd": [
"echo",
"1"
],
"delta": "0:00:00.002502",
"end": "2015-08-07 16:44:08.901313",
"invocation": {
"module_args": "echo 1",
"module_name": "command"
},
"item": 1,
"rc": 0,
"start": "2015-08-07 16:44:08.898811",
"stderr": "",
"stdout": "1",
"stdout_lines": [
"1"
],
"warnings": []
},
{
"changed": true,
"cmd": [
"echo",
"2"
],
"delta": "0:00:00.002516",
"end": "2015-08-07 16:44:09.038458",
"invocation": {
"module_args": "echo 2",
"module_name": "command"
},
"item": 2,
"rc": 0,
"start": "2015-08-07 16:44:09.035942",
"stderr": "",
"stdout": "2",
"stdout_lines": [
"2"
],
"warnings": []
}
]
}
}
}
PLAY RECAP ********************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0
A slightly different situation, which took a while to figure out. If you want to use the results of multiple items, but for changed_when, then the register variable will not have a var.results! Instead, changed_when, is evaluated for each item, and you can just directly use the register var.
Simple example, which will result in changed: false:
- action: command echo {{item}}
register: out
changed_when: "'z' in out.stdout"
with_items:
- hello
- foo
- bye
Another example:
- name: Create fulltext index for faster text searches.
mysql_db: name={{SO_database}} state=import target=/tmp/fulltext-{{item.tableName}}-{{item.columnName}}.sql
with_items:
- {tableName: Posts, columnName: Title}
- {tableName: Posts, columnName: Body}
- {tableName: Posts, columnName: Tags}
- {tableName: Comments, columnName: Text}
register: createfulltextcmd
changed_when: createindexcmd.msg.find('already exists') == -1
Finally, when you do want to loop through results in other contexts, it does seem a bit tricky to programmatically access the index as that is not exposed. I did find this one example that might be promising:
- name: add hosts to known_hosts
shell: 'ssh-keyscan -H {{item.host}}>> /home/testuser/known_hosts'
with_items:
- { index: 0, host: testhost1.test.dom }
- { index: 1, host: testhost2.test.dom }
- { index: 2, host: 192.168.202.100 }
when: ssh_known_hosts.results[{{item.index}}].rc == 1
Posting because I can't comment yet
Relating to gameweld's answer, since Ansible 2.5 there's another way to accessing the iteration index.
From the docs:
Tracking progress through a loop with index_var
New in version 2.5.
To keep track of where you are in a loop, use the index_var directive
with loop_control. This directive specifies a variable name to contain
the current loop index:
- name: count our fruit
debug:
msg: "{{ item }} with index {{ my_idx }}"
loop:
- apple
- banana
- pear
loop_control:
index_var: my_idx
This also allows you to gather results from an array and act later to the same array, taking into account the previous results
- name: Ensure directories exist
file:
path: "{{ item }}"
state: directory
loop:
- "mouse"
- "lizard"
register: reg
- name: Do something only if directory is new
debug:
msg: "New dir created with name '{{ item }}'"
loop:
- "mouse"
- "lizard"
loop_control:
index_var: index
when: reg.results[index].changed
Please note that the "mouse lizard" array should be exactly the same
If what you need is to register the output of two commands separately, use different variable names.
---
- hosts: Linux
serial: 1
tasks:
- name: Check first
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result0
changed_when: false
- debug: msg="{{result0.stdout}}"
- name: Check second
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result1
changed_when: false
- debug: msg="{{result1.stdout}}"

Resources