I have a list with shell lines that I want to execute on inventory hosts so I can determine if the database is working. For the test purposes I have 1 server with PostgreSQL and 1 with MySQL.
This is my playbook so far:
- name: Check db statuses
shell: "{{ item }}"
loop:
- ps -fp $(pgrep -u postgres) | grep /usr/lib/postgresql
- ps -fp $(pgrep -u mysql) | grep mysqld
register: http
ignore_errors: yes
changed_when: item.failed == false
this is failing with:
{
"http": {
"failed": true,
"msg": "The conditional check 'item.failed == false' failed. The error was: error while evaluating conditional (item.failed == false): 'ansible.parsing.yaml.objects.AnsibleUnicode object' has no attribute 'failed'"
}
}
I want to assign only the item.failed==false result in the register variable (http) but ignore the failed ones.
You can't select what will be registered in a loop. Instead, you'll have to evaluate the registered results in the next task(s), e.g.
- hosts: localhost
tasks:
- command: "{{ item }}"
loop:
- /bin/true
- /bin/false
register: http
ignore_errors: true
- debug:
msg: "{{ item.item }} failed: {{ item.failed }}"
loop: "{{ http.results }}"
loop_control:
label: "{{ item.cmd }}"
gives
ok: [localhost] => (item=['/bin/true']) =>
msg: '/bin/true failed: False'
ok: [localhost] => (item=['/bin/false']) =>
msg: '/bin/false failed: True'
Related
All,
Example: If i've got 20 hosts for a playbook and running them with Serial:10, below shell command runs on 10 hosts at a time. Once done handler task is called, wherein the task which creates dict (_dict) doesn't give a dictionary output thus the second task - Failed host - failed with mentioned error.
- name: Run some shell command
shell: "echo 2 > /abcd/abcd.txt"
when: random condition is satisfied
register: update2
ignore_errors: yes
notify: abc_handler
- handler:
- name: abcd_handler
set_fact:
_dict: "{{ dict(ansible_play_hosts|zip(
ansible_play_hosts|map('extract', hostvars, 'update2'))) }}"
run_once: true
- name: Find failed hosts
set_fact:
_failed: "{{ _dict|dict2items|json_query('[?value.failed].key') }}"
run_once: true
Handler First task output:
"changed: false"
"ansible_facts": {
"_dict": "{u'host1': {'stderr_lines': [], u'changed': True,...u'host2':.....u'host10'}"
2nd handler task gives the mentioned error when the dict2items is run for above values.
Thank you.
Q: "List of hosts where a certain task executed, changed something, or got failed."
A: For example, the command makes no changes at test_11 changes the file at test_12, and fails at test_13
- hosts: test_11,test_12,test_13
tasks:
- shell:
cmd: "echo 2 > /tmp/test/abcd.txt"
creates: /tmp/test/abcd.txt
register: update1
ignore_errors: true
TASK [shell] ***********************************************************
changed: [test_12]
fatal: [test_13]: FAILED! => changed=true
cmd: echo 2 > /tmp/test/abcd.txt
delta: '0:00:00.045992'
end: '2021-04-25 23:22:31.623804'
msg: non-zero return code
rc: 2
start: '2021-04-25 23:22:31.577812'
stderr: '/bin/sh: cannot create /tmp/test/abcd.txt: Permission denied'
stderr_lines: <omitted>
stdout: ''
stdout_lines: <omitted>
...ignoring
ok: [test_11]
Let's create a dictionary with the data first, e.g.
- set_fact:
_dict: "{{ dict(ansible_play_hosts|
zip(ansible_play_hosts|
map('extract', hostvars, 'update1'))) }}"
run_once: true
gives
_dict:
test_11:
changed: false
cmd: echo 2 > /tmp/test/abcd.txt
failed: false
rc: 0
stdout: skipped, since /tmp/test/abcd.txt exists
stdout_lines:
- skipped, since /tmp/test/abcd.txt exists
test_12:
changed: true
cmd: echo 2 > /tmp/test/abcd.txt
delta: '0:00:00.032474'
end: '2021-04-25 23:14:36.361510'
failed: false
rc: 0
start: '2021-04-25 23:14:36.329036'
stderr: ''
stderr_lines: []
stdout: ''
stdout_lines: []
test_13:
changed: true
cmd: echo 2 > /tmp/test/abcd.txt
delta: '0:00:00.054980'
end: '2021-04-25 23:14:35.565811'
failed: true
msg: non-zero return code
rc: 2
start: '2021-04-25 23:14:35.510831'
stderr: '/bin/sh: cannot create /tmp/test/abcd.txt: Permission denied'
stderr_lines:
- '/bin/sh: cannot create /tmp/test/abcd.txt: Permission denied'
stdout: ''
stdout_lines: []
Note that test_11 is reported ok not skipped despite the registered variable showing "stdout: skipped, since /tmp/test/abcd.txt exists".
The analysis is now trivial, e.g.
- set_fact:
_failed: "{{ _dict|dict2items|json_query('[?value.failed].key') }}"
run_once: true
gives the list of the failed hosts
_failed:
- test_13
and the next task
- set_fact:
_changed: "{{ (_dict|dict2items|json_query('[?value.changed].key'))|
difference(_failed) }}"
_ok: "{{ _dict|dict2items|json_query('[?value.changed == `false`].key') }}"
run_once: true
gives
_changed:
- test_12
_ok:
- test_11
Note that
The failed hosts need to be subtracted from the changed hosts because failed hosts are also reported as changed.
There will be no registered variable if a task is skipped.
Serial
Split the playbook into 2 plays if serial is used. e.g.
shell> cat playbook.yml
- hosts: all
serial: 10
tasks:
- shell:
cmd: "echo 2 > /tmp/test/abcd.txt"
creates: /tmp/test/abcd.txt
register: update1
ignore_errors: true
- hosts: all
tasks:
- set_fact:
_dict: "{{ dict(ansible_play_hosts|zip(
ansible_play_hosts|map('extract', hostvars, 'update1'))) }}"
run_once: true
It seems you would like to get the hosts on which the command task (shown in the question) failed or changed, and then target them for some other tasks.
There are two things required for this:
If the command task fails, playbook execution will stop and hence none of the following tasks will run. So we need to add ignore_errors flag to the task
add_host module to create a new group of hosts when the task failed or changed
So finally tasks like below should do the trick:
- hosts: some_group
serial: 1
- name: update file count
shell: "echo 2 > /home/ec2-user/abcd.txt"
when:
- count.stdout == "1"
register: update1
ignore_errors: true
- name: conditionally add the hosts from current play hosts to a new group
add_host:
groups:
- new_group
host: "{{ ansible_hostname }}"
when: >
cmd_stat is failed or
cmd_stat is changed
# Then have a play targeting the new group
- hosts: new_group
tasks:
# Tasks to be performed
Though the use of serial might make the whole playbook run longer if there are lot of hosts.
I have the following playbook:
---
- hosts: localhost
gather_facts: false
vars:
db_process:
- mysqld
- db2sys
- oracle*
- pmon
tasks:
- shell: ps -ef | grep '{{ item }}' | grep -v grep | wc -l ; pgrep -x '{{ item }}' | wc -l
loop: "{{ db_process }}"
changed_when: false
register: ps_out
- debug:
msg: "{{ item }}"
loop: "{{ ps_out.results }}"
- name: combine output
set_fact:
db_process_status: '{% if item.stdout_lines[0]|int == 0 and item.stdout_lines[1]|int == 0 %}pass{% else %}fail{% endif %}'
db_process_message: '{{ item.item }} {% if item.stdout_lines[0]|int == 0 and item.stdout_lines[1]|int == 0 %}Not Found{% else %}Found{% endif %}'
loop: "{{ ps_out.results }}"
After this task, i have to combine all the result of db_process_message into a proper JSON format.
Instead of grouping, Is there some way to accumulate the output across multiple process and register that to a variable?
Expected output:
[
"mysqld Not Found",
"db2sys Not Found",
"oracle Not Found",
"pmon Not Found"
]
In your set_fact task, you are overwritting in each iteration over the loop variable the values of db_process_status and db_process_message. Instead of that, you need to populate two lists, one for each variable, so that you keep their values from each loop run.
here is the task that can do it, your code was moved to the var1 and var2 lines, and the new code is appending these values to two lists:
task:
- name: combine output
set_fact:
db_process_status: '{{ db_process_status|default([]) + [var1] }}'
db_process_message: '{{ db_process_message|default([]) + [var2] }}'
vars:
var1: '{% if item.stdout_lines[0]|int == 0 and item.stdout_lines[1]|int == 0 %}pass{% else %}fail{% endif %}'
var2: '{{ item.item }} {% if item.stdout_lines[0]|int == 0 and item.stdout_lines[1]|int == 0 %}Not Found{% else %}Found{% endif %}'
loop: "{{ ps_out.results }}"
- name: print db_process_status
debug:
var: db_process_status
- name: print db_process_message
debug:
var: db_process_message
sample output (i added sshd to demonstrate it picks it up as running):
TASK [print db_process_status] ****************************************************************************************************************************************************************************************
ok: [localhost] => {
"db_process_status": [
"pass",
"pass",
"pass",
"pass",
"fail"
]
}
TASK [print db_process_message] ***************************************************************************************************************************************************************************************
ok: [localhost] => {
"db_process_message": [
"mysqld Not Found",
"db2sys Not Found",
"oracle* Not Found",
"pmon Not Found",
"sshd Found"
]
}
I am trying to shutdown databases in a loop, the catch is some databases run as a different user and others just run as oracle. I login as oracle user and run the playbook and if the database is run as oracle user it goes through fine. If it is running as a different user I would like to become that user (oracle user has permissions to do that).
Here is my main playbook:
[oracle#ansctrlsrv.localdomain epd3]$ cat test.yml
---
- hosts: testdrive
tasks:
- set_fact:
db_list: "{{ lookup('file', 'vars/' ~ inventory_hostname ~ '.dblist')|from_yaml }}"
- name: Shutdown running databases
include_tasks: shutdowndb.yml
loop: "{{ db_list }}"
DB list is as follows:
[oracle#ansctrlsrv.localdomain epd3]$ cat vars/dbsrv.localdomain.dblist
- ebs1
- ebs2
- ndb1
[oracle#ansctrlsrv.localdomain epd3]$ cat shutdowndb.yml
---
- debug: msg='Shutting down {{ item }}'
- name: Execute shutdown
shell: id "{{ item }}"
register: shutdown_output
become: "{{ item is search('ebs') | ternary('yes','no') }}"
become_user: "{{ item }}"
- debug: msg="{{ shutdown_output.stdout }}"
[oracle#ansctrlsrv.localdomain epd3]$ cat inventory
[testdrive]
dbsrv.localdomain
[oracle#ansctrlsrv.localdomain epd3]$ ansible-playbook -i inventory test.yml
TASK [Execute shutdown] ***
fatal: [dbsrv1.localdomain]: FAILED! => {"changed": false, "module_stderr": "Shared connection to dbsrv1.localdomain closed.\r\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} ...ignoring
I tried this question on another thread but got closed, so trying another hand after realizing a few issues myself such as being unable to run blocks of code with a loop, etc.
Appreciate any help.
Set become user conditionally
Fix the line
shell: id "{{ item }}"
correct syntax (use shell only when necessary)
command: "id {{ item }}"
The playbook below
- hosts: testdrive
tasks:
- set_fact:
db_list: "{{ lookup('file', 'vars/' ~ inventory_hostname ~ '.dblist')|from_yaml }}"
- name: Shutdown running databases
include_tasks: shutdowndb.yml
loop: "{{ db_list }}"
with the included tasks
$ cat shutdowndb.yml
- debug:
msg:
- "shell: {{ 'shutdown.sh ' ~ item }}"
- "become: {{ item is search('ebs')|ternary('yes', 'no') }}"
- "become_user: {{ item }}"
give
"msg": [
"shell: shutdown.sh ebs1",
"become: yes",
"become_user: ebs1"
]
"msg": [
"shell: shutdown.sh ebs2",
"become: yes",
"become_user: ebs2"
]
"msg": [
"shell: shutdown.sh ndb1",
"become: no",
"become_user: ndb1"
]
Q: "Why the command whoami is still giving oracle rather than ebs1?"
A: Short answer: Because become is not set to True.
Debugging
1) Is it possible to become all of the users in db_list? Yes.
- hosts: test_01
become: no
remote_user: admin
vars:
db_list: ['ebs1', 'ebs2', 'ndb1']
tasks:
- command: whoami
become_user: "{{ item }}"
become: true
register: result
loop: "{{ db_list }}"
- debug:
msg: "{{ result.results|json_query('[].stdout') }}"
give
"msg": [
"ebs1",
"ebs2",
"ndb1"
]
2) Does search and ternary work properly? Yes.
- debug:
msg: "{{ item is search('ebs')|ternary(true, false) }}"
loop: "{{ db_list }}"
gives
"msg": true
"msg": true
"msg": false
3) Does become and become_user work properly?. Yes.
- command: whoami
become_user: "{{ item }}"
become: "{{ item is search('ebs')|ternary(true, false) }}"
register: result
loop: "{{ db_list }}"
- debug:
msg: "{{ result.results|json_query('[].stdout') }}"
give
"msg": [
"ebs1",
"ebs2",
"admin"
]
Team, I have response from json_query which is a dict key:value and i would like to iterate over all values and run ssh command for each value
Below gets me list of all nodes
- name: "Fetch all nodes from clusters using K8s facts"
k8s_facts:
kubeconfig: $WORKSPACE
kind: Node
verify_ssl: no
register: node_list
- debug:
var: node_list | json_query(query)
vars:
query: 'resources[].{node_name: metadata.name, nodeType: metadata.labels.nodeType}'
TASK [3_validations_on_ssh : debug]
ok: [target1] => {
"node_list | json_query(query)": [
{
"nodeType": null,
"node_name": "host1"
},
{
"nodeType": "gpu",
"node_name": "host2"
},
{
"nodeType": "gpu",
"node_name": "host3"
}
]
}
playbook to write: parse node_name and use that in ssh command for all hosts 1-3
- name: "Loop on all nodeNames and ssh."
shell: ssh -F ~/.ssh/ssh_config bouncer#{{ item }}.internal.sshproxy.net "name -a"
register: ssh_result_per_host
failed_when: ssh_result_per_host.rc != 0
with_item: {{ for items in query.node_name }}
- debug:
var: ssh_result_per_host.stdout_lines
error output:
> The offending line appears to be:
failed_when: ssh_result_per_host.rc != 0
with_item: {{ for items in query.node_name }}
^ here
solution 2 also fails when i do loop:
shell: ssh -F ~/.ssh/ssh_config bouncer#{{ item.metadata.name }}.sshproxy.internal.net "name -a"
loop: "{{ node_list.resources }}"
loop_control:
label: "{{ item.metadata.name }}"
output sol 2:
failed: [target1] (item=host1) => {"msg": "Invalid options for debug: shell"}
failed: [target1] (item=host2) => {"msg": "Invalid options for debug: shell"}
fatal: [target1]: FAILED! => {"msg": "All items completed"}
To expand on Ash's correct answer:
- name: "Fetch all nodes from clusters using K8s facts"
k8s_facts:
kubeconfig: $WORKSPACE
kind: Node
verify_ssl: no
register: node_list
- set_fact:
k8s_node_names: '{{ node_list | json_query(query) | map(attribute="node_name") | list }}'
vars:
query: 'resources[].{node_name: metadata.name, nodeType: metadata.labels.nodeType}'
- name: "Loop on all nodeNames and ssh."
shell: ssh -F ~/.ssh/ssh_config bouncer#{{ item }}.internal.sshproxy.net "name -a"
register: ssh_result_per_host
with_items: '{{ k8s_node_names }}'
Separately, unless you quite literally just want to run that one ssh command, the way ansible thinks about that problem is via add_host::
- hosts: localhost
connection: local
gather_facts: no
tasks:
# ... as before, to generate the "k8s_node_names" list
- add_host:
hostname: '{{ item }}.internal.sshproxy.net'
groups:
- the_k8s_nodes
ansible_ssh_username: bouncer
# whatever other per-host variables you want
with_items: '{{ k8s_node_names }}'
# now, run the playbook against those nodes
- hosts: the_k8s_nodes
tasks:
- name: run "name -a" on the host
command: name -a
register: whatever
This is a contrived example, because if you actually wanted just to get the list of kubernetes nodes and use those in a playbook, you use would a dynamic inventory script
It is not with_item it is with_items
You cannot use with_item: {{ for items in query.node_name }}
Save the values to a variable and use the variable in with_items: {{ new_variable }}
I am fairly new to ansible and I am trying to determine how to test is a variable passed to my playbook matches against a list of substrings.
I have tried something like the following. Looping through my list of badcmds and then testing whether it is in the variable passed.
vars:
badcmds:
- clear
- no
tasks:
- name: validate input
debug:
msg: " {{ item }}"
when: item in my_command
with_items: "{{ badcmds }}"
I am getting the following error:
"msg": "The conditional check 'item in my_command' failed.
The error was: Unexpected templating type error occurred on
({% if item in my_command %} True {% else %} False {% endif %}):
coercing to Unicode: need string or buffer, bool found
Many thanks.
one problem with your playbook is that - no is automatically translated to boolean false. you should use "no" to make Ansible consider the variable as a string. without quotes:
---
- hosts: localhost
connection: local
gather_facts: false
vars:
badcmds:
- clear
- no
my_command: clear
tasks:
- name: print variable
debug:
msg: "{{ item }}"
with_items:
- "{{ badcmds }}"
output:
TASK [print variable] ***********************************************************************************************************************************************************************************************
ok: [localhost] => (item=None) => {
"msg": "clear"
}
ok: [localhost] => (item=None) => {
"msg": false
}
I guess you should enclose no in quotes, because this behavior was not your intention.
to make a loop and check if the variable matches any item from the badcmds list, you can use:
---
- hosts: localhost
connection: local
gather_facts: false
vars:
badcmds:
- "clear"
- "no"
tasks:
- name: validate input
debug:
msg: "{{ item }}"
when: item == my_command
with_items:
- "{{ badcmds }}"
hope it helps