Team,
I have my task that pulls hosts from a json_query output stdoud_lines.
Then i am looping over each hostname and performing an ssh connection plus shell commands on each of that host.
Finally, to list the output of each host, i am debugging out the results.[i] for every host. is there a way i can loop on results[i] where i = my dynamic variable having host entry? I want to loop on debug. any hint please?
task: "ssh and run shell command and manually list each host result"
- name: "RAID mount check for fscache on GPU Nodes"
shell: ssh -F {{ ssh_cfg_path.stdout }} {{ item.node_name }}.{{ ssh_host }} "df -kh /raid/"
ignore_errors: no
register: raid_info
failed_when: raid_info.rc != 0
with_items: "{{ gpu_nodes }}"
- name: raid_info result
debug:
var: raid_info
- name: raid_info results0_stdout_lines
debug:
var: raid_info.results[0].stdout_lines
- name: raid_info results1_stdout_lines
debug:
var: raid_info.results[1].stdout_lines
output:
TASK [team-services-pre-install-checks : raid_info results0_stdout_lines] ****
Friday 25 October 2019 17:03:07 +0000 (0:00:00.041) 0:00:25.292 ********
ok: [localhost] => {
"raid_info.results[0].stdout_lines": [
"Filesystem Size Used Avail Use% Mounted on",
"/dev/sdb1 7.0T 175G 6.5T 3% /raid"
]
}
TASK [team-services-pre-install-checks : raid_info results1_stdout_lines] ****
Friday 25 October 2019 17:03:07 +0000 (0:00:00.040) 0:00:25.333 ********
ok: [localhost] => {
"raid_info.results[1].stdout_lines": [
"Filesystem Size Used Avail Use% Mounted on",
"/dev/sdb1 7.0T 176G 6.5T 3% /raid"
]
}
Below approach does not work or lists any output.
- name: raid_info results loop over all hosts output/result
debug:
var: raid_info.results[{{ item }}].stdout_lines
with_items: "{{ raid_info }}"
I think i need to use hostvars but am new and not sure how to put up.
It looks like you were close, and just need to adjust your logic a little here. The with_items is going to loop through a list of whatever you feed it, so if you move more of your variable structure into the with_items list like this:
- name: raid_info results loop over all hosts output/result
debug:
msg: "{{ item.stdout }}"
with_items: "{{ raid_info.results }}"
loop_control:
label: "{{ item.item.node_name }}"
That will iterate over the indexed list that is the results created by the with_items of the previous task.
Related
I'm using community.vmware.vmware_guest_powerstate collection for Ansible to start VMs.
The problem is the time it takes for 1 VM can be 2-5 sec, which makes its very inefficient when I want to start 50 VMs ...
Is there any way to make it in parallel?
The playbook:
- hosts: localhost
gather_facts: false
collections:
- community.vmware
vars:
certvalidate: "no"
server_url: "vc01.x.com"
username: "{{ lookup('ansible.builtin.env', 'API_USER', default=Undefined) }}"
password: "{{ lookup('ansible.builtin.env', 'API_PASS', default=Undefined) }}"
tasks:
- name: "setting state={{ requested_state }} in vcenter"
community.vmware.vmware_guest_powerstate:
username: "{{ lookup('ansible.builtin.env', 'API_USER', default=Undefined) }}"
password: "{{ lookup('ansible.builtin.env', 'API_PASS', default=Undefined) }}"
hostname: "{{ server_url }}"
datacenter: "DC1"
validate_certs: no
name: "{{ item }}"
state: "powered-on"
loop: "{{ hostlist }}"
This is Ansible's output: (every line can take 2-5 sec ...)
TASK [setting state=powered-on in vcenter] ************************************************************************************************************
Monday 19 September 2022 11:17:59 +0000 (0:00:00.029) 0:00:08.157 ******
changed: [localhost] => (item=x1.com)
changed: [localhost] => (item=x2.com)
changed: [localhost] => (item=x3.com)
changed: [localhost] => (item=x4.com)
changed: [localhost] => (item=x5.com)
changed: [localhost] => (item=x6.com)
changed: [localhost] => (item=x7.com)
try this instead...
- hosts: all
gather_facts: false
collections:
- community.vmware
vars:
certvalidate: "no"
server_url: "vc01.x.com"
username: "{{ lookup('ansible.builtin.env', 'API_USER', default=Undefined) }}"
password: "{{ lookup('ansible.builtin.env', 'API_PASS', default=Undefined) }}"
tasks:
- name: "setting state={{ requested_state }} in vcenter"
community.vmware.vmware_guest_powerstate:
username: "{{ username }}"
password: "{{ password }}"
hostname: "{{ server_url }}"
datacenter: "DC1"
validate_certs: no
name: "{{ inventory_hostname }}"
state: "powered-on"
delegate_to: localhost
Then run it with your hostlist as the inventory and use forks:
ansible-playbook -i x1.com,x2.com,x3.com,... --forks 10 play.yml
... the time it takes for 1 VM can be 2-5 sec, which makes its very inefficient when I want to start 50 VMs ...
Right, this is the usual behavior.
Is there any way to make it in parallel?
As already mentioned within the comments by Vladimir Botka, asynchronous actions and polling is worth a try since
By default Ansible runs tasks synchronously, holding the connection to the remote node open until the action is completed. This means within a playbook, each task blocks the next task by default, meaning subsequent tasks will not run until the current task completes. This behavior can create challenges.
You see it in your case in the task and in a loop.
Probably the Best Practice to address the use case and to eliminate the cause is to enhance the module code.
According the documentation vmware_guest_powerstate module – Manages power states of virtual machines in vCenter and source ansible-collections/community.vmware/blob/main/plugins/modules/vmware_guest_powerstate.py, the parameter name: takes one name for one VM only. If it would be possible to provide a list of VM names "{{ hostlist }}" to the module directly, there would be one connection attempt only and the loop happening one the Remote Node instead of the Controller Node (... even if this is running localhost for both cases).
To do so one would need to start with name=dict(type='list') instead of str and implement all other logic, error handling and responses.
Further Documentation
Since the community vmware_guest_powerstate module is importing and utilizing additional libraries
pyVmomi library
pyVmomi Community Samples
Meanwhile and based on
Further Q&A and Tests
How do I optimize performance of Ansible playbook with regards to SSH connections?
I've setup another short performance test to simulate the behavior you are observing
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Gather subdirectories
shell:
cmd: "ls -d /home/{{ ansible_user }}/*/"
warn: false
register: subdirs
- name: Gather stats (loop) async
shell: "stat {{ item }}"
loop: "{{ subdirs.stdout_lines }}"
loop_control:
label: "{{ item }}"
async: 5
poll: 0
- name: Gather stats (loop) serial
shell: "stat {{ item }}"
loop: "{{ subdirs.stdout_lines }}"
loop_control:
label: "{{ item }}"
- name: Gather stats (list)
shell: "stat {% raw %}{{% endraw %}{{ subdirs.stdout_lines | join(',') }}{% raw %}}{% endraw %}"
register: result
- name: Show result
debug:
var: result.stdout
and found that adding async will add some additional overhead resulting into even longer execution time.
Gather subdirectories ------------------------ 0.57s
Gather stats (loop) async -------------------- 3.99s
Gather stats (loop) serial ------------------- 3.79s
Gather stats (list) -------------------------- 0.45s
Show result ---------------------------------- 0.07s
This is because of the "short" runtime of the executed task in comparison to "long" time establishing a connection. As the documentation pointed out
For example, a task may take longer to complete than the SSH session allows for, causing a timeout. Or you may want a long-running process to execute in the background while you perform other tasks concurrently. Asynchronous mode lets you control how long-running tasks execute.
one may take advantage from async in case of long running processes and tasks.
In respect the given answer from #Sonclay I've performed another test with
---
- hosts: all
become: false
gather_facts: false
tasks:
- name: Gather subdirectories
shell:
cmd: "ls -d /home/{{ ansible_user }}/*/"
warn: false
register: subdirs
delegate_to: localhost
- name: Gather stats (loop) serial
shell: "stat {{ item }}"
loop: "{{ subdirs.stdout_lines }}"
loop_control:
label: "{{ item }}"
delegate_to: localhost
whereby a call with
ansible-playbook -i "test1.example.com,test2.example.com,test3.example.com" --forks 3 test.yml
will result into an execution time of
Gather subdirectories ------------------------ 0.72s
Gather stats (loop) -------------------------- 0.39s
so it seems to be worth a try.
I am new to Ansible and don't have much coding experience in general. I am trying to figure out how to provision RHEL servers using the DellEMC OpenManage modules.
The first step to this is figuring out how to parse a CSV, that we are putting necessary information for the later templating. (IP, hostname, MAC, etc. ...) I can get it to print the data in general, but cant figure out how to parse/iterate through it.
CSV Sample
server_name,idrac_ip,idrac_user,idrac_pwd,idrac_nw,idrac_gw,idrac_mac,mgmt_ip,mgmt_nw,mgmt_gw,mgmt_mac,bond0_100_ip,bond0_200_ip,dns_1,bond0_100_nw,bond0_100_gw,bond0_100_vip,bond0_200_nw,bond0_200_gw,bond0_200_vip,os,fs,sio_type,mdm_name
test1,1.1.1.10,root,Password1234,1.1.1.0/24,1.1.1.1,00:00:00:00:00:01,1.1.1.142,1.1.62.0/24,1.1.2.1,98:03:9B:46:25:B3,1.1.1.22,1.1.1.21,1.1.1.26,1.1.61.15/24,1.1.61.1,1.1.61.29,1.1.66.0/24,1.1.66.1,1.1.1.29,RHEL 7.6,1,Master,MDM-1
Here is my playbook to print the info in general.
---
- name: Parse
hosts: localhost
connection: local
tasks:
- name: Load CSV Data into object
read_csv:
path: 'Test_Lab_Servers.csv'
fieldnames: server_name,idrac_ip,idrac_user,idrac_pwd,idrac_nw,idrac_gw,idrac_mac,mgmt_ip,mgmt_nw,mgmt_gw,mgmt_mac,bond0_100_ip,bond0_200_ip,dns_1,bond0_100_nw,bond0_100_gw,bond0_100_vip,bond0_200_nw,bond0_200_gw,bond0_200_vip,os,fs,sio_type,mdm_name
delimiter: ','
register: csv_output
delegate_to: localhost
- name: Print data
debug:
msg: "{{ csv_output }}"
Any advice?
According the read_csv module parameter documentation fieldnames are
needed if the CSV does not have a header
only. Therefore you could remove it to get a list object which might be easier to parse.
- name: Print data
debug:
msg: "{{ csv_output.list }}"
As already mentioned within the comments, to iterate over the list elements you may use loop.
- name: Print data
debug:
msg: "{{ item }}"
loop: "{{ csv_output.list }}"
With extended loop variables you will have a better control about the loop output.
- name: Print data
debug:
msg: "{{ item }}"
loop: "{{ csv_output.list }}"
loop_control:
extended: yes
label: "{{ ansible_loop.index0 }}"
Resulting into an output of
TASK [Print data] ***************
ok: [localhost] => (item=0) =>
msg:
bond0_100_gw: 1.1.61.1
bond0_100_ip: 1.1.1.22
bond0_100_nw: 1.1.61.15/24
bond0_100_vip: 1.1.61.29
bond0_200_gw: 1.1.66.1
bond0_200_ip: 1.1.1.21
bond0_200_nw: 1.1.66.0/24
bond0_200_vip: 1.1.1.29
dns_1: 1.1.1.26
fs: '1'
idrac_gw: 1.1.1.1
idrac_ip: 1.1.1.10
idrac_mac: 00:00:00:00:00:01
idrac_nw: 1.1.1.0/24
idrac_pwd: Password1234
idrac_user: root
mdm_name: MDM-1
mgmt_gw: 1.1.2.1
mgmt_ip: 1.1.1.142
mgmt_mac: 98:03:9B:46:25:B3
mgmt_nw: 1.1.62.0/24
os: RHEL 7.6
server_name: test1
sio_type: Master
...
Specific items (fields) like server_name you could get simply by using
msg: "{{ item.server_name }}"
resulting into an output of
TASK [Print data] ************
ok: [localhost] => (item=0) =>
msg: test1
ok: [localhost] => (item=1) =>
msg: test2
ok: [localhost] => (item=2) =>
msg: test3
...
I have a variable that is set from a Tower survey and I am using it to retrieve an associated IP address in netbox. I am not able to get it to match when I use square brackets and when I use {{ or ' or " everything is matched and my whole IPAM database is returned.
vars:
location: "{{ LOCATION }}"
c_description: "{{CIRCUIT_DESC}}"
prefix_length: "{{PREFIX}}"
tasks:
- name: "Print IP"
debug:
msg: "{{ query('netbox.netbox.nb_lookup', 'ip-addresses', api_filter= 'description=
[c_description]', api_endpoint='http://netbox', token='', validate_certs='False') }}"
Here is my output:
TASK [Print IP] ****************************************************************
ok: [localhost] => {
"msg": []
}
Solved in another forum using a tilda:
- name: "Print IP"
debug:
msg: "{{ query('netbox.netbox.nb_lookup', 'ip-addresses', api_filter='description=' ~ c_description,
api_endpoint='http://netbox.', token='', validate_certs='False') }}"
Basically i want modify an existing hostvars.
I have a dynamically generated array of hosts named "flash_hosts"
flash_hosts = ['host1', 'host2']
and a shell script which looks up a value for each host.
In following non-working code i try to assign each host the specific result of the script
- name: Assign values to to host var
shell: "get-assigned-value.sh {{ item }}"
register: "{{ hostvars[item].mac=stdout }}"
with_items: "{{ flash_hosts }}"
How can i make this work in in ansible? Basically i understand that register will not allow me to assign the value to hostvars directly, but how can this be solved then, as i need to iterate over the hosts?
Use set_fact to debug your register results on host wise..
- name: Assign values to to host var
shell: "/path/to/get-assigned-value.sh {{ item }}"
register: fileout
with_items:
- host1
- host2
- set_fact:
firstHost: "{{ fileout.results[0] }}"
secondHost: "{{ fileout.results[1] }}"
- debug:
var: firstHost.stdout
- debug:
var: secondHost.stdout
in the above example, firstHost is whole result of shell script running on first host, and firstHost.stdout gives the output of corresponding host shell script result.
It's possible set_facts and delegate_to the flash_hosts with delegate_facts.
But, to create a variable in hostvars the host must be declared in the inventory(static or dynamic). It's not necessary the host is accessible. For example
$ cat hosts
host1
host2
The play below
- hosts: localhost
vars:
flash_hosts: ['host1', 'host2']
tasks:
- name: Assign values to to host var
command: "{{ playbook_dir }}/get-assigned-value.sh {{ item }}"
register: result
loop: "{{ flash_hosts }}"
- set_fact:
mac: "{{ item.stdout }}"
loop: "{{ result.results }}"
delegate_to: "{{ item.item }}"
delegate_facts: true
- debug:
msg: "{{ hostvars[item].mac }}"
loop: "{{ flash_hosts }}"
gives
ok: [localhost] => (item=host1) =>
msg: mac address of host1
ok: [localhost] => (item=host2) =>
msg: mac address of host2
with the script
$ cat get-assigned-value.sh
#!/bin/sh
case $1 in
host1)
printf "mac address of host1"
;;
host2)
printf "mac address of host2"
;;
*)
printf "unknown host"
exit 1
;;
esac
exit 0
This question is NOT answered. Someone mentioned environment variables. Can you elaborate on this?
This seems like a simple problem, but not in ansible. It keeps coming up. Especially in error conditions. I need a global variable. One that I can set when processing one host play, then check at a later time with another host. In a nutshell, so I can branch later in the playbook, depending on the variable.
We have no control over custom software installation, but if it is installed, we have to put different software on other machines. To top it off, the installations vary, depending on the VM folder. My kingdom for a global var.
The scope of variables relates ONLY to the current ansible_hostname. Yes, we have group_vars/all.yml as globals, but we can't set them in a play. If I set a variable, no other host's play/task can see it. I understand the scope of variables, but I want to SET a global variable that can be read throughout all playbook plays.
The actual implementation is unimportant but variable access is (important).
My Question: Is there a way to set a variable that can be checked when running a different task on another host? Something like setGlobalSpaceVar(myvar, true)? I know there isn't any such method, but I'm looking for a work-around. Rephrasing: set a variable in one tasks for one host, then later in another task for another host, read that variable.
The only way I can think of is to change a file on the controller, but that seems bogus.
An example
The following relates to oracle backups and our local executable, but I'm keeping it generic. For below - Yes, I can do a run_once, but that won't answer my question. This variable access problem keeps coming up in different contexts.
I have 4 xyz servers. I have 2 programs that need to be executed, but only on 2 different machines. I don't know which. The settings may be change for different VM environments.
Our programOne is run on the server that has a drive E. I can find which server has drive E using ansible and do the play accordingly whenever I set a variable (driveE_machine). It only applies to that host. For that, the other 3 machines won't have driveE_machine set.
In a later play, I need to execute another program on ONLY one of the other 3 machines. That means I need to set a variable that can be read by the other 2 hosts that didn't run the 2nd program.
I'm not sure how to do it.
Inventory file:
[xyz]
serverxyz[1:4].private.mystuff
Playbook example:
---
- name: stackoverflow variable question
hosts: xyz
gather_facts: no
serial: 1
tasks:
- name: find out who has drive E
win_shell: dir e:\
register: adminPage
ignore_errors: true
# This sets a variable that can only be read for that host
- name: set fact driveE_machine when rc is 0
set_fact:
driveE_machine: "{{inventory_hostname}}"
when: adminPage.rc == 0
- name: run program 1
include: tasks/program1.yml
when: driveE_machine is defined
# program2.yml executes program2 and needs to set some kind of variable
# so this include can only be executed once for the other 3 machines
# (not one that has driveE_machine defined and ???
- name: run program 2
include: tasks/program2.yml
when: driveE_machine is undefined and ???
# please don't say run_once: true - that won't solve my variable access question
Is there a way to set a variable that can be checked when running a task on another host?
No sure what you actually want, but you can set a fact for every host in a play with a single looped task (some simulation of global variable):
playbook.yml
---
- hosts: mytest
gather_facts: no
vars:
tasks:
# Set myvar fact for every host in a play
- set_fact:
myvar: "{{ inventory_hostname }}"
delegate_to: "{{ item }}"
with_items: "{{ play_hosts }}"
run_once: yes
# Ensure that myvar is a name of the first host
- debug:
msg: "{{ myvar }}"
hosts
[mytest]
aaa ansible_connection=local
bbb ansible_connection=local
ccc ansible_connection=local
result
PLAY [mytest] ******************
META: ran handlers
TASK [set_fact] ******************
ok: [aaa -> aaa] => (item=aaa) => {"ansible_facts": {"myvar": "aaa"}, "ansible_facts_cacheable": false, "changed": false, "failed": false, "item": "aaa"}
ok: [aaa -> bbb] => (item=bbb) => {"ansible_facts": {"myvar": "aaa"}, "ansible_facts_cacheable": false, "changed": false, "failed": false, "item": "bbb"}
ok: [aaa -> ccc] => (item=ccc) => {"ansible_facts": {"myvar": "aaa"}, "ansible_facts_cacheable": false, "changed": false, "failed": false, "item": "ccc"}
TASK [debug] ******************
ok: [aaa] => {
"msg": "aaa"
}
ok: [bbb] => {
"msg": "aaa"
}
ok: [ccc] => {
"msg": "aaa"
}
https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#fact-caching
As shown elsewhere in the docs, it is possible for one server to reference variables about another, like so:
{{ hostvars['asdf.example.com']['ansible_os_family'] }}
This even applies to variables set dynamically in playbooks.
This answer doesn't pre-suppose your hostnames, nor how many hosts have a "drive E:". It will select the first one that is reachable that also has a "drive E:". I have no windows boxes, so I fake it with a random coin toss for whether a host does or doesn't; you can of course use your original win_shell task, which I've commented out.
---
- hosts: all
gather_facts: no
# serial: 1
tasks:
# - name: find out who has drive E
# win_shell: dir e:\
# register: adminPage
# ignore_errors: true
- name: "Fake finding hosts with drive E:."
# I don't have hosts with "drive E:", so fake it.
shell: |
if [ $RANDOM -gt 10000 ] ; then
exit 1
else
exit 0
fi
args:
executable: /bin/bash
register: adminPage
failed_when: false
ignore_errors: true
- name: "Dict of hosts with E: drives."
run_once: yes
set_fact:
driveE_status: "{{ dict(ansible_play_hosts_all |
zip(ansible_play_hosts_all |
map('extract', hostvars, ['adminPage', 'rc'] ) | list
))
}}"
- name: "List of hosts with E: drives."
run_once: yes
set_fact:
driveE_havers: "{%- set foo=[] -%}
{%- for dE_s in driveE_status -%}
{%- if driveE_status[dE_s] == 0 -%}
{%- set _ = foo.append( dE_s ) -%}
{%- endif -%}
{%- endfor -%}{{ foo|list }}"
- name: "First host with an E: drive."
run_once: yes
set_fact:
driveE_first: "{%- set foo=[] -%}
{%- for dE_s in driveE_status -%}
{%- if driveE_status[dE_s] == 0 -%}
{%- set _ = foo.append( dE_s ) -%}
{%- endif -%}
{%- endfor -%}{{ foo|list|first }}"
- name: Show me.
run_once: yes
debug:
msg:
- "driveE_status: {{ driveE_status }}"
- "driveE_havers: {{ driveE_havers }}"
- "driveE_first: {{ driveE_first }}"
It's working for me, you can directly use register var no need to use set_fact.
---
- hosts: manager
tasks:
- name: dir name
shell: cd /tmp && pwd
register: homedir
when: "'manager' in group_names"
- hosts: all
tasks:
- name: create a test file
shell: touch "{{hostvars['manager.example.io'['homedir'].stdout}}/t1.txt"
i have used "set_fact" module in the ansible tasks prompt menu, which is helped to pass user input into all the inventory hosts.
#MONITORING PLAYBOOK
- hosts: all
gather_facts: yes
become: yes
tasks:
- pause:
prompt: "\n\nWhich monitoring you want to perform?\n\n--------------------------------------\n\n1. Memory Utilization:\n2. CPU Utilization:\n3. Filesystem Utili
zation:\n4. Exist from Playbook: \n5. Fetch Nmon Report \n \nPlease select option: \n--------------------------------------\n"
register: menu
- set_fact:
option: "{{ menu.user_input }}"
delegate_to: "{{ item }}"
with_items: "{{ play_hosts }}"
run_once: yes
#1 Memory Monitoring
- pause:
prompt: "\n-------------------------------------- \n Enter monitoring Incident Number = "
register: incident_number_result
when: option == "1"
- name: Standardize incident_number variable
set_fact:
incident_number: "{{ incident_number_result.user_input }}"
when: option == "1"
delegate_to: "{{ item }}"
with_items: "{{ play_hosts }}"
run_once: yes
ansible playbook result is
[ansibusr#ansiblemaster monitoring]$ ansible-playbook monitoring.yml
PLAY [all] **************************************************************************
TASK [Gathering Facts] **************************************************************************
ok: [node2]
ok: [node1]
[pause]
Which monitoring you want to perform?
--------------------------------------
1. Memory Utilization:
2. CPU Utilization:
3. Filesystem Utilization:
4. Exist from Playbook:
5. Fetch Nmon Report
Please select option:
--------------------------------------
:
TASK [pause] **************************************************************************
ok: [node1]
TASK [set_fact] **************************************************************************
ok: [node1 -> node1] => (item=node1)
ok: [node1 -> node2] => (item=node2)
[pause]
--------------------------------------
Enter monitoring Incident Number = :
INC123456
TASK [pause] **************************************************************************
ok: [node1]
TASK [Standardize incident_number variable] ****************************************************************************************************************************
ok: [node1 -> node1] => (item=node1)
ok: [node1 -> node2] => (item=node2)