I'm writing a playbook to get a list of services deployed to a Mirantis' UCP/MKE cluster and check on all the Docker workers in that cluster, that each external swarm service port is open.
The playbook makes an API call from localhost to get an extensive JSON object of services deployed to the cluster, which is simplified using jmespath to just name, ID, ports.
As another play, my playbook runs a shell command on each worker in the cluster to obtain a list of open ports.
I'd like to loop through each port for each service and confirm if the port is open on each and every worker node.
My services/ports data object can look like this:
[
{
"ID": "aefgergergergergerg",
"Name": "application1_service",
"Ports": [
[
30950,
"tcp"
],
[
30951,
"tcp"
]
]
},
{
"ID": "sdfthtrhrthrthrthrthtrh",
"Name": "application2_service",
"Ports": [
[
31190,
"tcp"
]
]
},
...
]
(obtained via an API call and can be simplified with a jmespath query:
'[?Endpoint.Ports].{ ID: ID, Name: Spec.Name, Ports: Endpoint.Ports[?contains(#.PublishMode,`ingress`)].[PublishedPort, PublishMode, Protocol] }'
And my worker's open ports objects look like this:
ok: [worker1] => {
"msg": [
"tcp:31557",
"tcp:31501",
"tcp:31556",
"tcp:31500",
"tcp:30231",
"tcp:30230",
"tcp:30651",
"tcp:30650"
]
}
ok: [worker2] => {
"msg": [
"tcp:31557",
"tcp:31501",
"tcp:31556",
"tcp:31500",
"tcp:30231",
"tcp:30230",
"tcp:30651",
"tcp:30650"
]
}
ok: [worker3] => {
"msg": [
"tcp:31557",
"tcp:31501",
"tcp:31556",
"tcp:31500",
"tcp:30231",
"tcp:30230",
"tcp:30651",
"tcp:30650"
]
}
obtained with
iptables -L DOCKER-INGRESS | awk -F ' {2,}' '($1 == "ACCEPT") && ($6 ~ /dpt/) {print $6}' | sed 's/ dpt//g')
In my head, I want to combine a with_subelements loop (ports for each given service) with a with_nested loop (my subelements as the first list, and my open ports as the nested list), but I am sure this isn't quite possible.
This is the relevant part of my playbook (I've cut out the auth logic as it's not relevant)
- name: Ensure secrets are in the required collections
hosts: localhost
gather_facts: false
vars_files: vars.yaml
[SNIP]
- name: "Get a list of services from https://{{ endpoint }}/services"
ansible.builtin.uri:
url: "https://{{ endpoint }}/services"
body_format: json
headers:
Authorization: "Bearer {{ auth.json.auth_token }}"
validate_certs: "{{ validate_ssl_certs | default('yes') }}"
register: services
- name: "Create a simplified JSON object of services and ports"
ansible.builtin.set_fact:
services_ports: "{{ services.json | json_query(jmesquery) }}"
vars:
jmesquery: "{{ jmesquery_services }}"
- name: See what ports are open on which workers
hosts: workers
gather_facts: false
become: true
vars_files: vars.yaml
tasks:
- name: Get the list of open ports
shell: iptables -L DOCKER-INGRESS | awk -F ' {2,}' '($1 == "ACCEPT") && ($6 ~ /dpt/) {print $6}' | sed 's/ dpt//g'
register: iptables_rules
- name: debug
debug:
msg: "{{ iptables_rules.stdout_lines }}"
And relevant bit of vars.yaml:
---
jmesquery_services: '[?Endpoint.Ports].{ ID: ID, Name: Spec.Name, Ports: Endpoint.Ports[?contains(#.PublishMode,`ingress`)].[PublishedPort, PublishMode, Protocol] }'
How best to check each of these ports for a service against each open port on each worker?
You are on a right track with subelements lookup plugin, you should simply loop over the service/port pairs and check if such port exists in iptables_rules.stdout_lines.
Here is an example playbook with dummy data on how to so:
- hosts: localhost
gather_facts: false
become: false
tasks:
- name: check if all service ports are open
# Loop over service/port pairs
loop: "{{ lookup('subelements', services_ports, 'Ports') }}"
# Set variables for clarity
vars:
service_name: "{{ item[0]['Name'] }}"
iptables_port: "{{ item[1][1] ~ ':' ~ item[1][0] }}"
iptables_port_exists: "{{ iptables_port in iptables_rules.stdout_lines }}"
# Fail the module if port is not found
failed_when: "not iptables_port_exists"
# For demo, print out the service status
debug:
msg: "Service {{ service_name }} is {{ 'up' if iptables_port_exists else 'down' }} on port {{ iptables_port }}"
# Example data
vars:
services_ports:
- ID: aefgergergergergerg
Name: application1_service
Ports:
- ["30950", "tcp"]
- ["30951", "tcp"]
- ID: sdfthtrhrthrthrthrthtrh
Name: application2_service
Ports:
- ["31190", "tcp"]
iptables_rules:
stdout_lines: [
"tcp:30950",
"tcp:31190",
]
This produces output like so:
TASK [check if all service ports are open] *************************************
ok: [localhost] => (item=[{'ID': 'aefgergergergergerg', 'Name': 'application1_service'}, ['30950', 'tcp']]) => {
"msg": "Service application1_service is up on port tcp:30950"
}
failed: [localhost] (item=[{'ID': 'aefgergergergergerg', 'Name': 'application1_service'}, ['30951', 'tcp']]) => {
"msg": "Service application1_service is down on port tcp:30951"
}
ok: [localhost] => (item=[{'ID': 'sdfthtrhrthrthrthrthtrh', 'Name': 'application2_service'}, ['31190', 'tcp']]) => {
"msg": "Service application2_service is up on port tcp:31190"
}
fatal: [localhost]: FAILED! => {"msg": "One or more items failed"}
PS! I'm not familiar with jmespath, but you might need to use hostvars['localhost']['services_ports'] on the workers in order to access variables created on localhost
So I'm fairly new to Ansible. I'm trying to get the ip address and hostname from my inventory:
- set_fact:
ip_out: "{{hostvars[inventory_hostname].ansible_default_ipv4.address }}"
host_out: "{{hostvars[inventory_hostname].inventory_hostname}}"
And then want to add it in my monitoring system through an API. I'm just not sure how to make my loop work. It works when adding one host at a time but not multiple.
- name: Add host to Check_MK site via WebAPI
uri:
url: '{{ cmkclient__connection_string }}?action=add_host&_username={{ cmkclient_api_user }}&_secret={{ cmkclient_api_password }}&output_format=json'
method: 'POST'
body: 'request={"attributes":{"alias": "Test", "ipaddress": "{{item[0]}}", "hostname": "{{item[1]}}", "create_folders": "0", "folder": "" }'
return_content: yes
delegate_to: localhost
when: '"No such host" in cmkclient__host_query.json.result'
register: cmkclient__host_add
changed_when: (cmkclient__host_add.json is defined) and
(cmkclient__host_add.json.result_code == 0)
failed_when: (cmkclient__host_add.json is not defined) or
(cmkclient__host_add.json.result_code != 0)
with_nested:
- "{{ip_out}}"
- "{{host_out}}"
I get a JSON parsing error.
Any ideas would be helpful.
Thanks!
It seems you'd like to use the IP address and hostname of hosts under the targeted group for the body of API request. Instead of delegating this task to localhost, we could have a play on localhost like:
# gather required facts from all hosts
- hosts: all
gather_facts: false
tasks:
- setup:
gather_subset: network
- hosts: localhost
connection: local
gather_facts: false
tasks:
- debug:
msg: "ipaddress: {{ ip_address }}, hostname: {{ host_name }}"
vars:
ip_address: "{{ hostvars[item]['ansible_default_ipv4']['address'] }}"
host_name: "{{ hostvars[item]['inventory_hostname'] }}"
loop: "{{ groups['all'] }}"
I've used a debug task, but the same loop and vars can be applied to your uri task as well.
Note:
If you are running the uri task on my_group (or all) hosts, then you should be simply able to refer to the required variables directly, without delegating. In this case the task will run on each host of the group using its IP address and hostname.
body: 'request={"attributes":{"alias": "Test", "ipaddress": "{{ ansible_default_ipv4['address'] }}", "hostname": "{{ inventory_hostname }}", "create_folders": "0", "folder": "" }'
I am writing a playbook task to get the IP address from the output and store the value and use it for other tasks.
Ansible playbook
name: Extract the secondary router ipsec tunnel address
hosts: secondary
gather_facts: false
connection: local
tags:
"sec_tunnel_ip"
tasks:
name: Extract Tunnel1 ipsec interface address
ios_command:
commands: "sh ip int br | sec Tunnel1"
register: save_tunn_out
debug:
msg: "{{save_tunn_out.stdout}}"
I am getting the output like below:
ok: [172.16.12.1] => {
"msg": [
"Tunnel1 172.16.121.54 YES manual up up \nTunnel100 10.0.0.101 YES manual up up"
]
}
But I want to extract the first ip interface output(for tunnel1) like below, and store it in a variable.
172.16.121.54
I am not sure how to get it without regex and store it on the variable.
Please help!
If you don't want to use regex, which would be your best bet, you have to rely on splitting the string.
Assuming it will always be after Tunnel1 and followed by a space, you can do it like this.
- name: Extract Tunnel1 ipsec interface address
set_fact:
save_tunn_out: "Tunnel1 172.16.121.54 YES manual up up \nTunnel100 10.0.0.101 YES manual up up"
- name: Extract IP address
debug:
var: save_tunn_out.split("Tunnel1 ")[1].split(" ")[0]
Which gives you the desired output
ok: [localhost] => {
"save_tunn_out.split(\"Tunnel1 \")[1].split(\" \")[0]": "172.16.121.54"
}
To store in a variable afterward, you can use set_fact like this
- name: Store in a variable
set_fact:
ip_address: "{{save_tunn_out.split('Tunnel1 ')[1].split(' ')[0]}}"
- name: Debug variable
debug:
msg: "Ip address is : {{ip_address}}"
Output :
ok: [localhost] => {
"msg": "Ip address is : 172.16.121.54"
}
When running debug mode with Ansible Playbook i can clearly see that one of returned values is "invocation" however i struggle trying to get it form playbook. "register: xyz" Allows you only to get "msg, status failed, changed" form returned values (at least in task i use - proxmox_kvm). Is there a way of accessing rest of them?
My code:
---
- hosts: pve
become: yes
vars:
passwd: !vault |
$ANSIBLE_VAULT;1.1;AES256
<encrypted-password>
tasks:
- name: Stop VM
proxmox_kvm:
api_user : root#pam
api_password: "{{ passwd }}"
api_host : 10.0.0.1
name : "{{ vm_name }}"
node : my-node
state : current
register: output
- debug:
var: output
Output value from 'register':
"output": {
"changed": false,
"failed": false,
"msg": "VM RHEL8.1 with vmid = 101 is stopped",
"status": "stopped"
List of returned vars:
https://docs.ansible.com/ansible/latest/reference_appendices/common_return_values.html
When we check hostvars with:
- name: Display all variables/facts known for a host
debug: var=hostvars[inventory_hostname]
We get:
ok: [default] => {
"hostvars[inventory_hostname]": {
"admin_email": "admin#surfer190.com",
"admin_user": "root",
"ansible_all_ipv4_addresses": [
"192.168.35.19",
"10.0.2.15"
],...
How would I specify the first element of the "ansible_all_ipv4_addresses" list?
Use dot notation
"{{ ansible_all_ipv4_addresses.0 }}"
This should work just like it would in Python. Meaning you can access the keys with quotes and the index with an integer.
- set_fact:
ip_address_1: "{{ hostvars[inventory_hostname]['ansible_all_ipv4_addresses'][0] }}"
ip_address_2: "{{ hostvars[inventory_hostname]['ansible_all_ipv4_addresses'][1] }}"
- name: Display 1st ipaddress
debug:
var: ip_address_1
- name: Display 2nd ipaddress
debug:
var: ip_address_2
I had this same challenge when trying to parse the result of a command in Ansible.
So the result was:
{
"changed": true,
"instance_ids": [
"i-0a243240353e84829"
],
"instances": [
{
"id": "i-0a243240353e84829",
"state": "running",
"hypervisor": "xen",
"tags": {
"Backup": "FES",
"Department": "Research"
},
"tenancy": "default"
}
],
"tagged_instances": [],
"_ansible_no_log": false
}
And I wanted to parse the value of state into the result register in the ansible playbook.
Here's how I did it:
Since the result is an hash of array of hashes, that is state is in the index (0) hash of the instances array, I modified my playbook to look this way:
---
- name: Manage AWS EC2 instance
hosts: localhost
connection: local
# gather_facts: false
tasks:
- name: AWS EC2 Instance Restart
ec2:
instance_ids: '{{ instance_id }}'
region: '{{ aws_region }}'
state: restarted
wait: True
register: result
- name: Show result of task
debug:
var: result.instances.0.state
I saved the value of the command using register in a variable called result and then got the value of state in the variable using:
result.instances.0.state
This time when the command ran, I got the result as:
TASK [Show result of task] *****************************************************
ok: [localhost] => {
"result.instances.0.state": "running"
}
That's all.
I hope this helps