Ansible inventory same server in multiple groups, execute a task multiple times - ansible

Question

I have Ansible inventory as below:
[all]
[zookeeper]
host1
host2
host3
[broker]
host1
host2
host3
host4
[schemaregistry]
host2
host3
[ksql]
host5
host6
[connector]
host4
[restproxy]
host5
host6
[controlcenter]
host1
The group names are services installed on a server. A server can have more that one service installed. Through my playbook I expect to execute task of only one server group. For example, when I limit the playbook execute of server group [controlcenter] which has host1, the execution is done for other group also which has host1 in them.
How can I avoid this behavior?
ansible-playbook certificate_expiry_calculator.pb -i /home/ansible/inv_dev.yml -l controlcenter
this gives me results for controlcenter as well as for broker and zookeeper.
The playbook is
# FOR SIT
#
# source /home/ansible/highlight.sh;ansible-playbook /home/ansible/certificate_expiry_calculator.pb -i /home/ansible/inv_sit.yml |grep msg| sed 's/\\n/\n/g'|sed 's/\\r//g'|sed 's/\\t/ /'|sed 's/"msg": "//g'|sed 's/"//g'|sed 's/^ //g'|column -t|highlight red `date +%Y`|sed "s/^/ /"
# FOR DEV
#
# source /home/ansible/highlight.sh;ansible-playbook /home/ansible/certificate_expiry_calculator.pb -i /home/ansible/inv_dev.yml |grep msg| sed 's/\\n/\n/g'|sed 's/\\r//g'|sed 's/\\t/ /'|sed 's/"msg": "//g'|sed 's/"//g'|sed 's/^ //g'|column -t|highlight red `date +%Y`|sed "s/^/ /"
- hosts: broker
tasks:
- name: calculate keystore expiry for broker
become: yes
become_user: root
shell: |
keystore_location=`cat /etc/kafka/server.properties|grep keystore.location|cut -d "=" -f2|head -1`
#keystore_pwd=`cat /etc/kafka/server.properties|grep ssl.keystore.password|cut -d "=" -f2|head -1`
keystore_pwd="zzz"
/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd 2>/dev/null | grep Alias | awk '{print $3}' | while read ALIAS
do
EXPIRACY=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null| grep Valid`
UNTIL=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null | grep Valid | perl -ne 'if(/until: (.*?)\n/) { print "$1\n"; }'`
UNTIL_SECONDS=`date -d "$UNTIL" +%s`
UNTIL_DATE=`echo $UNTIL|sed 's/ /-/g'`
REMAINING_DAYS=$(( ($UNTIL_SECONDS - $(date +%s)) / 60 / 60 / 24 ))
THRESHOLD=`date -d "\`date -d "+90 days"\`" +%s`
if [[ $THRESHOLD -le $UNTIL_SECONDS ]]; then
#printf "${BRIGHT}${GREEN}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n" >> ../certificate_expiry_dates
printf "BROKER : `hostname` : ${BRIGHT}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
else
printf "BROKER : `hostname` : ${BRIGHT}${RED}[WARNING] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
RET=1
fi
done
register: output
- debug:
msg="{{output.stdout}}"
- hosts: zookeeper
tasks:
- name: calculate keystore expiry for zookeeper
become: yes
become_user: root
shell: |
keystore_location=`cat /etc/kafka/zookeeper.properties|grep keyStore.location|cut -d "=" -f2|head -1`
#keystore_pwd=`cat /etc/kafka/zookeeper.properties|grep ssl.keystore.password|cut -d "=" -f2|head -1`
keystore_pwd="zzzz"
/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd 2>/dev/null | grep Alias | awk '{print $3}' | while read ALIAS
do
EXPIRACY=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null| grep Valid`
UNTIL=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null | grep Valid | perl -ne 'if(/until: (.*?)\n/) { print "$1\n"; }'`
UNTIL_SECONDS=`date -d "$UNTIL" +%s`
UNTIL_DATE=`echo $UNTIL|sed 's/ /-/g'`
REMAINING_DAYS=$(( ($UNTIL_SECONDS - $(date +%s)) / 60 / 60 / 24 ))
THRESHOLD=`date -d "\`date -d "+90 days"\`" +%s`
if [[ $THRESHOLD -le $UNTIL_SECONDS ]]; then
#printf "${BRIGHT}${GREEN}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n" >> ../certificate_expiry_dates
printf "ZOOKEEPER : `hostname` : ${BRIGHT}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
else
printf "ZOOKEEPER : `hostname` : ${BRIGHT}${RED}[WARNING] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
RET=1
fi
done
register: output
- debug:
msg="{{output.stdout}}"
- hosts: connector
tasks:
- name: calculate keystore expiry for connector
become: yes
become_user: root
shell: |
keystore_location=`cat /etc/kafka/connect-distributed.properties|grep keystore.location|cut -d "=" -f2|head -1`
#keystore_pwd=`cat /etc/kafka/connect-distributed.properties|grep ssl.keystore.password|cut -d "=" -f2`
keystore_pwd="zzz"
/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd 2>/dev/null | grep Alias | awk '{print $3}' | while read ALIAS
do
EXPIRACY=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null| grep Valid`
UNTIL=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null | grep Valid | perl -ne 'if(/until: (.*?)\n/) { print "$1\n"; }'`
UNTIL_SECONDS=`date -d "$UNTIL" +%s`
UNTIL_DATE=`echo $UNTIL|sed 's/ /-/g'`
REMAINING_DAYS=$(( ($UNTIL_SECONDS - $(date +%s)) / 60 / 60 / 24 ))
THRESHOLD=`date -d "\`date -d "+90 days"\`" +%s`
if [[ $THRESHOLD -le $UNTIL_SECONDS ]]; then
#printf "${BRIGHT}${GREEN}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n" >> ../certificate_expiry_dates
printf "CONNECTOR : `hostname` : ${BRIGHT}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
else
printf "CONNECTOR : `hostname` : ${BRIGHT}${RED}[WARNING] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
RET=1
fi
done
register: output
- debug:
msg="{{output.stdout}}"
- hosts: controlcenter
tasks:
- name: calculate keystore expiry for control center
become: yes
become_user: root
shell: |
keystore_location=`cat /etc/confluent-control-center/control-center-production.properties|grep keystore.location|cut -d "=" -f2|head -n 1`
#keystore_pwd=`cat /etc/confluent-control-center/control-center-production.properties|grep keystore.password|cut -d "=" -f2|head -n 1`
keystore_pwd="zzz"
/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd 2>/dev/null | grep Alias | awk '{print $3}' | while read ALIAS
do
EXPIRACY=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null| grep Valid`
UNTIL=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null | grep Valid | perl -ne 'if(/until: (.*?)\n/) { print "$1\n"; }'`
UNTIL_SECONDS=`date -d "$UNTIL" +%s`
UNTIL_DATE=`echo $UNTIL|sed 's/ /-/g'`
REMAINING_DAYS=$(( ($UNTIL_SECONDS - $(date +%s)) / 60 / 60 / 24 ))
THRESHOLD=`date -d "\`date -d "+90 days"\`" +%s`
if [[ $THRESHOLD -le $UNTIL_SECONDS ]]; then
#printf "${BRIGHT}${GREEN}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n" >> ../certificate_expiry_dates
printf "CONTROL_CENTER : `hostname` : ${BRIGHT}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
else
printf "CONTROL_CENTER : `hostname` : ${BRIGHT}${RED}[WARNING] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
RET=1
fi
done
register: output
- debug:
msg="{{output.stdout}}"
- hosts: schemaregistry
tasks:
- name: calculate keystore expiry for schema_registry
become: yes
become_user: root
shell: |
keystore_location=`cat /etc/schema-registry/schema-registry.properties|grep keystore.location|cut -d "=" -f2`
#keystore_pwd=`cat /etc/schema-registry/schema-registry.properties|grep ssl.keystore.password|cut -d "=" -f2`
keystore_pwd="zzz"
/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd 2>/dev/null | grep Alias | awk '{print $3}' | while read ALIAS
do
EXPIRACY=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null| grep Valid`
UNTIL=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null | grep Valid | perl -ne 'if(/until: (.*?)\n/) { print "$1\n"; }'`
UNTIL_SECONDS=`date -d "$UNTIL" +%s`
UNTIL_DATE=`echo $UNTIL|sed 's/ /-/g'`
REMAINING_DAYS=$(( ($UNTIL_SECONDS - $(date +%s)) / 60 / 60 / 24 ))
THRESHOLD=`date -d "\`date -d "+90 days"\`" +%s`
if [[ $THRESHOLD -le $UNTIL_SECONDS ]]; then
#printf "${BRIGHT}${GREEN}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n" >> ../certificate_expiry_dates
printf "SCHEMA_REGISTRY : `hostname` : ${BRIGHT}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
else
printf "SCHEMA_REGISTRY : `hostname` : ${BRIGHT}${RED}[WARNING] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
RET=1
fi
done
register: output
- debug:
msg="{{output.stdout}}"
- hosts: restproxy
tasks:
- name: calculate keystore expiry for rest proxy
become: yes
become_user: root
shell: |
keystore_location=`cat /etc/kafka-rest/kafka-rest.properties|grep keystore.location|cut -d "=" -f2|head -n 1`
#keystore_pwd=`cat /etc/kafka-rest/kafka-rest.properties|grep ssl.keystore.password|cut -d "=" -f2|head -n 1`
keystore_pwd="zzz"
/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd 2>/dev/null | grep Alias | awk '{print $3}' | while read ALIAS
do
EXPIRACY=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null| grep Valid`
UNTIL=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null | grep Valid | perl -ne 'if(/until: (.*?)\n/) { print "$1\n"; }'`
UNTIL_SECONDS=`date -d "$UNTIL" +%s`
UNTIL_DATE=`echo $UNTIL|sed 's/ /-/g'`
REMAINING_DAYS=$(( ($UNTIL_SECONDS - $(date +%s)) / 60 / 60 / 24 ))
THRESHOLD=`date -d "\`date -d "+90 days"\`" +%s`
if [[ $THRESHOLD -le $UNTIL_SECONDS ]]; then
#printf "${BRIGHT}${GREEN}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n" >> ../certificate_expiry_dates
printf "REST_PROXY : `hostname` : ${BRIGHT}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
else
printf "REST_PROXY : `hostname` : ${BRIGHT}${RED}[WARNING] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
RET=1
fi
done
register: output
- debug:
msg="{{output.stdout}}"
- hosts: ksql
tasks:
- name: calculate keystore expiry for rest proxy
become: yes
become_user: root
shell: |
keystore_location=`cat /etc/ksqldb/ksql-server.properties|grep -i ssl.keystore.location|cut -d "=" -f2|head -n 1`
#keystore_pwd=`cat /etc/kafka-rest/ksql-server.properties|grep ssl.keystore.password|cut -d "=" -f2|head -n 1`
keystore_pwd="zzz"
/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd 2>/dev/null | grep Alias | awk '{print $3}' | while read ALIAS
do
EXPIRACY=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null| grep Valid`
UNTIL=`/usr/bin/keytool -list -v -keystore $keystore_location -storepass $keystore_pwd -alias $ALIAS 2>/dev/null | grep Valid | perl -ne 'if(/until: (.*?)\n/) { print "$1\n"; }'`
UNTIL_SECONDS=`date -d "$UNTIL" +%s`
UNTIL_DATE=`echo $UNTIL|sed 's/ /-/g'`
REMAINING_DAYS=$(( ($UNTIL_SECONDS - $(date +%s)) / 60 / 60 / 24 ))
THRESHOLD=`date -d "\`date -d "+90 days"\`" +%s`
if [[ $THRESHOLD -le $UNTIL_SECONDS ]]; then
#printf "${BRIGHT}${GREEN}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n" >> ../certificate_expiry_dates
printf "KSQL : `hostname` : ${BRIGHT}[OK] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
else
printf "KSQL : `hostname` : ${BRIGHT}${RED}[WARNING] : Certificate $ALIAS expires $UNTIL_DATE : days left $REMAINING_DAYS ${NORMAL}\n"
RET=1
fi
done
register: output
- debug:
msg="{{output.stdout}}"
the execution result is
ansible#host1[~] $ source /home/ansible/highlight.sh;ansible-playbook /home/ansible/certificate_expiry_calculator.pb -i /home/ansible/inv_dev.yml --limit controlcenter
PLAY [broker] *****************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************
ok: [host1]
TASK [calculate keystore expiry for broker] ***********************************************************************************
changed: [host1]
TASK [debug] ******************************************************************************************************************
ok: [host1] => {
"msg": "BROKER : host1 : [OK] : Certificate caroot expires Wed-Dec-03-17:49:33-UKT-2042 : days left 7231 \nBROKER : host1 : [OK] : Certificate localhost expires Thu-Dec-12-09:34:06-UKT-2024 : days left 665 "
}
PLAY [zookeeper] **************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************
ok: [host1]
TASK [calculate keystore expiry for zookeeper] ********************************************************************************
changed: [host1]
TASK [debug] ******************************************************************************************************************
ok: [host1] => {
"msg": "ZOOKEEPER : host1 : [OK] : Certificate caroot expires Wed-Dec-03-17:49:33-UKT-2042 : days left 7231 \nZOOKEEPER : host1 : [OK] : Certificate localhost expires Thu-Dec-12-09:34:06-UKT-2024 : days left 665 "
}
PLAY [connector] **************************************************************************************************************
skipping: no hosts matched
PLAY [controlcenter] **********************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************
ok: [host1]
TASK [calculate keystore expiry for control center] ***************************************************************************
changed: [host1]
TASK [debug] ******************************************************************************************************************
ok: [host1] => {
"msg": "CONTROL_CENTER : host1 : [OK] : Certificate caroot expires Wed-Dec-03-17:49:33-UKT-2042 : days left 7231 \nCONTROL_CENTER : host1 : [OK] : Certificate localhost expires Thu-Dec-12-09:34:06-UKT-2024 : days left 665 "
}
PLAY [schemaregistry] *********************************************************************************************************
skipping: no hosts matched
PLAY [restproxy] **************************************************************************************************************
skipping: no hosts matched
PLAY [ksql] *******************************************************************************************************************
skipping: no hosts matched
PLAY RECAP ********************************************************************************************************************
host1 : ok=9 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
As per my understanding play for broker & zookeeper shouldn't have been executed here.

Answer 1

the execution is done for other group also which has host1 in them.
Given the zoo.ini
[all]
[zookeeper]
host1
host2
host3
[broker]
host1
host2
host3
host4
[schemaregistry]
host2
host3
[ksql]
host5
host6
[connector]
host4
[restproxy]
host5
host6
[controlcenter]
host1
a minimal example playbook zoo.yml
---
- hosts: all
become: false
gather_facts: false
tasks:
- debug:
msg: "{{ inventory_hostname }} in {{ group_names }}"
called with
sshpass -p ${PWD} ansible-playbook --inventory zoo.ini --user ${USER} --ask-pass zoo.yml --limit 'controlcenter'
I expect to execute task of only one server group.
will result into the expected output of
TASK [debug] **********************************************
ok: [host1] =>
msg: host1 in [u'broker', u'controlcenter', u'zookeeper']
How can I avoid this behavior?
The execution is done for the given host and group only. For me it was not possible to reproduce an issue.
With the now given playbook and since you are using several - hosts: sections with hard-coded group names, you are actually not targeting hosts and groups with patterns. According Playbook execution you have more than one play.
The issue is reproducible with the following example
---
- hosts: all
become: false
gather_facts: false
tasks:
- debug:
msg: "{{ inventory_hostname }} in {{ group_names }}"
- hosts: broker
become: false
gather_facts: false
tasks:
- debug:
msg: "{{ inventory_hostname }} in {{ group_names }}"
resulting into an output of
PLAY [all] ************************************************
TASK [debug] **********************************************
ok: [host1] =>
msg: host1 in [u'broker', u'controlcenter', u'zookeeper']
PLAY [broker] *********************************************
TASK [debug] **********************************************
ok: [host1] =>
msg: host1 in [u'broker', u'controlcenter', u'zookeeper']
PLAY RECAP ************************************************
host1 : ok=2
How can I avoid this behavior?
In this case I like to recommend to address the playbook structure itself (annot.: and as it seems to be far away from good readable, maintainable, best practices, idempotency, etc.).

Answer 2

I was able to solve this using simple tags for each server group inside playbook. May be another dirty method, but for now it has worked.