Ansible: read csv file line by line and render it to template with loop - ansible

I am using a csv file to get data and render it to template and copy to remote servers. I am facing issue that CSV file read twice for same server and this results in loop to copy last line in each server instead of 1st line in first server and second on second servers and so on...
test.csv file:
Application,env,Datacenter,Hostname
Microsoft,Test,DC1,testserver1
Apple,Test,DC2,testserver2
main.yml:
- name: read csv
read_csv:
path: /u00/app/monitor/test.csv
key: Hostname
register: newrelic
- name: Print newrelic var
ansible.builtin.debug:
var: newrelic.dict[inventory_hostname]
- name: copy template
template:
src: /u00/ansible/Playbooks/files/infra-config.yml_template
dest: /u00/app/monitor/infra-config.yml
loop: "{{ newrelic.list }}"
- name: Start the New Relic Service
ansible.builtin.systemd:
name: newrelic-infra.service
state: started
become: yes
become_user: root
Template:
custom_attributes:
application : {{ item.Application }}
env : {{ item.env }}
datacenter : {{ item.Datacenter }}
log:
file: /u00/app/monitor/infra.log
Expected result is to get first entry in csv in testserver1 and second line in testserver2
**ssh admin#testserver1 - **
cat infra-config.yml
custom_attributes:
application : Microsoft
env : Test
datacenter : DC1
log:
file: /u00/app/monitor/infra.log
**ssh admin#testserver2- **
custom_attributes:
application : Apple
env : Test
datacenter : DC2
log:
file: /u00/app/monitor/infra.log
but I am getting
**ssh admin#testserver1 -**
cat infra-config.yml
custom_attributes:
application : Apple
env : Test
datacenter : DC2
log:
file: /u00/app/monitor/infra.log
**
ssh admin#testserver2- **
custom_attributes:
application : Apple
env : Test
datacenter : DC2
log:
file: /u00/app/monitor/infra.log

Given the CSV file on the controller
shell> cat /tmp/test.csv
Application,env,Datacenter,Hostname
Microsoft,Test,DC1,testserver1
Apple,Test,DC2,testserver2
The project below
shell> tree .
.
├── ansible.cfg
├── hosts
├── infra-config.yml.j2
└── pb.yml
0 directories, 4 files
shell> cat ansible.cfg
[defaults]
gathering = explicit
collections_path = $HOME/.local/lib/python3.9/site-packages/
inventory = $PWD/hosts
roles_path = $PWD/roles
remote_tmp = ~/.ansible/tmp
retry_files_enabled = false
stdout_callback = yaml
shell> cat hosts
[test]
testserver1 ansible_host=10.1.0.61
testserver2 ansible_host=10.1.0.63
[test:vars]
ansible_connection=ssh
ansible_user=admin
ansible_become=yes
ansible_become_user=root
ansible_become_method=sudo
ansible_python_interpreter=/usr/local/bin/python3.8
ansible_perl_interpreter=/usr/local/bin/perl
The playbook
shell> cat pb.yml
- hosts: all
tasks:
- block:
- read_csv:
path: /tmp/test.csv
key: Hostname
register: newrelic
- debug:
var: newrelic
delegate_to: localhost
run_once: true
- template:
src: infra-config.yml.j2
dest: /tmp/infra-config.yml
vars:
_data: "{{ newrelic.dict[inventory_hostname] }}"
shell> cat infra-config.yml.j2
custom_attributes:
application : {{ _data.Application }}
env : {{ _data.env }}
datacenter : {{ _data.Datacenter }}
log:
file: /u00/app/monitor/infra.log
creates the configuration files at the remote hosts
shell> ansible-playbook pb.yml
PLAY [all] ***********************************************************************************
TASK [read_csv] ******************************************************************************
ok: [testserver1 -> localhost]
TASK [debug] *********************************************************************************
ok: [testserver1 -> localhost] =>
newrelic:
changed: false
dict:
testserver1:
Application: Microsoft
Datacenter: DC1
Hostname: testserver1
env: Test
testserver2:
Application: Apple
Datacenter: DC2
Hostname: testserver2
env: Test
failed: false
list: []
TASK [template] ******************************************************************************
changed: [testserver1]
changed: [testserver2]
PLAY RECAP ***********************************************************************************
testserver1: ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
testserver2: ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
shell> ssh admin#10.1.0.61 cat /tmp/infra-config.yml
custom_attributes:
application : Microsoft
env : Test
datacenter : DC1
log:
file: /u00/app/monitor/infra.log
shell> ssh admin#10.1.0.63 cat /tmp/infra-config.yml
custom_attributes:
application : Apple
env : Test
datacenter : DC2
log:
file: /u00/app/monitor/infra.log

Related

How can i change YAML format to INI format?

I want to change from YAML format to INI format like this
How can I change?
tomcat_users:
- username: "admin"
password: "admin"
roles: "tomcat,admin,manager,manager-gui"
I try like this
[example]
127.0.0.1
[example:vars]
tomcat_user="{'username':'admin', 'password':'admin', 'roles':'tomcat, admin, manager, manager-gui'}"
See Templating. For example, create the template
shell> cat tomcat_users.ini.j2
[example]
127.0.0.1
[example:vars]
{% for user in tomcat_users %}
tomcat_user='{{ user|to_json }}'
{% endfor %}
and the task
- template:
src: tomcat_users.ini.j2
dest: /tmp/tomcat_users.ini
will create the file
shell> cat /tmp/tomcat_users.ini
[example]
127.0.0.1
[example:vars]
tomcat_user='{"username": "admin", "password": "admin", "roles": "tomcat,admin,manager,manager-gui"}'
Example of a complete playbook for testing
shell> cat pb.yml
- hosts: localhost
vars:
tomcat_users:
- username: "admin"
password: "admin"
roles: "tomcat,admin,manager,manager-gui"
tasks:
- template:
src: tomcat_users.ini.j2
dest: /tmp/tomcat_users.ini
The created file is a valid Ansible inventory file
shell> ansible-inventory -i /tmp/tomcat_users.ini --list --yaml
all:
children:
example:
hosts:
127.0.0.1:
tomcat_user: '{"username": "admin", "password": "admin", "roles": "tomcat,admin,manager,manager-gui"}'
ungrouped: {}
The playbook below
shell> cat pb2.yml
- hosts: example
gather_facts: false
tasks:
- debug:
var: tomcat_user
gives
shell> ansible-playbook -i /tmp/tomcat_users.ini pb2.yml
PLAY [example] *******************************************************************************
TASK [debug] *********************************************************************************
ok: [127.0.0.1] =>
tomcat_user:
password: admin
roles: tomcat,admin,manager,manager-gui
username: admin
PLAY RECAP ***********************************************************************************
127.0.0.1: ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

How to execute on multiple hosts in ansible

I have a script that will execute in two parts. First it will execute on localhost and query a database table to get a hostname. second part of the script should run on the host which was registered in the query before. I am not able to set the host with the set_fact I did in the first part of the code.
this is what iam trying to do:
- hosts: localhost
gather_facts: false
become: yes
become_user: oracle
vars_files:
- vars/main.yml
tasks:
- name: Get new hostname
tempfile:
state: file
register: tf
- name: create sql file
template:
src: get_hostname.sql.j2
dest:"{{ tf.path }}"
mode: 0775
- name: login
command:
argv:
- "sqlplus"
- -s
- "#{{ tf.path }}"
environment:
ORACLE_HOME: "oracle/home"
register: command_out
- set_fact:
NEW_HOST: "{{ command_out.stdout }}"
- hosts: "{{ NEW_HOST }}"
gather_facts: false
become: yes
become_user: oracle
vars_file:
- vars/main.yml
tasks:
- name: debug
command: hostname
register: new_host_out
- debug:
msg: "new host is {{ new_host_out.stdout }}"
Everything works fine in the first part of the code, but errors out at the second part saying it cannot find the NEW_HOST.
Use hostvars to reference such a variable. Create a dummy host to keep this variable. For example, given the inventory
shell> cat hosts
dummy
[test]
test_11
test_12
test_13
The playbook creates the variable. See Delegated facts
shell> cat pb.yml
- hosts: localhost
tasks:
- set_fact:
NEW_HOST: test_12
delegate_to: dummy
delegate_facts: true
- debug:
var: hostvars.dummy.NEW_HOST
- hosts: "{{ hostvars.dummy.NEW_HOST }}"
gather_facts: false
tasks:
- debug:
var: inventory_hostname
gives
shell> ansible-playbook pb.yml
PLAY [localhost] ****************************************************************************
TASK [set_fact] *****************************************************************************
ok: [localhost -> dummy]
TASK [debug] ********************************************************************************
ok: [localhost] =>
hostvars.dummy.NEW_HOST: test_12
PLAY [test_12] ******************************************************************************
TASK [debug] ********************************************************************************
ok: [test_12] =>
inventory_hostname: test_12
PLAY RECAP **********************************************************************************
localhost: ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test_12 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You can use localhost for this purpose as well. The playbook below works as expected
- hosts: localhost
tasks:
- set_fact:
NEW_HOST: test_12
- hosts: "{{ hostvars.localhost.NEW_HOST }}"
gather_facts: false
tasks:
- debug:
var: inventory_hostname

How store full host name as variable in ansible?

I need to use one out of two hosts as a variable. I do have inventory_hostname_short of both but I need a full host as a variable. Currently, for testing I am using a hardcoded value. My playbook will run on both hosts at a same time so How I can identify and store as a variable.
host_1_full = 123.abc.de.com
host_2_full = 345.abc.de.com
above both are hosts and I do have
---
- name: Ansible Script
hosts: all
vars:
host1_short : '123'
host2_short : '345'
tasks:
- name: set host
set_fact:
host1_full: "{{inventory_hostname}}"
when: inventory_hostname_short == host1_short
- name: print info
debug:
msg: "host - {{host1_full}}"
- name: block1
block:
- name:running PS1 file
win_shell: "script.ps1"
register: host1_output
when: inventory_hostname_short == host1_short
- name: block2
block:
- name: set host
set_fact:
IN_PARA: "{{ hostvars[host1_full]['host1_output']['stdout']}}"
- name:running PS1 file
win_shell: "main.ps1 -paramater {{ IN_PARA }}"
register: output
when: inventory_hostname_short == host2_short
SO to access any file from different host required full hostname. How can I get that full host name
Given the following inventories/test_inventory.yml
---
all:
hosts:
123.abc.de.com:
345.abc.de.com:
ansible will provide the needed result in inventory_hostname automagically as demonstrated by the following test.yml playbook
---
- name: print long and short inventory name
hosts: all
gather_facts: false
tasks:
- name: print info
debug:
msg: "Host full name is {{ inventory_hostname }}. Short name is {{ inventory_hostname_short }}"
which gives:
$ ansible-playbook -i inventories/test_inventory.yml test.yml
PLAY [print long and short inventory name] *********************************************************************************************************************************************************************************************
TASK [print info] **********************************************************************************************************************************************************************************************************************
ok: [345.abc.de.com] => {
"msg": "Host full name is 345.abc.de.com. Short name is 345"
}
ok: [123.abc.de.com] => {
"msg": "Host full name is 123.abc.de.com. Short name is 123"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
123.abc.de.com : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
345.abc.de.com : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
As already suggested, the ansible_hostname and ansible_fqdn are automatic facts about hosts in your inventory. However, your requirement is a little unique. So we might have to apply a unique method to accomplish this. How about something like this?
Consider example inventory as below:
---
all:
hosts:
192.168.1.101: # 123.abc.de.com
192.168.1.102: # 345.abc.de.com
We can have a play like this:
- hosts: all
vars:
host1_short: 123
host2_short: 345
tasks:
- command: "hostname -f"
register: result
- block:
- set_fact:
host1_full: "{{ result.stdout }}"
- debug:
msg: "Host1 fullname: {{ host1_full }}"
when: host1_short in result.stdout

Running Environment variable at Ansible Play level question

I have following commands declared in main playbook file:
---
- name: Install config
hosts: all
become: yes
become_method: sudo
become_user: root
roles:
- initial_config
environment:
http_proxy: http://{{ProxyHost}}:{{ProxyPort}}
https_proxy: http://{{ProxyHost}}:{{ProxyPort}}
ftp_proxy: http://{{ProxyHost}}:{{ProxyPort}}
Inside my group_vars/ all: I have declared these parameter:
ProxyHost: proxy.test.com
ProxyPort: 9999
no_proxy: 'test.com'
But I am receiving error when i run main playbook locally saying:
ERROR! Syntax Error while loading YAML.
found unexpected end of stream
The play below
shell> cat group_vars/all
ProxyHost: proxy.test.com
ProxyPort: 9999
no_proxy: test.com
shell> cat pb.yml
- hosts: test_01
environment:
http_proxy: "http://{{ ProxyHost }}:{{ ProxyPort }}"
https_proxy: "http://{{ ProxyHost }}:{{ ProxyPort }}"
ftp_proxy: "http://{{ ProxyHost }}:{{ ProxyPort }}"
tasks:
- shell: set | grep proxy
register: result
- debug:
var: result.stdout_lines
gives
shell> ansible-playbook pb.yml
PLAY [test_01] **********
TASK [shell] ************
changed: [test_01]
TASK [debug] ************
ok: [test_01] => {
"result.stdout_lines": [
"SUDO_COMMAND='/bin/sh -c echo BECOME-SUCCESS-hhojceebldohrrfvivcndrtjtvtrzgfg ; http_proxy=http://proxy.test.com:9999 https_proxy=http://proxy.test.com:9999 ftp_proxy=http://proxy.test.com:9999 /usr/local/bin/python3.7 /home/admin/.ansible/tmp/ansible-tmp-1588775688.2160842-43227888427690/AnsiballZ_command.py'",
"ftp_proxy=http://proxy.test.com:9999",
"http_proxy=http://proxy.test.com:9999",
"https_proxy=http://proxy.test.com:9999"
]
}
PLAY RECAP **********
test_01: ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Execute playbook with different sudo users on different hosts

There are two hosts in the host list DEV and QA host
I need to execute playbook with dev_sudo_user for development env and qa_sudo_user for QA environment parallely
inventory details
[hostlist]
host1 ansible_become_user=dev_sudo_user
host2 ansible_become_user=qa_sudo_user
---
- hosts: hostlist
connection: ssh
gather_facts: false
remote_user: abcd
serial: 1
become: true
tasks:
- name: run Script
shell: python apps.py
register: result
- debug: var=result
I am able to get result for one host. I want result for both hosts
you have to use the become and become_user arguments. see example and output below.
hosts files with the variable as you had:
[test_group]
rhel-green become_user=root
rhel-blue become_user=devops
playbook:
- hosts: test_group
gather_facts: false
tasks:
- name: step 1
shell: "id"
register: shell_output
become_user: "{{ become_user }}"
become: true
- debug:
var: shell_output.stdout
execution output:
[root#ansible]# ansible-playbook -i hosts become_loop.yml
PLAY [test_group] ***************************************************************************************************************************************************************************************************
TASK [step 1] *******************************************************************************************************************************************************************************************************
changed: [rhel-blue]
changed: [rhel-green]
TASK [debug] ********************************************************************************************************************************************************************************************************
ok: [rhel-green] => {
"shell_output.stdout": "uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023"
}
ok: [rhel-blue] => {
"shell_output.stdout": "uid=1000(devops) gid=1000(devops) groups=1000(devops) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023"
}
PLAY RECAP **********************************************************************************************************************************************************************************************************
rhel-blue : ok=2 changed=1 unreachable=0 failed=0
rhel-green : ok=2 changed=1 unreachable=0 failed=0
[root#ansible]#

Resources