trying to find and add hosts dynamically like so
---
- hosts: localhost
gather_facts: no
tasks:
- name: Gather EC2 remote facts.
ec2_remote_facts:
region: 'us-east-1'
register: ec2_remote_facts
- name: Debug.
debug:
msg: "{{ ec2_remote_facts }}"
- name: get instances for tags
add_host:
name: "{{ item }}"
group: dynamically_created_hosts
with_items: |
"{{ ec2_remote_facts.instances |
selectattr('tags.AppName', 'defined') | selectattr('tags.AppName', 'equalto', 'sql') |
selectattr('tags.AppType', 'defined') | selectattr('tags.AppType', 'equalto', 'infra') |
map(attribute='private_ip_address') | list }}"
- hosts:
- dynamically_created_hosts
become: yes
become_user: root
serial: 1
vars_files:
- group_vars/all
tasks:
- name: run command
shell: "uname -a"
I get following when i run in verbose mode
TASK [get instances for tags] **************************************************
task path: /Users/me/gitfork2/fornax/dynhst.yml:39
creating host via 'add_host': hostname="[u'10.112.114.241']"
changed: [localhost] => (item="[u'10.112.114.241']") => {"add_host": {"groups": ["dynamically_created_hosts"], "host_name": "\"[u'10.112.114.241']\"", "host_vars": {"group": "dynamically_created_hosts"}}, "changed": true, "invocation": {"module_args": {"group": "dynamically_created_hosts", "hostname": "\"[u'10.112.114.241']\""}, "module_name": "add_host"}, "item": "\"[u'10.112.114.241']\""}
PLAY [dynamically_created_hosts] ***********************************************
TASK [setup] *******************************************************************
<"[u'10.112.114.241']"> ESTABLISH SSH CONNECTION FOR USER: None
<"[u'10.112.114.241']"> SSH: ansible.cfg set ssh_args: (-F)(/Users/me/.ssh/config)
<"[u'10.112.114.241']"> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
<"[u'10.112.114.241']"> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<"[u'10.112.114.241']"> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<"[u'10.112.114.241']"> SSH: PlayContext set ssh_common_args: ()
<"[u'10.112.114.241']"> SSH: PlayContext set ssh_extra_args: ()
<"[u'10.112.114.241']"> SSH: EXEC ssh -C -vvv -F /Users/me/.ssh/config -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 '"[u'"'"'10.112.114.241'"'"']"' '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1509843772.58-176166317659656 `" && echo ansible-tmp-1509843772.58-176166317659656="` echo $HOME/.ansible/tmp/ansible-tmp-1509843772.58-176166317659656 `" ) && sleep 0'"'"''
fatal: ["[u'10.112.114.241']"]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh.", "unreachable": true}
to retry, use: --limit #./dynhst.retry
The odd thing here I see is
SSH: EXEC ssh -C -vvv -F /Users/me/.ssh/config -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 '"[u'"'"'10.112.114.241'"'"']"' '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1509843772.58-176166317659656 `" && echo ansible-tmp-1509843772.58-176166317659656="` echo $HOME/.ansible/tmp/ansible-tmp-1509843772.58-176166317659656 `" ) && sleep 0'"'"''
Seems like it is trying to ssh into '"[u'"'"'10.112.114.241'"'"']"' ... seems like the dynamically_created_hosts is being used as a string and not as a list
Any ideas why?
You pass a list (of IP addresses) to an argument name which requires a string:
hostname="[u'10.112.114.241']"
[ ] is a JSON representation of a list (single element in the example above).
If you want the first address from the list (and there seems to be no more for any of your hosts), then:
add_host:
name: "{{ item[0] }}"
group: dynamically_created_hosts
with_items: ...
Related
Setup
Passwords, not keys are used for ssh
The target is accessed via the bastion host (ssh -> bastion -> target)
The password is kept in an encrypted file that is accessed based on the ansible_user name
When decrypted, the password is bastion
Command
$ ansible -i ./inventory/debug -m debug -a 'var=foo' -kKu ansible all
Command Output
target | FAILED! => {
"msg": "The field 'ssh_common_args' has an invalid value, which includes an undefined variable. The error was: 'ansible_user' is undefined"
}
bastion | SUCCESS => {
"foo": "-o ProxyCommand=\"sshpass -p bastion ssh -o StrictHostKeyChecking=no -W %h:%p -q ansible#3.21.247.xxx.\""
}
Problem
The password is correctly accessed and resolved for foo for bastion, but foo cannot be resolved for the target host.
The inventory file (called "debug")
all:
vars:
env: 3.21.247.xxx
password: "{{lookup('file', inventory_dir + '/../users/' + ansible_user + '.yml')}}"
foo: "-o ProxyCommand=\"sshpass -p {{password}} ssh -o StrictHostKeyChecking=no -W %h:%p -q ansible#{{env}}.\""
children:
bastions:
hosts:
bastion:
ansible_host: "{{ env }}"
nv:
children:
targets:
hosts:
target:
ansible_host: 10.0.3.209
vars:
ansible_ssh_common_args: "{{ foo }}"
You cannot use jinja2 expansion inside jinja2 expansion. Try:
"{{lookup('file', inventory_dir + '/../users/' + ansible_user + '.yml')}}"
I would like to test if a user is able to SSH using SSH password. That's all I would like to do. I tried with modules: local_action, wait_for but those didn't get me the results. The playbook result must simply tell me where a connection succeeded or failed when trying to SSH.
The requirement is to test which user account succeeds in making a SSH connection to remote servers. The user who would be running the ansible script has multiple accounts on these servers but SSH login will succeed with just the right one which the user doesn't know. The user accounts all have the same password.
The inventory file:
all:
children:
FXO-Test:
hosts:
host1.abcd.com:
host2.abcd.com:
vars:
ansible_user: user1
The Playbook:
---
- hosts: "{{ targethosts }}"
gather_facts: no
tasks:
- name: Test connection
local_action: command ssh -q -o BatchMode=yes -o ConnectTimeout=3 {{ inventory_hostname }}
register: test_user
ignore_errors: true
changed_when: false
Invoked Using Command:
ansible-playbook checkLogin.yml -i ans_inventory_test --ask-pass --extra-vars "targethosts=FXO-Test" | tee verify_user.log
Expected to see which SSH connections failed and which ones worked.
Based on Vladimir Botka response, I tweaked the playbook a bit further to pull hostnames from an inventory file.
My Updated Playbook 'verifySSHLogin.yml':
- hosts: localhost
gather_facts: no
vars:
my_users:
- user1
- user2
my_hosts: "{{ query('inventory_hostnames', 'all') }}"
tasks:
- expect:
command: "ssh -o PubkeyAuthentication=no -o StrictHostKeyChecking=no {{ item.0 }}#{{ item.1 }}"
timeout: 2
responses:
(.*)password(.*):
- "password" # Fit the password
- "\x03" # Ctrl-C
(.*)\$(.*): "exit" # Fit the prompt
loop: "{{ my_users|product(my_hosts)|list }}"
register: result
ignore_errors: yes
- debug:
msg: "{{ (item.rc == 0)|ternary(item.invocation.module_args.command ~ ' [OK]',item.invocation.module_args.command ~ ' [KO]') }}"
loop: "{{ result.results }}"
Which I now invoke using below command:
ansible-playbook verifySSHLogin.yml -i ans_inventory_test --extra-vars "targethosts=FXO-Test" | tee verify_user.log
I can then do a grep against verify_user.log like this:
grep '\"msg\": \"ssh' verify_user.log
Which gives me below result which is what I was expecting:
"msg": "ssh -o PubkeyAuthentication=no -o StrictHostKeyChecking=no user1#host1.abc.corp.com [OK]"
"msg": "ssh -o PubkeyAuthentication=no -o StrictHostKeyChecking=no user1#host2.abc.corp.com [OK]"
"msg": "ssh -o PubkeyAuthentication=no -o StrictHostKeyChecking=no user1#host3.abc.corp.com [KO]"
"msg": "ssh -o PubkeyAuthentication=no -o StrictHostKeyChecking=no user2#host1.abc.corp.com [KO]"
"msg": "ssh -o PubkeyAuthentication=no -o StrictHostKeyChecking=no user2#host2.abc.corp.com [KO]"
"msg": "ssh -o PubkeyAuthentication=no -o StrictHostKeyChecking=no user2#host3.abc.corp.com [KO]"
Tweaked the playbook further to avoid hard-coding of SSH password. The final playbook looks like now:
- hosts: localhost
gather_facts: no
vars:
my_users:
- user1
- user2
my_hosts: "{{ query('inventory_hostnames', 'all') }}"
tasks:
- expect:
command: "ssh -o PubkeyAuthentication=no -o StrictHostKeyChecking=no {{ item.0 }}#{{ item.1 }}"
timeout: 2
responses:
(.*)password(.*):
- "{{ ansible_password }}" # Fit the password
- "\x03" # Ctrl-C
(.*)\$(.*): "exit" # Fit the prompt
loop: "{{ my_users|product(my_hosts)|list }}"
register: result
ignore_errors: yes
- debug:
msg: "{{ (item.rc == 0)|ternary(item.invocation.module_args.command ~ ' [OK]',item.invocation.module_args.command ~ ' [KO]') }}"
loop: "{{ result.results }}"
The SSH password can be passed to ansible-playbook command like this:
ansible-playbook verifySSHLogin.yml -i ans_inventory_test -k --extra-vars "targethosts=FXO-Test" | tee verify_user.log
expect module shall do the job. Given the
user1#test_01 is able to log in, the play below
- hosts: localhost
vars:
my_users:
- user1
- user2
my_hosts:
- test_01
- test_02
tasks:
- expect:
command: "ssh {{ item.0 }}#{{ item.1 }}"
timeout: 2
responses:
(.*)password(.*):
- "password" # Fit the password
- "\x03" # Ctrl-C
(.*)\$(.*): "exit" # Fit the prompt
with_nested:
- "{{ my_users }}"
- "{{ my_hosts }}"
register: result
ignore_errors: yes
- debug:
msg: "{{ (item.rc == 0)|ternary(item.invocation.module_args.command ~ ' [OK]',
item.invocation.module_args.command ~ ' [KO]') }}"
loop: "{{ result.results }}"
gives (grep msg):
"msg": "ssh user1#test_01 [OK]"
"msg": "ssh user1#test_02 [KO]"
"msg": "ssh user2#test_01 [KO]"
"msg": "ssh user2#test_02 [KO]"
I am not able to execute shell script remotely in Ansible. However, there are previous tasks in the same role (filebeat) that are executed in remote server successfully. I am running the following in local server 172.28.28.6 server to install and run filebeat in remote server 172.28.28.81
Playbook: install-filebeat.yml:
hosts: filebeat-servers
remote_user: wwwadm
sudo: yes
roles:
- { role: /vagrant/roles/filebeat}
Role filebeat: main.yml:
---
# tasks file for filebeat
- name: "Extract Filebeat"
unarchive:
src: "{{ tmp_artifact_cache }}/{{ filebeat_archive }}"
remote_src: yes
dest: "{{ filebeat_root_dir }}"
extra_opts: ['--transform=s,/*[^/]*,{{ filebeat_ver }},i', '--show-stored-names']
become: yes
become_user: "{{ filebeat_install_as }}"
when: not ansible_check_mode
tags: [ 'filebeat' ]
- name: Configure Filebeat
template:
src: "filebeat.yml.j2"
dest: "{{ filebeat_install_dir }}/filebeat.yml"
mode: 0775
become: yes
become_user: "{{ filebeat_install_as }}"
tags: [ 'filebeat' ]
- name: 'Filebeat startup script'
template:
src: "startup.sh.j2"
dest: "{{ filebeat_install_dir }}/bin/startup.sh"
mode: 0755
become: yes
become_user: "{{ filebeat_install_as }}"
tags: [ 'filebeat', 'start' ]
#This one does not get executed at all:
- name: "Start Filebeat"
# shell: "{{ filebeat_install_dir }}/bin/startup.sh"
command: "sh {{ filebeat_install_dir }}/bin/startup.sh"
become: yes
become_user: "{{ filebeat_install_as }}"
defaults:
# defaults file for filebeat
filebeat_ver: "6.6.0"
filebeat_archive: "filebeat-{{ filebeat_ver }}-linux-x86_64.tar.gz"
filebeat_archive_checksum : "sha1:d38d8fea7e9915582720280eb0118b7d92569b23"
filebeat_url: "https://artifacts.elastic.co/downloads/beats/filebeat/{{ filebeat_archive }}"
filebeat_root_dir: "{{ apps_home }}/filebeat"
filebeat_data_dir: "{{ apps_data }}/filebeat"
filebeat_log_dir: "{{ apps_logs }}/filebeat"
filebeat_install_dir: "{{ filebeat_root_dir }}/{{ filebeat_ver }}"
filebeat_cert_dir: "/etc/pki/tls/certs"
filebeat_ssl_certificate_file: "logstash.crt"
filebeat_ssl_key_file: "logstash.key"
filebeat_install_as: "{{ install_user | default('wwwadm') }}"
filebeat_set_as_current: yes
filebeat_force_clean_install: no
filebeat_java_home: "{{ sw_home }}/jdk"
inventory/local/hosts:
localhost ansible_connection=local
[filebeat-servers]
172.28.28.81 ansible_user=vagrant ansible_connection=ssh
Filebeat is installed and changes are done in the remote server except the last step which is the execution of shell script
When running the playbook as follows:
ansible-playbook -i /vagrant/inventory/local install-filebeat.yml -vvv
Getting the following output related to the shell execution:
TASK [/vagrant/roles/filebeat : Start Filebeat] ***************************************************************************************************************************************************************
task path: /vagrant/roles/filebeat/tasks/main.yml:184
<172.28.28.81> ESTABLISH SSH CONNECTION FOR USER: vagrant
<172.28.28.81> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/vagrant/.ansible/cp/f66f05c055 172.28.28.81 '/bin/sh -c '"'"'echo ~vagrant && sleep 0'"'"''
<172.28.28.81> (0, '/home/vagrant\n', '')
<172.28.28.81> ESTABLISH SSH CONNECTION FOR USER: vagrant
<172.28.28.81> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/vagrant/.ansible/cp/f66f05c055 172.28.28.81 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /var/tmp/ansible-tmp-1550178583.24-35955954120606 `" && echo ansible-tmp-1550178583.24-35955954120606="` echo /var/tmp/ansible-tmp-1550178583.24-35955954120606 `" ) && sleep 0'"'"''
<172.28.28.81> (0, 'ansible-tmp-1550178583.24-35955954120606=/var/tmp/ansible-tmp-1550178583.24-35955954120606\n', '')
Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py
<172.28.28.81> PUT /home/vagrant/.ansible/tmp/ansible-local-13658UX7cBC/tmpFzf2Ll TO /var/tmp/ansible-tmp-1550178583.24-35955954120606/AnsiballZ_command.py
<172.28.28.81> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/vagrant/.ansible/cp/f66f05c055 '[172.28.28.81]'
<172.28.28.81> (0, 'sftp> put /home/vagrant/.ansible/tmp/ansible-local-13658UX7cBC/tmpFzf2Ll /var/tmp/ansible-tmp-1550178583.24-35955954120606/AnsiballZ_command.py\n', '')
<172.28.28.81> ESTABLISH SSH CONNECTION FOR USER: vagrant
<172.28.28.81> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/vagrant/.ansible/cp/f66f05c055 172.28.28.81 '/bin/sh -c '"'"'setfacl -m u:wwwsvr:r-x /var/tmp/ansible-tmp-1550178583.24-35955954120606/ /var/tmp/ansible-tmp-1550178583.24-35955954120606/AnsiballZ_command.py && sleep 0'"'"''
<172.28.28.81> (0, '', '')
<172.28.28.81> ESTABLISH SSH CONNECTION FOR USER: vagrant
<172.28.28.81> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/vagrant/.ansible/cp/f66f05c055 -tt 172.28.28.81 '/bin/sh -c '"'"'sudo -H -S -n -u wwwsvr /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-ntzchfzqggiteuqwzpiurlloddbdhevp; /usr/bin/python /var/tmp/ansible-tmp-1550178583.24-35955954120606/AnsiballZ_command.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<172.28.28.81> (0, '\r\n{"changed": true, "end": "2019-02-14 13:09:44.800191", "stdout": "Starting Filebeat", "cmd": ["sh", "/apps_ux/filebeat/6.6.0/bin/startup.sh"], "rc": 0, "start": "2019-02-14 13:09:43.792122", "stderr": "+ export JAVA_HOME=/sw_ux/jdk\\n+ JAVA_HOME=/sw_ux/jdk\\n+ echo \'Starting Filebeat\'\\n+ /apps_ux/filebeat/6.6.0/bin/filebeat -c /apps_ux/filebeat/6.6.0/config/filebeat.yml -path.home /apps_ux/filebeat/6.6.0 -path.config /apps_ux/filebeat/6.6.0/config -path.data /apps_data/filebeat -path.logs /apps_data/logs/filebeat", "delta": "0:00:01.008069", "invocation": {"module_args": {"warn": true, "executable": null, "_uses_shell": false, "_raw_params": "sh /apps_ux/filebeat/6.6.0/bin/startup.sh", "removes": null, "argv": null, "creates": null, "chdir": null, "stdin": null}}}\r\n', 'Shared connection to 172.28.28.81 closed.\r\n')
<172.28.28.81> ESTABLISH SSH CONNECTION FOR USER: vagrant
<172.28.28.81> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/vagrant/.ansible/cp/f66f05c055 172.28.28.81 '/bin/sh -c '"'"'rm -f -r /var/tmp/ansible-tmp-1550178583.24-35955954120606/ > /dev/null 2>&1 && sleep 0'"'"''
<172.28.28.81> (0, '', '')
changed: [172.28.28.81] => {
"changed": true,
"cmd": [
"sh",
"/apps_ux/filebeat/6.6.0/bin/startup.sh"
],
"delta": "0:00:01.008069",
"end": "2019-02-14 13:09:44.800191",
"invocation": {
"module_args": {
"_raw_params": "sh /apps_ux/filebeat/6.6.0/bin/startup.sh",
"_uses_shell": false,
"argv": null,
"chdir": null,
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"warn": true
}
},
"rc": 0,
"start": "2019-02-14 13:09:43.792122",
"stderr": "+ export JAVA_HOME=/sw_ux/jdk\n+ JAVA_HOME=/sw_ux/jdk\n+ echo 'Starting Filebeat'\n+ /apps_ux/filebeat/6.6.0/bin/filebeat -c /apps_ux/filebeat/6.6.0/config/filebeat.yml -path.home /apps_ux/filebeat/6.6.0 -path.config /apps_ux/filebeat/6.6.0/config -path.data /apps_data/filebeat -path.logs /apps_data/logs/filebeat",
"stderr_lines": [
"+ export JAVA_HOME=/sw_ux/jdk",
"+ JAVA_HOME=/sw_ux/jdk",
"+ echo 'Starting Filebeat'",
"+ /apps_ux/filebeat/6.6.0/bin/filebeat -c /apps_ux/filebeat/6.6.0/config/filebeat.yml -path.home /apps_ux/filebeat/6.6.0 -path.config /apps_ux/filebeat/6.6.0/config -path.data /apps_data/filebeat -path.logs /apps_data/logs/filebeat"
],
"stdout": "Starting Filebeat",
"stdout_lines": [
"Starting Filebeat"
]
}
META: ran handlers
META: ran handlers
PLAY RECAP ****************************************************************************************************************************************************************************************************
172.28.28.81 : ok=18 changed=7 unreachable=0 failed=0
On remote server:
[6.6.0:vagrant]$ cd bin
[bin:vagrant]$ ls -ltr
total 36068
-rwxr-xr-x. 1 wwwadm wwwadm 36927014 Jan 24 02:30 filebeat
-rwxr-xr-x. 1 wwwadm wwwadm 478 Feb 14 12:54 startup.sh
[bin:vagrant]$ pwd
/apps_ux/filebeat/6.6.0/bin
[bin:vagrant]$ more startup.sh
#!/usr/bin/env bash
set -x
export JAVA_HOME="/sw_ux/jdk"
#To save pid into a file is an open feature: https://github.com/elastic/logstash/issues/3577. There is no -p flag for filebeat to save the pid and then kill it.
echo 'Starting Filebeat'
/apps_ux/filebeat/6.6.0/bin/filebeat -c /apps_ux/filebeat/6.6.0/config/filebeat.yml -path.home /apps_ux/filebeat/6.6.0 -path.config /apps_ux/filebeat/6.6.0/config -path.data /apps_data/filebeat -path.logs /a
pps_data/logs/filebeat &
No process running found by executing ps command
[bin:vagrant]$ ps -fea | grep filebeat | grep -v grep
However, if I connect to the remote server, I am able to run filebeat by executing the script with the user wwwadm and filebeat starts successfully:
[bin:wwwadm]$ pwd
/apps_ux/filebeat/6.6.0/bin
[bin:wwwadm]$ id
uid=778(wwwadm) gid=778(wwwadm) groups=778(wwwadm) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[bin:wwwadm]$ ./startup.sh
+ export JAVA_HOME=/sw_ux/jdk
+ JAVA_HOME=/sw_ux/jdk
+ echo 'Starting Filebeat'
Starting Filebeat
+ /apps_ux/filebeat/6.6.0/bin/filebeat -c /apps_ux/filebeat/6.6.0/config/filebeat.yml -path.home /apps_ux/filebeat/6.6.0 -path.config /apps_ux/filebeat/6.6.0/config -path.data /apps_data/filebeat -path.logs /apps_data/logs/filebeat
[bin:wwwadm]$ ps -fea | grep filebeat | grep -v grep
wwwadm 19160 1 0 15:12 pts/0 00:00:00 /apps_ux/filebeat/6.6.0/bin/filebeat -c /apps_ux/filebeat/6.6.0/config/filebeat.yml -path.home /apps_ux/filebeat/6.6.0 -path.config /apps_ux/filebeat/6.6.0/config -path.data /apps_data/filebeat -path.logs /apps_data/logs/filebeat
Thanks
You should use nohup to run it in background.
because when ansible exits, all processes associated with the session
will be terminated. To avoid this you should use nohup.
Correct command is:
- name: "Start Filebeat"
# shell: "{{ filebeat_install_dir }}/bin/startup.sh"
command: "nohup sh {{ filebeat_install_dir }}/bin/startup.sh &>> startup.log &"
become: yes
become_user: "{{ filebeat_install_as }}"
You have to use the disown built-in command to inform the shell that it should not kill background processes when you disconnect; you can also use nohup for that same effect
Having said that, you are for sure solving the wrong problem, because if^H^Hwhen filebeat falls over, there is nothing monitoring that service to keep it alive. You'll want to use systemd (or its equivalent on your system) to ensure that filebeat stays running, and by using the mechanism designed for that stuff, you side-step all the "disown or nohup" business that causes you to ask S.O. questions.
I am trying to use a variable and the results of a previous task as input. Here is my code.
---
- hosts: localhost
vars:
v_hostname: XXXXXXXXXXXXX
v_datacentre: DC-UK-LON-GS
v_username: YYYYYYYY
v_password: XXXXXXXX
connection: local
gather_facts: false
tasks:
- name: Find the folder for the VM
vmware_guest_find:
hostname: "{{ v_hostname }}"
datacenter: "{{ v_datacentre }}"
username: "{{ v_username }}"
password: "{{ v_password }}"
validate_certs: no
name: MYVMNAME
register: folder
- debug:
msg:
- "{{ folder.folders }}"
- name: Gather facts from VM
connection: local
gather_facts: false
vmware_guest_facts:
hostname: "{{ v_hostname }}"
datacenter: "{{ v_datacentre }}"
username: "{{ v_username }}"
password: "{{ v_password }}"
validate_certs: no
folder: "{{ v_datacentre }}{{ folder.folders }}"
name: MYVMNAME
The last line tries to use the defined variable v_datacentre and append the value from the registered fact from the previous task. However the concatenation produces some unwanted wrapping chars:
ansible-playbook 2.4.2.0
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/export/home/ansible/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /bin/ansible-playbook
python version = 2.7.5 (default, May 3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]
Using /etc/ansible/ansible.cfg as config file
Parsed /export/home/ansible/inventory_prod inventory source with ini plugin
[WARNING]: Ignoring invalid attribute: gather_facts
PLAYBOOK: mainvm.yml *****************************************************************************************************************************************
1 plays in mainvm.yml
PLAY [localhost] *********************************************************************************************************************************************
META: ran handlers
TASK [TEST1] *************************************************************************************************************************************************
task path: /export/home/ansible/mainvm.yml:12
Using module file /usr/lib/python2.7/site-packages/ansible/modules/cloud/vmware/vmware_guest_find.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: ansible
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /export/home/ansible/.ansible/tmp/ansible-tmp-1523621823.18-119950066241401 `" && echo ansible-tmp-1523621823.18-119950066241401="` echo /export/home/ansible/.ansible/tmp/ansible-tmp-1523621823.18-119950066241401 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpM_Pf2B TO /export/home/ansible/.ansible/tmp/ansible-tmp-1523621823.18-119950066241401/vmware_guest_find.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /export/home/ansible/.ansible/tmp/ansible-tmp-1523621823.18-119950066241401/ /export/home/ansible/.ansible/tmp/ansible-tmp-1523621823.18-119950066241401/vmware_guest_find.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /export/home/ansible/.ansible/tmp/ansible-tmp-1523621823.18-119950066241401/vmware_guest_find.py; rm -rf "/export/home/ansible/.ansible/tmp/ansible-tmp-1523621823.18-119950066241401/" > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
"changed": false,
"folders": [
"/vm/LON-GS-AD/LON-GS-AD-UNIX"
],
"invocation": {
"module_args": {
"datacenter": "DC-UK-LON-GS",
"hostname": "XXXXXXXXXXXXX",
"name": "MYVMNAME",
"password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"username": "XXXXXXXX",
"uuid": null,
"validate_certs": false
}
}
}
TASK [debug] *************************************************************************************************************************************************
task path: /export/home/ansible/mainvm.yml:23
ok: [localhost] => {
"msg": [
[
"/vm/LON-GS-AD/LON-GS-AD-UNIX"
]
]
}
TASK [Gather facts from standalone ESXi server] **************************************************************************************************************
task path: /export/home/ansible/mainvm.yml:28
Using module file /usr/lib/python2.7/site-packages/ansible/modules/cloud/vmware/vmware_guest_facts.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: ansible
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /export/home/ansible/.ansible/tmp/ansible-tmp-1523621827.79-78942665802166 `" && echo ansible-tmp-1523621827.79-78942665802166="` echo /export/home/ansible/.ansible/tmp/ansible-tmp-1523621827.79-78942665802166 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmp43NXf_ TO /export/home/ansible/.ansible/tmp/ansible-tmp-1523621827.79-78942665802166/vmware_guest_facts.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /export/home/ansible/.ansible/tmp/ansible-tmp-1523621827.79-78942665802166/ /export/home/ansible/.ansible/tmp/ansible-tmp-1523621827.79-78942665802166/vmware_guest_facts.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /export/home/ansible/.ansible/tmp/ansible-tmp-1523621827.79-78942665802166/vmware_guest_facts.py; rm -rf "/export/home/ansible/.ansible/tmp/ansible-tmp-1523621827.79-78942665802166/" > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
"changed": false,
"invocation": {
"module_args": {
"datacenter": "XXXXXX",
"folder": "DC-UK-LON-GS[u'/vm/LON-GS-AD/LON-GS-AD-UNIX']",
"hostname": "XXXXXXXXXXXXX",
"name": "MYVMNAME",
"name_match": "first",
"password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"username": "XXXXXXXX",
"uuid": null,
"validate_certs": false
}
},
"msg": "Unable to gather facts for non-existing VM MYVMNAME"
}
to retry, use: --limit #/export/home/ansible/mainvm.retry
PLAY RECAP ***************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=1
The results of folder.folders is correctly printed by debug but in the last step the concatenation gives:
"folder": "DC-UK-LON-GS[u'/vm/LON-GS-AD/LON-GS-AD-UNIX']",
I confess to being a bit of an Ansible newbie but I don't understand where the [u' and ] come from. I just want to use the combined strings as input. Can anyone please explain or show solution or point me at some beginners documents for this sort of variable use?
Thanks.
So after some further playing turns out that the vmware_guest_find returns a list (and it's in the docs!!!) so I need to either pick the 1st item if there will only be one group or loop though them.
folder: "{{ v_datacentre }}{{ folder.folders[0] }}"
This does the job.
I have the following playbook:
- hosts: localhost
connection: local
remote_user: test
gather_facts: no
vars_files:
- files/aws_creds.yml
- files/info.yml
environment:
AWS_ACCESS_KEY_ID: "{{ aws_id }}"
AWS_SECRET_ACCESS_KEY: "{{ aws_key }}"
s3cmd_access_key: "{{ aws_id }}"
s3cmd_secret_key: "{{ aws_key }}"
tasks:
- name: Basic provisioning of EC2 instance
ec2:
assign_public_ip: no
aws_access_key: "{{ aws_id }}"
aws_secret_key: "{{ aws_key }}"
region: "{{ aws_region }}"
image: "{{image_instance }}"
instance_type: "{{ free_instance }}"
key_name: "{{ ssh_keyname }}"
count: 3
state: present
group_id: "{{ secgroup_id }}"
vpc_subnet_id: "{{ private_subnet_id }}"
wait: no
instance_tags:
Name: Dawny33Template
#delete_on_termination: yes
register: ec2
- name: Add new instance to host group
add_host:
hostname: "{{ item.private_ip }}"
groupname: launched
with_items: "{{ ec2.instances }}"
- name: Wait for SSH to come up
wait_for:
host: "{{ item.private_ip }}"
port: 22
delay: 60
timeout: 320
state: started
with_items: "{{ ec2.instances }}"
- hosts: launched
sudo: true
remote_user: test
gather_facts: yes
vars_files:
- files/aws_creds.yml
- files/info.yml
environment:
AWS_ACCESS_KEY_ID: "{{ aws_id }}"
AWS_SECRET_ACCESS_KEY: "{{ aws_key }}"
s3cmd_access_key: "{{ aws_id }}"
s3cmd_secret_key: "{{ aws_key }}"
tasks:
- name: Add file system for the volume
command: mkfs -t ext4 /dev/xvdb
sudo: yes
- name: Create a directory for mounting
command: mkdir /home/ec2-user/EncryptedEBS
- name: Mount the volume
command: mount /dev/xvdb /home/ec2-user/EncryptedEBS
sudo: yes
- name: Owning the mounted folder
command: chown ec2-user /home/ec2-user/EncryptedEBS/lost+found/
sudo: yes
- name: check out a git repository
git: repo={{ repo_url }} dest=/home/ec2-user/EncryptedEBS/GitRepo accept_hostkey=yes force=yes
vars:
repo_url: https://github.com/Dawny33/AnsibleExperiments
become: yes
- name: Go to the folder and execute command
command: chmod 0755 /home/ec2-user/EncryptedEBS/GitRepo/processing.py
become: yes
become_user: root
- name: Run Py script
command: /home/ec2-user/EncryptedEBS/GitRepo/processing.py {{ N }} {{ bucket_name }}
become: yes
become_user: root
However, I get the "Permission denied" error, when Ansible tries to connect to my remote hosts, even though I have defined the env. variables in environment
Is there anything which I did wrong here?
Error:
fatal: [10.0.1.62]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n",
"unreachable": true
}
fatal: [10.0.1.177]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n",
"unreachable": true
}
fatal: [10.0.1.151]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n",
"unreachable": true
}
Adding the complete -vvv output:
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/core/system/setup.py
<10.0.1.170> ESTABLISH SSH CONNECTION FOR USER: ec2-user
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/core/system/setup.py
<10.0.1.11> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<10.0.1.170> SSH: EXEC ssh -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ec2-user -o ConnectTimeout=10 10.0.1.170 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" && echo ansible-tmp-1487158610.11-137345507492691="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" ) && sleep 0'"'"''
<10.0.1.11> SSH: EXEC ssh -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ec2-user -o ConnectTimeout=10 10.0.1.11 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" && echo ansible-tmp-1487158610.11-2307895121172="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" ) && sleep 0'"'"''
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/core/system/setup.py
<10.0.1.45> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<10.0.1.45> SSH: EXEC ssh -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ec2-user -o ConnectTimeout=10 10.0.1.45 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" && echo ansible-tmp-1487158610.12-3620848798638="` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" ) && sleep 0'"'"''
<10.0.1.170> ssh_retry: attempt: 0, ssh return code is 255. cmd (/bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" && echo ansible-tmp-1487158610.11-137345507492691="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" ) && sleep 0'...), pausing for 0 seconds
<10.0.1.170> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<10.0.1.170> SSH: EXEC ssh -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ec2-user -o ConnectTimeout=10 10.0.1.170 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" && echo ansible-tmp-1487158610.11-137345507492691="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" ) && sleep 0'"'"''
<10.0.1.11> ssh_retry: attempt: 0, ssh return code is 255. cmd (/bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" && echo ansible-tmp-1487158610.11-2307895121172="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" ) && sleep 0'...), pausing for 0 seconds
<10.0.1.11> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<10.0.1.11> SSH: EXEC ssh -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ec2-user -o ConnectTimeout=10 10.0.1.11 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" && echo ansible-tmp-1487158610.11-2307895121172="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" ) && sleep 0'"'"''
<10.0.1.45> ssh_retry: attempt: 0, ssh return code is 255. cmd (/bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" && echo ansible-tmp-1487158610.12-3620848798638="` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" ) && sleep 0'...), pausing for 0 seconds
<10.0.1.45> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<10.0.1.45> SSH: EXEC ssh -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ec2-user -o ConnectTimeout=10 10.0.1.45 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" && echo ansible-tmp-1487158610.12-3620848798638="` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" ) && sleep 0'"'"''
<10.0.1.170> ssh_retry: attempt: 1, ssh return code is 255. cmd (/bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" && echo ansible-tmp-1487158610.11-137345507492691="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" ) && sleep 0'...), pausing for 1 seconds
<10.0.1.11> ssh_retry: attempt: 1, ssh return code is 255. cmd (/bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" && echo ansible-tmp-1487158610.11-2307895121172="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" ) && sleep 0'...), pausing for 1 seconds
<10.0.1.45> ssh_retry: attempt: 1, ssh return code is 255. cmd (/bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" && echo ansible-tmp-1487158610.12-3620848798638="` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" ) && sleep 0'...), pausing for 1 seconds
<10.0.1.170> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<10.0.1.170> SSH: EXEC ssh -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ec2-user -o ConnectTimeout=10 10.0.1.170 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" && echo ansible-tmp-1487158610.11-137345507492691="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-137345507492691 `" ) && sleep 0'"'"''
<10.0.1.11> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<10.0.1.11> SSH: EXEC ssh -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ec2-user -o ConnectTimeout=10 10.0.1.11 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" && echo ansible-tmp-1487158610.11-2307895121172="` echo ~/.ansible/tmp/ansible-tmp-1487158610.11-2307895121172 `" ) && sleep 0'"'"''
<10.0.1.45> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<10.0.1.45> SSH: EXEC ssh -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ec2-user -o ConnectTimeout=10 10.0.1.45 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" && echo ansible-tmp-1487158610.12-3620848798638="` echo ~/.ansible/tmp/ansible-tmp-1487158610.12-3620848798638 `" ) && sleep 0'"'"''
fatal: [10.0.1.11]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n",
"unreachable": true
}
fatal: [10.0.1.170]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n",
"unreachable": true
}
fatal: [10.0.1.45]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n",
"unreachable": true
}
Don't forget, when you use ec2.py you should add your pem first, like this:
ssh-add /home/yourusername/.ssh/your.pem
Here, I was not sure why the key is not even allowing manual ssh. So, I generated a new key(pem file) and worked with it. [Manual ssh worked with that file]
Now, the problem lies in the following block:
- hosts: launched
sudo: true
remote_user: test
gather_facts: yes
I edited it to be:
- hosts: launched
sudo: no
connection: ssh
remote_user: ec2-user
gather_facts: yes
and it worked. The reason must be obvious. The connection have to be an ssh and not local, and the username should be ec2-user for an Amazon Linux Instance and Ubuntu for an ubuntu instance.
Generate ssh public key using ssh-keygen tool
and copy the ~/.ssh/id_rsa.pub key into ~/.ssh/authorized_keys file.