I'm very confused on how you are supposed to launch EC2 instances using Ansible.
I'm trying to use the ec2.py inventory scripts. I'm not sure which one is supposed to be used, because there is three installed with Ansible:
ansible/lib/ansible/module_utils/ec2.py
ansible/lib/ansible/modules/core/cloud/amazon/ec2.py
ansible/plugins/inventory/ec2.py
I thought running the one in inventory/ would make most sense, so I run it using:
ansible-playbook launch-ec2.yaml -i ec2.py
which gives me:
msg: Either region or ec2_url must be specified
So I add a region (even though I have a vpc_subnet_id specified) and I get:
msg: Region us-east-1e does not seem to be available for aws module boto.ec2. If the region definitely exists, you may need to upgrade boto or extend with endpoints_path
I'm thinking Amazon must have recently changed ec2 so you need to use a VPC? Even when I try and launch an instance from Amazon's console, the option for "EC2 Classic" is disabled.
When I try and use the ec2.py script in cloud/amazon/ I get:
ERROR: Inventory script (/software/ansible/lib/ansible/modules/core/cloud/amazon/ec2.py) had an execution error:
There are no more details than this.
After some searching, I see that ec2.py module in /module_utils has been changed so a region doesn't need to be specified. I try to run this file but get:
ERROR: The file /software/ansible/lib/ansible/module_utils/ec2.py is marked as executable, but failed to execute correctly. If this is not supposed to be an executable script, correct this with chmod -x /software/ansible/lib/ansible/module_utils/ec2.py.
So as the error suggests, I remove the executable permissions for the ec2.py file, but then get the following error:
ERROR: /software/ansible/lib/ansible/module_utils/ec2.py:30: Invalid ini entry: distutils.version - need more than 1 value to unpack
Does anyone have any ideas on how to get this working? What is the correct file to be using? I'm completely lost at this point on how to get this working.
There are several questions in your post. I'll try to summarise them in three items:
Is it still possible to launch instances in EC2 Classic (no VPC)?
How do I create a new EC2 instance using Ansible?
How to launch the dynamic inventory file ec2.py?
1. EC2 Classic
Your options will differ depending on when did you create your AWS account, the type of instance and the AMI virtualisation type used. Refs: aws account,instance type.
If none of the above parameters restricts the usage of EC2 classic you should be able to create a new instance without defining any VPC.
2. Create a new EC2 instance with Ansible
Since your instance doesn't exist yet a dynamic inventory file (ec2.py) is useless. Try to instruct ansible to run on your local machine instead.
Create a new inventory file, e.g. new_hosts with the following contents:
[localhost]
127.0.0.1
Then your playbook, e.g. create_instance.yml should use a local connection and hosts: localhost. See an example below:
--- # Create ec2 instance playbook
- hosts: localhost
connection: local
gather_facts: false
vars_prompt:
inst_name: "What's the name of the instance?"
vars:
keypair: "your_keypair"
instance_type: "m1.small"
image: "ami-xxxyyyy"
group: "your_group"
region: "us-west-2"
tasks:
- name: make one instance
ec2: image={{ image }}
instance_type={{ instance_type }}
keypair={{ keypair }}
instance_tags='{"Name":"{{ inst_name }}"}'
region={{ region }}
group={{ group }}
wait=true
register: ec2_info
- name: Add instances to host group
add_host: hostname={{ item.public_ip }} groupname=ec2hosts
with_items: ec2_info.instances
- name: Wait for SSH to come up
wait_for: host={{ item.public_dns_name }} port=22 delay=60 timeout=320 state=started
with_items: ec2_info.instances
This play will create an EC2 instance and it will register its public IP as an ansible host variable ec2hosts ie. as if you had defined it in the inventory file. This is useful if you want to provision the instance just created, just add a new play with hosts: ec2hosts.
Ultimately, launch ansible as follows:
export ANSIBLE_HOST_KEY_CHECKING=false
export AWS_ACCESS_KEY=<your aws access key here>
export AWS_SECRET_KEY=<your aws secret key here>
ansible-playbook -i new_hosts create_instance.yml
The purpose of the environment variable ANSIBLE_HOST_KEY_CHECKING=false is to avoid being prompted to add the ssh host key when connecting to the instance.
Note: boto needs to be installed on the machine that runs the above ansible command.
3. Use ansible's ec2 dynamic inventory
EC2 dynamic inventory is comprised of 2 files, ec2.py and ec2.ini. In your particular case, I believe that your issue is due to the fact that ec2.py is unable to locate ec2.ini file.
To solve your issue, copy ec2.py and ec2.ini to the same folder in the machine where you intend to run ansible, e.g. to /etc/ansible/.
Pre Ansible 2.0 release (change the branch accordingly).
cd /etc/ansible
wget https://raw.githubusercontent.com/ansible/ansible/stable-1.9/plugins/inventory/ec2.py
wget https://raw.githubusercontent.com/ansible/ansible/stabe-1.9/plugins/inventory/ec2.ini
chmod u+x ec2.py
For Ansible 2:
cd /etc/ansible
wget https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.py
wget https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.ini
chmod u+x ec2.py
Configure ec2.ini and run ec2.py, which should print an ini formatted list of hosts to stdout.
Related
I am using the ansible vsphere_guest module to spin up a base windows machine on a VMWare environment. In my playbook, to do this I set Hosts: 127.0.0.1 connection: local. The reason I am doing this is I beleive im not targeting this playbook at any particular host, as I dont have one yet. I instead want to run the playbook locally.
When this runs, I get a new shiny windows server VM. What I now want to do is rename that VM's computer name. To do this I am trying to upload and run a powershell script like so rename_host.ps1 $newHostname. As I understand, I need to use the script module to do this. However, this time I want to target my brand new VM, which I get the IP address of through a fact, {{ newvm_ipaddress }}.
However, when I try and run this script with delegate_to: "{{ newvm_ipaddress}}", its trying to run as SSH. SSH wont work, im targeting a windows machine with remote powershell.
is there any way to set the connection to use winRM in the context of delegate_to? Perhaps there is a better way of doing this?
Thank you for your help
I managed to work out how to solve it. The answer is the ansible module 'add_host'. I have a play under vsphere_guest as follows. This creates a new in memory host, which can then be accessed by a different play.
- add_host group=new_machine name={{ vm_ipaddress }} ansible_connection=winrm
After this, I then have a new play that can now target this host.
- host: new_machine
Also to note, variables do not span across different hosts. The solution was to use the set_fact module in play A, which can then be accessed from within play B
-set_fact:
vm_ipaddress: "{{ hw_eth0.ipaddresses[1] }}" #hw_eth0 is the fact returned from the vsphere_guest module
What about updating the inventory with the new hosts name and with ssh winrm connection params before using delegate_to, or perhaps setting some default catch-all naming scheme with these params?
For example:
[databases]
db-[a:f].example.com:5986 ansible_user=Administrator ansible_connection=winrm ansible_winrm_server_cert_validation=ignore
I'm creating an ansible playbook that goes through a group of AWS EC2 hosts and install some basic packages. Before the playbook can execute any tasks, the playbook needs to login to each host (2 type of distros AWS Linux or Ubuntu) with correct user: {{ userXXX }} this is the part that I'm not too sure how to pass in the correct user login, it would be either ec2-user or ubuntu.
- name: setup package agent
hosts: ec2_distros_hosts
user: "{{ ansible_user_id }}"
roles:
- role: package_agent_install
I was assuming ansible_user_id would work based of the reserved variable from ansible but that is not the case here. I don't want to create 2 separate playbook for different distros, is there an elegant solution to dynamically lookup user login and used as the user: ?
Here is the failed cmd with unknown user ansible-playbook -i inventory/ec2.py agent.yml
You have several ways to accomplish your task:
1. Create ansible user with the same name on every host
If you have one, you can use user: ansible_user in your playbook.
2. Tag every host with suitable login_name
You can create a tag (e.g. login_name) for every ec2 host and specify user in it. For Ubuntu hosts – ubuntu, for AWS Linux hosts – ec2-user.
After doing so, you can use user: "{{ec2_tag_login_name}}" in your playbook – this will take username from login_name tag of the host.
3. Patch the ec2.py script for your needs
It seems there is no decent way to get exact platform name from AMI, but you can use something like this:
image_name = getattr(conn.get_image(image_id=getattr(instance,'image_id')),'name')
login_name = 'user'
if 'ubuntu' in image_name:
login_name = 'ubuntu'
elif 'amzn' in image_name:
login_name = 'ec2-user'
setattr(instance, 'image_name', image_name)
setattr(instance, 'login_name', login_name)
Paste this code just before self.add_instance(instance, region) in ec2.py with the same indentation. It fetches image name and do some guess work to define login_name. Then you can use user: "{{ec2_login_name}}" in your playbook.
You can set variables based on EC2 instance tags. If you tag instances with the distro name then you can set Ansible's ssh username for each distro via group_vars files.
Example group_vars file for Ubuntu relative to your playbook: group_vars/tag_Distro_Ubuntu.yml
---
ansible_user: ubuntu
Any instances tagged Distro: Ubuntu will connect with the ubuntu user. Create a separate group_vars file per distro tag to accommodate other distros.
I have problems with my playbook which should create new EC2 instances trough built-in module and connect to them to set some default stuff.
I went trough lot of tutorials/posts, but none of them mentioned same problem, therefor i'm asking there.
Everything, in terms of creating goes well, but when i have instances created, and successfully waited for SSH to come up. I got error which says machine is unreachable.
UNREACHABLE! => {"changed": false, "msg": "ERROR! SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh", "unreachable": true}
I tried to connect manually (from terminal to the same host) and i was successful (while the playbook was waiting for connection). I also tried to increase timeout generally in ansible.cfg. I verified that given hostname is valid (and it is) and also tried public ip instead of public DNS, but nothing helps.
basically my playbook looks like that
---
- name: create ec2 instances
hosts: local
connection: local
gather_facts: False
vars:
machines:
- { type: "t2.micro", instance_tags: { Name: "machine1", group: "some_group" }, security_group: ["SSH"] }
tasks:
- name: lunch new ec2 instances
local_action: ec2
group={{ item.security_group }}
instance_type={{ item.type}}
image=...
wait=true
region=...
keypair=...
count=1
instance_tags=...
with_items: machines
register: ec2
- name: wait for SSH to come up
local_action: wait_for host={{ item.instances.0.public_dns_name }} port=22 delay=60 timeout=320 state=started
with_items: ec2.results
- name: add host into launched group
add_host: name={{ item.instances.0.public_ip }} group=launched
with_items: ec2.results
- name: with the newly provisioned EC2 node configure basic stuff
hosts: launched
sudo: yes
remote_user: ubuntu
gather_facts: True
roles:
- common
Note: in many tutorials are results from creating ec2 instances accessed in different way, but thats probably for different question.
Thanks
Solved:
I don't know how, but it suddenly started to work. No clue. In case i will find some new info, will update this question
A couple points that may help:
I'm guessing it's a version difference, but I've never seen a 'results' key in the registered 'ec2' variable. In any case, I usually use 'tagged_instances' -- this ensures that even if the play didn't create an instance (ie, because a matching instance already existed from a previous run-through), the variable will still return instance data you can use to add a new host to inventory.
Try adding 'search_regex: "OpenSSH"' to your 'wait_for' play to ensure that it's not trying to run before the SSH daemon is completely up.
The modified plays would look like this:
- name: wait for SSH to come up
local_action: wait_for host={{ item.public_dns_name }} port=22 delay=60 timeout=320 state=started search_regex="OpenSSH"
with_items: ec2.tagged_instances
- name: add host into launched group
add_host: name={{ item.public_ip }} group=launched
with_items: ec2.tagged_instances
You also, of course, want to make sure that Ansible knows to use the specified key when SSH'ing to the remote host either by adding 'ansible_ssh_private_key_file' to the inventory entry or specifying '--private-key=...' on the command line.
With ansible, I'm trying to copy an application artifact from a remote server "artifacts_host" to a target machine, i.e. a host in my inventory. The play I'm trying to run is something like:
- name: rsync WAR artifact from artifacts host
synchronize: >
src={{ artifacts_path }}/{{ artifact_filename }}.war
dest={{ artifact_installation_dir }}
delegate_to: "{{ artifacts_host }}"
I came very close to getting this to work by using ansible-vault to encrypt a "secrets.yml" variable file with the artifact_host's public key and then installed it on the target machine's auth file like:
- name: install artifacts_host's public key to auth file
authorized_key: >
user={{ ansible_ssh_user }}
key='{{ artifacts_host_public_key }}'
sudo: yes
but the problem is that my artifacts_host cannot resolve an IP address from the FQDN that Ansible passes to it. If I was able to "inform" the artifacts_host of the IP to use (what the fqdn should resolve to) then I would be fine. I would also be fine having the task fire off on the target machine to pull from the artifacts_host, but I can't find an idempotent way of accomplishing this, nor can I figure out how to feed the target machine a login/password OR ssh key to use.
Am I just gonna have to template out a script to push to my targets???
For anyone who comes across this and has the same question, I did not really figure it out, I just decided to install the private key in the target machines' /etc/ssh directory and chmod it to 0600. I figure it's basically as secure as it could get without a transient (in-memory only) key/password and with idempotence.
I am trying to configure with ansible my EC2 instances dynamically. I am having a problem working out how to find my RDS instances. I can set key tags but ansible ec2.py doesn't pick them up (https://github.com/ansible/ansible/issues/7564). Does any one have any suggestions?
So for instance I want an RDS instance for production, staging and for just for testing.
If you mean the ansible ec2.py inventory script doesn't pick up RDS instances then yes I believe you're right, it will only find EC2 instances.
We have a similar setup with a seperate RDS instance for staging and production environments. The way we solved it was for any playbooks/roles that need to run against the mysql database, we run them against the magic host "localhost", and have the RDS endpoints set in variables. We use a separate variable file per environment and load them in at the beginning of the play.
e.g.
|--vars/
| |--staging.yml
| |--production.yml
|
|--playbook.yml
Example "production.yml" file:
---
DB_SERVER: database-endpoint.cls4o6q35lol.eu-west-1.rds.amazonaws.com
DB_PORT: 3306
DB_USER: dbusername
DB_PASSWORD: dbpassword
Example playbook that creates a database
- name: Playbook name
hosts: localhost
vars_files:
- vars/{{ env }}.yml
tasks:
- mysql_db: login_host={{ DB_SERVER }}
login_user={{ DB_USER }}
login_password={{ DB_PASSWORD }}
login_port={{ DB_PORT }}
collation=utf8_general_ci
encoding=utf8
name=databasename
state=present
Then you can just specifiy the envionrment variable when you run the playbook.
ansible-playbook playbook.yml --extra-vars "env=production"
The other answer is wrong now (if I'm reading the question correctly). In the same dir as your ec2.py, add a ec2.ini file, and add:
ec2.ini
[ec2]
rds = true
I had a similar issue but the docs clearly state that ec2.py can be used to find other resources.
Ansible Dynamic Inventory
There are other config options in ec2.ini, including cache control and destination variables. By default, the ec2.ini file is configured for all Amazon cloud services, but you can comment out any features that aren’t applicable. For example, if you don’t have RDS or elasticache, you can set them to False
Edit: However, I'd also like to highlight that even though it states all resources are supported by default, I didn't get RDS results until I specified I wanted them in the ec2.ini file.