Ansible, EC2 and Instance Profile - amazon-ec2

I am trying to create via ansible with the following code:
– name: Create EC2 Instance(s)
ec2:
region: “{{ vpc_region }}”
instance_profile_name: “{{ instance_profile_name }}”
group: “{{ ec2_security_group_name }}”
keypair: “{{ ec2_key_name }}”
…..
Still it always output the following:
TASK [scanner : Create EC2 Instance(s)] ****************************************
fatal: [127.0.0.1]: FAILED! => {“changed”: false, “failed”: true, “msg”: “Instance creation failed => InvalidParameterValue: Value (my-role-for-ansible) for parameter iamInstanceProfile.name is invalid. Invalid IAM Instance Profile name”}
Although i think i defined the right policies to my user in AWS as follows:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt14844231360000",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::11111111:role/my-role-for-ansible"
]
}
]
}
Anything wrong with the above?

This occurs because you must create an additional IAM role, an instance profile.
You can attach an IAM instance profile to an Amazon EC2 instance as you launch it or to a previously launched instance. For more information, see Instance Profiles.
I resolved this issue by following the console instructions section of these AWS docs.
The key part is step 9 of the aforementioned instructions (linked above):
On the Create role page, choose AWS service, and from the Choose the
service that will use this role list, choose EC2.
Once you associate the role with the specific service type (in my case, EC2), then it all works.

In my case, Packer was throwing this error, and it was because a matching role did not exist.
For once an AWS doc was actually helpful: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html

RootInstanceProfile:
Type: 'AWS::IAM::InstanceProfile'
Properties:
Path: /
Roles:
- !Ref RootRole
RootRole:
Type: 'AWS::IAM::Role'
ec2:
IamInstanceProfile: !Ref RootInstanceProfile
If you were creating everything with cloudformation, you need a role, but you can't reference the role directly. You have to make an instance profile that references the role. Then in the ec2 properties you can reference the instance profile in the IamInstanceProfile property.

Related

Get AWS RDS Aurora cluster endpoint using Ansible

I need to be able to get the cluster endpoint for an existing AWS RDS Aurora cluster using Ansible by providing the "DB identifier" of the cluster.
When using community.aws.rds_instance_info in my playbook and referencing the DB instance identifier of the writer instance:
---
- name: Test
hosts: localhost
connection: local
tasks:
- name: Get RDS Aurora cluster
community.aws.rds_instance_info:
db_instance_identifier: "test-cluster-1" # the writer instance of the aurora db cluster
register: rds_aurora_cluster
It returns that instance as expected.
But if I use the cluster endpoint (test-cluster) it does not return any instances, or any cluster-level information:
ok: [localhost] => {
"changed": false,
"instances": [],
"invocation": {
"module_args": {
"aws_access_key": "<omitted>",
"aws_ca_bundle": null,
"aws_config": null,
"aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"db_instance_identifier": "test-cluster",
"debug_botocore_endpoint_logs": false,
"ec2_url": null,
"filters": null,
"profile": null,
"region": "us-east-1",
"security_token": null,
"validate_certs": true
}
}
}
I've also tried using the amazon.aws.aws_rds module in the amazon.aws.rds collection, which has an include_clusters parameter:
---
- name: Test
hosts: localhost
connection: local
vars:
collections:
- amazon.aws
tasks:
- name: Get RDS Aurora cluster
aws_rds:
db_instance_identifier: "test-cluster"
include_clusters: true
register: rds_aurora_cluster
When I run that playbook I get:
ERROR! couldn't resolve module/action 'aws_rds'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/Users/username/Desktop/test/test.yml': line 23, column 7, but may be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Get RDS Aurora cluster
^ here
I've confirmed that the latest version of the collection is installed:
❯ ansible-galaxy collection list
# /usr/local/Cellar/ansible/4.4.0/libexec/lib/python3.9/site-packages/ansible_collections
Collection Version
----------------------------- -------
amazon.aws 2.0.0
And I have verified the package:
❯ ansible-galaxy collection verify amazon.aws
Downloading https://galaxy.ansible.com/download/amazon-aws-2.0.0.tar.gz to /Users/username/.ansible/tmp/ansible-local-8367rrxw073b/tmpejakna6g/amazon-aws-2.0.0-_y4d1bqj
Verifying 'amazon.aws:2.0.0'.
Installed collection found at '/usr/local/Cellar/ansible/4.4.0/libexec/lib/python3.9/site-packages/ansible_collections/amazon/aws'
MANIFEST.json hash: 1286503f7bcc6dd26aecf9bec4d055e8f0d2e355522f97b620522a5aa754cb9e
Successfully verified that checksums for 'amazon.aws:2.0.0' match the remote collection.
I've also tried using the amazon.aws.aws_rds module in the amazon.aws.rds collection, which has an include_clusters parameter:
One will observe from the documentation you linked to that the aws_rds is an inventory plugin and not a module; it's unfortunate that they have a copy-paste error at the top alleging that one can use it in a playbook, but the examples section shows the correct usage by putting that yaml in a file named WHATEVER.aws_rds.yaml and then confirming the selection by running ansible-inventory -i ./WHATEVER.aws_rds.yaml --list
Based solely upon some use of grep -r, it seems that inventory plugin or command: aws rds describe-db-clusters ... are the only two provided mechanisms that are aurora-aware
Working example:
test.aws_rds.yml inventory file:
plugin: aws_rds
regions:
- us-east-1
include_clusters: true
test.yml playbook, executed with ansible-playbook test.yml -i ./test.aws_rds.yml:
---
- name: Test
hosts: localhost
connection: local
tasks:
- name: test
ansible.builtin.shell:
cmd: echo {{ hostvars['test-cluster'].endpoint }}

resize type of ec2 with ansible

I want to resize a ec2 type from ansible.
This is my code:
- name: resize the instance
ec2:
aws_access_key: "{{ aws_access_key_var }}"
aws_secret_key: "{{ aws_secret_key_var }}"
region: "{{ region }}"
instance_ids:
- "{{ instance_id }}"
instance_type: "{\"Value\": \"t2.small\"}"
wait: True
register: ec2_result_file
But I get this error:
fatal: [localhost]: FAILED! => {"changed": false, "msg": "image parameter is required for new instance"}
I try with command line all good
aws ec2 modify-instance-attribute --region reg --instance-id i-xx --instance-type "{\"Value\": \"t2.small\"}
Regards,
How to arrive at the solution:
Ansible tells you it wants to create "a new instance", but you already provided an existing instance ID.
Go to the docs for the ec2 module and check the argument in which you provided the ID of the current instance:
instance_ids list of instance ids, currently used for states: absent, running, stopped
Check what state you specified - you did not, so it's the default.
Check the same docs what is the default for state argument: it is present.
present is not listed in the instance_ids description, so the instance_ids is completely ignored.
Ansible thinks you really wanted to create a new instance.
Solution:
Add state: running to the ec2 module arguments.

How to change an existing AWS VPC resource with Ansible?

I have just created a vpc on AWS with ansible like so:
- name: Ensure VPC is present
ec2_vpc:
state: present
region: "{{ ec2_region }}"
cidr_block: "{{ ec2_subnet }}"
subnets:
- cidr: "{{ ec2_subnet }}"
route_tables:
- subnets:
- "{{ ec2_subnet }}"
routes:
- dest: 0.0.0.0/0
gw: igw
internet_gateway: yes # we want our instances to connect internet
wait: yes # wait for the VPC to be in state 'available' before returning
resource_tags: { Name: "my_project", environment: "production", tier: "DB" } # we tag this VPC so it will be easier for us to find it later
Note that this task is called from another playbook that takes care of filling the variables.
Also note that I am aware the ec2_vpc module is deprecated and I should update this piece of code soon.
But I think those points are not relevant to the question.
So you can notice in the resource_tags I have a name for that project. When I first ran this playbook, I did not have a name.
The VPC got created successfully the first time but then I wanted to add a name to it. So I tried adding it in the play, without knowing exactly how Ansible would know I want to update the existing VPC.
Indeed Ansible created a new VPC and did not update the existing one.
So the question is, how do I update the already existing one instead of creating a new resource?
After a conversation on the ansible irc (can't remember the usernames, sorry) it appears this specific module is not good at updating resources.
It looks like the answer lies in the new ec2_vpc_net module, specifically with the option multi_ok which by default won't duplicate VPCs with the same name and CIDR blocks.
docs: http://docs.ansible.com/ansible/ec2_vpc_net_module.html#ec2-vpc-net

Allocate EC2 through Ansible

When creating an EC2 instance through Ansible, how do you specify the security group (this is for a Amazon VPC environment that is not the account default)? In my case, I am attempting to assign a security group (that currently exists) to my webserver EC2 instance that restricts traffic to only the traffic coming from the ELB that sits in front of it. If I try the following:
- name: Create webserver instance
ec2:
key_name: "{{ project_name }}-{{ env }}-key"
image: "{{ image }}"
instance_type: "{{ instance_type }}"
instance_tags: '{"Name":"{{ project_name }}-{{ env }}-{{ webserver_name }}","Owner":"{{ project_name }}", "Type":"{{ webserver_name }}","Environment":"{{ env }}"}'
region: "{{ aws_region }}"
group: "{{ security_group_name }}"
wait: true
register: ec2
where {{ security_group_name }} is the 'Group Name' found in the AWS console, I receive the following error: 'Value () for parameter groupId is invalid. The value cannot be empty'
If I try the following:
- name: Create webserver instance
ec2:
key_name: "{{ project_name }}-{{ env }}-key"
image: "{{ image }}"
instance_type: "{{ instance_type }}"
instance_tags: '{"Name":"{{ project_name }}-{{ env }}-{{ webserver_name }}","Owner":"{{ project_name }}", "Type":"{{ webserver_name }}","Environment":"{{ env }}"}'
region: "{{ aws_region }}"
group_id: "{{ security_group_id }}"
wait: true
register: ec2
where {{ security_group_id }} is the 'Group Id' found in the AWS console (such as sg-xxxxxx), I receive the same error. The Ansible documentation stated that 'group' and 'group_id' are for the specification of the security group (http://docs.ansible.com/ansible/ec2_module.html).
The only thing I can think of is that AWS cannot find the security group because I am creating it and it does not know the VPC to place it in or it is placing it in my default VPC and cannot find the security group as it is in a different VPC.
So, maybe a better question is, how to I specify the VPC for a particular EC2 instance (when I have multiple VPCs in a region)?
I receive the following error: 'Value () for parameter groupId is invalid. The value cannot be empty'
It sounds like those variables aren't defined; where are you defining them? Does it work when you don't use a variable, but put in the value directly?
So, maybe a better question is, how to I specify the VPC for a particular EC2 instance (when I have multiple VPCs in a region)?
The VPC is implied by other values, notably group/group_id and vpc_subnet_id.
One of the main advantages of using group_id instead of group is that you'll get an error if you accidentally use a subnet for the wrong VPC, whereas if you have security groups with the same names in multiple VPCs, using group and an incorrect vpc_subnet_id will successfully launch a machine in the wrong place.
There is no way to specify the VPC ID in your playbook. Instead you specify the vpc_subnet_id. AWS can figure out the VPC based on subnet id. You are not specifying the subnet id, so AWS assumes that you want to launch it in EC2 Classic. Since the specified security group is not found in EC2 Classic, you are getting the not found or invalid error.
How to fix this? Find the id of the subnet where you want to launch the instance. Each VPC can have one more public and/or private subnets. From AWS dashboard, select VPC Service which will list the subnets and VPCs. Find the subnet you want to launch in, and specify its id in the playbook.
vpc_subnet_id: subnet-29e63245

Unable to create EC2 when subnet is used via Ansible(same works through AWS-CLI)

I am trying to create EC2 instance using ansible. If I try this without the subnet(and default security group), it works perfect and creates EC2. But this is not I want. I want to create instance using a specific 'sg' and using the subnet that's already existing(defined by my organization).
Same subnet and 'sg' works fine when using AWS-CLI(and via console too), same profile, same image, same key and same instance type. It creates instance under my subnet and assigns the sg passed in the command - Perfect!! Can we rule out the access/role related issues here(as CLI/console works fine)? If so, what else issue can be with Ansible/boto?
AWS CLI:
aws ec2 run-instances --image-id ami-3d401234 --count 1 --instance-type t2.large --region us-east-1 --key-name MyKeyNameHere --security-group-ids sg-766b1234 --subnet-id subnet-09871234 --profile MyProfileNameHere
Here is the playbook.
- name: Provision an EC2 node
hosts: local
connection: local
gather_facts: false
tags: provisioning
vars:
instance_type: t2.large
image: ami-3d401234
group_id: sg-766b1234
region: us-east-1
keypair: MyKeyNameHere
vpc_subnet_id: subnet-09871234
tasks:
- name: Launch new Instance
local_action: ec2 instance_tags="Name=MyInstance"
instance_type={{ instance_type}}
image={{ image }}
wait=true
group_id={{ group_id }}
profile=MyProfileNameHere
region={{ region }}
vpc_subnet_id={{ vpc_subnet_id }}
keypair={{ keypair }}
register: ec2
And here is the error, not sure why 401 again(got this earlier when profile was not mentioned in the playbook). I am sure access and secret keys are correct because i am able to create with default sg.
vpc_id = vpc.get_all_subnets(subnet_ids=[vpc_subnet_id])[0].vpc_id
File "/Library/Python/2.7/site-packages/boto-2.38.0-py2.7.egg/boto/vpc/__init__.py", line 1153, in get_all_subnets
return self.get_list('DescribeSubnets', params, [('item', Subnet)])
File "/Library/Python/2.7/site-packages/boto-2.38.0-py2.7.egg/boto/connection.py", line 1186, in get_list
raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 401 Unauthorized
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>6182f17d-f62e-4d57-b351-3498dc8a53b7</RequestID></Response>
And i have the access key and secret key information in ~/.boto file. Just the aws_access_key_id and aws_secret_access_key. No IAM role information, etc.
This was missing in boto, aws_security_token. Never thought this would be needed as i was already passing the access and secret keys. I guess it is needed as i am given access part of an organization group? I added this and it works now. Thanks #helloV for making me check the differences again with you comment :-)

Resources