How to obtain all available EC2 key pairs in boto3?
import boto3
ec2 = boto3.resource('ec2')
import boto3
client = boto3.client('ec2', 'ap-southeast-2')
keypairs = client.describe_key_pairs()
See: boto3 describe_key_pairs() documentation
Please note that, for security reasons, you cannot download the keypairs themselves. You'll simply be given their name and fingerprint.
Related
I am trying to launch an EMR cluster using AWS Lambda code written with boto3 and python. The Lambda is able to launch the cluster when there is no VPC configuration associated it. However, as soon as I add the VPC config it fails to launch the cluster and errors out and does not provide any error message.
I am trying to launch the lambda inside a default VPC and it has 3 public subnets and a default security group. I have checked the route table in the VPC is associated with an internet gateway and it is attached to the VPC.
The execution role provides full access to the cloudwatch elasticmapreduce and ec2 actions.
Any help in resolving this school boy error will be much appreciated.
I have a VPC enabled Lambda function which attempts to launch an EC2 using a launch template. The EC2 launch step (run_instances) fails with the below generic network error.
Calling the invoke API action failed with this message: Network Error
I can launch an instance successfully directly from the launch template, so I think everything is fine with the launch template. I have configured the following in the launch template
Amazon Machine Image ID
Instance type
Key Pair
A network interface (ENI) which I had created before using a specific (VPC, Subnet, Secutity Group) combo.
IAM role
The Lambda function includes the below code-
import json
import boto3
import time
def lambda_handler(event, context):
ec2_cl = boto3.client('ec2')
launch_temp = {"LaunchTemplateId": "<<Launch Template ID>>"}
resp_ec2_launch = ec2_cl.run_instances(MaxCount=1, MinCount=1, LaunchTemplate=launch_temp, SubnetId="<<Subnet ID>>")
Few things on the Lambda function-
I have used the subnet in the run_instances() call because this is not the default vpc/subnet.
The function is setup with the same (VPC, Subnet, Secutity Group) combo as used in the launch template
The execution role is setup to be the same IAM role as used in the launch template
The function as you see needs access only to the EC2, internet access is not needed
I replaced the run_instances() with describe_instance_status (using the instance id created directly from the launch template) and got the same error.
The error is a network error, so I assume all is fine (atleast as of now) with the privileges granted to the IAM role. I'm sure there would be a different error, if the IAM role missed any policies.
Can someone indicate what I might be missing?
It appears that the problem is with your AWS Lambda function being able to reach the Internet, since the Amazon EC2 API endpoint is on the Internet.
If a Lambda function is not attached to a VPC, it has automatic access to the Internet.
If a Lambda function is attached to a VPC and requires Internet access, then the configuration should be:
Attach the Lambda function only to private subnet(s)
Launch a NAT Gateway in a public subnet
Configure the Route Table on the private subnets to send Internet-bound traffic (0.0.0.0/0) through the NAT Gateway
It appears that your VPC does not have an Internet Gateway, but it does have a VPC Endpoint for EC2.
Therefore, to try and reproduce your situation, I did the following:
Created a new VPC with one subnet but no Internet Gateway
Added a VPC Endpoint for EC2 to the subnet
Created a Lambda function that would call DescribeInstances() and attached the Lambda function to the subnet
Opened the security group on the VPC Endpoint and Lambda function to allow all traffic from anywhere (hey, it's just a test!)
My Lambda function:
import json
import boto3
def lambda_handler(event, context):
ec2 = boto3.client('ec2',region_name='ap-southeast-2')
print(ec2.describe_instances())
The result: The Lambda function successfully received a response from EC2, with a list of instances in the region. No code or changes were required.
I am trying to connect my Lambda to RDS just as a learning exercise.
Currently, all resources are created through CloudFormation and I would like to continue to do that if possible.
My issue is with the following statement from https://docs.aws.amazon.com/lambda/latest/dg/vpc-rds.html which details how connect.
A second file contains connection information for the function.
Example rds_config.py
#config file containing credentials for RDS MySQL instance
db_username = "username"
db_password = "password"
db_name = "ExampleDB"
The statement AWS is making makes it seem like I should hardcode these values into a file which does not seem secure. I could try to use environment variables but I think the same issue will arise.
If anyone has any advice for how to connect lambda to rds securely I would greatly appreciate it!!!
If you don't want to use environment variables for whatever reason, you can have your Lambda function query the AWS Systems Manager Parameter Store for you.
So let's say once your function has been triggered, you can just query SSM to get the desired parameters and then pass it into your RDS connection.
Just remember that if your Lambda also needs Internet Access (and in this case it does, because it will need to access SSM), you'll need to attach 2 subnets to it: a private and a public. The private will route traffic to RDS and the public will route traffic to other AWS Services / or to the internet.
Setting up Environment Variables would be the easiest to get you off ground, though.
EDIT: Check this answer where I walk the OP through creating a VPC with both public and private subnets if you need a quick start.
EDIT 2: Good news. AWS has released VPC endpoints for SSM some time ago. So your Lambda won't need to go through the Internet anymore, you can just hit that VPC endpoint. You can see it in the official docs
I am running some python code on an AWS EC2 instance. The IAM role for this EC2 instance has adequate privileges to describe and create new EC2 instances.
My code calls ec2_client.describe_instances() and checks whether a specific instance is alive. If that instance is terminated, it will call ec2_client.run_instances and create a new instance.
However, when I call ec2_client.run_instances(), I get an error Could not connect to the endpoint URL: "https://ec2.us-west-2.amazonaws.com/".
ec2_client is a boto3 EC2 client object.
In some other examples, I've seen people getting this error because they had an incorrect region. That is not the case here. I've made sure that the region is correct.
Any ideas why this happens?
I am using boto3 (1.4.4).
Having created a boto connection with IAM user credentials, I'm not able to get Elastic IPs are being allocated by the same user:
conn = boto.ec2.connect_to_region(region, **iam_user_only_credentials)
addresses = con.get_all_addresses(filters={'public_ip': "my EIP address"})
# addresses array is empty here
Are Elastic IP addresses only associated to root credentials (AWS root credentials)? Is it possible to associate them to a user? (as mine was created being myself an IAM user -- w/ pretty much full access)
I agree. It appears that boto2 does not return addresses as documented.
Fortunately, it does function in boto3:
import boto3
client = boto3.client('ec2')
client.describe_addresses(Filters=[{'Name':'public-ip','Values':['54.XX.XX.XX']}])