single keypair for all regions - amazon-ec2

Can I use the same key pair file across all regions? I want to keep the ssh access as simple as possible.
I have set up one file for each region. I tried to upload the same file in different region but it did not not work as expected.

This has meanwhile been enabled by means of importing your own keypair(s), see the introductory post New Amazon EC2 Feature: Bring Your Own Keypair:
You can now import your own RSA keypair (or the public half, to be
precise) for use with your Amazon EC2 instances.
Why would you want to do this? Here are a couple of reasons:
Trust - By importing your own keypair you can ensure that you have
complete control over your keys.
Security -You can be confident that
your private key has never been transmitted over the wire.
Management of Multiple Regions - You can use the same public key across multiple
AWS Regions. [emphasis mine]
This approach is quite beneficial/convenient indeed - I'm using it for a while already, and it has considerably simplified EC2 usage across multiple regions and accounts.
Eric Hammond provides more details and a short tutorial in his article Uploading Personal ssh Keys to Amazon EC2:
Amazon recently launched the ability to upload your own ssh public key
to EC2 so that it can be passed to new instances when they are
launched. Prior to this you always had to use an ssh keypair that was
generated by Amazon.
Accordingly, I recommend to follow his article regarding the (few and simple) steps required to achieve this.

If you can import your own RSA public key, then you can share an Amazon generated key across regions:
Using the key you want to share, log in to an instance in the region where it was generated.
Open ~/.ssh/authorized_keys
Find the line with the key you want to share (the end of the line should show the keypair name you specified when you generated it)
Copy all of the line up to the first character of the keypair name. Should be 382 characters.
Save the copied data to a file locally (e.g. to the machine you launched the SSH session FROM).
Log on to the EC2 console and switch to the region that you want to share the key with.
Select the "Key Pairs" management
Click "Import Key Pair" and point it at the file created at step 5. You probably want to give the keypair the same name as you gave it in the region you're importing it form.
Launch instances using this keypair. You should be able to log into instances in the new region using the same private key as in the region where you generated it.

If you want to create a ssh key pair in the AWS Console and upload this key pair to all other regions.
Create a new ssh key pair in the console or use the one that you already own.
Generate a ssh public key from the ssh private key (pem > pub)
$ ssh-keygen -y -f ~/.ssh/MySSHKeyPair.pem >> ~/.ssh/MySSHKeyPair.pub
Upload the ssh public key to all regions. (remove from the list the region that already have the ssh key)
$ for region in us-east-1 us-east-2 us-west-1 us-west-2 ap-south-1 ap-northeast-2 ap-southeast-1 ap-southeast-2 ap-northeast-1 eu-central-1 eu-west-1 eu-west-2 ; do aws ec2 import-key-pair --key-name MySSHKeyPair --public-key-material file://~/.ssh/MySSHKeyPair.pub --region $region ; done
Get the list of all ssh key pairs in all regions.
$ for region in us-east-1 us-east-2 us-west-1 us-west-2 ap-south-1 ap-northeast-2 ap-southeast-1 ap-southeast-2 ap-northeast-1 eu-central-1 eu-west-1 eu-west-2 ; do aws ec2 describe-key-pairs --region $region ; done

The AWS generated key can be transferred to another region:
Log into the instance that was created with the key that you want to transfer.
Install ec2-api-tools if not already present (sudo apt-get install ec2-api-tools)
Use this command
ec2-import-keypair keypair-name --public-key-file ~/.ssh/authorized_keys --region aws-region
With above steps, I transferred the AWS created key from Singapore region to Oregon region and same key worked perfectly fine for me.
Please visit this link as well:
https://forums.aws.amazon.com/thread.jspa?threadID=52654

You can also use the aws cli to transfer the key-pair to another region.
aws ec2 import-key-pair --key-name my-key --public-key-material file://~/.ssh/id_rsa.pub --region my-region
See this issue on GitHub for more.

Create a new ssh key pair in the console or use the one that you already own.
Generate a ssh public key from the ssh private key (pem > pub)
$ ssh-keygen -y -f ~/.ssh/rainman_key.pem >> ~/.ssh/rainman_public.pub
Upload the ssh public key to all regions except the one where it exist.then Using python export it. Below is the code:
import os
regions=["us-east-1","us-east-2","us-west-1","us-west-2","ap-east-1","ap-south-1","ap-northeast-2","ap-southeast-1","ap-southeast-2","ap-northeast-1","ca-central-1" ,"eu-central-1","eu-west-1","eu-west-2","eu-west-3","eu-north-1","sa-east-1"]
for region in regions:
os.system("aws ec2 import-key-pair --key-name rainman_key --public-key-material file://rainman_public.pub --region "+region)**

Upload the pem file that I need to copy to other regions and then ...
# chmod 400 /home/ec2-user/mydec15a.pem
# ssh-keygen -y -f /home/ec2-user/mydec15a.pem >> ~/.ssh/MySSHKeyPair.pub
# aws ec2 import-key-pair --key-name mydec15a --public-key-material file://~/.ssh/MySSHKeyPair.pub --region us-west-2

You can also do this do the region list with expansion to get the list of regions dynamically...
for region in $(aws ec2 describe-regions --all-regions --query "Regions[].{Name:RegionName}" --output text | tr '\n' ' '); do
aws ec2 import-key-pair --key-name MYKEYNAME --public-key-material file://~/.ssh/MY_PUB_KEY.pub --region $region; done

Related

SSH ProxyCommand using aws SSM session manager and bash script with select command

In my company when we SSH to our AWS EC2 instances we are required to use the aws CLI session-manager plugin for auth. Using this SSH config snippet works:
Host my-aws-host
ProxyCommand bash -c "aws ssm start-session --target 'i-0abc123def456hij' \
--document-name AWS-StartSSHSession --parameters 'portNumber=22' \
--region us-west-1 --profile MAIN"
However, when the EC2 instance is relaunched, which happens semi-regularly, the 'target' instance ID changes. When this happens, all users need to update their SSH config with the new ID. We don't have any sort of DNS that resolves these instances to a static hostname unfortunately, and so would need to somehow publish the new instance ID to all interested users.
So instead I wrote a bash script (ssh_proxy_command.sh) that first queries our AWS account to grab the current instance ID based on a known tag value, and use that for the target - here's a cut-down version:
#!/bin/bash
INSTANCE_ID=$(aws ec2 describe-instances --region us-west-1 \
--filters Name=tag:Name,Values=my-server-nametag* \
--query "Reservations[*].Instances[*].{Instance:InstanceId}" --output text)
aws ssm start-session --target $INSTANCE_ID --document-name AWS-StartSSHSession --parameters 'portNumber=22' --region us-west-1 --profile MAIN
Now the SSH config looks like
Host my-aws-host
ProxyCommand bash -c "/path/to/my/ssh_proxy_command.sh %h"
This has been working fine. However, we have just started running multiple instances built from the same base image (AMI), and which use the same tags, etc. so the given describe-instances query now returns multiple instance IDs. So I tried wrapping the output returned by the query in a bash select loop, thinking I could offer the user a list of instance IDs and let them choose the one they want. This works when running the script directly, but not when it's used as the ProxyCommand. In the latter case when it reaches the select statement it prints out the options as expected, but doesn't wait for the user input - it just continues straight to the end of the script with an empty $INSTANCE_ID variable, which makes the aws ssm command fail.
I'm guessing this is a side-effect of the way SSH runs its ProxyCommands — from the ssh_config man page:
[the proxy command] is executed using the user's shell ‘exec’ directive [...]
I'm hoping I can find a way around this problem while still using SSH config and ProxyCommand, rather than resorting to a complete stand-alone wrapper around the ssh executable and requiring everyone use that. Any suggestions gratefully accepted...
host my-aws-host
ProxyCommand aws ssm start-session --target $(aws ec2 describe-instances --filter "Name=tag:Name,Values=%h" --query "Reservations[].Instances[?State.Name == 'running'].InstanceId[]" --output text) --document-name AWS-StartSSHSession --parameters portNumber=%p
The above will dynamically filter for your target-id based on host name (%h) provided so you can login using ssh my-aws-host. I personally have a prefix for all my machines in AWS so I ssh config is as follows:
host your-custom-prefix-*
ProxyCommand aws ssm start-session --target $(aws ec2 describe-instances --filter "Name=tag:Name,Values=%h" --query "Reservations[].Instances[?State.Name == 'running'].InstanceId[]" --output text) --document-name AWS-StartSSHSession --parameters portNumber=%p
This works only when the name of your machines in AWS match the host name provided.

Problem with creating EBS snapshot on server(Linux EC2 instance)

I am working on a task that required to run a script on a server, The script will grab instance id, create snapshot and run yum update -y command and reboot the server.
#!/bin/bash
set -eu
# Set Vars
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
export REGION=$(curl --silent http://169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/[a-z]$//')
export INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id)
echo $AWS_ACCOUNT_ID
echo $REGION
# Fetch VolumeId
volumeid=$(aws ec2 describe-instances --region $REGION --instance-id "$INSTANCE_ID" --filters Name=instance-state-name,Values=running --query "Reservations[*].Instances[].[BlockDeviceMappings[*].{VolumeName:Ebs.VolumeId}]" --output text)
echo $INSTANCE_ID
echo $volumeid
# Create snapshot
aws ec2 create-snapshot --region $REGION --volume-id $volumeid --description "Test-Snapshot-$INSTANCE_ID"
read -p "waiting a while to complete creation of EBS snapshot" -t 100
echo -e "\x1B[01;36m Snapshot has been created \x1B[0m"
I can get the Instance id but when I am trying to create snapshot id from Instance id, I am getting following error:
ERROR
us-east-1
An error occurred (UnauthorizedOperation) when calling the DescribeInstances operation: You are not authorized to perform this operation.
Thank you so much in advance for your support.
Your instance, and with that your script is missing the ec2:DescribeInstances permission to run the aws ec2 describe-instances command.
You should attach that permission to the instance role that is assigned to the instance (or create a new role with the permissions attached if there is none assigned yet).
Your IAM permissions do not grant access to DescribeInstances.
If you’re using an IAM role for the instance check it’s policies.
If it’s a user then make sure the credentials are being retrieved, either via aws credentials file or via environment variable

shell script to start ec2 instances and ssh and introducing delay in second command

I have written a small shell script to automate the starting and loggin in to my aws instances via terminal.
#!/bin/bash
aws ec2 start-instances --instance-ids i-070107834ab273992
public_ip=aws ec2 describe-instances --instance-ids i-070107834ab273992 \
--query 'Reservations[*].Instances[*].PublicDnsName' --output text
AWS_KEY="/home/debian/cs605 data management/assignment6/mumbai instance keys"
ssh -v -i "$AWS_KEY"/mumbai-instance-1.pem\
ec2-user#$public_ip
~
~
The problem is public_ip variable I want it to be used in line ssh
1) how do I get value of a variable to use in a command.
2) The instance takes some time to boot when it is switched on from power off to power on so how do I keep checking that instances has been powered on after aws start instance command in the script or retrieve the public ip once it has started fully and then ssh into it.
I am not good at python know just basics so is there a pythonic way of doing it.If there is an example script some where that would be better for me to have a look at it.
You do not set the variable public_ip in your script. It would not surprise me if the script complained about "ec2: command not found".
To set the variable:
public_ip=$(aws ec2 describe-instances --instance-ids i-070107834ab273992 --query 'Reservations[*].Instances[*].PublicDnsName' --output text)
(disclaimer: I have not used aws so I assume that the command is correct).
The information on whether an instance is running should be available with
aws ec2 describe-instance-status
You may want to apply some filters and/or grep for a specific result. You could try polling with a while loop:
while ! aws ec2 describe-instance-statusv --instance-ids i-070107834ab273992 | grep 'something that characterizes running' ; do
sleep 5
done

How to share private key with other developer

I am very new to AWS.I am working on a POC, where I get request from Developers for provisioning the instance in EC2. Once instance provisioned, developer would expect to share the private key to access the instance.
I am using terraform to
1.provision,
2.generate key pair
3.Output it and store it in secret manager.
Next step is to , share the private key to developer so that he can access only his instance.
Sending private key through, deemed bad idea.
Is there any best solutions or channel to share the private key?
It is not a good idea to share a private key at all.
If all developers use the same private key:
you cannot simply deactivate the key of a developer that leaves the company.
you cannot see who does what on the server.
You can copy the public key of the developers to the default user's (ec2-user, ubuntu...) ~/.ssh/authorized_keys file. So each user uses his/her own private key to connect to the server. This approach however does not solve the issue #2 above.
ssh -i dev1.pub ec2-user#instance-ip
The recommended way is to create a new user for each developer on the instance and copy the public key of each user to authorized_users. Depending on what you want to achieve, this method has more administrative burden but is more secure.
e.g. Created a user dev1, uploaded his public key to /home/dev1/.ssh/authorized_users. Now dev1 can connect using:
ssh -i dev1.pub dev1#instance-ip
For small organizations, to allow multiple users to get access to AWS EC2 Linux instances without having to share keys or accounts is always a challenge.Definitely sharing keys across multiple users is not good practice.
The public / private key pair is generated on your local machine and the private key is uploaded to S3. When launching the EC2 instance via the wizard, you can now choose to Proceed without a key pair.
For Linux / Mac users :
To create Public and Private keys use the following command
$ ssh-keygen -t rsa -b 4096 (This creates a 4096 bit RSA key pair)
Upload the public key to a folder in your S3 bucket. For example :
S3 > MyBucket > Keypair
Save and secure your private key.
For Windows users :
Use puttygen to generate the keys.
Follow DigitalOcean to create SSH keys.
Upload the public key to S3 > MyBucket > Keypair
Save and secure your private key.
The following steps are important during the launch of any Linux AMI.
Ensure the IAM role has a role created with AmazonS3FullAccess policy. This allows the instance to assume a role to access the S3 buckets. This is needed to read the public keys from S3 and copy them to the user profile.
Add the following code under the user-data section in Configure Instance details > Advanced Details (as Text) :
# FOR AWS LINUX #
#!/bin/bash
useradd user1
usermod -aG wheel user1
mkdir /home/user1/.ssh/
aws s3 cp s3://MyBucket /Keypair/user1-pub.pub /home/user1/.ssh/authorized_keys
useradd user2
usermod -aG wheel user2
mkdir /home/user2/.ssh/
aws s3 cp s3://MyBucket /Keypair/user2-pub.pub /home/user2/.ssh/authorized_keys
sudo -i
echo “user1 ALL=(ALL) NOPASSWD:ALL” >> /etc/sudoers
echo “user2 ALL=(ALL) NOPASSWD:ALL” >> /etc/sudoers
yum update -y
# FOR UBUNTU #
#!/bin/bash
apt-get install -y awscli
useradd user1
usermod -aG sudo user1
mkdir /home/user1/.ssh/
aws s3 cp s3://MyBucket /Keypair/user1-pub.pub /home/user1/.ssh/authorized_keys
useradd user2
usermod -aG sudo user2
mkdir /home/user2/.ssh/
aws s3 cp s3://MyBucket /Keypair/user2-pub.pub /home/user2/.ssh/authorized_keys
sudo -i
echo “user1 ALL=(ALL) NOPASSWD:ALL” >> /etc/sudoers
echo “user2 ALL=(ALL) NOPASSWD:ALL” >> /etc/sudoers
exit
apt-get update -y
This setup creates User1 and User2 and adds them to sudo users. The aws s3 cp command copies the users public keys from the S3 folder to their .ssh/authorized_keys path. The last section is to run commands as admin without needing passwords.
To read in details with screenshots - refer here. There are lots of security improvements that can be recommended here. While not explicitly used in this example, limiting S3 bucket access to a specific bucket and knowing the security implications of disabling password usage in sudo, are few things that can be highlighted. Use them wisely based on your particular needs.
An alternate way to connect is by using EC2 Instance Connect that allows using IAM policies and principals to connect via SSH to the instances thus avoiding sharing of SSH keys anymore. You can also use the browser-based SSH connection to the instances.
More details of EC2 Instance connect is available at https://aws.amazon.com/about-aws/whats-new/2019/06/introducing-amazon-ec2-instance-connect/

How to get the instance Name from the instance in AWS?

I'm trying to set up a means to register an instance in route53 automatically when the instance is created, using salt and this article.
The article uses ec2-metadata to get the instance-id and and the hostname. I'm wondering if there is a way, using bash within the instance, to get the instance Name instead. ec2-metadata only seems to show the instance-id. Thanks in advance.
First, you need to get the instance-id.
AWS_INSTANCE_ID=`curl -s http://169.254.169.254/latest/meta-data/instance-id`
Than you can get the ec2 instance name using below command.
EC2_NAME=$(aws ec2 describe-tags --region $REGION --filters "Name=resource-id,Values=$AWS_INSTANCE_ID" "Name=key,Values=Name" --output text | cut -f5)
Please ensure that you have AWS Cli Installed.
I hope this helps.
Thanks!
First and foremost, the Amazon EC2 Instance Metadata Service also provides quite some other Names besides the instance-id, if these might be what you are looking for - see Instance Metadata Categories:
hostname - The private hostname of the instance. In cases where multiple network interfaces are present, this refers to the eth0 device (the device for which the device number is 0).
local-hostname - The private DNS hostname of the instance. In cases where multiple network interfaces are present, this refers to the eth0 device (the device for which the device number is 0).
public-hostname - The instance's public DNS. If the instance is in a VPC, this category is only returned if the enableDnsHostnames attribute is set to true.
If you are looking for the Name as exposed in the AWS Management Console though, you would indeed need to resort to using one of the Tools for Amazon Web Services to retrieve it - that Name is in fact just a regular tag with the key Name (see Tagging Your Amazon EC2 Resources), which happens to be used across most AWS services for the obvious purpose.
Here's how to get it with the AWS Command Line Interface for example (skipping region and credentials):
aws ec2 describe-tags \
--filters Name=resource-id,Values=i-abcd1234 Name=key,Values=Name \
--query Tags[].Value --output text
For more advanced CLI JSON output processing than what's possible with the built in --query option, you could resort to jq (a lightweight and flexible command-line JSON processor).
Overthink's answer provides an example based on the now legacy Amazon EC2 API Tools (please note the comments, which correctly point out that you'd nowadays deal with credentials differently, see Tell the CLI Tools Who You Are and IAM Roles for EC2 instances for details).
Not sure what it would look like with bash, but you could use an SDK from the instance itself if you can get the instance id. You would query the ec2 recourse and pass in the ec2 instance id. Using the ruby sdk it would look like:
i = ec2.instances["i-12345678"]
puts i.dns_name
Found that describe-tags not working in my config, failed with 'UnauthorizedOperation' error. Got this working with describe-instances:
aws ec2 describe-instances --filters Name=instance-id,Values=$(wget -qO- http://instance-data/latest/meta-data/instance-id) --query Reservations[].Instances[].Tags[].Value --output text
Command using region and access keys from current user's aws config file's [default] section: ~/.aws/config . If need to use another user's region/keys (can be found at AWS console's IAM dashboard), you can add them to another section in that file, for example [user2] and use in command like this:
aws --profile user2 ec2 describe-instances --filters Name=instance-id,Values=$(wget -qO- http://instance-data/latest/meta-data/instance-id) --query Reservations[].Instances[].Tags[].Value --output text
Use this command to show which metadata is available
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/
You can chain any one of the below file/folders to display the required info
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
events/
hostname
iam/
identity-credentials/
instance-action
instance-id
instance-life-cycle
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-keys/
reservation-id
For eg. instance-type can be chained this to the above command as follows:
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/instance-type
Reference from AWS
ec2metadata tool is useful to get information about the EC2 server.
You can use the following;
ec2metadata --instance-id
in bash;
INSTANCE_ID=$(ec2metadata --instance-id)
You can also access other useful information like the following;
--ami-id
--ami-launch-index
--ami-manifest-path
--ancestor-ami-ids
--availability-zone
--block-device-mapping
--instance-action
--instance-id
--instance-type
--local-hostname
--local-ipv4
--kernel-id
--mac
--profile
--product-codes
--public-hostname
--public-ipv4
--public-keys
--ramdisk-id
--reserveration-id
--security-groups

Resources