I am trying to use Amazon's ECS cli to create a cluster. I keep getting the error:
reason="The key pair 'my-key-pair' does not exist" resourceType="AWS::AutoScaling::LaunchConfiguration"
I have also run:
ecs-cli configure profile --profile-name grantspilsbury --access-key foo --secret-key bar
ecs-cli configure --cluster cluster_test --region us-east-1 --config-name myclusterconfig
I have added my-key-pair to ECS and to EC2.
The full log is:
~ $ ecs-cli up --keypair my-key-pair --capability-iam --size 2 --instance-type t2.small --force
INFO[0002] Created cluster cluster=default region=us-east-1
INFO[0003] Waiting for your CloudFormation stack resources to be deleted...
INFO[0003] Cloudformation stack status stackStatus="DELETE_IN_PROGRESS"
INFO[0038] Waiting for your cluster resources to be created...
INFO[0038] Cloudformation stack status stackStatus="CREATE_IN_PROGRESS"
INFO[0101] Cloudformation stack status stackStatus="CREATE_IN_PROGRESS"
INFO[0164] Cloudformation stack status stackStatus="CREATE_IN_PROGRESS"
ERRO[0197] Failure event reason="The key pair 'my-key-pair' does not exist" resourceType="AWS::AutoScaling::LaunchConfiguration"
FATA[0197] Error executing 'up': Cloudformation failure waiting for 'CREATE_COMPLETE'. State is 'ROLLBACK_IN_PROGRESS'
I ran into the same issue. My issue was I was giving it the full path to the pem file rather than the name of the key on EC2 (link to Ohio region). Turning
ecs-cli up --keypair /home/me/keyPair.pem --capability-iam --size 2 --instance-type t2.medium --cluster-config ec2-tutorial --force
into
ecs-cli up --keypair keyPair --capability-iam --size 2 --instance-type t2.medium --cluster-config ec2-tutorial --force
works as long as there is a key pair on EC2 named keyPair
My problem was that I was passing the filename (keypair.pem) instead of the name of the keypair on AWS. Make sure you pass the keypair name as you see on AWS and not the name of the file.
It's possible that your key is in a different region from the one where you're attempting to create the image. Jeff's answer gave me the clue that my keypair was in the default instance (Ohio), but I was creating my instance in my local region.
I was following this tutorial until I found the same issue:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cli-tutorial-ec2.html
It had nothing to do with the region, the problem was my Key-Pair wasn't uploaded to the ECS. Maybe it is obvious, but if you are AWS beginner like me, the tutorial instructions are not clear about this.
Previous step: Create Key-Pair
If you haven't already done it, first you have to create the Key-Pair that is stored on a ".pem" file. Following this instructions with the AWS Console, was quite easy: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html
Solution:
The ".pem" file has to be uploaded. Can be done using the AWS console, following the menu:
EC2 > Network & Security > Key Pairs > "Create Key Pair"
Menu URL: https://eu-central-1.console.aws.amazon.com/ec2/v2/home?region=eu-central-1#CreateKeyPair:
(Be careful about the 2 regions defined on this URL)
As it is described on:
https://www.cloudbooklet.com/how-to-add-new-user-with-key-pair-in-aws-ec2/
That means that keypair name does not exist in your EC2 account.
Simply create it with s per the aws guide
Related
It should be possible to disable SourceDestCheck since it is documented
"SourceDestCheck -> (boolean)"
but using run-instances with
aws ec2 run-instances ...
--SourceDestCheck false
or
--sourceDestCheck=false
Fails with
Unknown options: --SourceDestCheck, false
It seems I can run it later with a modify command
aws ec2 modify-instance-attribute --resource=$INSTANCE_ID --no-source-dest-check
but it should be possible to set that at instantiation. I just can't figure out the actual syntax.
I know this is old but I ran into the same issue today and solved it this way. In the resource block of your terraform file add:
provisioner "local-exec" {
command = "aws ec2 modify-instance-attribute --no-source-dest-check --instance-id ${self.id}"
}
assuming you have the was cli tools installed.
As far as I can tell, you can't set that on initial launch with the AWS CLI. It's not a supported option. You have to call aws ec2 modify-instance-attribute --no-source-dest-check documented here.
As #mark pointed out, this isn't an option in the RunInstances API. I just want to add that the SourceDestCheck in the AWS CLI doc you referenced is an output. If you look closely, it's an attribute of the ENI.
I have a Jenkins Job to start and stop AWS EC2 instances.
Probably the profile is misconfigured and I'm stuck at this :
botocore.exceptions.ProfileNotFound: The config profile xxxx could not be found
Using this command through Execute Shell:
aws ec2 stop-instances --region $AWS_DEFAULT_REGION --profile $AWS_PROFILE --instance-ids $INSTANCE
Any suggestions will be appreciated to modify the job or resolve this error.
Please check the profile name you passed in the command. It should be configured in your credentials file, usually found at this location ~/.aws/credentials
You can follow this guide (https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) to set up profile.
Best of luck
I am trying to launch a Spark cluster on an EC2 that I created in a development AWS instance. I was able to successfully connect to the EC2 instance using the AWSCLI as ec2-user. I used the existing VPC and AMI to create this EC2. Unzipped the Spark files on EC2 and using the private key tried starting the cluster using the below:
export AWS_SECRET_ACCESS_KEY=xxx
export AWS_ACCESS_KEY_ID=xxx
/home/ec2-user/spark-1.2.0/ec2$ ./spark-ec2 -k test -i /home/ec2-user/identity_files/test.pem launch test-spark-cluster
Got the Error:
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request
InvalidKeyPair.NotFoundThe key pair 'test' does not existxxx
I thought, this might have been due to the region issue, so I used the region and zone parameters while launching spark
/home/ec2-user/spark-1.2.0/ec2$ ./spark-ec2 -k test -i /home/ec2-user/identity_files/test.pem -r us-west-2 -z us-west-2a launch test-spark-cluster
However, when I run this, I encounter a different error:
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request
VPCIdNotSpecifiedNo default VPC for this userxxx
How can I resolve this issue?
I am no expert on this area, but I would recommend setting more parameters on your script call, something like:
./spark-ec2 -k test
-i /home/ec2-user/identity_files/test.pem
-s 5
--instance-type=m3.medium
--region=eu-west-1
--spark-version=1.2.0
launch myCluster
The -s refers to the instante quantity to be created. Furthermore, you might want to check the following, pay special attention to the last one:
The key pair test exists on your account
The key pair test.pem is present on the EC2-console
The region for both key pair and instances is the same
Searching on the web I have found out that most of the errors related to key pairs not being found are caused by region mismatching.
I am running into problems when I am trying to run a mapreduce job on AWS via the command line. I have to perform a large set of steps (approx 100) that are all chained to each other. Since I am not looking forward to configuring that by hand with the AWS graphic interface, I am trying to get it done with the CLI.
However, even the most easy command does not work:
$ aws emr list-clusters
hostname 'elasticmapreduce.us-west-1.amazonaws.com' doesn't match u'us-west-1.elasticmapreduce.amazonaws.com'
On S3 my configurations seem to work fine, since this command creates the bucket without any problems:
$ aws s3 mb s3://randombigdatabucket
These are my configurations:
$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************fooo shared-credentials-file
secret_key ****************fooo shared-credentials-file
region us-west-1 config-file ~/.aws/config
I hope somebody can help me out with this one!
try to install AWS CLI v1.6.6 or later
I am creating an ec2 instance through knife . i gave the following command to create
knife ec2 server create -r "role[webserver]" -I ami-b84e04ea --flavor t1.micro --region ap-southeast-1 -G default -x ubuntu -N server01 -S ec2keypair
but getting error as Fog::Compute::AWS::Error: InvalidBlockDeviceMapping => iops must be specified with the volumeType of device '/dev/sda1' . I am unable to solve this issue , Any help will be appreciated .
Its possible that the ami you are trying to launch requires an EBS. With an EBS you can set the IOPS value which seems like it is not set and is giving you the issue.
Having a look at the documentation it seems you might need to add
--ebs-size 10
SIZE as an option.
I got that from the knife documentation
http://docs.opscode.com/plugin_knife_ec2.html
Also taking a look at the source code for the knife ec2 plugin it looks like you can add.
--ebs-optimized
Enabled optimized EBS I/O