Terraform fails to import key pair with Amazon EC2 - amazon-ec2

Using Terraform 0.7.7.
I have a simple Terraform file with the following:
provider "aws" {
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
region = "${var.region}"
}
resource "aws_instance" "personal" {
ami = "${lookup(var.amis, var.region)}"
instance_type = "t2.micro"
}
resource "aws_eip" "ip" {
instance = "${aws_instance.personal.id}"
}
resource "aws_key_pair" "personal" {
key_name = "mschuchard-us-east"
public_key = "${var.public_key}"
}
Terraform apply yields the following error:
aws_key_pair.personal: Creating...
fingerprint: "" => "<computed>"
key_name: "" => "mschuchard-us-east"
public_key: "" => "ssh-rsa pubkey hash mschuchard-us-east"
aws_instance.personal: Creating...
ami: "" => "ami-c481fad3"
availability_zone: "" => "<computed>"
ebs_block_device.#: "" => "<computed>"
ephemeral_block_device.#: "" => "<computed>"
instance_state: "" => "<computed>"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
network_interface_id: "" => "<computed>"
placement_group: "" => "<computed>"
private_dns: "" => "<computed>"
private_ip: "" => "<computed>"
public_dns: "" => "<computed>"
public_ip: "" => "<computed>"
root_block_device.#: "" => "<computed>"
security_groups.#: "" => "<computed>"
source_dest_check: "" => "true"
subnet_id: "" => "<computed>"
tenancy: "" => "<computed>"
vpc_security_group_ids.#: "" => "<computed>"
aws_instance.personal: Creation complete
aws_eip.ip: Creating...
allocation_id: "" => "<computed>"
association_id: "" => "<computed>"
domain: "" => "<computed>"
instance: "" => "i-0ab94b58b0089697d"
network_interface: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>"
vpc: "" => "<computed>"
aws_eip.ip: Creation complete
Error applying plan:
1 error(s) occurred:
* aws_key_pair.personal: Error import KeyPair: InvalidKeyPair.Duplicate: The keypair 'mschuchard-us-east' already exists.
status code: 400, request id: 51950b9a-55e8-4901-bf35-4d2be234abbf
The only help I found with googling was to blow away the *.tfstate files, which I tried and that did not help. I can launch an EC2 instance with the gui with this key pair and easily ssh into it, but Terraform is erroring when trying to use the same fully functional keypair.

The error is telling you that the keypair already exists in your AWS account but Terraform has no knowledge of it in its state files so is attempting to create it each time.
You have two options available to you here. Firstly, you could simply delete it from the AWS account and allow Terraform to upload it and thus allow it to be managed by Terraform and be in its state files.
Alternatively you could use the Terraform import command to import the pre-existing resource into your state file:
terraform import aws_key_pair.personal mschuchard-us-east

Use the ${uuid()} function to always get a random id for key pairs when generating, the selected/generated UUID makes it into the state file, so you will still be able to delete, but updating will not be possible. Every time you apply your terraform file there will be a new keypair generated...
While it is true that you can not generate a key pair from scratch using the AWS provider, you can generate a new key pair object in AWS using an RSA private key that the TLS provider generates.
resource "aws_key_pair" "test" {
key_name = "${uuid()}"
public_key = "${tls_private_key.t.public_key_openssh}"
}
provider "tls" {}
resource "tls_private_key" "t" {
algorithm = "RSA"
}
provider "local" {}
resource "local_file" "key" {
content = "${tls_private_key.t.private_key_pem}"
filename = "id_rsa"
provisioner "local-exec" {
command = "chmod 600 id_rsa"
}
}
Use the tls provider to generate a key, and import it as a new object every time.
Then export the private key so that you have it for access to the server(s) later on.
It is worthy to note that this breaks one of the paradigms that Terraform is attempting to use (infrastructure as code), but from a practical development standpoint that might be a bit too idealistic... Terraform builds fail midway through and states get invalidated. A better solution might be if the AWS plugin received an "already exists" error it imported automatically, or if that was an optional behavior that could be set.

The error says that key pair already exists in AWS, and it does not say whether it was created using Terraform or using console.
You should see it in AWS console EC2 -> Key Pairs for correct region. You should delete it using console before retrying import it using Terraform.

Related

Provisioning Windows VM including File Provisioner for AWS using Terraform results in Timeout

I'm aware that there already exists several posts similar to this one - I've went through them and adapted my Terraform configuration file, but it makes no difference.
Therefore, I'd like to publish my configuration file and my use case: I'd like to provision a (Windows) Virtual Machine on AWS, using Terraform. It works without the File Provisioning part - including them, the provisioning results in a timeout.
This includes adaptations from previous posts:
SSH connection restriction
SSH isnt working in Windows with Terraform provisioner connection type
Usage of a Security group
Terraform File provisioner can't connect ec2 over ssh. timeout - last error: dial tcp 92.242.xxx.xx:22: i/o timeout
I also get a timeout when using "winrm" instead of "ssh".
I'd be happy if you could provide any hint for following config file:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.0"
}
}
}
# Configure the AWS Provider
provider "aws" {
access_key = "<my access key>"
secret_key = "<my secret key>"
region = "eu-central-1"
}
resource "aws_instance" "webserver" {
ami = "ami-07dfec7a6d529b77a"
instance_type = "t2.micro"
security_groups = [aws_security_group.sgwebserver.name]
key_name = aws_key_pair.pubkey.key_name
tags = {
"Name" = "WebServer-Win"
}
}
resource "null_resource" "deployBundle" {
connection {
type = "ssh"
user = "Administrator"
private_key = "${file("C:/Users/<my user name>/aws_keypair/aws_instance.pem")}"
host = aws_instance.webserver.public_ip
}
provisioner "file" {
source = "files/test.txt"
destination = "C:/test.txt"
}
depends_on = [ aws_instance.webserver ]
}
resource "aws_security_group" "sgwebserver" {
name = "sgwebserver"
description = "Allow ssh inbound traffic"
ingress {
from_port = 0
to_port = 6556
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "sgwebserver"
}
}
resource "aws_key_pair" "pubkey" {
key_name = "aws-cloud"
public_key = file("key/aws_instance.pub")
}
resource "aws_eip" "elasticip" {
instance = aws_instance.webserver.id
}
output "eip" {
value = aws_eip.elasticip.public_ip
}
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
name = "my-vpc"
cidr = "10.0.0.0/16"
azs = ["eu-central-1a", "eu-central-1b", "eu-central-1c"]
private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
public_subnets = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
enable_nat_gateway = true
enable_vpn_gateway = true
tags = {
Terraform = "true"
Environment = "dev"
}
}
Thanks a lot in advance!
Windows EC2 instances don't support SSH, they support RDP. You would have to install SSH server software on the instance before you could SSH into it.
I suggest doing something like placing the file in S3, and using a user data script to trigger the Windows EC2 instance to download the file on startup.

Decrypting Windows Password in terraform

I'm trying to set up a Terraform script to deploy a windows server. When running terraform apply I get an error message referencing below
Error: Invalid reference
on main.tf line 44, in resource "aws_instance" "server":
44: password = "${rsadecrypt(aws_instance.server[0].password_data, file(KEY_PATH))}"
A reference to a resource type must be followed by at least one attribute
access, specifying the resource name.
AFAIK the resource is "aws_instance", the name is "server[0]" while the attribute is the "password_data". I know I'm missing something but don't know what. any assistance would be appreciated.
The full resource module is below in case there is something I'm missing contained in there.
Thanks
resource "aws_instance" "server" {
ami = var.AMIS[var.AWS_REGION]
instance_type = var.AWS_INSTANCE
vpc_security_group_ids = [module.networking.security_group_id_out]
subnet_id = module.networking.subnet_id_out
## Use this count key to determine how many servers you want to create.
count = 1
key_name = var.KEY_NAME
tags = {
# Name = "Server-Cloud"
Name = "Server-${count.index}"
}
root_block_device {
volume_size = var.VOLUME_SIZE
volume_type = var.VOLUME_TYPE
delete_on_termination = true
}
get_password_data = true
provisioner "remote-exec" {
connection {
host = coalesce(self.public_ip, self.private_ip)
type = "winrm"
## Need to provide your own .pem key that can be created in AWS or on your machine for each provisioned EC2.
password = ${rsadecrypt(aws_instance.server[0].password_data, file(KEY_PATH))}
}
inline = [
"powershell -ExecutionPolicy Unrestricted C:\\Users\\Administrator\\Desktop\\installserver.ps1 -Schedule",
]
}
provisioner "local-exec" {
command = "echo ${self.public_ip} >> ../public_ips.txt"
}
}
Use password = "${rsadecrypt(self.password_data, file("/root/.ssh/id_rsa"))}"
without user = "admin" as below :
resource "aws_instance" "windows_server" {
get_password_data = "true"
connection {
host = "${self.public_ip}"
type = "winrm"
https = false
password = "${rsadecrypt(self.password_data, file("/root/.ssh/id_rsa"))}"
agent = false
insecure = "true"
}
}

Trying to create an encrypted copy of an ami

I have the following ruby code:
sts = Aws::STS::Client.new
stsresp = sts.assume_role(
:role_arn => _role_arn,
:role_session_name => "provisioning_vpc_query"
)
ec2 = Aws::EC2::Client.new(
session_token: stsresp.credentials["session_token"],
region: _region,
access_key_id: stsresp.credentials["access_key_id"],
secret_access_key: stsresp.credentials["secret_access_key"]
)
# ...
p "got image: #{preimage_id}, create encrypted copy..."
resp = ec2.copy_image({
encrypted: true,
name: oname,
source_image_id: preimage_id,
source_region: _region,
dry_run: false
})
In the code above, preimage_id is a known image in the region _region referenced above.
When I run this, I get:
"got image: ami-71e9020b, create encrypted copy..."
Aws::EC2::Errors::InvalidRequest: The storage for the ami is not available in the source region.
I can do this manually from the console with no trouble.
Can you help me figure out what's wrong?
turns out, I was attempting to copy the ami before it was 'available'. Adding a single line to wait did the trick:
sts = Aws::STS::Client.new
stsresp = sts.assume_role(
:role_arn => _role_arn,
:role_session_name => "provisioning_vpc_query"
)
ec2 = Aws::EC2::Client.new(
session_token: stsresp.credentials["session_token"],
region: _region,
access_key_id: stsresp.credentials["access_key_id"],
secret_access_key: stsresp.credentials["secret_access_key"]
)
# ...
p "got image: #{preimage_id}, create encrypted copy..."
ec2.wait_until(:image_available, {image_ids: [preimage_id]})
resp = ec2.copy_image({
encrypted: true,
name: oname,
source_image_id: preimage_id,
source_region: _region,
dry_run: false
})

Terraform .tfvars cast decoding error

I'm trying to set up something really simple with Terraform, but it gives me an error I haven't seen before.
When I run terraform validate -var-file=secrets.tfvars I get the following error:
Error loading files open /home/MYUSER/Documents/git/packer-with-terraform/terratest/-var-file=secrets.tfvars: no such file or directory
And when I run terraform plan -var-file=secrets.tfvars I get this:
invalid value "secrets.tfvars" for flag -var-file: Error decoding Terraform vars file: At 1:10: root.variable: unknown type for string *ast.ObjectList
I have three files within the same folder, and their content is minimal:
providers.tf
provider "aws" {
region = "us-west-1"
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
main.tf
resource "aws_instance" "master_proxy" {
ami = "ami-123sample"
instance_type = "t2.micro"
}
secrets.tfvars
variable "access_key" { default = "sampleaccesskey" }
variable "secret_key" { default = "samplesecretkey" }
If I set access_key and secret_key directly, and not via variables, then it works. A similar setup with secrets-files and whatnot works on another project of mine; I just don't understand what's wrong here.
Firstly, terraform validate validates a folder of .tf files to check that the syntax is correct. You can't pass a separate vars file to the command. In fact, terraform validate won't even check your variables are even set properly.
Secondly, your secrets.tfvars file is using the wrong syntax. Instead you want it to look more like this:
secrets.tfvars:
access_key = "sampleaccesskey"
secret_key = "samplesecretkey"
But this will error because you haven't actually defined the variables in a .tf file:
providers.tf
variable "access_key" { default = "sampleaccesskey" }
variable "secret_key" { default = "samplesecretkey" }
provider "aws" {
region = "us-west-1"
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
If you don't have a sensible default for a variable (such as typically in this case) then you can remove the default argument to the variable and this will make Terraform error on the plan because a required variable is not set:
providers.tf
variable "access_key" {}
variable "secret_key" {}
provider "aws" {
region = "us-west-1"
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
Well, I messed up big time. I somehow managed to forget the supposed structure (and difference) of *.tf and *.tfvars files.
For those who might run into a similar problem later on:
*.tf files are for configuration and declaration, which means that any variables must be defined within a *.tf file.
*.tfvars files are for giving values to already defined variables. These files can be passed with the -var-file flag (which I had misused).
# Set a Provider
provider "aws" {
region = "${var.region}"
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
resource "aws_security_group" "test-server-sg" {
name = "test-server-sg"
ingress {
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_instance" "test-server" {
ami = "${var.ami}"
instance_type = "${var.instance_type}"
user_data = <<-EOF
#!/bin/bash
echo "Hello, World" > index.html
nohup busybox httpd -fp 8080 &
EOF
tags {
name = "Test Web Server"
environment = "${var.environment}"
project = "${var.project}"
}
}
variable "region" {
type = "string"
description = "AWS region"
}
variable "access_key" {
type = "string"
description = "AWS access key"
}
variable "secret_key" {
type = "string"
description = "AWS secret key"
}
variable "ami" {
type = "string"
description = "AWS image id"
}
variable "instance_type" {
type = "string"
description = "AWS instance type"
}
variable "environment" {
type = "string"
description = "AWS environment name"
}
variable "project" {
type = "string"
description = "AWS project name"
}
output "Test Server Public DNS" {
value = "${aws_instance.test-server.public_dns}"
}
output "Test Server Public IP" {
value = "${aws_instance.test-server.public_ip}"
}
region = "us-east-1"
access_key = "put your aws access key here"
secret_key = "put your aws secret key here"
ami = "ami-40d28157"
instance_type = "t2.micro"
environment = "Test"
project = "Master Terraform"

How do you use DynamoDB Local with the AWS Ruby SDK?

Amazon's documentation provides examples in Java, .NET, and PHP of how to use DynamoDB Local. How do you do the same thing with the AWS Ruby SDK?
My guess is that you pass in some parameters during initialization, but I can't figure out what they are.
dynamo_db = AWS::DynamoDB.new(
:access_key_id => '...',
:secret_access_key => '...')
Are you using v1 or v2 of the SDK? You'll need to find that out; from the short snippet above, it looks like v2. I've included both answers, just in case.
v1 answer:
AWS.config(use_ssl: false, dynamo_db: { api_verison: '2012-08-10', endpoint: 'localhost', port: '8080' })
dynamo_db = AWS::DynamoDB::Client.new
v2 answer:
require 'aws-sdk-core'
dynamo_db = Aws::DynamoDB::Client.new(endpoint: 'http://localhost:8080')
Change the port number as needed of course.
Now aws-sdk version 2.7 throws an error as Aws::Errors::MissingCredentialsError: unable to sign request without credentials set when keys are absent. So below code works for me
dynamo_db = Aws::DynamoDB::Client.new(
region: "your-region",
access_key_id: "anykey-or-xxx",
secret_access_key: "anykey-or-xxx",
endpoint: "http://localhost:8080"
)
I've written a simple gist that shows how to start, create, update and query a local dynamodb instance.
https://gist.github.com/SundeepK/4ffff773f92e3a430481
Heres a run down of some simple code:
Below is a simple command to run dynamoDb in memory
#Assuming you have downloading dynamoDBLocal and extracted into a dir called dynamodbLocal
java -Djava.library.path=./dynamodbLocal/DynamoDBLocal_lib -jar ./dynamodbLocal/DynamoDBLocal.jar -inMemory -port 9010
Below is a simple ruby script
require 'aws-sdk-core'
dynamo_db = Aws::DynamoDB::Client.new(region: "eu-west-1", endpoint: 'http://localhost:9010')
dynamo_db.create_table({
table_name: 'TestDB',
attribute_definitions: [{
attribute_name: 'SomeKey',
attribute_type: 'S'
},
{
attribute_name: 'epochMillis',
attribute_type: 'N'
}
],
key_schema: [{
attribute_name: 'SomeKey',
key_type: 'HASH'
},
{
attribute_name: 'epochMillis',
key_type: 'RANGE'
}
],
provisioned_throughput: {
read_capacity_units: 5,
write_capacity_units: 5
}
})
dynamo_db.put_item( table_name: "TestDB",
item: {
"SomeKey" => "somevalue1",
"epochMillis" => 1
})
puts dynamo_db.get_item({
table_name: "TestDB",
key: {
"SomeKey" => "somevalue",
"epochMillis" => 1
}}).item
The above will create a table with a range key and also add/query for the same data that was added. Not you must already have version 2 of the aws gem installed.

Resources