I am trying to create a spot instance in Terraform and the terraform code appears to be fine but I keep getting an error back saying MaxSpotInstanceCountExceeded.
NOTE: Right now this is just a test hence I am not including security groups, IPs, etc etc.
Steps I have taken:
Checked that I have 0 spot instance requests created in console.
Tried logging in to the console and creating a spot instance request. It works just fine.
Cancelled the spot instance request to ensure that I now have 0 spot instance requests.
Now I try and create virtually the same spot instance with the terraform script below, but I get the error: MaxSpotInstanceCountExceeded
Does anyone know why Terraform (or maybe AWS?) is not allowing me to create the spot instance using the terraform script, but it works just fine from the console?
Thanks!
provider "aws" {
profile = "terraform_enterprise_user"
region = "us-east-2"
}
resource "aws_spot_instance_request" "MySpotInstance" {
# Spot Request Settings
wait_for_fulfillment = "true"
spot_type = "persistent"
instance_interruption_behaviour = "stop"
# Instance Settings
ami = "ami-0520e698dd500b1d1"
instance_type = "c4.large"
associate_public_ip_address = "1"
root_block_device {
volume_size = "10"
volume_type = "standard"
}
ebs_block_device {
device_name = "/dev/sdb"
volume_size = "50"
volume_type = "standard"
delete_on_termination = "true"
}
tags = {
Name = "MySpotInstance"
Application = "MyApp"
Environment = "TEST"
}
}
Related
I am creating AWS ec2 instance(s) using auto scaling group and launch template. I would like to get instance ids of the newly launched instances. Is this possible?
For brevity purpose I have removed some code
resource "aws_launch_template" "service_launch_template" {
name_prefix = "${var.name_prefix}-lt"
image_id = var.ami_image_id
iam_instance_profile {
name = var.instance_profile
}
lifecycle {
create_before_destroy = true
}
}
resource "aws_lb_target_group" "service_target_group" {
name = "${var.name_prefix}-tg"
target_type = "instance"
vpc_id = var.vpc_id
lifecycle {
create_before_destroy = true
}
}
resource "aws_autoscaling_group" "service_autoscaling_group" {
name = "${var.name_prefix}-asg"
max_size = var.max_instances
min_size = var.min_instances
desired_capacity = var.desired_instances
target_group_arns = [aws_lb_target_group.service_target_group.arn]
health_check_type = "ELB"
launch_template {
id = aws_launch_template.service_launch_template.id
version = aws_launch_template.service_launch_template.latest_version
}
depends_on = [aws_alb_listener.service_frontend_https]
lifecycle {
create_before_destroy = true
}
}
resource "aws_alb" "service_frontend" {
name = "${var.name_prefix}-alb"
load_balancer_type = "application"
lifecycle {
create_before_destroy = true
}
}
resource "aws_alb_listener" "service_frontend_https" {
load_balancer_arn = aws_alb.service_frontend.arn
protocol = "HTTPS"
port = "443"
}
This is working. But I would like to output the instance ids of the newly launched instances. From terraform documentation looks like the aws_launch_template or aws_autoscaling_group does not export the instance ids. What are my options here?
Terraform is probably completing, and exiting, before the auto-scaling group has even triggered a scale-up event and created the instances. There's no way for Terraform to know about the individual instances, since Terraform isn't managing those instances, the auto-scaling group is managing them. You would need to use another tool, like the AWS CLI, to get the instance IDs.
I am running a terraform through Jenkins which starts up an ec2 then runs a shell script on it using user_data. I run this job 23 times in parallel, and for some reason each time only a few of them (anywhere from 1 to 8 and always different indices) will hang on "aws_instance.genomic-etl-ec2: Still creating..." until the connection times out after approximately an hour and throws a RequestExpired error, with no further details on why. The other instances start fine within around 2-3 minutes each.
My resource:
data "template_file" "my-user_data" {
template = file("scripts/my_script.sh")
}
data "template_cloudinit_config" "my-user-data" {
gzip = true
base64_encode = true
# user_data
part {
content_type = "text/x-shellscript"
content = data.template_file.my-user_data.rendered
}
}
resource "aws_instance" "genomic-etl-ec2" {
ami = var.ami-id
instance_type = "m5.12xlarge"
associate_public_ip_address = true
subnet_id = var.my-subnet-us-east-id
iam_instance_profile = "my-deployment-profile"
user_data = data.template_cloudinit_config.my-user-data.rendered
vpc_security_group_ids = [
aws_security_group.my-sg1.id,
aws_security_group.my-sg2.id
]
root_block_device {
delete_on_termination = true
encrypted = true
volume_size = 1000
}
provisioner "local-exec" {
command = "sleep 40"
}
tags = {
Owner = "Me"
Environment = "development"
Name = "My EC2 - ${id}"
automaticPatches = "1"
}
}
Sometimes AWS instances take a long time to become fully available. It's not uncommon for those to take longer than Terraform's default timeout, causing Terraform to fail.
As per the official documentation on the Terraform aws_instance resource, the create timeout defaults to 10 minutes. If a particular instance type is taking longer than 10 minutes to become available, then you need to increase the create timeout setting:
resource "aws_instance" "genomic-etl-ec2" {
# ...
timeouts {
create = "20m"
}
}
How to upload local file to the ec2 instance with the module terraform-aws-modules/ec2-instance/aws?
I placed provisioner inside module "ec2". It does not work.
I placed provisioner outsite of the module "ec2". It does not work either.
I got the error: "Blocks of type "provisioner" are not expected here".
"provisioner" is inside module "ec2". It does not work.
module "ec2" {
source = "terraform-aws-modules/ec2-instance/aws"
version = "4.1.4"
name = var.ec2_name
ami = var.ami
instance_type = var.instance_type
availability_zone = var.availability_zone
subnet_id = data.terraform_remote_state.vpc.outputs.public_subnets[0]
vpc_security_group_ids = [aws_security_group.sg_WebServerSG.id]
associate_public_ip_address = true
key_name = var.key_name
provisioner "file" {
source = "./foo.txt"
destination = "/home/ec2-user/foo.txt"
connection {
type = "ssh"
user = "ec2-user"
private_key = "${file("./keys.pem")}"
host = module.ec2.public_dns
}
}
}
"provisioner" is outsite of the module "ec2". It does not work.
module "ec2" {
source = "terraform-aws-modules/ec2-instance/aws"
version = "4.1.4"
name = var.ec2_name
ami = var.ami
instance_type = var.instance_type
availability_zone = var.availability_zone
subnet_id = data.terraform_remote_state.vpc.outputs.public_subnets[0]
vpc_security_group_ids = [aws_security_group.sg_WebServerSG.id]
associate_public_ip_address = true
key_name = var.key_name
}
provisioner "file" {
source = "./foo.txt"
destination = "/home/ec2-user/foo.txt"
connection {
type = "ssh"
user = "ec2-user"
private_key = "${file("./keys.pem")}"
host = module.ec2.public_dns
}
}
You can use a null ressource to make it work!
resource "null_resource" "this" {
provisioner "file" {
source = "./foo.txt"
destination = "/home/ec2-user/foo.txt"
connection {
type = "ssh"
user = "ec2-user"
private_key = "${file("./keys.pem")}"
host = module.ec2.public_dns
}
}
You can provision files on an EC2 instance with the YAML cloud-init syntax which is passed to the EC2 instance as user-data. Here is an example of passing cloud-init config to EC2.
cloud-init.yaml file:
#cloud-config
# vim: syntax=yaml
#
# This is the configuration syntax that the write_files module
# will know how to understand. Encoding can be given b64 or gzip or (gz+b64).
# The content will be decoded accordingly and then written to the path that is
# provided.
#
# Note: Content strings here are truncated for example purposes.
write_files:
- content: |
# Your TXT file content...
# goes here
path: /home/ec2-user/foo.txt
owner: ec2-user:ec2-user
permissions: '0644'
Terraform file:
module "ec2" {
source = "terraform-aws-modules/ec2-instance/aws"
version = "4.1.4"
name = var.ec2_name
ami = var.ami
instance_type = var.instance_type
availability_zone = var.availability_zone
subnet_id = data.terraform_remote_state.vpc.outputs.public_subnets[0]
vpc_security_group_ids = [aws_security_group.sg_WebServerSG.id]
associate_public_ip_address = true
key_name = var.key_name
user_data = file("./cloud-init.yaml")
}
The benefits of this approach over the approach in the accepted answer are:
This method creates the file immediately at instance creation, instead of having to wait for the instance to come up first. The null-provisioner/SSH connection method has to wait for the EC2 instance to be become available, and the timing of that could cause your Terraform workflow to become flaky.
This method doesn't require the EC2 instance to be reachable from your local computer that is running Terraform. You could be deploying the EC2 instance to a private subnet behind a load balancer, which would prevent the null-provisioner/SSH connect method from working.
This doesn't require you to have the SSH key for the EC2 instance available on your local computer. You might want to only allow AWS SSM connect to your EC2 instance, to keep it more secure than allowing SSH directly from the Internet, and that would prevent the null-provisioner/SSH connect method from working. Further, storing or referencing an SSH private key in your Terraform state adds a risk factor to your overall security profile.
This doesn't require the use of a null_resource provisioner, which the Terraform documentation states:
Important: Use provisioners as a last resort. There are better alternatives for most situations. Refer to Declaring Provisioners for more details.
I am new to terraform and aws, all I want to do is launch an aws ec2 instance with elastic load balancer with terraform. I get some of the configuration examples from various sites but don't know what is right way to implement those configurations, what should be the folder structure and everything. I had done it using GUI of aws but not getting much help with terraform.
Here the server should be apache2.
Any help is appreciated.
Thanks in advance
As per your requirement of creating Elastic Loadbalancer, using terraform. You will need to create the following resources.
If you already have the EC2 instance created and just want to attach them to ELB.
Create Target Group
Create ELB
Assign the Target Group to your ELB
Register your existing instance to your Target Group
If you don't have any instance created,
Create Target Group
Create ELB
Assign the Target Group to your ELB
Create Launch Template/Configuration
Create ASG, assign the ELB to ASG
The new instance created through ASG will auto-register to the ELB target group.
Terraform Resource example,
Launch Configuration
resource "aws_launch_configuration" "Your_Launch_Configuration" {
name = "launch_conf_name"
instance_type = "Instance_Type"
image_id = "AMI_image_id"
key_name = "Key_Name"
security_groups = "security_groups_id"
user_data = "User Data"
iam_instance_profile = "Instance IAM Role"
}
Auto Scaling Group
resource "aws_autoscaling_group" "Your_ASG" {
name = "ASG Name"
launch_configuration = aws_launch_configuration.Your_Launch_Configuration.id
max_size = "Max size"
min_size = "Min Size"
desired_capacity = "Desired Capacity"
vpc_zone_identifier = "Your Subnet List"
tags = [{
"key" = "Name"
"value" = "ASG Name"
"propagate_at_launch" = true
}]
health_check_grace_period = "300"
target_group_arns = "set of your ELB target Group"
}
Load Balancer Target Group
resource "aws_load_balancer_target_group" "Your_target_group" {
name = "Target_group_name"
port = "80"
protocol = "HTTP"
vpc_id = "Your_vpcid"
tags = {
name = "Target_group_name"
}
health_check {
enabled = true
interval = 300 # health check interval
protocol = "HTTP"
timeout = 300 # timeout seconds
path = "/" # your health check path
}
}
Load Balancer
resource "aws_load_balancer" "your_load_balancer" {
name = load_balancer_name
load_balancer_type = "application"
internal = true # if not internet facing
subnets = ["List of your subnet id"]
security_groups = ["List of your security group id"]
tags = {
"name" = load_balancer_Target_group_name
}
}
Load Balancer Listner
resource "aws_load_balancer_listener" "your_load_balancer_Listner" {
load_balancer_arn = listner_load_balancer_arn #arn of your load balancer
port = "80"
protocol = "http"
default_action {
target_group_arn = listner_Target_group_arn # arn of your target group
type = "forward"
}
}
As described in https://github.com/hashicorp/terraform/issues/17429
After 7 days the spot request is getting cancelled and the instance is still running. So when I run the "terraform apply" its trying to create a new spot. This happens with AWS provider >=1.13.0.
I'm using AWS provider 1.32.0, does anyone know a workaround to this issue? on future installation I will use the valid_until flag which will extend the request lifetime, but what about already installed spots?
Thanks
resource "aws_spot_instance_request" "cheap_worker" {
count = "${var.kube_master_spot_num}"
ami = "${data.aws_ami.nat_ami.id}"
availability_zone ="${element(slice(data.aws_availability_zones.available.names,var.kube_master_on_demand_num,var.availability_zones_num),count.index)}"
spot_price = "3"
instance_type = "${var.kube_master_type}"
subnet_id = "${element(module.aws-vpc.aws_subnet_ids,count.index + var.kube_master_on_demand_num)}" # adjusting to a case with spots & on-demand servers
vpc_security_group_ids = [ "${module.aws-vpc.cluster_sg_id}", "${module.aws-vpc.route53_sg_id}" ]
key_name = "${basename(var.local_ssh_key)}"
associate_public_ip_address = true
root_block_device = [{volume_type="gp2",volume_size="50",delete_on_termination=true}]
spot_type = "one-time"
wait_for_fulfillment = true
tags {
Name = "${var.kube_identify}-${var.kube_type}-master-${count.index}"
}
provisioner "local-exec" {
command = "sleep 120"
}
connection {
type = "ssh"
user = "ubuntu"
private_key = "${file(var.local_ssh_key)}"
}
provisioner "remote-exec" {
inline = [
"sudo apt-get update"
]
}
}