I am trying to integrate Terraform go-SDK with my go-code where I already have an terraform template file and I need to create the infrastructure by importing that file to the go-SDK. I am not able to find any related documents. I tried with the godoc for terraform. With a little understanding, I tried to implement a simple terraform template. My terraform template file is,
provider "aws" {
access_key = "xxxx"
secret_key = "xxxx"
region = "xxxx"
}
resource "aws_instance" "example" {
ami = "ami-xxxx"
instance_type = "t2.micro"
# The name of our SSH keypair.
key_name = "xxxx"
# Security groupID
vpc_security_group_ids = ["sg-xxxx"]
#Subnet ID
subnet_id = "subnet-xxxx"
}
I tried via commandLine and I am able to bring up an instance with this file.
I tried the same via my go-code. My understanding is that I need to create a Context in order to apply it. For creating a Context, I need a module.Tree. So I tried creating a module.Tree with config.Config. I imported the above terraform template file to my go-code to create a config and a tree. My go-code looks like,
package main
import (
"github.com/hashicorp/terraform/config"
"github.com/hashicorp/terraform/config/module"
"fmt"
)
func main() {
con, err := config.LoadFile(path to .tf file)
fmt.Println("Config: ", con)
fmt.Println("Error: ", err)
fmt.Println("Validate: ", con.Validate())
tree := module.NewTree("testTree", con)
fmt.Println("Tree: ", tree.String())
}
When I execute my code, I am not getting any errors for loading the config from file, or validating the same. But I am getting the output as "tree not loaded",
Output:
Config: &{ <nil> <nil> [] [0xc4201cd0c0] [0xc4201f03c0] [] [] [] []}
Error: <nil>
Validate: <nil>
Tree: testTree not loaded
Can someone help me out to implement this? Thanks in advance.
Related
This was a previous question I asked: How to get AWS Lambda ARN using Terraform?
This question was answered but turns out didn't actually solve my problem so this is a follow up.
The terraform code I have written provisions a Lambda function:
Root module:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
}
provider "aws" {
region = var.region
profile = var.aws_profile
}
module "aws_lambda_function" {
source = "./modules/lambda_function"
}
Child module:
resource "aws_lambda_function" "lambda_function" {
function_name = "lambda_function"
handler = "lambda_function.lambda_handler"
runtime = "python3.8"
filename = "./task/dist/package.zip"
role = aws_iam_role.lambda_exec.arn
}
resource "aws_iam_role" "lambda_exec" {
name = "aws_iam_lambda"
assume_role_policy = file("policy.json")
}
What I want the user to be able to do to get the Lambda ARN:
terraform output
The problem: I cannot seem to include the following code anywhere in my terraform code as it causes a "ResourceNotFOundException: Function not found..." error.
data "aws_lambda_function" "existing" {
function_name = "lambda_function"
}
output "arn" {
value = data.aws_lambda_function.existing.arn
}
Where or how do I need to include this to be able to get the ARN or is this possible?
You can't lookup the data for a resource you are creating at the same time. You need to output the ARN from the module, and then output it again from the main terraform template.
In your Lambda module:
output "arn" {
value = aws_lambda_function.lambda_function.arn
}
Then in your main file:
output "arn" {
value = module.aws_lambda_function.arn
}
I've created a key pair for EC2 called terraform, downloaded the pem file to the same directory where my terraform files live, I issue a terraform apply and I get:
aws_instance.windows: Creating...
Error: Error launching source instance: InvalidKeyPair.NotFound: The key pair 'terraform' does not exist
status code: 400, request id: 1ac563d4-244a-4371-bde7-ee9bcf048830
I'm specifying the name of the key-value pair via an envrionment variable. This is the start of the block I'm using to create the Windows virtual machine:
resource "aws_instance" "windows" {
ami = data.aws_ami.Windows_2019.image_id
instance_type = var.windows_instance_types
key_name = var.key_name
vpc_security_group_ids = [aws_security_group.allow_rdp_winrm.id]
associate_public_ip_address = true
subnet_id = aws_subnet.subnet1.id
get_password_data = "true"
user_data = file("scripts/user_data.txt")
There is obviously something I'm doing wrong, do I need to tell terraform which aws region then key pair resides in ?
The key pairs are regional, so if you created them in one region, they aren't available in the other.
Terraform will always try to find and use the key in the region that you tell it to run in and if the key is not present, AWS will complain about this error.
Terraform also doesn't like it when things are created out of band and you might run into complications. It's also much cleaner to create the keypair using terraform and you can reference it as Atul has posted in his answer.
You could also import the key into Terraform or use Terraform's data sources to search and find the key as alternatives but these are a bit advanced, especially if you're getting started with Terraform.
You need to create a key pair first before consuming it.
resource "aws_key_pair" "my_key_pair" {
key_name = var.key_name
public_key = file("${abspath(path.cwd)}/my-key.pub")
}
Now use the key as
resource "aws_instance" "windows" {
ami = data.aws_ami.Windows_2019.image_id
instance_type = var.windows_instance_types
key_name = aws_key_pair.my_key_pair.key_name
I'll only provide an answer that pertains to your error message directly, as this question came up first on a Bing search.
The Data Source documentation doesn't make mention of Keypair. It might be presumed from the Instances Availability Zone.
data "aws_key_pair" "example" {
key_name = "terraform"
filter {
name = "tag:Component"
values = ["web"]
}
}
resource "aws_instance" "web" {
ami = data.aws_ami.ubuntu.id
key_name = data.aws_key_pair.example.key_name
instance_type = "t3.micro"
tags = {
Name = "HelloWorld"
}
}
Instance resource declaration: https://registry.terraform.io/providers/hashicorp%20%20/aws/latest/docs/resources/instance
AWS Keypair Data Source: https://registry.terraform.io/providers/hashicorp%20%20/aws/latest/docs/data-sources/key_pair
I am fairly new to terraform. I am trying to create number of ec2 instances and want to run bootstrap script once instance is ready, and in bootstrap script I need to pass private ips of instance created ( part of terraform script).
After so much of googling I came to know that I have to use terraform_file but, not able to use it correctly.
Terraform version: 0.11.13
/* Security group for the instances */
resource "aws_security_group" "test-reporting-sg"
{
name = "${var.environment}-test-reporting-sg"
vpc_id = "${var.vpc_id}"
}
data "template_file" "init"
{
count = "${var.instance_count}"
template = "${file("${path.module}/wrapperscript.sh")}"
vars = {
ip_addresses= "${aws_instance.dev-testreporting.*.private_ip}"
}
}
resource "aws_instance" "dev-testreporting"
{
count = "${var.instance_count}"
ami="${var.ami_id}"
instance_type ="${var.instance_type}"
key_name ="${var.key_name}"
security_groups = [ "${aws_security_group.test-reporting-sg.id}" ]
subnet_id = "${var.subnet_ids}"
user_data = "${data.template_file.init.*.rendered.[count.index]}"
}
Thanks
In resource module you can add private_ip like below
private_ip = "${lookup(var.private_ips,count.index)}"
and add private_ip values to variable file
I am trying to create ec2 instance using terraform. Passing credentials through terraform cli fails, while hardcoding it in main.tf works fine
This is to create ec2 instance dynamically using terraform
terraform apply works with following main.tf
provider "aws" {
region = "us-west-2"
access_key = "hard-coded-access-key"
secret_key = "hard-coded-secret-key"
}
resource "aws_instance" "ec2-instance" {
ami = "ami-id"
instance_type = "t2.micro"
tags {
Name = "test-inst"
}
}
while the following does not work:
terraform apply -var access_key="hard-coded-access-key" -var secret_key="hard-coded-secret-key"
Is there any difference in the above two ways of running the commands? As per terraform documentation both of the above should work.
Every terraform module can use input variables, including the main module. But before using input variables, you must declare them.
Create a variables.tf file on the same folder you have your main.tf file:
variable "credentials" {
type = object({
access_key = string
secret_key = string
})
description = "My AWS credentials"
}
Then you can reference input variables in your code like that:
provider "aws" {
region = "us-west-2"
access_key = var.credentials.access_key
secret_key = var.credentials.secret_key
}
And you can either run:
terraform apply -var credentials.access_key="hard-coded-access-key" -var credentials.secret_key="hard-coded-secret-key"
Or you could create a terraform.tfvars file with the following content:
# ------------------
# AWS Credentials
# ------------------
credentials= {
access_key = "hard-coded-access-key"
secret_key = "hard-coded-secret-key"
}
And then simply run terraform apply.
But the key point is that you must declare input variables before using them.
The #Felipe answer is right, But I will never recommend defining Access key and secret key in variables.tf, What you have to to do is to left it blinks and set keys using aws configure or other options is to create keys for terraform deployment purpose only using aws configure --profile terraform or without profile aws configur
so your connection.tf or main.tf will look like this,
provider "aws" {
#You can use an AWS credentials file to specify your credentials.
#The default location is $HOME/.aws/credentials on Linux and OS X, or "%USERPROFILE%\.aws\credentials" for Windows users
region = "us-west-2"
# profile configured during aws configure --profile
profile = "terraform"
# you can also restrict account here, to allow particular account for deployment
allowed_account_ids = ["12*****45"]
}
You can also tell separate file for secret key and access key, The reason behind this is, as Variables.tf is part of your configuration language or bitbucket, so it's better to not place these sensitive keys in variables.tf
You can create a file somewhere in your system and give the path of the keys in the provider section.
provider "aws" {
region = "us-west-2"
shared_credentials_file = "$HOME/secret/credentials"
}
Here is the format of the credentials file
[default]
aws_access_key_id = A*******Z
aws_secret_access_key = A*******/***xyz
I'm trying to set up something really simple with Terraform, but it gives me an error I haven't seen before.
When I run terraform validate -var-file=secrets.tfvars I get the following error:
Error loading files open /home/MYUSER/Documents/git/packer-with-terraform/terratest/-var-file=secrets.tfvars: no such file or directory
And when I run terraform plan -var-file=secrets.tfvars I get this:
invalid value "secrets.tfvars" for flag -var-file: Error decoding Terraform vars file: At 1:10: root.variable: unknown type for string *ast.ObjectList
I have three files within the same folder, and their content is minimal:
providers.tf
provider "aws" {
region = "us-west-1"
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
main.tf
resource "aws_instance" "master_proxy" {
ami = "ami-123sample"
instance_type = "t2.micro"
}
secrets.tfvars
variable "access_key" { default = "sampleaccesskey" }
variable "secret_key" { default = "samplesecretkey" }
If I set access_key and secret_key directly, and not via variables, then it works. A similar setup with secrets-files and whatnot works on another project of mine; I just don't understand what's wrong here.
Firstly, terraform validate validates a folder of .tf files to check that the syntax is correct. You can't pass a separate vars file to the command. In fact, terraform validate won't even check your variables are even set properly.
Secondly, your secrets.tfvars file is using the wrong syntax. Instead you want it to look more like this:
secrets.tfvars:
access_key = "sampleaccesskey"
secret_key = "samplesecretkey"
But this will error because you haven't actually defined the variables in a .tf file:
providers.tf
variable "access_key" { default = "sampleaccesskey" }
variable "secret_key" { default = "samplesecretkey" }
provider "aws" {
region = "us-west-1"
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
If you don't have a sensible default for a variable (such as typically in this case) then you can remove the default argument to the variable and this will make Terraform error on the plan because a required variable is not set:
providers.tf
variable "access_key" {}
variable "secret_key" {}
provider "aws" {
region = "us-west-1"
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
Well, I messed up big time. I somehow managed to forget the supposed structure (and difference) of *.tf and *.tfvars files.
For those who might run into a similar problem later on:
*.tf files are for configuration and declaration, which means that any variables must be defined within a *.tf file.
*.tfvars files are for giving values to already defined variables. These files can be passed with the -var-file flag (which I had misused).
# Set a Provider
provider "aws" {
region = "${var.region}"
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
resource "aws_security_group" "test-server-sg" {
name = "test-server-sg"
ingress {
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_instance" "test-server" {
ami = "${var.ami}"
instance_type = "${var.instance_type}"
user_data = <<-EOF
#!/bin/bash
echo "Hello, World" > index.html
nohup busybox httpd -fp 8080 &
EOF
tags {
name = "Test Web Server"
environment = "${var.environment}"
project = "${var.project}"
}
}
variable "region" {
type = "string"
description = "AWS region"
}
variable "access_key" {
type = "string"
description = "AWS access key"
}
variable "secret_key" {
type = "string"
description = "AWS secret key"
}
variable "ami" {
type = "string"
description = "AWS image id"
}
variable "instance_type" {
type = "string"
description = "AWS instance type"
}
variable "environment" {
type = "string"
description = "AWS environment name"
}
variable "project" {
type = "string"
description = "AWS project name"
}
output "Test Server Public DNS" {
value = "${aws_instance.test-server.public_dns}"
}
output "Test Server Public IP" {
value = "${aws_instance.test-server.public_ip}"
}
region = "us-east-1"
access_key = "put your aws access key here"
secret_key = "put your aws secret key here"
ami = "ami-40d28157"
instance_type = "t2.micro"
environment = "Test"
project = "Master Terraform"