please help
How to make sure EC2 is using the custom KMS key; we are using terraform to deploy the EC2 instance, every time the EC2 instance is launched in an auto-scaling group, it crashes with the below error. Seems like the EC2 instance has no access to KMS key
Error: Termination Reason: Client.InternalError: Client error on launch
resource "aws_autoscaling_group" "autoscaling-group" {
name = var.name
availability_zones = var.availability_zones
min_size = var.min_size
desired_capacity = var.desired_capacity
max_size = var.max_size
health_check_type = "EC2"
launch_configuration = aws_launch_configuration.launch_configuration.name
vpc_zone_identifier = local.subnet_id
termination_policies = ["OldestInstance"]
}
resource "aws_launch_configuration" "launch_configuration" {
name = var.name
image_id = var.ami
instance_type = var.instance_type
iam_instance_profile = var.iam_instance_profile_name
security_groups = [aws_security_group.security_group.id]
associate_public_ip_address = true
}
resource "aws_autoscaling_policy" "autoscaling-policy" {
name = var.name
policy_type = "TargetTrackingScaling"
estimated_instance_warmup = "90"
adjustment_type = "ChangeInCapacity"
autoscaling_group_name = aws_autoscaling_group.autoscaling-group.name
}
--
Thank you
Thank you All for your support, I was able to resolve; The issue was with the kms key grant for ec2 auto scaling service
we used the below module and the issue got resolved
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_grant
resource "aws_kms_grant" "a" {
name = "my-grant"
key_id = aws_kms_key.a.key_id
grantee_principal = aws_iam_role.a.arn
operations = ["Encrypt", "Decrypt", "GenerateDataKey"]
}
This is probably happening because the Auto Scaling Group can't attach the EBS volume to your EC2 instance. Looks like you chose your EBS volume to be encrypted, but the key policy in your customer managed key in KMS is not having the correct policy for the specific IAM role Autoscaling use, that is AWSServiceRoleForAutoScaling. You need to add the below policy blocks under the key policy for the KMS key used to encrypt the EBS volume:
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::*<AWS Account Number>*:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
]
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::*<AWS Account Number>*:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
]
},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
}
You can execute the plan with TF_LOG=DEBUG to get more details about what is missing. You mostly need a Service-Linked role in order to get pass the permission issue
I revolved this error, adding autoscaling role on user in KMS
Command to be executed on Target account:
aws kms create-grant --region <Region> --key-id arn:aws:kms:<Region>:111111111111:key/<Key-ID> --grantee-principal arn:aws:iam::999999999999:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling --operations "Encrypt" "Decrypt" "ReEncryptFrom" "ReEncryptTo" "GenerateDataKey" "GenerateDataKeyWithoutPlaintext" "DescribeKey" "CreateGrant"
Region: same region where AMI and ASG to be created
Source account: 111111111111 (AMI from which EBS is encrypted)
Target account: 999999999999 (ASG to be created from Source account AMI)
Snapshot KMS Key: (Go to AMI of Source account and check snapshot)
IAM Role of Target account:
arn:aws:iam::999999999999:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling
Related
I have a lambda function and an apigatewayv2. I am creating everything via terraform as below.
resource "aws_lambda_function" "prod_options" {
description = "Production Lambda"
environment {
variables = var.prod_env
}
function_name = "prod-func"
handler = "index.handler"
layers = [
aws_lambda_layer_version.node_modules_prod.arn
]
memory_size = 1024
package_type = "Zip"
reserved_concurrent_executions = -1
role = aws_iam_role.lambda_exec.arn
runtime = "nodejs12.x"
s3_bucket = aws_s3_bucket.lambda_bucket_prod.id
s3_key = aws_s3_bucket_object.lambda_node_modules_prod.key
source_code_hash = data.archive_file.lambda_node_modules_prod.output_base64sha256
timeout = 900
tracing_config {
mode = "PassThrough"
}
}
and role
resource "aws_iam_role_policy_attachment" "lambda_policy" {
role = aws_iam_role.lambda_exec.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}
resource "aws_iam_role" "lambda_exec" {
name = "api_gateway_role"
assume_role_policy = jsonencode({
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": [
"apigateway.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
})
}
and then permissions
resource "aws_lambda_permission" "prod_api_gtw" {
statement_id = "AllowExecutionFromApiGateway"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.prod_options.function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.gateway_prod.execution_arn}/*/*"
}
After I deploy and try to invoke the url , I gget the following error
"integrationErrorMessage": "The IAM role configured on the integration or API Gateway doesn't have permissions to call the integration. Check the permissions and try again.",
I've been stuck with this for a while now. How can I solve this error?
You may have to create a Lambda permission to allow execution from an API Gateway resource:
resource "aws_lambda_permission" "apigw_lambda" {
statement_id = "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.layout_editor_prod_options.function_name
principal = "apigateway.amazonaws.com"
# The /*/*/* part allows invocation from any stage, method and resource path
# within API Gateway REST API.
source_arn = "${aws_api_gateway_rest_api.rest_api.execution_arn}/*/*/*"
}
Also, for the Lambda lambda_exec, you don't need apigateway.amazonaws.com principal. The reason why we don't need this is that the execution role applies to the function and allows it to interact with other AWS services. In the other hand, this wont allow anything for the API Gateway, for that we need a Lambda permission.
resource "aws_iam_role" "lambda_exec" {
name = "lambda_exec_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
In the other hand, I would add a policy to the Lambda execution role to be able to log to CloudWatch. This might be useful for further debugging:
resource "aws_iam_policy" "lambda_logging" {
name = "lambda_logging"
path = "/"
description = "IAM policy for logging from a lambda"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*",
"Effect": "Allow"
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "lambda_logs" {
role = aws_iam_role.lambda_exec.name
policy_arn = aws_iam_policy.lambda_logging.arn
}
I'm trying to configure an AWS Lambda function to pipe its output into an SNS notification, but it doesn't seem to work. The function executes successfully in the Lambda console and I can see the output is correct, but SNS never seems to be getting notified or publishing anything. I'm working with Terraform to stand up my infra, here is the Terraform code I'm using, maybe someone can help me out:
resource "aws_lambda_function" "lambda_apigateway_to_sns_function" {
filename = "../node/lambda.zip"
function_name = "LambdaPublishToSns"
handler = "index.snsHandler"
role = aws_iam_role.lambda_apigateway_to_sns_execution_role.arn
runtime = "nodejs12.x"
}
resource "aws_iam_role" "lambda_apigateway_to_sns_execution_role" {
assume_role_policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow"
}
]
}
POLICY
}
resource "aws_iam_role_policy_attachment" "apigateway_to_sns_sns_full_access" {
policy_arn = "arn:aws:iam::aws:policy/AmazonSNSFullAccess"
role = aws_iam_role.lambda_apigateway_to_sns_execution_role.name
}
resource "aws_lambda_function_event_invoke_config" "example" {
function_name = aws_lambda_function.lambda_apigateway_to_sns_function.arn
destination_config {
on_success {
destination = aws_sns_topic.sns_topic.arn
}
on_failure {
destination = aws_sns_topic.sns_topic.arn
}
}
}
And here's my Lambda function code (in NodeJS):
exports.snsHandler = (event, context, callback) => {
context.callbackWaitsForEmptyEventLoop = false;
callback(null, {
statusCode: 200,
body: event.body + " apigateway"
);
}
(the function is supposed to take input from API Gateway, and whatever is in the body of the API Gateway request, just append "apigateway" to the end of it and pass the message on; I've tested the integration with API Gateway and that integration works perfectly)
Thanks!
I have been getting this error lately while creating a ES domain using Terraform. Nothing has changed in the way I define the ES domain. I did however start using SSL (AWS ACM cert) on the ALB layer but that should not have affected this. Any ideas what it might be complaining about ?
resource "aws_elasticsearch_domain" "es" {
domain_name = "${var.es_domain}"
elasticsearch_version = "6.3"
cluster_config {
instance_type = "r4.large.elasticsearch"
instance_count = 2
zone_awareness_enabled = true
}
vpc_options {
subnet_ids = "${var.private_subnet_ids}"
security_group_ids = [
"${aws_security_group.es_sg.id}"
]
}
ebs_options {
ebs_enabled = true
volume_size = 10
}
access_policies = <<CONFIG
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "es:*",
"Principal": "*",
"Effect": "Allow",
"Resource": "arn:aws:es:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:domain/${var.es_domain}/*"
}
]
}
CONFIG
snapshot_options {
automated_snapshot_start_hour = 23
}
tags = {
Domain = "${var.es_domain}"
}
depends_on = [
"aws_iam_service_linked_role.es",
]
}
resource "aws_iam_service_linked_role" "es" {
aws_service_name = "es.amazonaws.com"
}
EDIT: Oddly enough, when I removed using the ACM cert and moved back to using HTTP (port 80) for my ALB Listener, the ES domain was provisioned.
Not sure what to make of this but clearly the ACM cert is interfering with the ES domain creation. Or I am doing something wrong with the ACM creation. Here is how I do it and use it -
resource "aws_acm_certificate" "ssl_cert" {
domain_name = "api.xxxx.io"
validation_method = "DNS"
tags = {
Environment = "development"
}
lifecycle {
create_before_destroy = true
}
}
resource "aws_alb_listener" "alb_listener" {
load_balancer_arn = "${aws_alb.alb.id}"
port = "443"
protocol = "HTTPS"
ssl_policy = "ELBSecurityPolicy-2016-08"
certificate_arn = "${aws_acm_certificate.ssl_cert.arn}"
default_action {
target_group_arn = "${aws_alb_target_group.default.id}"
type = "forward"
}
}
The cert is validated and issued by AWS pretty fast as far as I can see in the console. And as seen, it has nothing to do with the ES domain per say.
It sometimes occurs that when it creates an ES-domain before enabling a service-linked role, even though using depends_on.
maybe you can try using local-exec provisioner to wait.
resource "aws_iam_service_linked_role" "es" {
aws_service_name = "es.amazonaws.com"
provisioner "local-exec" {
command = "sleep 10"
}
}
Below one is enough for service-linked role creation, also incl the role in the depends_on
resource "aws_iam_service_linked_role" "es" {
aws_service_name = "es.amazonaws.com"
}
I'm attempting to write an authorizer to protect calls to a lambda using the serverless framework. I'm using Ruby.
Configuration:
provider:
name: aws
runtime: ruby2.5
iamRoleStatements:
- Effect: Allow
Action:
- KMS:Decrypt
Resource: ${self:custom.kmsSecrets.keyArn}
functions:
authorize:
handler: handler.authorize
hello:
handler: handler.protected
events:
- http:
path: protected
method: get
authorizer: authorize
The authorizer:
def authorize(event:, context:)
if is_authorized?
{
"policyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": "execute-api:Invoke",
"Resource": [ context.invoked_function_arn ],
"Effect": "Allow"
}
]
},
"principalId": "seeker"
}.to_json
end
end
The authentication I'd like to implement is token based: the is_authorized? method would receive the token and then return a policy that would allow access to the protected lambda function.
I'm not entirely sure what goes in the PrincipalId argument - I have no user.id.
Right now it complains that: seeker-dev-authorize is not authorized to perform: iam:CreatePolicy on resource: policy seeker-allowed which leaves me quite confused: I can't create a policy... on the policy? And where should I set this permission? On IAM or serverless.yml? Because I've set the permissions to encode/decode keys in serverless, maybe I should do the same with this?
I haven't used custom authorizers before, but I put together a small hello world project to try this out and this is what I found.
The protected function and the authorizer function:
def authorize(event:, context:)
{
"principalId": "test",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": "execute-api:Invoke",
"Effect": "Allow",
"Resource": event["methodArn"]
}
]
}
}
end
def hello(event:, context:)
{ statusCode: 200, body: JSON.generate('This is a protected endpoint!') }
end
Note that I'm returning the hash and not a string with to_json, I got an error back from the authorizer when using to_json.
Also note that I'm using event["methodArn"] to get the protected lambda ARN, using context.invoked_function_arn also caused me an error.
Besides that, not including an Authorization header in the request, will return an "Unauthorized error":
curl -X GET https://endpoint/dev/hello -H 'Authorization: test'
Lastly, about the principalId:
The principalId is a required property on your authorizer response. It represents the principal identifier for the caller. This may vary from application-to-application, but it could be a username, an email address, or a unique ID.
Source: https://www.alexdebrie.com/posts/lambda-custom-authorizers/
I have AWS dynamo DB table where I store information for AWS Cognito users. I created the table to be private so that only the owner of a row in the table can read/write the data (based on cognito authentication). I need to get the data for the user through a lambda function. I created the IAM role for the function in this way:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:Query",
"dynamodb:UpdateItem"
],
"Resource": [
"arn:aws:dynamodb:XX-XXXX-X:XXXXXXX:table/tablename"
],
"Condition": {
"ForAllValues:StringEquals": {
"dynamodb:LeadingKeys": [
"${cognito-identity.amazonaws.com:sub}"
]
}
}
}
]
}
In the lamba function (node.js) I need to get the information stored from the user so I call:
let ddb = new AWS.DynamoDB({ apiVersion: 'latest' });
var params = {
TableName: tablename,
Key: { 'id': {S: event.queryStringParameters.user_id}
}
};
ddb.getItem(params, function(err, data) {
if (err) {
console.log("Error", err);
}
else {
console.log("Success", data);
}
});
I get the error:
Error { AccessDeniedException: User: arn:aws:sts::xxxxxx:assumed-role/lambda_dynamo/getPmtDetails is not authorized to perform: dynamodb:GetItem on resource
How can I call getItem with the cognito id in order to retrive the row that belong to the user?