AWS Elasticsearch failing for _refresh request - elasticsearch

I'm using IAM user ARN based access policy for AWS Elasticsearch. As per the documentation signing requests using AWS4Signer in RestHighLevelClient for elasticsearch.
All of the requests - put/get/search are working. However, POST /_refresh used at the end of saving a document is failing saying:
"The request signature we calculated does not match the signature you provided"
The policy used is as follows:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::accountId:user/esUser"
},
"Action": "es:*",
"Resource": "arn:aws:es:ap-south-1:accountId:domain/myDomain/*"
}
]
}
Is any action I'm missing?

Related

giving all lambda function access to secretmanager

I want to access aws secret manager in all my lambda functions(AWS::Serverless::Function). Currently, I have to give individual lambda function reference like below. Since I have many lambda functions it is tedious. I tried "Service": "lambda.amazonaws.com" but it didn't work.
{
"Version" : "2012-10-17",
"Statement" : [ {
"Effect" : "Allow",
"Principal" : {
"AWS" : ["arn:aws:sts::xxxxxxx:assumed-role/employer-api-getAllEmployeesFunctionRole-xxxxx/employer-api-4-getAllEmployeesFunction-xxxxx",
"arn:aws:sts::xxxxxxx:assumed-role/employee-backend-getEmployeeByIdFunctionRole-xxxxx/employee-backend-getEmployeeByIdFunction-xxxxx"
]
},
"Action" : "secretsmanager:GetSecretValue",
"Resource" : "*"
} ]
}
You can create a policy(Like below) and attach it to your lambda's execution role.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": "*"
}
]
}
This will allow your lambda function to get secret value of any Secret stored in secret manager.
According to best practice we should only allow our lambda or any other aws service to give minimal access required.
So if your lambda needs access to only one Secret manager it is best you pass the ARN of that secret in below policy and attach it to your role.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": "<ARN of secret required by lambda>"
}
]
}
You can use AWS CDK IAM module to easily create role and policies.

CloudWatch logs are not created automatically for lambdas after deployment

I'm deploying my next.js app with #sls-next/serverless-component#1.17.0-alpha.12 to AWS lambda.
I read that logs in CloudWatch should be created automatically (and actually one time it worked fine), but when trying to view logs for lambda, I'm getting this messages:
How can I create logs for lambda? or make the serverless-next.js tool to create one in deployment?
Thanks!
EDIT:
lambda execution role permissions
In JSON format:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Resource": "*",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
]
},
{
"Effect": "Allow",
"Resource": "arn:aws:s3:::myBucket/*",
"Action": [
"s3:GetObject",
"s3:PutObject"
]
}
]
}

AWS Lambda called via Cloudfront in front of S3 bucket can't assume role for Cognito User Pool to authenticate

I have an AWS Lambda that is run as a viewer request on a CloudFront instance that restricts access to an S3 bucket setup for static hosting of a website. It uses a Cognito User Pool to restrict access and verifies the credentials via AdminInitiateAuth.
The lambda runs fine using the test data directly obtained from logging the CloudFront event, however when actually called via the trigger from CloudFront I get the error:
An error occurred (AccessDeniedException) when calling the AdminInitiateAuth operation:
User: arn:aws:sts::<AWS_ACCOUNT_ID>:assumed-role/cloudfront_trigger_s3_auth_http_service/us-east-1.s3_service_resources_auth
is not authorized to perform: cognito-idp:AdminInitiateAuth on resource:
arn:aws:cognito-idp:us-west-2:<AWS_ACCOUNT_ID>:userpool/<USER_POOL_ID>
I've tried expanding my trust relationship and making sure that AWS STS can assume the role when it needs to.
cloudfront_trigger_s3_auth_http_service role trust relationship:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"cognito-idp.amazonaws.com",
"edgelambda.amazonaws.com",
"sts.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
arcimoto-service-resources-user-pool-auth policy attached to cloudfront_trigger_s3_auth_http_service role that allows cognito access
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"cognito-idp:AdminInitiateAuth",
"sts:AssumeRole"
],
"Resource": [
"arn:aws:iam::511596272857:role/cloudfront_trigger_s3_auth_http_service/us-east-1.s3_service_resources_auth",
"arn:aws:cognito-idp:us-east-1:511596272857:userpool/us-east-1_sES7sBpcg"
]
}
]
}

push data securely from lambda to elastic

I have this policy that accepts requests from a single assumed role. When I try to push data from lambda, I get an access denied error.
If I open the access to elastic server using this line, then it will work as expected.
"AWS": "*"
But it is not secure. How do I push data from lambda to elastic service that is restricted to Cognito users?
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:sts::51346970xxxx:assumed-role/document-search-CognitoAuthorizedRole-LZWR058L66O8/CognitoIdentityCredentials"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:51346970xxxx:domain/documentsearchapp/*"
}
]
}

Unable to update ElasticSearch access policy using CLI

I have an access policy that looks something like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:123456789012:domain/es01-vpc01-prod-useast1/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"123.456.123.456"
]
}
}
}
]
}
From my understanding, this should give all es related permissions to anyone at the IP 123.456.123.456. This indeed works for most things: I can access ElasticSearch, and I can also execute CLI commands such as aws es describe-elasticsearch-domain --domain-name es01-vpc01-prod-useast1
However, for some reason I cannot modify the access policy from the CLI using a command like this:
aws es update-elasticsearch-domain-config --domain-name es01-vpc01-prod-useast1 --access-policies '{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Principal\": {\"AWS\": \"*\"},\"Action\": \"es:*\",\"Resource\": \"arn:aws:es:us-east-1:123456789012:domain/es01-vpc01-prod-useast1/*\",\"Condition\": {\"IpAddress\": {\"aws:SourceIp\": [\"123.456.123.456\"]}}}]}'
I receive an error such as:
An error occurred (AccessDeniedException) when calling the
UpdateElasticsearchDomainConfig operation: User:
arn:aws:iam::123456789012:user/MyUser is not authorized to
perform: es:UpdateElasticsearchDomainConfig on resource:
arn:aws:es:us-east-1:123456789012:domain/es01-vpc01-prod-useast1
I have tried variations on the access policy, such as adding the specific role to the access policy, e.g.
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:role/MyRole"
},
"Action": [
"es:UpdateElasticsearchDomainConfig"
],
"Resource": "arn:aws:es:us-east-1:123456789012:domain/es01-vpc01-prod-useast1"
}
But still no success. What is the trick to make this work?
I know this is an old topic, but in case anyone else has the same issue.
Note that IP address whitelisting for ES only works if the ES domain is outside the VPC.
The following command works for me, the main thing is that you don't need to escape the double quotes if you are surrounding the JSON in single quotes.
Change 'my-domain', the account ID of '1234567890' and the IP address of '127.0.0.1' to your own and it should work for you.
aws es update-elasticsearch-domain-config --domain-name my-domain --access-policies '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:1234567890:domain/my-domain/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": "127.0.0.1"
}
}
}
]
}'

Resources