I am trying to deploy serverless with the credentials stored in my aws-vault keychain.
However when I run aws-vault exec myprofile -- sls deploy I receive the following error:
An error occurred: MyLambdaFunction - AccessDenied. User doesn't have permission to call iam:GetRole.
The user has AdministratorAccess and I also gave him IAMFullAccess (which should not be needed)
When I deploy with the same credentials stored in ~.aws/credentials with sls deploy
it works.
According to the docs the session token generated by aws-vault has some restrictions:
You cannot call any IAM API operations unless MFA authentication information is included in the request.
You cannot call any AWS STS API except AssumeRole or GetCallerIdentity.
A work around is to use the --no-session
so aws-vault exec myprofile --no-session -- sls deploy works without any error.
can you help me?
I'm doing a continuous deployment with circle.ci...
The unity tests, build and push is working.
But when i try update the ecs task, i'm getting the error:
ERROR: Could not connect to the endpoint URL: https://ecs.********.amazonws.com/
in the circle.ci i'm using the orbs
aws-ecs: circleci/aws-ecs#2.2.1
Any idea what is happen?
I don't know what to think/search anymore..
Update:
I tried run the command
aws ecs update-service --service ng-prod-service --cluster ng-cluster --force-new-deployment
And to my surprise i get the same error..
But my aws login works..
aws ecr get-login-password --region sa-east-1 | docker login --username AWS --password-stdin 119120169187.dkr.ecr.sa-east-1.amazonaws.com
i think the problem is with my credentials...
So i made new credentials with new iam rules and registerd on circle.ci and then it works...
It was probably the iam rules or iam credentials...
I have configured specific Lambda alias (Please note this) as Cognito trigger using CLI as there is no provision in web console to do so. Now I am getting PreSignUp invocation failed due to the error AccessDeniedException while signing up. I am not sure about it but this might be happening as I have configured trigger using CLI. I tried to locate Cognito specific role in IAM but I didn't find such role.
So how can I update missing permissions in IAM?
I used the following CLI command to update the permissions.
aws lambda add-permission --function-name <ARN of the lambda alias> --source-arn <user-pool-arn> --action lambda:InvokeFunction --principal cognito-idp.amazonaws.com
The question is very similar to Google Cloud Tasks cannot authenticate to Cloud Run, but I am not using my own domain. I am using cloud run domain itself.
I have followed both below tutorial (both are kind of similar):
https://cloud.google.com/run/docs/triggering/using-tasks
https://cloud.google.com/tasks/docs/creating-http-target-tasks
I am not sure if there is more to the tutorial that I am missing, but below is the complete scenario.
I have some Django REST based APIs running on Google Cloud Run, and they are public. Furthermore, I can use them without any issue (though then need authentication).
I have created a Google Tasks Queue, and I am sending tasks to it using the following code
class QueueTaskMixin:
def send_task(self, payload=None):
url = 'https://my-public-cloud-run-app.a.run.app/api/conversation_types/'
client = tasks_v2.CloudTasksClient(credentials=credentials)
# `credentials` above belongs to a service account which have all sort of accesses
# Cloud SQL Admin
# Cloud SQL Client
# Cloud SQL Editor
# Cloud Tasks Admin
# Cloud Tasks Task Runner
# Service Account Token Creator
# Service Account User
# Cloud Run Admin
# Cloud Run Invoker
# Cloud Run Service Agent
# Storage Admin
parent = client.queue_path(
project=settings.GS_PROJECT_ID,
location=settings.GS_Q_REGION,
queue=settings.GS_Q_NAME)
task = {
'http_request':
{
'headers': {
'Content-type': 'application/json',
},
'http_method': tasks_v2.HttpMethod.POST,
'url': url,
'oidc_token': {
'service_account_email': settings.GS_Q_IAM_EMAIL,
# GS_Q_IAM_EMAIL is a another Service account that has
# Cloud Tasks Enqueuer
# Service Account User
# Cloud Run Invoker
}
}
}
if payload:
if isinstance(payload, dict):
payload = json.dumps(payload)
converted_payload = payload.encode()
task['http_request']['body'] = converted_payload
response = client.create_task(
request={'parent': parent, 'task': task}
)
print('Created task {}'.format(response.name))
Now I am getting PERMISSION_DENIED(7): HTTP status code 403 error.
My API logs shows the same:
"POST /api/conversation_types/ HTTP/1.1" 403 58 "-" "Google-Cloud-Tasks"
Forbidden: /api/conversation_types/
Now what I am not sure is whether this 403 error is thrown by
Two Google services trying to authorize each other
Or my API. Because my API requires authentication/authorization. As in, a user needs to log in using their username and password, and they will get a JWT token, and then they can call this API.
Referring to the documentation, I am not sure where I have to provide my APIs username/password/JWT token. The documentation says:
To authenticate between Cloud Tasks and an HTTP Target handler, Cloud Tasks creates a header token. This token is based on the credentials in the Cloud Tasks Enqueuer service account, identified by its email address.
Do I need to add this service account email address into my APIs as a user? Do I use oidc or oauth?
Any comments or answers much appreciated.
Update 1 - After removing auth from my API, Cloud Tasks is able to call the API successfully. So now how do I auth Cloud Task to be able to run my API?
Update 2 - Tried using OAuthToken and got error
400 Invalid url for HttpRequest.oauth_token. The hostname in the url must end with ".googleapis.com"
Looks like will have to go for OIDC token only.
Update 3 - Google Docs says:
OIDC tokens are signed JSON Web Tokens (JWT) and are used primarily to assert identity and not to provide any implicit authorization against a resource, unlike OAuth tokens, which do provide access.
Does that mean OIDC tokens cannot be used for authorization? Because I am getting authorization error here. Can't use OIDC, can't use OAuth, what to use then?
Update 4
As per comments from #johnhanley, I have updated my API to accept Bearer token for authentication. But even if my API is getting the token correctly, it won't be able to authorize it and will give invalid token error (I verified it using curl command. In both cases where token format is incorrect or token is incorrect, API simply return 403 Forbidden error).
Anyone can tell how to give the password (I can generate that in my API for the service account user email) to the service account so that using that password and email ID as username, OIDC can generate a token and use it to authenticate. Or am I going in the wrong direction?
Check the presence of Cloud Tasks service agent (service-[project-number]#gcp-sa-cloudtasks.iam.gserviceaccount.com) in your IAM Admin Console Page and grant him the roles/iam.serviceAccountTokenCreator role.
Don't turn off authentication, it does work. Here are some steps to follow in the form of bash shell commands and comments to explain the steps
Set some variables
PROJECT_ID='my-gcp-project-123'
TASK_ENQUER_SVC_ACCOUNT='tasks-creator-svc'
TASKS_SRV_ACCOUNT='cloud-tasks-svc'
CLOUD_RUN_SRV_ACCOUNT='cloud-run-svc'
GPC_REGION='australia-southeast1'
Create a cloud task queue
gcloud tasks queues create my-task-queue
Create a service account for sending the tasks - you need cloudtasks.enqueuer and serviceAccountUser
gcloud iam service-accounts create $TASK_ENQUER_SVC_ACCOUNT \
--display-name "Service account for sending cloud tasks"
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member \
serviceAccount:$TASK_ENQUER_SVC_ACCOUNT#${PROJECT_ID}.iam.gserviceaccount.com \
--role "roles/logging.logWriter"
# cloudtasks.enqueuer role is needed to create tasks
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member \
serviceAccount:$TASK_ENQUER_SVC_ACCOUNT#${PROJECT_ID}.iam.gserviceaccount.com \
--role "roles/cloudtasks.enqueuer"
# service account user role is needed for permission iam.serviceAccounts.actAs
# so a task can be generated as service account $TASKS_SRV_ACCOUNT
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member \
serviceAccount:$TASK_ENQUER_SVC_ACCOUNT#${PROJECT_ID}.iam.gserviceaccount.com \
--role "roles/iam.serviceAccountUser"
Create the service account for running the Cloud Run service
gcloud iam service-accounts create $CLOUD_RUN_SRV_ACCOUNT \
--display-name "Service account for running cloud run service"
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member \
serviceAccount:$CLOUD_RUN_SRV_ACCOUNT#${PROJECT_ID}.iam.gserviceaccount.com \
--role "roles/logging.logWriter"
Create the service account that cloud-tasks will use - no special permissions needed
gcloud iam service-accounts create $TASKS_SRV_ACCOUNT \
--display-name "Service account for the cloud-tasks"
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member \
serviceAccount:$TASKS_SRV_ACCOUNT#${PROJECT_ID}.iam.gserviceaccount.com \
--role "roles/logging.logWriter"
Deploy your cloud run service with ingress=all and no-allow-unauthenticated
Cloud tasks does not use serverless VPC connectors, so ingress needs to be all
gcloud run deploy my-cloud-run-service \
--image=us.gcr.io/${PROJECT_ID}/my-docker-image \
--concurrency=1 \
--cpu=2 \
--memory=250Mi \
--max-instances=10 \
--platform=managed \
--region=$GPC_REGION \
--no-allow-unauthenticated \
--port=8080 \
--service-account=$CLOUD_RUN_SRV_ACCOUNT#${PROJECT_ID}.iam.gserviceaccount.com \
--ingress=all \
--project=${PROJECT_ID}
You must now give permission for TASKS_SRV_ACCOUNT to call the Cloud Run target service
gcloud run services add-iam-policy-binding my-cloud-run-service \
--member=serviceAccount:$TASKS_SRV_ACCOUNT#${PROJECT_ID}.iam.gserviceaccount.com \
--role=roles/run.invoker \
--region=$GPC_REGION
You can then send your cloud tasks with authentication tokens as per instructions at this step
https://cloud.google.com/tasks/docs/creating-http-target-tasks#token
The service_account_email parameter is the TASKS_SRV_ACCOUNT created above
I can add permission to Lambda function with the aws lambda add-permission command in AWS CLI. But I can't remove permission using aws lambda remove-permission command.
I run this command:
aws lambda remove-permission --function-name LAMBDA_NAME --statement-id SID
Here is the error:
An error occurred (ServiceException) when calling the RemovePermission
operation (reached max retries: 4): An error occurred and the request
cannot be processed.
Hope someone helps.