google deployment manager assigning IAM policies at project - google-deployment-manager

I am using to update a project with IAM policies. in GCP deployment manager's templates, they are using python Jinja file, but I would like to add IAM policy (assign a user/service account some role). Can someone modify the Jinja/ config file and pinpoint how I can modify?
https://github.com/GoogleCloudPlatform/deploymentmanager-samples/blob/master/examples/v2/project_creation/config.yaml
https://github.com/GoogleCloudPlatform/deploymentmanager-samples/blob/master/examples/v2/project_creation/project.py

Please follow Adam Ocsvari's example to assign IAM policy. The old method was to get all the IAM binding policies, add a few role -> members bindings, then set all the bindings. He's providing a new method using 'type': 'gcp-types/cloudresourcemanager-v1:virtual.projects.iamMemberBinding'. I used one of the links he provided to find the python template that assigned IAM policy bindings. The code there has a nested loop. I only needed to create a single service account and assign 1 binding:
service-accounts.py
def GenerateConfig(context):
project_id = context.env['project']
service_account = context.properties['service-account']
resources = [
{
'name': service_account,
'type': 'iam.v1.serviceAccount',
'properties': {
'accountId': service_account,
'displayName': service_account,
'projectId': project_id
}
},
{
'name': 'bind-iam-policy',
'type': 'gcp-types/cloudresourcemanager-v1:virtual.projects.iamMemberBinding',
'properties': {
'resource': project_id,
'role': 'roles/dataflow.admin',
'member': 'serviceAccount:$(ref.' + service_account + '.email)'
},
'metadata': {
'dependsOn': [service_account]
}
}
]
return {'resources': resources}
service-accounts.yaml
imports:
- path: service-accounts.py
resources:
- name: service-accounts
type: service-accounts.py
properties:
project: [*YOUR_PROJECT_ID*]
service-account: k8s-service-account
this example creates a k8s-service-account and assigns Dataflow admin role to it. Make sure you Grant Deployment Manager permission to set IAM policies before you start.

Here's a jinja snippet that creates a new service account and adds it as an owner to an existing project. This requires assigning deployment manager the proper access to manage IAM for the project.
{% set deployment = env['deployment'] %}
{% set project = env['project'] %}
resources:
- name: {{ deployment }}-svc-account
type: iam.v1.serviceAccount
properties:
accountId: {{ deployment }}-svc-account
displayName: {{ deployment }}-svc-account
- name: get-iam-policy
action: gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.getIamPolicy
properties:
resource: {{ project }}
metadata:
runtimePolicy:
- 'UPDATE_ALWAYS'
- name: patch-iam-policy
action: gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.setIamPolicy
properties:
resource: {{ project }}
policy: $(ref.get-iam-policy)
gcpIamPolicyPatch:
add:
- role: roles/owner
members:
- serviceAccount:$(ref.{{ deployment }}-svc-account.email)

Please avoid using these solutions:
gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.getIamPolicy
gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.setIamPolicy
It can cause concurrent IAM policy update errors. The Deployment Manager team is providing a new type binding this 2 actions together:
'type': 'gcp-types/cloudresourcemanager-v1:virtual.projects.iamMemberBinding',
Check out the following implementations as part of the Cloud Foundation Toolkit provided by Google Cloud:
Cloud Foundation Toolkit NEW repo - IAM binding
Cloud Foundation Toolkit OLD repo - IAM binding
Cloud Foundation Toolkit NEW repo - Project Creation Factory

You need to make changes to the below part of the config.yaml file and add the users or service accounts according to your need under the members line.
iam-policy:
bindings:
- role: roles/owner
members:
- serviceAccount:98765432111#cloudservices.gserviceaccount.com
- serviceAccount:98765432100#cloudservices.gserviceaccount.com
- role: roles/viewer
members:
- user:iamtester#deployment-manager.net
For example: You can add -user:foo#bar.com under members tab in proper section to make it owner or viewer of the project.

My code to add permissions to a service account.
{% set deployment = env['deployment'] %}
{% set project = env['project'] %}
resources:
- name: get-iam-policy
action: gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.getIamPolicy
properties:
resource: {{ project }}
metadata:
runtimePolicy:
- 'UPDATE_ALWAYS'
- name: patch-iam-policy
action: gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.setIamPolicy
properties:
resource: {{ project }}
policy: $(ref.get-iam-policy)
gcpIamPolicyPatch:
add:
- role: roles/bigquery.dataEditor
members:
- serviceAccount: <service account>

According to Google, the preferred way is to NOT use actions. Instead use type providers that introduce state within deployment manager. For a full list of available types, use the following command:
gcloud beta deployment-manager types list --project gcp-types
The example that Hil Liao uses is the correct one for setting the bindings.

Related

How to pass api endpoint parameter to lambda function in serverless

I created a lambda function behind an API gateway, and I am trying to implement a healthcheck for it as well, but right now I need to do this in 2 steps:
run serverless deploy so it spits out the endpoint for the api gateway.
manually insert the endpoint into the healthcheck.environment.api_endpoint param, and run serverless deploy a second time
functions:
app:
handler: wsgi_handler.handler
events:
- http:
method: ANY
path: /
- http:
method: ANY
path: '{proxy+}'
healthcheck:
environment:
api_endpoint: '<how to reference the api endpoint?>'
handler: healthcheck.handler
events:
- schedule: rate(1 minute)
custom:
wsgi:
app: app.app
pythonBin: python3
packRequirements: false
pythonRequirements:
dockerizePip: non-linux
Is there a way to get the reference to the api gateway on creation time, so it can be passed as an environment variable to the healthcheck app? the alternative I can think of, is to basically create a specific serverless.yaml just for the healthcheck purpose.
I noticed I can reconstruct the endpoint in the lambda, and grab the id like so:
healthcheck:
environment:
api_endpoint: !Ref ApiGatewayRestApi
region: ${env:region}
handler: healthcheck.handler
events:
- schedule: rate(1 minute)
and then reconstruct:
def handler(event, context):
api_id = os.environ['api_endpoint']
region = os.environ['region']
endpoint = f"https://{api_id}.execute-api.{region}.amazonaws.com/dev"
With a bit of cloudformation you can inject it directly! That way you don't need to compute it everytime in your lambda handler.
api_endpoint: {
'Fn::Join': [
'',
[
{ Ref: 'WebsocketsApi' },
'.execute-api.',
{ Ref: 'AWS::Region' },
'.',
{ Ref: 'AWS::URLSuffix' },
"/${opt:stage, self:provider.stage, 'dev'}",
],
],
},
(this is an example for a websocket API)
Is your API created through the same/another Cloudformation Stack? If so, you can reference is directly (same stack) or through a CloudFormation variable export.
https://carova.io/snippets/serverless-aws-cloudformation-output-stack-variables
https://carova.io/snippets/serverless-aws-reference-other-cloudformation-stack-variables
If you created it outside of CloudFormation, ie. in the aws console, then you'll need to add the ids into the template. Most likely by creating different environment variables based on the stage.

Call service from existing api gateway using base path mappings

Our API has the following endpoints:
POST /users - create a user
GET /users/{userId} - get a particular user
GET /posts/{postId} - get a particular post
GET /posts/{postId}/users - get the users who contributed to this post
I have defined two services: users-service and posts-service. In these two services I define the lambdas like so. I'm using the serverless-domain-manager plugin to create base path mappings:
/users-service/serverless.yaml:
service: users-service
provider:
name: aws
runtime: nodejs10.x
stage: dev
plugins:
- serverless-domain-manager
custom:
customDomain:
domainName: 'serverlesstesting.example.com'
basePath: 'users'
stage: ${self:provider.stage}
createRoute53Record: true
functions:
create:
name: userCreate
handler: src/create.handler
events:
- http:
path: /
method: post
get:
name: userGet
handler: src/get.handler
events:
- http:
path: /{userId}
method: get
/rooms-service/serverless.yaml:
service: posts-service
provider:
name: aws
runtime: nodejs10.x
stage: dev
plugins:
- serverless-domain-manager
custom:
customDomain:
domainName: 'serverlesstesting.example.com'
basePath: 'posts'
stage: ${self:provider.stage}
createRoute53Record: true
functions:
get:
name: postsGet
handler: src/get.handler
events:
- http:
path: /{postId}
method: get
getUsersForPost:
handler: userGet ?
events: ??
The problem is that the GET /posts/{postId}/users actually calls the same userGet lambda from the users-service. But the source for that lambda lives in the users-service, not the posts-service.
So my question becomes:
How do I reference a service from another service using base path mappings? In other words, is it possible for the posts service to actually make a call to the parent custom domain and into the users base path mapping and its service?
Consider or refer below approach
https://serverless-stack.com/chapters/share-an-api-endpoint-between-services.html

Developer authenticated identities federated via an Identity Pool cannot make AppSync calls in Amplify project

Currently, I'm authenticating Linkedin users and calling GetOpenIdTokenForDeveloperIdentity and GetCredentialsForIdentity in a Lambda, and signing in those users with Auth.federatedSignIn(), I am able to retrieve the current authenticated user and credentials.
Using Amplify's APIClass or AWSAppSyncClient, however, I cannot get these users to make GraphQL calls through AppSync, which is configured to authorize Cognito User Pool and AWS IAM users. The Cognito User Pool users have no problem making AppSync calls when the authentication type is changed accordingly.
I have tried each of the following for the federated users:
API.graphql({
query: queries.getUserProfile,
variables: {
input: {
email,
}
},
authMode: 'AWS_IAM'
})
const client = new AWSAppSyncClient({
url: process.env.GRAPHQL_ENDPOINT,
region: process.env.AWS_REGION,
auth: {
type: AUTH_TYPE.AWS_IAM,
credentials: async () => Auth.currentCredentials(),
},
disableOffline: true,
});
client.query({
query: gql(queries.getUserProfile),
variables: {
input: {
email,
},
},
fetchPolicy: 'no-cache',
})
The Identity Pool is provisioned through Serverless resources/CloudFormation templates, and I've set the trust relationship and authenticated role like so:
ProjectAuthRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Federated: "cognito-identity.amazonaws.com"
Action:
- "sts:AssumeRoleWithWebIdentity"
Condition:
StringEquals:
"cognito-identity.amazonaws.com:aud":
- Ref: ProjectIdentityPool
ForAnyValue:StringLike:
"cognito-identity.amazonaws.com:amr": authenticated
AuthPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- mobileanalytics:PutEvents
- cognito-sync:*
- cognito-identity:*
Resource:
- "*"
- Effect: Allow
Action:
- appsync:GraphQL
Resource:
- "*"
Roles:
- Ref: ProjectAuthRole
I have been able to make GraphQL calls by adding the #aws_iam schema directive to every type and input definition, but it sounds like this shouldn't even be necessary, as the authorization should be happening AWS AppSync GraphQL API level.
Any ideas where else I might have forgotten to configure?
It looks like the issue was the additionalAuthenticationProviders in our serverless-appsync-plugin YAML file.
My understanding is that schema-level directives must be applied to type definitions in one's schema if you want additional authentication providers (in our case, AWS_IAM) to have access alongside the default authentication provider (in our case, AMAZON_COGNITO_USER_POOLS).
Source for further details:
https://docs.aws.amazon.com/appsync/latest/devguide/security.html#using-additional-authorization-modes

AWS - API keys available on the Serverless Offline framework?

I use Serverless Offline to develop a Web project.
I need of API Keys to access to resource on Serverless AWS Lamda.
I have a serverless.yml with my service and my provider.
In Postman, I access to my route (http://127.0.0.1:3333/segments/UUID/test), and I haven't any error (as Forbidden message), the Lambda is executed...
test:
handler: src/Api/segment.test
events:
- http:
path: segments/{segmentUuid}/test
method: post
request:
parameters:
paths:
segmentUuid: true
private: true
The route in question is not protected by private.
https://www.npmjs.com/package/serverless-offline#token-authorizers
Serverless-offline will emulate the behaviour of APIG and create a
random token that's printed on the screen. With this token you can
access your private methods adding x-api-key: generatedToken to your
request header. All api keys will share the same token. To specify a
custom token use the --apiKey cli option.
Command will look like this:
sls offline --apiKey any-pregenerated-key
For local dev use this inside serverless.yml:
custom:
serverless-offline:
apiKey: 'your-key-here'
Or this inside serverless.ts:
custom: {
'serverless-offline': {
apiKey: 'your-key-here',
},
},
Given latest changes this configuration worked for me with serverless offline:
provider: {
name: 'aws',
region: region,
runtime: 'nodejs14.x',
stage: stage,
apiGateway:{
apiKeys: [{
name: 'test name',
value: 'sadasfasdasdasdasdafasdasasd'
}],
},
},
https://github.com/dherault/serverless-offline/issues/963

how to generate swagger document with tags using serverless-aws-documentation plugin for serverless

I am using serverless-aws-documentation plugin to auto-generate swagger-doc. Followed all the steps provided at : https://github.com/9cookies/serverless-aws-documentation. Under documentation key I am defining tags but it is not getting generated in the output swagger doc. Following is the sample handler :
functions:
get_tickets:
handler: handler.ticket_handler.get_tickets
events:
- http:
path: tickets
method: get
cors: true
documentation:
tags:
- private
- admin
summary: "Get list of ticket"
description: "This ticket will provide you list of tickets"
I want to segrigate APIs depending on the tags, but not able to achieve it. Thanks in advance for the help.
Try to add the serverless-aws-documentation plugin in the serverless.yml
plugins:
- serverless-aws-documentation
Add the infor and models documentation in the custom section:
custom:
myStage: ${opt:stage, self:provider.stage}
profiles:
dev: road-we-go
prod: road-we-
documentation:
info:
version: "1.0.0"
name: "Example API"
description: "Example API description"
termsOfService: "https://example.com/terms-of-service"
contact:
name: "Example API"
url: "https://example.com"
email: "dev#example.com"
licence:
name: "Licensing"
url: "https://example.com/licensing"
models:
-
name: "StoreAudioSuccess"
description: "Model for store audio"
contentType: "application/json"
schema: ${file(swagger/audios/storeResponse.
Add the function documentation:
If you want to add the custom models like RequestStore and StoreAudioSuccess check the serverless-aws-documentation documentation and the json-schema docs
functions:
Update:
handler: src/functions/update.handler
timeout: 30
memory: 128
events:
- http:
method: put
private: true
cors: true
path: api/v1/update
documentation:
summary: "Update "
description: "Update a record"
tags:
- "Users"
requestModels:
"application/json": "RequestStore"
methodResponses:
-
statusCode: "200"
responseModels:
"application/json": "StoreUserSuccess"
To download the swagger documentation you need to run this command:
First you need to deploy you project
sls downloadDocumentation --outputFileName=swagger.json
Which version are you using?
According to their latest documentation https://github.com/9cookies/serverless-aws-documentation, you need to provide tags as follows i.e. within double quotes.
documentation:
tags:
- "private"
- "admin"

Resources