Parse user/metadata/runtime data from cloudformation into EC2 - windows

I am trying to setup my windows 2016 instances in cloudformation stack and It requires some inputs from user that is taken in parameter section to be transferred into the instance and the software inside would require those parameters. Till now i only found that i can send it by parsing it inside a powershell script while saving the data into a text file. Is there anyother method? or more efficient method to do this???

See Bootstrapping AWS CloudFormation Windows Stacks.
In short, you typically use AWS::CloudFormation::Init metadata and cfn-init in UserData to bootstrap and initialize your instances.

Here is an AWS CloudFormation template that can take a parameter from the CloudFormation stack and put that information onto the Windows instance:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Username:
Type: String
Description: Ask for a username, will be placed on the instance
WindowsAmiId:
Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
Default: /aws/service/ami-windows-latest/Windows_Server-2019-English-Full-Base
KeyName:
Type: String
Description: Keyname for the keypair to use when launching EC2 instances
Resources:
AppInstance:
Type: AWS::EC2::Instance
Properties:
KeyName: !Ref KeyName
ImageId: !Ref WindowsAmiId
InstanceType: t2.large
UserData:
Fn::Base64: !Sub |
<powershell>
Set-Content -Path 'C:\username.txt' -Value '${Username}'
</powershell>
The result is a file in C:\username.txt that contains the Username parameter that was provided when the stack was launched.
Alternatively, you could call describe-stack to retrieve the parameters.

Related

SSM parameters SAM local aws set up: unable to set properties to work in local

people.
I have been trying to set up a project to test my lambdas in a local environment. We are using sam to mimic AWS resources and everything works pretty fine with one exception: for business reasons, I had to include SSM parameters. When we try to read the parameters using SAM with sam local start-lambda ... whatever, the lambda code is not able to recover the parameters. I know that AWS credentials are fine as I can connect to AWS, but we don't want to use real AWS services to do that, we want to pass, set, define; whatever you want to call it is fine, SSM parameters in my local environment and then use them with sam local start-lambda whatever, without AWS connection therefore we can use parameters for testing only in local.
I have read the following post, How to access SSM Parameter Store from SAM lambda local in node
And this issue in github:
https://github.com/aws/aws-sam-cli/issues/616
It is mentioned that the way to do it is by using --env-vars but it is not working so far.
This is my template.
Parameters:
IdentityNameParameter:
Type: AWS::SSM::Parameter::Value<String>
Resources:
MyLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Handler: handler::handleRequest
CodeUri: lambdaFunction
Runtime: java11
Timeout: 40
Environment:
Variables:
AWS_ACCESS_KEY_ID: "keyid"
AWS_SECRET_ACCESS_KEY: "accesskey"
AWS_DEFAULT_REGION: "us-west-1"
AWS_REGION: "us-west-1"
This is what I use to start the lambda
sam local start-lambda --host 0.0.0.0 -d 5859 --docker-volume-basedir /folderWithClasses --container-host host.docker.internal --debug --env-vars env.json
This is the env.json:
{
"Parameters": {
"IdentityNameParameter": "admin"
}
}
I guess there is no support to do this. If so, what's the point of having SAM to test it in local if you need AWS to actually test?
Any clue?

Cannot add lambda layer via GUI or programmatically, but works via cloud formation. Failed to unzip archive: Zip file contains invalid files/folders;

I'm following this excellent article: https://github.com/vittorio-nardone/selenium-chromium-lambda/
End to end the example works correctly - I just want to re-use the layers that are created in my own function.
Whatever method I use to try and add the layer fails. Manually using GUI,Boto3 in python or the AWS CLI, although it is working on the function setup up by the cloud formation script.
aws lambda update-function-configuration --function-name='test_headless' --layers='arn:aws:lambda:eu-west-1:366134052888:layer:SeleniumChromiumLayer:1'
An error occurred (InvalidParameterValueException) when calling the UpdateFunctionConfiguration operation: Failed to unzip archive: Zip file contains invalid files/folders;
Clearly I'm missing something here:
Partial Extract from cloud formation script:
ScreenshotFunction:
Type: AWS::Lambda::Function
Properties:
Runtime: python3.7
Description: Function to take a screenshot of a website.
Handler: src/lambda_function.lambda_handler
Role:
Fn::GetAtt: [ "ScreenshotFunctionRole", "Arn" ]
Environment:
Variables:
PYTHONPATH: "/var/task/src:/opt/python"
PATH: "/opt/bin:/opt/bin/lib"
URL:
Ref: WebSite
BUCKET:
Ref: BucketName
DESTPATH:
Ref: ScreenshotsFolder
Timeout: 60
MemorySize: 2048
Code:
S3Bucket:
Ref: BucketName
S3Key:
Fn::Sub: '${SourceFolder}/ScreenshotFunction.zip'
Layers:
- Ref: SeleniumChromiumLayer
SeleniumChromiumLayer:
Type: AWS::Lambda::LayerVersion
Properties:
CompatibleRuntimes:
- python3.7
- python3.6
Content:
S3Bucket:
Ref: BucketName
S3Key:
Fn::Sub: '${SourceFolder}/SeleniumChromiumLayer.zip'
Description: Selenium and Chromium Layer for Python3.6
How is it that the contents of the zip used can be OK to add via cloudformation but not in any other manner?
Seems there was some corruption on a function - add the layer to another function worked successfully

How to debug and run multiple lambdas locally

I would like to build .NET HTTP API using aws lambdas. These lambdas will be called by UI and some other systems via api gateway. Obviously in local environment I would like to run/debug these.
What I have tried:
a) Using the mock tool that comes with AWS Visual Studio templates. You can call individual lambdas but I couldn't figure out how I can call them from e.g. postman using normal rest calls. I don't know how mock tool makes those calls as chrome/firefox doesn't show them.
b) Using sam local start-api. Here is what I did:
sam --version
SAM CLI, version 1.22.0
sam init (choose aws quick start template, package type Image and amazon/dotnet5.0-base as base image)
I can build the solution with sam build, run it wit sam local start-api and I can browse to http://localhost:3000/hello and it works. Problem is that I would need to do build in VS + do those steps every time I change code. Also no easy way to attach debugger.
So what is the recommended way to do this? I know you can run whole .NET web api inside lambda but that doesn't sound like a good technical solution. I am assuming I am not the first person building HTTP api using lambdas.
It might be worth considering running a lambda-like environment in Docker.
While including the dotnet tools you need might not be feasable in actual Lambda, It might be feasible to either include them in a Docker image, or bind mounted to a docker container. These images from lambci can help with that: https://hub.docker.com/r/lambci/lambda/
You can use sam local
https://github.com/thoeni/aws-sam-local
Create API with API gateway example
Resources:
ApiGatewayToLambdaRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: ['sts:AssumeRole']
Effect: Allow
Principal:
Service: ['apigateway.amazonaws.com']
Version: '2012-10-17'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaRole
- arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs
ApiGateway:
Type: AWS::Serverless::Api
Properties:
StageName: test
EndpointConfiguration: REGIONAL
DefinitionBody:
swagger: "2.0"
info:
title: "TestAPI"
description: TestAPI description in Markdown.
paths:
/create:
post:
x-amazon-apigateway-integration:
uri:
!Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MyLambda.Arn}/invocations
credentials: !GetAtt ApiGatewayToLambdaRole.Arn
responses: {}
httpMethod: POST
type: aws
x-amazon-apigateway-request-validators:
Validate query string parameters and headers:
validateRequestParameters: true
validateRequestBody: false
LambdaRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: ['sts:AssumeRole']
Effect: Allow
Principal:
Service: [lambda.amazonaws.com]
Version: '2012-10-17'
Path: /
Policies:
- PolicyName: CodeBuildAccess
PolicyDocument:
Version: '2012-10-17'
Statement:
- Action:
- logs:*
- lambda:*
- ec2:CreateNetworkInterface
- ec2:DescribeNetworkInterfaces
- ec2:DeleteNetworkInterface
Effect: Allow
Resource: "*"
Version: '2012-10-17'
MyLambda:
Type: AWS::Serverless::Function
Properties:
Role: !GetAtt LambdaRole.Arn
Handler: myfunctionname.lambda_handler
CodeUri: ./src/myfunctionname
Events:
SCAPIGateway:
Type: Api
Properties:
RestApiId: !Ref ApiGateway
Path: /create
Method: POST
...
Build :
Time sam build --use-container --template backend/template.yam
Invoke Lambda Locally:
The command to invoke Lambda locally is sam local invoke and -e flag is used to specify the path to the Lambda event.
$ sam local invoke -e event.json
When it is run, it will look something like this:
$ sam local invoke MyLambda -e event.json
2021-04-20 11:11:09 Invoking index.handler
2021-04-20 11:11:09 Found credentials in shared credentials file:
~/.aws/credentials
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-invoke.html
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-start-api.html
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-debugging.html
https://github.com/ashiina/lambda-local

aws serverless - exporting output value for cognito authorizer

I'm trying to share cognito authorizer between my stacks for this I'm exporting my authorizer but when I try to reference it in another service I get the error
Trying to request a non exported variable from CloudFormation. Stack name: "myApp-services-test" Requested variable: "ExtApiGatewayAuthorizer-test".
Here is my stack where I have authorizer defined and exported:
CognitoUserPool:
Type: AWS::Cognito::UserPool
Properties:
# Generate a name based on the stage
UserPoolName: ${self:provider.stage}-user-pool
# Set email as an alias
UsernameAttributes:
- email
AutoVerifiedAttributes:
- email
ApiGatewayAuthorizer:
Type: AWS::ApiGateway::Authorizer
Properties:
Name: CognitoAuthorizer
Type: COGNITO_USER_POOLS
IdentitySource: method.request.header.Authorization
RestApiId: { "Ref": "ProxyApi" }
ProviderARNs:
- Fn::GetAtt:
- CognitoUserPool
- Arn
ApiGatewayAuthorizerId:
Value:
Ref: ApiGatewayAuthorizer
Export:
Name: ExtApiGatewayAuthorizer-${self:provider.stage}
this is successfully exported as I can see it in stack exports list from my aws console.
I try to reference it in another stack like this:
myFunction:
handler: handler.myFunction
events:
- http:
path: /{userID}
method: put
cors: true
authorizer:
type: COGNITO_USER_POOLS
authorizerId: ${myApp-services-${self:provider.stage}.ExtApiGatewayAuthorizer-${self:provider.stage}}
my env info
Your Environment Information ---------------------------
Operating System: darwin
Node Version: 12.13.1
Framework Version: 1.60.5
Plugin Version: 3.2.7
SDK Version: 2.2.1
Components Core Version: 1.1.2
Components CLI Version: 1.4.0
Answering my own question
it looks like I should have imported by output name not output export name, which is bit weird and all the docs I have seen point to export name, but this is how I was able to make it work
replaced this -
authorizerId:${myAppservices-${self:provider.stage}.ExtApiGatewayAuthorizer-${self:provider.stage}}
with -
authorizerId: ${myApp-services-${self:provider.stage}.ApiGatewayAuthorizerId}
If you come across Trying to request a non exported variable from CloudFormation. Stack name: "myApp-services-test" Requested variable: "ExtApiGatewayAuthorizer-test"., when exporting profile i.e.,
export AWS_PROFILE=your_profile
It must be done on the terminal window where you are doing sls deploy not on another terminal window. It is a silly mistake but I don't want anyone else waste their time around that

How to access private AWS resources in AWS SAM LOCAL when start-api testing

I've been working with AWS SAM Local to create and test a lambda / api gateway stack before shipping it to production. I have recently ran into a brick wall when trying to access private resources (RDS) when testing locally (sam local start-api --profile [profile]). I'm able to connect to some of these private resources if I do some ssh tunneling, but was wondering if I am able to test locally without tunneling using VPC.
Below is an example sam template:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Example Stack
Globals:
Function:
Timeout: 3
Resources:
ExampleFunction:
Type: 'AWS::Serverless::Function'
Properties:
Handler: index.example
Runtime: nodejs8.10
CodeUri: .
Description: 'Just an example'
MemorySize: 128
Role: 'arn:aws:iam::[arn-role]'
VpcConfig:
SecurityGroupIds:
- sg-[12345]
SubnetIds:
- subnet-[12345]
- subnet-[23456]
- subnet-[34567]
Events:
Api1:
Type: Api
Properties:
Path: /example
Method: GET
After reading through a lot of documentation and searching stackoverflow for anything that would help... I ended up joining the #samdev slack channel and asked for help. I was provided some guidance and a great guide on setting up OpenVPN on an EC2 instance.
The set up was super easy (completed in under 30 minutes) and the EC2 instance uses a pre-baked AMI image. Make sure you assign the new EC2 instance to the appropriate VPC containing the resources you need access to.
Here is a link to the OpenVPN guide: https://openvpn.net/index.php/access-server/on-amazon-cloud.html
You can request an invite to the #samdev slack channel here: https://awssamopensource.splashthat.com/

Resources