I'm newbies on heat yaml template loaded by OpenStack
I've got this command which works fine :
openstack server create --image RHEL-7.4 --flavor std.cpu1ram1 --nic net-id=network-name.admin-network --security-group security-name.group-sec-default value instance-name
I tried to write this heat file with the command above :
heat_template_version: 2014-10-16
description: Simple template to deploy a single compute instance with an attached volume
resources:
my_instance:
type: OS::Nova::Server
properties:
name: instance-name
image: RHEL-7.4
flavor: std.cpu1ram1
networks:
- network: network-name.admin-network
security_group:
- security_group: security-name.group-sec-default
security-group:
type: OS::Neutron::SecurityGroup
properties:
rules: security-name.group-sec-default
my_volume:
type: OS::Cinder::Volume
properties:
size: 10
my_attachment:
type: OS::Cinder::VolumeAttachment
properties:
instance_uuid: { get_resource: my_instance }
volume_id: { get_resource: my_volume }
mountpoint: /dev/vdb
The stack creation failed with the following message error :
openstack stack create -t my_first.yaml First_stack
openstack stack show First_stack
.../...
| stack_status_reason | Resource CREATE failed: BadRequest: resources.my_instance: Unable to find security_group with name or id 'sec_group1' (HTTP 400) (Request-ID: req-1c5d041c-2254-4e43-8785-c421319060d0)
.../...
Thanks for helping,
According to the template guide it is expecting the rules type is of list.
So, change the content of template as below for security-group:
security-group:
type: OS::Neutron::SecurityGroup
properties:
rules: [security-name.group-sec-default]
OR
security-group:
type: OS::Neutron::SecurityGroup
properties:
rules:
- security-name.group-sec-default
After digging, I finally found what was wrong in my heat file. I had to declare my instance like this :
my_instance:
type: OS::Nova::Server
properties:
name: instance-name
image: RHEL-7.4
flavor: std.cpu1ram1
networks:
- network: network-name.admin-network
security_groups: [security-name.group-sec-default]
Thanks for your support
Related
I have a CloudFormation template yml file that passes OptionSettings for an ElasticBeanstalk applicaiton. I can hard code values, and I can pass values from Parameters. However, I am unable to determine how to pass Parameters or Variables as the namespace.
This works:
- Namespace: aws:elasticbeanstalk:environment:process:lbtargetgroup
OptionName: Port
Value: 3000
This works (where PORTNUMBER is a parameter)
Parameters:
PORTNUMBER:
Type: String
Description: Port number
ElasticBeanstalkConfig:
Properties:
OptionSettings:
- Namespace: aws:elasticbeanstalk:environment:process:lbtargetgroup
OptionName: Port
Value: !Ref PORTNUMBER
However, this does not work (where LBTARGETGROUP is a parameter):
Parameters:
LBTARGETGROUP:
Type: String
Description: Target Group Name
ElasticBeanstalkConfig:
Properties:
OptionSettings:
- Namespace: aws:elasticbeanstalk:environment:process:!Ref LBTARGETGROUP
OptionName: Port
Value: 3000
From what I have tried, you cannot use typical Variables in a CFT (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html#template-anatomy-sections). I have also tried mappings. But I can't seem to figure out how to pass the name as a parameter.
The answer is often very simple... the following works:
Parameters:
LBTARGETGROUP:
Type: String
Description: Target Group Name
ElasticBeanstalkConfig:
Properties:
OptionSettings:
- Namespace: !Join ["", ["aws:elasticbeanstalk:environment:process:", Ref: LBTARGETGROUP]]
OptionName: Port
Value: 3000
I have am trying to deploy an aws lambda function using the SAM cli. I have some layers defined in the sam template. Testing locally using sam local start-api works quite well. The but deploying using the sam deploy --guided command throws the following error
Error: Failed to create changeset for the stack: sam-app, ex: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state: For expression "Status" we matched expected path: "FAILED" Status: FAILED. Reason: Template format error: Unresolved resource dependencies [arn:aws:lambda:us-west-1:338231645678:layer:ffmpeg:1] in the Resources block of the template
The SAM template is as follows
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
video-processor-functions
Functions to generate gif and thumbnail from uploaded videos
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Tracing: Active
Resources:
VideoProcessorFunctions:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: src/
Handler: app.lambdaHandler
Runtime: nodejs14.x
# timeout in seconds - 2 minutes
Timeout: 120
Layers:
- !Ref VideoProcessorDepLayer
- !Ref arn:aws:lambda:us-west-1:338231645678:layer:ffmpeg:1
Architectures:
- x86_64
Events:
HelloWorld:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /hello
Method: get
VideoProcessorDepLayer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: mh-video-processor-dependencies
Description: Dependencies for sam app [video-processor-functions]
ContentUri: dependencies/
CompatibleRuntimes:
- nodejs14.17
LicenseInfo: 'MIT'
RetentionPolicy: Retain
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
HelloWorldApi:
Description: "API Gateway endpoint URL for Prod stage for Hello World function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
VideoProcessorFunctions:
Description: "Generate GIF and Thumnail from Video"
Value: !GetAtt VideoProcessorFunctions.Arn
VideoProcessorFunctionsIamRole:
Description: "Implicit IAM Role created for MH Video Processor function"
Value: !GetAtt VideoProcessorFunctionsRole.Arn
Any ideas what i'm doing wrong?
You are trying to reference a logical id that doesn’t exist.
Just put the layer's ARN of 'ffmpeg':
Layers:
- !Ref VideoProcessorDepLayer
- arn:aws:lambda:us-west-1:338231645678:layer:ffmpeg:1
My question: How can Codepipeline read the value of a field in a json file which is in SourceCodeArtifact?
I have Gthub repo that contains a file imageManifest.json which looks like this:
{
"image_id": "docker.pkg.github.com/my-org/my-repo/my-app",
"image_version": "1.0.1"
}
I want my AWS Codepipeline Source stage to be able to read the value of image_version from imageManifest.json and pass it as a parameter to a CloudFormation action in a subsequent stage of my pipeline.
For reference, here is my source stage.
Stages:
- Name: GitHubSource
Actions:
- Name: SourceAction
ActionTypeId:
Category: Source
Owner: ThirdParty
Version: '1'
Provider: GitHub
OutputArtifacts:
- Name: SourceCodeArtifact
Configuration:
Owner: !Ref GitHubOwner
Repo: !Ref GitHubRepo
OAuthToken: !Ref GitHubAuthToken
And here is my deploy stage:
- Name: DevQA
Actions:
- Name: DeployInfrastructure
InputArtifacts:
- Name: SourceCodeArtifact
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: '1'
Configuration:
StackName: !Ref AppName
Capabilities: CAPABILITY_NAMED_IAM
RoleArn: !GetAtt [CloudFormationRole, Arn]
ParameterOverrides: !Sub '{"ImageId": "${image_version??}"}'
Note that image_version in the last line above is just my aspirational placeholder to illustrate how I hope to use the image_version json value.
How can Codepipeline read the value of a field in a json file which is in SourceCodeArtifact?
StepFunctions? Lambda? CodeBuild?
You can use a CodeBuild step in between Source and Deploy stages.
In CodeBuild step, read the image_version from SourceArtifact (artifact produced by soruce stage) and write to an artifact 'Template configuration' file 1 which is a configuration property of the CloudFormation action. This file can hold parameter values for your CloudFormation stack. Use this file instead of ParameterOverrides you are currently using.
Fn::GetParam is what you want. It can returns a value from a key-value pair in a JSON-formatted file. And the JSON file must be included in an artifact.
Here is the documentation and it gives you some examples: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-parameter-override-functions.html#w2ab1c13c20b9
It should be something like:
ParameterOverrides: |
{
"ImageId" : { "Fn::GetParam" : ["SourceCodeArtifact", "imageManifest.json", "image_id"]}
}
Creating private gke cluster with yaml.
Currently looking into creating a private gke. tried adding private settings in yaml file but getting error
resources:
- name: myclus
type: gcp-types/container-v1:projects.locations.clusters
properties:
parent: projects/[PROJECT_ID]/locations/[REGION]
cluster:
name: my-clus
zone: [ZONE]
network: [NETWORK]
subnetwork: [SUBNETWORK] ### leave this field blank if using the default network###
initialClusterVersion: "1.13"
nodePools:
- name: my-clus-pool1
initialNodeCount: 1
autoscaling:
enabled: true
minNodeCount: 1
maxNodeCount: 12
management:
autoUpgrade: true
autoRepair: true
config:
machineType: n1-standard-1
diskSizeGb: 15
imageType: cos
diskType: pd-ssd
oauthScopes: ###Change scope to match needs###
- https://www.googleapis.com/auth/cloud-platform
preemptible: false
Looking for it to create a private cluster with no external IPs.
Did you ever had the chance to go over this documentation?
https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters
https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#public_master
Well, I also found this other Official Google Document that can help you achieve what you want:
https://cloud.google.com/solutions/creating-kubernetes-engine-private-clusters-with-net-proxies
On the "Creating the Docker Image" section there's a Dockerfile example.
Best of Luck!
Is there any way to have IPv6 addresses auto-assigned to EC2 instances within an autoscaling group+launch configuration?
VPC and subnets are all set up for IPv6. Manually created instances are ok.
I can also manually assign them, but I can't seem to find a way to do it in CloudFormation.
The current status is that CloudFormation support for IPv6 is workable. Not fun or complete, but you can build a stack with it - I had to use 2 custom resources:
The first is a generic resource that I use for other things and also reused here, to work around the missing feature to construct a subnet /64 CIDR block from a VPC's /56 auto-provided network
The other I had to add specifically to work around a bug in the EC2 API that CloudFormation uses correctly.
Here is my setup:
1. Add IPv6 CIDR block to your VPC:
VPCipv6:
Type: "AWS::EC2::VPCCidrBlock"
Properties:
VpcId: !Ref VPC
AmazonProvidedIpv6CidrBlock: true
2. Extract the network prefix for creating /64 subnets:
As explained in this answer.
VPCipv6Prefix:
Type: Custom::Variable
Properties:
ServiceToken: !GetAtt [ IdentityFunc, Arn ]
Value: !Select [ 0, !Split [ "00::/", !Select [ 0, !GetAtt VPC.Ipv6CidrBlocks ] ] ]
IdentityFunc is an "identity function" implemented in Lambda for "custom variables", as described in this answer. Unlike this linked answer, I implement the function directly in the same stack so it is easier to maintain. See here for the gist.
3. Add an IPv6 default route to your internet gateway:
RouteInternet6:
Type: "AWS::EC2::Route"
Properties:
RouteTableId: !Ref RouteTableMain
DestinationIpv6CidrBlock: "::/0"
GatewayId: !Ref IGWPublicNet
DependsOn:
- IGWNetAttachment
IGWNetAttachment is a reference to the AWS::EC2::VPCGatewayAttachment defined in the stack. If you don't wait for it, the route may fail to be set properly
4. Add an IPv6 CIDR block to your subnets:
SubnetA:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: !Select [ 0, !GetAZs { Ref: "AWS::Region" } ]
CidrBlock: 172.20.0.0/24
MapPublicIpOnLaunch: true
# The following does not work if MapPublicIpOnLaunch because of EC2 bug
## AssignIpv6AddressOnCreation: true
Ipv6CidrBlock: !Sub "${VPCipv6Prefix.Value}00::/64"
VpcId:
Ref: VPC
Regarding the AssignIpv6AddressOnCreation being commented out - this is normally what you want to do, but apparently, there's a bug in the EC2 API that prevents this from working - through no fault of CloudFormation. This is documented in this AWS forums thread, as well as the solution which I'll present here next.
5. Fix the AssignIpv6AddressOnCreation problem with another lambda:
This is the lambda setup:
IPv6WorkaroundRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: !Sub "ipv6-fix-logs-${AWS::StackName}"
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: arn:aws:logs:*:*:*
- PolicyName: !Sub "ipv6-fix-modify-${AWS::StackName}"
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- ec2:ModifySubnetAttribute
Resource: "*"
IPv6WorkaroundLambda:
Type: AWS::Lambda::Function
Properties:
Handler: "index.lambda_handler"
Code: #import cfnresponse below required to send respose back to CFN
ZipFile:
Fn::Sub: |
import cfnresponse
import boto3
def lambda_handler(event, context):
if event['RequestType'] is 'Delete':
cfnresponse.send(event, context, cfnresponse.SUCCESS)
return
responseValue = event['ResourceProperties']['SubnetId']
ec2 = boto3.client('ec2', region_name='${AWS::Region}')
ec2.modify_subnet_attribute(AssignIpv6AddressOnCreation={
'Value': True
},
SubnetId=responseValue)
responseData = {}
responseData['SubnetId'] = responseValue
cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, "CustomResourcePhysicalID")
Runtime: python2.7
Role: !GetAtt IPv6WorkaroundRole.Arn
Timeout: 30
And this is how you use it:
IPv6WorkaroundSubnetA:
Type: Custom::SubnetModify
Properties:
ServiceToken: !GetAtt IPv6WorkaroundLambda.Arn
SubnetId: !Ref SubnetA
This call races with the autoscaling group to complete the setup, but it is very unlikely to lose - I ran this a few dozen times and it never had a problem to set the field correctly before the first instance boots.
I had a very similar issue and had a chat with AWS Support concerning this. The current state is that IPv6 support in CloudFormation is very limited.
We ended up creating Custom Resources for lots of IPv6-specific things. We have a Custom Resource that:
Enables IPv6-allocation on a subnet
Creates an Egress-Only Internet Gateway
Adds a route to the Egress-Only Internet Gateway (the built-in Route resource says it "fails to stabilize" when pointing to an EIGW)
The Custom Resources are just Lambda functions that do the "raw" API call, and a IAM Role that grants the Lambda enough permissions to do that API call.