I'm trying to add an event stream for DynamoDB in YAML, DynamoDB table is a variable that changes based on
the environment it gets deployed to, I'm using Fn::GetAtt to determine TABLE_NAME but it's giving me an exception.
Is the syntax right?
Is there a way to configure the stream in YAML other than Fn::GetAtt ?
Exception:
missed comma between flow collection entries in "erverless.yml" at line 60,
column28:Fn::GetAtt: [ ${self:custom.TABLE_NAME ...
Code:
custom:
TABLE_NAME : mix of account + env + name
events:
- stream:
type: dynamodb
arn:
Fn::GetAtt: [ ${self:custom.TABLE_NAME}, StreamArn ]
Inside YAML flow collections, strings with characters like {, }, [ etc. have to be quoted, because { starts a flow mapping and [ starts a flow sequence.
There is no space required for that.
So this is how it should look like:
arn:
Fn::GetAtt: [ '${self:custom.TABLE_NAME}', StreamArn ]
For information on pretty much all types of quoting in YAML you can have a look at https://www.yaml.info/learn/quote.html (disclaimer: I'm the author).
Related
Sometimes, when processing log events in vector, a log source might have hyphens as key names - for example json structured logs. Assuming it is from a third party and changing them there is not an option, how can we handle these keys?
A sample log message (contrived for demonstration) is:
{
"labels":{"no_hypens":"normal field","this-has-hypens":"this is a test"},
"message":"a message",
"timestamp":"2022-11-01T12:03:00.941866394Z"
}
Note the field labels.this-has-hyphens there.
I managed to put together a test case, and find out the syntax, both in providing test data like this and in extracting the data in VRL:
The test case
---
tests:
- name: hypens
inputs:
- insert_at: hypens
type: log
log_fields:
labels."this-has-hypens": "this is a test"
labels.no_hypens: "normal field"
outputs:
- extract_from: hypens
conditions:
- type: vrl
source: |
assert_eq!("normal field", .no_hypens)
assert_eq!("this is a test", .output_without_hypens)
This will insert and check for two fields from the input data. Note the hyphenated key segment needs to be quoted.
Next the VRL in the transform:
---
transforms:
hypens:
type: remap
inputs:
- route
source: |
log(., "error")
. = {
"no_hypens": .labels.no_hypens,
"output_without_hypens": .labels."this-has-hypens",
}
The log message is there as while debugging this, I had to figure out that the test hyphenated field didn't even get to the transform until I had quotes around it.
Then the field reference itself needs to have quotes too, after the dot (not square brackets).
This will pass the tests, and output the right data.
I'm using pykwalify to validate a schema.
Given this yaml:
variables:
dev:
options:
key: value
uat:
key: value
key2: value
prd:
key: value
key2: value
Under variables, any map should be allowed.
Under that second level (dev, uat, prd) - any key should be allowed, EXCEPT options. "options" should not be allowed here.
I've tried using a regex, but this is only evaluating the top level, and I'm not quite sure how evaluate the level nested under that "dev, uat, prd" level.
variables:
type: map
matching-rule: all
mapping:
regex;([^,]+):
type: any
regex;(^(?!.*options:).*$):
type: any
Another potential option would be if I have to explicitly list the values that are allowed, that would work too.
I see two issues in your snippet:
The regex to match anything but "options" is wrong.
The schema isn't properly structured for the nested mapping.
The following schema should provide what you need:
variables:
type: map
matching-rule: all
mapping:
regex;([^,]+):
type: map
mapping:
regex;(^(?!options$).*):
type: any
I want to combine 3 functions in CloudFormation YAML but failing to do so.
I've got an exported parameter that I want to access foo-exportedParam
Then I want to import it, but taking into account that the prefix foo is dynamic and comes from a parameter in the template.
So I want something like
Name: Fn::ImportValue Fn::Join ['', [Fn::Ref prefix, "-exportedParam"]]
If I have the param prefix = foo, then this should translate into
Name: !ImportValue foo-exportedParam
Is this possible?
The documentation gave me a clue on the syntax. This works
Name:
Fn::ImportValue:
!Join ['', [!Ref prefix, "-exportedParam"]]
See nested simple sample: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html
I have a CloudFormation dashboard where I link/reference alarms using Alarm Annotations.
The names/ARNs of these alarms differ based on a condition. I want to a include a simple condition using the Fn::If intrinsic condition as below:
I have tried the following:
"properties": {
"annotations": {
"alarms": [
{'Fn::If': ["<condition>", "<value_if_true>", "<value_if_false>"]}
]
}
}
This gives me an error. I want to include a simple if condition.
Is there something that I am missing?
Based on the commends, the issue reported was caused by the use of single quotes in 'Fn::If'.
The solution was to use double quotes, as single quotes are not json complient: "Fn::If".
I have the following cloudformation template. The goal is to only create a parameter group if the user wants to and then to populate the RDS parameters in the parameter group with the contents of the cloudformation template parameters.
Parameters:
UseCustomParameterGroup:
Description: Toggle to 'Yes' to create a new parameter group.
Type: String
AllowedValues: ['Yes', 'No']
Default: 'No'
CustomParameters:
Description: Add custom parameters to your custom parameter group. Creating a custom parameter group without parameters creates a mirror of the default group.
Type: String
Conditions:
UseCustomParameterGroup: !Equals [!Ref 'UseCustomParameterGroup', 'Yes']
Resources:
CustomParameterGroup:
Type: AWS::RDS::DBParameterGroup
Condition: 'UseCustomParameterGroup'
Properties:
Family: "postgres10"
Parameters: !Ref "CustomParameters"
If I call this template from another template as so, it will fail with the error Value of property Parameters must be an object
Parameters:
USECUSTOMPARAMETERGROUP: 'Yes'
CUSTOMPARAMETERS: '{
"shared_preload_libraries": "pg_stat_statements",
"pg_stat_statements.track": "all"
}'
Resources:
Postgres:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: https://..../rds-postgres-instance.yaml
TimeoutInMinutes: '60'
Parameters:
UseCustomParameterGroup: !Ref USECUSTOMPARAMETERGROUP
CustomParameters: !Ref CUSTOMPARAMETERS
The documentation for AWS::RDS::DBParameterGroup states the following for the Parameters parameter:
Type: A JSON object consisting of string key-value pairs, as shown in the following example
"Parameters" : { "Key1" : "Value1", "Key2" : "Value2", "Key3" : "Value3" }
I think this may be out of date for the YAML version of Cloudformation, but there is no documentation on how to pass this value multiple parameters.
I want the user to be able to define as few or many RDS parameters as they please without having to account for any of the thousands of possible parameters available to RDS.
Cloudformation doesn't allow you to convert string parameters to JSON objects (or YAML for that matter), your parameters are meant to be used as values for your definition keys.
Some other frameworks like serverless overcome this limitation by using a different template language that generates a Cloudformation compatible artifact after some processing, if this feature is critical to your process I advise you to migrate to one of those frameworks.