I want to optionally apply a VPC configuration based on whether an environment variable is set.
Something like this:
custom:
vpc:
securityGroupIds:
- ...
subnetIds:
- ...
functions:
main:
...
vpc: !If
- ${env:USE_VPC}
- ${self:custom.vpc}
- ~
I'd also like to do similar for alerts (optionally add emails to receive alerts) and other fields too.
How can this be done?
I've tried the above configuration and a variety of others but just receive various different errors
For example:
Configuration error:
at 'functions.main.vpc': must have required property 'securityGroupIds'
at 'functions.main.vpc': must have required property 'subnetIds'
at 'functions.main.vpc': unrecognized property 'Fn::If'
Currently, the best way to achieve such behavior is to use JS/TS-based configuration instead of YAML. With TS/JS, you get full power of a programming language to shape your configuration however you want, including use of such conditional checks to exclude certain parts of the configuration. It's not documented too well, but you can use this as a starting point: https://github.com/serverless/examples/tree/v3/legacy/aws-nodejs-typescript
In general, you can do whatever you want, as long as you export a valid object (or a promise that resolves to a valid object) with serverless configuration.
Related
I am experiencing weird behavior with YAML variables, parameters, and Azure pipeline resource references. The following shows the original implementation that works compared to my new implementation with a single line change that fails.
Working Implementation
Template A (makes a call to template B):
- template: Templates\TemplateB.yml
serviceBuildResourceName: resourceName
Template B (uses serviceBuildResourceName param to get pipeline run information):
$projectId = '$(resources.pipeline.${{ parameters.serviceBuildResourceName }}.projectID)'
$pipelineId ='$(resources.pipeline.${{ parameters.serviceBuildResourceName }}.PipelineID)'
Template B goes on to use the values in $projectId and $pipelineId (along with other values not listed here since it is irrelevant) to successfully retrieve information about the a pipeline run from the specific pipeline resource, serviceBuildResourceName. Note that all pipeline resources are correctly defined at the beginning yaml file for the pipeline. In this implementation above, everything works perfectly.
Failing Implementation
Template A (makes a call to template B):
- template: Templates\TemplateB.yml
serviceBuildResourceName: $(ServiceBuildResourceName)
Template B (uses serviceBuildResourceName param to get pipeline run information):
$projectId = '$(resources.pipeline.${{ parameters.serviceBuildResourceName }}.projectID)'
$pipelineId ='$(resources.pipeline.${{ parameters.serviceBuildResourceName }}.PipelineID)'
Note that the only difference is the following: instead of passing the hard-coded string into the serviceBuildResourceName parameter, I pass in a variable, which has the same value as before, resourceName. The variable is defined in an earlier template as such:
- name: ServiceBuildResourceName
value: resourceName
I feel it should still work the same, but I know get the following error in my pipeline run:
WARNING: 2023-02-12 15:52:29.5071 Response body: {"$id":"1","innerException":null,"message":"The value is not an integer.
$(resources.pipeline.resourceName.PipelineID)
I know that the variable is being correctly populated since the error message above contains "resourceName" in resources.pipeline.resourceName.PipelineID, as it should.
However, for reasons unknown to me, it now throughs an error. It seems like it doesn't recognize the pipeline resource, and instead recognizes it as a string.
Any help or insight here would be greatly appreciated, thanks!
As far as I can tell, this is because of how predefined variables work in YAML. Since resources.pipeline... is a predefined variable, it gets resolved at compile time. Thus, you can't use run-time defined variables like I am doing. Instead of resolving it as a predefined variable, it will get resolved to be a string at runtime.
Given I have the following defined in a YML file for a resource:
Type: AWS::Cognito::UserPoolClient
CallbackURLs:
- "my-first-url"
How can I add a second item when only certain conditions are met? Unfortuantely it is not possible to simply set it to an empty string or null as it fails deployment validation. E.G something like:
Type: AWS::Cognito::UserPoolClient
CallbackURLs:
- "my-first-url"
- myCondition ? "" : undefined // Omit item
Is this possible in any way at all? Happy to use Plugin solutions etc.
You can use a CloudFormation condition like Fn::If to conditionally create stack resources. The CloudFormation documentation about conditions has all the details but somehting like this should get you started:
resources:
Conditions:
my_condition: !Equals [value_1, value_2]
Resources:
MyUserPool:
Type: AWS::Cognito::UserPoolClient
CallbackURLs:
- "my-first-url"
- !If [my_condition, "...", !Ref "AWS::NoValue"]
Replace the content of my_condition with your condition. It is referenced later in the Fn::If (the example uses the shorthand for Fn::If).
The AWS::NoValue is a pseudo parameter which can be used as a return value to remove the corresponding property. It should work here to remove the list item but I'm not sure about it, you'll need to test.
I have my property file like this:
integrations:
- operationCode: CD001
connectionFactoryName: cf1
senderName: sn1
host: 192.168.1.1
port: 1416
queueManager: QM_TSTIN
channel: JAVA.CHANNEL
username: user
password: pass
receiveTimeout: 10000
sendQueue: SEND
receiveQueue: RECEIVE
How can i get my integration entity by operationCode? This is how i try to do this. But it does not work.
#Value("\${integrations.?[operationCode == 'CD001'].receiveQueue}")
var receiveQueue: String? = null
This is not possible using the property placeholder syntax ${} and also not possible by the SPEL syntax #{} as far as I know.
The property placeholder is a very simple solution it provides direct access ${a.b.c} with the option of default value ${a.b.c:default_value}. In terms of syntax that's it (javadoc).
The Spring Expression Language is very powerful comparing to the property placeholder but can not access properties directly. You can access a property by #{'${a.b.c}'} but the a.b.c must be a leaf in the yaml it can not access internal nodes. So you can not use the #{'${a.b}'.c} as the a.b is an internal node. So even you could use the #{bean.collection.?[attribute1 > 5].attribute2} in SPEL you can not use that for properties.
I suggest the standard yml attributes instead of the collection:
integrations:
CD001:
operationCode: CD001
receiveQueue: RECEIVER_2
CD002:
operationCode: CD002
receiveQueue: RECEIVER_2
and then use the #Value("${integrations.CD002.receiveQueue}") to get the value.
Unfortunately as far as I know there is no expression to solve your problem if you can not change the yaml's format.
I am trying to rebuild my ci-cd within the new github actions yaml format (new), the issue is that I can't seem to use computed values as arguments in a step.
I have tried the following
- name: Download Cache
uses: ./.github/actions/cache
with:
entrypoint: restore_cache
args: --bucket=gs://[bucket secret] --key=node-modules-cache-$(checksum package.json)-node-12.7.0
However "$(checksum package.json)" is not valid as part of an argument.
Please not this has nothing to do with if the command checksum exists, it does exist within the container.
I'm trying to copy this kind of setup in google cloud build
- name: gcr.io/$PROJECT_ID/restore_cache
id: restore_cache_node
args:
- '--bucket=gs://${_CACHE_BUCKET}'
- '--key=node-modules-cache-$(checksum package.json)-node-${_NODE_VERSION}'
I expected to be able to use compute arguments in a similar way to other ci-cd solutions.
Is there a way to do this that I am missing? Maybe being able to use 'run:' within a docker container to run some commands.
The only solution I'm aware of at the moment is to compute the value in a previous step so that you can use it in later steps.
See this answer for a method using set-output. This is the method I would recommend for passing computed values between workflow steps.
Github Actions, how to share a calculated value between job steps?
Alternatively, you can create environment variables. Computed environment variables can also be used in later steps.
How do I set an env var with a bash expression in GitHub Actions?
I want to set the value of 'teamcity.build.branch' dynamically according to the result of another TC build configuration part of the build pipeline.
Is that even possible? It looks like the value is evaluated and used at the start of the build pipeline.
UseCase:
I am executing a TC build configuration that will generate a unique number
in the connected TC build configuration part of the same pipeline I want the number to be used in the 'teamcity.build.branch' - just for visualization purposes
I am already using message service to overwrite the parameter, but the change is not taken into account. It looks like the value is read in the very early stage of the build process.
Check below reference containing build number and git branch name
https://octopus.com/blog/teamcity-version-numbers-based-on-branches
You could overwrite the value of the parameter by using a simple script that emits a "set parameter" service message.
By using a dedicated service message in your build script, you can dynamically update build parameters of the build right from a build step (...)
With that approach, here are the steps that you need to perform:
In the first build config, define a custom build parameter and set its value to the unique number you're generating. Do this directly from the script that generates the unique number by writing something like this to STDOUT:
##teamcity[setParameter name='magicNumber' value='1234']
In the dependent build config, you now have access to that parameter. Using a second build script, you can overwrite the teamcity.build.branch with the same mechanism:
##teamcity[setParameter name='teamcity.build.branch' value='the new value']
Note 1: I recommend against overwriting the built-in parameters, because this might have strange side-effects. Rather, define a custom parameter in the second build config and use that for your visualization purposes.
Note 2: In case you decide to ignore Note 1, it may be necessary to overwrite the build parameters by setting the dependency property as outlined in the docs in section "Overriding Dependencies Properties":
##teamcity[setParameter name='reverse.dep.*.teamcity.build.branch' value='the new value']