I'm trying to use gomplate as a generator of configuration. The problem I'm facing now is having multiple mutations and environments where the application needs to be configured in a different way. I'd like to achieve some user-friendly and readable way with the least possible repetitions in the template and source data.
The motivation behind this is to have generated source data app_config which can be used in a following gomplate as following:
feature_a={{ index (datasource "app_config").features.feature_a .Env.APP_MUTATION .Env.ENV_NAME | required }}
feature_b={{ index (datasource "app_config").features.feature_b .Env.APP_MUTATION .Env.ENV_NAME | required }}
Basically I'd like to have this source data
features:
feature_a:
~: true
feature_b:
mut_a:
~: false
dev: true
test: true
mut_b:
~: true
converted into this result (used as app_config gomplate datasource)
features:
feature_a:
mut_a:
dev: true
test: true
load: true
staging: true
prod: true
mut_b:
dev: true
test: true
load: true
staging: true
prod: true
feature_b:
mut_a:
dev: true
test: true
load: false
staging: false
prod: false
mut_b:
dev: true
test: true
load: true
staging: true
prod: true
given that datasource platform is defined as
mutations:
- mut_a
- mut_b
environments:
- dev
- test
- load
- staging
- prod
I chose to use the ~ to state that every environment or mutation that is not defined will get the value behind ~.
This should work under assumption that the lowest level is environment and the level before the lowest is mutation. Unless environments are not defined, in that case mutation level is lowest and applies for all mutations and environments. However I know this brings extra complexity, so I'm wiling to use simplified variant where mutations are always defined:
features:
feature_a:
mut_a: true
mut_b: true
feature_b:
mut_a:
~: false
dev: true
test: true
mut_b:
~: true
However, since I'm fairly new to gomplate, I'm not sure whether it is the right tool for the job.
I welcome every feedback.
After further investigation I decided that this issue will be better solved with separate tool.
Related
I need rundeck to parse an option based on the value selected in another option. I have an option ${option.env} and other options like ${option.id_dev}, ${option.id_qa}
I want to achieve something like below for extra-vars, so that "env" option value determines which id(dev or qa) to read.
ansible-playbook /build.yml -e id=${option.id_${option.env.value}}
Is this possible or Could I pass extra-vars like a conditional case based on env value ?. I'm using rundeck 3.0.X
Update :
To give clear info, If I select 'dev' for the option 'env', I need to use its value like ${option.id_${option.env.value}} , so it translates to ${option.id_dev} to get other option in the command line
You can use cascade remote options in a tricky way. The explanation is at the end of this answer.
I made a little example to see how to achieve this:
The branches.json file (referenced on the job options as "branches"):
[
{"name":"branch1", "value":"branch1.json"},
{"name":"branch2", "value":"branch2.json"},
{"name":"branch3", "value":"branch3.json"}
]
The branch1.json is the first tentative value of the branches option:
[
{"name":"v1", "value":"1"},
{"name":"v2", "value":"2"},
{"name":"v3", "value":"3"}
]
The branch2.json is the second tentative value of the branches option:
[
{"name":"v4", "value":"4"},
{"name":"v5", "value":"5"},
{"name":"v6", "value":"6"}
]
The branch3.json is the third tentative value of the branches option:
[
{"name":"v7", "value":"7"},
{"name":"v8", "value":"8"},
{"name":"v9", "value":"9"}
]
Full job definition to test:
- defaultTab: summary
description: ''
executionEnabled: true
id: ed0d84fe-135b-41ee-95b6-6daeaa94894b
loglevel: INFO
name: CascadeTEST
nodeFilterEditable: false
options:
- enforced: true
name: branches
valuesUrl: file:/Users/myuser/branches.json
- enforced: true
name: level2
valuesUrl: file:/Users/myuser/${option.branches.value}
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- fileExtension: .sh
interpreterArgsQuoted: false
script: |+
#!/bin/sh
# getting the options
first=#option.branches#
second=#option.level2#
# this just an example
echo "this an example: ${first%%.*}.$second"
scriptInterpreter: /bin/bash
keepgoing: false
strategy: node-first
uuid: ed0d84fe-135b-41ee-95b6-6daeaa94894b
Explanation
As you see, the value of the first option is always a file name, if you take the value directly you always obtain an "option1.json" like string, so, the trick here is to cut the file extension and take only the name as the value, for that, I used this approach.
So, with the first selection value and the second one, you can do anything later in a script, for example, launch the ansible-playbook command in the inline script.
Check the example result.
UPDATED ANSWER:
The closest way is just using two options and concatenating them in the command step but isn't possible to get the option value inside the other one as you say.
Just like this:
- defaultTab: nodes
description: ''
executionEnabled: true
id: 4e8df698-c7ca-4a10-9f70-bc68c1007a10
loglevel: INFO
name: NewJob
nodeFilterEditable: false
options:
- enforced: true
name: env
value: qa
values:
- qa
- prod
- stage
valuesListDelimiter: ','
- enforced: true
name: id_dev
value: '1'
values:
- '1'
- '2'
- '3'
valuesListDelimiter: ','
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- exec: echo ${option.env}_${option.id_dev}
keepgoing: false
strategy: node-first
uuid: 4e8df698-c7ca-4a10-9f70-bc68c1007a10
Result.
(the cascade option is another approach)
In my yaml spring-boot application config I have
additional-properties[auto.register.schemas]: false
additional-properties[use.latest.version]: true
and it works! I haven't found this syntax in the YAML specification. What does it mean? How can it be re-written using standard YAML? Is this the same as
additional-properties:
- auto.register.schemas: false
- use.latest.version: true
?
AFAIK:
Every element (separated by a dot) has to be on its own line and tabed accordingly.
foo:
bar:
name: value
name2: value2
fez: value
So your example would be:
additional-properties:
auto:
register:
schemas: false
After experimenting and after finding this answer, I conclude that (at least in Spring application.yaml):
camel.component.kafka:
additional-properties[auto.register.schemas]: false
additional-properties[use.latest.version]: true
is equivalent to
camel.component.kafka.additional-properties:
"[auto.register.schemas]": false
"[use.latest.version]": true
and this is equivalent to
camel:
component:
kafka:
additional-properties:
"[auto.register.schemas]": false
"[use.latest.version]": true
I got the serverless-plugin-warmup 4.2.0-rc.1 working fine with serverless version 1.36.2
But it only executes with one single warmup call instead of the configured five.
Is there any problem in my serverless.yml config?
It is also strange that I have to add 'warmup: true' to the function section to get the function warmed up. According to the docs on https://github.com/FidelLimited/serverless-plugin-warmup the config at custom section should be enough.
plugins:
- serverless-prune-plugin
- serverless-plugin-warmup
custom:
warmup:
enabled: true
concurrency: 5
prewarm: true
schedule: rate(2 minutes)
source: { "type": "keepLambdaWarm" }
timeout: 60
functions:
myFunction:
name: ${self:service}-${opt:stage}-${opt:version}
handler: myHandler
environment:
FUNCTION_NAME: myFunction
warmup: true
in AWS Cloud Watch I only see one execution every 2 minutes. I would expect to see 5 executions every 2 minutes, or do I misunderstand something here?
EDIT:
Now using the master branch concurrency works but now the context that is deliverd to the function which should be warmed is broken: Using Spring Cloud Functions => "Error parsing Client Context as JSON"
Looking at the JS of the generated warmup function the delivered source looks not ok =>
const functions = [{"name":"myFunction","config":{"enabled":true,"source":"\"\\\"{\\\\\\\"source\\\\\\\":\\\\\\\"serverless-plugin-warmup\\\\\\\"}\\\"\"","concurrency":3}}];
Config is:
custom:
warmup:
enabled: true
concurrency: 3
prewarm: true
schedule: rate(5 minutes)
timeout: 60
Added Property sourceRaw: true to warmup config which generates a clean source in the Function JS.
const functions = [{"name":"myFunctionName","config":{"enabled":true,"source":"{\"type\":\"keepLambdaWarm\"}","concurrency":3}}];
Config:
custom:
warmup:
enabled: true
concurrency: 3
prewarm: true
schedule: rate(5 minutes)
source: { "type": "keepLambdaWarm" }
sourceRaw: true
timeout: 60
boot config *.yml file.
server.port: 2222
spring:
application:
name: x-service
data:
mongodb:
host: db.x
database: x
# userName: ${db.userName}
# password: ${db.password}
rabbitmq:
# port: ${queue.port}
host: queue.x
username: ${queue.userName}
password: ${queue.password}
listener:
max-concurrency: 1
prefetch: 1
acknowledge-mode: auto
auto-startup: true
dynamic: true
###########DEV##############
spring.profiles: dev
#queue.virtual.host: xuser
queue.userName: guest
queue.password: guest
queue.port: 5672
#db.userName:
#db.password:
falconUrl: http://x.y.com
##########DEFAULT###########
spring.profiles: qa
queue.virtual.host: xuser
queue.userName: xuser
queue.password: xpassword
queue.port: 3456
db.userName: xuser
db.password: xpassword
falconUrl: http://x.z.com
It gives me org.yaml.snakeyaml.parser.ParserException: while parsing MappingNode
in 'reader', line 1, column 1:
server.port: 2222
^
Duplicate key: spring.profiles
in 'reader', line 47, column 1:
error. If I comment properties of one of the profile.It works fine.
Can anyone please suggest what is wrong here?
The error message is actually quite specific and accurate: in the top-level mapping of your YAML file (the one starting with the key-value pair server.port and 2222 you have two identical keys (the scalar spring.profiles). And duplicate keys are not allowed in YAML, as the are required to be unique according to the specification.
The underlying problem is that if you want to change the configuration depending on the environment, you'll have to follow the documented specification, which states that:
A YAML file is actually a sequence of documents separated by --- lines, and each document is parsed separately to a flattened map.
If a YAML document contains a spring.profiles key, then the profiles value (comma-separated list of profiles) is fed into the Spring Environment.acceptsProfiles() and if any of those profiles is active that document is included in the final merge (otherwise not)
Your YAML file is a single implicit YAML document because it lacks the directive indicator --- that occurs at the beginning of an explicit YAML document. (the YAML directive ... that indicates end-of-document might not be supported properly supported by snake-yaml, at least it is not mentioned in the examples).
Your code should look like:
server.port: 2222
spring:
application:
name: x-service
data:
mongodb:
host: db.x
database: x
# userName: ${db.userName}
# password: ${db.password}
rabbitmq:
# port: ${queue.port}
host: queue.x
username: ${queue.userName}
password: ${queue.password}
listener:
max-concurrency: 1
prefetch: 1
acknowledge-mode: auto
auto-startup: true
dynamic: true
###########DEV##############
---
spring.profiles: dev
#queue.virtual.host: xuser
queue.userName: guest
queue.password: guest
queue.port: 5672
#db.userName:
#db.password:
falconUrl: http://x.y.com
##########DEFAULT###########
---
spring.profiles: qa
queue.virtual.host: xuser
queue.userName: xuser
queue.password: xpassword
queue.port: 3456
db.userName: xuser
db.password: xpassword
falconUrl: http://x.z.com
The statement in the documentation that "each document is parsed separately to a flattened map" is of course only true, if each of the documents has a mapping at the top level. That is what spring-boot expects, but you can as easily have a scalar or sequence at the top level of a document, and such documents are certainly not parsed by snake-yaml to a flattened map.
I managed to interactively set the debugging mode of my symfony application On and Off for a user session with something like this:
$configuration = ProjectConfiguration::getApplicationConfiguration($app,
$env, $debugging);
I know that the Web Debug Toolbar showing up or not does not depend on the value of $debugging, but the configuration of the current environment.
To this moment the only way that the toolbar appears is when $env = 'dev'.
I'd like to activate it when accessing the "prod" environment also.
I have this app setting:
prod:
.settings:
no_script_name: true
logging_enabled: false
web_debug: true
error_reporting: <?php echo (E_ALL | E_STRICT)."\n" ?>
dev:
.settings:
error_reporting: <?php echo (E_ALL | E_STRICT)."\n" ?>
web_debug: true
cache: false
no_script_name: false
etag: false
The toolbar is not being shown, apparently ignoring the "web_debug" setting.
If I echo(sfConfig::get('sf_web_debug')) I get "true".
¿How could I get the toolbar working?
you also need to change the factory.yml . By default, there's no Logger in the 'prod'-environment.
Just comment out like this:
prod:
# logger:
# class: sfNoLogger
# param:
# level: err
# loggers: ~
From memory you have to change a value in your frontend php file. Compare the frontend.php and frontend_dev.php files in your web directory. Look for a difference where one is true and the other is false (I think it's the last parameter).
The lines are:
require_once(dirname(__FILE__).'/../config/ProjectConfiguration.class.php');
$configuration = ProjectConfiguration::getApplicationConfiguration('frontend', 'prod', false));
sfContext::createInstance($configuration)->dispatch();
change to:
require_once(dirname(__FILE__).'/../config/ProjectConfiguration.class.php');
$configuration = ProjectConfiguration::getApplicationConfiguration('frontend', 'prod', true));
sfContext::createInstance($configuration)->dispatch();