I have some yaml like this
SECRETS: |
- TEST_SECRET_A
- TEST_SECRET_B
- dev:
- TEST_SECRET_B:
env: OVERRIDE_TEST_SECRET_B
- TEST_SECRET_C:
file: /etc/secrets/secret_c
- TEST_SECRET_D:
env: OVERRIDE_TEST_SECRET_D
I need to pull TEST_SECRET_[A|B|D] into an "all" map of environment variables and the two under dev into a "dev" map.
The first two elements are the same type as the other TEST_ elements the'll get an 'env' element in the code once I import them.
Related
It is possible to access all vars from a azure DevOps variable group as environment vars in a bash step?
At best I want a way to access all group vars as environment var inside of a python script.
So something like the yaml below I have in mind, but open for other ideas. Right know I add every var from my group as an env param, but so I have always to update the azure DevOps yaml if I want to add a new var in the group. This just don't feel right....
pool:
vmImage: ubuntu-latest
variables:
- group: myGroup
stages:
- stage: test
jobs:
- job: test
displayName: Test
steps:
- bash: |
echo $path.myGroup
group:
- myGroup
This is not possible. However you can try to use azure CLI to fetch both names and values for non secret variables inside the group. It doesn't returna value for you for secret variables, however for them you still have to map them explicit.
And if you do this:
pool:
vmImage: ubuntu-latest
stages:
- stage: test
jobs:
- job: test
displayName: Test
variables:
- group: myGroup
steps:
- bash: |
echo 'You have all non secret variables ampped to env variables`
echo `You can use Azure Cli to get all names of the mapped variables from variables group`
You will get all non-secret variables mapped.
You can also use
variables:
- group: myGroup
on the top level achieving the same, but then you will get it mapped accross all jobs.
I am trying to represent the following in the helmfile.yaml but I am getting an error. Can anyone help me to set it up?
values.yaml
extraVolumes:
- name: google-cloud-key
secret:
secretName: gcloud-auth
I tried the following in helmfile.yaml
repositories:
- name: loki
url: https://grafana.github.io/loki/charts
releases:
- name: loki
namespace: monitoring
chart: loki/loki
set:
- name: extraVolumes.name
value: google-cloud-key
- name: extraVolumes.secret.secretName
value: gcloud-auth
The error I am getting is
coalesce.go:160: warning: skipped value for extraVolumes: Not a table.
I also tried with the following in helmfile.yaml
- name: extraVolumes.name[]
value: google-cloud-key
This gave me the following error
Error: failed parsing --set data: key map "extraVolumes" has no value
Any idea?
Helmfile has two ways to provide values to the charts it installs. You're using set:, which mimics the finicky helm install --set option. However, Helmfile also supports values:, which generally maps to helm install -f. Helmfile values: supports two extensions: if a filename in the list ends in *.gotmpl then the values file itself is processed as a template file before being given to Helm; or you can put inline YAML-syntax values directly in helmfile.yaml.
This last option is probably easiest. Instead of using set:, use values:, and drop that block of YAML directly into helmfile.yaml.
releases:
- name: loki
namespace: monitoring
chart: loki/loki
values: # not `set:`
- extraVolumes: # inline YAML content as a single list item
- name: google-cloud-key
secret:
secretName: gcloud-auth
values: is set to a list of either filenames or inline mappings. If you're not deeply familiar with YAML syntax, this means you need to put a - list-item indicator before the inline YAML block. If you already have a list of values: files you can add this additional item into the list wherever appropriate.
Error parsing config file: yaml: line 22: did not find expected key
Cannot find a job named build to run in the jobs: section of your configuration file.
I got those errors, but I'm really new to yaml so I can't really find reaons why It's not working. any ideas? Some says It might have extra spaces or something, but I can't really find it.
yaml file
defaults: &defaults:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package.json" }}
- v1-dependencies-
- run: npm install
- save_cache:
paths:
- node_modules
key: v1-dependencies-{{ checksum "package.json" }}
version: 2
jobs:
build:
docker:
- image: circleci/node:10.3.0
working_directory: ~/repo
steps:
<<: *defaults // << here
- run: npm run test
- run: npm run build
deploy:
docker:
- image: circleci/node:10.3.0
working_directory: ~/repo
steps:
<<: *defaults
- run:
name: Deploy app scripts to AWS S3
command: npm run update-app
workflows:
version: 2
build-deploy:
jobs:
- build
- deploy:
requires:
- build
filters:
branches:
only: master
What you are trying to do is trying to merge two sequences. ie all elements of default are merged into steps. Which is not supported in YAML spec. Only you can merge maps and nested sequences.
This is invalid:
steps:
<<: *defaults
- run:
as <<: is for merging map elements, not sequences
If you do this:
step_values: &step_values
- run ...
steps:
- *defaults
- *step_values
You will end up with nested sequences, which is not what you intend.
Its not possible for now. Unfortunately, the only solution is to repeat the whole list. Many users are requesting the same feature.
it looks like your YAML is not written properly. You can always check the structure validation of YAML from an open-source website such as http://www.yamllint.com/.
On checking the yaml file, on line 22 you are doing wrong. As explained by Srikanth, that you are trying to do is merging two sequences. i.e. all elements of default are merged into steps. Which is not supported in YAML at the moment.
Only you can merge maps and nested sequences
If you do this:
step_values: &step_values
- run ...
-----------------------------------------------
steps:
- *defaults
- *step_values
You will end up with nested sequences, which is not what you intend.
In some ansible roles (e.g. roles/my-role/) I've got quite some big default variables files (defaults/main.yml). I'd like to split the main.yml into several smaller files. Is it possible to do that?
I've tried creating the files defaults/1.yml and defaults/2.yml, but they aren't loaded by ansible.
The feature I'm describing below has been available since Ansible 2.6, but got a bugfix in v2.6.2 and another (minor) one in v2.7.
To see a solution for older versions, see Paul's answer.
defaults/main/
Instead of creating defaults/main.yml, create a directory — defaults/main/ — and place all YAML files in there.
defaults/main.yml → defaults/main/*.yml
Ansible will load any *.yml file inside that directory, so you can name your files like roles/my-role/defaults/main/{1,2}.yml.
Note, the old file — defaults/main.yml — must not exist. See this Github comment.
vars/main/
By the way, the above solution also works for vars/:
vars/main.yml → vars/main/*.yml
further details
The feature has been introduced in v2.6 — git commit, Pull Request, main Github issue.
There have been two bugfixes:
v2.7 fix: git commit, Pull Request — backported to v2.6.2: commit, Pull Request
v2.7 fix: git commit, Pull Request, bug discussion
If you aren't using 2.6 (which you probably should, but I understand that isn't always an option), then you might find include_vars useful.
- name: Include vars of stuff.yaml into the 'stuff' variable (2.2).
include_vars:
file: stuff.yaml
name: stuff
- name: Conditionally decide to load in variables into 'plans' when x is 0, otherwise do not. (2.2)
include_vars:
file: contingency_plan.yaml
name: plans
when: x == 0
- name: Load a variable file based on the OS type, or a default if not found. Using free-form to specify the file.
include_vars: "{{ item }}"
with_first_found:
- "{{ ansible_distribution }}.yaml"
- "{{ ansible_os_family }}.yaml"
- default.yaml
- name: Bare include (free-form)
include_vars: myvars.yaml
- name: Include all .json and .jsn files in vars/all and all nested directories (2.3)
include_vars:
dir: vars/all
extensions:
- json
- jsn
- name: Include all default extension files in vars/all and all nested directories and save the output in test. (2.2)
include_vars:
dir: vars/all
name: test
- name: Include default extension files in vars/services (2.2)
include_vars:
dir: vars/services
depth: 1
- name: Include only files matching bastion.yaml (2.2)
include_vars:
dir: vars
files_matching: bastion.yaml
Note that this is a task directive, though. It isn't as neat as just being able to include it into the defaults file itself.
We have a gitlab-ci yaml file with duplicate parts.
test:client:
before_script:
- node -v
- yarn install
cache:
untracked: true
key: client
paths:
- node_modules/
script:
- npm test
build:client:
before_script:
- node -v
- yarn install
cache:
untracked: true
key: client
paths:
- node_modules/
policy: pull
script:
- npm build
I would like to know, with the merge syntax, if I can extract the common part to reuse it efficiently in the context of these two parts.
.node_install_common: &node_install_common
before_script:
- node -v
- yarn install
cache:
untracked: true
key: client
paths:
- node_modules/
But the real question is: at which indent level do I have to merge the block to ensure policy: pull is applied to the cache section. I tried to so that:
test:client:
<<: *node_install_common
script:
- npm test
test:build:
<<: *node_install_common
policy: pull
script:
- npm build
But I get an invalid yaml error. How to indent to get the correct merge behavior?
Note that merge keys are not part of the YAML specification and therefore are not guaranteed to work. They are also specified for the obsolete YAML 1.1 version and have not been updated for the current YAML 1.2 version. We intend to explicitly remove merge keys in upcoming YAML 1.3 (and possibly provide a better alternative).
That being said: There is no merge syntax. the merge key << must be placed like a normal key in a mapping. This means that the key must have the same indentation as other keys. So this would be valid:
test:client:
<<: *node_install_common
script:
- npm test
While this is not:
test:build:
<<: *node_install_common
policy: pull
script:
- npm build
Note that compared to your code, I added : to the test:client and test:build lines.
Now merge is specified to place all key-value pairs of the referenced mapping into the current mapping if they do not already exist in it. This means that you can not, as you want to, replace values deeper in the subtree – merge does not support partial replacement of subtrees. However, you can use merge multiple times:
.node_install_common: &node_install_common
before_script:
- node -v
- yarn install
cache: &cache_common
untracked: true
key: client
paths:
- node_modules/
test:client:
<<: *node_install_common
script:
- npm test
test:build:
<<: *node_install_common
cache: # define an own cache mapping instead of letting merge place
# its version here (which could not be modified)
<<: *cache_common # load the common cache content
policy: pull # ... and place your additional key-value pair
script:
- npm build