I'm attempting to use Ansible to better manage my Kubernetes configmaps in a multienvironment project (dev, stage, and prod). I've generalized each of the config maps as j2 templates, and I'll override the variables depending on how they might change in different environments (so that they aren't duplicated three times for basically the same file).
My playbook currently looks something like this:
---
- hosts: localhost
vars_files:
- "vars/{{ env }}.yml"
tasks:
- name: Generate YAML from j2 template
template:
src: templates/foo.j2
dest: output/foo.yml
And this has been working great for testing so far. However, I'm at the point where I want to incorporate this into my already existing Jenkins CI/CD, but I'm having trouble understanding how it might work with what I am doing currently.
After generating what is basically a Kuberenets ConfigMap from the j2, I'll somehow do this within Jenkins:
kubectl apply -f <yaml>
However, the playbook is creating a YAML file every time I run it, and I am wondering if there is an alternative that would allow me to pipe the contents of the YAML file or somehow retrieve it from stdout.
Basically, I want to evaluate the template and retrieve it without necessarily creating a file.
If I do this, I could do something like the following:
echo result | kubectl apply -f -
where result of course is the contents of the YAML file that results after the templating, and the short dash after the f flag specifies Kubernetes to use the process' stdout.
Sorry for so much explaining, I can clarify anything if needed.
I would like to retrieve the result of the template, and pipe it into that command, such as "echo result | kubectl apply -f -"
In which case, you'd use the stdin: parameter of the command: module:
- name: generate kubernetes yaml
command: echo "run your command that generates yaml here"
register: k8s_yaml
- name: feed the yaml to kubectl apply
command: kubectl apply -f -
stdin: '{{ k8s_yaml.stdout }}'
It isn't super clear what the relationship is in your question between the top part, dealing with the template:, and the bottom part about apply -f -, but if you mean "how can I render a template to a variable, instead of a file?" the the answer is the template lookup plugin:
- name: render the yaml
set_fact:
k8s_yaml: '{{ lookup("template", "templates/foo.j2") }}'
- name: now feed it to apply
command: kubectl apply -f -
stdin: '{{ k8s_yaml }}'
You've got a couple options here. I usually try to stay away from shelling out to command wherever possible. Check out the k8s module in ansible. Note that as long as state is present ansible will patch your object.
- name: Apply your previously generated configmap if you so choose.
k8s:
state: present
definition: "{{ lookup('file', '/output/foo.yml') }}"
Or even better, you could just directly create the configmap
- name: Create the configmap for {{ env }}
k8s:
state: present
definition:
apiVersion: v1
kind: ConfigMap
metadata:
name: ConfigMap
namespace: "{{ foo_namespace }}"
labels:
app: bar
environment: "{{ bizzbang }}"
Related
We're using Ansible playbook with GitLab CI in this project, where we'd pass some variables from ENV_FILE through Ansible playbook, then rendering JJ2 template with them.
Now the problem occurs when some variable has $ in its value, which seems interpreted as shell variable at some point, and the final value is rendered incorrect.
For example, in ENV_FILE
(set via GitLab CI Settings > CI/CD > Variables menu):
export FIRST_VAR=...
export SOME_VAR='123$abc#xyz'
export SOME_OTHER_VAR=...
And the final result in docker-compose.yaml becomes 123#xyz
EDIT: We just tried changing to export SOME_VAR='123''$''abc#xyz', the final result becomes 123abc#xyz, still missing the $.
gitlab-ci.yaml
deploy:
stage: deploy
environment:
name: dev
script:
- source $ENV_FILE
- cd ansible && ansible-playbook -i inventory/dev.ini runapp.yaml --vault-password-file=${ANSIBLE_VAULT_FILE}
runapp.yaml
- hosts: app
become: yes
roles:
- { role: some_app }
vars:
SOME_VAR: "{{ lookup('env', 'SOME_VAR') }}"
Task File:
- name: "Templating docker-compose file"
become: yes
template:
src: app-docker-compose.yaml.j2
dest: /opt/someapp/docker-compose.yaml
app-docker-compose.yaml.j2
someapp-svc:
image: someapp:version
restart: always
ports:
- ####:####
environment:
SOME_VAR: {{ SOME_VAR }}
Any hint about this?
Thanks!
I can reproduce that behavior when setting a CI/CD variable containing $; the docs kind of hint at it, although the docs are written as if the problem only applies when setting variables inside .gitlab-ci.yml which is demonstrably false
If you want a CI/CD variable to contain a literal $, it needs to be doubled, so SOME_VAR would need to be written as 123$$abc#xyz in the CI/CD configuration page in order for it to materialize as 123$abc#xyz inside the pipeline (although as the comments correctly point out, one will want to be exceedingly careful about the use of source to avoid further interpolation)
I am creating an ecs_taskdefinition in ansible, but I would like the task-defintion in a sperate file. Can I somehow do something like this:
ecs_taskdefintion:
containers: {{ load_external_yaml containers.yaml }}
volumes: {{ load_external_yaml_volumes.yaml }}
So I want to load the yaml data from external files.
You may try to combine file lookup and from_yaml filter like this:
{{ lookup('file','containers.yaml') | from_yaml }}
Remember that lookups are local, so containers.yaml should be on ansible control host.
Since your file is YAML, you may use include_vars
from https://docs.ansible.com/ansible/latest/collections/ansible/builtin/include_vars_module.html :
- name: Setup vars
tags: ["always"]
include_vars:
file: "./vars/tintin.yaml"
name: tintin
Use tintin as a normal var everywhere!
In order to perform some operations locally (not on the remote machine), I need to put the content of an ansible variable inside a temporary file.
Please note that I am looking for a solution that takes care of generating the temporary file to a location where it can be written (no hardcoded names) and also that takes care of the removal of the file as we do not want to leave things behind.
You should be able to use the tempfile module, followed by either the copy or template modules. Like so:
- hosts: localhost
tasks:
# Create a file named ansible.{random}.config
- tempfile:
state: file
suffix: config
register: temp_config
# Render template content to it
- template:
src: templates/configfile.j2
dest: "{{ temp_config.path }}"
vars:
username: admin
Or if you're running it in a role:
- tempfile:
state: file
suffix: config
register: temp_config
- copy:
content: "{{ lookup('template', 'configfile.j2') }}"
dest: "{{ temp_config.path }}"
vars:
username: admin
Then just pass temp_config.path to whatever module you need to pass the file to.
It's not a great solution, but the alternative is writing a custom module to do it in one step.
Rather than do it with a file, why not just use the environment? This wan you can easily work with the variable and it will be alive through the ansible session and you can easily retrieve it in any steps or outside of them.
Although using the shell/application environment is probably, if you specifically want to use a file to store variable data you could do something like this
- hosts: server1
tasks:
- shell: cat /etc/file.txt
register: some_data
- local_action: copy dest=/tmp/foo.txt content="{{some_data.stdout}}"
- hosts: localhost
tasks:
- set_fact: some_data="{{ lookup('file', '/tmp/foo.txt') }}"
- debug: var=some_data
As for your requirement to give the file a unique name and clean it up at the end of the play. I'll leave that implementation to you
I want to create a generic ansible playbook yaml that will pull and run any image based on extra-vars parameter received.
E.g. ansible-playbook my generic.yml --extra-vars "image=helloworld version=2.1"
In generic.yml, I have the following lines, I am only showing the lines that are pertinent to my problem.
vars: imageName=image
imageVersion=version
pullImage="{{ imageName }}:{{ imageVersion }}"
tasks:
- docker
name: helloworld
image: "{{ pullImage }}"
When I run my playbook, it is erroring on the last line image: "{{ pullImage }}"
The error msg said it is expecting a dictionary.
Is the formatting only a problem of your post (copy & paste & remove non-relevant noise) or is that actually the problem?
You can see this is no valid YAML by running it through any YAML parser.
Correct tasks notation would look like this:
tasks:
- docker:
name: helloworld
image: "{{ pullImage }}"
I am trying to craft a list of environment variables to use in tasks that may have slightly different path on each host due to version differences.
For example, /some/common/path/v_123/rest/of/path
I created a list of these variables in variables.yml file that gets imported via roles.
roles/somerole/varables/main.yml contains the following
somename:
somevar: 'coolvar'
env:
SOME_LIB_PATH: /some/common/path/{{ unique_part.stdout }}/rest/of/path
I then have a task that runs something like this
- name: Get unique path part
shell: 'ls /some/common/path/'
register: unique_part
tags: workflow
- name: Perform some actions that need some paths
shell: 'binary argument argument'
environment: somename.env
But I get some Ansible errors about variables not being defined.
Alternatively I tried to predefine the unique_part.stdout in hopes of register overwriting predefined variable, but then I got other ansible errors - failure to template.
Is there another way to craft these variables based on command returns?
You can also use facts:
http://docs.ansible.com/set_fact_module.html
# Prepare unique variables
- hosts: myservers
tasks:
- name: Get unique path part
shell: 'ls /some/common/path/'
register: unique_part
tags: workflow
- name: Add as Fact per for each hosts
set_fact:
library_path: "{{ unique_part.stdout }}"
# launch roles that use those unique variables
- hosts: myservers
roles:
- somerole
This way you can dynamicaly add variable to your hosts before using them.
The vars files gets evaluated when it is read by Ansible. Your only chance would be to include a placeholder which you then later have to replace yourself, like this:
somename:
somevar: 'coolvar'
env:
SOME_LIB_PATH: '/some/common/path/[[ unique_part.stdout ]]/rest/of/path'
And then later in your playbook you can replace that placeholder:
- name: Perform some actions that need some paths
shell: 'binary argument argument'
environment: '{{ somename.env | replace("[[ unique_part.stdout ]]", unique_part.stdout) }}'