I want to set a playbook level environment, but after I execute a couple of tasks. I have found that I could define a playbook level environment variable before definition of any tasks or task level environment variables. But, I haven't found how can I set-up an environment variable that can be used by all tasks following a task.
- name: server properties
hosts: kafka_broker
vars:
ansible_ssh_extra_args: "-o StrictHostKeyChecking=no"
ansible_host_key_checking: false
date: "{{ lookup('pipe', 'date +%Y%m%d-%H%M%S') }}"
copy_to_dest: "/export/home/kafusr/kafka/secrets"
server_props_loc: "/etc/kafka"
secrets_props_loc: "{{ server_props_loc }}/secrets"
environment:
CONFLUENT_SECURITY_MASTER_KEY: "{{ extract_key2 }}"
tasks:
- name: Create a directory if it does not exist
file:
path: "{{ copy_to_dest }}"
state: directory
mode: '0755'
- name: Find files from "{{ server_props_loc }}"
find:
paths: /etc/kafka/
patterns: "server.properties*"
# ... the rest of the task
register: etc_kafka_server_props
- name: Find files from "{{ secrets_props_loc }}"
find:
paths: /etc/kafka/secrets
patterns: "*"
# ... the rest of the task
register: etc_kafka_secrets_props
- name: Copy the files
copy:
src: "{{ item.path }}"
dest: "{{ copy_to_dest }}"
remote_src: yes
loop: "{{ etc_kafka_server_props.files + etc_kafka_secrets_props.files }}"
- name: set masterkey content value
set_fact:
contents: "{{ lookup('file', '/export/home/kafusr/kafka/secrets/masterkey.txt') }}"
extract_key2: "{{ contents.split('\n').2.split('|').2|trim }}"
I want to set CONFLUENT_SECURITY_MASTER_KEY after the set_facts task
Is it possible to set playbook level environment variable, but after defining some tasks
Thank you
UPDATE
Initially, when I was executing the playbook as originally defined, I was getting the error
fatal: [kafkaserver1]: FAILED! => {"msg": "The field 'environment' has an invalid value,
which includes an undefined variable. The error was: 'extract_key2' is undefined"}
which was expected as the variable extract_key2 was not set - before copying the files to desired directory.
After #Zeitounator's suggestion, when I added default to the environment variable's definition,
CONFLUENT_SECURITY_MASTER_KEY: "{{ extract_key2 | default('') }}"
I now get a different error
The new error is
TASK [set masterkey content value] ******************** fatal: [kafkaserver1]: FAILED! =>
{"msg": "The task includes an option with an undefined variable. The error was: 'contents' is undefined\n\n
The error appears to be in '/export/home/kafuser/tmp/so-71538207-question.yml': line 43, column 7, but may\n
be elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n
- name: set masterkey content value\n ^ here\n"}
Getting this on all 3 brokers in the console and I checked the file it exists
I did do a cat on that file, copying the path from error to make sure there is no typo, and the contents of that file are displayed on console.
Update 2
I am trying to figure out how to use slurp to get the info, with the same approach as #Zeitounator's example about using lookup.
This is what I am trying. The current definition, is of course, erroneous. Just wanted to show what I am trying to do. But, can it be done for slurp and am I on the right path?
environment:
CONFLUENT_SECURITY_MASTER_KEY: >-
{{
(
((slurp: src: /export/home/z8tpush/kafka/secrets/masterkey.txt)['content'] | b64decode ).split('\n').2.split('|').2|trim
)
}}
#Zeitounator - Will you be able to direct me to an example where a slurp or fetch module is defined to set-up an environment variable and where the value will get updated after the tasks that create the file are executed, similar to what you have shown with lookup filter? I would really appreciate it.
Note:
Ultimately, I want to use ansible to create a new kafka user using confluents CLI commands ( using shell or command module ), verify it in my directory and once satisfied, I will encrypt the security.properties file using the masterkey and copy it to the appropriate location where confluent is installed.
As already mentioned, you can
With Ansible Configuration Settings set environment variables globally
Setting the remote environment in a task
Regarding your question
I haven't found how can I set-up an environment variable that can be used by all tasks following a task.
You can set the environment on Block level, a logical groups of tasks too
Setting the remote environment: "When you set a value with environment: at the play or block level, it is available only to tasks within the play or block that are executed by the same user."
This means you would need to define a block for the next tasks
- name: Block of next task(s)
block:
- name: Next task
...
environment:
CONFLUENT_SECURITY_MASTER_KEY: "{{ extract_key2 }}"
Regarding your question
Is it possible to set playbook level environment variable, but after defining some tasks?
No, not on that level in that run as the playbook is already running.
Another option might be to distribute the tasks in question into an other role, playbook or task file and include_* it.
You cannot set_fact a var depending on an other var in the same block. Moreover, there is absolutely no need to set_fact here as long as your relevant tasks can live with an empty environment var until it is fully defined. The following environment declaration (untested) should work and return the key for every task running after your file exists.
environment:
CONFLUENT_SECURITY_MASTER_KEY: >-
{{
(
(
lookup('file', '/export/home/kafusr/kafka/secrets/masterkey.txt', errors='ignore')
| default('')
).split('\n').2
| default('')
).2
| default('')
| trim
}}
Related
I created a Worflow job in awx containing 2 jobs:
Job 1 is using the credentials of the windows server where we get the json file from. It reads the content and put it in a variable using set_stats
Job2 is using the credential of the server where to upload the json file. It reads the content of the variable set in the job 1 in the set_stats task and creates a json file with the content.
First job:
- name: get content
win_shell: 'type {{ file_dir }}{{ file_name }}'
register: content
- name: write content
debug:
msg: "{{ content.stdout_lines }} "
register: result
- set_fact:
this_local: "{{ content.stdout_lines }}"
- set_stats:
data:
test_stat: "{{ this_local }}"
- name: set hostname in a variable
set_stats:
data:
current_hostname: "{{ ansible_hostname }}"
per_host: no
Second job
- name: convert to json and copy the file to destination control node.
copy:
content: "{{ test_stat | to_json }}"
dest: "/tmp/{{ current_hostname }}.json"
How can I get the current_hostname, so that the the created json file is named <original_hostname>.json? In my case its concatenating the two hosts which I passed in the first job.
In my case its concatenating the two hosts which I passed in the first job
... which is precisely what you asked for since you used per_host: no as parameter to set_stats to gather the current_hostname stat globally for all host and that aggregate: yes is the default.
Anyhow, this is not exactly the intended use of set_stats and you are making this overly complicated IMO.
You don't need two jobs. In this particular case, you can delegate the write task to a linux host in the middle of a play dedicated to windows hosts (and one awx job can use several credentials).
Here is a pseudo untested playbook to give you the idea. You'll want to read the slurp module documentation which I used to replace your shell task to read the file (which is a bad practice).
Assuming your inventory looks something like:
---
windows_hosts:
hosts:
win1:
win2:
linux_hosts:
hosts:
json_file_target_server:
The playbook would look like:
- name: Gather jsons from win and write to linux target
hosts: windows_hosts
tasks:
- name: Get file content
slurp:
src: "{{ file_dir }}{{ file_name }}"
register: json_file
- name: Push json content to target linux
copy:
content: "{{ json_file.content | b64decode | to_json }}"
dest: "/tmp/{{ inventory_hostname }}.json"
delegate_to: json_file_target_server
I am trying to do a custom install of openedx and I have a bunch of .yml files with environment variables in them inside paths that looks like this
playbooks/roles/<component-name>/defaults/main.yml
Then, while running a playbook that installs all such components, I'm using a command like this
ansible-playbook ./openedx_native.yml -e"#roles/<component-name-1>/defaults/main.yml" -e"#roles/<component-name-2>/defaults/main.yml"
Now I want to be able to use the main.yml files from all components and there are about 20-25 of them, so I'm looking for a way to include them using a wildcard, something like this
ansible-playbook ./openedx_native.yml -e"#roles/*/defaults/main.yml"
This, of course, doesn't work and Ansible throws an error like this
ERROR! the file_name
'/var/tmp/configuration/playbooks/roles/*/defaults/main.yml' does not
exist, or is not readable
How do I achieve this? Please help!
An option would be to find the files and include_vars.
tasks:
- command: "sh -c 'find {{ playbook_dir }}/roles/*/defaults/main.yml'"
register: result
- include_vars:
file: "{{ item }}"
loop: "{{ result.stdout_lines }}"
If you have flexibility to change & re-arrange environment variables and its values in /group/all.yaml like environments:
- { name: ‘development’, profile: 'small' }
- { name: ‘staging’, profile: ‘medium’ }
- { name: ‘production’, profile: ‘complex’ }
And then you can use this variable for any task say for example you want to create folder with environment name
- name: create folders for Environment
file:
path: "{{ target }}/{{ item.name }}"
state: directory
mode: 0755
with_items: "{{ environments }}"
I have the following ansible playbook that writes the content of the variable "hello" as a message (I got this code from an example online). I tried modifying it so it would write this to a local file however I got an error. The modified code and error message are below:
original code (successful):
- hosts: all
vars:
hello: world
tasks:
- name: Ansible Basic Variable Example
debug:
msg: "{{ hello }}"
modified code (unsuccessful):
- hosts: all
vars:
hello: world
tasks:
- name: Ansible Basic Variable Example
- local_action: copy content={{hello}} dest=~/ansible_logfile
debug:
msg: "{{ hello }}"
error message:
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
The error appears to have been in '/space/mathewLewis/towerCodeDeploy/playBooks/test.yml': line 5, column 5, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
tasks:
- name: Ansible Basic Variable Example
^ here
I would like to know how to write a variable to a file correctly
It's a simple syntax error.
A Task is an entry in the list of tasks, which in YAML is designated by a - (dash).
Task names are optional in Ansible.
Both copy and debug are modules, which are supposed to be a task's "action".
What the error message is telling you is that the task with name: Ansible Basic Variable Example does not have an action, which is because your local_action is a separate task, indicated by a -.
Fixing your example with appropriate names for the tasks:
- name: Write variable to file
local_action: copy content="{{hello}}" dest=~/ansible_logfile
- name: Output the variable
debug:
msg: "{{ hello }}"
Thomas Hirsh's answer is correct. However, I found this representation less confusing (I'm a newbie to ansible):
- name: "Controller"
hosts: "controller.jeff.ddns.net"
tasks:
- name: "Register a variable to be shared with the clients"
set_fact: shared_fact="Brother"
- name: "Client"
hosts: "client.jeff.ddns.net"
tasks:
- name: "writing to hostvars.json"
local_action: copy content="{{hostvars | to_json(indent=4) }}" dest="hostvars.json"
This example has two plays. The controller play only sets a variable. The client is what actually writes to the file. In this case, hostvars has a complicated structure, so I used the to_json(indent=4) filter to convert to a good .json file, suitable for use with jq .
In my ansible vars file, I have a variable that will sometimes be set and other times I want to dynamically set it. For example, I have an RPM that I want to install. I can manually store the location in a variable, or if I don't have a particular one in mind, I want to pull the latest from Jenkins. My question is, how can I check if the variable is not defined or empty, and if so, just use the default from Jenkins (already stored in a var)?
Here is what I have in mind:
...code which gets host_vars[jenkins_rpm]
- hosts: "{{ host }}"
tasks:
- name: Set Facts
set_fact:
jenkins_rpm: "{{ hostvars['localhost']['jenkins_rpm'] }}"
- name: If my_rpm is empty or not defined, just use the jenkins_rpm
set_fact: my_rpm=jenkins_rpm
when: !my_rpm | my_rpm == ""
There is default filter for that:
- set_fact:
my_rpm: "{{ my_rpm | default(jenkins_rpm) }}"
I cannot get this seemingly simple example to work in Ansible 1.8.3. The variable interpolation does not kick in the task name. All examples I have seen seem to suggest this should work. Given that the variable is defined in the vars section I expected the task name to print the value of the variable. Why doesn't this work?
Even the example from the Ansible documentation seems to not print the variable value.
---
- hosts: 127.0.0.1
gather_facts: no
vars:
vhost: "foo"
tasks:
- name: create a virtual host file for {{ vhost }}
debug: msg="{{ vhost }}"
This results in the following output:
PLAY [127.0.0.1]
**************************************************************
TASK: [create a virtual host file for {{ vhost }}]
****************************
ok: [127.0.0.1] => {
"msg": "foo"
}
PLAY RECAP
********************************************************************
127.0.0.1 : ok=1 changed=0 unreachable=0 failed=0
Update
This works with 1.7.2 but does not work with 1.8.3. So either this is a bug or a feature.
Variables are not resolved inside the name. Only inside the actual tasks/conditions etc. the placeholders will be resolved. I guess this is by design. Imagine you have a with_items loop and use the {{ item }}in the name. The tasks name will only be printed once, but the {{ item }} would change in every iteration.
I see the examples, even the one in the doc you linked to, use variables in the name. But that doesn't mean the result would be like you expected it. The docs are community managed. It might be someone just put that line there w/o testing it - or maybe it used to work like that in a previous version of Ansible and the docs have not been updated then. (I'm only using Ansible since about one year). But even though it doesn't work like we wish it would, I'm still using variables in my name's, just to indicate that the task is based on dynamic parameters. Might be the examples have been written with the same intention.
An interesting observation I recently made (Ansible 1.9.4) is, default values are written out in the task name.
- name: create a virtual host file for {{ vhost | default("foo") }}
When executed, Ansible would show the task title as:
TASK: [create a virtual host file for foo]
This way you can avoid ugly task names in the output.
Explanation
Whether the variable gets interpolated depends on where it has been declared.
Imagine You have two hosts: A and B.
If variable foo has only per-host values, when Ansible runs the play, it cannot decide which value to use.
On the other hand, if it has a global value (global in a sense of host invariance), there is no confusion which value to use.
Source: https://github.com/ansible/ansible/issues/3103#issuecomment-18835432
Hands on playbook
ansible_user is an inventory variable
greeting is an invariant variable
- name: Test variable substitution in names
hosts: localhost
connection: local
vars:
greeting: Hello
tasks:
- name: Sorry {{ ansible_user }}
debug:
msg: this won't work
- name: You say '{{ greeting }}'
debug:
var: ansible_user
I experienced the same problem today in one of my Ansible roles and I noticed something interesting.
When I use the set_fact module before I use the vars in the task name, they actually get translated to their correct values.
In this example I wanted to set the password for a remote user:
Notice that I use the vars test_user and user_password that I set as facts before.
- name: Prepare to set user password
set_fact:
user_password: "{{ linux_pass }}"
user_salt: "s0m3s4lt"
test_user: "{{ ansible_user }}"
- name: "Changing password for user {{ test_user }} to {{ user_password }}"
user:
name: "{{ ansible_user }}"
password: "{{ user_password | password_hash('sha512', user_salt) }}"
state: present
shell: /bin/bash
update_password: always
This gives me the following output:
TASK [install : Changing password for user linux to LiNuXuSeRPaSs#]
So this solved my problem.
It might be ugly, but you can somewhat workaround with something like this:
- name: create a virtual host file
debug:
msg: "Some command result"
loop: "{{ [ vhost ] }}"
or
- name: create a virtual host file
debug:
msg: "Some command result"
loop_control:
label: "{{ vhost }}"
loop: [1]
I wouldn't do this in general, but it shows how you can use items or label to give information outside of the command result. While it might not
Source: https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html