I have a file called values.txt on each of the server in the directory /tmp/values.txt and it has some values. And I have a jinja template and I am substituting some values from the the values.txt
But the problem is, when i use the lookup command it looks in the controller server and not the remote servers
Here's what I tried:
- name: Create /etc/systemd/system/etcd.service
template:
src: etcd.service.j2
dest: /etc/systemd/system/etcd.service
vars:
value_from_file: "{{ lookup('file', '/tmp/values.txt').split('\n') }}"
vars_from_jinja: [SELF_NAME, SELF_IP, NODE_1_NAME, NODE_1_IP, NODE_2_NAME, NODE_2_IP, NODE_3_NAME, NODE_3_IP, NODE_4_NAME, NODE_4_IP]
my_dict: "{{ dict(vars_from_jinja|zip(value_from_file)) }}"
How can I can this task remotely? Or is there another workaround to substitute the values to the jinja template?
PS: I can't fetch the values.txt to the controller because the content of values.txt in each server is slightly different from the other.
Can someone please help me?
The lookup find the file (or stream) on the controller. If you file is on the remote node, you can't use the lookup
Lookup plugins are an Ansible-specific extension to the Jinja2 templating language. You can use lookup plugins to access data from outside sources (files, databases, key/value stores, APIs, and other services) within your playbooks. Like all templating, lookups execute and are evaluated on the Ansible control machine. Ansible makes the data returned by a lookup plugin available using the standard templating system. You can use lookup plugins to load variables or templates with information from external sources.
try the cat file instead:
- name: read the values.txt
shell: cat /tmp/values.txt
register: data
- name: Create /etc/systemd/system/etcd.service
template:
src: etcd.service.j2
dest: /etc/systemd/system/etcd.service
vars:
value_from_file: "{{ data.stdout_lines }}"
vars_from_jinja: [ SELF_NAME, SELF_IP, NODE_1_NAME, NODE_1_IP, NODE_2_NAME, NODE_2_IP, NODE_3_NAME, NODE_3_IP, NODE_4_NAME, NODE_4_IP ]
my_dict: "{{ dict(vars_from_jinja|zip(value_from_file)) }}"
Use slurp:
- name: read remote values.txt
register: values
ansible.builtin.slurp:
src: /tmp/values.txt
- name: Create /etc/systemd/system/etcd.service
template:
src: etcd.service.j2
dest: /etc/systemd/system/etcd.service
vars:
value_from_file: "{{ (values.content | b64decode).split('\n') }}"
vars_from_jinja: [SELF_NAME, SELF_IP, NODE_1_NAME, NODE_1_IP, NODE_2_NAME, NODE_2_IP, NODE_3_NAME, NODE_3_IP, NODE_4_NAME, NODE_4_IP]
my_dict: "{{ dict(vars_from_jinja|zip(value_from_file)) }}"
Related
I created a Worflow job in awx containing 2 jobs:
Job 1 is using the credentials of the windows server where we get the json file from. It reads the content and put it in a variable using set_stats
Job2 is using the credential of the server where to upload the json file. It reads the content of the variable set in the job 1 in the set_stats task and creates a json file with the content.
First job:
- name: get content
win_shell: 'type {{ file_dir }}{{ file_name }}'
register: content
- name: write content
debug:
msg: "{{ content.stdout_lines }} "
register: result
- set_fact:
this_local: "{{ content.stdout_lines }}"
- set_stats:
data:
test_stat: "{{ this_local }}"
- name: set hostname in a variable
set_stats:
data:
current_hostname: "{{ ansible_hostname }}"
per_host: no
Second job
- name: convert to json and copy the file to destination control node.
copy:
content: "{{ test_stat | to_json }}"
dest: "/tmp/{{ current_hostname }}.json"
How can I get the current_hostname, so that the the created json file is named <original_hostname>.json? In my case its concatenating the two hosts which I passed in the first job.
In my case its concatenating the two hosts which I passed in the first job
... which is precisely what you asked for since you used per_host: no as parameter to set_stats to gather the current_hostname stat globally for all host and that aggregate: yes is the default.
Anyhow, this is not exactly the intended use of set_stats and you are making this overly complicated IMO.
You don't need two jobs. In this particular case, you can delegate the write task to a linux host in the middle of a play dedicated to windows hosts (and one awx job can use several credentials).
Here is a pseudo untested playbook to give you the idea. You'll want to read the slurp module documentation which I used to replace your shell task to read the file (which is a bad practice).
Assuming your inventory looks something like:
---
windows_hosts:
hosts:
win1:
win2:
linux_hosts:
hosts:
json_file_target_server:
The playbook would look like:
- name: Gather jsons from win and write to linux target
hosts: windows_hosts
tasks:
- name: Get file content
slurp:
src: "{{ file_dir }}{{ file_name }}"
register: json_file
- name: Push json content to target linux
copy:
content: "{{ json_file.content | b64decode | to_json }}"
dest: "/tmp/{{ inventory_hostname }}.json"
delegate_to: json_file_target_server
I feel I must be missing this answer as it seems obvious but I've read a number of posts and have not been able to get this working.
Currently I am loading and then templating vars from files depending on inventory hostnames, like so:
- name: load unique dev vars from file
include_vars:
file: ~/ansible/env-dev.yml
when: inventory_hostname in groups[ 'devs' ]
- name: load unique prod vars from file
include_vars:
file: ~/ansible/env-prod.yml
when: inventory_hostname == 'prod'
- name: copy .env dev file with templated vars
ansible.builtin.template:
src: ~/ansible/env-dev.j2
dest: /home/{{ inventory_hostname }}/.env
owner: '{{ inventory_hostname }}'
group: '{{ inventory_hostname }}'
mode: '0600'
when: inventory_hostname in groups[ 'devs' ]
This works fine but ultimately it is requiring me to create a ton of .yml files when I would rather include some variables in certain steps instead.
Is it possible to load vars for a specific task? I've tried a number of solutions but haven't been able to make it work yet. See below for one method I tried using vars at the end of the task.
- name: copy .env dev file with templated vars
ansible.builtin.template:
src: ~/ansible/env-dev.j2
dest: /home/{{ inventory_hostname }}/.env
owner: '{{ inventory_hostname }}'
group: '{{ inventory_hostname }}'
mode: '0600'
when: inventory_hostname in groups[ 'devs' ]
vars:
NODE_ENV: development
PORT: 66
The key to organize your Ansible code is to rely on group vars.
This feature permits to load variables according to the group a host belong to. You have several ways to do that, one of the clearest way is to use yaml files named with the name of the group in the group_vars folder (plus a all.yaml matching all hosts). Ansible will pick automatically them for you, so you can get rid of your first two include_vars. You can combine them with variables specific to the role and or the playbook. So you end with a set of variables coming from the host (the target) and from the role / playbook (the task to achieve).
To replace the hardcoded src: ~/ansible/env-dev.j2 you could for example define a variable in each group.
---
# dev.yaml
template_name: "env-dev.j2"
---
# prod.yaml
template_name: "env-prod.j2"
And then use it in your playbook / role src: "{{ template_name }}".
In order to perform some operations locally (not on the remote machine), I need to put the content of an ansible variable inside a temporary file.
Please note that I am looking for a solution that takes care of generating the temporary file to a location where it can be written (no hardcoded names) and also that takes care of the removal of the file as we do not want to leave things behind.
You should be able to use the tempfile module, followed by either the copy or template modules. Like so:
- hosts: localhost
tasks:
# Create a file named ansible.{random}.config
- tempfile:
state: file
suffix: config
register: temp_config
# Render template content to it
- template:
src: templates/configfile.j2
dest: "{{ temp_config.path }}"
vars:
username: admin
Or if you're running it in a role:
- tempfile:
state: file
suffix: config
register: temp_config
- copy:
content: "{{ lookup('template', 'configfile.j2') }}"
dest: "{{ temp_config.path }}"
vars:
username: admin
Then just pass temp_config.path to whatever module you need to pass the file to.
It's not a great solution, but the alternative is writing a custom module to do it in one step.
Rather than do it with a file, why not just use the environment? This wan you can easily work with the variable and it will be alive through the ansible session and you can easily retrieve it in any steps or outside of them.
Although using the shell/application environment is probably, if you specifically want to use a file to store variable data you could do something like this
- hosts: server1
tasks:
- shell: cat /etc/file.txt
register: some_data
- local_action: copy dest=/tmp/foo.txt content="{{some_data.stdout}}"
- hosts: localhost
tasks:
- set_fact: some_data="{{ lookup('file', '/tmp/foo.txt') }}"
- debug: var=some_data
As for your requirement to give the file a unique name and clean it up at the end of the play. I'll leave that implementation to you
I needed to be able to invert variables stored in a JSON file that is passed to the playbook from the command line.
These are the tasks that I set up (they are identical except for vars), this is a fragment of a playbook:
- name: Prepare a .sql file
delegate_to: 127.0.0.1
mysql_db:
name: "{{ source['database']['db_name'] }}"
state: dump
login_host: "{{ source['database']['host'] }}"
login_user: "{{ source['database']['user'] }}"
login_password: "{{ source['database']['password'] }}"
target: test_db.sql
when: invert is not defined
- name: Prepare a .sql file (inverted)
delegate_to: 127.0.0.1
mysql_db:
name: "{{ target['database']['db_name'] }}"
state: dump
login_host: "{{ target['database']['host'] }}"
login_user: "{{ target['database']['user'] }}"
login_password: "{{ target['database']['password'] }}"
target: test_db.sql
when: invert is defined
So consequently when I execute
ansible-playbook -i hosts playbook.yml --extra-vars "#dynamic_vars.json"
the first task is executed. If I execute
ansible-playbook -i hosts playbook.yml --extra-vars "#dynamic_vars.json" --extra-vars "invert-yes"
the second task is executed that takes the same hash as parameters, but only swaps source for target (which essentially becomes a source in my playbook).
As you can see, this is a very simplistic approach, there is a lot of unnecessary duplication, I just do not like it. However, I cannot think of a better way to be able to revert variables at the command line without building some more complex include logic.
Perhaps you can advice me on how I can do it better? Thanks!
I'm a big fan of YAMLs anchors and references when it comes to the topic of avoiding repetition. Since the content is dynamic, you could take advantage of with_items, which can be used to pass a parameter like so:
- &sqldump
name: Prepare a .sql file
delegate_to: 127.0.0.1
mysql_db:
name: "{{ item['database']['db_name'] }}"
state: dump
login_host: "{{ item['database']['host'] }}"
login_user: "{{ item['database']['user'] }}"
login_password: "{{ item['database']['password'] }}"
target: test_db.sql
when: invert is not defined
with_items:
- source
- <<: *sqldump
name: Prepare a .sql file (inverted)
when: invert is defined
with_items:
- target
The 2nd task is a perfect clone of the first one, you then override the name, condition and the loop with_items to pass the target instead of the source.
After reading your answer to #ydaetskcoR it sounds like you have quite some cases where you need to use the data from one or the other dict. Maybe in that case it then would make sense to just define the var globally depending on the invert parameter. Your vars file could look like that:
---
source:
database: ...
db_name: ...
target:
database: ...
db_name: ...
data: "{{ target if invert is defined else source }}"
You then simply can use data in all your tasks without dealing with conditions any further.
- name: Prepare a .sql file
delegate_to: 127.0.0.1
mysql_db:
name: "{{ data['database']['db_name'] }}"
state: dump
login_host: "{{ data['database']['host'] }}"
login_user: "{{ data['database']['user'] }}"
login_password: "{{ data['database']['password'] }}"
target: test_db.sql
Of course, this way you have a fixed task name which does not change with the param you pass.
If you are attempting to do the same thing but just want to specify different variables depending on the host/group then a better approach may be to simply set these as host/group vars and run it as a single task.
If we set up our inventory file a bit like this:
[source_and_target-nodes:children]
source-nodes
target-nodes
[source-nodes]
source database_name='source_db' database_login_user='source_user' database_login_pass='source_pass'
[target-nodes]
target database_name='target_db' database_login_user='target_user' database_login_pass='target_pass'
Then we can target the task at the source_and_target-nodes like so:
- name: Prepare a .sql file
hosts: source_and_target-nodes
mysql_db:
name: "{{ database_name }}"
state: dump
login_host: "{{ inventory_hostname }}"
login_user: "{{ database_login_user }}"
login_password: "{{ database_login_pass }}"
target: test_db.sql
You won't be able to access the host vars of a different host this easily if you need to use delegate_to as you are in your question but if you are simply needing to run the play locally you can instead set ansible_connection to local in your host/group vars or setting connection: local in the play.
I'm using the ec2 module with ansible-playbook I want to set a variable to the contents of a file. Here's how I'm currently doing it.
Var with the filename
shell task to cat the file
use the result of the cat to pass to the ec2 module.
Example contents of my playbook.
vars:
amazon_linux_ami: "ami-fb8e9292"
user_data_file: "base-ami-userdata.sh"
tasks:
- name: user_data_contents
shell: cat {{ user_data_file }}
register: user_data_action
- name: launch ec2-instance
local_action:
...
user_data: "{{ user_data_action.stdout }}"
I assume there's a much easier way to do this, but I couldn't find it while searching Ansible docs.
You can use lookups in Ansible in order to get the contents of a file, e.g.
user_data: "{{ lookup('file', user_data_file) }}"
Caveat: This lookup will work with local files, not remote files.
Here's a complete example from the docs:
- hosts: all
vars:
contents: "{{ lookup('file', '/etc/foo.txt') }}"
tasks:
- debug: msg="the value of foo.txt is {{ contents }}"
You can use the slurp module to fetch a file from the remote host: (Thanks to #mlissner for suggesting it)
vars:
amazon_linux_ami: "ami-fb8e9292"
user_data_file: "base-ami-userdata.sh"
tasks:
- name: Load data
slurp:
src: "{{ user_data_file }}"
register: slurped_user_data
- name: Decode data and store as fact # You can skip this if you want to use the right hand side directly...
set_fact:
user_data: "{{ slurped_user_data.content | b64decode }}"
You can use fetch module to copy files from remote hosts to local, and lookup module to read the content of fetched files.
lookup only works on localhost. If you want to retrieve variables from a variables file you made remotely use include_vars: {{ varfile }} . Contents of {{ varfile }} should be a dictionary of the form {"key":"value"}, you will find ansible gives you trouble if you include a space after the colon.