I have a JSON object of files and the destinations they need to be symlinked moved to. I would like to be able to have environment variables in the destinations and have them evaluated to know where they should be moved to. I'm attempting to do a regex_search with a lookup, but it isn't giving me the desired result.
Here is the json:
dotfiles.json
{
"mac": [
{
"name": ".gitconfig",
"destination": "$HOME/"
}
]
}
playbook: ansible/bootstrap.yaml
---
- name: "Bootstrapping Machine"
hosts: localhost
connection: local
ignore_errors: true
vars:
dotfiles: "{{ lookup('file', '../dotfiles.json') | from_json }}"
roles:
- role: "MacOSX"
when: "ansible_distribution == 'MacOSX'"
tasks: ansible/roles/MacOSX/tasks/main.yaml
- name: Symlink dotfiles
ansible.builtin.file:
src: "../../../dotfiles/{{ item.name }}"
dest: "{{ destination }}"
state: link
vars:
destination: "{{ item.destination | regex_replace('\\$\\w+', lookup('env', '\\1')) }}"
with_items: "{{ dotfiles.mac | list }}"
The destination variable evaluates to /.
Below is a quick and dirty fix of your original idea. This will only work if you target localhost (as lookups only run on localhost) hence will fail for a remote target (as the env var will be gathered for the current user on your local environment).
Moreover, the regexes/replace/lookups will probably fail as soon as you introduce more than one env var in your string(s). But at least you get a first working example for your particular situation and can build over it.
The following example playbook (using the same json file as in your question):
- name: "Replace dollar env var with local env value"
hosts: localhost
gather_facts: false
vars:
dotfiles: "{{ lookup('file', 'dotfiles.json') }}"
tasks:
- name: Replace dollar env var with local env value
vars:
env_var: >-
{{ item.destination | regex_replace('\$(\w+)/', '\1') }}
destination: >-
{{ item.destination | regex_replace('\$\w+[^/]', lookup('env', env_var)) }}{{ item.name }}
debug:
msg: "{{ destination }}"
loop: "{{ dotfiles.mac }}"
Gives (username redacted)
$ ansible-playbook bootstrap.yml
PLAY [Replace dollar env var with local env value] *****************************************************************************
TASK [Replace dollar env var with local env value] *****************************************************************************
ok: [localhost] => (item={'name': '.gitconfig', 'destination': '$HOME/'}) => {
"msg": "/home/somelocaluser/.gitconfig"
}
PLAY RECAP *****************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Related
I have a variable reference to a variable that is not known when the Playbook starts and the reference is evaluated. Is there a way to lazy load the variable when it is actually accessed?
In the inventory:
all:
hosts:
l01lin08:
bamboo_agents_it:
- bamboo_agent_app_name: bamboo-agent-it
bamboo_agent_install_dir: /data/bamboo-agent-it
bamboo_agent_capabilities: "{{ bamboo_agent_capabilities_it }}"
- # ...
- # ...
bamboo_agent_capabilities_it:
# Error: bamboo_agent_install_dir is not known when evaluating the variable
capabilitiy: '{{ bamboo_agent_install_dir }}/bla/blubb'
another.one: 123
is.something: true
bamboo_agent_capabilities_it.capability references to bamboo_agents_it[].bamboo_agent_install_dir which is not known in this context.
The capabilities is a set of key-value-pairs which can be everything. The values are not always paths and the keys are not always the same. Thus I cannot tell when to add a parent path and when not in the target task.
In the playbook I call a role in a loop over bamboo_agents_it:
tasks:
Install all bamboo agents
- include_role:
name: bamboo-agent-linux
vars:
bamboo_agent_app_name: '{{ item.bamboo_agent_app_name }}'
bamboo_agent_install_dir: '{{ item.bamboo_agent_install_dir }}'
bamboo_agent_capabilities: '{{ item.bamboo_agent_capabilities }}'
loop: '{{ bamboo_agents_it }}'
In this loop item.bamboo_agent_install_dir is available but I want to have a generic config definition. I don't want to handle these single keys separately.
Is there a way to solve this problem in a generic way?
Maybe can I lazyload the variable in the task when it is accessed?
A workaround I'm using is to replace a placeholder insteed. But I hope there is more idiomatic way:
# Inventory
bamboo_agent_capabilities_it:
# Error: bamboo_agent_install_dir is not known when evaluating the variable
capabilitiy: '[bamboo_agent_install_dir]/bla/blubb'
# Task:
lineinfile:
path: '{{ bamboo_agent_install_dir }}/bin/bamboo-capabilities.properties'
line: '{{ item.key }}={{ item.value | replace("[bamboo_agent_install_dir]", bamboo_agent_install_dir) }}'
loop: '{{ bamboo_agent_default_capabilities | dict2items + bamboo_agent_capabilities | dict2items }}'
What you could do is to declare the name that you would expect the variable to be named after:
bamboo_agent_capabilities_it:
capability: /foo/bar
dir: bamboo_agent_install_dir
Then use the vars lookup to fetch the value of that variable, for example:
bamboo_agent_capabilities: "{{ lookup('vars', item.bamboo_agent_capabilities.dir) }}{{ item.bamboo_agent_capabilities.capability }}"
Given the playbook:
- hosts: localhost
gather_facts: no
vars:
bamboo_agents_it:
- bamboo_agent_capabilities: "{{ bamboo_agent_capabilities_it }}"
bamboo_agent_capabilities_it:
capability: /foo/bar
dir: bamboo_agent_install_dir
tasks:
- set_fact:
bamboo_agent_install_dir: /path/to/dir
- debug:
msg:
bamboo_agent_capabilities: "{{ lookup('vars', item.bamboo_agent_capabilities.dir) }}{{ item.bamboo_agent_capabilities.capability }}"
loop: "{{ bamboo_agents_it }}"
This yields:
PLAY [localhost] ***************************************************************************************************
TASK [set_fact] ****************************************************************************************************
ok: [localhost]
TASK [debug] *******************************************************************************************************
ok: [localhost] => (item={'bamboo_agent_capabilities': {'capability': '/foo/bar', 'dir': 'bamboo_agent_install_dir'}}) =>
msg:
bamboo_agent_capabilities: /path/to/dir/foo/bar
PLAY RECAP *********************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
When use json_query in Ansible, got the error along with my code as shown below.
The Python used in Ansible is 3.6.8, and used the same version installed jmespath: pip install jmespath, so, this should not be the issue. The Ansible code should be fine as well.
fatal: [localhost]: FAILED! => {"msg": "template error while templating string: no filter named 'json_query'. String: {{ jsondata | json_query(jmesquery) }}"
The following is the Ansible codes:
---
- name: ReadJsonfile
hosts: localhost
tasks:
- name: Display the JSON file content
shell: cat config.json
register: result
- name: save the JSON data to a Variable as a Fact
set_fact:
jsondata: "{{ result.stdout | from_json }}"
- name: setDomainName
set_fact:
domain_name: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'domain.name'
- name: setDomainUsername
set_fact:
domain_username: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'domain.user'
- name: setDomainPassword
set_fact:
domain_password: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'domain.password'
- name: setadmin_Listenport
set_fact:
admin_ListenPort: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'domain.admin.listenport'
- name: Debug the values
debug: msg=" Admin Listen Port => {{ admin_ListenPort }}, DomainName => {{ domain_name }}, DomainUserName => {{ domain_username }} , Domain Password => {{ domain_password }}"
From the error message, I suspect you are using ansible 2.10
The json_query filter is part of the community.general collection which needs to be installed separately starting from this version (this is clearly stated in the documentation)
ansible-galaxy collection install community.general
That being said:
your actual jmespath queries will not return what you expect
using shell to get a file content from remote is a bad practice
that's lots of set_facts and json_query to simply get data that is readily available
Since yaml is a strict superset of json, any valid json is also a valid yaml. In your case, you only have to load the content of the file and you have the data.
One solution is to fetch the file locally and then use include_vars.
In this particular case, using slurp looks like the best option.
This is what (I believe, since you did not provide an example) your config file looks like. I pushed that file in /tmp/config.json for my example.
{
"domain": {
"name": "toto",
"user": "doejohn",
"password": "sosecret",
"admin": {
"listenport": 5501
}
}
}
This is the demo playbook
---
- name: Load remote json content demo
hosts: localhost
gather_facts: false
vars:
config_file: /tmp/config.json
# Of course the below vars will fire errors
# if you call them before slurping data from remote.
# Note: loading `data` in a jinja expression on its own
# forces to transform it back rom string to json data.
# You can accomplish the same result in one go
# using the `from_json` filter
# i.e. => domain: "{{ (slurped_config.content | b64decode | from_json).domain }}"
data: "{{ slurped_config.content | b64decode }}"
domain: "{{ data.domain }}"
# Note2: the above will work and adapt to N hosts in your play: you will get
# the correct data for each host in every task.
tasks:
- name: Slurp config file content
slurp:
src: "{{ config_file }}"
register: slurped_config
- name: Show result
vars:
message: |-
name is: {{ domain.name }}
user is: {{ domain.user }}
password is: {{ domain.password }}
port is: {{ domain.admin.listenport }}
debug:
msg: "{{ message.split('\n') }}"
which gives:
PLAY [Load remote json content demo] ******************************************************
TASK [Slurp config file content] **********************************************************
Wednesday 21 April 2021 17:57:47 +0200 (0:00:00.010) 0:00:00.010 *******
ok: [localhost]
TASK [Show result] ************************************************************************
Wednesday 21 April 2021 17:57:48 +0200 (0:00:00.199) 0:00:00.209 *******
ok: [localhost] => {
"msg": [
"name is: toto",
"user is: doejohn",
"password is: sosecret",
"port is: 5501"
]
}
PLAY RECAP ********************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Wednesday 21 April 2021 17:57:48 +0200 (0:00:00.027) 0:00:00.237 *******
===============================================================================
Slurp config file content ---------------------------------------------------------- 0.20s
Show result ------------------------------------------------------------------------ 0.03s
I would like to modify vars for drupal configuration file settings.php with ansible and regex_replace module. My initial var is an IP xxx.xxx.xxx.xxx and I want this as my new var xxx\.xxx\.xxx\.xxx
My playbook:
---
- hosts: localhost
remote_user: cal
become: yes
become_user: cal
tasks:
- set_fact:
ip_front: "10.11.12.13"
ip_front_back: "{{ ip_front | regex_replace('\\.', '\\.') }}"
- name: "show all var"
debug:
msg:
- "{{ ip_front }}"
- "{{ ip_front_ok }}"
Output:
ok: [localhost] => {
"msg": [
"10.11.12.13",
"10\\.11\\.12\\.13"
]
}
How can I use pattern to resolve it?
I am following the redhat ansible course online and I followed the below steps to use multi-valued variable in a playbook. But I am hitting an error stating there's an undefined variable
Below is my arrays.yaml file
---
- name: show arrays
hosts: ansible1.example.com
vars_files:
- vars/users
tasks:
- name: print array values
debug:
msg: "User {{ item.username }} has homedirectory {{ item.homedir }} and shell {{ item.shell }}"
with_items: "{{ users }}"
And below is vars/users file
users:
linda:
username: linda
homedir: /home/linda
shell: /bin/bash
gokul:
username: gokul
homedir: /home/gokul
shell: /bin/bash
saha:
username: saha
homedir: /home/gokul/saha
shell: /bin/bash
And below is the error that I am hitting
ansible-playbook arrays.yaml
PLAY [show arrays] *****************************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] *************************************************************************************************************************************************************************************************************************************
ok: [ansible1.example.com]
TASK [print array values] **********************************************************************************************************************************************************************************************************************************
fatal: [ansible1.example.com]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'username'\n\nThe error appears to be in '/home/ansible/install/arrays.yaml': line 7, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n - name: print array values\n ^ here\n"}
PLAY RECAP *************************************************************************************************************************************************************************************************************************************************
ansible1.example.com : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
However, when I try to refer individual values like below in the playbook, it works fine.
---
- name: show arrays
hosts: ansible1.example.com
vars_files:
- vars/users
tasks:
- name: print array values
debug:
msg: "User {{ users.gokul.username }} has homedirectory {{ users.gokul.homedir }} and shell {{ users.gokul.shell }}"
It's not possible to iterate a dictionary. Either the code or the data should be changed.
1. Change data
Use a list of users instead of a dictionary. For example, the list below
users:
- username: linda
homedir: /home/linda
shell: /bin/bash
- username: gokul
homedir: /home/gokul
shell: /bin/bash
- username: saha
homedir: /home/gokul/saha
shell: /bin/bash
would work as expected
- name: print array values
debug:
msg: "User {{ item.username }}
has homedirectory {{ item.homedir }}
and shell {{ item.shell }}"
loop: "{{ users }}"
2. Change code
It's possible to use the users dictionary with the filter dict2items. For example, the task below would give the same result
- name: print array values
debug:
msg: "User {{ item.value.username }}
has homedirectory {{ item.value.homedir }}
and shell {{ item.value.shell }}"
loop: "{{ users|dict2items }}"
I've set up some application information in my Ansible group_vars like this:
applications:
- name: app1
- name: app2
- name: app3
- name: app4
settings:
log_dir: /var/logs/app4
associated_files:
- auth/key.json
- name: app5
settings:
log_dir: /var/logs/app5
repo_path: new_apps/app5
I'm struggling to get my head around how I can use these "sub loops".
My tasks for each application are:
Create some folders based purely on the name value
Create a log folder if a settings/log_dir value exists
Copy associated files over, if specified
The syntax for these tasks isn't the problem here, I'm comfortable with those - I just need to know how to access the information from this applications variable. Number 3 in particular seems troublesome to me - I need to loop within a loop.
To debug this, I've been trying to run the following task:
- debug:
msg: "{{ item }}"
with_subelements:
- "{{ applications }}"
- settings
Here's the output:
with_items: I get the error with_items expects a list or a set
with_nested: I can see the top level information (e.g. msg: {{ item }} outputs an array of app1, app2 etc)
with_subelements: I get the error subelements lookup expects a dictionary, got 'None'
It's possible/probable that the way I've set the variable up in the first instance is wrong. If there's a better way to do this, it's not a problem to change it.
You can't use with_subelements because settings is a dictionary, not a list. If you were to restructure your data so that settings is a list, like this:
applications:
- name: app1
- name: app2
- name: app3
- name: app4
settings:
- name: log_dir
value: /var/logs/app4
- name: associated_files
value:
- auth/key.json
- name: app5
settings:
- name: log_dir
value: /var/logs/app5
- name: repo_path
value: new_apps/app5
You could then write something like the following to iterate over each setting for each application:
---
- hosts: localhost
gather_facts: false
vars_files:
- applications.yml
tasks:
- debug:
msg: "set {{ item.1.name }} to {{ item.1.value }} for {{ item.0.name }}"
loop: "{{ applications|subelements('settings', skip_missing=true) }}"
loop_control:
label: "{{ item.0.name }}.{{ item.1.name }} = {{ item.1.value }}"
(I'm using loop_control here just to make the output nicer.)
Using the sample data you posted in applications.yml, this will produce as output:
PLAY [localhost] *********************************************************************
TASK [debug] *************************************************************************
ok: [localhost] => (item=app4.log_dir = /var/logs/app4) => {
"msg": "set log_dir to /var/logs/app4 for app4"
}
ok: [localhost] => (item=app4.associated_files = ['auth/key.json']) => {
"msg": "set associated_files to ['auth/key.json'] for app4"
}
ok: [localhost] => (item=app5.log_dir = /var/logs/app5) => {
"msg": "set log_dir to /var/logs/app5 for app5"
}
ok: [localhost] => (item=app5.repo_path = new_apps/app5) => {
"msg": "set repo_path to new_apps/app5 for app5"
}
PLAY RECAP ***************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0