I've got this yaml object
users:
user1: some description
user2: another desc
user3: ...
How can I get this object's keys? Result should be [user1, user2, user3]
EDIT: I'm using yaml variables in ansible
Array:
{{ users.keys() }}
Task:
- name: with_dict
debug:
msg: "{{ item.key }}: {{ item.value }}"
with_dict: "{{ users }}"
Template:
{% for key, value in users.iteritems() %}
{{ key|e }}: {{ value|e }}
{% endfor %}
Ansible uses Jinja, so you can do:
[ {{ obj.keys() | join(", ") }} ]
Related
I want to generate a list of dictionaries through the loop. I came across that we can use with_sequence to generate the integer sequences in ansible. I am getting the following error:
The offending line appears to be:
- { "dir": "{{ mat_cleaner_input_file_path_flow }}/{{ item }}" }
with_sequence: start=0 end={{ http_range }}
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
My config.j2:
{{ cleaner_config | to_nice_yaml(indent=2) }}
My task.yml:
---
- name: Test dictionaries playbook
hosts: localhost
connection: local
vars:
mat_cleaner_input_file_path_flow: "/var/opt/miq/sftp/edr-flow"
mat_cleaner_input_file_path_http: "/var/opt/miq/sftp/edr-http"
mat_cleaner_input_retention_period: 21600
http_range: 5
cleaner_config:
- { "dir": "{{ mat_cleaner_input_file_path_flow }}/{{ item }}" }
with_sequence: start=0 end={{ http_range }}
tasks:
- name: Set Cleaner Config
template:
src: /home/osboxes/CleanerConfig/cleaner.config.j2
dest: /home/osboxes/CleanerConfig/output
delegate_to: localhost
with_items: cleaner_config
I want the final output file to be:
dir: /var/opt/miq/sftp/edr-flow/0
dir: /var/opt/miq/sftp/edr-flow/1
dir: /var/opt/miq/sftp/edr-flow/2
The format of the output file you want is text. It's not a list. You can create it by Jinja, e.g.
mat_cleaner_input_file_path_flow: "/var/opt/miq/sftp/edr-flow"
http_range: 3
cleaner_config: |
{% for i in range(http_range) %}
{{ mat_cleaner_input_file_path_flow }}/{{ i }}
{% endfor %}
gives
cleaner_config: |-
/var/opt/miq/sftp/edr-flow/0
/var/opt/miq/sftp/edr-flow/1
/var/opt/miq/sftp/edr-flow/2
Then, because it's not a list, you don't have to use to_nice_yaml in the template. Use indent instead if you want to indent the lines, e.g.
shell> cat cleaner.config.j2
{{ cleaner_config|indent(2) }}
- template:
src: cleaner.config.j2
dest: output
gives
shell> cat output
/var/opt/miq/sftp/edr-flow/0
/var/opt/miq/sftp/edr-flow/1
/var/opt/miq/sftp/edr-flow/2
Under normal circumstances, you would simplify the code, omit the cleaner_config variable, and put the Jinja iteration into the template, e.g.
mat_cleaner_input_file_path_flow: "/var/opt/miq/sftp/edr-flow"
http_range: 3
shell> cat cleaner.config.j2
{% for i in range(http_range) %}
{{ mat_cleaner_input_file_path_flow }}/{{ i }}
{% endfor %}
If you want to use a list
mat_cleaner_input_file_path_flow: "/var/opt/miq/sftp/edr-flow"
http_range: 3
cleaner_config_text: |
{% for i in range(http_range) %}
- {{ mat_cleaner_input_file_path_flow }}/{{ i }}
{% endfor %}
cleaner_config: "{{ cleaner_config_text|from_yaml }}
gives
cleaner_config:
- /var/opt/miq/sftp/edr-flow/0
- /var/opt/miq/sftp/edr-flow/1
- /var/opt/miq/sftp/edr-flow/2
Now you need to_nice_yaml to format the list, e.g.
shell> cat cleaner.config.j2
{{ cleaner_config|to_nice_yaml|indent(2) }}
gives
shell> cat output
- /var/opt/miq/sftp/edr-flow/0
- /var/opt/miq/sftp/edr-flow/1
- /var/opt/miq/sftp/edr-flow/2
How to loop over dict and filetree? I want to recursively template files with .j2 suffix (key) to destination location (value), also basename should be renamed (remove .j2 suffix). Its a perfect use case. Unfortunatelly ansible is not good with complex data structures.
Input:
vars:
applications:
application1:
svcpaths:
localfolder/bardir1: remotefolder/bardir1
localfolder/bardir2: remotefolder/bardir2
localfolder/bardir3: remotefolder/bardir3
application2:
svcpaths:
localfolder/bardir5: remotefolder/bardir5
localfolder/bardir6: remotefolder/bardir6
My try:
- name: Files to template
template:
src: "{{ item.src }}"
dest: "{{ item.destination }}/{{ item.name | regex_replace('.j2','') }}"
loop: |
[
{% for c in applications %}
{% if applications[c]['svcpaths'] is defined and applications[c]['svcpaths'] |list|length >0 %}
{% for o,m in applications[c]['svcpaths'].items() %}
{% for i in lookup('filetree', o ) %}
{% if i.state == 'file' and i.path | regex_search('.\.j2') %}
{
"name": "{{ i.path }}",
"src": "{{ i.src }}",
"destination": "{{ m }}"
},
{% endif %}
{% endfor %}
{% endfor %}
{% endif %}
{% endfor %}
]
I know that using jinja in plays is not good and I want to avoid it, if its possible. Also input datastructure should not be changed.
Thannks
If I understand what you're trying to do, I think there is a reasonably simple solution. If you write a task file like this called template_files.yml:
---
- name: render templates to dest_dir
loop: "{{ query('filetree', src_dir) }}"
# we need this to avoid conflicts with the "item" variable in
# the calling playbook.
loop_control:
loop_var: template
when: template.src.endswith('.j2')
template:
src: "{{ template.src }}"
dest: "{{ dest_dir }}/{{ (template.src|basename)[:-3] }}"
Then you can write a playbook like this:
---
- hosts: localhost
gather_facts: false
vars:
applications:
application1:
svcpaths:
localfolder/bardir1: /tmp/remotefolder/bardir1
localfolder/bardir2: /tmp/remotefolder/bardir2
localfolder/bardir3: /tmp/remotefolder/bardir3
application2:
svcpaths:
localfolder/bardir5: /tmp/remotefolder/bardir5
localfolder/bardir6: /tmp/remotefolder/bardir6
tasks:
# generate a list of {key: src, value: destination}
# dictionaries from your data structure.
- set_fact:
templates: "{{ templates|default([]) + item|dict2items }}"
loop: "{{ applications|json_query('*.svcpaths')}}"
# show what the generated variable looks like
- debug:
var: templates
# template all the things
- include_tasks: template_files.yml
loop: "{{ templates }}"
vars:
src_dir: "{{ item.key }}"
dest_dir: "{{ item.value }}"
Given that I have a set of local files that look like this:
localfolder/bardir1/example.txt.j2
localfolder/bardir2/example.txt.j2
localfolder/bardir3/example.txt.j2
localfolder/bardir5/example.txt.j2
localfolder/bardir6/example.txt.j2
Running the playbook results in:
/tmp/remotefolder/bardir6/example.txt
/tmp/remotefolder/bardir5/example.txt
/tmp/remotefolder/bardir3/example.txt
/tmp/remotefolder/bardir2/example.txt
/tmp/remotefolder/bardir1/example.txt
I think that's probably easier to read and understand than the Jinja-template based solution you're using.
I have a json structure like this:
{
"group_a": {
"setting_one": "text one",
"setting_two": "text two"
},
"group_b": {
"setting_three": "text three",
"setting_four": "text four"
}
}
And I need to do iterate over the dicts inside the dicts and create tasks, kinda like this:
- name: Task group_a setting_one
...
- name: Task group_a setting_two
...
- name: Task group_b setting_three
...
- name: Task group_b setting_four
...
I was hoping to make something like this:
- name: Insert {{ group }} {{ setting }}
ini_file:
path: settings.ini
section: "{{ group }}"
option: "{{ setting }}"
value: "{{ setting_value }}"
with_nested:
- "{{ data.keys() }}" # Can I get this key and use in the next loop?
- "{{ data.key.keys() }}"
How can I do this? I have been messing about with loop, with_items, with_dict etc, but I can't manage to get it exactly the way I need. I also tried with_nested, but that does not seem to do what I need either.
Kind regards, Jonas
Let's reduce the structure to one with the same piece of information
my_data:
group_a:
- one
- two
group_b:
- three
- four
Then the play below
- debug:
msg: "{{ item.0.key }} setting_{{ item.1 }}"
with_subelements:
- "{{ my_data|dict2items }}"
- value
gives:
"msg": "group_b setting_three"
"msg": "group_b setting_four"
"msg": "group_a setting_one"
"msg": "group_a setting_two"
It's not straightforward to use the original data. The play below separates the keys and values
tasks:
- set_fact:
my_groups: "{{ my_data.keys()}}"
- debug:
msg: "{{ item }} {{ my_data|dict2items|json_query(my_query)|to_yaml }}"
loop: "{{ my_groups }}"
vars:
my_query: "[?key=='{{ item }}'].value"
and gives:
"msg": "group_b - - {setting_three: text three}\n - {setting_four: text four}\n"
"msg": "group_a - - {setting_one: text one}\n - {setting_two: text two}\n"
Multi-step loops are needed to proceed. For example the play below
- set_fact:
my_data1: |
{% for group, values in my_data.items() %}
{% for value in values %}
{% for item_key, item_val in value.items() %}
Task {{ group }} {{ item_key }},
{% endfor %}{% endfor %}{% endfor %}
- debug:
msg: "{{ my_data1.split('\n') }}"
gives:
"msg": [
"Task group_b setting_three,",
"Task group_b setting_four,",
"Task group_a setting_one,",
"Task group_a setting_two,",
""
]
I want to write an Ansible role to be able to alter a given Kafka topic. I am using a dictionary of key/value pairs.
The command module is then used to execute a Kafka script that take a string of comma separated values. For instance, use app_kafka_topic list:
---
app_kafka_topic:
cleanup.policy :
- "delete"
retention.ms :
- "146800000"
partitions :
- "6"
replication-factor :
- "2"
and create the string:
"cleanup.policy=delete,retention.ms=146800000,partitions=6,replication-factor=2"
This is what I have so far.
- name: Reading the Default Topic Properties
set_fact:
app_kafka_topic_properties_dicts: |
{% set res = [] -%}
{% for key in app_kafka_topic.keys() -%}
{% for value in app_kafka_topic[key] -%}
{% set ignored = res.extend([{'topic_property': key, 'value':value}]) -%}
{%- endfor %}
{%- endfor %}
{{ res }}
- name: Create Topic with Default Properties
command: "{{ kafka_bin_dir }}/{{ kafka_config_script }}
--zookeeper {{ prefix }}-kafka-{{ Kafka_node }}.{{ DNSDomain}}:{{ zookeeper_port }}
--entity-type topics
--alter
--entity-name {{ kafka_topic }}
--add-config
{{ properties }}"
with_items: "{{ app_kafka_topic_properties_dicts }}"
register: createdTopic
vars:
properties: |-
{% for key in app_kafka_topic.keys() %}
{% for value in app_kafka_topic[key] %}
"{{ key }}={{ value }}"
{%- endfor %}
{%- endfor %}
However, the properties variable is not concatenating the values to the end of a string. Is there a way to append the values to a string and separate them by a comma?
Is this the code that you're looking for?
play.yml
- hosts: localhost
gather_facts: no
vars:
string: ""
app_kafka_topic:
cleanup.policy :
- "delete"
retention_ms :
- "146800000"
partitions :
- "6"
replication_factor :
- "2"
tasks:
- set_fact:
string: "{{ string }}{{ (index > 0)|ternary(',','') }}{{ item.key }}={{ item.value[0] }}"
loop: "{{ app_kafka_topic|dict2items }}"
loop_control:
index_var: index
- debug:
var: string
$ ansible-playbook play.yml | grep string
"string": "retention_ms=146800000,cleanup.policy=delete,replication_factor=2,partitions=6"
My playbook contains vars that are passed to a role. When I run it, I get [DEPRECATION WARNING]: Skipping task due to undefined Error, in the future this will be a fatal error..
Here's what I have:
---
- hosts: hadoopL0X
become: yes
become_method: sudo
vars:
logrotate_scripts:
- name: "{{ item }}"
with_items:
- zookeeper
- sa
path: "/var/log{{ item }}/{{ item }}.log "
options:
- daily
- rotate 3
- missingok
- compress
- notifempty
roles:
- log-rotation
...
The role is as such:
log-rotation/tasks/main.yml
---
- name: Setup logrotate.d scripts
template:
src: logrotate.d.j2
dest: "{{ logrotate_conf_dir }}{{ item }}"
with_items: "{{ logrotate_scripts }}"
...
log-rotation/defaults/main.yml
---
logrotate_conf_dir: "/etc/logrotate.d/"
logrotate_scripts: []
...
log-rotation/templates/logrotate.d.j2
# {{ ansible_managed }}
"{{ item.path }}" {
{% if item.options is defined -%}
{% for option in item.options -%}
{{ option }}
{% endfor -%}
{% endif %}
{%- if item.scripts is defined -%}
{%- for name, script in item.scripts.iteritems() -%}
{{ name }}
{{ script }}
endscript
{% endfor -%}
{% endif -%}
}
Any help would be much appreciated!
with_items can only be used with tasks, it cannot be used when defining variables, and because of that item isn't defined. It also looks like that the service variable isn't defined as well.