I am trying to read a file with some key=value data. I am able to do it using file lookup feature of ansible. Now the problem is I want to exclude the comments from the input file.But not sure how to do it. How to negate '^#' (line starting with #). I tried using this command {{ input_parms.split("\n")|regex_search('^[^#]') }} but it did not worked as expected.
This is my input file
->cat mydata.cfg
#parms
name: 'foo'
place: 'bar'
id: 1
My playbook:
---
- hosts: localhost
vars:
parm_file: 'mydata.cfg'
input_parms: "{{ lookup('file', parm_file) }}"
tasks:
- debug: var=input_parms.split("\n")
Current output:
ok: [localhost] => {
"input_parms.split(\"\n\")": [
"#parms",
"name: 'foo'",
"place: 'bar'",
"id: 1"
]
}
My expected output, without the commented line(#parms):
ok: [localhost] => {
"input_parms.split(\"\n\")": [
"name: 'foo'",
"place: 'bar'",
"id: 1"
]
}
You can use the select and the match filter to select only lines that don't begin with #:
- debug:
var: input_parms.splitlines()|select('match', '^[^#]')|list
Which produces:
TASK [debug] *************************************************************************
ok: [localhost] => {
"input_parms.splitlines()|select('match', '^[^#]')|list": [
"name: 'foo'",
"place: 'bar'",
"id: 1"
]
}
Related
The idea is to add an item into the array if the item does not exist. The select attribute function returns an empty list even though there are items. Therefore the length of an empty is always 0, which in turn will create a new list every time in my case. So the union function in add a new comment section is returning a list with the only new comment erasing all the old ones.
I understand that the issue is with t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list|length == 0 conditional statement but I am not sure what I am doing wrong. I have tried many variations of this command. Few of them are commented. Any advice/suggestion is appreciated :)
main.yml:
- name: Get comment array from facts
set_fact:
common_motd_qsc_comments_array: "{{ ansible_local['snps']['motd']['comment_array'] }}"
register: t_array_exists
when:
- ansible_local['snps']['motd'] is defined
- ansible_local['snps']['motd']['comment_array'] is defined
- debug:
var: t_array_exists
- debug:
var: t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list
# var: ansible_facts|selectattr("common_motd_qsc_comments_array", "defined")|list
# var: t_array_exists|selectattr("common_motd_qsc_comments_array", "defined")|list
- name: Create an empty array if there is no array
set_fact:
common_motd_qsc_comments_array: []
when:
- t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list|length == 0
- debug:
var: t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list|length
- name: Deleting a comment if it exists
set_fact:
common_motd_qsc_comments_array: "{{ common_motd_qsc_comments_array | difference([t_new_entry]) }}"
loop: "{{ common_motd_qsc_delete_comment }}"
when: t_new_entry in common_motd_qsc_comments_array
vars:
t_new_entry: "{{ item | trim }}"
- name: Add a new comment if it doesn't exist
set_fact:
common_motd_qsc_comments_array: "{{ common_motd_qsc_comments_array | union([t_new_entry]) }}"
loop: "{{ common_motd_qsc_add_comment }}"
when: t_new_entry not in common_motd_qsc_comments_array
vars:
t_new_entry: "{{ item | trim }}"
- name: Saving comments to snps.fact file
ini_file:
dest: "/etc/ansible/facts.d/snps.fact"
section: 'motd' # header
option: 'comment_array' # key
value: "{{ common_motd_qsc_comments_array }}" # value
Ansible output:
TASK [common/motd_scratch/v1 : Get comment array from facts] ***************************************************************************************
ok: [ansible-poc-cos6]
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"t_array_exists": {
"ansible_facts": {
"common_motd_qsc_comments_array": [
"new comment 1"
]
},
"changed": false,
"failed": false
}
}
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"ansible_facts|selectattr(\"common_motd_qsc_comments_array\", \"defined\")|list": []
}
TASK [common/motd_scratch/v1 : Create an empty array if there is no array] *************************************************************************
ok: [ansible-poc-cos6]
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"t_array_exists[\"ansible_facts\"]|selectattr(\"common_motd_qsc_comments_array\", \"defined\")|list|length": "0"
}
TASK [common/motd_scratch/v1 : Add a new comment if it doesn't exist] ******************************************************************************
ok: [ansible-poc-cos6] => (item=new comment 22)
TASK [common/motd_scratch/v1 : Saving comments to snps.fact file] **********************************************************************************
changed: [ansible-poc-cos6]
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"common_motd_qsc_add_comment": [
"new comment 22"
]
}
snps.fact output:
Before:
[motd]
comment_array = ['new comment 1']
After passing new_comment as "new comment 2":
[motd]
comment_array = ['new comment 2']
Expected:
[motd]
comment_array = ['new comment 1', 'new comment 2']
You can do all of the above in one single easy step. Here is an MCVE playbook to illustrate.
Prior to running the playbook, I added the following file on my machine:
/etc/ansible/facts.d/snps.fact
{
"motd_example1": [
"One message",
"A message",
"Message to delete"
],
"motd_example2": [
"Some messages",
"Mandatory message",
"Other message"
]
}
The playbook
---
- name: Add/Remove list elements demo
hosts: localhost
gather_facts: true
vars:
# The (list of) custom message(s) I want to add if not present
my_custom_mandatory_messages:
- Mandatory message
# The (list of) custom message(s) I want to remove if present
my_custom_messages_to_delete:
- Message to delete
# The list of custom vars I'm going to loop over for demo
# In your case, you can simply use your single motd var
# directly in the below task without looping. I added this
# for convenience as the expression is exactly the same
# in all cases. Only the input data changes
my_motd_example_vars:
- motd_example1
- motd_example2
- motd_example3 # This one does not even exist in ansible_local.snps
tasks:
- name: Show result using local facts for each demo test var
debug:
msg: "{{ ansible_local.snps[item] | default([])
| union(my_custom_mandatory_messages)
| difference(my_custom_messages_to_delete) }}"
loop: "{{ my_motd_example_vars }}"
- name: Proove it works with a totaly undefined var
debug:
msg: "{{ totally.undefined.var | default([])
| union(my_custom_mandatory_messages)
| difference(my_custom_messages_to_delete) }}"
The result
TASK [Gathering Facts] **********************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Show result using local facts for each demo test var] *********************************************************************************************************************************************************************************
ok: [localhost] => (item=motd_example1) => {
"msg": [
"One message",
"A message",
"Mandatory message"
]
}
ok: [localhost] => (item=motd_example2) => {
"msg": [
"Some messages",
"Mandatory message",
"Other message"
]
}
ok: [localhost] => (item=motd_example3) => {
"msg": [
"Mandatory message"
]
}
TASK [Simple proof that it works with a totaly undefined var] *******************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"Mandatory message"
]
}
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Not sure I understood exactly what the problem is, but if you are trying to merge two dictionaries (with one overwriting the other if the key already exists), then you might want to take a look at:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_advanced_syntax.html#yaml-anchors-and-aliases-sharing-variable-values
The very first example shows exactly that, with just YAML. If that doesn't do the trick, then you might want to try the combine filter:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#combining-hashes-dictionaries
HTH
Alex
I was able to fix it with the help of my colleague. I just had to check if comment_array is not defined instead of trying to find its length. A simple alternate idea which works :)
- name: Create an empty array if there is no array
set_fact:
common_motd_qsc_comments_array: []
when:
- ansible_local['snps']['motd']['comment_array'] is not defined
Thank you all for your help :)
I want to run a task in ansible something similar to the following.
#Task in Playbook
- name : Include tasks
block:
- name: call example.yml
include_tasks: "example.yml"
vars:
my_var: item
with_items:
- [1, 2]
# example.yml
- name: Debug.
debug:
msg:
- "my_var: {{ my_var }}"
with_inventory_hostnames:
- 'all'
I expect the output to be printing the my_var as values 1 in the first iteration and 2 in second iteration of the loop in the playbook. But instead, it is printing the hostnames
# Output
TASK [proxysql : Debug.] ************************************************************************************************
[WARNING]: The loop variable 'item' is already in use. You should set the `loop_var` value in the `loop_control` option for the task to something else to avoid variable collisions and unexpected behavior.
ok: [10.1xx.xx.xx] => (item=None) => {
"msg": [
"my_var: 10.134.34.34"
]
}
ok: [10.1xx.xx.xx] => (item=None) => {
"msg": [
"my_var: 10.123.23.23"
]
}
ok: [10.1xx.xx.xx] => (item=None) => {
"msg": [
"my_var: 10.112.12.12"
]
}
TASK [proxysql : Debug.] ************************************************************************************************
[WARNING]: The loop variable 'item' is already in use. You should set the `loop_var` value in the `loop_control` option for the task to something else to avoid variable collisions and unexpected behavior.
ok: [10.1xx.xx.xx] => (item=None) => {
"msg": [
"my_var: 10.134.34.34"
]
}
ok: [10.1xx.xx.xx] => (item=None) => {
"msg": [
"my_var: 10.123.23.23"
]
}
ok: [10.1xx.xx.xx] => (item=None) => {
"msg": [
"my_var: 10.112.12.12"
]
}
Thanks in advance
There are two problems:
In the playbook, tasks are included in a loop that has the loop variable name item and the included task also has a loop and
the default variable name is again item. This is why the warning
messages and to solve that use loop_control.
my_var: item assignment needs to be in format my_var: "{{ item }}" for correct assignment.
After both the corrections, playbook would look like this.
- name : Include tasks
block:
- name: call example.yml
include_tasks: "example.yml"
vars:
my_var: "{{ outer_item }}"
with_items:
- [1, 2]
loop_control:
loop_var: outer_item
I am trying to get the "count" value from the dictionary
"{ \"_id\" : ObjectId(\"5d3a1643c43c898d01a3c740\"), \"count\" : 2 }"
present at the last element of the ansible stdout_lines.
TASK [version_update : debug] ******************************************************************************************************************************************
ok: [192.168.27.125] => {
"count_info.stdout": "MongoDB shell version v4.0.6\nconnecting to: mongodb://127.0.0.1:27017/configure-db?gssapiServiceName=mongodb\nImplicit session: session { \"id\" : UUID(\"4bfad3ba-981f-47de-86f9-a1fadbe28e12\") }\nMongoDB server version: 4.0.6\n{ \"_id\" : ObjectId(\"5d3a1643c43c898d01a3c740\"), \"count\" : 2 }"
}
TASK [version_update : debug] ******************************************************************************************************************************************
ok: [192.168.27.125] => {
"count_info.stdout_lines": [
"MongoDB shell version v4.0.6",
"connecting to: mongodb://127.0.0.1:27017/configure-db?gssapiServiceName=mongodb",
"Implicit session: session { \"id\" : UUID(\"4bfad3ba-981f-47de-86f9-a1fadbe28e12\") }",
"MongoDB server version: 4.0.6",
"{ \"_id\" : ObjectId(\"5d3a1643c43c898d01a3c740\"), \"count\" : 2 }"
]
}
I tried the following two ways but not successful.
- debug:
msg: "{{ (count_info.stdout_lines[-1] | from_json).count }}"
- name: count value
debug:
msg: "{{ count_info.stdout_lines[-1] | json_query('count') }}"
Error log:
TASK [version_update : debug] ******************************************************************************************************************************************
fatal: [192.168.27.125]: FAILED! => {"msg": "the field 'args' has an invalid value ({u'msg': u'{{ (count_info.stdout_lines[-1] | from_json).count }}'}), and could not be converted to an dict.The error was: No JSON object could be decoded\n\nThe error appears to have been in '/home/admin/playbook-3/roles/version_update/tasks/version_update.yml': line 73, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- debug:\n ^ here\n"}
to retry, use: --limit #/home/admin/playbook-3/version_update.retry
TASK [version_update : count value] ************************************************************************************************************************************
ok: [192.168.27.125] => {
"msg": ""
}
Your last line in the output is not a pure json string (probably bson from your MongoDB output). The error you get is actually coming from the filter itself not getting a correct input and failing.
You will have to translate that to pure json before you can use the from_json filter. The offending data is the ObjectId(\"5d3a1643c43c898d01a3c740\") that cannot be deserialized by the filter. This should be changed in the task/command you use to register your variable. You can have a look at this interesting question on the subject with many answers that will probably give you some clues.
Once this is done, accessing your data becomes easy as you had already figured out. Here is an example with a modified sample data (the way I think you should finally get it) just to confirm your where on the right track.
- name: Get count in json serialized string
hosts: localhost
gather_facts: false
vars:
"count_info":
"stdout_lines": [
"MongoDB shell version v4.0.6",
"connecting to: mongodb://127.0.0.1:27017/configure-db?gssapiServiceName=mongodb",
"Implicit session: session { \"id\" : UUID(\"4bfad3ba-981f-47de-86f9-a1fadbe28e12\") }",
"MongoDB server version: 4.0.6",
"{ \"_id\" : \"someDeserializedId\", \"count\" : 2 }"
]
tasks:
- name: Get count
debug:
msg: "{{ (count_info.stdout_lines[-1] | from_json).count }}"
And the result
PLAY [Get count in json serialized string] ********************************************************************************************************************************************************************************
TASK [Get count] **********************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "2"
}
although the element has the structure of a dictionary, its in fact a string, which i couldnt filter it with to_json or to_nice_json to convert it to a dictionary.
using the below sequence of tasks will get the value you want to pick up, unfortunately i didnt find a way to do it in one task. the logic is as follows:
get the last element from the list, and split the string into key-value substrings, separated by the ,.
parse this list and find the element that contains the keyword count (you can enhance it here if you think the count may appear in other lines too). then with regex, get the numerical value out of it.
PB:
---
- hosts: localhost
gather_facts: false
vars:
final_count_value: -1
count_info:
stdout_lines:
- MongoDB shell version v4.0.6
- 'connecting to: mongodb://127.0.0.1:27017/configure-db?gssapiServiceName=mongodb'
- 'Implicit session: session { "id" : UUID("4bfad3ba-981f-47de-86f9-a1fadbe28e12")
}'
- 'MongoDB server version: 4.0.6'
- '{ "_id" : ObjectId("5d3a1643c43c898d01a3c740"), "count" : 2 }'
tasks:
- name: prepare list var
set_fact:
temp_list: "{{ (count_info.stdout_lines | last).split(', ') | list }}"
- name: find count
set_fact:
final_count_value: "{{ item | regex_replace('\"count\" : ', '') | regex_replace(' }', '') }}"
when: item is search('count')
with_items:
- "{{ temp_list }}"
- name: print result
debug:
var: final_count_value
output:
PLAY [localhost] *******************************************************************************************************************************************************************************************************
TASK [prepare list var] ************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [find count] ******************************************************************************************************************************************************************************************************
skipping: [localhost] => (item={ "_id" : ObjectId("5d3a1643c43c898d01a3c740"))
ok: [localhost] => (item="count" : 2 })
TASK [print result] ****************************************************************************************************************************************************************************************************
ok: [localhost] => {
"final_count_value": "2"
}
UPDATE
to subtract 1 from the result, you should use:
- name: find count and subtract 1
set_fact:
final_count_value: "{{ item | regex_replace('\"count\" : ', '') | regex_replace(' }', '') | int - 1 }}"
when: item is search('count')
with_items:
- "{{ temp_list }}"
hope it helps!.
I made a UI that I use to configure our servers. I put the new IP's or servernames into a txt field and have ansbible playbooks ran on them depending on the type of server it is instead of using an inventory file. I'm trying to get ansible to add lines to a file for each value in the {{ips}} variable and not have them all on the same line.
I have tried a few different ways including lineinfile, blockinfile and replace seems to get me closer than the others, but I am still not able to get the results I want.
- name: Add new lines
replace:
path: /foo/bar
regexp: '^# Test Line'
replace: "# Test Line\n/foo/bar {{ ips }}"
This add one line with all the IP's in the ips variable.
# Test Line
/foo/bar test1,test2
What I am trying to get is.
# Test Line
/foo/bar test1
/foo/bar test2
It is hard to account for how many ip's will be in the variable at a time. Sometimes it's one sometimes it's 10.
In the following solution:
I'm using the python split inside a Jinja2 expression to create a list out of your string of comma separated ip strings. This will only work if the format stays the same.
I loop over that list with lineinfile, using the insertafter option
I made sure the solution handles dupes nicely if needed.
This is the demo playbook
---
- name: test for So
hosts: localhost
vars:
test_file: /tmp/test_file.txt
first_ips: 127.0.0.1,127.0.0.2
more_ips: 10.35.26.1,10.35.26.2
dupe_ips: 127.0.0.2,10.35.26.1
tasks:
- name: Make sure we start from scratch
shell: echo "I'm a line of text\n\n# Test Line\n\nThis is the end, my only friend" > {{ test_file }}
- name: Show file at start
debug:
msg: "{{ lookup('file', test_file).split('\n') }}"
- name: Add first ips
lineinfile:
path: "{{ test_file }}"
insertafter: "# Test Line"
line: "/foo/bar {{ item }}"
loop: "{{ first_ips.split(',') }}"
- name: Show file with first ips
debug:
msg: "{{ lookup('file', test_file).split('\n') }}"
- name: Add second list of ips
lineinfile:
path: "{{ test_file }}"
insertafter: "# Test Line"
line: "/foo/bar {{ item }}"
loop: "{{ more_ips.split(',') }}"
- name: Show file with more ips
debug:
msg: "{{ lookup('file', test_file).split('\n') }}"
- name: Test if dupes are handled correctly
lineinfile:
path: "{{ test_file }}"
insertafter: "# Test Line"
line: "/foo/bar {{ item }}"
loop: "{{ dupe_ips.split(',') }}"
- name: Show file that should not have changed
debug:
msg: "{{ lookup('file', test_file).split('\n') }}"
And here is the result. My debug task is showing the result file as a list of lines for display purpose when running the playbook. cat the file yourself if you want to see the result without the extra quotes and commas.
PLAY [test for So] *******************************************************************
TASK [Gathering Facts] ***************************************************************
ok: [localhost]
TASK [Make sure we start from scratch] ***********************************************
changed: [localhost]
TASK [Show file at start] ************************************************************
ok: [localhost] => {
"msg": [
"I'm a line of text",
"",
"# Test Line",
"",
"This is the end, my only friend"
]
}
TASK [Add first ips] *****************************************************************
changed: [localhost] => (item=/foo/bar 127.0.0.1)
changed: [localhost] => (item=/foo/bar 127.0.0.2)
TASK [Show file with first ips] ******************************************************
ok: [localhost] => {
"msg": [
"I'm a line of text",
"",
"# Test Line",
"/foo/bar 127.0.0.2",
"/foo/bar 127.0.0.1",
"",
"This is the end, my only friend"
]
}
TASK [Add second list of ips] ********************************************************
changed: [localhost] => (item=/foo/bar 10.35.26.1)
changed: [localhost] => (item=/foo/bar 10.35.26.2)
TASK [Show file with more ips] *******************************************************
ok: [localhost] => {
"msg": [
"I'm a line of text",
"",
"# Test Line",
"/foo/bar 10.35.26.2",
"/foo/bar 10.35.26.1",
"/foo/bar 127.0.0.2",
"/foo/bar 127.0.0.1",
"",
"This is the end, my only friend"
]
}
TASK [Test if dupes are handled correctly] *******************************************
ok: [localhost] => (item=/foo/bar 127.0.0.2)
ok: [localhost] => (item=/foo/bar 10.35.26.1)
TASK [Show file that should not have changed] ****************************************
ok: [localhost] => {
"msg": [
"I'm a line of text",
"",
"# Test Line",
"/foo/bar 10.35.26.2",
"/foo/bar 10.35.26.1",
"/foo/bar 127.0.0.2",
"/foo/bar 127.0.0.1",
"",
"This is the end, my only friend"
]
}
PLAY RECAP ***************************************************************************
localhost : ok=9 changed=3 unreachable=0 failed=0
Is there any way in Ansible to set a variable to some structured value (e.g. a list or a dictionary) that is constructed from an expression?
For example, I have these variables:
all_flavours: ["vanilla", "chocolate", "strawberry", "banana"]
flavours_to_keep: ["chocolate", "banana"]
I want to evaluate the expression all_flavours | difference(flavours_to_keep) and store the resulting structured collection (list or indeed set) in the variable flavours_to_drop, so that for these values it's as if I had defined:
flavours_to_drop: ["vanilla", "strawberry"]
However, I can't find a syntax to do this. Here are my attempts:
Attempt 1
flavours_to_drop: all_flavours | difference(flavours_to_keep)
This is simply interpreted as a string:
flavours_to_drop: "all_flavours | difference(flavours_to_keep)"
Attempt 2
flavours_to_drop: {{ all_flavours | difference(flavours_to_keep) }}
This is an error because YAML interprets the "{" to mean a dictionary.
Attempt 3
flavours_to_drop: "{{ all_flavours | difference(flavours_to_keep) }}"
This does the set difference correctly, but then converts it to a string, so I end up with:
flavours_to_drop: "set([\"vanilla\", \"strawberry\"])"
How can I evaluate the expression, but store the structured object straight back into a variable instead of converting it into a string?
I should add that I'm using Ansible version 1.6.2. Is this behaviour that has changed between Ansible versions?
The formatting, etc. can be a little tricky. You need to quote the filter in order for it to be parsed properly. You can do it either as a var or in set_fact. You want something like this:
vars:
all_flavours: ["vanilla", "chocolate", "strawberry", "banana"]
flavours_to_keep: ["chocolate", "banana"]
flavours_to_drop_1: "{{ all_flavours | difference(flavours_to_keep) }}"
tasks:
- debug: var=all_flavours
- debug: var=flavours_to_keep
- set_fact:
flavours_to_drop_2: "{{ all_flavours | difference(flavours_to_keep) }}"
- debug: var=flavours_to_drop_1
- debug: var=flavours_to_drop_2
The result of the above is:
TASK: [debug var=all_flavours] ************************************************
ok: [localhost] => {
"all_flavours": [
"vanilla",
"chocolate",
"strawberry",
"banana"
]
}
TASK: [debug var=flavours_to_keep] ********************************************
ok: [localhost] => {
"flavours_to_keep": [
"chocolate",
"banana"
]
}
TASK: [set_fact ] *************************************************************
ok: [localhost]
TASK: [debug var=flavours_to_drop_1] ********************************************
ok: [localhost] => {
"flavours_to_drop_1": [
"vanilla",
"strawberry"
]
}
TASK: [debug var=flavours_to_drop_2] ********************************************
ok: [localhost] => {
"flavours_to_drop_2": [
"vanilla",
"strawberry"
]
}