Receiving Error when I read file line by line in ansible - ansible

The goal is to read the input file for a string that matches the regex expression then to match the expression to my config file.. when there is a match ansible is to replace the existing line in the config file with the matching line in the delta file.
I was able to perform this task but noticed that ansible would read one line and essentially be done. I added the .splitlines() option to my code so that it would read line by line and perform the same action but i received the following error:
- name: Search for multiple reg expressions and replace in config file
vars:
# pull data from config file
input: "{{ lookup('file', '{{ record }}').splitlines() }}"
delta: "{{ input | regex_search('^.*[a-zA-Z]+.*_.*[a-zA-Z]+.*?', multiline=True )}}"
delta1: "{{ input | regex_search('^.*[a-zA-Z]+.*_.*[a-zA-Z]+.*', multiline=True)}}"
record: "/etc/ansible/roles/file_config/files/records/records.config.{{ inventory_hostname }}"
lineinfile:
path: /dir/dir/WCRUcachedir/records.config
# Line to search/Match against
regexp: "{{item.From}}"
# Line to replace with
line: "{{item.To}}"
state: present
backup: yes
with_items:
- { From: '{{delta}}', To: '{{delta1}}' }
This happened to be my end result
"msg": "An unhandled exception occurred while templating '{{ input | regex_search('^.*[a-zA-Z]+.*_.*[a-zA-Z]+.*', multiline=True )}}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Unexpected templating type error occurred on ({{ input | regex_search('^.*[a-zA-Z]+.*_.*[a-zA-Z]+.*', multiline=True )}}): expected string or buffer"
}
these are what i believe my conflicting lines are
input: "{{ lookup('file', '{{ record }}').splitlines() }}"
AND
delta1: "{{ input | regex_search('^.[a-zA-Z]+._.[a-zA-Z]+.', multiline=True)}}"

OK, you do have another problem.
In the vars section, when you're setting delta and delta1, regex_search is expecting a string, but your passing a list (which splitlines() created). But you need it to work on one line at a time.
So, rather that input, use item, which will be set in the loop:
vars:
input: "{{ lookup('file', '{{ record }}').splitlines() }}"
delta: "{{ item | regex_search('^.*[a-zA-Z]+.*_.*[a-zA-Z]+.*?')}}"
delta1: "{{ item | regex_search('^.*[a-zA-Z]+.*_.*[a-zA-Z]+.*')}}"
Obviously, you don't need the multiline=True any more.
Now, the loop will look like this:
lineinfile:
path: /etc/opt/CCURcache/records.config
regexp: "{{ delta }}"
line: "{{ delta1 }}"
state: present
backup: yes
loop: "{{ input }}"
when: delta != ""

Yes, you only have one item to loop over. That item has two elements, From and To.
From is {{ input | regex_search('^.*[a-zA-Z]+.*_.*[a-zA-Z]+.*?', multiline=True )}}
To is {{ input | regex_search('^.*[a-zA-Z]+.*_.*[a-zA-Z]+.*', multiline=True)}}
All you did in the vars section was define strings, which will get executed later, when the regex and line are used in the module.
Now, assuming you need to put these together, you need to zip them:
lineinfile:
path: /etc/opt/CCURcache/records.config
regexp: "{{item.0}}"
line: "{{item.1}}"
state: present
backup: yes
loop: "{{ delta|zip(delta1)|list }}"
Here's a simple example:
---
- hosts: localhost
connection: local
gather_facts: no
vars:
list1:
- key_1
- key_2
- key_n
list2:
- value_1
- value_2
- value_n
tasks:
- name: Loop over lists
debug:
msg: "key is {{ item.0 }}; value is {{ item.1 }}"
loop: "{{ list1|zip(list2)|list }}"
And the results:
PLAY [localhost] *******************************************************************************
TASK [Loop over lists] *************************************************************************
ok: [localhost] => (item=['key_1', 'value_1']) => {
"msg": "key is key_1; value is value_1"
}
ok: [localhost] => (item=['key_2', 'value_2']) => {
"msg": "key is key_2; value is value_2"
}
ok: [localhost] => (item=['key_n', 'value_n']) => {
"msg": "key is key_n; value is value_n"
}
PLAY RECAP *************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Good luck!

Related

Ansible conditionals with with_items/nested

Given the following tasks:
- name: Gather security group info
amazon.aws.ec2_group_info:
filters:
"tag:vpn_ports": "*"
register: sec_group_info_output
- name: Extract security groups and ports
set_fact:
vpn_groups: "{{ vpn_groups + [{ 'group_id': item.group_id, 'ports': item.tags.vpn_ports.split(',') }] }}"
with_items:
- "{{ sec_group_info_output | json_query('security_groups') }}"
vars:
vpn_groups: []
when: sec_group_info_output != []
- name: Generate list with CIDRs
set_fact:
vpn_rules: "{{ vpn_rules + [{ 'group_id': item.0.group_id , 'port': item.1, 'cidr': item.2 }] }}"
with_nested:
- "{{ vpn_groups|subelements('ports') }}"
- "{{ cidr_ranges }}"
vars:
vpn_rules: []
when: sec_group_info_output != []
I am trying to skip the last two tasks if the first task returns an empty set.
My understanding is that the when conditional is evaluated for every loop, and not just for the task as a whole.
The below therefor gives me:
TASK [security_groups : Gather security group info] ****************************
ok: [localhost]
TASK [security_groups : Extract security groups and ports] *********************
TASK [security_groups : Generate list with CIDRs] ******************************
fatal: [localhost]: FAILED! => {"msg": "obj must be a list of dicts or a nested dict"}
PLAY RECAP *********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=1 skipped=1 rescued=0 ignored=0
🚨 Error: The command exited with status 2
How would I go about fixing this error? I've tried putting |default([]) into my nested_items like below:
- name: Generate list with CIDRs
set_fact:
vpn_rules: "{{ vpn_rules + [{ 'group_id': item.0.group_id , 'port': item.1, 'cidr': item.2 }] |default([])}}"
with_nested:
- "{{ vpn_groups|subelements('ports') |default([])}}"
- "{{ cidr_ranges |default([])}}"
vars:
vpn_rules: []
when: sec_group_info_output != []
The error remains the same.
I've also tried putting both tasks in a block, but this had no effect and the error remains the same.
How would I be able to skip these tasks based on my condition?
Firstly, your condition is completely wrong. sec_group_info_output is a registered variable, so it can never be equal to an empty list. It will always be a dictionary containing information about the task execution. In order to have a chance of working as intended, it would need to be:
when: sec_group_info_output.security_groups != []
# more idiomatically, an empty list is false so you can just treat the value as a boolean
when: sec_group_info_output.security_groups
# or you can check the length
when: sec_group_info_output['security_groups'] | length > 0
However, in this case you don't need conditions at all. You're looping over the same list you're checking, and an empty loop will not execute any tasks. You just need a | default([]) in the loop definition on the third task in case the second didn't execute, and everything's fine.
- name: Gather security group info
amazon.aws.ec2_group_info:
filters:
"tag:vpn_ports": "*"
register: sec_group_info_output
- name: Extract security groups and ports
set_fact:
vpn_groups: "{{ vpn_groups | default([]) + [{ 'group_id': item.group_id, 'ports': item.tags.vpn_ports.split(',') }] }}"
loop: "{{ sec_group_info_output.security_groups }}"
- name: Generate list with CIDRs
set_fact:
vpn_rules: "{{ vpn_rules | default([]) + [{ 'group_id': item.0.0.group_id , 'port': item.0.1, 'cidr': item.1 }] }}"
loop: "{{ vpn_groups | default([]) | subelements('ports') | product(cidr_ranges) }}"
{{ vpn_groups | subelements('ports') | default([]) }} was headed in the right direction, but you put the default() in the wrong place. It needs to be before the subelements() filter so that that filter receives an empty list, not an undefined variable.

Errors when use "json_query" in Ansible

When use json_query in Ansible, got the error along with my code as shown below.
The Python used in Ansible is 3.6.8, and used the same version installed jmespath: pip install jmespath, so, this should not be the issue. The Ansible code should be fine as well.
fatal: [localhost]: FAILED! => {"msg": "template error while templating string: no filter named 'json_query'. String: {{ jsondata | json_query(jmesquery) }}"
The following is the Ansible codes:
---
- name: ReadJsonfile
hosts: localhost
tasks:
- name: Display the JSON file content
shell: cat config.json
register: result
- name: save the JSON data to a Variable as a Fact
set_fact:
jsondata: "{{ result.stdout | from_json }}"
- name: setDomainName
set_fact:
domain_name: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'domain.name'
- name: setDomainUsername
set_fact:
domain_username: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'domain.user'
- name: setDomainPassword
set_fact:
domain_password: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'domain.password'
- name: setadmin_Listenport
set_fact:
admin_ListenPort: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'domain.admin.listenport'
- name: Debug the values
debug: msg=" Admin Listen Port => {{ admin_ListenPort }}, DomainName => {{ domain_name }}, DomainUserName => {{ domain_username }} , Domain Password => {{ domain_password }}"
From the error message, I suspect you are using ansible 2.10
The json_query filter is part of the community.general collection which needs to be installed separately starting from this version (this is clearly stated in the documentation)
ansible-galaxy collection install community.general
That being said:
your actual jmespath queries will not return what you expect
using shell to get a file content from remote is a bad practice
that's lots of set_facts and json_query to simply get data that is readily available
Since yaml is a strict superset of json, any valid json is also a valid yaml. In your case, you only have to load the content of the file and you have the data.
One solution is to fetch the file locally and then use include_vars.
In this particular case, using slurp looks like the best option.
This is what (I believe, since you did not provide an example) your config file looks like. I pushed that file in /tmp/config.json for my example.
{
"domain": {
"name": "toto",
"user": "doejohn",
"password": "sosecret",
"admin": {
"listenport": 5501
}
}
}
This is the demo playbook
---
- name: Load remote json content demo
hosts: localhost
gather_facts: false
vars:
config_file: /tmp/config.json
# Of course the below vars will fire errors
# if you call them before slurping data from remote.
# Note: loading `data` in a jinja expression on its own
# forces to transform it back rom string to json data.
# You can accomplish the same result in one go
# using the `from_json` filter
# i.e. => domain: "{{ (slurped_config.content | b64decode | from_json).domain }}"
data: "{{ slurped_config.content | b64decode }}"
domain: "{{ data.domain }}"
# Note2: the above will work and adapt to N hosts in your play: you will get
# the correct data for each host in every task.
tasks:
- name: Slurp config file content
slurp:
src: "{{ config_file }}"
register: slurped_config
- name: Show result
vars:
message: |-
name is: {{ domain.name }}
user is: {{ domain.user }}
password is: {{ domain.password }}
port is: {{ domain.admin.listenport }}
debug:
msg: "{{ message.split('\n') }}"
which gives:
PLAY [Load remote json content demo] ******************************************************
TASK [Slurp config file content] **********************************************************
Wednesday 21 April 2021 17:57:47 +0200 (0:00:00.010) 0:00:00.010 *******
ok: [localhost]
TASK [Show result] ************************************************************************
Wednesday 21 April 2021 17:57:48 +0200 (0:00:00.199) 0:00:00.209 *******
ok: [localhost] => {
"msg": [
"name is: toto",
"user is: doejohn",
"password is: sosecret",
"port is: 5501"
]
}
PLAY RECAP ********************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Wednesday 21 April 2021 17:57:48 +0200 (0:00:00.027) 0:00:00.237 *******
===============================================================================
Slurp config file content ---------------------------------------------------------- 0.20s
Show result ------------------------------------------------------------------------ 0.03s

Issue passing variable to include_task file having add_host in Ansible

I am dealing with nested loops inorder to build dynamic host using add_host.
Outer Loop: with_items: "{{ command_result.stdout_lines }}" // gets me the list of users
Inner Loop: with_items: "{{ dest_ip.split(',') }}" // gets me the list of IP addresses seperated by comma (,)
The below playbook works fine.
- name: Add hosts
include_tasks: "{{ playbook_dir }}/gethosts.yml"
vars:
dest_ip: "{{ item.split('\t')[0] }}"
file_dets: "{{ item.split('\t')[1] }}"
USER_pass: "anystring"
# USER_pass: "{% if item.split('\t')[3] == 'FrontEnd' %}user1{% elif item.split('\t')[3] == 'BackEnd' %}user2{% else %}{% endif %}"
ansible_host: localhost
install_dir_pass: "anystring"
# install_dir_pass: "{{ item.split('\t')[2] }}"
with_items: "{{ command_result.stdout_lines }}"
Below is my include_task gethost.yml file:
---
- name: Generate JSON data
add_host:
name: "{{ item }}"
groups: dest_nodes
ansible_user: "{{ USER_pass }}"
install_dir: "{{ install_dir_pass }}"
with_items: "{{ dest_ip.split(',') }}"
I get the below error if I uncomment either USER_pass or install_dir_pass and comment the existing value:
TASK [Generate JSON data]
*********************************************************************************************************************************** task path: /app/deployment/gethosts.yml:2 [WARNING]: The loop
variable 'item' is already in use. You should set the loop_var value
in the loop_control option for the task to something else to avoid
variable collisions and unexpected behavior.
fatal: [localhost]: FAILED! => {
"msg": "The task includes an option with an undefined variable. The error was: list object has no element 2\n\nThe error appears to be
in '/app/deployment/gethosts.yml': line 2, column 4, but may\nbe
elsewhere in the file depending on the exact syntax problem.\n\nThe
offending line appears to be:\n\n---\n - name: Generate JSON data\n
^ here\n" }
NO MORE HOSTS LEFT
PLAY RECAP
************************************************************************************************************************************************** localhost : ok=12 changed=1 unreachable=0
failed=1 skipped=0 rescued=0 ignored=0
Requesting a solution to this issue and an explanation of a few questions I have.
The dest_ip is read and works fine with .split(,) method inside of include_task get_hosts.yml file when other variables like install_dir_pass dont seem to work.
When USER_pass and install_dir_pass are given simple string "AnyString" it works and is read fine inside of get_hosts.yml where as if they are assigned values using item.split('\t')[] the playbook errors as above.
I have already tested using debug that all the entries in command_result are good and the values should be populated correctly as below.
- debug:
msg: "{{ item.split('\t')[0] }}"
# msg: "{{ item.split('\t')[1] }}"
#msg: "{{ item.split('\t')[2] }}"
# msg: "{{ item.split('\t')[3] }}"
with_items: "{{ command_result.stdout_lines }}"
As this is very well explained in your error message:
There is a name collision between the item of each nested loops.
You have to set the loop_var name in the loop_control option to resolve the conflict in either of the loops.
I suggest you do this on the include_task so that it solves the problem for any other loop inside your included file. As I have no idea what your command_result.stdout_lines contains exactly, I used in the below example the var name my_result that you should change with something fitting your situation. Note that this var will be available directly in the included file if necessary.
- name: Add hosts
include_tasks: "{{ playbook_dir }}/gethosts.yml"
vars:
dest_ip: "{{ my_result.split('\t')[0] }}"
file_dets: "{{ my_result.split('\t')[1] }}"
USER_pass: "{% if my_result.split('\t')[3] == 'FrontEnd' %}user1{% elif my_result.split('\t')[3] == 'BackEnd' %}user2{% else %}{% endif %}"
ansible_host: localhost
install_dir_pass: "{{ my_result.split('\t')[2] }}"
with_items: "{{ command_result.stdout_lines }}"
loop_control:
loop_var: my_result

Splitting a debug msg to extract a certain part of the msg in ansible

So I need to take a particular part of my debug msg that I have in my playbook that goes like
---
- name: extract
shell: grep "ScriptAlias /.*/" /etc/httpd/conf/httpd.conf
register: st
- debug:
msg: "{{ st.stdout_lines | map('trim') | list }}"
and that playbook prints out when ran
ok: [52.61.71.178] => {
"msg": [
"ScriptAlias /cgi-bin/ \"/var/www/cgi-bin/\""
]
}
So what I need to do is put that msg "ScriptAlias /cgi-bin/ \"/var/www/cgi-bin/\""in an array and extract the last element in that which is the "/var/www/cgi-bin/"
What is the best approach in extracting the last element in my msg to where it can get only the "/var/www/cgi-bin/"?
Try as below. Only added regex_replace (Answer From Valdimir) to get exactly what you want.
- hosts: localhost
vars:
lines:
- "ScriptAlias /cgi-bin/ \"/var/www/cgi-bin/\""
tasks:
- debug:
msg: "{{ item.split(' ')[2] | regex_replace('\"', '') }}"
loop: "{{ lines }}"
Output should be exactly as ::
"msg": "/var/www/cgi-bin/"
The split method works fine. Output is abridged.
> ansible-playbook split_test.yml
"msg": "\"/var/www/cgi-bin/\""
split_test.yml
- hosts: localhost
vars:
lines:
- "ScriptAlias /cgi-bin/ \"/var/www/cgi-bin/\""
tasks:
- debug:
msg: "{{ item.split(' ')[2] }}"
loop: "{{ lines }}"

How do I test in a var matches against a list of substrings in ansible?

I am fairly new to ansible and I am trying to determine how to test is a variable passed to my playbook matches against a list of substrings.
I have tried something like the following. Looping through my list of badcmds and then testing whether it is in the variable passed.
vars:
badcmds:
- clear
- no
tasks:
- name: validate input
debug:
msg: " {{ item }}"
when: item in my_command
with_items: "{{ badcmds }}"
I am getting the following error:
"msg": "The conditional check 'item in my_command' failed.
The error was: Unexpected templating type error occurred on
({% if item in my_command %} True {% else %} False {% endif %}):
coercing to Unicode: need string or buffer, bool found
Many thanks.
one problem with your playbook is that - no is automatically translated to boolean false. you should use "no" to make Ansible consider the variable as a string. without quotes:
---
- hosts: localhost
connection: local
gather_facts: false
vars:
badcmds:
- clear
- no
my_command: clear
tasks:
- name: print variable
debug:
msg: "{{ item }}"
with_items:
- "{{ badcmds }}"
output:
TASK [print variable] ***********************************************************************************************************************************************************************************************
ok: [localhost] => (item=None) => {
"msg": "clear"
}
ok: [localhost] => (item=None) => {
"msg": false
}
I guess you should enclose no in quotes, because this behavior was not your intention.
to make a loop and check if the variable matches any item from the badcmds list, you can use:
---
- hosts: localhost
connection: local
gather_facts: false
vars:
badcmds:
- "clear"
- "no"
tasks:
- name: validate input
debug:
msg: "{{ item }}"
when: item == my_command
with_items:
- "{{ badcmds }}"
hope it helps

Resources