How to search for a string in a remote file using Ansible? - ansible

Based on a questions
How to search for a string in a file using Ansible?
Ansible: How to pull a specific string out of the contents of a file?
Can slurp be used as a direct replacement for lookup?
and considerations like
By using the slurp module one is going to transfer the whole file from the Remote Node to the Control Node over the network just in order to process it and looking up a string. For log files these can be several MB and whereby one is mostly interested only in the information if the file on the Remote Node contains a specific string and therefore one would only need to transfer that kind of information, true or false.
How to execute a script on a Remote Node using Ansible?
I was wondering how this can be solved instead of using the shell module?
---
- hosts: localhost
become: false
gather_facts: false
vars:
SEARCH_STRING: "test"
SEARCH_FILE: "test.file"
tasks:
- name: Search for string in file
command:
cmd: "grep '{{ SEARCH_STRING }}' {{ SEARCH_FILE }}"
register: result
# Since it is a reporting task
# which needs to deliver a result in any case
failed_when: result.rc != 0 and result.rc != 1
check_mode: false
changed_when: false
Or instead of using a workaround with the lineinfile module?
---
- hosts: localhost
become: false
gather_facts: false
vars:
SEARCH_STRING: "test"
SEARCH_FILE: "test.file"
tasks:
- name: Search for string
lineinfile:
path: "{{ SEARCH_FILE }}"
regexp: "{{ SEARCH_STRING }}"
line: "SEARCH_STRING FOUND"
state: present
register: result
# Since it is a reporting task
changed_when: false
failed_when: "'replaced' not in result.msg" # as it means SEARCH_STRING NOT FOUND
check_mode: true # to prevent changes and to do a dry-run only
- name: Show result, if not found
debug:
var: result
when: "'added' in result.msg" # as it means SEARCH_STRING NOT FOUND
Since I am looking for a more generic approach, could it be a feasible case for Should you develop a module?

Following Developing modules and Creating a module I've found the following simple solution with
Custom Module library/pygrep.py
#!/usr/bin/python
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
from ansible.module_utils.basic import AnsibleModule
def run_module():
module_args = dict(
path=dict(type='str', required=True),
search_string=dict(type='str', required=True)
)
result = dict(
changed=False,
found_lines='',
found=False
)
module = AnsibleModule(
argument_spec=module_args,
supports_check_mode=True
)
with open(module.params['path'], 'r') as f:
for line in f.readlines():
if module.params['search_string'] in line:
result['found_lines'] = result['found_lines'] + line
result['found'] = True
result['changed'] = False
module.exit_json(**result)
def main():
run_module()
if __name__ == '__main__':
main()
Playbook pygrep.yml
---
- hosts: localhost
become: false
gather_facts: false
vars:
SEARCH_FILE: "test.file"
SEARCH_STRING: "test"
tasks:
- name: Grep string from file
pygrep:
path: "{{ SEARCH_FILE }}"
search_string: "{{ SEARCH_STRING }}"
register: search
- name: Show search
debug:
var: search
when: search.found
For a simple test.file
NOTEST
This is a test file.
It contains several test lines.
321tset
123test
cbatset
abctest
testabc
test123
END OF TEST
it will result into an output of
TASK [Show search] ******************
ok: [localhost] =>
search:
changed: false
failed: false
found: true
found_lines: |-
This is a test file.
It contains several test lines.
123test
abctest
testabc
test123
Some Links
What is the grep equivalent in Python?
cat, grep and cut - translated to Python
What is the Python code equivalent to Linux command grep -A?

Related

Ansible: 'regex_replace' removing every character starting with "#"

I have following registered variable with stdout and I want to remove every character starting with "#".
ok: [localhost] => {
"msg": [
"wazuh#4.3.10-4311"
]
}
Example: wazuh#4.3.10-4311 should become wazuh.
remove every character after "#"
---
- hosts: localhost
gather_facts: false
vars:
my_string: wazuh#4.3.10-4311
tasks:
- debug:
msg: "{{ my_string | regex_replace('#.*', '') }}"
Provides:
wazuh
One could also look at the use case as just interested in the left part of the #
delimitered string.
---
- hosts: localhost
gather_facts: false
vars:
my_string: wazuh#4.3.10-4311
tasks:
# For Ansible v2.9 and later
- name: Use of Python string method
debug:
msg: "{{ my_string.split('#') | first }}"
So not removing the part behind but only deliver the part before.
# For Ansible v2.10 and later
- name: Show left part with filter
debug:
msg: "{{ my_string | split('#') | first }}"
Also this solution it's working:
regex_replace('#.*$','')

How do I copy files to a machine based on a vars flag using ansible?

I'm trying to get a playbook to copy a different set of files to a server, depending an a flag set on the group it belongs too in the ansible inventory.
The code is as follows
- name: check if they exist already
stat:
path: /home/me/set1files
register: st
- name: copy pre-prod if they don't
copy:
src: /home/me/set1files
dest: /home/me
owner: me
group: me
mode: "0644"
when: (st.stat.exists==false) and (is_emms_pp==true)
- name: check if prod files exist
stat:
path: /home/me/set2files
register: stprod
- name: copy prod files
copy:
src: /home/me/set2files
dest: /home/me
owner: me
group: me
mode: "0644"
when: (stprod.stat.exists==false) and (is_emms_pp==false)
and in the inventory file
[PPEMMS]
emmspp1
emmspp2
emmspp3
[PPEMMS:vars]
is_emms_pp = true
[PRODEMMS]
emms1
emms2
emms3
[PRODEMMS:vars]
is_emms_pp = false
[EMMS:children]
PPEMMS
PRODEMMS
When I run the script on a new PP machine, it all seems to work fine, but when I run it on a new Prod machine, it still copies the PP files and skips the step for the Prod machine.
Is this a valid test or do I have something wrong?
One thought I had was that because I'm limiting the script to one machine, rather than the group, its not picking up the group_var?
I'm using the script as
ansible-playbook -i inventory/hosts -l emms1 install_emms.yml --ask-vault-pass --extra-vars '#passwd.yml'
For each machine.
The variable is_emms_pp = true is a string. Quoting from Defining variables at runtime
"Values passed in using the key=value syntax are interpreted as strings. Use the JSON format if you need to pass non-string values such as Booleans, integers, floats, lists, and so on."
Given the inventory to simplify the test
test_01 is_emms_pp=true
test_02 is_emms_pp=false
The playbook
- hosts: test_01,test_02
tasks:
- debug:
var: is_emms_pp|type_debug
gives
ok: [test_01] =>
is_emms_pp|type_debug: str
ok: [test_02] =>
is_emms_pp|type_debug: str
Comparison between a string and a boolean will fail
- debug:
msg: Copy file
when: is_emms_pp == true
gives
skipping: [test_01]
skipping: [test_02]
One option is to compare strings. The task below works as expected
- debug:
msg: Copy file
when: is_emms_pp == 'true'
gives
ok: [test_01] =>
msg: Copy file
skipping: [test_02]
The cleaner option is to convert the variable to boolean. The task below gives the same result
- debug:
msg: Copy file
when: is_emms_pp|bool
See CONDITIONAL_BARE_VARS
Example of how to fix the conditions
- name: check if they exist already
stat:
path: /home/me/set1files
register: st
- name: copy pre-prod if they don't
debug:
msg: Copy set1files
when:
- not st.stat.exists
- is_emms_pp|bool
- name: check if prod files exist
stat:
path: /home/me/set2files
register: st
- name: copy prod files
debug:
msg: Copy set2files
when:
- not st.stat.exists
- not is_emms_pp|bool

using ansible variable in task conditonal

When I use an Ansible variable in the failed_when conditional, Ansible complains and fails by telling me that I cannot use Jinja2 expressions with failed_when.
Below snippet does not work:
vars:
var1: "var"
tasks:
- name: "sth"
raw: "anothersth"
register: status
failed_when: var1 in status.stdout
I want to use a regex in the above example and use variable var1 within a regex. This also fails.
I want to fail the task if var1 is repeated in the beginning and at the end of the stdout_lines. I was thinking something like this:
failed_when: regex("^{{var1}}/.*/{{var1}}$") in status.stdout
I wonder whether there is a way to achieve what I intend to do?
===============================================================
Exact play and the description:
Let me explain what I am trying to do:
run a command with "raw" module and register the stdout_lines.
iterate over a list of items, where list is a fact of current ansible host
use failed_when to search a pattern in the registered stdout and fail the task if it is not found
peers_underlay is a list defined in the host file:
peers_underlay:
- ip: "172.16.1.2"
- ip: "172.16.1.6"
This is the play:
- hosts: leaf-2, spine-1
gather_facts: no
tags: [ verify ]
vars:
bgp_underlay: "show ip bgp summary"
tasks:
- name: "underlay bgp test"
raw: "{{ bgp_underlay }}"
register: underlay_status
with_items: "{{ peers_underlay }}"
failed_when: not regex(".*/{{ item['ip'] }}/.*/Estab/.*") in underlay_status.stdout_lines
Q: "Fail the task if var1 is repeated in the beginning and at the end."
A: It's possible to test strings. For example, use regex
- hosts: localhost
vars:
my_stdout1: 'ABC dlkjfsldkfjsldkfj ABC'
my_stdout2: 'alkdjflaksdjflakdjflaksjdflaksdjflasj'
var1: 'ABC'
my_regex: '^{{ var1 }}(.*){{ var1 }}$'
tasks:
- debug:
msg: my_regex match my_stdout1
when: my_stdout1 is regex(my_regex)
- debug:
msg: my_regex match my_stdout2
when: my_stdout2 is regex(my_regex)
gives
"msg": "my_regex match my_stdout1"
The particular task sequence would be
vars:
var1: "var"
my_regex: '^{{ var1 }}(.*){{ var1 }}$'
tasks:
- name: "sth"
raw: "anothersth"
register: status
failed_when: status.stdout is regex(my_regex)

Need Syntax to add Ansible meta module to existing Playbook

I wish to search for a string ("AC245") in all files with extension *.db under /home/examples directory. Below is what i tried.
---
- name: "Find the details here "
hosts: localhost
any_errors_fatal: true
serial: 1
tasks:
- name: Ansible find files multiple patterns examples
find:
paths: /home/examples
patterns: "*.db"
recurse: yes
register: files_matched
- name: Search for String in the matched files
command: grep -i {{ myString }} {{ item.path }}
register: command_result
failed_when: command_result.rc == 0
with_items:
- "{{ files_matched.files }}"
run the above find.yml using this command:
ansible-playbook find.yml -e "myString=AC245"
My requirement is that if the string is found I wish to abort the play immediately using "meta: end_play" marking the playbook as FAILED.
Can you help suggest how can I update my current code to add the end_play feature as soon as the string is found in any *.db file ?
One possible solution is to move the task into a separate tasks file and loop over it. That will allow discrete control over each iteration of the task. For example:
playbook.yml:
---
- name: "Find the details here "
hosts: localhost
any_errors_fatal: true
serial: 1
tasks:
- name: Ansible find files multiple patterns examples
find:
paths: /home/mparkins/bin/playbooks_sandpit/outputs/dir
patterns: "*.db"
recurse: yes
register: files_matched
- name: Search for String in the matched files
include_tasks:
tasks.yml
with_items:
- "{{ files_matched.files }}"
tasks.yml:
- command: grep -i {{ myString }} {{ item.path }}
register: command_result
failed_when: command_result.rc == 0
You can add additional tasks in the tasks file if you wish to fail in a different way, for example using the fail module or meta: end_play
Q: "The requirement is that if the string is found I wish to abort the play immediately using "meta: end_play" marking the playbook as FAILED."
(ansible 2.7.9)
A: It's not possible.
1) It's not possible to execute meta: end_play in a loop on a condition (string is found)
2) It's not possible to execute both meta: end_play and fail/assert in one play
1) It is possible to write a filter_plugin (Python2)
$ cat filter_plugins/file_filters.py
import mmap
def file_list_search(list, string):
for file in list:
if file_search(file, string):
break
return file_search(file, string)
def file_search(file, string):
f = open(file)
s = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
if s.find(string) != -1:
return True
else:
return False
class FilterModule(object):
''' Ansible filters for operating on files '''
def filters(self):
return {
'file_search' : file_search,
'file_list_search' : file_list_search
}
and use it in a play. For example
- hosts: localhost
vars:
myString: "test string"
tasks:
- find:
paths: /home/examples
patterns: "*.db"
register: files_matched
- name: End of play when myString found
meta: end_play
when: "files_matched.files|
json_query('[*].path')|
file_list_search(myString)"
- debug:
msg: continue
2) It is possible to set_stats. The play below
- name: End of play when myString found
block:
- set_stats:
data:
FAILED: 1
- meta: end_play
when: "files_matched.files|
json_query('[*].path')|
file_list_search(myString)"
- debug:
msg: continue
- set_stats:
data:
FAILED: 0
gives the output below if the string is found
PLAY RECAP **********************************************************************************
127.0.0.1 : ok=4 changed=0 unreachable=0 failed=0
CUSTOM STATS: *******************************************************************************
RUN: { "FAILED": 1}
Allow show_custom_stats
$ grep stats ansible.cfg
show_custom_stats = True

How to write varibles/hard code values in nested json in ansible?

I'm trying to create a json file with hard codes valuesas a output in nested json.But the second play is overwriting the first play value.So do we have any best option to do this?
I have tried with to_nice_json template to copy the variable to json file.But not able to keep multiple variable values in imported_var to copy to json file
---
- hosts: localhost
connection: local
gather_facts: false
tasks:
- name: load var from file
include_vars:
file: /tmp/var.json
name: imported_var
- name: Checking mysqld status
shell: service mysqld status
register: mysqld_stat
ignore_errors: true
- name: Checking mysqld status
shell: service httpd status
register: httpd_stat
ignore_errors: true
- name: append mysqld status to output json
set_fact:
imported_var: "{{ imported_var | combine({ 'status_checks':[{'mysqld_status': (mysqld_stat.rc == 0)|ternary('good', 'bad') }]})}}"
# - name: write var to file
# copy:
# content: "{{ imported_var | to_nice_json }}"
# dest: /tmp/final.json
- name: append httpd status to output json
set_fact:
imported_var: "{{ imported_var| combine({ 'status_checks':[{'httpd_status': (httpd_stat.rc == 0)|ternary('good', 'bad') }]})}}"
# - debug:
# var: imported_var
- name: write var to file
copy:
content: "{{ imported_var | to_nice_json }}"
dest: /tmp/final.json
Expected result:
{
"status_checks": [
{
"mysqld_status": "good"
"httpd_status": "good"
}
]
}
Actual result:
{
"status_checks": [
{
"httpd_status": "good"
}
]
}
You're trying to perform the sort of data manipulation that Ansible really isn't all that good at. Any time you attempt to modify an existing variable -- especially if you're trying to set a nested value -- you're making life complicated. Having said that, it is possible to do what you want. For example:
---
- hosts: localhost
gather_facts: false
vars:
imported_var: {}
tasks:
- name: Checking sshd status
command: systemctl is-active sshd
register: sshd_stat
ignore_errors: true
- name: Checking httpd status
command: systemctl is-active httpd
register: httpd_stat
ignore_errors: true
- set_fact:
imported_var: "{{ imported_var|combine({'status_checks': []}) }}"
- set_fact:
imported_var: >-
{{ imported_var|combine({'status_checks':
imported_var.status_checks + [{'sshd_status': (sshd_stat.rc == 0)|ternary('good', 'bad')}]}) }}
- set_fact:
imported_var: >-
{{ imported_var|combine({'status_checks':
imported_var.status_checks + [{'httpd_status': (httpd_stat.rc == 0)|ternary('good', 'bad')}]}) }}
- debug:
var: imported_var
On my system (which is running sshd but is not running httpd, this will output:
TASK [debug] **********************************************************************************
ok: [localhost] => {
"imported_var": {
"status_checks": [
{
"sshd_status": "good"
},
{
"httpd_status": "bad"
}
]
}
}
You could dramatically simplify the playbook by restructuring your data. Make status_checks a top level variable, and instead of having it be a list, have it be a dictionary that maps a service name to the corresponding status. Combine this with some loops and you end up with something that is dramatically simpler:
---
- hosts: localhost
gather_facts: false
tasks:
# We can use a loop here instead of writing a separate task
# for each service.
- name: Checking service status
command: systemctl is-active {{ item }}
register: services
ignore_errors: true
loop:
- sshd
- httpd
# Using a loop in the previous task means we can use a loop
# when creating the status_checks variable, which again removes
# a bunch of duplicate code.
- name: set status_checks variable
set_fact:
status_checks: "{{ status_checks|default({})|combine({item.item: (item.rc == 0)|ternary('good', 'bad')}) }}"
loop: "{{ services.results }}"
- debug:
var: status_checks
The above will output:
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => {
"status_checks": {
"httpd": "bad",
"sshd": "good"
}
}
If you really want to add this information to your imported_var, you can do that in a single task:
- set_fact:
imported_var: "{{ imported_var|combine({'status_checks': status_checks}) }}"

Resources