Replacing / by \ in a string with ansible - ansible

I need to replace all the / by \ in a string stored in a variable.
I'm just trying to do it a simple as possible to test it with a debug, but no matter how I try it I dont get the expected result of just replacing character to character. I think it's probably just a single/double quote problem or maybe the \ needs to be escaped in a certain way I don't know.
vars:
- SecGroup: '/stuff/foo/thing'
tasks:
- name: Display modified var
debug:
msg: "{{ SecGroup | replace('/','\') }}"
Expected output : \stuff\foo\thing
Output with differents tries :
- name: Display modified var
debug:
msg: "{{ SecGroup | replace('/','\') }}"
TASK [Display modified var]
ok: [localhost] => {
"msg": "stufffoothing"
}
- name: Display modified var
debug:
msg: "{{ SecGroup | replace('/','\\') }}"
TASK [Display modified var]
fatal: [localhost]: FAILED! => {"msg": "Unexpected failure during module execution."}
- name: Display modified var
debug:
msg: "{{ SecGroup | replace('/','\\\') }}"
TASK [Display modified var]
fatal: [localhost]: FAILED! => {"msg": "Unexpected failure during module execution."}
- name: Display modified var
debug:
msg: "{{ SecGroup | replace('/','\\\\') }}"
TASK [Display modified var]
ok: [localhost] => {
"msg": "\\\\stuff\\\\foo\\\\thing"
}
I also tried to revert the quotes :
- name: Display modified var
debug:
msg: '{{ SecGroup | replace("/","\") }}'
TASK [Display modified var]
fatal: [localhost]: FAILED! => {"msg": "Unexpected failure during module execution."}
I can't explain the output of this one
- name: Display modified var
debug:
msg: '{{ SecGroup | replace("/","\\") }}'
TASK [Display modified var]
ok: [localhost] => {
"msg": "\\\\stuff\\\\foo\\\\thing"
}

I think you've stumbled upon an edge case that involves the interaction between YAML escaping and Python escaping. The only way I was able to get it to work was introducing a guard character -- something to ensure that the \ isn't the last character in the expression, which we then remove with a subsequent replace() filter. Here I'm using a semicolon (;), but you could use anything that you're certain won't be in your SecGroup string. Note that your choice of quotes is significant; quoting the entire string with single quotes inhibits YAML escaping:
- name: With guard character
debug:
msg: '{{ SecGroup | replace("/","\;") | replace(";", "") }}'
Outputs:
TASK [With guard character] *******************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "\\stuff\\foo\\thing"
}
Which is exactly what you want (remembering that a single \ is encoded as \\ in JSON output).
Regarding this:
- name: Display modified var
debug:
msg: '{{ SecGroup | replace("/","\\") }}'
TASK [Display modified var]
ok: [localhost] => {
"msg": "\\\\stuff\\\\foo\\\\thing"
}
You are successfully replacing / with two backslashes, \\. Since a backslash must be encoded as \\ in JSON output, a double backslash will end up represented as \\\\, so this:
"msg": "\\\\stuff\\\\foo\\\\thing"
Means you actually have the string:
\\stuff\\foo\\thing

I wanted to add an alternative solution:
If you're familiar with Python, you can just write a custom filter module and avoid multiple layers of escaping. E.g., if you were to create filter_plugins/reslash.py with the following content:
#!/usr/bin/python
def filter_reslash(val):
return val.replace('/', '\\')
class FilterModule(object):
filter_map = {
'reslash': filter_reslash
}
def filters(self):
return self.filter_map
You could then write your playbook like this:
---
- hosts: localhost
gather_facts: false
vars:
- SecGroup: '/stuff/foo/thing'
tasks:
- debug:
msg: "{{ SecGroup | reslash }}"
That's arguably a cleaner solution.

The solution by #larsks didn't entirely work for me as described. I needed to escape the backslash with double slashes \ plus the guard character in order for it to work in the Ansible Playbook.
This works: replace('/','\\;') | replace(';', '')

Another easy solution is to leave escaping backslash to ansible itself. This is how i would have done.
- set_fact:
replacer: '\'
- name: With guard character
debug:
msg: '{{ SecGroup | replace("/",replacer)}}'

Same workaround if you want replace 1 backslash with double backslash on a windows path.
- hosts: localhost
gather_facts: False
vars:
- iis_manager_logdir: 'C:\inetpub\logs\manager-logs'
tasks:
- set_fact:
iis_mng_logs: "{{ iis_manager_logdir | regex_replace('\\\\', '\\\\;') | regex_replace(';', '\\\\') }}"
- name: Original path
debug:
msg: "{{ iis_manager_logdir }}"
- name: New path
debug:
msg: "{{ iis_mng_logs }}"

Thanks to the #larsks's answer i've managed to replace backslashes in ansible string variable value without intermediate replace. It's possible by supplying into regex_replace expression a regex quantifier {1} between last backslash and closing quote.
For example, expression like {{ install_path | regex_replace('\\\\{1}', '/') }} replaces all occurences of backslash \ to forward slash /. It was used to replace Windows path delimiters with Unix-like ones:
- name: install libs
win_shell: "pip install --no-index --find-links \"file://{{ install_path | regex_replace('\\\\{1}', '/') }}/libs\" attrs requests"
become: true

For what its worth, after countless struggles, this is what has worked for me without any workarounds:
Forward to Back Slash
ForwardtoBackSlash: "{{ 'c:/test' | regex_replace('\\\/', '\\\\') }}"
output:
c:\test
Single Slash to Double Slash
SingleSlashtoDoble: "{{ 'C:\test\logs\logfile.txt'| regex_replace('\\\\', '\\\\\\\\') }}"
Output:
C:\\test\\logs\\logfile.txt
I hope it helps someone.

Related

How to remove u' from ansible list

In my playbook, I am trying to get list of sub-directory names using the find module and then extracting the basename from path. I have been able to get the list but the elements are prepended with u'. How can I remove those from the output?
Ansible version 2.9
I tried to look at these SO posts here and here, but couldn't get it to work.
I may not have fully understood how they should be applied
This is part of my playbook:
- name: set item.path | basename
set_fact: dir_name_list2_basename="{{ item.path | basename}}"
with_items: "{{ zookeeper_data_dir.files}}"
register: item_path_basename_list
- debug:
msg: "{{item_path_basename_list.results}}"
- name: debug item.path | basename as list
debug:
var: item.ansible_facts.dir_name_list2_basename
with_items: "{{item_path_basename_list.results}}"
- debug: msg="item_path_basename_list.results {{ item_path_basename_list.results | map(attribute='ansible_facts.dir_name_list2_basename') | list }}"
- name: set fact to array
set_fact: basename_array="{{ item_path_basename_list.results | map(attribute='ansible_facts.dir_name_list2_basename') | list }}"
- debug:
msg: "basename_array &&&&&&&& {{basename_array}}"
And this is the output of the last debug:
ok: [zk3-dev] => {
"msg": "basename_array &&&&&&&& [u'version-2_backup', u'version-2']"
}
ok: [zk2-dev] => {
"msg": "basename_array &&&&&&&& [u'version-2_backup', u'version-2']"
}
ok: [zk1-dev] => {
"msg": "basename_array &&&&&&&& [u'version-2_backup', u'version-2']"
}
I would like the basename_array to show up as ["version-2_backup", "version-2"] without the u prefix
How should I change my set fact to array task, so I will get the desired result?
Since ["version-2_backup", "version-2"] is actually a JSON array, you could use the to_json filter.
This said, your long set of tasks looks like an overcomplicated process for a requirement that can be achieved with the right set of map filters, since map can apply the same filter to all the elements of a list, you can easily fit your basename in it.
So, given:
- debug:
msg: >-
basename_array &&&&&&&&
{{
zookeeper_data_dir.files
| map(attribute='path')
| map('basename')
| to_json
}}
This yields:
ok: [localhost] => {
"msg": "basename_array &&&&&&&& [\"version-2_backup\", \"version-2\"]"
}
Note that the double quotes are escaped because you are using the JSON stdout callback. But, if you change the callback to YAML, this would yield exactly what you expected:
ok: [localhost] =>
msg: basename_array &&&&&&&& ["version-2_backup", "version-2"]

Remove only trailling whitespaces from stdout_lines

I need to assert the policy-maps from Cisco devices. And Cisco for some reason adds trailing whitespaces on some lines, but not all. I want to remove them, but, only the trailing whitespaces.
- name: Get running class-map & policy-map config
vars:
ansible_connection: network_cli
ios_command:
commands:
- 'show run | sec class-map|policy-map'
register: show_policy
- name: Print trim
debug:
var: show_policy.stdout_lines | trim
- name: Ansible block with assert module
block:
- name: Validate running line
ansible.builtin.assert:
that:
- "lookup('template', 'policy_desired.j2').splitlines() in show_policy.stdout_lines"
success_msg: "TEST: {{ UNIT_HOSTNAME }}: VALIDATE RUNNING POLICY: PASSED"
fail_msg: "TEST: {{ UNIT_HOSTNAME }}: VALIDATE RUNNING POLICY: FAILED"
This gives the following output:
{
"show_policy.stdout_lines | trim": [
[
"class-map match-any CM-QOS-GENERIC-BESTEFFORT-MARK",
" description Generic - Best Effort",
" match access-group name ACL-QOS-GENERIC-BESTEFFORT",
"class-map match-any CM-QOS-1P3Q-Q1",
" match dscp cs4 cs5 ef ", <--- Notice the whitespace
"class-map match-any CM-QOS-1P3Q-Q2",
" match dscp cs6 cs7 ",
"class-map match-any CM-QOS-1P3Q-Q3",
" match dscp cs1 "
]
]
}
The filter should then be added in the assert module so that it is gone when being asserted.
I have tried multiple things, but nothing seems to do the trick:
- name: Print trim
debug:
var: show_policy.stdout_lines | trim
- name: Print trim
debug:
var: show_policy.stdout_lines | strip
- name: Print trim
debug:
var: show_policy.stdout_lines.strip()
- name: Print trim
debug:
var: "{{ show_policy.stdout_lines | map('trim') }}"
- name: Print trim
debug:
var: "{{ show_policy.stdout_lines | trim }}"
You can use a regex_replace filter in order to achieve that.
Note: It seems, from you debug of show_policy.stdout_lines that it is actually a list of list, so you need the trim to happen on show_policy.stdout_lines.0.
Mind that, in the example below, I am purposely using a YAML multiline syntax, to save me from escaping the backslashes.
If you don't want to use this syntax, you will have to escaping those backslashes by doubling them (i.e. \\1 and \\s instead of \1 and \s).
- debug:
msg: >-
{{
show_policy.stdout_lines.0 | map('regex_replace', '(.*)\s+$', '\1')
}}
Given the task:
- debug:
msg: >-
{{
show_policy.stdout_lines | map('regex_replace', '(.*)\s+$', '\1')
}}
vars:
show_policy:
stdout_lines:
-
- "class-map match-any CM-QOS-GENERIC-BESTEFFORT-MARK"
- " description Generic - Best Effort"
- " match access-group name ACL-QOS-GENERIC-BESTEFFORT"
- "class-map match-any CM-QOS-1P3Q-Q1"
- " match dscp cs4 cs5 ef "
- "class-map match-any CM-QOS-1P3Q-Q2"
- " match dscp cs6 cs7 "
- "class-map match-any CM-QOS-1P3Q-Q3"
- " match dscp cs1 "
This yields:
msg:
- class-map match-any CM-QOS-GENERIC-BESTEFFORT-MARK
- ' description Generic - Best Effort'
- ' match access-group name ACL-QOS-GENERIC-BESTEFFORT'
- class-map match-any CM-QOS-1P3Q-Q1
- ' match dscp cs4 cs5 ef'
- class-map match-any CM-QOS-1P3Q-Q2
- ' match dscp cs6 cs7'
- class-map match-any CM-QOS-1P3Q-Q3
- ' match dscp cs1'
Since you've tried already to use the Python method .strip(), a minimal example playbook
---
- hosts: localhost
become: false
gather_facts: false
vars:
pol:
stdout_lines:
- "First line"
- " Second line with leading whitespace"
- "Third line with trailing whitespace "
- " Forth line with leading and trailing whitespaces "
- "Last line"
tasks:
- name: Print stripped
debug:
msg: "\"{{ item.rstrip() }}\""
loop: "{{ pol.stdout_lines }}"
will result into the output of
TASK [Print stripped] ***********************************************************
ok: [localhost] => (item=First line) =>
msg: '"First line"'
ok: [localhost] => (item= Second line with leading whitespace) =>
msg: '" Second line with leading whitespace"'
ok: [localhost] => (item=Third line with trailing whitespace ) =>
msg: '"Third line with trailing whitespace"'
ok: [localhost] => (item= Forth line with leading and trailing whitespaces ) =>
msg: '" Forth line with leading and trailing whitespaces"'
ok: [localhost] => (item=Last line) =>
msg: '"Last line"'
You may also have a look into .lstrip().
Unfortunately there is no direct Jinja2 Builtin Filter for, so a simple approach like
- name: Print trimmed
debug:
msg: "{{ pol.stdout_lines | map('trim') }}"
won't work since it
Strip leading and trailing characters, by default whitespace.
Further Q&A
How do I trim whitespace from a string?
How to proceed further?
If there are several use cases where one could take advantages from a filter like strip(), lstrip() and rstrip() it might be feasible to write an own filter plugin.
cat plugins/filter/rstrip.py
#!/usr/bin/python
class FilterModule(object):
def filters(self):
return {
'rstrip': self.rstrip,
}
def rstrip(self, String):
return String.rstrip()
which can then be used as follow
- name: Print stripped
debug:
msg: "{{ pol.stdout_lines | map('rstrip') }}"
and producing the required output.
Further Q&A reagarding Filter Plugins
How to convert an AnsibleUnsafeText HEX value to int?
Is there any way to convert integer to MAC address in Ansible?

Line too long for setting up variable

This is my Ansible task:
- name: get the custom job id
ansible.builtin.set_fact:
custom_job_id: >
"{{ train_custom_image_unmanaged_response.stderr_lines |select('search', 'describe') |list |regex_search('.*/customJobs/(\\d+)', '\\1') |first }}"
when: "(gcs_model_list.stdout is not defined) or ('saved_model.pb' not in gcs_model_list.stdout)"
I am getting "line too long" as Ansible lint error for custom_job_id line.
Any idea how can I break it down in smaller parts?
You can do it using YAML multi lines syntaxes, as you started doing it.
With this syntax, the indentation is what is defining a block, so, as long as you are indented inward of the fact name custom_job_id, all the following code is considered as being the expression that is going to be assigned to that fact.
For example:
- name: get the custom job id
ansible.builtin.set_fact:
custom_job_id: >-
{{
train_custom_image_unmanaged_response.stderr_lines
| select('search', 'describe')
| list
| regex_search('.*/customJobs/(\d+)', '\1')
| first
}}
when: >-
gcs_model_list.stdout is not defined
or 'saved_model.pb' not in gcs_model_list.stdout
Here is a playbook complying with the Ansible linting demonstrating this:
- hosts: localhost
gather_facts: true
tasks:
- name: Get the custom job id
ansible.builtin.set_fact:
custom_job_id: >-
{{
train_custom_image_unmanaged_response.stderr_lines
| select('search', 'describe')
| list
| regex_search('.*/customJobs/(\d+)', '\1')
| first
}}
when: >-
gcs_model_list.stdout is not defined
or 'saved_model.pb' not in gcs_model_list.stdout
vars:
train_custom_image_unmanaged_response:
stderr_lines:
- foo
- bar
- describe - /customJobs/123
- baz
gcs_model_list:
- name: Display `custom_job_id`
ansible.builtin.debug:
var: custom_job_id
Which yields:
PLAY [localhost] **********************************************************
TASK [Get the custom job id] **********************************************
ok: [localhost]
TASK [Display `custom_job_id`] ********************************************
ok: [localhost] =>
custom_job_id: '123'

How to convert to valid JSON when running an Ansible command?

So, I have this list:
ip_range:
- "1.x.x.x/24"
- "2.x.x.x/24"
And I'm trying to pass it on to an AWS cli command as a valid JSON:
- name: list of IPs
set_fact:
allowed_cidrs: "{{ ip_range | ipaddr('network/prefix') }}"
- debug:
msg: "{{ allowed_qualys_cidrs | to_json }}"
- name: send command
command: >
aws wafv2 update-ip-set
--addresses "{{ allowed_cidrs | to_json }}"
That debug prints:
["1.x.x.x/24", "2.x.x.x/24"]
But what the command tries to send is:
"cmd": [
"aws",
"wafv2",
"update-ip-set",
"--addresses",
"[1.x.x.x/24, 2.x.x.x/24]",
]
Which, because the double quotes are gone, is not a valid JSON.
I already tried every possible approach with Ansible and Jinja2, but I can't figure out a way to send that command with a valid JSON.
Use single quotes in your command rather than double quotes:
- name: list of IPs
set_fact:
allowed_cidrs: "{{ ip_range | ipaddr('network/prefix') }}"
vars:
ip_range:
- "1.1.1.1/24"
- "2.2.2.2/24"
- name: send command
command: >-
aws wafv2 update-ip-set
--addresses '{{ allowed_cidrs | to_json }}'
register: _cmd
- debug:
var: _cmd.cmd
Yields:
TASK [list of IPs] **********************************************************
ok: [localhost]
TASK [send command] *********************************************************
changed: [localhost]
TASK [debug] ****************************************************************
ok: [localhost] => {
"_cmd.cmd": [
"aws",
"wafv2",
"update-ip-set",
"--addresses",
"[\"1.1.1.0/24\", \"2.2.2.0/24\"]"
]
}

Unable to add curly braces within curly braces in a jinja2 syntax

I have a set of jinja2 actions within curly braces separated by pipes. Within that set of actions I need to add a variable, but I keep getting syntax errors.
debug:
msg: "{{ item.path | basename | regex_replace('{{ variable }}', '') }}"
with_items: "{{ content.files }}"
Note that, the variable will contain some regex string for example...
The problem ansible has with this, is that it contains a double quote inside a double quote. I tried escaping, inverting double quotes to single quotes...nothing worked.
When I run the above as is, it considers the variable as a literal value.
You don't need curly braces to denote variables inside of curly braces. Here's a simple playbook to demonstrate:
---
- name: test
hosts: localhost
gather_facts: false
vars:
content:
files:
- path: /path1/itemXXX.jpg
- path: /path2/itXem.pdf
regex_pattern: '[X]+' # Match one or more X's
tasks:
- debug:
msg: "{{ item.path | basename | regex_replace(regex_pattern, '') }}"
with_items: "{{ content.files }}"
Results are:
TASK [debug] ***********************************************************************************************************************************************************************
ok: [localhost] => (item={'path': '/path1/itemXXX.jpg'}) => {
"msg": "item.jpg"
}
ok: [localhost] => (item={'path': '/path2/itXem.pdf'}) => {
"msg": "item.pdf"
}

Resources