Trying to replace a line which has multiple special characters, which gets interpreted as escape characters by ansible.
Tried using "\" and "." for every character, and tried using !unsafe.
- name: Update file
replace:
path: /some/file
regexp: "*[0-9a-zA-Z._-]* )" #<<=== This is line to be replaced
replace: "*[0-9a-z._-]* )" #<<== With this
backup: yes
Getting errors like:
raise error, v # invalid expression\r\nsre_constants.error: nothing to repeat\r\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
You should escape the special characters with one backslash each:
regexp: '\*\[0\-9a\-zA\-Z\._\-\]\* \)'
And you should use single quotes.
Tried this and it worked:
regexp: '\*\[0-9a-zA-Z\._-\]\* \)'
replace: '*[0-9a-z._-]* )'
Related
I'm trying to do a replace where I first escape all double quotes and then I want to use the result of this replace to updated a value. But in the last replace the backslashes are removed, why is this and how do I avoid that?
Example in bash:
>TEST_OBJECT='{"val1": "a", "val2": "b"}'
>ESCAPED_OBJ="$(echo $TEST_OBJECT | sed 's/"/\\"/g')"
>echo $ESCAPED_OBJ
{\"val1\": \"a\", \"val2\": \"b\"}
>echo 'value: "_REPLACE_ME"' | sed "s#_REPLACE_ME#$ESCAPED_OBJ#g"
value: "{"val1": "a", "val2": "b"}"
I'm expecting this on the last row:
value: "{\"val1\": \"a\", \"val2\": \"b\"}"
EDIT
I realize I presented the issue wrong, the reason why I do it in 2 steps is because the first replace happens in one step and then the second replace happens in a later step. This is part of a github workflow and the last replace actually replaces a string in a different yaml file.
sed "s#_REPLACE_ME#$ESCAPED_OBJ#g" > ${{ github.action_path }}/config/job.yml
So I don't think I can do the replace in one step, first I need to update the string and then replace a value in another file.
job.yml
...
env:
- name: CUSTOM_DATA_OBJECT
value: "_REPLACE_ME"
I need the value to be escaped so that it doesn't break the yaml.
Using sed
$ sed "s#_REPLACE_ME#${TEST_OBJECT//\"/\\\\\"}#" input_file
...
env:
- name: CUSTOM_DATA_OBJECT
value: "{\"val1\": \"a\", \"val2\": \"b\"}"
Maybe letting yq take care of the correct escaping for YAML?
TEST_OBJECT='{"val1": "a", "val2": "b"}' \
yq eval '.env[0].value = strenv(TEST_OBJECT)' file.yaml
env:
- name: CUSTOM_DATA_OBJECT
value: "{\"val1\": \"a\", \"val2\": \"b\"}"
I tried using regex_replace option this way:
Here groups[group_names[0]] is list of node names
"groups[group_names[0]]": [
"node1.in.labs.corp.netin",
"node2.in.labs.corp.netin"
]
- set_fact:
groups[group_names[0]]={{ groups[group_names[0]] |
map('regex_replace', _regex, _replace)|list }}
vars:
_regex: '^(.*?)\.(.*)$'
_replace: '-n \1'
Hitting the following error:
{"changed": false, "msg": "The variable name 'groups[group_names[0]]' is not valid. Variables must start with a letter or underscore character, and contain only letters, numbers and underscores."}
Can i assign back to same list ? after replacing the regex ?
Also -n option is using so that my expected output should be
-n node1 -n node2
I am using [file lookup] which reads the whole file and stores the content in a variable. My play looks something like this:
- name: Store foo.xml contents in a variable
set_fact:
foo_content: "{{ lookup('file', 'foo.xml' ) | replace('\n', '')}}"
So the above code reads the foo.xml file and stores it in the variable, but the problem is when the foo.xml has line breaks in it, it also includes the line break in the variable.
My foo.xml is this file:
<?xml version="1.0" encoding="utf-8"?>
<initialize_param>
<secrets>
<my_secret id="99">3VMjII6Hw+pd1zHV5THSI712y421USUS8124487128745812sajfhsakjfasbfvcasvnjasjkvbhasdfasgfsfaj5G8A9+n8CkLxk7Dqu0G8Jclg0eb1A5xeFzR3rrJHrb2GBBa7PJNVx8tFJP3AtF6ek/F/WvlBIs2leX2fq+/bGryKlySuFmbcwBsThmPJC5Z5AwPJgGZx</my_secret>
</secrets>
</initialize_param>
The output removes line break \n but also incudes the tabs \r & \t
I need to got rid of the \n , need to get rid of extra formatting too (\r & \t), Moreover after the replace filter I get the error while firing a DB Update query as
stderr: /bin/sh: 1: cannot open ?xml: No such file
Use the Jinja trim filter:
"{{ lookup('file', 'foo.xml' ) | trim }}"
You can do that with the replace filter?
contents: "{{ lookup('file', '/etc/foo.txt') | replace('\n', '')}}"
You may use the regex_replace filter since the trim doesn't clear other line break characters as you mentioned in the question.
"{{ some_stdout_to_clear | regex_replace('[\\r\\n\\t]+','') }}"
Using in an ansible playbook for ansible 2.4.2.0
- name: Check the Tomcat version
shell: "unzip -c {{ mount_path_instance }}/tomcat9/lib/catalina.jar META-INF/MANIFEST.MF |awk '/Implementation-Version: / {sub(/[^ ]+ /, \"\"); print \$0}'"
register: tomcat_version
when: instance_check_v9.stat.exists
I get the error
TASK [redhat-tomcat-update : include_tasks] ************************************
fatal: [localhost]: FAILED! => {"reason": "Syntax Error while loading YAML.\n\n\nThe error appears to have been in '/home/ansible/work/play_redhat_tomcat_update/roles/redhat-tomcat-update/tasks/variables.yml': line 39, column 153, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- name: Check the Tomcat version\n shell: \"unzip -c {{ mount_path_instance }}/tomcat9/lib/catalina.jar META-INF/MANIFEST.MF |awk '/Implementation-Version: / {sub(/[^ ]+ /, \\\"\\\"); print \\$0}'\"\n ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n with_items:\n - {{ foo }}\n\nShould be written as:\n\n with_items:\n - \"{{ foo }}\"\n\nexception type: <class 'yaml.scanner.ScannerError'>\nexception: while parsing a quoted scalar\n in \"<unicode string>\", line 39, column 10\nfound unknown escape character\n in \"<unicode string>\", line 39, column 153"}
running the next command works fine
ansible localhost -m shell -a "unzip -c {{ mount_path_instance }}/tomcat9/lib/catalina.jar META-INF/MANIFEST.MF |awk '/Implementation-Version: / {sub(/[^ ]+ /, \"\"); print \$0}'" --extra-vars='{"mount_path_instance": "/appl/tomcat/paul_1_uat", "instance_name": "paul_1_uat" }'
[WARNING]: Consider using unarchive module rather than running unzip
localhost | SUCCESS | rc=0 >>
9.0.7.redhat-12
does some one see what is wrong in the yml definition?
YAML expands backslashes in double-quoted strings, and \$ is not a legal escape in YAML (the \" is fine)
What you'll want is to use the alternate syntax that side-steps the double-quoting issue:
shell: |
unzip -c {{ mount_path_instance }}/tomcat9/lib/catalina.jar META-INF/MANIFEST.MF | awk '/Implementation-Version: / {sub(/[^ ]+ /, ""); print $0}'
Ironically, you don't actually have to escape that dollarsign because the awk command is already in single-quotes, so you could also just drop the \$ from your existing command, too, but I am pretty sure you'll get less backslash-itis using the shell: | syntax
With regard to:
ansible localhost -m shell -a "unzip -c {{ mount_path_instance }}/tomcat9/lib/catalina.jar META-INF/MANIFEST.MF |awk '/Implementation-Version: / {sub(/[^ ]+ /, \"\"); print \$0}'"
running fine, that's because your shell actually collapsed the \$ for you, which one can see with a simple echo:
echo "thingy \"\" \$0"
I have to use Ansible modules in order to edit the /etc/ssh/sshd_config file - every time I create a new user I want to append it at these two lines:
AllowUsers root osadmin <new_user>
AllowGroups root staff <new_group>
At this moment I'm using the shell module to execute a sed command but would like to use lineinfile, if possible
- shell: "sed -i '/^Allow/ s/$/ {{ user_name }}/' /etc/ssh/sshd_config"
Any suggestions would be sincerely appreciated.
The replace module will replace all instances of a regular expression pattern within a file. Write a task to match the AllowUsers line and replace it with the original line appended with the user name. To ensure the task is idempotent, a negative lookahead assertion in the regular expression checks if the user name already appears in the line. For example:
- name: Add user to AllowUsers
replace:
backup: yes
dest: /etc/ssh/sshd_config
regexp: '^(AllowUsers(?!.*\b{{ user_name }}\b).*)$'
replace: '\1 {{ user_name }}'
You could do it in a single play with a newline, but I think it's cleaner to use two lineinfile plays for this.
- hosts: '127.0.0.1'
vars:
usernames:
- larry
- curly
- moe
usergroups:
- stooges
- admins
tasks:
- lineinfile:
dest: /etc/ssh/sshd_config
regexp: '^AllowUsers'
line: "AllowUsers {{usernames | join(' ')}}"
- lineinfile:
dest: /etc/ssh/sshd_config
regexp: '^AllowGroups'
line: "AllowGroups {{usergroups | join(' ')}}"
Note that groups is a reserved word so don't use that as a variable name.
The selected answer assumes that the complete list of users is available at runtime, while the most popular answer can fail when there is a dash in the username, because \b interprets it as a word boundary. The following solution assumes that the playbook cannot regenerate the complete list of usernames from scratch, and tries and handle the corner case of dashes:
name: add a user to the list of AllowUsers if not present
lineinfile:
path: /etc/ssh/sshd_config
backrefs: yes
backup: yes
regexp: "^AllowUsers((?:(?:\s+\S+(?!\S))(?<!\s{{ username }}))+\s*?)(\n?)$"
line: "AllowUsers\1 {{ username }}\2"
validate: /usr/sbin/sshd -t -f %s
As a bonus, I threw in sshd_config backup and verification.
How the (interesting part of the) regular expression works:
--------------------------+----------------------------------------------------
( |
--------------------------+----------------------------------------------------
(?: | This group is not captured
--------------------------+----------------------------------------------------
(?:\s+\S+(?!\S)) | Matches any sequence of whitespace characters fol-
| lowed by any sequence of non-whitespace characters,
| that is to say a leading space and a username. The
| negative look-ahead at the end prevents a "catast-
| rophic backtracking". Also, this group is not cap-
| tured.
--------------------------+----------------------------------------------------
(?<!\s{{ username }}) | Applies a negative look-behind on the username, so
| that if the username found by the previous expres-
| sion matches, the regular expression fails. The
| match on a leading whitespace character ensures
| that the comparison is made on the complete string.
--------------------------+----------------------------------------------------
)+ | Groups the detection of a username and its negative
| look-behind together. The "+" quantifier is used
| here on the assumption that the file already cont-
| ains at least one username, but "*" could be used
| for a more relaxed matching.
--------------------------+----------------------------------------------------
\s*? | Matches any trailing whitespace. The match is lazy
| in order to detect the newline character later on.
--------------------------+----------------------------------------------------
) | Captures the whole text after "AllowUsers" (this
| will be \1).
--------------------------+----------------------------------------------------
(\n?) | Captures either a newline character or an empty
| string (this will be \2).
--------------------------+----------------------------------------------------
If the regular expression matches, it means that the line exists and that it does not contain {{ username }}, so we append it.
If the regular expression does not match, it means that either the line does not exist or that it contains {{ username }}, and we do nothing.
I had the same problem. I needed add user to sudoers group, let's say 'testuser' to line:
User_Alias SOMEADMIN = smoeuser1, someuser2, someuser3
This worked well for me:
- name: add testuser to end of line
lineinfile:
dest: /etc/sudoers.d/somegroup
state: present
regexp: '^(User_Alias(.*)$)'
backrefs: yes
line: '\1, testuser'
The point is that if I had '^User_Alias(..)$'* in regexp and not '^(User_Alias(..)$)'* it didn't work and whole line was replaced. With () arround searched text the result was OK:
User_Alias SOMEADMIN = smoeuser1, someuser2, someuser3, testuser
So then anything can work in line:, included ansible variables like "{{ usernames | join(', ') }}"
This worked for me
- name: Add Group to AllowGroups
lineinfile:
dest=/etc/ssh/sshd_config
backup=True
backrefs=True
state=present
regexp='^(AllowGroups(?!.*\b{{ groupname }}\b).*)$'
line='\1 {{ groupname }}'