I am writing a script, which should alter the Users of all Tenants of the Database. When i print the Command and execute it from hand on the Server, it works fine. With the Command Module i get this weird Error:
ERROR! failed at splitting arguments, either an unbalanced jinja2 block or quotes: {{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a "{{ item.stdout | replace('"', '')}}"
The error appears to be in '/tmp/awx_313202_z7e4l568/project/ansible_labor/scripts/update_users.yml': line 37, column 5, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Execute Querys on Tenants
^ here
I think it has something to do with the Quotes in the Command Module.
This is the Script:
---
- name: Update all Users in DB
hosts: '*'
become: true
vars:
_User: "{{ lookup('env', 'USER') | upper }}_M"
_pattern: ^\"[a-zA-Z0-9]*\"
_tenants: []
_query: select 'alter user ' || user_name || ' IDENTIFIED EXTERNALLY AS ''' || user_name ||'#INTERN'';' as todo from users where EXTERNAL_IDENTITY LIKE '%COMPANY.DE';
_hdbclient: "/hana/shared/{{ SID }}/hdbclient/hdbsql -i {{ Instanz }} -n {{ inventory_hostname }}"
tasks:
- name: "Tenants der DB auslesen"
command: "{{ _hdbclient }} -U BASIS_{{ SID }}_SYSTEMDB -x -a 'SELECT * FROM M_DATABASES'"
register: tenants
- debug:
msg: "{{ tenants }}"
- set_fact:
_tenants: "{{ _tenants + [ item | regex_search(_pattern) | replace('\"', '') ] }}"
with_items: "{{ tenants.stdout_lines }}"
- name: Print extracted Tenants
debug:
msg: "{{ _tenants }}"
- name: Create Query on Tenant
command: '{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item }} -x -a "{{ _query }}"'
register: query_tenants
with_items: "{{ _tenants }}"
- name: Print All Queries
debug:
msg: "{{ query_tenants.results[1] }}"
- name: Execute Querys on Tenants
command: "{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a \"{{ item.stdout | replace('\"', '')}}\" "
with_items: "{{ query_tenants.results }}"
This is an escape issue inside your replace filter. You need to replace double quotes which translates as a string to \". Since that string is itself inside a double quoted string you have to escape the escape => \\\n
So keeping your initial quoting this gives:
command: "{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a \"{{ item.stdout | replace('\\\"', '') }}\""
Meanwhile there are a few (non exhaustive list) alternatives you might want to consider:
command: '{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a "{{ item.stdout | replace("\"", "") }}"'
command: >-
{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a "{{ item.stdout | replace("\"", "") }}"
Related
I'm trying to find a way to mask the credentials being passed in a command when executing the below tasks in case there is an execution failure,
- name: Define string variables for username and password
set_fact:
temp_user: "{{ user }}"
temp_pass: "{{ password }}"
- name: run-script
command: "{{ abc }} {{ def }}/ghi/jkl.js {{ temp_user }} {{
temp_pass }} {{ mno }} {{ pqr }}"
no_log: true
register: output
become: yes
become_user: root
Firstly, I tried using fact "no_log" but it hides the command in logs and only refers that there is an execution failure without any additional details which is not desired, The required approach is to view the log but with masked credentials so trying the below,
- name: Define string variables for username and password
set_fact:
temp_user: "{{ user }}"
temp_pass: "{{ password }}"
- name: run-script
command: "{{ abc }} {{ def }}/ghi/jkl.js {{ temp_user }} {{
temp_pass }} {{ mno }} {{ pqr }}"
ignore_errors: True
no_log: true
register: output
become: yes
become_user: root
- debug:
msg: "{{ output.stderr | regex_replace(temp_user, '*****') |
regex_replace(temp_pass, '*****') }}"
when: output.stderr != ""
- debug:
msg: "{{ output.stdout | regex_replace(temp_user, '*****') |
regex_replace(temp_pass, '*****') }}"
when: output.stdout != ""
I used the facts "no_log" & "ignore_erros" followed by the "debug" module to hide the command when there is a failure and also view a level of debugging info with replacing credentials with asterisks so that it cannot be exposed but this approach doesn't output the desired level of debugging, Is there a better approach to mask the credentials in logs when there is a failure ?
I've used the below approach and it works,
- name: run-script
command: "{{ abc }} {{ def }}/ghi/jkl.js {{ user }} {{ pass }} {{ mno }} {{
pqr }}"
no_log: true
register: output
become: yes
become_user: root
- name: debug error
debug:
msg: "{{ output | replace(rhsso_user, '*****') | replace(rhsso_password,
'*****') }}"
when: output.stderr != ""
- name: debug output
debug:
msg: "{{ output | replace(rhsso_user, '*****') | replace(rhsso_password,
'*****') }}"
when: output.stdout != ""
I have created the following ansible role for samba, which works without problems.
The user will be created and the password will be set:
role:
---
- name: "Include OS-specific variables"
include_vars: "{{ ansible_os_family }}.yml"
- name: "Ensure Samba-related packages are installed"
apt:
name:
- samba
- samba-common
state: present
when: ansible_os_family == 'Debian'
- name: "Configure smb.conf"
template:
src: etc/samba/smb.conf
dest: /etc/samba/smb.conf
notify: restart_smbd
- name: "Create system user and home directory"
ansible.builtin.user:
name: "{{ item.name }}"
home: "{{ item.home }}"
system: yes
skeleton: no
shell: /sbin/nologin
group: nogroup
state: present
with_items: "{{ smb.user_details }}"
- name: "Create samba users"
shell: >
set -e -o pipefail
&& (pdbedit --user={{ item.name }} 2>&1 > /dev/null)
|| (echo '{{ item.smbpassword }}'; echo '{{ item.smbpassword }}')
| smbpasswd -s -a {{ item.name }}
args:
executable: /bin/bash
register: samba_create_users
changed_when: "'Added user' in samba_create_users.stdout"
loop: "{{ smb.user_details }}"
- name: "Set samba passwords correctly"
shell: >
set -e -o pipefail
&& (smbclient -U {{ item.name }}%{{ item.smbpassword }} -L 127.0.0.1 2>&1 > /dev/null)
|| (echo '{{ item.smbpassword }}'; echo '{{ item.smbpassword }}')
| smbpasswd {{ item.name }}
args:
executable: /bin/bash
register: samba_verify_users
changed_when: "'New SMB password' in samba_verify_users.stdout"
loop: "{{ smb.user_details }}"
- name: "Ensure Samba is running and enabled"
service:
name: "{{ samba_daemon }}"
state: started
enabled: true
host vars:
.
.
.
smb:
user_details:
- name: test
home: /backup/test
smbpassword: testtest
When i know want to improve the role, that an absent user is getting deleted, it will not work:
- name: "Configure smb.conf"
template:
src: etc/samba/smb.conf
dest: /etc/samba/smb.conf
notify: restart_smbd
- name: "Create system user and home directory"
become: yes
user:
name: "{{ item.key }}"
state: "{{ item.value.state | default('present') }}"
append: yes
system: yes
skeleton: no
group: nogroup
home: "{{ item.value.home }}"
shell: "{{ item.value.shell | default('/sbin/nologin') }}"
loop: "{{ samba_users | dict2items }}"
when: "'state' not in item.value or item.value.state == 'present'"
- name: "Create samba users"
shell: >
set -e -o pipefail
&& (pdbedit --user={{ item.key }} 2>&1 > /dev/null)
|| (echo '{{ item.value.name }}'; echo '{{ item.smbpassword }}')
| smbpasswd -s -a {{ item.key }}
args:
executable: /bin/bash
register: samba_create_users
changed_when: "'Added user' in samba_create_users.stdout"
loop: "{{ samba_users }}"
- name: "Set samba passwords correctly"
shell: >
set -e -o pipefail
&& (smbclient -U {{ item.key }}%{{ item.smbpassword }} -L 127.0.0.1 2>&1 > /dev/null)
|| (echo '{{ item.smbpassword }}'; echo '{{ item.smbpassword }}')
| smbpasswd {{ item.key }}
args:
executable: /bin/bash
register: samba_verify_users
changed_when: "'New SMB password' in samba_verify_users.stdout"
loop: "{{ samba_users }}"
- name: "Remove unwanted users."
become: yes
user:
name: "{{ item.key }}"
state: "{{ item.value.state | default('absent') }}"
remove: true
loop: "{{ samba_users | dict2items }}"
when: "'state' in item.value and item.value.state == 'absent'"
- name: "Ensure Samba is running and enabled"
service:
name: "{{ samba_daemon }}"
state: started
enabled: true
...
host vars:
.
.
.
samba_users:
test:
state: absent
The following error happens:
msg": "Invalid data passed to 'loop', it requires a list, got this instead: {'test': {'state': 'absent'}}. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup."
Is it even possible to do this easily?
The problem is that if the user is absent, the "Create samba users" task should not actually be run through.
I would have to write a second task that deletes the Samba user.
But how can I control that when the user is absent he only uses the new task, e.g. "delete samba users"?
I want to uncomment multiple lines in a file using Ansible playbook.
Let's say this as a content which I need to comment out.
if [xxxx]; then
echo xxxxxxxx
exit;
fi
These 4 lines I need to comment out using Ansible playbook.
PLease help me how I can comment out.
An option would be to use blockinfile if you control the content of the file. Set the BEGIN and END markers in the file. Then replace the block as you wish.
If the content of the file is not under your control you might want to try and set the markers with lineinfile (use line, insertbefore, insertafter, firstmatch ...)
The best practice, however, is to use the module template.
Example of blockinfile
Given the file below for testing
shell> cat /tmp/test.txt
line1
line2
if [yyy]; then echo yyyyyyyyy exit;
cmd
cmd
fi
# if [xxxx]; then echo xxxxxxxx exit;
# cmd
# cmd
# fi
lineX
the goal is to uncomment the if statement. Create the markers first. Declare the variables
marker: marker_1
path: /tmp/test.txt
regex1: '# if \[xxxx\]; then echo xxxxxxxx exit;'
replace1: '# if [xxxx]; then echo xxxxxxxx exit;'
regex2: '# fi'
replace2: '# fi'
Test if the BEGIN marker exists. If it does not exist, create the markers in the block
- name: "Check begin marker {{ marker }}"
ansible.builtin.command:
cmd: >
grep -q '# BEGIN ANSIBLE MANAGED BLOCK {{ marker }}' {{ path }}
register: checkmarker
ignore_errors: true
changed_when: false
- block:
- name: "Create begin marker {{ marker }}"
ansible.builtin.replace:
path: '{{ path }}'
regexp: '{{ regex1 }}'
replace: |-
{{ '#' }} BEGIN ANSIBLE MANAGED BLOCK {{ marker }}
{{ replace1 }}
- name: "Create end marker {{ marker }}"
ansible.builtin.replace:
path: '{{ path }}'
regexp: '({{ regex1 }}[\s\S]*?){{ regex2 }}'
replace: |-
\g<1>{{ replace2 }}
{{ '#' }} END ANSIBLE MANAGED BLOCK {{ marker }}
when: checkmarker.rc != 0
gives
shell> cat /tmp/test.txt
line1
line2
if [yyy]; then echo yyyyyyyyy exit;
cmd
cmd
fi
# BEGIN ANSIBLE MANAGED BLOCK marker_1
# if [xxxx]; then echo xxxxxxxx exit;
# cmd
# cmd
# fi
# END ANSIBLE MANAGED BLOCK marker_1
lineX
Declare the variables
test_content: "{{ (test_out['content']|b64decode).splitlines() }}"
marker_1_begin: "{{ test_content|select('regex', '^.*BEGIN.*marker_1.*$')|first }}"
marker_1_end: "{{ test_content|select('regex', '^.*END.*marker_1.*$')|first }}"
block_1_begin_index: "{{ test_content.index(marker_1_begin) + 1 }}"
block_1_end_index: "{{ test_content.index(marker_1_end) }}"
block_1: "{{ test_content[block_1_begin_index|int:block_1_end_index|int] }}"
block_1_update: "{{ block_1|
map('regex_replace', block_1_regex, block_1_replace)|
join('\n') }}"
block_1_regex: '^#\s(.*)$'
block_1_replace: '\1'
Read and update the block
- name: "Read file {{ path }}"
slurp:
src: "{{ path }}"
register: test_out
gives
marker_1_begin: '# BEGIN ANSIBLE MANAGED BLOCK marker_1'
block_1_begin_index: '9'
marker_1_end: '# END ANSIBLE MANAGED BLOCK marker_1'
block_1_end_index: '13'
block_1:
- '# if [xxxx]; then echo xxxxxxxx exit;'
- '# cmd'
- '# cmd'
- '# fi'
block_1_update: |-
if [xxxx]; then echo xxxxxxxx exit;
cmd
cmd
fi
Update the block
- name: "Update block {{ marker }} in {{ path }}"
ansible.builtin.blockinfile:
path: "{{ path }}"
marker: '# {mark} ANSIBLE MANAGED BLOCK {{ marker }}'
block: |
{{ block_1_update }}
gives
shell> cat /tmp/test.txt
line1
line2
if [yyy]; then echo yyyyyyyyy exit;
cmd
cmd
fi
# BEGIN ANSIBLE MANAGED BLOCK marker_1
if [xxxx]; then echo xxxxxxxx exit;
cmd
cmd
fi
# END ANSIBLE MANAGED BLOCK marker_1
lineX
Example of a complete playbook for testing
- hosts: localhost
vars:
marker: marker_1
path: /tmp/test.txt
regex1: '# if \[xxxx\]; then echo xxxxxxxx exit;'
replace1: '# if [xxxx]; then echo xxxxxxxx exit;'
regex2: '# fi'
replace2: '# fi'
test_content: "{{ (test_out['content']|b64decode).splitlines() }}"
marker_1_begin: "{{ test_content|select('regex', '^.*BEGIN.*marker_1.*$')|first }}"
marker_1_end: "{{ test_content|select('regex', '^.*END.*marker_1.*$')|first }}"
block_1_begin_index: "{{ test_content.index(marker_1_begin) + 1 }}"
block_1_end_index: "{{ test_content.index(marker_1_end) }}"
block_1: "{{ test_content[block_1_begin_index|int:block_1_end_index|int] }}"
block_1_update: "{{ block_1|
map('regex_replace', block_1_regex, block_1_replace)|
join('\n') }}"
block_1_regex: '^#\s(.*)$'
block_1_replace: '\1'
tasks:
- name: "Check begin marker {{ marker }}"
ansible.builtin.command:
cmd: >
grep -q '# BEGIN ANSIBLE MANAGED BLOCK {{ marker }}' {{ path }}
register: checkmarker
ignore_errors: true
changed_when: false
- block:
- name: "Create begin marker {{ marker }}"
ansible.builtin.replace:
path: '{{ path }}'
regexp: '{{ regex1 }}'
replace: |-
{{ '#' }} BEGIN ANSIBLE MANAGED BLOCK {{ marker }}
{{ replace1 }}
- name: "Create end marker {{ marker }}"
ansible.builtin.replace:
path: '{{ path }}'
regexp: '({{ regex1 }}[\s\S]*?){{ regex2 }}'
replace: |-
\g<1>{{ replace2 }}
{{ '#' }} END ANSIBLE MANAGED BLOCK {{ marker }}
when: checkmarker.rc != 0
- name: "Read file {{ path }}"
slurp:
src: "{{ path }}"
register: test_out
- debug:
var: test_content
- debug:
var: marker_1_begin
- debug:
var: block_1_begin_index
- debug:
var: marker_1_end
- debug:
var: block_1_end_index
- debug:
var: block_1
- debug:
var: block_1_update
- name: "Update block {{ marker }} in {{ path }}"
ansible.builtin.blockinfile:
path: "{{ path }}"
marker: '# {mark} ANSIBLE MANAGED BLOCK {{ marker }}'
block: |
{{ block_1_update }}
See:
Example of blockinfile markers
files-markers.yml
I have a txt file like:
test1:group1:real name1
test2:group1:real name2
test3:group2:real name3
test4:group2:real name4
and I want to use it as an Ansible variable for a shell.
- shell: test.sh -u {{ username }} -g {{ group }} -r {{ real_name }}
With lookup, I was able to get a line from the file such as test1:group1:real name1 However, I am not sure how to separate name or group.
file_lines: "{{ lookup('file', './files/user.txt').splitlines() }}"
Is there a way to make it work?
You can split the string into a list with split:
- shell: test.sh -u {{ username }} -g {{ group }} -r {{ real_name }}
loop: "{{ lookup('file', './files/user.txt').splitlines() }}"
vars:
params: "{{ item.split(':') }}"
username: "{{ params[0] }}"
group: "{{ params[1] }}"
real_name: "{{ params[2] }}"
You can, of course, write it in a single line, depending on how readable you want your code to be.
Although the following command works when typing in in shell
echo -ne "myser\nmypass\n" | smbpasswd -a -s myuser
The following task fails in ansible
- name: add dms samba user
command: echo -ne "myuser\nmypass\n" | smbpasswd -a -s myuser
notify: restart samba
It does not produce any errors, but the user is not created.
Working with ansible 2.3.0.0 on Ubuntu 16.0.4.
As stated, pipes won't work with the command module. I've used something like this in the past to create Samba users:
- name: Configure Samba users.
shell: >
(pdbedit --user={{ item.username }} 2>&1 > /dev/null)
|| (echo '{{ item.password }}'; echo '{{ item.password }}')
| smbpasswd -s -a {{ item.username }}
register: smbpasswd
changed_when: "'Added user' in smbpasswd.stdout"
with_items: "{{ samba_users }}"
loop_control:
label: "{{ item.username }}"
The task will only run if the user does not exist yet. So changing passwords won't work with this example.
Please try this approach with your Ansible Playbook:
- name: set Samba passwords for each user
shell: "printf '{{ item.passwd }}\n{{ item.passwd }}\n' | smbpasswd -a {{ item.name }}"
with_items:
- "{{ users }}"
tags: smbpasswd
Please note that you will need to map your variables file that includes users: with the format of:
users:
- name: userName
passwd: myClearTextPassword
Please note that to support smbpasswd you will be passing this password as clear text. Additionally, noting this is only a single task that would need to be included in your playbook.
I improved the code from siwyd and Tormod Macleod slightly.
Thanks to both of you!
- name: shell - create samba users
shell: >
set -e -o pipefail
&& (pdbedit --user={{ item.username }} 2>&1 > /dev/null)
|| (echo '{{ item.password }}'; echo '{{ item.password }}')
| smbpasswd -s -a {{ item.username }}
args:
executable: /bin/bash
register: samba_create_users
changed_when: "'Added user' in samba_create_users.stdout"
loop: "{{ samba_users }}"
no_log: true
- name: shell - set samba passwords correctly
shell: >
set -e -o pipefail
&& (smbclient -U {{ item.username }}%{{ item.password }} -L 127.0.0.1 2>&1 > /dev/null)
|| (echo '{{ item.password }}'; echo '{{ item.password }}')
| smbpasswd {{ item.username }}
args:
executable: /bin/bash
register: samba_verify_users
changed_when: "'New SMB password' in samba_verify_users.stdout"
loop: "{{ samba_users }}"
no_log: true
Changes:
Added pipefail to satisfy Ansible Lint (https://ansible-lint.readthedocs.io/en/latest/default_rules.html#risky-shell-pipe)
Changed executable to /bin/bash, beacause /bin/sh doesn't know pipefail
Added no_log to prevent password logging if the task fails
Removed loop_control label, since logging is disabled
Used loop instead of with_items
The answer by siwyd above is excellent. I was struggling to figure out how to solve this problem in an idempotent way until I saw this. For my use-case, I'd like to keep the passwords in sync so I've added another play to do this. Might be useful for someone
- name: shell - create samba users
shell: >
(pdbedit --user={{ item.username }} 2>&1 > /dev/null)
|| (echo '{{ item.password }}'; echo '{{ item.password }}')
| smbpasswd -s -a {{ item.username }}
register: create_samba_users
changed_when: "'Added user' in create_samba_users.stdout"
become: true
with_items: "{{ samba_users }}"
loop_control:
label: "{{ item.username }}"
- name: shell - set samba passwords correctly
shell: >
(smbclient -U {{ item.username }}%{{ item.password }} -L 127.0.0.1 2>&1 > /dev/null)
|| (echo '{{ item.password }}'; echo '{{ item.password }}')
| smbpasswd {{ item.username }}
register: verify_samba_users
changed_when: "'New SMB password' in verify_samba_users.stdout"
become: true
with_items: "{{ samba_users }}"
loop_control:
label: "{{ item.username }}"
the command module does not support pipelining. use the shell module for stuff like this.
see:
command module
shell module
Another variant using dictionary lists:
ad_users: [
{ username: john.doe, password: P4ssw0rd*, givenname: John, surname: Doe, mail: john.doe#domain, ou: "OU=Department,OU=Division" },
{ username: jane.doe, password: P455w0rd*, givenname: Jane, surname: Doe, mail: jane.doe#domain, ou: "OU=Department,OU=Division" },
]
- name: Add user to AD
command: samba-tool user create {{ item.username }} {{ item.password }} --given-name='{{ item.givenname }}' --surname='{{ item.surname }}' --mail-address={{ item.mail }} --userou='{{ item.ou }}'
loop: "{{ ad_users }}"
Just remember to vault sensitive data.