I have a txt file like:
test1:group1:real name1
test2:group1:real name2
test3:group2:real name3
test4:group2:real name4
and I want to use it as an Ansible variable for a shell.
- shell: test.sh -u {{ username }} -g {{ group }} -r {{ real_name }}
With lookup, I was able to get a line from the file such as test1:group1:real name1 However, I am not sure how to separate name or group.
file_lines: "{{ lookup('file', './files/user.txt').splitlines() }}"
Is there a way to make it work?
You can split the string into a list with split:
- shell: test.sh -u {{ username }} -g {{ group }} -r {{ real_name }}
loop: "{{ lookup('file', './files/user.txt').splitlines() }}"
vars:
params: "{{ item.split(':') }}"
username: "{{ params[0] }}"
group: "{{ params[1] }}"
real_name: "{{ params[2] }}"
You can, of course, write it in a single line, depending on how readable you want your code to be.
Related
I'm trying to find a way to mask the credentials being passed in a command when executing the below tasks in case there is an execution failure,
- name: Define string variables for username and password
set_fact:
temp_user: "{{ user }}"
temp_pass: "{{ password }}"
- name: run-script
command: "{{ abc }} {{ def }}/ghi/jkl.js {{ temp_user }} {{
temp_pass }} {{ mno }} {{ pqr }}"
no_log: true
register: output
become: yes
become_user: root
Firstly, I tried using fact "no_log" but it hides the command in logs and only refers that there is an execution failure without any additional details which is not desired, The required approach is to view the log but with masked credentials so trying the below,
- name: Define string variables for username and password
set_fact:
temp_user: "{{ user }}"
temp_pass: "{{ password }}"
- name: run-script
command: "{{ abc }} {{ def }}/ghi/jkl.js {{ temp_user }} {{
temp_pass }} {{ mno }} {{ pqr }}"
ignore_errors: True
no_log: true
register: output
become: yes
become_user: root
- debug:
msg: "{{ output.stderr | regex_replace(temp_user, '*****') |
regex_replace(temp_pass, '*****') }}"
when: output.stderr != ""
- debug:
msg: "{{ output.stdout | regex_replace(temp_user, '*****') |
regex_replace(temp_pass, '*****') }}"
when: output.stdout != ""
I used the facts "no_log" & "ignore_erros" followed by the "debug" module to hide the command when there is a failure and also view a level of debugging info with replacing credentials with asterisks so that it cannot be exposed but this approach doesn't output the desired level of debugging, Is there a better approach to mask the credentials in logs when there is a failure ?
I've used the below approach and it works,
- name: run-script
command: "{{ abc }} {{ def }}/ghi/jkl.js {{ user }} {{ pass }} {{ mno }} {{
pqr }}"
no_log: true
register: output
become: yes
become_user: root
- name: debug error
debug:
msg: "{{ output | replace(rhsso_user, '*****') | replace(rhsso_password,
'*****') }}"
when: output.stderr != ""
- name: debug output
debug:
msg: "{{ output | replace(rhsso_user, '*****') | replace(rhsso_password,
'*****') }}"
when: output.stdout != ""
I am writing a script, which should alter the Users of all Tenants of the Database. When i print the Command and execute it from hand on the Server, it works fine. With the Command Module i get this weird Error:
ERROR! failed at splitting arguments, either an unbalanced jinja2 block or quotes: {{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a "{{ item.stdout | replace('"', '')}}"
The error appears to be in '/tmp/awx_313202_z7e4l568/project/ansible_labor/scripts/update_users.yml': line 37, column 5, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Execute Querys on Tenants
^ here
I think it has something to do with the Quotes in the Command Module.
This is the Script:
---
- name: Update all Users in DB
hosts: '*'
become: true
vars:
_User: "{{ lookup('env', 'USER') | upper }}_M"
_pattern: ^\"[a-zA-Z0-9]*\"
_tenants: []
_query: select 'alter user ' || user_name || ' IDENTIFIED EXTERNALLY AS ''' || user_name ||'#INTERN'';' as todo from users where EXTERNAL_IDENTITY LIKE '%COMPANY.DE';
_hdbclient: "/hana/shared/{{ SID }}/hdbclient/hdbsql -i {{ Instanz }} -n {{ inventory_hostname }}"
tasks:
- name: "Tenants der DB auslesen"
command: "{{ _hdbclient }} -U BASIS_{{ SID }}_SYSTEMDB -x -a 'SELECT * FROM M_DATABASES'"
register: tenants
- debug:
msg: "{{ tenants }}"
- set_fact:
_tenants: "{{ _tenants + [ item | regex_search(_pattern) | replace('\"', '') ] }}"
with_items: "{{ tenants.stdout_lines }}"
- name: Print extracted Tenants
debug:
msg: "{{ _tenants }}"
- name: Create Query on Tenant
command: '{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item }} -x -a "{{ _query }}"'
register: query_tenants
with_items: "{{ _tenants }}"
- name: Print All Queries
debug:
msg: "{{ query_tenants.results[1] }}"
- name: Execute Querys on Tenants
command: "{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a \"{{ item.stdout | replace('\"', '')}}\" "
with_items: "{{ query_tenants.results }}"
This is an escape issue inside your replace filter. You need to replace double quotes which translates as a string to \". Since that string is itself inside a double quoted string you have to escape the escape => \\\n
So keeping your initial quoting this gives:
command: "{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a \"{{ item.stdout | replace('\\\"', '') }}\""
Meanwhile there are a few (non exhaustive list) alternatives you might want to consider:
command: '{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a "{{ item.stdout | replace("\"", "") }}"'
command: >-
{{ _hdbclient }} -U BASIS_{{ SID }}_{{ item.item }} -x -a "{{ item.stdout | replace("\"", "") }}"
This has been a challenge and I am stuck on what else to try. Basically, I have a text file that has an unknown amount of lines. I wish to read through each line and set facts for each line that is then used as variables for a win_domain_user module.
If there is only one line in the text file, then I can accomplish my task with no issues or errors. But, if there is more than one line, I do not know how to start the process again with the next line.
For example:
- name: "Pre-task AD | Get user account info"
win_shell: |
Get-Content C:\AD\user_info.txt
register: 'read_output'
The line of text in the user_info.txt file looks like this:
first-John;last-Doe;display_name-Doe, John (US);title-Engineer;employee_type-Employee;user-john.doe;email-john.doe#company.com;customer-Some_Business;dept_groups-Test_Group
Example of two lines:
first-John;last-Doe;display_name-Doe, John (US);title-Engineer;employee_type-Employee;user-john.doe;email-john.doe#company.com;customer-Some_Business;dept_groups-Test_Group
first-Jane;last-Doe;display_name-Doe, Jane (US);title-Engineer II;employee_type-Employee;user-jane.doe;email-jane.doe#company.com;customer-Some_Business;dept_groups-Test_Group
The set facts looks like:
- set_fact:
first: "{{ read_output.stdout | regex_search(...) }}"
last: "{{ read_output.stdout | regex_search(...) }}"
user: "{{ read_output.stdout | regex_search(...) }}"
email: "{{ read_output.stdout | regex_search(...) }}"
customer: "{{ read_output.stdout | regex_search(...) }}"
title: "{{ read_output.stdout regex_search(...) }}"
display_name: "{{ read_output.stdout regex_search(...) }}"
employee_type: "{{ read_output.stdout | regex_search(...) }}"
dept_groups: "{{ read_output.stdout | regex_search(...) }}"
The winrm Active Directory related module looks like:
- name: "Active Directory | Add domain user account to domain controller"
win_domain_user:
firstname: "{{ first }}"
surname: "{{ last }}"
name: "{{ first }}{{ space }}{{ last }}"
description: "{{ title }}"
password: "{{ password_var_here}}"
upn: "{{ user }}{{ domain_fqdn_var_here }}"
email: "{{ email }}"
state: 'present'
path: "{{ ou_path_here }}"
groups: "Domain Users, {{ dept_groups }}"
attributes:
displayName: "{{ first }}{{ space }}{{ last }}"
domain_username: "{{ domainusername_var_here }}"
domain_password: "{{ domainpassword_var_here }}"
domain_server: "{{ dc_var_here }}"
register: 'add_ad_user'
I tried using a win_shell ForEach (PowerShell) loop to action the lines in C:\AD\user_info.txt but I was not able to figure out how to use set_facts within the ForEach loop and use the win_domain_user module within it. So, instead, I looked into using with_items which from what I understand is now loop. From the info above, I was going to create 2 yml files, one that has the 'Get-Content C:\AD\user_info.txt' / register and then use the include_tasks yml with block for the tasks to run against the data from the Get-Content.
The tricky part is that I do not know how many lines are going to be in that user_info.txt file. Am I on the right track as far as using two files for this or is there a more efficient way?
Each line from the file will be in the registered variable read_output.stdout_lines. We can loop with this variable and split the line using semi-colon (;) to get the individual attributes of the user.
If we take only the first line, the example below with return John:
"{{ read_output.stdout_lines[0].split(';')[0].split('-')[1] }}"
Similarly a debug task:
- name: "Pre-task AD | Get user account info"
win_shell: |
Get-Content C:\AD\user_info.txt
register: read_output
- name: Show user details after splitting
debug:
msg:
- "First: {{ first }}"
- "Last: {{ last }}"
- "Display name: {{ display_name }}"
vars:
first: "{{ item.split(';')[0].split('-')[1] }}"
last: "{{ item.split(';')[1].split('-')[1] }}"
display_name: "{{ item.split(';')[2].split('-')[1] }}"
loop: "{{ read_output.stdout_lines }}"
will produce:
"msg": [
"First: John",
"Last: Doe",
"Display name: Doe, John (US)"
]
Continuing with this theme, we can have the win_domain_user task like (not tested):
- name: "Active Directory | Add domain user account to domain controller"
win_domain_user:
firstname: "{{ first }}"
surname: "{{ last }}"
name: "{{ first }} {{ last }}"
description: "{{ title }}"
password: "{{ password_var_here }}"
upn: "{{ user }}{{ domain_fqdn_var_here }}"
email: "{{ email }}"
state: 'present'
path: "{{ ou_path_here }}"
groups: "Domain Users, {{ dept_groups }}"
attributes:
displayName: "{{ first }} {{ last }}"
domain_username: "{{ domainusername_var_here }}"
domain_password: "{{ domainpassword_var_here }}"
domain_server: "{{ dc_var_here }}"
vars:
first: "{{ item.split(';')[0].split('-')[1] }}"
last: "{{ item.split(';')[1].split('-')[1] }}"
display_name: "{{ item.split(';')[2].split('-')[1] }}"
name: "{{ first }} {{ last }}"
title: "{{ item.split(';')[3].split('-')[1] }}"
email: "{{ item.split(';')[6].split('-')[1] }}"
user: "{{ item.split(';')[5].split('-')[1] }}"
dept_groups: "{{ item.split(';')[8].split('-')[1] }}"
loop: "{{ read_output.stdout_lines }}"
As already stated in the comments, the real good solution would be to change your file format at source to something that is easy to parse. Being in ansible the best format would indeed be yaml/json. But any other standard would make this much easier (xml, csv...).
Note: I know your current data is actually csv with semicolon delimiters. But it doesn't have a header and values contain the field name that must be extracted on each line
As an other remark: registering the output of a cat command (e.g. Get-Content) from a (win-)shell to get a file content is a bad practice. This is precisely why the fetch and slurp modules exist.
The below is just a workaround to get a dict out of your custom and not really ideal format. It will break as soon as a value does not strictly respect the current inferred schema (i.e. contains a dash, a semicolon, a new line...) and will probably become a real nightmare to maintain every time you encounter an exception. But at least it does what you expect from your example and you can build upon that to adapt it to your exact needs.
Edit: my original solution was using a regex (see history if you want to look at it). But #seshadri_c opened my eyes on a much simpler solution that leads to the same result using only split. I borrowed the idea while keeping the key/value mapping dynamic.
Given the following files/user_data.txt
first-John;last-Doe;display_name-Doe, John (US);title-Engineer;employee_type-Employee;user-john.doe;email-john.doe#company.com;customer-Some_Business;dept_groups-Test_Group
first-Jane;last-Doe;display_name-Doe, Jane (US);title-Engineer II;employee_type-Employee;user-jane.doe;email-jane.doe#company.com;customer-Some_Business;dept_groups-Test_Group
The loop_bad_file.yml playbook
---
- name: Loop on a poorly formated file
hosts: localhost
gather_facts: false
vars:
# File name on the remote (here relative to the current dir for the example....)
bad_file_name: files/user_data.txt
# list of user dicts we get by:
#
# 1. spliting each line from the original file (`select` removes the last trainling blank line)
# So we get ['line1....', 'line2....']
#
# 2. split each line on semicolons so we get:
# [['key1-value1', 'key2-value2', ....], ['key1-value1', 'key2-value2', ....], ...]
#
# 3. split each field-value on a dash so we get:
# [[['key1', 'value1'], ['key2', 'value2'], ..., [['key1', 'value1'], ['key2', 'value2'], ...], ...]
#
# 4. Turn this back into a list of dicts using the key/values pairs in each element
user_list: "{{
(bad_file.content | b64decode).split('\n') | select
| map('split', ';')
| map('map', 'split', '-')
| map('items2dict', key_name=0, value_name=1)
}}"
tasks:
- name: Get file content from remote
slurp:
src: "{{ bad_file_name }}"
register: bad_file
- name: Show the result using the transformed list of dicts
vars:
debug_msg: |-
This is entry for {{ item.display_name }}
first: {{ item.first }}
last: {{ item.last }}
title: {{ item.title }}
employee type: {{ item.employee_type }}
email: {{ item.email }}
customer: {{ item.customer }}
department group: {{ item.dept_groups }}
debug:
msg: "{{ debug_msg.split('\n') }}"
loop: "{{ user_list }}"
Gives:
$ ansible-playbook loop_bad_file.yml
PLAY [Loop on a poorly formated file] *********************************************************************
TASK [Get file content from remote] *********************************************************************
ok: [localhost]
TASK [Show the result using the transformed list of dicts] *********************************************************************
ok: [localhost] => (item={'first': 'John', 'last': 'Doe', 'display_name': 'Doe, John (US)', 'title': 'Engineer', 'employee_type': 'Employee', 'user': 'john.doe', 'email': 'john.doe#company.com', 'customer': 'Some_Business', 'dept_groups': 'Test_Group'}) => {
"msg": [
"This is entry for Doe, John (US)",
"first: John",
"last: Doe",
"title: Engineer",
"employee type: Employee",
"email: john.doe#company.com",
"customer: Some_Business",
"department group: Test_Group"
]
}
ok: [localhost] => (item={'first': 'Jane', 'last': 'Doe', 'display_name': 'Doe, Jane (US)', 'title': 'Engineer II', 'employee_type': 'Employee', 'user': 'jane.doe', 'email': 'jane.doe#company.com', 'customer': 'Some_Business', 'dept_groups': 'Test_Group'}) => {
"msg": [
"This is entry for Doe, Jane (US)",
"first: Jane",
"last: Doe",
"title: Engineer II",
"employee type: Employee",
"email: jane.doe#company.com",
"customer: Some_Business",
"department group: Test_Group"
]
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You may want to try to use csvfile lookup plugin [1] to load data from csv file as a variable or iterable object.
[1] https://docs.ansible.com/ansible/latest/collections/ansible/builtin/csvfile_lookup.html
I Have a file /tmp/srv_info.txt as below:
INVENTORY_HOSTNAME: hostname
BIOS_VERSION: biosversion
TOTAL_MEMORY: memory
VDA_SIZE: vdasize
VDB_SIZE: vdbsize
XVDA_SIZE: xvdasize
XVDC_SIZE: xvdcsize
I have to fill it with correct values per host, I already set the variables for each value:
- name: Save results
set_fact:
HN: "{{ ansible_hostname | default('NONE', true) }}"
BV: "{{ ansible_bios_version | default('NONE', true) }}"
MEM: "{{ ansible_memtotal_mb | default('NONE', true) }} MB"
XVDA: "{{ ansible_devices.xvda.size.split(',') | first | default('NONE', true) }}"
XVDC: "{{ ansible_devices.xvdc.size.split(',') | first | default('NONE', true) }}"
VDA: "{{ ansible_devices.vda.size | default('NONE', true) }}"
VDB: "{{ ansible_devices.vdb.size | default('NONE', true) }}"
How can I change /tmp/srv_info.txt with appropriate values, for example:
HOSTNAME: host_one.lab.infra
BIOS_VERSION: 4.13
TOTAL_MEMORY: 3900 MB
VDA_SIZE: 25 GB
VDB_SIZE: 10 GB
XVDA_SIZE: NONE
XVDC_SIZE: NONE
I Can use module replace, or lineinfile for each value, but then need to call 7 times the module.
I would like to loop the variable list, find the corresponding script, and replace the value in the file.
But could not achieve yet.
Can you help?
If you know exactly how the file should look like on the target machine, use a template. That's what they are for.
Template:
HOSTNAME: {{HN}}
BIOS_VERSION: {{BV}}
TOTAL_MEMORY: {{MEM}}
VDA_SIZE: {{VDA}}
VDB_SIZE: {{VDB}}
XVDA_SIZE: {{XVDA}}
XVDC_SIZE: {{XVDB}}
Playbook:
- template:
src: src.j2
dest: /tmp/srv_info.txt
[owner, group, mode...]
For example, use copy
- copy:
content: |
HOSTNAME: {{ HN }}
BIOS_VERSION: {{ BV }}
TOTAL_MEMORY: {{ MEM }}
VDA_SIZE: {{ VDA }}
VDB_SIZE: {{ VDB }}
XVDA_SIZE: {{ XVDA }}
XVDC_SIZE: {{ XVDC }}
dest: /tmp/srv_info.txt
The next option is to put the content into a file and use template.
If you want to change only some of the values read the hashes from the file into a dictionary and use this dictionary to set the defaults. For example
- include_vars:
file: /tmp/srv_info.txt
name: srv_info
- set_fact:
HN: "{{ ansible_hostname | default(srv_info.INVENTORY_HOSTNAME) }}"
BV: "{{ ansible_bios_version | default(srv_info.BIOS_VERSION) }}"
MEM: "{{ ansible_memtotal_mb | default(srv_info.TOTAL_MEMORY) }} MB"
XVDA: "{{ ansible_devices.xvda.size.split(',') | first | default(srv_info.XVDA_SIZE) }}"
XVDC: "{{ ansible_devices.xvdc.size.split(',') | first | default(srv_info.XVDC_SIZE) }}"
VDA: "{{ ansible_devices.vda.size | default(srv_info.VDA_SIZE) }}"
VDB: "{{ ansible_devices.vdb.size | default(srv_info.VDB_SIZE) }}"
I am writing playbook to check user principal in kerberos servers. If principal exists it should skip task and if not it should create user principal. I am not sure how to use string with when condition I am trying below but getting errors
"ERROR! Syntax Error while loading YAML.
expected <block end>, but found '<scalar>'
The error appears to be in '/home/revyas/RHELProjects/Atlas/eda-datalake/playbooks/provision-emr.yml': line 42, column 31, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
stat:
path: "{{ user_dir }}"/"{{ keytab_name }}"
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
Playbook:
- name: Check if user principals exist
command: 'kadmin -w "{{ emr_kdc_admin_password }}" -p kadmin/admin listprincs'
register: user_princs
delegate_to : "{{ emr_kerberos_master }}"
tags: "emr_acct"
- name: Create user kerberos principal if not exist
command: 'kadmin -w {{ emr_kdc_admin_password }} -p kadmin/admin addprinc -randkey {{ kerberos_username }}#{{ emr_kerberos_realm }}'
when: "{{ kerberos_username }}#{{ emr_kerberos_realm }}" not in user_princs.stdout
delegate_to: "{{ emr_kerberos_master }}"
tags: "emr_acct"
User principal from kdc have format given below:
emr-test1-aren-reetika#abd.xyz.com
emr-test-aren#bd.xyz.com
emr-test-integration-test#bd.xyz.com
For the first cited issue, yaml doesn't behave like python or shell which automatically concatenate string literals together
You'll want:
stat:
path: "{{ user_dir }}/{{ keytab_name }}"
And the second error is because yaml believes the leading " is the start of a YAML literal, but in fact it's the start of a Jinja2 literal, thus:
when: '"{{ kerberos_username }}#{{ emr_kerberos_realm }}" not in user_princs.stdout'
Or you can use any of the scalar folding syntaxes, if you prefer that:
when: >-
"{{ kerberos_username }}#{{ emr_kerberos_realm }}"
not in user_princs.stdout
when: "{{ kerberos_username }}#{{ emr_kerberos_realm }}" not in user_princs.stdout
change to
when: "{{ kerberos_username }}\\#{{ emr_kerberos_realm }}" not in user_princs.stdout