I have an ansible playbook which accepts a variable, passing a variable with quotes in it(its needed), that variable will used to query against DB
Playbook
- name: Execute clear script
script: scripts/clear-documents.sh {{ids}}
Command
ansible-playbook playbooks/maintenance.yml -i hosts -t clear -e ids=["foo", "bar"]
in this process script receives the input as [foo, bar] instead of ["foo", "bar"]
I tried escaping using backslash but that did not help
ansible-playbook playbooks/maintenance.yml -i hosts -t clear -e ids=[\"foo\", \"bar\"]
Adding double quotes in playbook, makes the input like "[foo,bar]" and not ["foo", "bar"]
script: scripts/clear-documents.sh "{{ids}}"
I searched a lot but did not get any proper solution, is there a way to handler this
Note:
ansible version - 2.2.3.0
The thing you are looking for is quote, in combination with #JGK's correct usage of -e ids='["foo", "bar"]' because you were not quoting them on the way into ansible, and then you were not quoting them on the way out of ansible in that shell: task
- shell: scripts/clear-documents.sh {{ ids | quote }}
As is mentioned in How to escape backslash and double quote in Ansible (script module) you need to surround the {{ ids }} with ' ' in your shell command.
#!/usr/bin/env ansible-playbook
- hosts: localhost
gather_facts: false
become: false
tasks:
- name: Escape characters for fun and profit
vars:
string_list: '["one", "two"]'
shell: "echo '{{ string_list }}'"
register: output1
- name: Print it out
debug:
msg: "{{ output1 }}"
- name: Don't escape characters
vars:
string_list: '["one", "two"]'
shell: "echo {{ string_list }}"
register: output2
- name: Print it out
debug:
msg: "{{ output2 }}"
PLAY [localhost] ************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
TASK [Escape characters for fun and profit] *********************************************************************************************************************************************************************************************************************************************************************************************************************************
changed: [localhost]
TASK [Print it out] *********************************************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": {
"changed": true,
"cmd": "echo '[\"one\", \"two\"]'",
"delta": "0:00:00.003400",
"end": "2019-05-07 12:02:32.897856",
"failed": false,
"rc": 0,
"start": "2019-05-07 12:02:32.894456",
"stderr": "",
"stderr_lines": [],
"stdout": "[\"one\", \"two\"]",
"stdout_lines": [
"[\"one\", \"two\"]"
]
}
}
TASK [Don't escape characters] **********************************************************************************************************************************************************************************************************************************************************************************************************************************************
changed: [localhost]
TASK [Print it out] *********************************************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": {
"changed": true,
"cmd": "echo [\"one\", \"two\"]",
"delta": "0:00:00.002990",
"end": "2019-05-07 12:02:33.192049",
"failed": false,
"rc": 0,
"start": "2019-05-07 12:02:33.189059",
"stderr": "",
"stderr_lines": [],
"stdout": "[one, two]",
"stdout_lines": [
"[one, two]"
]
}
}
PLAY RECAP ******************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
localhost : ok=4 changed=2 unreachable=0 failed=0
Related
I have to execute below 2 commands whose value depends on the system.
# sysctl -w kernel.shmmax= parse_from_shm.sh script #For example 17179869184
# sysctl -w kernel.shmall= parse_from_shm.sh script #For example 4194304
./shm.sh will echo both system values required in the below format
kernel.shmmax=4185686016
kernel.shmall=1021896
So I have to parse & get integer value above result & execute ultimately below 2 commands
# sysctl -w kernel.shmmax=4185686016
# sysctl -w kernel.shmall=1021896
I have tried to register & parse the integer values using regex. But I couldn't able to process it perfectly. Any help would be of great help.
---
- hosts: fossology_test
become: true
become_user: root
environment:
HOME: /usr/ansible
gather_facts: no
tasks:
- name: run shell script
become: true
become_user: root
command: ./shm.sh
args:
chdir: /usr/local/src/
register: results
- set_fact:
shmmax: "{{ results.stdout | regex_search(shmmaxregexp, '\\1' ) }}"
shmall: "{{ results.stdout | regex_search(shmallregexp, '\\1' ) }}"
vars:
shmmaxregexp: 'shmmax=([^\"]+)'
shmallregexp: 'shmall=([^\"]+)'
- name: sysctl -w kernel.shmmax="{{ shmmax | int }}"
become: true
become_user: root
command: sysctl -w kernel.shmmax="{{ shmmax | int }}"
- name: sysctl -w kernel.shmall="{{ shmall }}"
become: true
become_user: root
command: sysctl -w kernel.shmall="{{ shmall }}"
This is the output
dinesh#dinesh-VirtualBox:~/Documents/remote/Ansible-Playbook/fossology_playbook$ ansible-playbook regex.yml -K -v
Using /etc/ansible/ansible.cfg as config file
BECOME password:
PLAY [fossology_test] ************************************************************************************
TASK [run shell script] **********************************************************************************
changed: [fossology_test] => {"changed": true, "cmd": ["./shm.sh"], "delta": "0:00:00.005912", "end": "2020-03-28 05:25:42.022156", "rc": 0, "start": "2020-03-28 05:25:42.016244", "stderr": "", "stderr_lines": [], "stdout": "kernel.shmmax=4185686016\nkernel.shmall=1021896", "stdout_lines": ["kernel.shmmax=4185686016", "kernel.shmall=1021896"]}
TASK [set_fact] ******************************************************************************************
ok: [fossology_test] => {"ansible_facts": {"shmall": ["1021896"], "shmmax": ["4185686016\nkernel.shmall=1021896"]}, "changed": false}
TASK [sysctl -w kernel.shmmax="0"] ***********************************************************************
changed: [fossology_test] => {"changed": true, "cmd": ["sysctl", "-w", "kernel.shmmax=0"], "delta": "0:00:00.003133", "end": "2020-03-28 05:25:42.574223", "rc": 0, "start": "2020-03-28 05:25:42.571090", "stderr": "", "stderr_lines": [], "stdout": "kernel.shmmax = 0", "stdout_lines": ["kernel.shmmax = 0"]}
TASK [sysctl -w kernel.shmall="[u'1021896']"] ************************************************************
changed: [fossology_test] => {"changed": true, "cmd": ["sysctl", "-w", "kernel.shmall=[u'1021896']"], "delta": "0:00:00.003558", "end": "2020-03-28 05:25:43.071811", "rc": 0, "start": "2020-03-28 05:25:43.068253", "stderr": "sysctl: setting key \"kernel.shmall\": Invalid argument", "stderr_lines": ["sysctl: setting key \"kernel.shmall\": Invalid argument"], "stdout": "kernel.shmall = [u'1021896']", "stdout_lines": ["kernel.shmall = [u'1021896']"]}
PLAY RECAP ***********************************************************************************************
fossology_test : ok=4 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
I am using ansible 2.9.6
dinesh#dinesh-VirtualBox:/$ ansible --version
ansible 2.9.6
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/dinesh/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.17 (default, Nov 7 2019, 10:07:09) [GCC 7.4.0]
As you can very clearly see in the set_fact results dict, the output of regexp_search is a list of matched strings, not the just the capture group. And, because your regex is imprecise, that's why your shmmax is the numbers plus a newline plus the rest of the text.
The accurate regex is shmmax=([0-9]+) because those values aren't "any character except a double quote" it's "any number after the equals sign"
I have a playbook where I am configuring environment variables for multiple hosts.
these are the global vars set in defaults\main.yml:
environment:
http_proxy: blabla
https_proxy: blabla
Now I have a single task where I need to set another environment variable for the python library.
However, when I set the environment var for that single task it overwrites the global environment vars.
- name: some task
command: command
environment:
ENV_VAR: "blabla"
I want the ENV_VAR for the single task to be added on top of the global vars. But is that even possible?
https://docs.ansible.com/ansible/latest/user_guide/playbooks_environment.html
this page didn't give me anything conclusive.
Could I use the with_items option to achieve this possibly?
The following playbook demonstrates a possible implementation for your requirement using the combine filter associated with default. Run this with -v option to see output of shell commands:
---
- name: Parse several results as json strings
hosts: localhost
gather_facts: false
# This will automatically combine `default_env` (default to empty if it does not exists)
# with `more_env` if this latest var is defined somewhere
environment: "{{ default_env | default({}) | combine(more_env | default({})) }}"
vars:
# Define default environment variables
default_env:
http_proxy: blabla
https_proxy: blabla
tasks:
- name: Show some vars
vars:
# Inject more environment variables for this task
more_env:
toto: titi
shell: |-
echo $http_proxy
echo $https_proxy
echo $toto
- name: Same with default env
shell: |-
echo $http_proxy
echo $https_proxy
echo $toto
Which gives:
$ ansible-playbook test.yml -v
PLAY [Default env override] ****************************************************************************************************************************************************************************************************************************
TASK [Show some vars] ***************************************************************************************************************************************************************************************************************************************************
changed: [localhost] => {"changed": true, "cmd": "echo $http_proxy\necho $https_proxy\necho $toto", "delta": "0:00:00.002207", "end": "2020-01-07 09:57:30.178989", "rc": 0, "start": "2020-01-07 09:57:30.176782", "stderr": "", "stderr_lines": [], "stdout": "blabla\nblabla\ntiti", "stdout_lines": ["blabla", "blabla", "titi"]}
TASK [Same with default env] ********************************************************************************************************************************************************************************************************************************************
changed: [localhost] => {"changed": true, "cmd": "echo $http_proxy\necho $https_proxy\necho $toto", "delta": "0:00:00.001875", "end": "2020-01-07 09:57:30.626163", "rc": 0, "start": "2020-01-07 09:57:30.624288", "stderr": "", "stderr_lines": [], "stdout": "blabla\nblabla", "stdout_lines": ["blabla", "blabla"]}
PLAY RECAP **************************************************************************************************************************************************************************************************************************************************************
localhost : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Let me try and explain my need:
As part of the regular deployment of our application, we have a SQL script(which would alter tables, add tables or update, etc) which needs to be run on 3 schemas in a region and 5 schemas in another for example. The application is in AWS and the database is Arora db(RDS)- MySQL. This schema can take anywhere between 30 minutes to 3 hours.
This SQL script needs to be run in parallel and with a delay of 2 minutes between each schema run.
This is what I have achieved till now:
A file having DB details- dbdata.yml
---
conn_details:
- { host: localhost, user: root, password: "Password1!" }
- { host: localhost, user: root, password: "Password1!" }
The playbook:
- hosts: localhost
vars:
script_file: "{{ path }}"
vars_files:
- dbdata.yml
tasks:
- name: shell command to execute script in parallel
shell: |
sleep 30s
"mysql -h {{ item.host }} -u {{ item.user }} -p{{ item.password }} < {{ script_file }} >> /usr/local/testscript.log"
with_items: "{{ conn_details }}"
register: sql_query_output
async: 600
poll: 0
- name: Wait for sql execution to finish
async_status:
jid: "{{ item.ansible_job_id }}"
register: _jobs
until: _jobs.finished
delay: 20 # Check every 20 seconds. Adjust as you like.
retries: 10
with_items: "{{ sql_query_output.results }}"
1st part- executes the script in parallel and this also includes a time gap of 30 seconds before each execution.
2nd part- picks the ansible job id from the registered output and checks if the job is completed or not.
Please note: before including the 30 seconds sleep, this playbook was working fine.
We have following erroneous output upon execution:
ansible-playbook parallel_local.yml --extra-vars "path=RDS_script.sql"
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
PLAY [localhost] **************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [localhost]
TASK [sample command- ansible-playbook my_sqldeploy.yml --extra-vars "path=/home/NICEONDEMAND/bsahu/RDS_create_user1.sql"] ****************************************************************************************
changed: [localhost] => (item={u'host': u'localhost', u'password': u'Password1!', u'user': u'root'})
changed: [localhost] => (item={u'host': u'localhost', u'password': u'Password1!', u'user': u'root'})
TASK [Wait for creation to finish] ********************************************************************************************************************************************************************************
FAILED - RETRYING: Wait for creation to finish (10 retries left).
FAILED - RETRYING: Wait for creation to finish (9 retries left).
failed: [localhost] (item={'ansible_loop_var': u'item', u'ansible_job_id': u'591787538842.77844', 'item': {u'host': u'localhost', u'password': u'Password1!', u'user': u'root'}, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/root/.ansible_async/591787538842.77844'}) => {"ansible_job_id": "591787538842.77844", "ansible_loop_var": "item", "attempts": 3, "changed": true, "cmd": "sleep 30s\n\"mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log\"\n", "delta": "0:00:30.073191", "end": "2019-11-28 17:01:57.632285", "finished": 1, "item": {"ansible_job_id": "591787538842.77844", "ansible_loop_var": "item", "changed": true, "failed": false, "finished": 0, "item": {"host": "localhost", "password": "Password1!", "user": "root"}, "results_file": "/root/.ansible_async/591787538842.77844", "started": 1}, "msg": "non-zero return code", "rc": 127, "start": "2019-11-28 17:01:27.559094", "stderr": "/bin/sh: line 1: mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log: No such file or directory", "stderr_lines": ["/bin/sh: line 1: mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log: No such file or directory"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item={'ansible_loop_var': u'item', u'ansible_job_id': u'999397686792.77873', 'item': {u'host': u'localhost', u'password': u'Password1!', u'user': u'root'}, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/root/.ansible_async/999397686792.77873'}) => {"ansible_job_id": "999397686792.77873", "ansible_loop_var": "item", "attempts": 1, "changed": true, "cmd": "sleep 30s\n\"mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log\"\n", "delta": "0:00:30.120136", "end": "2019-11-28 17:01:58.694713", "finished": 1, "item": {"ansible_job_id": "999397686792.77873", "ansible_loop_var": "item", "changed": true, "failed": false, "finished": 0, "item": {"host": "localhost", "password": "Password1!", "user": "root"}, "results_file": "/root/.ansible_async/999397686792.77873", "started": 1}, "msg": "non-zero return code", "rc": 127, "start": "2019-11-28 17:01:28.574577", "stderr": "/bin/sh: line 1: mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log: No such file or directory", "stderr_lines": ["/bin/sh: line 1: mysql -h localhost -u root -pPassword1! < RDS_script.sql >> /usr/local/testscript.log: No such file or directory"], "stdout": "", "stdout_lines": []}
PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
Any suggestions how to overcome this. Thanks in advance for all the help.
My bad. I had a silly mistake which was creating trouble. I needed to remove "" from the line executing the sql file. Reproducing the correct yaml file below
- hosts: localhost
vars:
script_file: "{{ path }}"
vars_files:
- dbdata.yml
tasks:
- name: sample command- ansible-playbook my_sqldeploy.yml --extra-vars "path=/home/NICEONDEMAND/bsahu/RDS_create_user1.sql"
shell: |
sleep 30s
mysql -h {{ item.host }} -u {{ item.user }} -p{{ item.password }} < {{ script_file }} >> /usr/local/testscript.log
with_items: "{{ conn_details }}"
register: sql_query_output
async: 600
poll: 0
- name: Wait for creation to finish
async_status:
jid: "{{ item.ansible_job_id }}"
register: _jobs
until: _jobs.finished
delay: 20 # Check every 5 seconds. Adjust as you like.
retries: 10
with_items: "{{ sql_query_output.results }}"
Thanks all for the help.
Below is my simple playbook
name: "test"
hosts: webservers
tasks:
- name: Echo my_env_var
shell: "echo $MY_ENV_VARIABLE"
environment:
MY_ENV_VARIABLE: whatever_value
- name: Echo my_env_var again
shell: "echo $MY_ENV_VARIABLE"
register: stdd
- debug: msg={{stdd.stdout_lines}}
My output is always msg:"" or msg: []. Why am i not able to see the value of variable
I took your example and changed it from debug msg to debug var. I also simplified it by only running the task once, and found the error in the process. The environment argument is specific to a task. You aren't including it in your second shell task.
Here's the example I used.
echo.yml
- hosts: localhost
tasks:
- name: Echo my_env_var
shell: "echo $MY_ENV_VARIABLE"
environment:
MY_ENV_VARIABLE: whatever_value
register: stdd
- debug: var=stdd
execution
$ ansible-playbook -c local -i "localhost," echo.yml
PLAY [localhost] **************************************************************
GATHERING FACTS ***************************************************************
ok: [localhost]
TASK: [Echo my_env_var] *******************************************************
changed: [localhost]
TASK: [debug var=stdd] ********************************************************
ok: [localhost] => {
"var": {
"stdd": {
"changed": true,
"cmd": "echo $MY_ENV_VARIABLE",
"delta": "0:00:00.005332",
"end": "2016-07-25 19:42:54.320667",
"invocation": {
"module_args": "echo $MY_ENV_VARIABLE",
"module_complex_args": {},
"module_name": "shell"
},
"rc": 0,
"start": "2016-07-25 19:42:54.315335",
"stderr": "",
"stdout": "whatever_value",
"stdout_lines": [
"whatever_value"
],
"warnings": []
}
}
}
PLAY RECAP ********************************************************************
localhost : ok=3 changed=1 unreachable=0 failed=0
I was given a task to verify some routing entries for all Linux server and here is how I did it using an Ansible playbook
---
- hosts: Linux
serial: 1
tasks:
- name: Check first
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result
changed_when: false
- debug: msg="{{result.stdout}}"
- name: Check second
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result
changed_when: false
- debug: msg="{{result.stdout}}"
You can see I have to repeat same task for each routing entry and I believe I should be able to avoid this. I tried use with_items loop but got following error message
One or more undefined variables: 'dict object' has no attribute 'stdout'
is there a way to register variable for each command and loop over them one by one ?
Starting in Ansible 1.6.1, the results registered with multiple items are stored in result.results as an array. So you can use result.results[0].stdout and so on.
Testing playbook:
---
- hosts: localhost
gather_facts: no
tasks:
- command: "echo {{item}}"
register: result
with_items: [1, 2]
- debug:
var: result
Result:
$ ansible-playbook -i localhost, test.yml
PLAY [localhost] **************************************************************
TASK: [command echo {{item}}] *************************************************
changed: [localhost] => (item=1)
changed: [localhost] => (item=2)
TASK: [debug ] ****************************************************************
ok: [localhost] => {
"var": {
"result": {
"changed": true,
"msg": "All items completed",
"results": [
{
"changed": true,
"cmd": [
"echo",
"1"
],
"delta": "0:00:00.002502",
"end": "2015-08-07 16:44:08.901313",
"invocation": {
"module_args": "echo 1",
"module_name": "command"
},
"item": 1,
"rc": 0,
"start": "2015-08-07 16:44:08.898811",
"stderr": "",
"stdout": "1",
"stdout_lines": [
"1"
],
"warnings": []
},
{
"changed": true,
"cmd": [
"echo",
"2"
],
"delta": "0:00:00.002516",
"end": "2015-08-07 16:44:09.038458",
"invocation": {
"module_args": "echo 2",
"module_name": "command"
},
"item": 2,
"rc": 0,
"start": "2015-08-07 16:44:09.035942",
"stderr": "",
"stdout": "2",
"stdout_lines": [
"2"
],
"warnings": []
}
]
}
}
}
PLAY RECAP ********************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0
A slightly different situation, which took a while to figure out. If you want to use the results of multiple items, but for changed_when, then the register variable will not have a var.results! Instead, changed_when, is evaluated for each item, and you can just directly use the register var.
Simple example, which will result in changed: false:
- action: command echo {{item}}
register: out
changed_when: "'z' in out.stdout"
with_items:
- hello
- foo
- bye
Another example:
- name: Create fulltext index for faster text searches.
mysql_db: name={{SO_database}} state=import target=/tmp/fulltext-{{item.tableName}}-{{item.columnName}}.sql
with_items:
- {tableName: Posts, columnName: Title}
- {tableName: Posts, columnName: Body}
- {tableName: Posts, columnName: Tags}
- {tableName: Comments, columnName: Text}
register: createfulltextcmd
changed_when: createindexcmd.msg.find('already exists') == -1
Finally, when you do want to loop through results in other contexts, it does seem a bit tricky to programmatically access the index as that is not exposed. I did find this one example that might be promising:
- name: add hosts to known_hosts
shell: 'ssh-keyscan -H {{item.host}}>> /home/testuser/known_hosts'
with_items:
- { index: 0, host: testhost1.test.dom }
- { index: 1, host: testhost2.test.dom }
- { index: 2, host: 192.168.202.100 }
when: ssh_known_hosts.results[{{item.index}}].rc == 1
Posting because I can't comment yet
Relating to gameweld's answer, since Ansible 2.5 there's another way to accessing the iteration index.
From the docs:
Tracking progress through a loop with index_var
New in version 2.5.
To keep track of where you are in a loop, use the index_var directive
with loop_control. This directive specifies a variable name to contain
the current loop index:
- name: count our fruit
debug:
msg: "{{ item }} with index {{ my_idx }}"
loop:
- apple
- banana
- pear
loop_control:
index_var: my_idx
This also allows you to gather results from an array and act later to the same array, taking into account the previous results
- name: Ensure directories exist
file:
path: "{{ item }}"
state: directory
loop:
- "mouse"
- "lizard"
register: reg
- name: Do something only if directory is new
debug:
msg: "New dir created with name '{{ item }}'"
loop:
- "mouse"
- "lizard"
loop_control:
index_var: index
when: reg.results[index].changed
Please note that the "mouse lizard" array should be exactly the same
If what you need is to register the output of two commands separately, use different variable names.
---
- hosts: Linux
serial: 1
tasks:
- name: Check first
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result0
changed_when: false
- debug: msg="{{result0.stdout}}"
- name: Check second
command: /sbin/ip route list xxx.xxx.xxx.xxx/24
register: result1
changed_when: false
- debug: msg="{{result1.stdout}}"