I want to know if it's possible deploy 6 different tasks in parallel (3|3) for 2 different hosts in ansible 2.2.0, because i'm having issues run them in parallel with one playbook, they just run sequencial.
You can do this with async tasks. Something like this:
- hosts: host1,host2
tasks:
- command: /task1a.sh
async: 1000
poll: 0
register: task1a
when: inventory_hostname == 'host1'
- command: /task1b.sh
async: 1000
poll: 0
register: task1b
when: inventory_hostname == 'host2'
# ... repeat for tasks 2 and 3
- command: /check_done.sh
register: check_done
until: check_done | success
retries: 30
This will start task1a.sh and task1b.sh as async jobs on host1 and host2 respectively and then wait for check_done.sh to return zero.
You can see strategy: free in a combination with when: inventory_hostname == 'hostXX', for example:
- hosts: all
strategy: free
tasks:
- block:
- name: task1
...
- name: task2
...
- name: task3
...
when: inventory_hostname == 'host1'
- block:
- name: task4
...
- name: task5
...
- name: task6
...
when: inventory_hostname == 'host2'
strategy: free means all tasks will run on all hosts in parallel as fast as they can, not waiting for each task to finish on each host. Which task is going to be run on which host is decided by the when clause.
I used block here, so that you only need to write one when for each block, but you don't have to use it, you can write a when on each task.
Related
I am implementing a role in ansible where I need to:
Start an application (retry max of 3 times). If the app was successfully started, it will return "OK", otherwise it will return "NOK".
If the app was not successfully started (NOK), we should try to delete a xpto directory (max 3 times - after each NOK status). After this 3 times, (if the app was not successfully started) we should get fail status, and we should abort execution.
If the app starts OK, there is no need to clean the directory, and we are ready to run the app.
We need to be aware of this:
Each time I try to start the app, if I get status "NOK" I must run a task to delete the xpto directory.
We can retry up to 3 times to start the app (and to delete the directory).
Each time we try to start the app with NOK status we must run the task to delete the directory.
If at any attempt of starting the app we get status OK (app started with success), we don't want to run task to delete the directory - In this case we should move to last task to run the app.
The role has only this 3 tasks (start app, delete directory, run the app)
For now I have only this with where I am missing a lot of the mentioned features:
---
- name: Start app
command: start app
register: result
tags: myrole
- name: Delete directory xpto if app didn't started ok
command: rm -rf xpto
when:
- result.stdout is search("NOK")
tags: myrole
- name: Run the application
command: app run xpto
when:
- result.stdout is search("OK")
tags: myrole
I have been pointed to an other question with a response which allows me to implement the 3 retries with abort option.
I am still missing the way to implement the option if the app starts ok (task1) and proceed directly to run the app (task3) (not going throw task2) and I don't know where to start.
Building-up on the response you have been pointed to (which was inspired by a blog post on dev.to), adapting to your use case and adding some good practice.
start_run_app.yml would contain the needed tasks that can be retried:
---
- name: Group of tasks to start and run the application which could fail
block:
- name: Increment attempts counter
ansible.builtin.set_fact:
attempt_number: "{{ attempt_number | d(0) | int + 1 }}"
- name: Start the application
ansible.builtin.command: start app
register: result_start
failed_when: result_start.rc != 0 or result_start.stdout is search('NOK')
# If we get here then application started ok above
# register run result to disambiguate possible failure
# in the rescue section
- name: Run the application
ansible.builtin.command: app run xpto
register: result_run
rescue:
- name: Fail/stop here if application started but did not run
ansible.builtin.fail:
msg: "Application started but did not run. Exiting"
when:
- result_run is defined
- result_run is failed
- name: Delete directory xpto since app didn't start ok
ansible.builtin.command: rm -rf xpto
- name: "Fail if we reached the max of {{ max_attempts | d(3) }} attempts"
# Default will be 3 attempts if max_attempts is not passed as a parameter
ansible.builtin.fail:
msg: Maximum number of attempts reached
when: attempt_number | int == max_attempts | int | d(3)
- name: Show number of attempts
ansible.builtin.debug:
msg: "group of tasks failed on attempt {{ attempt_number }}. Retrying"
- name: Add delay if configured
# no delay if retry_delay is not passed as parameter
ansible.builtin.wait_for:
timeout: "{{ retry_delay | int | d(omit) }}"
when: retry_delay is defined
- name: Include ourselves to retry.
ansible.builtin.include_tasks: start_run_app.yml
And you can include this file like so (example for a full playbook, adapt to your exact need).
---
- name: Start and run my application
hosts: my_hosts
tasks:
- name: Include retry-able set of tasks to start and run application
ansible.builtin.include_tasks: start_run_app.yml
vars:
max_attempts: 6
retry_delay: 5
Considering that none of the tasks shown have any error handling, it seems that the scripts are all returning the exit code "0"; it would be better that in the logic where "action terminated" is printed to the output, it would also change the exit code to a custom value that then can be included in the Ansible logic:
# changes in the shell script
...
echo "action terminated"
exit 130
...
that way the task 2 can be set with
- name: Task2
command: "{{ home }}/script2.sh"
when:
- result.rc == 130
tags: role1
After the execution of the task2, include an additional task that retries task1
- name: Task2.5
command: "{{ home }}/script1.sh"
register: result2
until: "result2.rc != 130"
ignore_errors: yes
retries: 3
delay: 5
when:
- result.rc == 130
tags: role1
- name: Fail if script1 failed after all the attempts
fail:
msg: "script1 could not be completed"
when:
- result.rc == 130
- result2.failed
tags: role1
note that the when evaluates if the first attempt failed, as the register will keep track the status of the task in a different variable, this one is used in the evaluation of until. The Fail task will be executed only if both attempts were unsuccessful.
EDIT
If changing the exit code is not possible, you need to replace that condition to the search by text
- name: Task1
command: "{{ home }}/script1.sh"
register: result
tags: role1
- name: Task2
command: "{{ home }}/script2.sh"
when:
- result.stdout is search("action terminated")
tags: role1
- name: Task2.5
command: "{{ home }}/script1.sh"
register: result2
until: "'action terminated' not in result.stdout"
ignore_errors: yes
retries: 3
delay: 5
when:
- result.stdout is search("action terminated")
tags: role1
- name: Exit the execution if script1 failed after all the attempts
fail:
msg: "script1 could not be completed"
when:
- result.stdout is search("action terminated")
- result2.failed
tags: role1
- name: Task3
command: "{{ home }}/script3.sh"
tags: role1
When a task failed on any host on ansible, it should not be trigger again by the next task. When a task failed on host, the host is removed from the current bash of ansible and the rest of tasks will not be run on that host. If you want to try a task for n, you will need to use the until and loop.
- name: Task1
command: "{{ home }}/script1.sh"
register: result
tags: role1
- name: Task2
command: "{{ home }}/script2.sh"
when:
- result.stdout is search("action terminated")
tags: role1
- name: Task1 again
command: "{{ home }}/script1.sh"
register: result2
tags: role1
until: result2.stdout is search("action terminated")
retries: 3
- name: Task2 Again
command: "{{ home }}/script2.sh"
when:
- result2.stdout is not search("not action terminated")
- result2.stdout is search("action terminated")
tags: role1
- name: Task3
command: "{{ home }}/script3.sh"
tags: role1
when:
- result.stdout is search("not action terminated")
But you can only trigger a task by using the handler. But the playbook is going from top tasks to down and never go back on a previous tasks.
I have the following two playbooks / tasks files. I want to re-run the children tasks until the result is neither changed nor failed, but max 6 times.
I have the impression that the until statement is simply ignored with the import statement.
The child file is executed only once, with no errors or failures.
I inserted my test task directly in the until tasks in the parent file -> everything works fine.
But I need to use more than one child task (run the main task and then restart).
I know that you can't use until-loop with blocks and include_tasks don't work with until either. But I read the documentation as saying that import_tasks and until should work together (https://docs.ansible.com/ansible/latest/collections/ansible/builtin/import_tasks_module.html#attributes)
Is this behavior correct / intended or am I doing something wrong? If this behavior is intended, how could I solve my problem?
playbook.yaml
- name: "make this working"
hosts: mygroup
tasks:
- name: rerun up to 6 times if not everything is ok
until: (not result_update.changed) and (not result_update.failed)
retries: 5
ansible.builtin.import_tasks: "./children.yaml"
children.yaml
- name: shell1
ansible.windows.win_shell: echo "test1"
register: result_update
# changed_when: false
- name: shell2
ansible.windows.win_shell: echo "test2"
I couldn't solve this problem with the until statement. Instead I build my own loop system which is clearly not really ansible like and not an ideal solution but it works fine (at least for my needs).
playbook.yaml
- name: "make this working"
hosts: mygroup
tasks:
- name: create variable for maximum tries
ansible.builtin.set_fact:
tries_max: 10
- name: create variable for counting tries
ansible.builtin.set_fact:
tries_counter: 1
- name: run this ({{ tries_counter }} / {{ tries_max }})
ansible.builtin.include_tasks: "./children.yaml"
children.yaml
- name: shell1
ansible.windows.win_shell: echo "test1"
register: result_update
# changed_when: false
- name: shell2
ansible.windows.win_shell: echo "test2"
- name: increase try counter by 1
ansible.builtin.set_fact:
tries_counter: "{{ (tries_counter | int) + 1 }}"
- name: run tasks again if not everything is ok ({{ tries_counter }} / {{ tries_max }})
when: ((result_update.changed) or (result_update.failed)) and ((tries_counter | int) <= (tries_max | int))
ansible.builtin.include_tasks: "./children.yaml"
Real scenario is,
I have n hosts in a inventory group and playbook has to run a specific *command for the specific inventory hostname(done with ansible when condition statement), but whenever the condition met and I need to register a variable for the above *command result.
so this variable creation should be done dynamically and these created variable should be appended into a list and then at end of the same playbook by passing the list to a loop I have to check the job async_status.
So could some one help me here?
tasks:
-name:
command:
when: invenory_hostname == x
async: 360
poll:0
regsiter: "here dynamic variable"
-name:
command:
when: invenory_hostname == x
async: 360
poll:0
regsiter: "here dynamic variable"
-name:
command:
when: invenory_hostname == x
async: 360
poll:0
regsiter: "here dynamic variable" #his will continue based on the requirments
-name: collect the job ids
async_status:
jid:{item}
with_items:"list which has all the dynamically registered variables"
If you can write this as a loop instead of a series of independent tasks this becomes much easier. E.g:
tasks:
- command: "{{ item }}"
register: results
loop:
- "command1 ..."
- "command2 ..."
- name: show command output
debug:
msg: "{{ item.stdout }}"
loop: "{{ results.results }}"
The documentation on "Registering variables with a loop" discusses what the structure of results would look like after this task executes.
If you really need to write independent tasks instead, you could use the
vars lookup to find the results from all the tasks like this:
tasks:
- name: task 1
command: echo task1
register: task_result_1
- name: task 2
command: echo task2
register: task_result_2
- name: task 3
command: echo task3
register: task_result_3
- name: show results
debug:
msg: "{{ item }}"
loop: "{{ q('vars', *q('varnames', '^task_result_')) }}"
loop_control:
label: "{{ item.cmd }}"
You've updated the question to show that you're using async tasks, so
that changes things a bit. In this example, we use an until loop
that waits for each job to complete before checking the status of the
next job. The gather results task won't exit until all the async
tasks have completed.
Here's the solution using a loop:
- hosts: localhost
gather_facts: false
tasks:
- name: run tasks
command: "{{ item }}"
async: 360
poll: 0
register: task_results
loop:
- sleep 1
- sleep 5
- sleep 10
- name: gather results
async_status:
jid: "{{ item.ansible_job_id }}"
register: status
until: status.finished
loop: "{{ task_results.results }}"
- debug:
var: status
And the same thing using individual tasks:
- hosts: localhost
gather_facts: false
tasks:
- name: task 1
command: sleep 1
async: 360
poll: 0
register: task_result_1
- name: task 2
command: sleep 5
async: 360
poll: 0
register: task_result_2
- name: task 3
command: sleep 10
async: 360
poll: 0
register: task_result_3
- name: gather results
async_status:
jid: "{{ item.ansible_job_id }}"
register: status
until: status.finished
loop: "{{ q('vars', *q('varnames', '^task_result_')) }}"
- debug:
var: status
I am dynamically creating the hostgroup by site and it did well with below recipe.
- name: "generate batches from batch_processor"
batch_processor:
inventory: "{{ inventory_url }}"
ignore_applist: "{{ ignore_applist | default('None') }}"
ignore_podlist: "{{ ignore_podlist | default('None') }}"
register: batch_output
- name: "Creating batches by site"
include: batch_formatter.yml hostkey="{{ 'batch_' + item.key }}" hostlist="{{ item.value | list | join(',') }}"
with_dict: "{{ batch_output.result['assignments'] }}"
In my playbook, i have like this
- hosts: batch_*
serial: 10
gather_facts: False
tasks:
- include: roles/deployment/tasks/dotask.yml
I initially had strategy: free but in some reason it didn't pickup parallel. Currently i am using all batch hosts 10 at time to deploy.
I am thinking of below items
batch_site1 - 10 in parallel
batch_site2 - 10 in parallel
batch_site3 - 10 in parallel
But in the playbook, i don't want to specify the hostgroup execution by site as they are dynamic. Sometime, we will have siteX and sometime it wont be there. Please suggest the best approach.
- hosts: batch_*
gather_facts: False
strategy: free
tasks:
- include: roles/deployment/tasks/dotask.yml
when: "'batch_site1' not in group_names"
async: 600
poll: 0
register: job1
- name: Wait for asynchronous job to end
async_status:
jid: '{{ job1.ansible_job_id }}'
register: job_result1
until: job_result1.finished
retries: 30
- include: roles/deployment/tasks/dotask.yml
when: "'batch_site2' not in group_names"
register: job2
async: 600
poll: 0
- name: Wait for asynchronous job to end
async_status:
jid: '{{ job2.ansible_job_id }}'
register: job_result2
until: job_result2.finished
retries: 30
- include: roles/deployment/tasks/dotask.yml
when: "'batch_site3' not in group_names"
register: job3
async: 600
poll: 0
- name: Wait for asynchronous job to end
async_status:
jid: '{{ job3.ansible_job_id }}'
register: job_result3
until: job_result3.finished
retries: 30
This config slightly working on my purpose, however i couldn't pull the async_status to show better results in screen.
{"msg": "The task includes an option with an undefined variable. The error was: 'job1' is undefined\n\nThe error appears to have been in '/../../../../play.yml': line 12, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Wait for asynchronous job to end\n ^ here\n"}
not sure why ?
This is something I often do
- name: check whether ready
shell: do_something_complex
changed_when: false
ignore_errors: true
register: result
- name: make it ready
when: result.rc != 0
It is messy when there are many tasks after that, which perform the same check.
I would like instead to wrap the result somehow, and use it like this:
- name: do this if ready
when: isReady
- name: do that if not ready
when: not isReady
How can I do that? (Preferably without intermediate dummy tasks that exist just to set variables.)
That's the thing that import_tasks:, or even custom modules, are designed to solve:
# tasks/make_ready.yml
- name: check whether ready
shell: '{{ check_ready_shell }}'
changed_when: false
# you'll want to be **very careful** using this
ignore_errors: true
register: isReady
- name: make it ready
shell: '{{ make_ready_shell }}'
when: isReady.rc != 0
then, in your main playbook:
- import_tasks: make_ready.yml
vars:
check_ready_shell: echo 'hello from checking'
make_ready_shell: systemctl start-the-thing-or-whatever
- import_tasks: make_ready.yml
vars:
check_ready_shell: echo 'hello from other checking'
make_ready_shell: systemctl start-the-other-thing