Ansible: Unable to run docker compose in an ansible playbook - ansible

I appear to be unable to run docker compose tasks in an ansible playbook. I get stuck in a loop.
The first error I get when running sudo ansible-playbook playbook.yml is the following
fatal: [10.0.3.5]: FAILED! => {"changed": false, "msg": "Unable to load docker-compose. Try `pip install docker-compose`. Error: No module named compose"}
so I remote to that machine and did sudo pip install docker-compose and try running the playbook again. This time I get...
fatal: [10.0.3.5]: FAILED! => {"changed": false, "msg": "Cannot have both the docker-py and docker python modules installed together as they use the same namespace and cause a corrupt installation. Please uninstall both packages, and re-install only the docker-py or docker python module"}
so I try uninstalling docker python...
sudo uninstall docker python
Then I get the following when attempting to run the playbook again
fatal: [10.0.3.5]: FAILED! => {"changed": false, "msg": "Failed to import docker-py - No module named docker. Try `pip install docker-py`"}
However this is already install on the machine, as when I run sudo pip install docker-py I see the following...
Requirement already satisfied (use --upgrade to upgrade): docker-py in /usr/local/lib/python2.7/dist-packages
Cleaning up...
Does anyone know how to escape this loop and successfully get an ansible playbook that uses docker-compose to run?
The machine os is linux 14.04
Thanks,

What worked for me was to first uninstall everything docker related in the virtualenv for Ansible.
pip uninstall docker docker-py docker-compose
And then install the docker-compose module, which will install the docker module as well as a dependency.
pip install docker-compose
The Ansible docker module will try to import docker, which will also succeed with the docker module, and as such not provide an error with the misleading instruction to install docker-py.

I had the just had the same error message while trying to run sudo ansible playbook-with-docker.yaml
{"changed": false, "msg": "Unable to load docker-compose. Try `pip install docker-compose`. Error: No module named compose"}
I took me about 2 hours to figure out that in Linux pip install is not the same as sudo pip install (quite obvious, once you know what's happening ).
So in case someone has the same issue - make sure you're you are running everything consistently either as sudo or not sudo, but don't mix stuff :)
...and use sudo pip list | grep docker to verify.

As already stated in other answers, docker-compose python module is missing.
You can install it manually as previous answers indicate or you can use a "pure" Ansible solution that is to install via a task.
For that, use ansible.builtin.pip to install docker-compose module (you can add more modules to install if needed in the same task).
- hosts: all
gather_facts: no
tasks:
- name: Install docker-compose python package
ansible.builtin.pip:
name: docker-compose
Reference: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/pip_module.html

To add more context to Thermostat's answer
I was using pip3 and not pip with the following:
Ubuntu 20.04
Ansible 2.10.7
Python 3.8.10
pip 20.0.2 (pip3)
Here's how I fixed mine:
So first I ran the command to remove all existing copies of docker, docker-py and docker-compose python libraries:
pip3 uninstall docker docker-py docker-compose
And then ran the command below to install the python docker-compose library alongside the python docker library
pip3 install docker-compose
That's all.

Related

ansible returns with "Failed to import the required Python library (Docker SDK for Python: docker (Python >= 2.7) or docker-py (Python 2.6))

I am running myserver in ubuntu:
+ sudo cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
I use ansible and when I run it I get the following error:
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (Docker SDK for Python: docker (Python >= 2.7) or docker-py (Python 2.6)) on dd63315fad06's Python /usr/bin/python. Please read module documentation and install in the appropriate location, for example via `pip install docker` or `pip install docker-py` (Python 2.6). The error was: No module named docker"}
when I run
python -c "import sys; print(sys.path)"
I see:
['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/dist-packages/pip-19.2.2-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/fasteners-0.15-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/monotonic-1.5-py2.7.egg', '/usr/lib/python2.7/dist-packages']
and python versions are as follows:
+ python --version
Python 2.7.12
+ python3 --version
Python 3.5.2
Then as I see everything is fine and I am not sure why I get
"Failed to import the required Python library (Docker SDK for Python: docker (Python >= 2.7) or docker-py (Python 2.6)) on dd63315fad06's Python /usr/bin/python. Please read module documentation and install in the appropriate location, for example via `pip install docker` or `pip install docker-py` (Python 2.6). The error was: No module named docker"
in ansible?
It appears that you don't have the docker module installed.
You will need to install it via your system package manager (apt install python-docker, for example), or using pip (pip install docker).
If you have multiple Python versions, make sure that you've installed the docker module into the version that Ansible is using.
Here, in May 2021 for Ubuntu 20.04 you need to run apt install python3-docker because no python 2.x is shipped by default anymore
I have faced the same issue for the Ansible docker-compose module. I was able to fix it by selecting python3 for those tasks.
Before (Not working)
- name: Create docker service services
docker_compose:
project_src: /root/
become: true
After (Working)
We can get the python location by $which python3
- name: Create fleuntd services
docker_compose:
project_src: /root/
become: true
vars:
ansible_python_interpreter: /bin/python3
I started getting this same error in April 2021, with the release of Version 5.0 of the Docker SDK For Python. The error message was almost the exact same as the original question, with the only difference being that at the end of the error message was one of the statements:
The error was: No module named parse
or
The error was: No module named selectors
This was ultimately due to the older version of pip that Ansible was using, incorrectly installing a Python3 library on a Python2.7 setup. The fix was to pin the version of the docker Python library to something earlier than Version 5.0, and the 'websocket-client' library to something earlier than Version 1.0:
- name: Install Docker SDK for Python
pip:
name: "docker<5"
become: yes
- name: Setup more docker dependencies
pip:
name: "websocket-client<1"
become: yes
Alternatively, this set of commands would have also worked, given that Python2 was still in use:
pip install docker<5
pip install websocket-client<1
Once these older versions of the Docker SDK for Python and Websocket Client were installed, Ansible was able to again successfully manage Docker on my behalf.
In my case (Ubuntu 20 with installed docker) these commands was required
apt update
apt install python3 python3-pip
pip3 install docker docker-compose
Detailed requirements:
https://docs.ansible.com/ansible/latest/collections/community/general/docker_compose_module.html

Getting error when using Ansible playbook. error message: "msg": "boto required for this module"

I'm trying to execute a playbook for starting an ec2 instance. Have boto3 and python installed on both controlling machine and client node. But it still gives me the error saying:
"msg": "boto required for this module"
Have tried everything suggested here: Ansible ec2: "boto required for this module". But nada.
You are missing the python boto package on the host that is executing the Ansible playbooks.
You can install it via the python package manager
$ pip install boto --user
If you are using linux you can also install it via the package manager:
# RHEL/CentOS
$ sudo yum -y install python-pip
# Fedora
$ sudo dnf -y install python-pip
Package dependencies are listed on the Ansible website for each module. For example for the ec2 module the Ansible host dependencies are:
python >= 2.6
boto
It is worth noting that that boto, botocore, and boto3 are all separate packages.

Ansible DistributionNotFound jinja2

When I run the following ansible command
ansible webservers -m ping
It reports "DistributionNotFound jinja2<2.9". I then updated jinja to 2.9.4 :
pip install --upgrade jinja2
But ansible still reports the same error when running it. Does anybody can help? Thanks!
I eventually figured out the solution, which is to update distribute rather than jinja2. Then reinstall ansible (my ansible was initially installed via pip):
pip install -U distribute
pip uninstall ansible
pip install ansible
Thanks to #KonstantinSuvorov and #techraf for your help.

Ansible installed from source - what should my library setting be?

I have installed Ansible from source as per the instructions at:
http://docs.ansible.com/ansible/intro_installation.html
However when I try to use any command other than script, I get the following error:
fatal: [...]: FAILED! => {"failed": true, "msg": "ERROR! The module get_url was not found in configured module paths"}
If the source Ansible directory is /home/cloud/ansible, and I have done a make install, what should I set the library path setting to in ansible.cfg?
As #udondan says, make sure you used:
git clone https://github.com/ansible/ansible --recursive
to clone the Ansible repo, and then run:
cd ./ansible
source ./hacking/env-setup
You don't need to run make install.
The machine that has Ansible running on it needs some other Python modules too, they are listed at the bottom of the http://docs.ansible.com/ansible/intro_installation.html#running-from-source section. Best to install the pip Python package manager with:
sudo easy_install pip
and then install the required packages:
sudo pip install paramiko PyYAML Jinja2 httplib2 six

Ansible "Permission denied" when trying to install/check a gem

I recently decided to switch my Ansible deployment to install Ruby via rbenv rather than from apt-get via ruby1.9.1. Now I'm getting an error when trying to install the gem via Ansible.
TASK: [nginx | s3cp gem] ******************************************************
failed: [staging.myapp.com] => {"cmd": ["/usr/local/bin", "query", "-n", "^s3cp$"], "failed": true, "item": "", "rc": 13}
msg: [Errno 13] Permission denied
FATAL: all hosts have already failed -- aborting
Ansible playbook entry for this command:
- name: s3cp gem
gem: name=s3cp state=present executable=/usr/local/bin
I have sudo set to "yes" in a higher-level call to this playbook part. So I am not sure why it's tripping up. I also am able to login with the same user used for Ansible and navigate to that directory and also install this gem.
It was working fine when I was using apt-get to install ruby1.9.1. Any ideas?
This is deployed to an Ubuntu 13.04 server, by the way.
MORE INFORMATION:
Apparently it's not just tripping up on s3cp. I skipped that one and went on to another command to install bundler. This command also would not work (failed in the same way). I am wondering if there's a default ruby that's conflicting with the rbenv ruby (though, which ruby when ssh'ed in is yielding the expected rbenv directory).
MORE-MORE INFO:
I tried to install ruby via rvm instead. I had the same error. :(
What happens when you run ansible with -vvvv? It should provide full verbose output of the tasks, hopefully including any errors that it encounters. With a bit of luck it will show you what the problem is.
Another thing to check is what user you're running the tasks as. How do you have the following parameters set at the top of your play (or do you not specify any of these)?
- hosts: myhosts
user: someuser
sudo: True
sudo_user: another_user
As far as I know, the gem ansible module is not rbenv-aware. This means that when you call the gem module, it will try to install a gem system wide. This, of course, will fail if you're not acting as root on your node.
To install a gem with rbenv, you must use rbenv's gem shim. The only way to do this is to be able to trigger rbenv init by sending the command thru bash :
- name: Install Bundler
command: bash -lc "gem install bundler"
This has been already adressed here :
Install Bundler gem using Ansible

Resources