I am getting an error trying to execute docker-compose in a ansible playbook.
I am running from a Mac OSX, but the destination node is a AWS ec2 (Ubuntu 20.04)
On Ubuntu 20.04, I installed the following
sudo apt-get update -y && sudo apt-get install git python3 python3-setuptools
python3-pip software-properties-common docker docker.io unzip acl -y
sudo apt-add-repository -y --update ppa:ansible/ansible
sudo apt-get install ansible -y
sudo apt install python3-pip -y
sudo apt-get install python3-docker
pip3 install docker
pip3 install docker-compose
In the main.yml
name: Run "docker-compose up"
community.docker.docker_compose:
project_src: /data
project_name: test
register: output
I am getting the following error
fatal: [ndr2]: FAILED! => {"changed": false, "msg": "Unable to load docker-compose. Try `pip install docker-compose`. Error: Traceback (most recent call last):\n File \"/tmp/ansible_community.docker.docker_compose_payload_566twi2g/ansible_community.docker.docker_compose_payload.zip/ansible_collections/community/docker/plugins/modules/docker_compose.py\", line 494, in <module>\nModuleNotFoundError: No module named 'compose'\n"}
Any help is appreciated.
I had to run sudo apt install docker-compose -y on the target host, instead of installing docker-compose via pip3. Not sure why it had to be installed this way.
Related
I am trying to establish a connection from Azure pipelines to oracle using cx_oracle.
I'm using a Dockerfile:
FROM python:3.9-slim-buster
RUN apt-get update && apt-get install -y git && apt-get install telnet
RUN apt-get install -y gcc
RUN apt-get -y install wget
RUN apt -y install unzip
RUN wget https://download.oracle.com/otn_software/linux/instantclient/215000/instantclient-basic-linux.x64-21.5.0.0.0dbru.zip
RUN unzip instantclient-basic-linux.x64-21.5.0.0.0dbru.zip
RUN apt-get -y install libaio1
RUN sh -c "echo /instantclient_21_5 > /etc/ld.so.conf.d/oracle-instantclient.conf"
RUN ldconfig
RUN apt -y remove unzip
RUN rm instantclient-basic-linux.x64-21.5.0.0.0dbru.zip
CMD ["/bin/sh"]
Im doing automated tests using robotframework, i just updated to cx-Oracle 8.3.0. I still get the error
DatabaseError: DPI-1047:
Cannot locate a 64-bit Oracle Client library:
"libclntsh.so: cannot open shared object file:
No such file or directory".
I'm running a gitlab ci pipeline with a Centos image.
The pipeline has a before script that runs a set of commands.
gitlab-ci.yaml
variables:
WORKSPACE_HOME: '$CI_PROJECT_DIR'
DELIVERY_HOME: delivery
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
default:
image: centos:latest
cache:
paths:
- .cache/pip
before_script:
- chmod u+x devops/scripts/*.sh
- devops/scripts/install-ci.sh
- python3 -m ensurepip --upgrade
- cp .env.docker.dist .env
- pip3 install --upgrade pip
- pip3 install -r requirements.txt
install-ci.yaml
sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-Linux-* &&\
sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-Linux-*
yum -y update
yum -y install gcc gcc-c++ make
yum -y install python3.8
yum install python3-setuptools
yum -y groupinstall "Development Tools"
yum -y install python3-devel
yum -y install postgresql-server
yum -y install postgresql-devel
yum -y install postgresql-libs
yum -y install python3-pip
timedatectl set-timezone Europe/Paris
yum -y install sqlite-devel
The issue is that everytime I run the ci pipeline it takes time to install centos and all it's packages.
Is there a way to avoid this ? or cache this operation somewhere ?
You could create your own image in which all your dependencies are installed and use this in your job instead of installing the dependencies all over again. I would create a dedicated project on your gitlab instance, something like "centos-python-postgress" and within this project you create a Dockerfile in which you install everything you need. (You can either copy your install-ci.sh or RUN the commands directly within your dockerfile) :
FROM centos:latest
RUN sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-Linux-* && sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-Linux-*
RUN yum -y update
RUN yum -y install gcc gcc-c++ make
...
You can now either build the Dockerfile on your machine and push it manually to the container registry in this project or you create a CI Pipeline that builds and pushes that image automatically:
stages:
- build
stage: build
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
- >-
/kaniko/executor
--context "${CI_PROJECT_DIR}"
--dockerfile "${CI_PROJECT_DIR}/Dockerfile"
--destination "${CI_REGISTRY_IMAGE}:latest"
Now, Instead of using centos:latest in your origin project/job, you can use your own image:
variables:
WORKSPACE_HOME: '$CI_PROJECT_DIR'
DELIVERY_HOME: delivery
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
default:
image: registry.gitlab.com/snowfire/centos-python-postgress:latest
cache:
paths:
- .cache/pip
before_script:
- ...
docker image builds successfully. But when actually I run docker image using docker run command to check if my application works fine. I got the below issue.
Please need guidance over this issue. Thanks a lot
issue:
sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found (0) (SQLDriverConnect)")
(Background on this error at: http://sqlalche.me/e/dbapi)
connection string:
def connectiondb():
params = urllib.parse.quote_plus(
"DRIVER={SQL Server};SERVER=servername;DATABASE=dbname;UID=username;PWD=password")
engine = sqlalchemy.create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
metadata = MetaData(engine)
con = engine.connect()
dockerfile:
FROM python:latest
Bundle app source
COPY . /app
WORKDIR /app
Update
RUN pip install --upgrade pip
RUN apt-get update &&
apt-get install -y apt-transport-https &&
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - &&
curl https://packages.microsoft.com/config/debian/9/prod.list > /etc/apt/sources.list.d/mssql-release.list &&
apt-get update &&
ACCEPT_EULA=Y apt-get install msodbcsql17 unixodbc-dev -y
RUN apt-get install libssl-dev
RUN pip install pyodbc==4.0.28
RUN pip install numpy==1.18.1
RUN pip install scikit-multiflow==0.4.1
RUN pip install Cython
RUN pip install scikit-learn==0.21.3
RUN pip install -r requirements.txt
EXPOSE 5000
CMD python ./app.py
return con, metadata,engine
I have installed AWX using the galaxy role obtained with ansible-galaxy install geerlingguy.awx.
It installs the devel branch of awx, however jobs still use the stable version of ansible.
Is there a way to control the version of ansible used by jobs when running playbooks from AWX?
I solved the issue leveraging virtualenvs as per an hint from https://groups.google.com/forum/#!topic/awx-project/AlWbCrlpIC0
So it was a matter of creating a virtualenv on docker containers and enabling it for the organization:
docker exec -it awx_web bash
virtualenv /var/lib/awx/venv/ansible27
yum install -y gcc
yum install -y python-devel
/var/lib/awx/venv/ansible27/bin/pip install python-memcached psutil
/var/lib/awx/venv/ansible27/bin/pip install -U "ansible == 2.7.0"
exit
docker exec -it awx_task bash
virtualenv /var/lib/awx/venv/ansible27
yum install -y gcc
yum install -y python-devel
/var/lib/awx/venv/ansible27/bin/pip install python-memcached psutil
/var/lib/awx/venv/ansible27/bin/pip install -U "ansible == 2.7.0"
exit
curl -u 'admin:password' -X PATCH -H 'Content-Type: application/json' http://127.0.0.1/api/v2/organizations/MYORG/ -d '
{
"custom_virtualenv": "/var/lib/awx/venv/ansible27"
}
'
Trying to build a simple ubuntu apache web server docker image, I get an error while the docker build command tries installing packages. I am using the Ubuntu base image in Docker to do this. Below is the code in my Dockerfile;
FROM ubuntu
RUN apt-get update
RUN apt-get install apache2
RUN apt-get install apache2-utils
RUN apt-get clean
EXPOSE 80
CMD ["apache2ctl", "-D", "FOREGROUND"]
My Host OS is Mac OSX El Capitan, and the error I get when the build fails is;
The command '/bin/sh -c apt-get install apache2' returned a non-zero code: 1
and my docker build command is;
docker build -t="webserver" .
Any help please. Thanks in Advance.
You should use the '-y' apt-get flag when building an image.
apt-get will ask you permission to continue with apache installation, and since you can't interact with the apt-get while building the image, you must pass the '-y' flag that means 'yes' to apt-get prompt.
Try changing it to:
FROM ubuntu
RUN apt-get update
RUN apt-get install apache2 -y
RUN apt-get install apache2-utils -y
RUN apt-get clean
EXPOSE 80
CMD ["apache2ctl", "-D", "FOREGROUND"]
or even:
FROM ubuntu
RUN apt-get update && apt-get install apache2 apache2-utils -y
RUN apt-get clean
EXPOSE 80
CMD ["apache2ctl", "-D", "FOREGROUND"]