Run Ansible playbook command from gitlabci - ansible

I recently learned cloudformation module in ansible and looking to use the same with gitlab CI.
Below is my playbook:
- name: provision stack
hosts: localhost
connection: local
gather_facts: false
# Launch the cloudformation-example.yml template. Register the output.
tasks:
- name: launch ansible cloudformation example
cloudformation: >
stack_name="ansible-cloudformation" state=present
region=us-east-1 disable_rollback=true
template=files/simple-rds.yml
args:
template_parameters:
vpcid: vpc-0123456
application: abc
appSubnetCidr1: 201.0.0.0/20
appSubnetCidr2: 201.0.0.0/22
dbCreateNewParamsGroup: true
dbInstanceType: db.t2.micro
dbName: testdb
dbSubnetId1: subnet-87654321
dbSubnetId2: subnet-12345678
dbUsername: master_user
environment: development
product: ""
dbPassword: ""
techContact: ""
register: stack
- name: show stack events
debug: msg="My stack events are {{stack.events}}"
Now I want to run this playbook using gitlab CI. For that, I have created a generic ansible image, So that I use the same image for running different playbooks.
Ansible image Docker file:
FROM ubuntu:18.04
# File Author / Maintainer
# Install ansible
RUN apt-get update
RUN apt-get install software-properties-common -y
RUN apt-get update
#RUN apt-add-repository -y ppa:ansible/ansible
#RUN apt-get update
RUN apt-get -y install ansible
# Install Pip
RUN apt install python-pip -y
# Install boto3
RUN pip install boto3
# Install boto
RUN pip install boto
Now In .gitlab-ci.yml.
stages:
- build
build:master:
image: vrathore/ansibleimage:latest
script:
- touch ~/.boto
- echo "[Credentials]" > ~/.boto
- echo "aws_access_key_id = AKIAIACZXXXXXXXXXXXX" >> ~/.boto
- echo "aws_secret_access_key = P5lO8H9tXXXXXXXXXXXXX" >> ~/.boto
- cp simple-rds.yml /etc/ansible/playbook/files/simple-rds.yml
- cp eb_playbook.yml /etc/ansible/playbook/eb_playbook.yml
- cd /etc/ansible/playbook && ansible-playbook eb_playbook.yml -vvv
stage: build
only:
- master
In the above script I am adding credential in boto file and copying playbook script (eb_playbook.yml) and cloudformation script(simple-rds.yml) and running the playbook run command but I am getting
cp: cannot stat 'simple-rds.yml': No such file or directory
How can I copy both files (eb_playbook.yml and simple-rds.yml). both files present in gitlab project and run the ansible playbook command.

Related

How to activate python virtual environment on remote machine with ansible?

I'm learning Vagrant and Ansible, I'm trying to setup a local development environment for a basic flask app in ubuntu20.04 with Nginx.
my vagrantfile looks like this:
Vagrant.configure("2") do |config|
config.vm.define :ubuntuserver do | ubuntuserver |
ubuntuserver.vm.box = "bento/ubuntu-20.04"
ubuntuserver.vm.hostname = "ubuntuserver"
ubuntuserver.vm.provision :ansible do | ansible |
ansible.playbook = "development.yml"
end
ubuntuserver.vm.network "private_network", ip:"10.11.1.105"
ubuntuserver.vm.network "forwarded_port", guest: 80, host: 8080
ubuntuserver.vm.network "public_network", bridge: "en1: Wi-Fi (AirPort)"
ubuntuserver.vm.provider :virtualbox do |vb|
vb.memory = "1024"
end
ubuntuserver.vm.synced_folder "./shared", "/var/www"
end
end
my ansible-playbook like so:
-
name: local env
hosts: ubuntuserver
tasks:
- name: update and upgrade apt packages
become: yes
apt:
upgrade: yes
update_cache: yes
- name: install software properties common
apt:
name: software-properties-common
state: present
- name: install nginx
become: yes
apt:
name: nginx
state: present
update_cache: yes
- name: ufw allow http
become: yes
community.general.ufw:
rule: allow
name: "Nginx HTTP"
- name: installing packages for python env
become: yes
apt:
name:
- python3-pip
- python3-dev
- build-essential
- libssl-dev
- libffi-dev
- python3-setuptools
- python3-venv
update_cache: yes
- name: Create app directory if it does not exist
ansible.builtin.file:
path: /var/www/app
state: directory
mode: '0774'
- name: Install virtualenv via pip
become: yes
pip:
name: virtualenv
executable: pip3
- name: Set python virual env
command:
cmd: virtualenv /var/www/app/ -p python3
creates: "/var/www/app/"
- name: Install requirements
pip:
requirements: /var/www/requirements.txt
virtualenv: /var/www/app/appenv
virtualenv_python: python3
My playbook fails at the next task with error:
- name: Activate /var/www/app/appenv
become: yes
command: source /var/www/app/appenv/bin/activate
fatal: [ubuntuserver]: FAILED! => {"changed": false, "cmd": "source /var/www/app/appenv/bin/activate", "msg": "[Errno 2] No such file or directory: b'source'", "rc": 2}
Rest of the playbook
- name: ufw allow 5000
become: yes
community.general.ufw:
rule: allow
to_port: 5000
- name: Run app
command: python3 /var/www/app/appenv/app.py
From what I understand from this thread, The "source" command must be used from inside the vagrant machine. (I tried solutions from the thread but couldn't get it to work)
If I ssh into the vagrant machine and execute the three last commands of my playbook manually:
source /var/www/app/appenv/bin/activate
sudo ufw allow 5000
python3 /var/www/app/appenv/app.py
my basic flask app is running on port 5000 at the IP set in the vagrantfile 10.11.1.105
My questions are:
How can I get the playbook to work and not have to ssh into the machine to accomplish the same?
Is my approach even correct, knowing that my end goal is to replicate in the vagrant machine a similar environment to what would be the production environment and develop the flask app from my local machine in the synced folder?
to give a maximum of information, if one wants to reproduce this.
I also have a shared/app/appenv/app.py file containing the basic flask app
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "<h1 style='color:blue'>Hello There!</h1>"
if __name__ == "__main__":
app.run(host='0.0.0.0')
and shared/requirements.txt file
wheel
uwsgi
flask
This question was coming from a misconception about python venv.
I thought that in order to install packages inside the virtual env it had to be activated.
for example:
source env/bin/activate
pip install package_name
I understood later that I can install packages in the venv without activating it by doing:
env/bin/pip install package_name
So the solution with ansible is not to activate the venv to install packages but instead
- name: "Install python packages with the local instance of pip"
shell: "{{virtualenv_path}}/bin/pip3 install package_name"
become: no
or better even with the pip module and the packages in a requirements.txt file:
- name: Install project requirements in venv
pip:
requirements: '{{project_path}}/requirements.txt'
virtualenv: '{{virtualenv_path}}'
virtualenv_python: python3
I wrote this article for a Linux computer with Python 3.x. In this scenario, this is your Ansible development machine. First, verify the installed Python version and path:
# check Python version
$ python3 -V
Python 3.6.8
$ which python3
/usr/bin/python3
I recommend setting up a directory for the virtual environment:
$ mkdir python-venv
$ cd !$
Create a new virtual environment
$ python3 -m venv ansible2.9
$ ls
ansible2.9
activate python venv
$ source ansible2.9/bin/activate
(ansible2.9)$ python3 -V
Python 3.6.8
upgrade pip
(ansible2.9)$ python3 -m pip install --upgrade pip
Install Ansible in a virtual environment
(ansible2.9)$ python3 -m pip install ansible==2.9
(ansible2.9)$ which ansible
~/python-venv/ansible2.9/bin/ansible
Verify your new installation:
(ansible2.9)$ ansible --version
ansible 2.9.0
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/devops/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/devops/python-venv/ansible2.9/lib64/python3.6/site-packages/ansible
executable location = /home/devops/python-venv/ansible2.9/bin/ansible
python version = 3.6.8 (default, Jan 09 2021, 10:57:11) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
Install Ansible roles or collections
(ansible2.9)$ ansible-galaxy collection install \
kubernetes.core:==1.2.1 -p collections
Deactivate a Python virtual environment
(ansible2.9)$ deactivate
Create another Python virtual environment for Ansible 3.0
$ python3 -m venv ansible3.0
$ ls -1
ansible2.9
ansible3.0
$ source ansible3.0/bin/activate
(ansible3.0)$ which python
~/python-venv/ansible3.0/bin/python
(ansible3.0)$ python3 -m pip install --upgrade pip
(ansible3.0)$ python3 -m pip install ansible==3.0

How to install ansible on a RHEL machine which doesn't have access to Internet

How to install ansible on a RHEL 6.8 machine which doesn't have access to Internet?
I'd propose to use raw: module. Copy rpm (scp) to server, and run rpm -i.
Raw module does not use a usual ansible workflow and do things 'directly'. Here my code to install python if it's not present on server, I think you can adopt it for rpm for ansible too.
- hosts: all
gather_facts: no
tasks:
- name: Remember sever key and install python
raw: test -x /usr/bin/python || sudo apt-get update && sudo apt-get -y install python
changed_when: True

Can't find the syntax issue with this Ansible playbook

I'm attempting to run an Ansible playbook and I can't find out, through documentation or by looking at examples, what is wrong with my playbook.
---
- hosts: all
sudo: yes
pre_tasks:
ignore_errors: True
tasks:
command: sudo apt-get install build-essential libssl-dev libffi-dev python-dev python-pip
command: sudo pip install caravel
command: fabmanager create-admin --app caravel
command: caravel db upgrade
command: caravel init
command: caravel runserver -p 8088
- copy: src=../zika.db dest=zika.db
failed_when: false
I've been chasing my tail and I don't understand this error:
The offending line appears to be:
---
- hosts: all
^ here
command is a task, and tasks is a list, so you should prefix every task with a dash.
tasks:
- command: ....
- command: ....
....

How to make sure an ansible playbook runs completely

I am running this ansible playbook:
---
- hosts: localhost
remote_user: root
tasks:
- name : update system
apt : update_cache=yes
- name : install m4
apt : name=m4 state=present
- name : install build-essential
apt : name=build-essential state=present
- name : install gcc
apt : name=gcc state=present
- name : install gfortran
apt : name=gfortran state=present
- name : install libssl-dev
apt : name=libssl-dev state=present
- name : install python-software-properties
apt : name=python-software-properties state=present
- name : add sage ppa repo
apt_repository: repo='ppa:aims/sagemath'
- name : update system
apt : update_cache=yes
- name : install dvipng
apt : name=dvipng state=present
- name : install sage binary
apt : name=sagemath-upstream-binary state=present
- name : invoke create_sagenb script
command: /usr/bin/screen -d -m sudo /root/databases-and-datamining-iiith/python-scripts/create_sagenb -i -y
- name : invoke start_sage script
command: /usr/bin/screen -d -m sudo /root/databases-and-datamining-iiith/python-scripts/start_sage -i -y
This playbook fails during task "install build-essential" and stops with error asking to run dpkg --configure -a.
How can I make sure that the playbook runs again after facing this error by running the command
dpkg --configure -a
first and then continue with other tasks.
Ansible in general is idempotent. That means you can simply run your playbook again after resolving the issue without conflicts.
This is not always true. In case you have a more complex play and execute tasks depending on the result of another task, this can break easily and a failed task then would bring you into a state that is not so easy to be fixed with Ansible. But that is not the case with the tasks you provided.
If you want to speed things up and skip all the tasks and/or hosts that did not fail, you can work with --limit and/or --start-at-task:
When the playbook fails, you might notice Ansible shows a message including a command which will enable you to limit the play to hosts which failed. So if only 1 host failed you do not need to run the playbook on all hosts:
ansible-playbook ... --limit #/Users/your-username/name-of-playbook.retry
To start at a specific task, you can use --start-at-task. So if your playbook failed at the task "install build-essential" you can start again at right this task and skip all previous tasks:
ansible-playbook ... --start-at-task="install build-essential"
On a side note, the apt module is optimized to work with loops. You can speed up your play by combining the tasks into one single apt task:
tasks:
- name: Install packages that we need for need for apt_repository
apt: update_cache=yes
name={{ item }}
state=present
cache_valid_time=3600
with_items:
- python-software-properties
- python-software-properties-common
- name: add sage ppa repo
apt_repository: repo='ppa:aims/sagemath'
- name: Install packages
apt: update_cache=yes
cache_valid_time=3600
name={{ item }}
state=present
with_items:
- m4
- build-essential
- gcc
- gfortran
- libssl-dev
- dvipng
- sagemath-upstream-binary

Using Ansible, how can I install PythonBrew system-wide?

I'm trying to create a playbook with Ansible (v 1.3.3) to install Pythonbrew system-wide on a Debian server following the instructions in the Pythonbrew readme file.
I am able to get Pythonbrew installed but I cannot install the specific version of Python that I want with it. I suspect the issue has to do with the shell environment Ansible is running under.
Here's my playbook script:
- name: Install and configure PythonBrew
hosts: dev
user: root
vars_files:
- vars.yml
gather_facts: false
tasks:
- name: Install PythonBrew Debian packages
apt: pkg=${item} state=installed update-cache=yes
with_items: ${pythonbrew_packages}
- name: Install PythonBrew system-wide
shell: curl -kL http://xrl.us/pythonbrewinstall | bash creates=/usr/local/pythonbrew executable=/bin/bash
- name: Update bashrc for PythonBrew
lineinfile:
dest=~/.bashrc
regexp='^'
line='[[ -s $HOME/.pythonbrew/etc/bashrc ]] && source $HOME/.pythonbrew/etc/bashrc'
state=present
create=True
- name: Install python binary
shell: pythonbrew install -v ${python_version} executable=/bin/bash
When I run this playbook, it fails with the following output
failed: [devserver] => {"changed": true, "cmd": "pythonbrew
install -v 2.7.3 ", "delta": "0:00:00.016639", "end": "2013-10-11
15:21:40.989677", "rc": 127, "start": "2013-10-11 15:21:40.973038"}
stderr: /bin/bash: pythonbrew: command not found
I've been tweaking things for the last hour or so to no avail. Does anybody have any suggestions for fixing this?
By peeking at the PythonBrew install script, I was able to figure this out. (And just in time for the deprecation of PythonBrew!)
Here's the playbook that installs PythonBrew without manual intervention. This may be of interest to anyone trying to script PythonBrew to install automatically.
vars.yml
#
# Python/PythonBrew Settings
# TODO: replace old-style Ansible ${vars} with jinja-style {{ vars }}
#
project_name: MY_PROJECT
python:
version: 2.7.3
pythonbrew:
root: /usr/local/pythonbrew
bashrc_path: $HOME/.pythonbrew/etc/bashrc
packages:
- curl
- zlib1g-dev
- libsqlite3-dev
- libssl-dev
- libxml2
- libxml2-dev
- libxslt1-dev
- libmysqlclient-dev
- libbz2-dev
pythonbrew.yml
---
#
# Install and Configure PythonBrew
#
- name: Install and configure PythonBrew
hosts: MY_HOST
user: root
vars_files:
- vars.yml
gather_facts: false
tasks:
- name: Install PythonBrew Debian packages
apt: pkg=${item} state=installed update-cache=yes
with_items: ${pythonbrew.packages}
- name: Install PythonBrew system-wide
shell: curl -kL http://xrl.us/pythonbrewinstall | bash
executable=/bin/bash
creates=${pythonbrew.root}
- name: Update bashrc for PythonBrew
lineinfile:
dest=/root/.bashrc
regexp='^'
line='[[ -s ${pythonbrew.bashrc_path} ]] && source ${pythonbrew.bashrc_path}'
state=present
create=True
# This step allows install to continue without new shell. Pulled from:
# https://github.com/utahta/pythonbrew/blob/master/pythonbrew/installer/pythonbrewinstaller.py#L91
- name: Install python binary
shell: export PYTHONBREW_ROOT=${pythonbrew.root}; source ${pythonbrew.root}/etc/bashrc; pythonbrew install ${python.version}
executable=/bin/bash
- name: Switch to python version
shell: export PYTHONBREW_ROOT=${pythonbrew.root}; source ${pythonbrew.root}/etc/bashrc; pythonbrew switch ${python.version}
executable=/bin/bash

Resources