I have a requirement where i need to use the group_vars in Ansible but it does not need to be on the same path. Basically my playbooks are in SVN and whenever we want to deploy we actually checkout the latest playbook with release version and then deploy.
Playbook which actually executes the deployment has this structure -
/home/ansible/myapp/ansible-myapp-release/deploy/ansible-myapp-5.80.0-05
- bin
- roles
- myapp-main.yml
- myapp-full-deployment.yml
- myapp-config.yml
The other thing which i checkout from SVN is this which contains the configuration for my application and it does not contain any plays and it could be different for different environments like preprod, prod, test etc.
/home/ansible/myapp/ansible-myapp-release/configuration/5.80.0-05/preprod
- config
- group_vars
- preprod.yml
- inventory
- myapp
inventory myapp -
[preprod]
myapp-preprod
Now when i execute my first playbook I want to use this group_vars because that's where my configuration is in preprod.yml which will be loaded into jinja templates at run time.
How can i use this inventory and group_vars from this path OR do i need to copy these folders from here to the original playbook root directory and then execute the playbook?
Please let me know if someone can advice on this.
The documentation says the path should be relative to where your inventory is stored
https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#organizing-host-and-group-variables
Quoting from Organizing host and group variables
Ansible loads host and group variable files by searching paths relative to the inventory file or the playbook file
Below is the example of how to put the group_vars relative to inventory file
shell> tree /home/ansible/myapp/ansible-myapp-release/configuration/5.80.0-05/preprod
/home/ansible/myapp/ansible-myapp-release/configuration/5.80.0-05/preprod
└── inventory
├── group_vars
│ └── preprod.yml
└── myapp
Related
Tree structure:
.
├── README.md
├── galaxy
│ ├── requirements.yaml
│ └── roles
│ └── github-runner.yaml
└── inventories
└── hosts.yaml
3 directories, 4 files
galaxy/requirements.yaml content:
---
roles:
- name: monolithprojects.github_actions_runner
galaxy/roles/github-runner.yaml content:
---
- name: GitHub self-hosted runner
hosts: my_host
roles:
- role: monolithprojects.github_actions_runner
become: true
runner_user: ubuntu
github_account: my-github-username
github_repo: my-github-repository
access_token: redacted
I'm using this role to install GitHub Actions self-hosted runner on one of my servers. But, I can't find a way to load and pass that access_token parameter from another file (e.g. .env or secrets.yaml) or from my local environment variable.
I don't prefer to store any secrets in my Git repo even they can be revoked and I may apply some Git tricks to clean my Git history. I just want to start clean and right because I don't want to deal with the mess later.
I tried using vars_files and include_vars attributes in the YAML file above but failed. I also tried passing from my local environment variable (like in code block below) - but that failed too.
... content of galaxy/roles/github-runner.yaml ...
access_token: "{{ lookup('env', 'PERSONAL_ACCESS_TOKEN') }}"
Is it possible to set that access_token from a secret file or environment variable? Or it's not possible in this case?
Note: I'm planning to reuse more roles from Ansible Galaxy in future and I will add them as needed under that galaxy/roles folder.
The ansible docu says
If Ansible were to load ansible.cfg from a world-writable current working directory, it would create a serious security risk.
That makes sense but causes a problem in my ci-pipeline fir my project:
.
├── group_vars
├── host_vars
├── playbooks
├── resources
├── roles
| ├── bootstrap
| └── networking
├── ansible.cfg
├── inventory.yml
├── requirements.yml
├── site.yml
└── vault.yml
I have two "local" roles which are checked in under source control of the ansible project under ./roles, but the roles are not found when i run ansible-playbook --syntax-check site.yml
$ ansible-playbook --syntax-check site.yml
[WARNING] Ansible is being run in a world writable directory (/builds/papanito/infrastructure), ignoring it as an ansible.cfg source. For more information see https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-writable-dir
[WARNING]: provided hosts list is empty, only localhost is available. Note
that the implicit localhost does not match 'all'
ERROR! the role 'networking' was not found in /builds/papanito/infrastructure/playbooks/roles:/root/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/builds/papanito/infrastructure/playbooks
The error appears to have been in '/builds/papanito/infrastructure/playbooks/networking.yml': line 14, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
roles:
- { role: networking, become: true }
^ here
ERROR: Job failed: exit code 1
--------------------------------------------------------
Obviously cause roles are searched
A roles/ directory, relative to the playbook file.
Thus my ansible.cfg defined to look in ./roles
# additional paths to search for roles in, colon separated
roles_path = ./roles
So based on the ansible docu I can use the environment variable ANSIBLE_CONFIG as I do as follows in the gitlab-ci.yml
variables:
SITE: "site.yml"
PLAYBOOKS: "playbooks/**/*.yml"
ANSIBLE_CONIG: "./ansible.cfg"
stages:
- verify
before_script:
.....
ansible-verify:
stage: verify
script:
- ansible-lint -v $SITE
- ansible-lint -v $PLAYBOOKS
- ansible-playbook --syntax-check $SITE
- ansible-playbook --syntax-check $PLAYBOOKS
But I still get the error above. What do I miss?
site.yml
- import_playbook: playbooks/networking.yml
- import_playbook: playbooks/monitoring.yml
playbooks/networking.yml
- name: Setup default networking
hosts: all
roles:
- { role: networking, become: true }
- { role: oefenweb.fail2ban, become: true }
I know the topic is old, but you have a typo in your config file. You are missing an F in ANSIBLE_CONFIG, so write this instead
variables:
SITE: "site.yml"
PLAYBOOKS: "playbooks/**/*.yml"
ANSIBLE_CONFIG: "./ansible.cfg"
BTW, it helped to solve my problem
Looks like a Hierarchy setup issue, there is no task associated within the roles bootstrap, networking; Instead looks like the playbooks are in a different folder called playbooks.
Refer directory layout: https://docs.ansible.com/ansible/latest/user_guide/playbooks_best_practices.html
I'm trying to organize my playbooks according to the Directory Layout structure. The documentation doesn't seem to have a recommendation for host-specific files/templates.
I have 2 plays for a single site
example.com-provision.yml
example.com-deploy.yml
These files are located in the root of my structure. The provisioning playbook simply includes other roles
---
- hosts: example.com
roles:
- common
- application
- database
become: true
become_method: su
become_user: root
The deployment playbook doesn't include roles, but has it's own vars and tasks sections. I have a couple template and copy tasks, and am wondering what the 'best practice' is for where to put these host-specific templates/files within this directory structure.
Right now I have them at ./roles/example.com/templates/ and ./roles/example.com/files/, but need to reference the files with their full path from my deployment playbook, like
- name: deployment | copy httpd config
template:
src: ./roles/example.com/templates/{{ host }}.conf.j2
# ...
instead of
- name: deployment | copy httpd config
template:
src: {{ host }}.conf.j2
# ...
Facing the same problem the cleanest way seems for me the following structure:
In the top-level directory (same level as playbooks) I have a files folder (and if I needed also a templates folder). In the files folder there is a folder for every host with it's own files where the folder's name is the same as the host name in inventory.
(see the structure below: myhost1 myhost2)
.
├── files
│ ├── common
│ ├── myhost1
│ ├── myhost2
|
├── inventory
│ ├── group_vars
│ └── host_vars
├── roles
│ ├── first_role
│ └── second_role
└── my_playbook.yml
Now in any role you can access the files with files modules relatively:
# ./roles/first_role/main.yml
- name: Copy any host based file
copy:
src: "{{ inventory_hostname }}/file1"
dest: /tmp
Explanation:
The magic variable inventory_hostname is to get the host, see here
The any file module (as for example copy) looks up the files directory in the respective role directory and the files directory in the same level as the calling playbook. Of course same applies to templates (but if you have different templates for the same role you should reconsider your design)
Semantically a host specific file does not belong into a role, but somewhere outside (like host_vars).
I'm designing a kind of playbook lib with individual tasks
so in the usual roles repo, I have something like:
roles
├── common
│ └── tasks
│ ├── A.yml
│ ├── B.yml
│ ├── C.yml
│ ├── D.yml
│ ├── login.yml
│ ├── logout.yml
│ └── save.yml
├── custom_stuff_workflow
│ └── tasks
│ └── main.yml
└── other_stuff_workflow
└── tasks
└── main.yml
my main.yml in custom_stuff_workflow then contain something like:
---
- include: login.yml
- include: A.yml
- include: C.yml
- include: save.yml
- include: logout.yml
and this one in the other workflow:
---
- include: login.yml
- include: B.yml
- include: A.yml
- include: D.yml
- include: save.yml
- include: logout.yml
I can't find a way to do it in a natural way:
one way that worked was having all tasks in a single role and tagging the relevant tasks while including a custom_stuff_workflow
The problem I have with that is that tags cannot be set in the calling playbook: it's only to be set at command line
as I'm distributing this ansible repo with many people in the company, I can't rely on command line invocations (it would be nice to have a #! header in yml to be processed by ansible-playbook command)
I could also copy the relevant tasks (inside common in the above tree) in each workflow, but I don't want to repeat them around
Can someone see a solution to achieve what I'd like without repeating the tasks over different roles?
I guess the corner stone of my problem is that I define tasks as individual and it looks not natural in ansible...
Thanks a lot
PS: note that the tasks in the workflow have to be done in specific order and the only natural steps to abstract would be the login and save/logout
PPS: I've seen this question How do I call a role from within another role in Ansible? but it does not solve my problem as it's invoking a full role and not a subset of the tasks in a role
Just in case someone else bumps into this, version 2.2 of Ansible now has include_role. You can now do something like this:
---
- name: do something
include_role:
name: common
tasks_from: login
Check out the documentation here.
Yes, Ansible doesn't really like tasks as individual components. I think it wants you to use roles, but I can see why you wouldn't want to use roles for simple, reusable tasks.
I currently see two possible solutions:
1. Make those task-files into roles and use dependencies
Then you could do something like this in e.g. custom_stuff_workflow
dependencies:
- { role: login }
See: https://docs.ansible.com/playbooks_roles.html#role-dependencies
2. Use include with "hardcoded" paths to the task files
- include: ../../common/tasks/login.yml
That worked pretty well in a short test playbook I just did. Keep in mind, you can also use parameters etc. in those includes.
See: http://docs.ansible.com/ansible/latest/playbooks_reuse.html
I hope I understood that question correctly and this helps.
Using include_role: with option tasks_from is a good idea. However this still includes parts of the role. For example it loads role vars and meta dependencies. If apply is used to apply tags to an included file, then same tags are applied to meta dependencies. Also, the ansible output lists as the included role's name in its output, which is confusing.
It is possible to dynamically locate a role and include a file using first_found. One can find the role path searching DEFAULT_ROLES_PATH and load a file from tasks folder. Ansible uses the same variable when sarching a role, so long as the role is in a path that Ansible can find, then the file will be loaded.
This method is as dynamic as using include_role with option tasks_from
Example:
- name: Include my_tasks.yml from my_ansible_role
include_tasks: "{{lookup('first_found', params)}}"
vars:
params:
files: my_ansible_role/tasks/my_tasks.yml
paths: "{{ lookup('config', 'DEFAULT_ROLES_PATH') }}"
You can use the built-in variable playbook_dir to include tasks from other roles.
- name: "Create SSL certificates."
include_tasks: "{{ playbook_dir }}/roles/common/tasks/ssl.yml"
with_items: "{{ domains }}"
The ssl.yml is in the common role. It can have any number of tasks; also, passing variables like the example is possible.
I have two Ansible projects. One contains common roles and playbooks, the second playbooks, roles specific to a customer including the inventory.
I want to be able to run the common playbooks using the inventory from the second project
ansible-playbook -vvvv -s -K -i \
~/inventory/hosts/staging.int.openbet.com site.yml
~/inventory >
.
├── group_vars
│ └── all
├── hosts
│ └── staging.int.openbet.com
└── host_vars
└── staging.int.openbet.com
This runs against the host correctly but the host_vars aren't picked up. I was expecting the host_vars and group_vars to be relative to the inventory file but this doesn't seem to happen.
http://docs.ansible.com/intro_inventory.html#splitting-out-host-and-group-specific-data
Tip: In Ansible 1.2 or later the group_vars/ and host_vars/ directories can exist in either the playbook directory OR the inventory directory. If both paths exist, variables in the playbook directory will be loaded second.
Any help most appreciated, i'm running 1.7
You need to define the inventory file path in ansible.cfg in your first project where you will define the path of inventory from the second project.
Example:
inventory = ./_inventory/hosts.ini