Tree structure:
.
├── README.md
├── galaxy
│ ├── requirements.yaml
│ └── roles
│ └── github-runner.yaml
└── inventories
└── hosts.yaml
3 directories, 4 files
galaxy/requirements.yaml content:
---
roles:
- name: monolithprojects.github_actions_runner
galaxy/roles/github-runner.yaml content:
---
- name: GitHub self-hosted runner
hosts: my_host
roles:
- role: monolithprojects.github_actions_runner
become: true
runner_user: ubuntu
github_account: my-github-username
github_repo: my-github-repository
access_token: redacted
I'm using this role to install GitHub Actions self-hosted runner on one of my servers. But, I can't find a way to load and pass that access_token parameter from another file (e.g. .env or secrets.yaml) or from my local environment variable.
I don't prefer to store any secrets in my Git repo even they can be revoked and I may apply some Git tricks to clean my Git history. I just want to start clean and right because I don't want to deal with the mess later.
I tried using vars_files and include_vars attributes in the YAML file above but failed. I also tried passing from my local environment variable (like in code block below) - but that failed too.
... content of galaxy/roles/github-runner.yaml ...
access_token: "{{ lookup('env', 'PERSONAL_ACCESS_TOKEN') }}"
Is it possible to set that access_token from a secret file or environment variable? Or it's not possible in this case?
Note: I'm planning to reuse more roles from Ansible Galaxy in future and I will add them as needed under that galaxy/roles folder.
Related
The ansible docu says
If Ansible were to load ansible.cfg from a world-writable current working directory, it would create a serious security risk.
That makes sense but causes a problem in my ci-pipeline fir my project:
.
├── group_vars
├── host_vars
├── playbooks
├── resources
├── roles
| ├── bootstrap
| └── networking
├── ansible.cfg
├── inventory.yml
├── requirements.yml
├── site.yml
└── vault.yml
I have two "local" roles which are checked in under source control of the ansible project under ./roles, but the roles are not found when i run ansible-playbook --syntax-check site.yml
$ ansible-playbook --syntax-check site.yml
[WARNING] Ansible is being run in a world writable directory (/builds/papanito/infrastructure), ignoring it as an ansible.cfg source. For more information see https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-writable-dir
[WARNING]: provided hosts list is empty, only localhost is available. Note
that the implicit localhost does not match 'all'
ERROR! the role 'networking' was not found in /builds/papanito/infrastructure/playbooks/roles:/root/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/builds/papanito/infrastructure/playbooks
The error appears to have been in '/builds/papanito/infrastructure/playbooks/networking.yml': line 14, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
roles:
- { role: networking, become: true }
^ here
ERROR: Job failed: exit code 1
--------------------------------------------------------
Obviously cause roles are searched
A roles/ directory, relative to the playbook file.
Thus my ansible.cfg defined to look in ./roles
# additional paths to search for roles in, colon separated
roles_path = ./roles
So based on the ansible docu I can use the environment variable ANSIBLE_CONFIG as I do as follows in the gitlab-ci.yml
variables:
SITE: "site.yml"
PLAYBOOKS: "playbooks/**/*.yml"
ANSIBLE_CONIG: "./ansible.cfg"
stages:
- verify
before_script:
.....
ansible-verify:
stage: verify
script:
- ansible-lint -v $SITE
- ansible-lint -v $PLAYBOOKS
- ansible-playbook --syntax-check $SITE
- ansible-playbook --syntax-check $PLAYBOOKS
But I still get the error above. What do I miss?
site.yml
- import_playbook: playbooks/networking.yml
- import_playbook: playbooks/monitoring.yml
playbooks/networking.yml
- name: Setup default networking
hosts: all
roles:
- { role: networking, become: true }
- { role: oefenweb.fail2ban, become: true }
I know the topic is old, but you have a typo in your config file. You are missing an F in ANSIBLE_CONFIG, so write this instead
variables:
SITE: "site.yml"
PLAYBOOKS: "playbooks/**/*.yml"
ANSIBLE_CONFIG: "./ansible.cfg"
BTW, it helped to solve my problem
Looks like a Hierarchy setup issue, there is no task associated within the roles bootstrap, networking; Instead looks like the playbooks are in a different folder called playbooks.
Refer directory layout: https://docs.ansible.com/ansible/latest/user_guide/playbooks_best_practices.html
I'm building an AMI with packer and providing it with Ansible. The AMI needs to be build with different variables depending on the environment.Therefore I used the below structure based on this article:
├── ansible.cfg
├── bamboo-server-dev.yml
├── environments
├── group_vars
│ ├── bamboo_packer_dev.yml
│ └── bamboo_packer_prod.yml
└── roles/
├── bamboo-server/
│ └── vars/
│ ├── bamboo-dev.yml
│ ├── bamboo-prod.yml
│ └── main.yml
└── ubuntu-server/
My environment file is where I define my inventory and looks like this:
[bamboo_packer_dev]
localhost
[bamboo_packer_prod]
localhost
The group_vars/bamboo_packer_dev.yml looks like the below:
---
# Setup specific variables per environment
environment: "dev"
The playbook bamboo-server-dev.yml looks like this:
---
- hosts: bamboo_packer_dev
become: true
roles:
- ubuntu-server
- bamboo-server
And finally the roles/bamboo-server/vars/main.yml looks like this:
---
# Provide variables according to the environment
- name: provide variables
include_vars:
file: "bamboo-{{ environment }}.yml"
But when I run the playbook I get the following message:
ERROR! failed to combine variables, expected dicts but got a 'dict' and a 'AnsibleSequence':
{}
[{"name": "provide variables", "include_vars": {"file": "bamboo-{{ environment }}.yml"}}]
It seems the "environment" variable is not getting picked up. And I cannot figure it why. Can someone please suggest what might be wrong with my playbook? Or abetter way to achieve this? My Ansible version is 2.9.1 running on Mojave.
Your vars file cannot include ansible tasks, as you have done. Files in vars must be -- as ansible very clearly informed you -- dict and not list, since the keys of the vars file becomes variable names, and [] is not a variable name
Perhaps you meant to put that content into roles/bamboo-server/tasks/main.yml, which will allow you to have include_vars: as an action
I have a playbook organized as follows (simplified for the sake of this question):
├── deploy.yml
├── hosts
├── requirements.yml
├── roles
│ └── web
│ ├── meta
│ │ └── main.yml
│ └── tasks
│ └── main.yml
└── site.retry
My simplified deploy.yml is:
---
- name: Everything I need
hosts: somewhere
roles:
- web
And my simplified roles/web/tasks/main.yml is
---
- name: Various things that work
become: yes
[whatever]
- name: the thing that I have a problem with
become: yes
davidedelvento.nbextension: name=foo state=present
This fails with:
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
So I tried to change roles/web/tasks/main.yml to
---
- name: Various things that work
become: yes
[whatever]
- name: the thing that I have a problem with
become: yes
roles:
- { role: davidedelvento.nbextension, name: foo, state: present}
which fails in the same way. I understand the failure (since I cannot call a role from a task, which instead I'm doing -- but the error could be clearer....)
However I am not clear how can I accomplish what I'd like, namely doing whatever nbextension is doing at that point in time. I could move that role from roles/web/tasks/main.yml to roles/web/meta/main.yml and that works, but it is executed before the Various things that work and I need it to be executed after. How to accomplish that?
Note that I wrote nbextension, however the same problem happen with similar other roles from the galaxy.
EDIT: Note also that the extension is correctly installed and can be used from a standalone, single-file playbook such as
---
- name: Example
hosts: all
become: yes
roles:
- { role: davidedelvento.nbextension, name: foo, state: present}
however I need it to "integrate" in the larger project described above for the "web" role (I have more roles that I'm not showing)
EDIT2: note that the galaxy ansible role used for this question has been renamed to jupyterextension but as I said the issue (and solution) is the same for any role
Ok, so I've found two ways to deal with this issue.
split the role in two (or more) parts and use the galaxy's role as a dependency to the things that it needs to prepend. In general, I like this idea, but in my particular use case I don't, since I would need create 3 roles for something that is really one.
Use include_role module, with the caveat that at the moment it is "flagged as preview", namely it is not guaranteed to have a backwards compatible interface. However it works quite well for my current setup:
- name: the thing that I have not a problem with anymore
become: yes
include_role:
name: davidedelvento.nbextension
with_items:
- foo
- bar
loop_control:
loop_var: name
I'm trying to organize my playbooks according to the Directory Layout structure. The documentation doesn't seem to have a recommendation for host-specific files/templates.
I have 2 plays for a single site
example.com-provision.yml
example.com-deploy.yml
These files are located in the root of my structure. The provisioning playbook simply includes other roles
---
- hosts: example.com
roles:
- common
- application
- database
become: true
become_method: su
become_user: root
The deployment playbook doesn't include roles, but has it's own vars and tasks sections. I have a couple template and copy tasks, and am wondering what the 'best practice' is for where to put these host-specific templates/files within this directory structure.
Right now I have them at ./roles/example.com/templates/ and ./roles/example.com/files/, but need to reference the files with their full path from my deployment playbook, like
- name: deployment | copy httpd config
template:
src: ./roles/example.com/templates/{{ host }}.conf.j2
# ...
instead of
- name: deployment | copy httpd config
template:
src: {{ host }}.conf.j2
# ...
Facing the same problem the cleanest way seems for me the following structure:
In the top-level directory (same level as playbooks) I have a files folder (and if I needed also a templates folder). In the files folder there is a folder for every host with it's own files where the folder's name is the same as the host name in inventory.
(see the structure below: myhost1 myhost2)
.
├── files
│ ├── common
│ ├── myhost1
│ ├── myhost2
|
├── inventory
│ ├── group_vars
│ └── host_vars
├── roles
│ ├── first_role
│ └── second_role
└── my_playbook.yml
Now in any role you can access the files with files modules relatively:
# ./roles/first_role/main.yml
- name: Copy any host based file
copy:
src: "{{ inventory_hostname }}/file1"
dest: /tmp
Explanation:
The magic variable inventory_hostname is to get the host, see here
The any file module (as for example copy) looks up the files directory in the respective role directory and the files directory in the same level as the calling playbook. Of course same applies to templates (but if you have different templates for the same role you should reconsider your design)
Semantically a host specific file does not belong into a role, but somewhere outside (like host_vars).
I'm designing a kind of playbook lib with individual tasks
so in the usual roles repo, I have something like:
roles
├── common
│ └── tasks
│ ├── A.yml
│ ├── B.yml
│ ├── C.yml
│ ├── D.yml
│ ├── login.yml
│ ├── logout.yml
│ └── save.yml
├── custom_stuff_workflow
│ └── tasks
│ └── main.yml
└── other_stuff_workflow
└── tasks
└── main.yml
my main.yml in custom_stuff_workflow then contain something like:
---
- include: login.yml
- include: A.yml
- include: C.yml
- include: save.yml
- include: logout.yml
and this one in the other workflow:
---
- include: login.yml
- include: B.yml
- include: A.yml
- include: D.yml
- include: save.yml
- include: logout.yml
I can't find a way to do it in a natural way:
one way that worked was having all tasks in a single role and tagging the relevant tasks while including a custom_stuff_workflow
The problem I have with that is that tags cannot be set in the calling playbook: it's only to be set at command line
as I'm distributing this ansible repo with many people in the company, I can't rely on command line invocations (it would be nice to have a #! header in yml to be processed by ansible-playbook command)
I could also copy the relevant tasks (inside common in the above tree) in each workflow, but I don't want to repeat them around
Can someone see a solution to achieve what I'd like without repeating the tasks over different roles?
I guess the corner stone of my problem is that I define tasks as individual and it looks not natural in ansible...
Thanks a lot
PS: note that the tasks in the workflow have to be done in specific order and the only natural steps to abstract would be the login and save/logout
PPS: I've seen this question How do I call a role from within another role in Ansible? but it does not solve my problem as it's invoking a full role and not a subset of the tasks in a role
Just in case someone else bumps into this, version 2.2 of Ansible now has include_role. You can now do something like this:
---
- name: do something
include_role:
name: common
tasks_from: login
Check out the documentation here.
Yes, Ansible doesn't really like tasks as individual components. I think it wants you to use roles, but I can see why you wouldn't want to use roles for simple, reusable tasks.
I currently see two possible solutions:
1. Make those task-files into roles and use dependencies
Then you could do something like this in e.g. custom_stuff_workflow
dependencies:
- { role: login }
See: https://docs.ansible.com/playbooks_roles.html#role-dependencies
2. Use include with "hardcoded" paths to the task files
- include: ../../common/tasks/login.yml
That worked pretty well in a short test playbook I just did. Keep in mind, you can also use parameters etc. in those includes.
See: http://docs.ansible.com/ansible/latest/playbooks_reuse.html
I hope I understood that question correctly and this helps.
Using include_role: with option tasks_from is a good idea. However this still includes parts of the role. For example it loads role vars and meta dependencies. If apply is used to apply tags to an included file, then same tags are applied to meta dependencies. Also, the ansible output lists as the included role's name in its output, which is confusing.
It is possible to dynamically locate a role and include a file using first_found. One can find the role path searching DEFAULT_ROLES_PATH and load a file from tasks folder. Ansible uses the same variable when sarching a role, so long as the role is in a path that Ansible can find, then the file will be loaded.
This method is as dynamic as using include_role with option tasks_from
Example:
- name: Include my_tasks.yml from my_ansible_role
include_tasks: "{{lookup('first_found', params)}}"
vars:
params:
files: my_ansible_role/tasks/my_tasks.yml
paths: "{{ lookup('config', 'DEFAULT_ROLES_PATH') }}"
You can use the built-in variable playbook_dir to include tasks from other roles.
- name: "Create SSL certificates."
include_tasks: "{{ playbook_dir }}/roles/common/tasks/ssl.yml"
with_items: "{{ domains }}"
The ssl.yml is in the common role. It can have any number of tasks; also, passing variables like the example is possible.