I have my directory structure setup such that the playbooks directory is under /var/lib/awx/projects/ and roles is a symlink under playbooks.
I need symlinks for roles as I'm using git push from my local system to push playbooks/roles to Tower and both playbooks and roles are under a different git repo.
When I try to run a job in Tower, it complains that it can't find my roles. When I look at every path that ansible states that it has looked in, they are there.
Any ideas why my roles can't be found?
If the symlinks indeed are the problem (I can't tell) then git submodules might be a solution for you. You could add your "roles" repository as a submodule to the repository where the playbooks are located. You need to clone the parent repository then recursive:
git clone --recursive playbooks
Be aware submodules can be a huge PITA, but if you are required to have playbooks and roles in separate repositories and symlinks are indeed not followed by tower, this might be a solution.
Another solution would be to create a separate git repository for every role. Then you could install each role separately via ansible-galaxy. For that you need a requirements.yml with all your roles defined like this:
- src: https://gitsource.host/user/some_role
version: master
name: some_role
Ansible Tower would install all listed roles automatically, as described here.
Related
I always use composer packages in Laravel but I never changed one. This is my first time and I don't want to do it incorrect.
I need to use and change a packages foo/bar. Everything that follows now is just guessed:
I forked the repo
I created a develop branch
I added a vcs to my composer.json
"require": {
//...
"foo/bar": "dev-develop",
},
"repositories": [
{
"type": "vcs",
"url": "https://github.com/thisisme/bar"
}
],
composer update
Now I have the thisisme/bar fork in my vendor folder in foo.
So far so good. Now I can use my own fork.
But currently, as I don't know what is good practice to modify the repo, I cloned the repo to a completely different location. Then I push my changes there and run composer update in my project to get the changes. But this is a pain.
Do I need to have a sub git in my project in vendor/foo with
git remote add origin https://github.com/thisisme/bar.git. Because "git in git" feels wrong and finally is not really working as git commands seem to interact with the "parent git".
While VonCs answer is correct regarding git, I'm not certainly sure that git submodule support is well aligned with composer(1) vendor dir for packages from a VCS repository. At least I have not experimented much with it and when I use a composer configuration with a VCS git repository, I normally don't need that1.
While composer(1) has support for git for vendor packages, it is on repository level, that is, you can have your own repository for your package (as you have configured it shown in your question) and then composer takes care of updating (or giving a warnings about local changes).
composer(1) supports this with its own remote for the packages (non-bare) clone (in the source install, read on).
So yes, what you describe ("But this is a pain."), is as long as you don't use it to your benefit. While you develop your (cloned) package, you don't need to run composer update all the time.
.git
composer.json
vendor/foo/bar/.git
A Composer project with two Git repositories
This is why IMHO "git in git" must not feel wrong. Similar to git sub-modules, git supports this very well. By default it even keeps track in the parent project of the current revision (changes) of the sub-project but without having the information of the remote - as it is local (gitlink).
You won't see this thought as within the tree, the gitlink would be at vendor/foo/bar and commonly (& given that) vendor is git ignored, no version tracking in the main project for vendor/foo/bar/.git - but there in the sub-project.
This is not a problem as Composer manages that git sub-project for you (the initial clone and further checkouts) in terms of your main project.
And git realizes it is a different project.
You should be able to cd into the package directory within the vendor folder (vendor/foo/bar) and configure your remote(s) there. You can then work within that project and git(1) will work there and not within the parent repository.
To have this work with composer(1) it is important that you configure composer to prefer the source install variant for that repository. This is the preferred-install option and you can configure it for your repository specifically.
{
"config": {
"preferred-install": {
"foo/bar": "source"
}
}
}
From the wording in your question, I assume that you have not yet configured it.
And this is somewhat important as only with the source install, there will be a (non-bare) git clone in vendor/foo/bar and therefore the git checkout with the overall git configuration within the packages folder in the vendor directory (as you have Github configured as the repository source and composer optimizes to take the dist version by default IIRC).
After you changed your configuration to the source install and updated it, cd into vendor/foo/bar and then run git remote -v. It now should show you the "composer" remote(s) for that package.
As you use the develop branch, you can add changes locally but mind the gap that you would also need to push them to the remote repository (Github) before you use composer again to update (at least) that foo/bar package - as while you use git for the development of the foo/bar package now, in your main project you use composer to manage the dependency.
This is the price you have on the payroll using Github instead of a configuration that is more near to the place of work, but at least locally, you can handle the package with "git in git".
This is normally straight forward. One overall price remains thought, due to managing two instead of one repository but that you can't prevent with this kind of composer project [composer only versioned vendor folder]).
Note: If development takes longer than a few hours, it may also make sense to include the new Git sub-project in the backup routine of your parenting project, so that when you remove the folder vendor/foo/bar you have a backup of the (local) Git repository and working tree in it. However, this depends on the project configuration and is your own responsibility.
A bit of a workflow with some hints is also outlined in the composer documentation in Loading a package from a VCS repository.
1 There is a type of setup for a composer project where vendor itself is under git version control, with that git sub-modules can work (very well), but this is most likely not the kind of setup you have for your project, so I skip it for this answer.
If you're working with sail or docker-compose and linking the foo/bar project in the vendor dir is only a temporary until 'it works' solution you could just add it as a volume link. This is what I usually do.
Eg: I'm working on my-project in ~/projects/my-project, I clone the foo/bar repo to ~/projects/bar
Then in the docker-compose.yml I can add the volume:
volumes:
- .:/var/www/html
- ../bar:/var/www/html/vendor/foo/bar
Again, this has a huge assumption on docker being used, but I like to think that everybody is using it these days.
Do I need to have a sub git in my project in vendor/foo with git remote add origin https://github.com/thisisme/bar.git.
That could be achieved with a submodule which allows for your parent Git repository to only store a reference to another repository.
You would use git submodule add for that.
A git clone --recurse-submodule would therefore clone your project with the submodule Git repository in it cloned as well, and checked out to the exact reference you previously committed.
My current playbook is structured this way
projectroot
|
|--ubuntu2004
|
|--00_setup
|
|--vars
|--playbook.yml
|--readme.md
Because my playbook uses ansible.posix and I also commit my playbook into a github repo. I was hoping if there's a way to include the required collection in this case ansible.posix as a requirement and how do I install it?
I saw that there are multiple ways https://docs.ansible.com/ansible/latest/user_guide/collections_using.html#installing-collections
I was wondering what's the best practice way that makes sense when using a github repository as version control for the playbook?
There's a few ways that you could do this. I'd suggest a requirements file since it's the easiest to set up and to manage.
Create a requirements file that you use to install the required modules.
The best way would be to create a requirements file that references the collection(s) your playbook needs. Which you can then use to install the required collection(s) and / or role(s).
---
collections:
- name: ansible.posix
Store the module in a repository and install it through ansible-galaxy.
You could upload the module to a git repository and then install it. I wouldn't recommend this as storing dependencies in your source-code isn't considered a good practise when there's a tool to manage dependencies available.
ansible-galaxy collection install
git+https://github.com/organization/repo_name.git,devel
Install the module through a playbook
I've set up a master node to run playbooks on with Ansible before and installed modules through the command task in a playbook. As long as you don't reference / include any tasks or plays that use the module, this will work fine.
- name: Install ansible posix module
command: ansible-galaxy collection install ansible.posix
The ideal way would be to have to playbook install the required module(s) prior to executing the tasks. However it's not possible to ignore the error that is thrown when a module is missing, so you would have to have to create a play that doesn't include / reference any of the tasks using the module.
Straight forward question. MY playbook creates a new project on gitlab and then It clones it on all the host. Now I want for one host to enter the project and create a .gitmodules files with "git submodule add git#some-url.git" only once.
the git module for ansible only has track_submodules parameter but that updates the submodule which will be useful later. And I wouldn't like using the shell module in this case.
Essentially I just want an automatic ansible way to create a new project with a specific submodule.
You'll need to first check if the submodule exists. You can just use the stat module to verify that the directory exists, and then only run your git submodule add ... command if the submodule directory doesn't exist. Something like:
- name: check if submodule exists
stat:
path: mysubmodule
register: mysubmodule_check
- name: add submodule
command: git submodule add {{ submodule_url }} mysubmodule
when: not mysubmodule_check.stat.exists
I want to build a project in Azure Pipelines, but I want to know what the idiomatic way is to obtain the latest tag, latest tag distance, and repo remote path/URL in order to pass those values into the actual build script that is inside the repository.
Previously our build script would invoke hg log -r . --template with a clever template, but we found when moving to Continua CI build server that the build agent doesn't have access to the actual repository during a build, and had to find another way.
I'm assuming the same issue would crop up with Azure Pipelines and haven't quite found the relevant docs yet on artifact versioning.
Many thanks in advance.
For git at least, Azure Pipelines does a full clone of the repo by default, unless you explicitly denote that you're doing a shallow clone (source: https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/pipeline-options-for-git?view=azure-devops).
Deriving the version/tag can be done via normal git commands (i.e. git describe --tags or whatever you prefer), which can then be saved as VSO variables to be accessed in later steps in the same job (see https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#set-variables-using-expressions for more info on how to do that).
I have a playbook and a bunch of modules I wrote.
Now I want to reuse the same modules in my next playbook for a different project.
I really want to push those modules to a public git repository and then somehow tell ansible to use the modules from the git repository.
(kinda like npm package.json referencing github)
I can't seem to find any documentation on how to do that.
For now, I am using a workaround where I tell people to npm install the repository, and then define ANSIBLE_LIBRARY variable.
How can I tell the playbook to load modules from a github repository or some other remote location?
Actually modules can be nested inside roles since quite a long time. Since Ansible 2 this even is possible with most of the plugins.
The folders where the modules & plugins need to be stored inside the role is the same as on playbook level. Modules go into library, plugins go into *_plugins (action_plugins, callback_plugins, filter_plugins etc)
To make the module/plugin then available the role has to be applied to the playbook (or added as a dependency of another role)
Only exception known to me are variable plugins and that perfectly makes sense. Since variable plugins are executed when the inventory is read, which happens before roles are interpreted.
vars_plugins still can be distributed in roles, but the path needs to be added in the ansible.cfg. Fortunately you can also use wildcards in paths:
vars_plugins = roles/*/vars_plugins
And no, all of this is not documented in any way. :)
Finally, to distribute roles you can use Ansible Galaxy:
ansible-galaxy install foo
Nothing wrong with directly using git. Ansible Galaxy actually only is a tool to install git repositories. But since Galaxy is the Ansible standard I suggest to at least provide a Galaxy compatible format. A good (best?) practice how to install Galaxy roles separate from the projects roles can be found here.
Here's an example for an action plugin: https://galaxy.ansible.com/udondan/ssh-reconnect/
There is no solution for that at the moment. Maybe you can add a playbook to download the modules to your project to avoid npm, but thats even not that nice.
I have my custom modules in a directory next to my playbooks. This directory is defined in my global ansible.cfg file:
library = /usr/share/ansible
The only drawbag here is that i allways have the same version for modules on all playbboks.