We are working on converting our project to Ansible. Due to the scale of the project, we will need a large amount of roles (30+). Where we're running into problems is how to store and manage these roles. Things we have considered:
1) Github repo per role -> This is unrealistic. We don't want to manage 30+ git repositories simply for the purpose of maintaining our roles
2) Ansible Galaxy -> This would be valuable if we could have a local instance of Ansible Galaxy, but the central instance won't work
3) We can simply store the roles in a flat directory, however we lose the benefit of being able to version them in this case. There is also the matter of how to automatically push our ansible roles directories to the ansible controller host into the correct directory
Is there a solution I'm missing?
I would suggest keeping the roles in a single git repo.
For the automatic push to the ansible controller, you could either create a standalone playbook that uses the git module to retrieve the appropriate version of the roles. This could then be run on a regular basis (or scheduled via cron).
Alternatively, you could add the git retrieval to your existing playbooks, and then it would check/update the roles prior to executing them.
Related
For one of our repositories we set "Custom CI configuration path" inside GitLab to a remote gitlab-ci.yml. We want to do this to prevent Developers to change the gitlab-ci.yml file (as protected files are available in EE Premium and up). But except this purpose, the Custom CI configuration path feature should work anyway for Merge Requests.
Being in repo
group1/repo1
we set
.gitlab-ci.yml#group1/repo1-ci
repo1-ci repository exists and ci works correctly when we push to configured branches etc.
For Merge Request functionality GitLab tells us:
Detached merge request pipeline #123 failed for ...
Project group1/repo1-ci not found or access denied!
We added the developers to repo1-ci repo as developers, to be able to read the files. It does not help. Anyway the expectation is, that it is not run with user permissions, so it should simply find the gitlab-ci.yml file.
Any ideas on this?
So our expectations were right an it seems that we have to add one important thing into our considerations:
If a user interacts in the GitLab UI with the Merge Request features and you are using "Custom CI configuration path" for your gitlab-ci.yml file, please ensure
this user needs at least read permissions to that remote file, even if you moved it to another repo on purpose (e.g. use enhanced file protection in PREMIUM/ULTIMATE or push/merge protect the branches for the Developer role)
the user got this permission change applied in a running session
The last part failed for our users, as it worked one day later. Seems that they just continued working from their open merge request page and GitLab checks the accessibility out of this session (using a cookie, token or something which was not updated with the the access to the remote repo/file)
It works!
I manage MapR based large scale infrastructure running on on prem dc's. As part of configuration management enhancement we have written several of playbooks and keeping everything in github. Now I dont want anyone to download/clone those repo local to Ansible client nodes and run it from there. Is there a way where i can run playbooks from ansible without downloading to local machine. So basically what i want, a script/playbook where i pass which playbook to run, it should download that playbook and run it locally.
You're looking for some web interface that users will simply run your tasks, and in the background it will execute Ansible.
There are many methods to achieve what you need, however most likely you're looking for any of this:
AWX project - official ansible web interface
Jenkins or Rundeck - more bloated software that you can create your own "jobs" for users to interact with, create CI/CD flows and cron tasks to run any time you need.
You can also look into workflow automation, such as Airflow
There are alternatives to all the mentions I put, so be sure to check everything when deciding what you need.
I'm trying to achieve an "password-free" deployment workflow using Gitlab CI and Ansible.
Some steps do require a password (I'm already using SSH Keys whenever I can) so I've stored those password inside an Ansible Vault. Next, I would just need to provide the Vault password when running the playbook.
But how could I integrate this nicely with Gitlab CI?
May I register a gitlab-ci job (or jobs are suitable for builds only?), which just runs the playbook providing the vault password somehow?! Can this be achieved without a password laying around in plain text?!
Also, I would be really happy if someone can point me some material that shows how we can deploy builds using Ansible. As you can notice, I've definitively found nothing about that.
You can set an environment variable in the GitLab CI which would hold the Ansible Vault password. In my example i called it $ANSIBLE_VAULT_PASSWORD
Here is the example for .gitlab-ci.yml:
deploy:
only:
- master
script:
- echo $ANSIBLE_VAULT_PASSWORD > .vault_password.txt
- ansible-playbook -i ansible/staging.yml --vault-password-file .vault_password.txt
Hope this trick helps you out.
I'm not super familiar with gitlab ci, or ansible vault for that matter, but one strategy that I prefer for this kind of situation is to create a single, isolated, secure, and durable place where your password or other secrets exist. A private s3 bucket, for example. Then, give your build box or build cluster explicit access to that secure place. Of course, you'll then want to make sure your build box or cluster are also locked down, such as within a vpc that isn't publicly accessible and can only be accessed via vpn or other very secure means.
The idea is to give the machines that need your password explicit knowledge of where to get it AND seperately the permission & access they need to get it. The former does not have to be a secret (so it can exist in source control) but the latter is virtually impossible to attain without compromising the entire system, at which point you're already boned anyway.
So, more specifically, the machine that runs ansible could be inside the secure cluster. It knows how to access the password. It has permission to do so. So, it can simply get the pw, store as a variable, and use it to access secure resources as it runs. You'll want to be careful not to leak the password in the process (like piping ansible logs with the pw to somewhere outside the cluster, or even anywhere perhaps). If you want to kick off the ansible script from outside the cluster, then you would vpn in to run the ansible playbook on the remote machine.
I have recently started using ansible for configuration management of linux servers.
My habbit is that if I learn one tool then I try to use it as much as possible.
Initially for my php web apps I had a long Makefile which used to download, install packages , make php.ini file chnages , extract zip files , copy files between folders etc to deploy my application in as automated way.
Now, I am thinking of converting that Makefile deployment to Ansible because then I can arrange the separate yml file for separate areas rather than one big makefile for the whole project.
I want to know that is it good idea to use ansible for that or Makefile will be good for that.
Sure, Ansible is great for that. You can separate all your different steps into different playbooks that are identified by yaml files.
You can define common tasks and then include them in your specific playbooks.
You can also make use of Ansible roles to create complete set of playbooks depending on the role of the server. For example, one set servers' role could be webservers and another set of servers' role could be databases.
You can find more info on roles here: http://docs.ansible.com/playbooks_roles.html
There are's also a few modules on the web out there that you can also use to get you started and you can also use Ansible Galaxy to import roles.
Of course, you can accomplish the same by breaking down your Makefile but maybe you want to learn a new tool.
Hope it helps.
I have installed Nexus 2.0.6 and the integration with my LDAP server is working (authentication only). Is there a way to configure Nexus so that only a set of developers are able to deploy a given artifact, or group?
I basically do not want a developer in Team A deploying an artifact that Team B is reponsible for.
You may partition a repository using repository targets, privileges, and roles as described in this Sonatype blog post. I tested this a few months back and it seemed to work well.
Under Nexus 2.0.1, this is the solution that I've worked through and verified.
Caveats:
This allows one to restrict the "Artifact Upload" option to a directory hierarchy.
This is not quite the same as "deploy" or in conjunction with "release prepare".
Steps:
Remove the single privilege of "Artifact Upload" from all that presently have it, that you do not want to have it.
Create the "Repository-Target" that includes the directory mask of where you want to give access. Ex. .*/com/mycompany/target-dir/.* Double check the mask, it starts with period-asterisk AND ends with period-asterisk.
Create the "Repository-Privilege" that maps to the Repository-Target you just created. Typically I include the repo-name in this repository-privilege-name, like "releases.com.mycompany.target-dir". This creates the CRUD entries (4) for said directory.
Next, create the "Role" based on the Repository Privileges you just created (4), plus "Artifact Upload". Total of 5 entries for the role. Use the "Apply Filter" here to help you find these items. I name this role something like "Upload.Role.Releases.Target-Dir". Again, when finished there should be 5 entries for this role:
Artifact Upload
com.mycompany.target-dir(create)
com.mycompany.target-dir(delete)
com.mycompany.target-dir(read)
com.mycompany.target-dir(update)
This is what constrains the upload role to this repo.directory.
Create the User if it does not already exist, assign password.
Give the user the following Roles:
Nexus Deployment
Repo: All Maven Repositories(Read)
UI: Base UI Privileges
Upload.Role.Relases.Target-Dir
Hit save and you've got it. Later on, you can use this same approach to create a new Role for other repos (say Snapshot or 3rd Party), and keep the same directory mask.