Our whole infrastructure is managed through ansible. We have a gitlab repo where all our ansible playbooks and roles are stored in. We recently added working gitlab-runner working, therefore we have to possibility to have a CI for each project. As our ansible repo is quite important, I was wondering if there is anything interesting to check ? I thought doing a ansible-playbook --syntax-check, but I'd like to know if there are more tests useful (maybe a linter ?).
I don't want to automatically run the playbooks, but only assert that the newly pushed roles/modifications are "good".
Thanksies,
The first thing you definitely want to run is yamllint which will make sure that all your yaml files are ok and respecting your coding standard. Make sure you configure the checks correctly in your .yamllint to be compatilble with ansible-lint below if you decide to use it (especially the line length). The following basic config should be a good start:
extends: default
rules:
line-length:
max: 160
level: warning
You can then have your current syntax check
Finally, you probably want to run ansible-lint. Although it will report most errors you will find above, this one will concentrate on good ansible practices, like not using shell when there is a module available, having names for all tasks, etc...
Side note: These are all default basic checks made by molecule when you use it to test your roles. If you are developing independant reusable roles, I suggest you have a look at this tool.
If you don't want to run your playbooks, this is pretty much all I can see.
Related
I have several roles that run actions on a remote database that executes sentences for user and privilege creation.
I have seen molecule used to test playbooks that run against a single host, but I am unsure of how you could setup a second container to run a docker instance in the same network as the molecule container (similar to a docker-compose setup). However I have not been able to find a setup like this in the documentation.
Is there a recommended way to run molecule tests with external dependencies? Or should I just use docker-compose or similar to run my tests?
There is a 'prepare' stage in Molecule specifically for that. You need to separate questions:
where external resource (database) is run?
why and how it's configured?
Those are very separate, and mixing them together is a bad idea.
For 1 there are different answers:
It is (out of blue, configured by other people). Use non-managed hosts in molecule.yml.
We OK to run it on the same host as the host we run our code. Shovel installation into 'prepare' stage.
We want it to be on separate server. Put additional host in platforms in a different group and configure it in prepare stage.
If you find your driver is not good enough, you always can opt for 'delegated' driver. In this case you need to write playbooks for create/destroy of hosts. It's relatively easy. The main trick is to use 'platforms' variable to get information about content of molecule.yaml's platform section.
I manage MapR based large scale infrastructure running on on prem dc's. As part of configuration management enhancement we have written several of playbooks and keeping everything in github. Now I dont want anyone to download/clone those repo local to Ansible client nodes and run it from there. Is there a way where i can run playbooks from ansible without downloading to local machine. So basically what i want, a script/playbook where i pass which playbook to run, it should download that playbook and run it locally.
You're looking for some web interface that users will simply run your tasks, and in the background it will execute Ansible.
There are many methods to achieve what you need, however most likely you're looking for any of this:
AWX project - official ansible web interface
Jenkins or Rundeck - more bloated software that you can create your own "jobs" for users to interact with, create CI/CD flows and cron tasks to run any time you need.
You can also look into workflow automation, such as Airflow
There are alternatives to all the mentions I put, so be sure to check everything when deciding what you need.
I have a github project that uses travis for continuous integration. I would like to deploy my project on amazon ec2. In order to simplify deployment, I would like the deployed system to have the same configuration as the test system. Is this possible?
AFAICT, this requires two things: First, an ec2 preconfigured instance that matches the settings used by travis. Does one exist? And second, a way to execute travis.yml scripts from the command line. How can I do that?
As for executing .travis.yml scripts from the command line, if I were you I would instead take it the other way around and replace your .travis.yml script with something like this:
language: bleh
etc etc...
install:
- ./travis-scripts/install.sh
before_script:
- ./travis-scripts/before_script.sh
script:
- ./travis-scripts/script.sh
Of course, you will still have to write a script for installing whatever language versions, Travis plugins etc you need on your Amazon EC2 instance.
As for an Amazon EC2 instance that matches Travis VMs, I don't know about that because I'm not so familiar with Amazon AWS, but I can tell you that Travis VMs are based on Ubuntu 12.04, and there is a lot more specific information in the page about The Build Environment.
So you want something on a EC2 instance that can read your .travis.yml file and configure it in the same way that travis does when it tests?
I think that's a pretty long shot for a relatively simple problem like this. Travis is an integration and testing platform that uses a lot of other systems (like chef and docker) to do what it does with the .yml files. To use this system to run a single app sounds a bit overkill.
I would recommend using chef (or similar like puppet) to configure your production environment and deploy your app.
You could have one chef recipe that configures the production environment (DB's, configuration files, install stuff, etc...) and another that deploys, configures and starts your app. When you want to make changes to the production environment, you make changes to these files. They can easily be bundled with the project.
I have recently started using ansible for configuration management of linux servers.
My habbit is that if I learn one tool then I try to use it as much as possible.
Initially for my php web apps I had a long Makefile which used to download, install packages , make php.ini file chnages , extract zip files , copy files between folders etc to deploy my application in as automated way.
Now, I am thinking of converting that Makefile deployment to Ansible because then I can arrange the separate yml file for separate areas rather than one big makefile for the whole project.
I want to know that is it good idea to use ansible for that or Makefile will be good for that.
Sure, Ansible is great for that. You can separate all your different steps into different playbooks that are identified by yaml files.
You can define common tasks and then include them in your specific playbooks.
You can also make use of Ansible roles to create complete set of playbooks depending on the role of the server. For example, one set servers' role could be webservers and another set of servers' role could be databases.
You can find more info on roles here: http://docs.ansible.com/playbooks_roles.html
There are's also a few modules on the web out there that you can also use to get you started and you can also use Ansible Galaxy to import roles.
Of course, you can accomplish the same by breaking down your Makefile but maybe you want to learn a new tool.
Hope it helps.
I'm investigating Puppet as our future deployment and provisioning tool in our shop, but now I'm stuck at how to make a clever Continuous Integration/Delivery tool chain with deployment through Puppet.
In any of our environments (dev, test, qa, demo, prod) we have a range of components. We need to be able to deploy each component separately and possibly even concurrently.
I'd like a way to initiate (through script) a deploy of a single component package (=Puppet module) and gather the output and success status of that.
Simply waiting for a scheduled agent pull, or doing a 'puppet agent --test' on each node on the environment isn't good enough, because it may pick up other pending changes (I don't know if another component is also in the process of being deployed).
In my tool chain I would like the deployment output and status from component A and component B to be recorded separately and not mixed up.
So my question is: Can I use puppet to deploy one single named package (module) at a time?
And if not, where did I take a wrong turn when I drove down this path?
I realise a master-less Puppet set-up with modules and manifests replicated to each node perhaps could do it, but IMHO a master-less Puppet set-up kind of defeats the purpose of Puppet.
PS: I think what I'm trying to achieve is called 'Directed Orchestration' in Damon Edwards' very enlightening video at Integrating DevOps tools into a Service Delivery Platform (at timestamp around 22:30).
So my question is: Can I use puppet to deploy one single named package (module) at a time?
Yes, you can, via puppet apply. First you need to create a moduledir and a module that will contain your manifests. e.g. :
/scratch/user/puppet/local/ # This is your modulepath for local deployment
# Following contains the manifests for a module name "localmod"
/scratch/user/puppet/local/localmod/manifests/init.pp
# example content of init.pp
class localmod {
notify{"I am in in local module....":}
}
On that local machine you can test this module via puppet apply :
puppet apply -v --modulepath=/scratch/user/puppet/local -e "include localmod"
echo $? # Get the exit status of the above command
I watched the video at the point your video. There are two types of automation you can do.
Application build/deploy automation, which can be achieved via maven/ant (Build) and ant/capistrano/chrome/bash/msdeploy (Deploy) or as termed on that slide "Installer".
System/Infrastructure automation can be achieved via Chef/Puppet/CFEngine.
This question seems to be ... "How do I do applications build using puppet (implied as a system automation tool)"
So quite simply, oval tool in round hole. (I didn't say square)
At my company, we use Jenkins and the Build Pipeline Integration plugin to build massive multi component projects. As an example, a Java app will use ant in a build job, the next chained job will be a "deploy to dev" job which uses Capistrano to deploy the application, then the next job in the chain is "Configure Dev" which calls Chef to update the system configurations in the DEV environment. Chef is used to configure the application. Each of these jobs can be set to run automatically and sequentially.
a master-less Puppet set-up kind of defeats the purpose of Puppet.
Only if you discount
The rich DSL puppet has to offer
So many peer reviewed community modules
Otherwise, something like this gives you remote directed orchestration.
#update manifests etc (version control is the source of truth)
ssh user#host git pull
#run puppet
ssh user#host sudo puppet-apply