I'm new to chef and planning for disaster recovery plan such that will be having all my configuration files, list of packages installed on system in chef cookbooks. I wondering how could i mention a text file in recipe for the package installation.
I have taken list of packages with dpkg --get-selections > packages.txt.
I would like chef to read a package from that list one by one and install it. Would be great if someone can guide me on this.
Thanks,
Swaroop.
Blueprint could help you for your task. It inspects the system and generates a chef cookbook out of it.
Disclaimer: I haven't tried it, yet. But it looks very interesting.
I am not aware of a way to read a file from a cookbook. But you can install a number of packages with the following code in a recipe:
%w{
package-1
package-2
}.each do |pkg|
package pkg
end
Related
Some Background
I am using Vagrant and chef-zero; chef server is not part of my
equation
I have two roles: base and infrastructure, and two
cookbooks: apt and php. The purpose of the base role is to do some basic provisioning and package management and to clone a repository from github, etc. The apt cookbook has a recipe, update_upgrade.rb which runs an execute resource to effectively call apt update, apt upgrade, apt-get autoremove and so forth
The infrastructure role sets up PHP via the php cookbook. It'll include more later, but it serves as a good example of the need I have.
The php cookbook has two recipes: add_repository.rb and install.rb. The add_repository recipe runs an execute resource to add the ondrej/php repository to apt. The install recipe iterates over a list of packages in order to install php from the added repository. This is where the need for the utility comes into play.
The crux is, I need to run apt update after I add the PHP repo and before I install its packages. What's the best way to achieve this?
What I've Tried
Initially, I had my add_repository and install resources combined into a single install recipe and had planned to use include_recipe apt::update_upgrade to insert my apt cookbook's execute resource into that recipe. However, after scouring the chef docs, and this handy article: Should I use include_recipe or add the recipe to run_list?, it seems that include_recipe doesn't run immediately where it's placed. The suggestion in said article was to use a run list to dictate precise execution of recipes, and the chef docs do suggest that the recipes execute in the order they are listed.
This led me to break up the php resources into add_repository and install and to use the following role run list for infrastructure:
"run_list": [
"recipe[php::add_repository]",
"recipe[apt::update_upgrade]",
"recipe[php::install]"
]
My php cookbook's metadata.rb has depends apt, and Chef doesn't error out. However, it also doesn't trigger the update_upgrade recipe after the add_repository one, either. Instead, the install recipe runs after add_repository which causes an error.
This all leads me to assume that role run lists are also executed in stages similar to include_recipe. Is this accurate?
My Conclusions
If these assumptions are accurate, then it seems the only sure way to update apt with the new repository is to duplicate my execute resource from my apt cookbook into my `php cookbook. The DRY-ist in me screams at this; hence the question of a utility recipe/resource.
EDIT
Using the apt Resource
As the original answerer indicates, for this specifically-described use case is to use a combination of apt_package, apt_update, and apt_repository. Here's how I've set it up to get what I want:
In the afore-mentioned apt cookbook, I now have two recipes, install.rb and update.rb. The key recipe is update:
apt_update 'update_packages' do
action :update
end
According to the documentation (https://docs.chef.io/resource_apt_update.html), the :update action will update the Apt repository at the start of a Chef Infra Client run. This is useful for me, as I want to update my OS packages when my VM is first spun up.
Here's my base role's run list which makes use of the apt::update recipe:
"run_list": [
"recipe[apt::install]",
"recipe[apt::update]",
...
]
Next, PHP. First, I want to add the ondrej/php repository, then loop through a list of packages and install them. Lastly, I want to run apt update after installing the new packages.
The first thing I do is to ensure, again, that my php cookbook's metadata.rb has:
depends 'apt'
which will allow me to reuse the apt_update['update_packages'] resource I defined in my apt cookbook.
My install.rb recipe in my php cookbook now contains the following:
# Add PHP repository
apt_repository 'php7' do
uri "ppa:#{node[:infrastructure][:php][:repository][:name]}"
action :add # I know this is unnecessary, but I like to be explicit
end
# Install PHP
node[:infrastructure][:php][:extensions][:list].each do |package|
apt_package package do
action :install
notifies :update, 'apt_update[update_packages]', :delayed
end
end
Since I loop through a list of packages stored in attributes, I notify the apt_update[update_packages] resource after all of the php packages are installed with the :delayed timer.
Finally, here's a look at the infrastructure role which handles the PHP cookbook execution:
"run_list": [
"recipe[php::install]"
]
This explanation handles the above use case beautifully and now hopefully makes use of the appropriate Chef resources. There are, however, some additional Apt actions which aren't covered with resources and which may still require some specific execute resources. Consider the recipes in this apt_cleanup cookbook which cover things like:
Cleaning up apt-cache
Purging leftovers from removed packages
Removing old unused kernels (for a cleaner /boot)
Removes dependencies dragged in by already deleted packages
Looking at those recipes, I see a collection of execute resources, which implies that for these actions, the execute approach may still be necessary. (Albeit within a cookbook).
the best practice is to use the appropriate chef resource in conjunction with notification.
since you are using apt, use the appropriate apt resources, such as apt_update (to make sure apt repositories are up to date), apt_package (to manipulate packages using apt, apt_repository (to add apt respository) and try to avoid using execute resource to do all of that - this will take advantage of chef idompotance (you might like reading Thinking Like a Chef)
each chef resource, can notify and\or subscribe to notifications. but you might not need to use it if you write the resources in order.
if you these advises won't lead you the right way, please post a relevant snippet where you have an issue and i will try to help you farther.
UPDATE
since apt suffers from many failures during update, i would give it few retries and also a periodic update.
apt_update 'update_packages' do
retries 3
action [:update, :periodic]
end
depending of the chef-client version that you are using, apt resouce has been bundles into chef standard resource collection, rather than using the apt cookbook.
so you might not need to specify the dependency on apt cookbook in the metadata.rb file. also, make sure you have no collision between the builtin apt and the apt cookbook functionality.
if you like all your nodes to have the apt_update step at boot time, then place the recipe which utilizes apt_update at your base chef role and make sure that each node runs this recipe (by running the base role) first in the node run-list.
when using apt_repository you did not specify any options for the resource, like arch and others. make sure that this is what you want.
when php packages are being installed using apt_package, there is no need to update apt repositories again. thus, there is no need to notify apt_update each time apt_package is invoked.
I am experimenting with Puppet using Vagrant. I'm new to Puppet.
I'm installing modules in my Puppet manifest using the approach suggested at: Can I install puppet modules through puppet manifest?
My default.pp contains something like:
$dsesterojava = 'dsestero-java'
exec { 'dsestero-java':
command => "puppet module install ${dsesterojava}",
unless => "puppet module list | grep ${dsesterojava}",
path => ['/usr/bin', '/bin']
}
include java::java_7
I'm trying to import a module and then immediately use the classes defined in it.
Currently, I get:
Error: Could not find class java::java_7
If I comment out the include line and re-run it. The module installs. If I then removed the comment and run the provisioning again then it works.
There is some kind of "chicken and egg" situation here. Can I use a module in the same Puppet manifest that installs it?
How should I solve it?
No, you cannot do this. When your catalog is compiled, Puppet will search in the appropriate directories for all of the required code and data. Since the java module does not exist until catalog application, the compilation of a catalog (occurs prior to application) depending upon it will fail. You are absolutely dealing with a "chicken and egg" situation here. I highly recommend against using Puppet code to install Puppet code.
Alternatively, the recommended approach to install and manage your Puppet modules is to use one of these solutions:
librarian-puppet: http://librarian-puppet.com/
r10k: https://github.com/puppetlabs/r10k
code-manager (PE only): https://puppet.com/docs/pe/2017.3/code_management/code_mgr.html
These will also solve the problem for you within the Vagrant if you are using the agent provisioner and subscribing the Vagrant instance to a Puppet Master.
If you are using the apply provisioner inside of Vagrant, then you will need to go a different route. The simplest solution is to use the shell provisioner to install Puppet modules via module install after the Puppet installation (unless you are using a Vagrant box with Puppet baked in, in which case you are probably not installing Puppet on it). Alternatively, you could share a directory with the host where your modules are installed, or install the librarian-puppet or r10k gems onto the Vagrant box and then use them to install into the appropriate path. I can go into more detail on these upon request.
I have found this DNSimple ansible module:
http://docs.ansible.com/ansible/dnsimple_module.html
but can not find anywhere on that page to download and install it? How do I go about downloading and installing ansible modules like this. Thanks.
The accepted answer solved the questioner's problem but didn't address the broader scope of the question.
How to install an Ansible module? The documentation is currently vague as to how to achieve this simple requirement!
An excellent general guide to writing modules (I've no connection to the author) can be found here.
The quickest way is to simply have a folder called library/ in the same folder as your playbook. Inside this folder, place the python script for the Ansible Module. You should now have a corresponding task available to your playbook.
If you want to share your module across multiple projects, then you can add an entry to /etc/ansible/ansible.cfg pointing to a shared library location, eg:
library = /usr/share/ansible/library
The module itself is part of ansible since version 1.6 (as stated here). To use it, you need to have dnsimple on your host machine (also stated in the above description). Install it with sudo pip install dnsimple
It is important to know that base ansible modules are not installed by default on devel version, which is the default installed version when you build from source.
Only few modules are present for developpment purpose.
So when you'll run your playbook it'll complain about not found module with following error message
couldn't resolve module/action 'xxx'
If you have no choice but building for source, don't forget to checkout the stable branch to install all basic ansible modules!
So I am tasked with packaging a Ruby application. The idea is to eliminate the need to apt-get any packages or gem install anything. the package is all-inclusive.
I've gotten as far as finding Omnibus (there is also fmp-cookery but it doesnt seem to be as well known)
I've figured out the general structure of omnibus projects but now getting problems with specifics:
1) I have a bunch of recipies that run with Chef AT INSTALL TIME. These setup the DB and nginx configs. Whats the best way to run through these on target machine?
2) I added a bundle install line in my build scripts, but this now dubplicates my requirements. For example nokogiri gets installed twice. Once as my actual project Gem and another time as another requirement of omnibus. So then I end up with 2 binaries one in /#{instal_dir}/embedded/bin and one in ${install_dir}/embedded/lib/ruby/.../gems any way to prevent this?
Building an installer is bit more involved than creating a chef cookbook.
User needs to write a DSL for each module (this has instructions on gathering the artifacts and installing it on specific OS).
After creating the omnibus project, refer to the README file to get started with basics.
Refer to this github project for details on how to build your own DSLs. Gitlab omnibus project has DSLs for some modules like postgresql etc.
https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master
Example DSL for embedding nginx module in your installer looks like this:
name "nginx"
default_version "1.9.10"
dependency "pcre"
dependency "openssl"
source url: "http://nginx.org/download/nginx-#{version}.tar.gz",
md5: "64cc970988356a5e0fc4fcd1ab84fe57"
relative_path "nginx-#{version}"
build do
command ["./configure",
"--prefix=#{install_dir}/embedded",
"--with-http_ssl_module",
"--with-http_stub_status_module",
"--with-http_gzip_static_module",
"--with-http_v2_module",
"--with-ipv6",
"--with-debug",
"--with-ld-opt=-L#{install_dir}/embedded/lib",
"--with-cc-opt=\"-L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include\""].join(" ")
command "make -j #{workers}", :env => {"LD_RUN_PATH" => "#{install_dir}/embedded/lib"}
command "make install"
end
I'm working with someone else's chef recipe and it consists of these references to the process of installing 1.9.3p0 on my server:
package 'ruby1.9.3'
package 'ruby1.9.1-dev'
# set ruby 1.9 to be default
execute 'update-alternatives --set ruby /usr/bin/ruby1.9.1'
execute 'update-alternatives --set gem /usr/bin/gem1.9.1'
ohai "reload" do
action :reload
end
I'm new to chef so I'm not sure where these packages reside, but seeing no other reference them to them in the repo of recipes, I'm guessing it's referring to a central repo. In that case, how could I modify this recipe to get chef (solo) to prepare my servers with a different patch level?
The documentation may clear things up a little here:
package tells the chef-client to use one of sixteen different
providers during the chef-client run, where the provider that is used
by chef-client depends on the platform of the machine on which the
chef-client run is taking place
So on Debian-based systems like the one that recipe was written for, Chef will automatically resolve the package resource to an apt_package resource, which will call apt-get to install ruby1.9.3.
Now, given none of the mainstream Linux distros or FreeBSD package up multiple patchlevels of Ruby (and, in some cases, stated patchlevels are not what they seem), you probably don't want to use package to get Ruby. Most likely you'll end up wanting to build it from source using something like the bash resource.
package will still be useful for installing the Ruby prerequisites, which you can use from your vendor's package repository without issue.