I have followed below mentioned article to configure apache, mysql, php using chef cookbook.
http://gettingstartedwithchef.com/first-steps-with-chef.html
My purpose was to write a standalone application, which will do all needed setup for mysql, apache, php etc.
By following the article, I observed they are executing the chef commands to install stuffs & getting modifying the configuration files manually.
for example for the following command I wrote respective ruby equivalent:
rvm install ruby-2.1.2
ruby equivalent : `rvm install ruby-2.1.2`
Similar way, I found alternatives for the shell commands & prepared my ruby code.
I am not sure whether that's the right approach.
2nd: how we can do custom functionality, let's say bundling a gemfile inside a project repo or running rails migrations.
How to perform those operations, please guide.
There're many community created cookbooks for many applications.
You can find many cookbooks on chef supermarket.
In general, you can install packet by use your packet manager this way:
For another supported install options you can look to cookbook.
package "#{res_name} :create nginx" do
package_name 'nginx'
action :install
end
If you want install ruby rvm package, you can use gem_package statement:
gem_package 'name' do
clear_sources TrueClass, FalseClass
gem_binary String
notifies # see description
options String
package_name String, Array # defaults to 'name' if not specified
provider Chef::Provider::Package::Rubygems
source String
subscribes # see description
timeout String, Integer
version String, Array
action Symbol # defaults to :install if not specified
end
Related
Some Background
I am using Vagrant and chef-zero; chef server is not part of my
equation
I have two roles: base and infrastructure, and two
cookbooks: apt and php. The purpose of the base role is to do some basic provisioning and package management and to clone a repository from github, etc. The apt cookbook has a recipe, update_upgrade.rb which runs an execute resource to effectively call apt update, apt upgrade, apt-get autoremove and so forth
The infrastructure role sets up PHP via the php cookbook. It'll include more later, but it serves as a good example of the need I have.
The php cookbook has two recipes: add_repository.rb and install.rb. The add_repository recipe runs an execute resource to add the ondrej/php repository to apt. The install recipe iterates over a list of packages in order to install php from the added repository. This is where the need for the utility comes into play.
The crux is, I need to run apt update after I add the PHP repo and before I install its packages. What's the best way to achieve this?
What I've Tried
Initially, I had my add_repository and install resources combined into a single install recipe and had planned to use include_recipe apt::update_upgrade to insert my apt cookbook's execute resource into that recipe. However, after scouring the chef docs, and this handy article: Should I use include_recipe or add the recipe to run_list?, it seems that include_recipe doesn't run immediately where it's placed. The suggestion in said article was to use a run list to dictate precise execution of recipes, and the chef docs do suggest that the recipes execute in the order they are listed.
This led me to break up the php resources into add_repository and install and to use the following role run list for infrastructure:
"run_list": [
"recipe[php::add_repository]",
"recipe[apt::update_upgrade]",
"recipe[php::install]"
]
My php cookbook's metadata.rb has depends apt, and Chef doesn't error out. However, it also doesn't trigger the update_upgrade recipe after the add_repository one, either. Instead, the install recipe runs after add_repository which causes an error.
This all leads me to assume that role run lists are also executed in stages similar to include_recipe. Is this accurate?
My Conclusions
If these assumptions are accurate, then it seems the only sure way to update apt with the new repository is to duplicate my execute resource from my apt cookbook into my `php cookbook. The DRY-ist in me screams at this; hence the question of a utility recipe/resource.
EDIT
Using the apt Resource
As the original answerer indicates, for this specifically-described use case is to use a combination of apt_package, apt_update, and apt_repository. Here's how I've set it up to get what I want:
In the afore-mentioned apt cookbook, I now have two recipes, install.rb and update.rb. The key recipe is update:
apt_update 'update_packages' do
action :update
end
According to the documentation (https://docs.chef.io/resource_apt_update.html), the :update action will update the Apt repository at the start of a Chef Infra Client run. This is useful for me, as I want to update my OS packages when my VM is first spun up.
Here's my base role's run list which makes use of the apt::update recipe:
"run_list": [
"recipe[apt::install]",
"recipe[apt::update]",
...
]
Next, PHP. First, I want to add the ondrej/php repository, then loop through a list of packages and install them. Lastly, I want to run apt update after installing the new packages.
The first thing I do is to ensure, again, that my php cookbook's metadata.rb has:
depends 'apt'
which will allow me to reuse the apt_update['update_packages'] resource I defined in my apt cookbook.
My install.rb recipe in my php cookbook now contains the following:
# Add PHP repository
apt_repository 'php7' do
uri "ppa:#{node[:infrastructure][:php][:repository][:name]}"
action :add # I know this is unnecessary, but I like to be explicit
end
# Install PHP
node[:infrastructure][:php][:extensions][:list].each do |package|
apt_package package do
action :install
notifies :update, 'apt_update[update_packages]', :delayed
end
end
Since I loop through a list of packages stored in attributes, I notify the apt_update[update_packages] resource after all of the php packages are installed with the :delayed timer.
Finally, here's a look at the infrastructure role which handles the PHP cookbook execution:
"run_list": [
"recipe[php::install]"
]
This explanation handles the above use case beautifully and now hopefully makes use of the appropriate Chef resources. There are, however, some additional Apt actions which aren't covered with resources and which may still require some specific execute resources. Consider the recipes in this apt_cleanup cookbook which cover things like:
Cleaning up apt-cache
Purging leftovers from removed packages
Removing old unused kernels (for a cleaner /boot)
Removes dependencies dragged in by already deleted packages
Looking at those recipes, I see a collection of execute resources, which implies that for these actions, the execute approach may still be necessary. (Albeit within a cookbook).
the best practice is to use the appropriate chef resource in conjunction with notification.
since you are using apt, use the appropriate apt resources, such as apt_update (to make sure apt repositories are up to date), apt_package (to manipulate packages using apt, apt_repository (to add apt respository) and try to avoid using execute resource to do all of that - this will take advantage of chef idompotance (you might like reading Thinking Like a Chef)
each chef resource, can notify and\or subscribe to notifications. but you might not need to use it if you write the resources in order.
if you these advises won't lead you the right way, please post a relevant snippet where you have an issue and i will try to help you farther.
UPDATE
since apt suffers from many failures during update, i would give it few retries and also a periodic update.
apt_update 'update_packages' do
retries 3
action [:update, :periodic]
end
depending of the chef-client version that you are using, apt resouce has been bundles into chef standard resource collection, rather than using the apt cookbook.
so you might not need to specify the dependency on apt cookbook in the metadata.rb file. also, make sure you have no collision between the builtin apt and the apt cookbook functionality.
if you like all your nodes to have the apt_update step at boot time, then place the recipe which utilizes apt_update at your base chef role and make sure that each node runs this recipe (by running the base role) first in the node run-list.
when using apt_repository you did not specify any options for the resource, like arch and others. make sure that this is what you want.
when php packages are being installed using apt_package, there is no need to update apt repositories again. thus, there is no need to notify apt_update each time apt_package is invoked.
I'm attempting to create a Chef cookbook that, for now, is mostly just a wrapper cookbook for another cookbook (the audit cookbook). I'm still learning Chef, but from what I can gather from the About Recipes documentation and the Resources Reference documentation, Chef recipes should execute in the order that they're defined (via Ruby code and/or Chef resources).
In the About Recipes documentation, it mentions that
When a recipe is included, the resources found in that recipe will be
inserted (in the same exact order) at the point where the
include_recipe keyword is located.
In the Resources Reference documentation, they have an apt_update resource that presumably executes before the include_recipe method due to the fact that it's defined earlier in the recipe.
My wrapper cookbook has a single recipe, default.rb, which is literally these two lines:
package 'ruby-dev'
include_recipe 'audit'
However, during a chef-client or chef-solo run I see that the audit::inspec recipe runs before the security::default recipe which causes things to break because InSpec has some other dependencies that need to be installed beforehand. Before I used the package resource I was using the execute resource to explicitly run apt-get install ruby-dev or yum install ruby-dev depending on the platform using a case statement, but same problem (all that code was skipped and the include_recipe method called first).
In case it's useful, I'm using Chef 12 which I realize is EOL but I have other dependencies that require me to stick with this version of Chef for now.
I may very well just be misunderstanding how Chef converges work and the order in which execution occurs, but it's causing me a lot of grief so I'd really appreciate some pointers! Does include_recipe always occur before other code within your recipe? Is there any way around this? Am I missing something?
-- EDIT --
I was able to get the desired functionality (install other packages and gems before an include_recipe call triggered installation of a dependency gem) using the following code in my cookbook recipe:
package 'build-essential' do
action :nothing
end.run_action(:install)
chef_gem 'train' do
version "1.4.4"
action :install
end
chef_gem 'signet' do
version "0.11.0"
action :install
end
include_recipe 'audit'
Note that I ended up installing the build-essential package rather than the ruby-dev package from my original code snippet, and also installed two gems for Chef client to use. This all gets installed in the order I expected during the compile phase of the Chef run.
Sources:
https://docs.chef.io/resource_reference.html#run-in-compile-phase
https://docs.chef.io/resource_chef_gem.html
if you would examine the audit::inspec rescipe, you will find that it uses a compile time installation of the inspec rubygem (see the last line)
inspec_gem 'inspec' do
version node['audit']['inspec_version']
source node['audit']['inspec_gem_source']
action :nothing
end.run_action(:install)
from chef documentation:
run_action
Use .run_action(:some_action) at the end of a resource block to run the specified action during the compile phase.
I am experimenting with Puppet using Vagrant. I'm new to Puppet.
I'm installing modules in my Puppet manifest using the approach suggested at: Can I install puppet modules through puppet manifest?
My default.pp contains something like:
$dsesterojava = 'dsestero-java'
exec { 'dsestero-java':
command => "puppet module install ${dsesterojava}",
unless => "puppet module list | grep ${dsesterojava}",
path => ['/usr/bin', '/bin']
}
include java::java_7
I'm trying to import a module and then immediately use the classes defined in it.
Currently, I get:
Error: Could not find class java::java_7
If I comment out the include line and re-run it. The module installs. If I then removed the comment and run the provisioning again then it works.
There is some kind of "chicken and egg" situation here. Can I use a module in the same Puppet manifest that installs it?
How should I solve it?
No, you cannot do this. When your catalog is compiled, Puppet will search in the appropriate directories for all of the required code and data. Since the java module does not exist until catalog application, the compilation of a catalog (occurs prior to application) depending upon it will fail. You are absolutely dealing with a "chicken and egg" situation here. I highly recommend against using Puppet code to install Puppet code.
Alternatively, the recommended approach to install and manage your Puppet modules is to use one of these solutions:
librarian-puppet: http://librarian-puppet.com/
r10k: https://github.com/puppetlabs/r10k
code-manager (PE only): https://puppet.com/docs/pe/2017.3/code_management/code_mgr.html
These will also solve the problem for you within the Vagrant if you are using the agent provisioner and subscribing the Vagrant instance to a Puppet Master.
If you are using the apply provisioner inside of Vagrant, then you will need to go a different route. The simplest solution is to use the shell provisioner to install Puppet modules via module install after the Puppet installation (unless you are using a Vagrant box with Puppet baked in, in which case you are probably not installing Puppet on it). Alternatively, you could share a directory with the host where your modules are installed, or install the librarian-puppet or r10k gems onto the Vagrant box and then use them to install into the appropriate path. I can go into more detail on these upon request.
I'm working with someone else's chef recipe and it consists of these references to the process of installing 1.9.3p0 on my server:
package 'ruby1.9.3'
package 'ruby1.9.1-dev'
# set ruby 1.9 to be default
execute 'update-alternatives --set ruby /usr/bin/ruby1.9.1'
execute 'update-alternatives --set gem /usr/bin/gem1.9.1'
ohai "reload" do
action :reload
end
I'm new to chef so I'm not sure where these packages reside, but seeing no other reference them to them in the repo of recipes, I'm guessing it's referring to a central repo. In that case, how could I modify this recipe to get chef (solo) to prepare my servers with a different patch level?
The documentation may clear things up a little here:
package tells the chef-client to use one of sixteen different
providers during the chef-client run, where the provider that is used
by chef-client depends on the platform of the machine on which the
chef-client run is taking place
So on Debian-based systems like the one that recipe was written for, Chef will automatically resolve the package resource to an apt_package resource, which will call apt-get to install ruby1.9.3.
Now, given none of the mainstream Linux distros or FreeBSD package up multiple patchlevels of Ruby (and, in some cases, stated patchlevels are not what they seem), you probably don't want to use package to get Ruby. Most likely you'll end up wanting to build it from source using something like the bash resource.
package will still be useful for installing the Ruby prerequisites, which you can use from your vendor's package repository without issue.
I have a custom Ruby library directory that I'd like to have automatically added to Ruby's load path whenever Ruby is executed. I know I can use the -I option to Ruby, but is there something like an environment variable that I can set that will globally determine Ruby's load path.
I want to install Ruby Gems on a Linux box where I don't have root privileges, so I need to have a Ruby load path in a non-standard location. I installed RubyGems per "Installing RubyGems in a User Directory", but the gem command isn't picking up the non-standard load path.
Maybe I'm missing something obvious here and making things harder for myself?
See the "Ruby and Its World" chapter from The Pickaxe Book, specifically the section on environment variables. Excerpt:
RUBYLIB
Additional search path for Ruby programs ($SAFE must be 0).
DLN_LIBRARY_PATH
Search path for dynamically loaded modules.
RUBYLIB_PREFIX
(Windows only) Mangle the RUBYLIB search path by adding this
prefix to each component.
Make sure that you've placed the installed bin directory in your $PATH for the gem
command to work. It should modify the RUBYLIB itself, but if not, try Martin's answer to fix that.
Then, you can have your gem home (where the gems that rubygems installs are stored) be local.
Just use $GEM_HOME (or set things up in your ~/.gemrc) and check that everything took with gem environment.
% mkdir ~/.gems
% export GEM_HOME=~/.gems
% gem help environment
Usage: gem environment [arg] [options]
Common Options:
-h, --help Get help on this command
-V, --[no-]verbose Set the verbose level of output
-q, --quiet Silence commands
--config-file FILE Use this config file instead of default
--backtrace Show stack backtrace on errors
--debug Turn on Ruby debugging
Arguments:
packageversion display the package version
gemdir display the path where gems are installed
gempath display path used to search for gems
version display the gem format version
remotesources display the remote gem servers
display everything
Summary:
Display information about the RubyGems environment
Description:
The RubyGems environment can be controlled through command line arguments,
gemrc files, environment variables and built-in defaults.
Command line argument defaults and some RubyGems defaults can be set in
~/.gemrc file for individual users and a /etc/gemrc for all users. A gemrc
is a YAML file with the following YAML keys:
:sources: A YAML array of remote gem repositories to install gems from
:verbose: Verbosity of the gem command. false, true, and :really are the
levels
:update_sources: Enable/disable automatic updating of repository metadata
:backtrace: Print backtrace when RubyGems encounters an error
:bulk_threshold: Switch to a bulk update when this many sources are out of
date (legacy setting)
:gempath: The paths in which to look for gems
gem_command: A string containing arguments for the specified gem command
Example:
:verbose: false
install: --no-wrappers
update: --no-wrappers
RubyGems' default local repository can be overriden with the GEM_PATH and
GEM_HOME environment variables. GEM_HOME sets the default repository to
install into. GEM_PATH allows multiple local repositories to be searched
for
gems.
If you are behind a proxy server, RubyGems uses the HTTP_PROXY,
HTTP_PROXY_USER and HTTP_PROXY_PASS environment variables to discover the
proxy server.
If you are packaging RubyGems all of RubyGems' defaults are in
lib/rubygems/defaults.rb. You may override these in
lib/rubygems/defaults/operating_system.rb
Make life easy and install RVM. It will install whatever version of Ruby you want and let you switch between them and it doesn't require root access. It has many other killer features you will become addicted to after using it for a while.
I used #MartinCarpenter's solution to run a specific/particular/single test method with minitest. Where I normally add the test directory to the $LOAD_PATH with Rake::TestTask, e.g., t.libs << 'test', I was able to do it with the command line, like so:
RUBYLIB=test ruby test/user_test.rb --name test_create
I added test to $LOAD_PATH because user_test.rb calls require 'test_helper' to load lib/test_helper.rb.