Chef Solo - Role Data in Ruby DSL or JSON? - ruby

I am playing with Roles with Chef Solo (11.4.4 and 11.6.0). A bit Confused.
For Chef Solo runs, should roles be written in Ruby or JSON?
As per the official docs: About Roles, Roles can be stored as domain-specific Ruby (DSL) files or JSON data.
NOTE: chef-client uses Ruby for Roles, when these files are uploaded to Chef Server, they are converted to JSON. Whenever chef-repo is refreshed, the contents of all domain-specific Ruby files are re-compiled to JSON and re-uploaded to the server.
My question is, if the requirement is to run Chef in solo mode without a server and roles are needed, should the roles be written in Ruby or JSON (we don't have a server to convert Ruby to JSON)?
My guess is the latter. Does anyone know the correct answer?
BTW: I've seen people mixing Ruby and JSON in role files...
What is the Ruby DSL equivalent for rbenv.rb below?
Example, run rbenv + ruby-build cookbooks to install rbenv on Ubuntu.
rbenv.json
{
"run_list": ["role[rbenv]"]
}
roles/rbenv.rb
name "rbenv"
description "rbenv + ruby-build"
run_list(
"recipe[rbenv]",
"recipe[ruby_build]"
)
override_attributes(
:rbenv => {
:git_repository => "https://github.com/sstephenson/rbenv.git"
},
:ruby_build => {
:git_repository => "https://github.com/sstephenson/ruby-build.git"
}
)
Chef Solo run chef-solo -c solo.rb -j rbenv.json -l debug works as expected. This is to achieve cloning via HTTPS because it easier behind the firewall.
However, using a Ruby DSL version of role rbenv.rb like below
name "rbenv"
description "rbenv + ruby-build"
run_list "recipe[rbenv]", "recipe[ruby_build]"
# default_attributes ":rbenv" => {":install_prefix" => "/opt"}
override_attributes ":rbenv" => {":git_repository" => "https://github.com/sstephenson/rbenv.git"}, ":ruby_build" => {":git_repository" => "https://github.com/sstephenson/ruby-build.git"}
It didn't seem to work because it still used the default attributes (clone via git URL instead of HTTPS).
I am new to Ruby so most likely I made some mistakes in the DSL code, please help;-)
* git[/opt/rbenv] action sync[2013-09-03T03:44:53+00:00] INFO: Processing git[/opt/rbenv] action sync (rbenv::default line 91)
[2013-09-03T03:44:53+00:00] DEBUG: git[/opt/rbenv] finding current git revision
[2013-09-03T03:44:53+00:00] DEBUG: git[/opt/rbenv] resolving remote reference
================================================================================
Error executing action `sync` on resource 'git[/opt/rbenv]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '128'
---- Begin output of git ls-remote "git://github.com/sstephenson/rbenv.git" master* ----
STDOUT:
STDERR: fatal: unable to connect to github.com:
github.com[0: 192.30.252.128]: errno=Connection timed out
---- End output of git ls-remote "git://github.com/sstephenson/rbenv.git" master* ----
Ran git ls-remote "git://github.com/sstephenson/rbenv.git" master* returned 128

I prefer to use JSON format wherever possible for one simple reason - it's easy to parse and validate with a script. Here are three things that you can do if all your Chef data is in JSON format:
Easily perform a syntax check in a git pre-commit hook, something that's much harder to do when the file is in the Ruby DSL format.
Validate the keys and values in a data bag entry. This can be useful to check that you are not going to deploy invalid or nonsensical data bag entries to production.
Compare (with a little extra work - key ordering in a dictionary needs to be taken into account) the value of an object on a server with what's in git. The --format json argument is useful here.

Related

compute sha-256 using chef

I have a code to compare sha-256 value from the file and artifactory which throws error if the checksum does not match. But I want to compute checksum in my own recipe rather than reading it from artifactory. Below is the code I have so far and trying to find out if there are any functions or methods which I can use in my recipe to compute sha-256 for a file.
Thanks in advance
only_if { node['abc'] }
not_if { ::File.exist?(checksum_file) and ::File.read(checksum_file).strip==coordinates['checksum'].strip }
message 'The previously deployed checksum is not aligned with the actual value'
level :debug
notifies :create, 'remote_file[download file]', :immediately
end```
chef is built on top of ruby, so we can unleash the power of ruby within recipes.
if you would like to compute the sha-256 checksome for a file, the following ruby code might be handy
require 'digest'
Digest::SHA256.file('/path/to/file').hexdigest
the snippet in your post, does not specify which chef resource are you using, although it feels like you are using the the log resource.
since you mentioned that you are interested in downloading the file from artifactory, remote_file resource can do that and it has a checksum property:
checksum: Optional, see use_conditional_get. The SHA-256 checksum of the file. Use to prevent a file from being re-downloaded. When the local file matches the checksum, Chef Infra Client does not download it.

Puppet: generate statement fails when trying to retrieve default path of an executable

I have built a stanza to remove a ruby gem package from our servers. The problem is that the ruby gem executable is installed in different paths on the servers, so on one server it could be in /opt/ruby/bin/gem on other servers it's in /usr/local/rvm/rubies/ruby-2.0.0-p353/bin/gem
My stanza uses the generate function in puppet to pull out the default ruby gem installation as follows:
$ruby_gem_location = generate('which', 'gem')
exec { "remove-remote_syslog":
command => "gem uninstall remote_syslog",
path => "$ruby_gem_location:/opt/ruby/bin:/usr/bin:/usr/sbin",
onlyif => "$ruby_gem_location list|grep remote_syslog"
}
When I run puppet agent I get the following error:
Generators must be fully qualified at ****redacted*
I have also tried to provide a default path for the which command as follows:
$ruby_gem_location = generate('/usr/bin/which', 'gem')
and now the error says : Could not evaluate: Could not find command '/usr/bin/gem
I checked the target server and the gem command is in
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/gem
What am I doing wrong?
How can I pull out the default ruby gem location on our servers?
Thank you in advance
Your code
$ruby_gem_location = generate('/usr/bin/which', 'gem')
will generate a full path to your gem command (if it succeeds). From the result you describe, I think it is generating '/usr/bin/gem', which is perhaps a symlink to the real gem command. You are putting that into your command path instead of just the directory part, and that will not be helpful. It is not, however, the source of the error message you report.
The real problem here is that generate(), like all DSL fucntions, runs during catalog building. I infer from your results that you are using a master / agent setup, so generate() is giving you a full path to gem -- evidently /usr/bin/gem -- on the master. Since the whole point is that different servers have gem installed in different places, this is unhelpful. The actual error message arises from an attempt to execute your onlyif command with the wrong path to gem.
Your best way forward is probably to create a custom fact with which each node can report the appropriate location of the gem binary. You can then use that fact's value in your Exec, maybe:
exec { "remove-remote_syslog":
command => "$::ruby_gem_path uninstall remote_syslog",
onlyif => "$::ruby_gem_path list | grep remote_syslog"
}
Note that you don't need a path attribute if you give a complete path to the executable in the first place.
Details on creating the $::ruby_gem_path custom fact depend on a number of factors, and in their full generality they are rather too broad for SO, but PL provides good documentation.

Unable to use application_ruby cookbook with Chef 11.8.0, Cannot find a resource for bundle_options

I have been attempting to setup a chef recipe which installs ruby using RVM and then uses the application_ruby cookbook to configure the application, however I keep running into the error
NameError: Cannot find a resource for bundle_options on ubuntu version 12.04
I am using the following code
application "application setup" do
owner "ubuntu"
group "ubuntu"
repository "https://github.com/me/myapplication.git" // Real address removed
path rails_app_path
revision "master"
rails do
bundler true
precompile_assets true
bundler_deployment true
end
end
I noticed that the bundle_options was recently added, https://github.com/opscode-cookbooks/application_ruby/commit/e7719170a661a957796e8e5d58ba8f4ecd937487 however I am unable to track down if this is causing the issue. I have included
depends "application"
depends "application_ruby"
in my metadata.rb and made sure all my dependencies are installed so I am unsure what I am doing wrong at this point.
According to documentation bundle_options is an attribute of the rails resource, not a resource itself.
The only correct way of using it is INSIDE the "rails" block, so you got the message because you either used it as :
an attribute of the application resource (but outside of the "rails" block)
standalone resource (outside of any resource).
Message you mentioned is being displayed when nonexistent resource is being referenced. e.g. if you had tried to execute following code on your system:
nonexistent_resource "failure gonna happen" do
some_attribute "whatever_value"
end
you would've got a message
NameError: Cannot find a resource for nonexistent_resource on Ubuntu version 12.04
I ran into this problem today as well. It appears the problem is that commit e771917 forgot to add the necessary getter for the bundle_option. Someone filed a PR to fix it (https://github.com/poise/application_ruby/pull/44), but it has not yet been merged. I can confirm that when I made that change locally, this error went away. The forked branch in the PR is located at https://github.com/mauriciosilva/application_ruby/tree/bundle_options_fix.

Git hook post-receive not running (Ruby script)

I'm trying to get this script working: https://github.com/zmilojko/git-trello/
In .git/hooks/post-receive (with proper values of course):
#!/usr/bin/env ruby
require 'git-trello'
GitHook.new(
:api_key => 'API_KEY',
:oauth_token => 'OAUTH_TOKEN',
:board_id => 'TRELLO_BOARD_ID',
:list_id_in_progress => 'LIST_ID_IN_PROGRESS',
:list_id_done => 'LIST_ID_IN_DONE',
:commit_url_prefix => 'https://github.com/zmilojko/git-trello/commits/'
).post_receive
File is executable. If I run it in bash ($ .git/hooks/post-receive), it seems to work mostly ok (except for the fact it doesn't receive git's input to stdin).
When doing git push, the script is not run, and no error whatsoever is thrown. Also, the remote URL is of the form git#github.com:...
I'm using rbenv, although I don't see how that could be a problem, could it? If it was, at least an error should be shown, like the ruby command wasn't found or something?
post-receive is a server-side hook. I assume that you are expecting this to run on your local machine, when, from your local machine, you push to GitHub. It doesn't work like that.
Here is a link to all of the server and client side hooks for Git.
http://git-scm.com/book/ch7-3.html#Server-Side-Hooks

Make chef cookbook recipe only run once

So I use the following recipe:
include_recipe "build-essential"
node_packages = value_for_platform(
[ "debian", "ubuntu" ] => { "default" => [ "libssl-dev" ] },
[ "amazon", "centos", "fedora", "centos" ] => { "default" => [ "openssl-devel" ] },
"default" => [ "libssl-dev" ]
)
node_packages.each do |node_package|
package node_package do
action :install
end
end
bash "install-node" do
cwd Chef::Config[:file_cache_path]
code <<-EOH
tar -xzf node-v#{node["nodejs"]["version"]}.tar.gz
(cd node-v#{node["nodejs"]["version"]} && ./configure --prefix=#{node["nodejs"]["dir"]} && make && make install)
EOH
action :nothing
not_if "#{node["nodejs"]["dir"]}/bin/node --version 2>&1 | grep #{node["nodejs"]["version"]}"
end
remote_file "#{Chef::Config[:file_cache_path]}/node-v#{node["nodejs"]["version"]}.tar.gz" do
source node["nodejs"]["url"]
checksum node["nodejs"]["checksum"]
notifies :run, resources(:bash => "install-node"), :immediately
end
It successfully installed nodejs on my Vagrant VM but on restart it's getting executed again. How do I prevent this? I'm not that good in reading ruby code.
To make the remote_file resource idempotent (i.e. to not download a file already present again) you have to correctly specify the checksum of the file. You do this in your code using the node["nodejs"]["checksum"] attribute. However, this only works, if the checksum is correctly specified as the SHA256 hash of the downloaded file, no other algorithm (esp. not MD5) is supported.
If the checksum is not correct, your recipe will still work. However, on the next run, Chef will notice that the checksum of the existing file is different from the one you specified and will download the file again, thus notify the install node ressource and do the whole compile stuff.
With chef, it's important that recipes be idempotent. That means that they should be able to run over and over again without changing the outcome. Chef expects to be able to run all the recipes on a node periodically, and that should be ok.
Do you have a way of knowing which resource within that recipe is causing you problems? The remote_file one is the only one I'm suspicious of being non-idempotent, but I'm not sure offhand.
Looking at the Chef wiki, I find this:
Deprecated Behavior In Chef 0.8.x and earlier, Remote File is also
used to fetch files from the files/ directory in a cookbook. This
behavior is now provided by #Cookbook File, and use of Remote File for
this purpose is deprecated (though still valid) in Chef 0.9.0 and
later.
Anyway, the way chef tends to work, it will look to see if whatever "#{Chef::Config[:file_cache_path]}/node-v#{node["nodejs"]["version"]}.tar.gz" resolves to exists, and if it does, it should skip that resource. Is it possible that install-node deletes that file when it's finished installing? If so, chef will re-fetch it every time.
You can run a recipe only once overriding the run-list with -o modifier.
sudo chef-client -o "recipe[cookbook::recipe]"
-o RunlistItem,RunlistItem..., Replace current run list with specified items
--override-runlist
In my experience remote_file always runs when executing chef-client, even if the target file already exists. I'm not sure why (haven't dug into the Chef code to find the exact cause of the bug), though.
You can always write a not_if or only_if to control the execution of the remote_file resource, but usually it's harmless to just let it run every time.
The rest of your code looks like it's already idempotent, so there's no harm in running the client repeatedly.
There's an action you can specify for remote_file that will make it run conditionally:
remote_file 'target' do
source 'wherever'
action :create_if_missing
end
See the docs.
If you want to test whether your recipe is idempotent, you may be interested in ToASTER, a framework for systematic testing of Chef scripts.
http://cloud-toaster.github.io/
Chef recipes are executed with different configurations in isolated container environments (Docker VMs), and ToASTER reports various metrics such as system state changes, convergence properties, and idempotence issues.

Resources