I have built a cookbook for installing Jenkins CI. It uses the key and repository resources from the yum cookbook, so I end up with the following recipe:
yum_key "RPM-GPG-KEY-jenkins" do
url "http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key"
action :add
end
yum_repository "jenkins" do
description "Jenkins-CI 3rd party repository"
url "http://pkg.jenkins-ci.org/redhat"
key "RPM-GPG-KEY-jenkins"
action :add
end
When I include this recipe in another recipe:
include_recipe 'sp_jenkins::default'
and I test this with the following ChefSpec test
it 'includes the `sp_jenkins::default` recipe' do
expect(chef_run).to include_recipe('sp_jenkins::install')
end
my ChefSpec test fails with the following output:
NameError:
Cannot find a resource for yum_key on chefspec version 0.6.1
(I'm not sure why it says version 0.6.1, gem list tells me it's using 3.0.2)
The sp_jenkins cookbook does depend on the yum cookbook (metadata.rb), and runs fine, however, the cookbook I'm currently writing does not depend on the yum cookbook and therefore doesn't have the yum_key and yum_repository methods available.
Is there a way to prevent ChefSpec from 'descending' into included recipes/cookbooks and just test the current cookbook?
Ohai! Julian is correct - ChefSpec actually does a Chef Solo run in memory on your local machine. It rewrites the provider actions to be a noop, but creates a registry of all the actions taken (including those that would be taken if notifications were executed).
So just like you need the yum cookbook to converge this recipe on a real node, you need it to converge during your unit tests with ChefSpec. The easiest way to accomplish this is by using the Berkshelf or Librarian resolvers. To use the Berkshelf resolver, simply require 'chefspec/berkshelf' after requiring chefspec:
# spec_helper.rb
require 'chefspec'
require 'chefspec/berkshelf'
If you have Berkshelf installed on your system, it will pull all the cookbooks into a temporary directory and run ChefSpec for you.
You may also want to take a look at Strainer, which aims to solve a similar problem.
On a somewhat unrelated note, I am working on a fairly large refactor to the Jenkins cookbook that may better suit your needs.
Sources:
I wrote it...
No, there's no way to prevent it from descending, because it's trying to converge an entire Chef run in memory.
However, if you use the Berkshelf functionality in ChefSpec, the Berkshelf dependency resolver will feed all dependent cookbooks to the in-memory Chef run, and you'll be golden.
It is absolutely valid to expect to test your cookbook in isolation, and not include other projects' code into the scope of your tests. Unfortunately there appears to be no supported, "clean" way to do this, that I can find. I was able to achieve this, but it comes at a price.
To use this technique, do not require 'chefspec/berkshelf' anywhere in your test code, only chefspec itself, as you are intentionally not gathering other cookbook source. Here is a template of my working test module (not my complete test code, as I have omitted RSpec config options):
describe 'mycookbook::recipe' do
let(:chef_run) do
ChefSpec::SoloRunner.new(platform: 'x', version: 'x') {
# ...
}.converge(described_recipe)
end
before :each do
allow_any_instance_of(Chef::RunContext::CookbookCompiler).to receive(:cookbook_order) do
Chef::Log.debug 'Attempt to source external cookbooks blocked'
[described_cookbook]
end
allow_any_instance_of(Chef::Recipe).to receive(:include_recipe) do |recipe|
Chef::Log.debug "Attempt to include #{recipe} blocked"
end
end
it 'works' do
# ...
end
end
You need both of these in your before. The one I had to work for is the intercept of the :cookbook_order method. I had to drill down into the Chef internals to discover this. Keep in mind, this worked for me using Chef 14, but there is no guarantee that this will be future-safe. After upgrading Chef you might have to find another solution, if the implementation of CookbookCompiler ever changes. (The intercept of Chef::Recipe.include_recipe however is a supported API and therefore should be at least somewhat future-safe.)
And, I mention that this comes at a price. (Other than using an unsupported hack!) You will not be able to do any expects for your recipe or attribute includes, except within your own cookbook. A test case like this will fail, because the recipe can't actually be included, as you are preventing that:
it 'includes othercookbook::recipe' do
expect_any_instance_of(Chef::Recipe).to receive(:include_recipe).with('othercookbook::recipe')
end
Also, you must now satisfy in your before blocks all attributes and other preconditions that might otherwise be fulfilled by other recipes in your run list. So you may be signing yourself up for considerable pain by doing this. But, once you have finished, you will have much less brittle tests. (Although to achieve 100% purity regarding external dependencies, you must also surrender fauxhai, which will be even more painful.)
Related
I have an RSpec suite, run via Bundler, that is testing a number of different command-line applications using Aruba. It works fine ... as long as the command being tested is not itself written in Ruby using Bundler. But I cannot figure out how to prevent the RSpec suite's bundler config from interfering with the execution of commands that themselves use Bundler - at least, not without extreme measures.
I have tried various permutations of unset_bundler_env_vars and with_clean_env, to no avail. Here's an example of a technique I thought would work:
describe 'my ruby app' do
before :each { unset_bundler_env_vars }
it 'should work' do
Bundler.with_clean_env { run_simple ruby_command_name }
end
end
I also tried unset_bundler_env_vars without with_clean_env, and vice-versa, in case they interfered with each other. No dice.
The only way I've gotten it to work is to massage Aruba's copy of the environment manually, like this:
before :all do
aruba.environment.tap do |env|
if env.include? 'BUNDLE_ORIG_PATH' then
env['PATH'] = env['BUNDLE_ORIG_PATH']
%w(BUNDLE_BIN_PATH BUNDLE_GEMFILE BUNDLE_ORIG_PATH GEM_HOME RBENV_DIR
RBENV_HOOK_PATH RUBYLIB RUBYOPT).each do |key|
env.delete key
end
end
end
end
There must be a better way. Neither the test suite nor the command being tested should know or care what language the other is written in. And my test code that uses Aruba and Bundler should not need to know the details of how bundle exec affects the process environment.
So what am I doing wrong? How should I be doing this?
It looks like unset_bundler_env_vars is deprecated and replaced by delete_by_environment_variable which requires a string param (source).
You might try before :each { delete_environment_variable('BUNDLE_GEMFILE') } in your spec. If that does not work, you may need to iterate through the PATH variable list to delete each one.
In the deprecation notice, there is a work-around, though I am not sure how brittle that would be moving forward.
unset_bundler_env_vars
aruba.environment.clear.update(ENV)
Hope this helps.
I'm working on a infrastructure where some servers don't have access to the internet, so I have to push the packages to the local repo before declaring them to be installed on Chef.
However we've been on a situation where Chef failed to install a package since the package wasn't there on some boxes and it has been successful on some other boxes.
What I want to do is to run a Ruby/RSpec test before applying Chef config on the nodes to make sure the packages declared on the recipes do actually exist on the repo.
In order to do that I need to be able to list all the packages exists in the our recipes.
My question is: Is there anyway to list all the declared packages in Chef? I had a quick look at Chef::Platform and ChefSpec but unfortunately couldn't find anything useful to my problem.
Do you have any idea where is the best place to look at?
If you use ChefSpec you can find all the packages by calling chef_run.find_resources(:package) inside some test. See the source code. Like this:
require 'chefspec'
describe 'example::default' do
let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) }
it 'does something' do
chef_run.find_resources(:package)...
end
end
You could install one or more of the community ohai plugins. For example the following will return information about installed sofware:
debian
Redhat
windows
Once the plugins are enabled they will add additional node attributes that will be searchable from chef-server.
I have a question for the Ruby and Chef hackers.
I have very limited knowledge of Chef and even less on Ruby programming language, however, I need to implement on Chef (chef-solo) something similar to "augeas" (which works with Puppet, but here I need a solution for Chef).
I got the example code below but it's not working and I am now for a few days trying to figure out what is wrong.
Basically I need to be able to select specific strings in a text file and modify these values. I could use sed but perhaps I can do it in a more elegant way using the ruby_block from Chef.
Please let me know what can be possibly wrong with the code below. Why is my /etc/hosts not being updated with new values?
Always when I re-run chef-solo, I get the following error:
NoMethodError
-------------
undefined method `chef' for Chef::Resource::RubyBlock
Thanks for your help.
Follows my default.rb file:
ruby_block "edit etc hosts" do
block do
rc = Chef::Util::FileEdit.new("/etc/hosts")
rc.search_file_replace_line(
/^127\.0\.0\.1 localhost$/,
"127.0.0.1 #{new_fqdn} #{new_hostname} localhost"
)
rc.write_file
end
end
Add this line as the first line of your ruby block:
require 'chef/util/file_edit'
According to your case, you should use the cookbook hostsfile:
hostsfile_entry '127.0.0.1' do
hostname new_hostname
aliases [new_fqdn]
comment 'Append by Recipe X'
action :append
end
It shouldn't be too hard to get Augeas to work with Chef. Augeas is a C library, and Puppet simply uses its Ruby bindings. You just need to make use of these bindings in Chef.
There is a PoC Augeas resource provider for Chef here: https://github.com/craigtracey/augeas.
Note: http://lists.opscode.com/sympa/arc/chef/2013-02/msg00337.html mentions Augeas integration into Chef, but apparently the participants misunderstand Augeas as they mention idempotency issues and deltas. Most uses of Augeas don't lead to managing deltas, but desired states.
I am trying to use the main gem for making command line utilities. This was presented in a recent Ruby Rogues podcast.
If I put all the code in one file and require that file, then rspec gives me an error, as the main dsl regards rpsec as a command line invocation of the main utility.
I can break out a method into a new file and have rspec require that file. Suppose you have this program, but want to put the do_something method in a separate file to test with rspec:
require 'main'
def do_something(foo)
puts "foo is #{foo}"
end
Main {
argument('foo'){
required # this is the default
cast :int # value cast to Fixnum
validate{|foo| foo == 42} # raises error in failure case
description 'the foo param' # shown in --help
}
do_something(arguments['foo'].value)
}
What is the convenient way to distribute/deploy a ruby command line program with multiple files? Maybe create a gem?
You are on the right track for testing - basically you want your "logic" in separate files so you can unit test them. You can then use something like Aruba to do an integration test.
With multiple files, your best bet is to distribute it as a RubyGem. There's lots of resources out there, but the gist of it is:
Put your executable in bin
Put your files in lib/YOUR_APP/whatever.rb where "YOUR_APP" is the name of your app. I'd also recommend namespacing your classes with modules named for your app
In your executable, require the files in lib as if lib were in the load path
In your gemspec, make sure to indicate what your bin files are and what your lib files are (if you generate it with bundle gem and are using git, you should be good to go)
This way, your app will have access to the files in lib at runtime, when installed with RubyGems. In development, you will need to either do bundle exec bin/my_app or RUBYLIB=lib bin/my_app. Point is, RubyGems takes care of the load path at runtime, but not at development time.
So here's what I'm attempting to do. I'm building an ember.js application, with a java backend running on GAE.
I'm using handlebars, but I want them divided up into separate files, not just all pasted into the index.html.
Via the ember.js irc I was turned on to rake-pipeline along with minispade
Along with the web filters and a custom handlebars filter I started building the assetfile. I don't know Ruby, or gem files, etc.
So I'm trying to figure out the best way to be able to compile my coffeescript/handlebars files on the fly, minispade them, but keep the individual files accessible while in dev mode so I can debug them. What makes that hard is that the rake pipeline is running on a different port than GAE. So I'm not sure exactly how to handle this. Do I make my index file in GAE point to individual files at the 9292 port (rakep) during development, but in production mode point to the fully concatenated version? I'm not sure.
So I was attempting to do that here: https://gist.github.com/1495740 by having only one section that was triggered by the 'build' flag. Not even sure if that works that way.
I know there's a lot of confusion here. Apologies, like I said I'm not even remotely familiar with the Ruby style of doing things.
Since you're not a Ruby person, here are the most reliable steps for getting a stock OSX environment set up with rake pipeline:
Step 1: Install bundler
# on OSX, using built-in Ruby
$ sudo gem install bundler --pre
Step 2: Create a Gemfile
# inside your app directory
$ bundle init
# will create a file named Gemfile in the root
Step 3: Add rake-pipeline to the Gemfile
# inside the Gemfile
gem "rake-pipeline-web-filters"
Step 4: Install your gems
$ bundle install --binstubs
Step 5: Set up Assetfile
However you were already doing it...
Step 6: Run Rake::Pipeline
# to run the preview server
$ bin/rakep
# to build your assets
$ bin/rakep build
Rake::Pipeline.build is the method that evaluates an Assetfile. You can imagine that your entire Assetfile is wrapped inside a Rake::Pipeline.build {} block; you shouldn't ever need to write one inside an Assetfile.
Some of the filters in the docs are hypothetical, most of those docs were written before there were any filters at all. A CoffeeScript compiler has been recently added, though.
As to your main question, I'm not sure there's a clean way to do it with the current rakep implementation. An Assetfile is just Ruby, though, so it's possible to hack something together that should work. Here's how I would write yours:
require "json"
require "rake-pipeline-web-filters"
require "rake-pipeline-web-filters/helpers"
class HandlebarsFilter < Rake::Pipeline::Filter
def initialize(&block)
block ||= proc { |input| input.sub(/\.handlebars$/, '.js') }
super(&block)
end
def generate_output(inputs, output)
inputs.each do |input|
output.write "return Ember.Handlebars.compile(#{input.read.to_json})"
end
end
end
# process all js, css and html files in app/assets
input "assets"
# processed files should be outputted to public
output "public"
# process all coffee files
match "**/*.coffee" do
# compile all CoffeeScript files. the output file
# for the compilation should be the input name
# with the .coffee extension replaced with .js
coffee_script
# The coffee_script helper is exactly equivalent to:
# filter Rake::Pipeline::Web::Filters::CoffeeScriptCompiler
end
match "**/*.js" do
minispade
if ENV['RAKEP_ENV'] == "production"
concat "application.js"
else
concat
end
end
match "**/*.handlebars" do
filter HandlebarsFilter
minispade
concat "templates.js"
end
The if ENV['RAKEP_ENV'] bit reads an environment variable to decide whether to concatenate your JS to a single file.
So now you can run RAKEP_ENV="production" rakep build for a concatenated build, or just rakep build for a development build.