I'm quite new to all of this. I am trying to test out a puppet module using Beaker. I keep getting this:
NoMethodError: undefined method `describe' for
#Beaker::TestCase:0x007fd6f95e6460
/Users/user1/beaker/Puppet/puppet-files/spec/classes/unit_spec.rb:3
/Users/user1/.rvm/gems/ruby-2.2.7/gems/beaker-3.24.0/bin/beaker:9
/Users/user1/.rvm/gems/ruby-2.2.7/bin/ruby_executable_hooks:15
/Users/user1/.rvm/gems/ruby-2.2.7/bin/ruby_executable_hooks:15.
This is the command that I'm running - "beaker --hosts myhost.yaml --pre-suite spec".
My unit_spec.rb contains this:
require 'puppetlabs_spec_helper/rake_tasks'
describe 'application' do
context 'applied to supported operating system' do
on_supported_os.each do |os, facts|
context "#{os}" do
let(:facts) do
facts
end
context "without any parameters" do
let(:params) {{ }}
it { is_expected.to compile.with_all_deps }
it { is_expected.to contain_class('files') }
end
end
end
end
context 'applied to unsupported operating system' do
describe 'ubuntu-14-x86_64' do
let(:facts) {{
:osfamily => 'Debian',
:operatingsystem => 'Ubuntu'
}}
it { is_expected.to raise_error(Puppet::Error, /Ubuntu not supported/) }
end
end
end
Any help would be much appreciated! Btw, I am using 'puppetlabs_spec_helper/rake_tasks' due to the fact that when I just used 'spec_helper' it gave me an error that it could not "load such file" even though it was there.
Also, I have tried doing
RSpec.Describe
That did not fix the issue either. I get the following error -
NameError: undefined local variable or method `on_supported_os' for #Class:0x007f92a61d5e58
I realise that this might be an RSpec Puppet issue, as this module was previously tested through puppet RSpec, however now I am trying to test using Beaker and not quite sure how to fully achieve that!
You're mixing up unit- and VM-based testing.
rspec-puppet (and rspec-puppet-facts' on_supported_os) are for catalog-based unit testing. They do not require a VM, and can give you quick feedback on syntax, and logic of your module.
beaker, and the recommended beaker-rspec add-on, provide full end-to-end testing capabilities, using actual VMs, and testing a complete stack deploy (as defined in you tests).
The main entry point for existing modules is usually rake. Take a look at the existing rake tasks in the module using rake -T. In a well-written module it should have tasks for both rspec-puppet (usually called spec), and beaker (often called beaker, or acceptance).
If it is your own module, you also might want to look into the new Puppet Development Kit to get the most important tools in a single installer.
Related
I'm still getting up to speed on Puppet and rspec and all that, but...
We've currently got a CI runner that tests our Puppet module code using a Docker container running on Linux. Well, we're now delving into using Windows-specific features of Puppet, and our tests are failing. So we're wondering if there's any way to have the tests get ignored if the platform running them is Linux?
For example, if we need to run our unit tests against our code that manages the local groups (https://puppet.com/docs/pe/2018.1/managing_windows_configurations.html#manage_local_groups), is there a way so that when we run it locally on our Windows Dev boxes, it works, but when it runs on our (Linux-based) CI runner, it skips that particular test?
Per request, here is an example of the code we're looking to use to manage a local group:
class my_repo::profile::windows::remote_desktop_users (
Array $members = ['MyDomain\MyUserAccount', 'MyDomain\ARandomDomainGroup'],
) {
group{'Set local Remote Desktop Users memberships':
ensure => present,
name => 'Remote Desktop Users',
members => $members,
auth_membership => false
}
}
Note: We're using the Role-Profile pattern
The above code seems to work. It just bombs out when our unit tests run via our CI:
describe 'my_repo::profile::windows::remote_desktop_users' do
on_supported_os.select { |_, f| f[:os]['family'] == 'windows' }.each do |os, os_facts|
context "on #{os}" do
let(:facts) { os_facts }
it { is_expected.to compile }
end
end
end
Thanks
As I remarked in a comment, I was able to reproduce a test failure using the manifest and Spec code (now) presented in the question. If I understand correctly that the failure is observed only when running the unit tests and not when serving up catalogs for real, then it follows that the problem is in the test environment's configuration, which may or may not be on you.
But as for the actual question:
So we're wondering if there's any way to have the tests get ignored if the platform running them is Linux?
Sure you can. Rspec tests are written in Ruby, and you can use substantially all standard Ruby features in them, including control flow statements and mechanisms for executing system commands. Thus, as a temporary workaround, you can put your breaks-when-not-running-on-Windows tests into a conditional statement, like this:
describe 'my_repo::profile::windows::remote_desktop_users' do
on_supported_os.select { |_, f| f[:os]['family'] == 'windows' }.each do |os, os_facts|
if %x{facter os.family}.chomp == 'windows'
context "on #{os}" do
let(:facts) { os_facts }
it { is_expected.to compile }
end
end
end
end
Note in particular the use of a %x expression to execute a system command on the host where the test runs, and in it use of facter to request the specific system fact that tells you whether the test is running on Windows. That resolves the issue for me. Note that this particular implementation will require facter to be installed on your CI machine.
I have 2 spec files that use different docker images and therefore are suppose to start separate and different docker containers to run the examples.
In the snippets below I'm using the serverspec gem to test my containers
spec/dockerfile/ember_spec.rb
require 'spec_helper'
require 'shared_examples/release'
describe 'ember' do
before(:all) do
#image = Docker::Image.build_from_dir(image_path('ember'))
set :os, family: :alpine
set :backend, :docker
set :docker_image, #image.id
set :docker_container_create_options, { 'Entrypoint' => ['/bin/sh'] }
end
describe command('ember version') do
its(:stdout) { should contain 'ember-cli: 3.3.0' }
its(:stdout) { should contain 'node: 10.10.0' }
end
include_examples 'os release', 'Alpine Linux'
end
spec/dockerfile/gerbv_spec.rb
require 'spec_helper'
require 'shared_examples/release'
describe 'gerbv' do
before(:all) do
#image = Docker::Image.build_from_dir(image_path('gerbv'))
set :os, family: :debian
set :backend, :docker
set :docker_image, #image.id
set :docker_container_create_options, { 'Entrypoint' => ['/bin/sh'] }
end
describe package('gerbv') do
it { should be_installed }
end
include_examples 'os release', 'Ubuntu 18.04'
end
However when running bundle exec rspec it is quite clear that the same container is being used to run each spec file. I have confirmed this by printing out the running containers before each of the examples. This is of course causing the specs to fail for one of the files (whichever runs second).
When the files are run independently using bundle exec rspec path/to/file then all the specs pass.
Is there any way to force a container to be spun down after the examples in one file have run and a new container created for the other set of examples?
I found a way to solve the problem, albeit a pretty hacky one. The key to this problem lay in how the container is finally released. When there are no longer any references pointing to the Docker instance it will be garbage collected and the container killed and deleted. However the object instance is held in a class level variable as a singleton in the base class. It would seem to me the only way to "reset" specinfra is to call the clear method inherited on the Docker class.
In the end the following solved the problem and the correct class is being used to run each spec.
after(:all) {
Specinfra.backend.class.clear
}
It would be great to know there is a better way to access this methods without having to rely on a method not exposed through the serverspec gem.
I have an RSpec suite, run via Bundler, that is testing a number of different command-line applications using Aruba. It works fine ... as long as the command being tested is not itself written in Ruby using Bundler. But I cannot figure out how to prevent the RSpec suite's bundler config from interfering with the execution of commands that themselves use Bundler - at least, not without extreme measures.
I have tried various permutations of unset_bundler_env_vars and with_clean_env, to no avail. Here's an example of a technique I thought would work:
describe 'my ruby app' do
before :each { unset_bundler_env_vars }
it 'should work' do
Bundler.with_clean_env { run_simple ruby_command_name }
end
end
I also tried unset_bundler_env_vars without with_clean_env, and vice-versa, in case they interfered with each other. No dice.
The only way I've gotten it to work is to massage Aruba's copy of the environment manually, like this:
before :all do
aruba.environment.tap do |env|
if env.include? 'BUNDLE_ORIG_PATH' then
env['PATH'] = env['BUNDLE_ORIG_PATH']
%w(BUNDLE_BIN_PATH BUNDLE_GEMFILE BUNDLE_ORIG_PATH GEM_HOME RBENV_DIR
RBENV_HOOK_PATH RUBYLIB RUBYOPT).each do |key|
env.delete key
end
end
end
end
There must be a better way. Neither the test suite nor the command being tested should know or care what language the other is written in. And my test code that uses Aruba and Bundler should not need to know the details of how bundle exec affects the process environment.
So what am I doing wrong? How should I be doing this?
It looks like unset_bundler_env_vars is deprecated and replaced by delete_by_environment_variable which requires a string param (source).
You might try before :each { delete_environment_variable('BUNDLE_GEMFILE') } in your spec. If that does not work, you may need to iterate through the PATH variable list to delete each one.
In the deprecation notice, there is a work-around, though I am not sure how brittle that would be moving forward.
unset_bundler_env_vars
aruba.environment.clear.update(ENV)
Hope this helps.
I have built a cookbook for installing Jenkins CI. It uses the key and repository resources from the yum cookbook, so I end up with the following recipe:
yum_key "RPM-GPG-KEY-jenkins" do
url "http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key"
action :add
end
yum_repository "jenkins" do
description "Jenkins-CI 3rd party repository"
url "http://pkg.jenkins-ci.org/redhat"
key "RPM-GPG-KEY-jenkins"
action :add
end
When I include this recipe in another recipe:
include_recipe 'sp_jenkins::default'
and I test this with the following ChefSpec test
it 'includes the `sp_jenkins::default` recipe' do
expect(chef_run).to include_recipe('sp_jenkins::install')
end
my ChefSpec test fails with the following output:
NameError:
Cannot find a resource for yum_key on chefspec version 0.6.1
(I'm not sure why it says version 0.6.1, gem list tells me it's using 3.0.2)
The sp_jenkins cookbook does depend on the yum cookbook (metadata.rb), and runs fine, however, the cookbook I'm currently writing does not depend on the yum cookbook and therefore doesn't have the yum_key and yum_repository methods available.
Is there a way to prevent ChefSpec from 'descending' into included recipes/cookbooks and just test the current cookbook?
Ohai! Julian is correct - ChefSpec actually does a Chef Solo run in memory on your local machine. It rewrites the provider actions to be a noop, but creates a registry of all the actions taken (including those that would be taken if notifications were executed).
So just like you need the yum cookbook to converge this recipe on a real node, you need it to converge during your unit tests with ChefSpec. The easiest way to accomplish this is by using the Berkshelf or Librarian resolvers. To use the Berkshelf resolver, simply require 'chefspec/berkshelf' after requiring chefspec:
# spec_helper.rb
require 'chefspec'
require 'chefspec/berkshelf'
If you have Berkshelf installed on your system, it will pull all the cookbooks into a temporary directory and run ChefSpec for you.
You may also want to take a look at Strainer, which aims to solve a similar problem.
On a somewhat unrelated note, I am working on a fairly large refactor to the Jenkins cookbook that may better suit your needs.
Sources:
I wrote it...
No, there's no way to prevent it from descending, because it's trying to converge an entire Chef run in memory.
However, if you use the Berkshelf functionality in ChefSpec, the Berkshelf dependency resolver will feed all dependent cookbooks to the in-memory Chef run, and you'll be golden.
It is absolutely valid to expect to test your cookbook in isolation, and not include other projects' code into the scope of your tests. Unfortunately there appears to be no supported, "clean" way to do this, that I can find. I was able to achieve this, but it comes at a price.
To use this technique, do not require 'chefspec/berkshelf' anywhere in your test code, only chefspec itself, as you are intentionally not gathering other cookbook source. Here is a template of my working test module (not my complete test code, as I have omitted RSpec config options):
describe 'mycookbook::recipe' do
let(:chef_run) do
ChefSpec::SoloRunner.new(platform: 'x', version: 'x') {
# ...
}.converge(described_recipe)
end
before :each do
allow_any_instance_of(Chef::RunContext::CookbookCompiler).to receive(:cookbook_order) do
Chef::Log.debug 'Attempt to source external cookbooks blocked'
[described_cookbook]
end
allow_any_instance_of(Chef::Recipe).to receive(:include_recipe) do |recipe|
Chef::Log.debug "Attempt to include #{recipe} blocked"
end
end
it 'works' do
# ...
end
end
You need both of these in your before. The one I had to work for is the intercept of the :cookbook_order method. I had to drill down into the Chef internals to discover this. Keep in mind, this worked for me using Chef 14, but there is no guarantee that this will be future-safe. After upgrading Chef you might have to find another solution, if the implementation of CookbookCompiler ever changes. (The intercept of Chef::Recipe.include_recipe however is a supported API and therefore should be at least somewhat future-safe.)
And, I mention that this comes at a price. (Other than using an unsupported hack!) You will not be able to do any expects for your recipe or attribute includes, except within your own cookbook. A test case like this will fail, because the recipe can't actually be included, as you are preventing that:
it 'includes othercookbook::recipe' do
expect_any_instance_of(Chef::Recipe).to receive(:include_recipe).with('othercookbook::recipe')
end
Also, you must now satisfy in your before blocks all attributes and other preconditions that might otherwise be fulfilled by other recipes in your run list. So you may be signing yourself up for considerable pain by doing this. But, once you have finished, you will have much less brittle tests. (Although to achieve 100% purity regarding external dependencies, you must also surrender fauxhai, which will be even more painful.)
I'm new to Ruby, and I've been trying to learn Rake, RSpec, and Cucumber. I found some code that will help me test my Rake tasks, but I'm having trouble getting it to work. I was told here: http://blog.codahale.com/2007/12/20/rake-vs-rspec-fight/ to drop this:
def describe_rake_task(task_name, filename, &block)
require "rake"
describe "Rake task #{task_name}" do
attr_reader :task
before(:all) do
#rake = Rake::Application.new
Rake.application = #rake
load filename
#task = Rake::Task[task_name]
end
after(:all) do
Rake.application = nil
end
def invoke!
for action in task.instance_eval { #actions }
instance_eval(&action)
end
end
instance_eval(&block)
end
end
into my spec_helper.rb file.
I've managed to take this code out and run it in my cucumber steps like this:
When /^I run the update_installers task$/ do
#rake = Rake::Application.new
Rake.application = #rake
load "lib/tasks/rakefile.rb"
#task = Rake::Task["update_installers"]
for action in #task.instance_eval { #actions }
instance_eval(&action)
end
instance_eval(&block)
Rake.application = nil
end
but when I try to get things working in rspec, I get the following error.
ArgumentError in 'Rake task
install_grapevine should install to
the mygrapevine directory'
wrong number of arguments (1 for 2)
/spec/spec_helper.rb: 21:in instance_eval'
/spec/spec_helper.rb: 21:inblock in invoke!'
/spec/spec_helper.rb: 20:in each'
/spec/spec_helper.rb: 20:ininvoke!'
/spec/tasks/rakefile_spec.rb:12:in `block (2 levels) in
'
Unfortunately, I've got just under a week of ruby under by belt, so the metaprogramming stuff is over my head. Could anyone point me in the right direction?
This works for me: (Rails3/ Ruby 1.9.2)
When /^the system does it's automated tasks$/ do
require "rake"
#rake = Rake::Application.new
Rake.application = #rake
Rake.application.rake_require "tasks/cron"
Rake::Task.define_task(:environment)
#rake['cron'].invoke
end
Substitute your rake task name here and also note that your require may be "lib/tasks/cron" if you don't have the lib folder in your load path.
I agree that you should only do minimal work in the Rake task and push the rest to models for ease of testing. That being said I think it's important to ensure that the code is ACTUALLY run in my cron tasks during my integration tests so I think very mild testing of the rake tasks is justified.
Since testing rake is just too much for me, I tend to move this problem around. Whenever I find myself with a long rake task that I want to test, I create a module/class in lib/ and move all the code from the task there. This leaves the task to a single line of Ruby code, that delegates to something more testable (class, module, you name it). The only thing that remains untested is whether the rake task invokes the right line of code (and passes the right parameters), but I think that is OK.
It might be useful to tell us which is the 21nd line of your spec_helper.rb. But given that the approach you posted digs deep in rake (referring to its instance variables), I would entirely abandon it for what I suggested in the previous paragraph.
I've just spent a little while getting cucumber to run a rake task so I thought I'd share my approach. Note: This is using Ruby 2.0.0 and Rake 10.0.4, but I don't think the behaviour has changed since previous versions.
There are two parts to this. The first is easy: with a properly set up instance of Rake::Application then we can access tasks on it by calling #[] (eg rake['data:import']). Once we have a task we can run it by calling #invoke and passing in the arguments (eg rake['data:import'].invoke('path/to/my/file.csv').
The second part is more awkward: properly setting up an instance of Rake::Application to work with. Once we've done require 'rake' we have access to the Rake module. It already has an application instance, available from Rake.application, but it's not yet set up — it doesn't know about any of our rake tasks. It does, however, know where to find our Rakefile, assuming we've used one of the standard file names: rakefile, Rakefile, rakefile.rb or Rakefile.rb.
To load the rakefile we just need to call #load_rakefile on the application, but before we can do that we need to call #handle_options. The call to #handle_options populates options.rakelib with a default value. If options.rakelib is not set then the #load_rakefile method will blow up, as it expects options.rakelib to be enumerable.
Here's the helper I've ended up with:
module RakeHelper
def run_rake_task(task_name, *args)
rake_application[task_name].invoke(*args)
end
def rake_application
require 'rake'
#rake_application ||= Rake.application.tap do |app|
app.handle_options
app.load_rakefile
end
end
end
World(RakeHelper)
Pop that code into a file in features/support/ and then just use run_rake_task in your steps, eg:
When /^I import data from a CSV$/ do
run_rake_task 'data:import', 'path/to/my/file.csv'
end
The behavior might have changed since the correct answer was posted. I was experiencing problems executing two scenarios that needed to run the same rake task (only one was being executed despite me using .execute instead of .invoke). I thought to share my approach to solve the issue (Rails 4.2.5 and Ruby 2.3.0).
I tagged all the scenarios that require rake with #rake and I defined a hook to setup rake only once.
# hooks.rb
Before('#rake') do |scenario|
unless $rake
require 'rake'
Rake.application.rake_require "tasks/daily_digest"
# and require other tasks
Rake::Task.define_task(:environment)
$rake = Rake::Task
end
end
(Using a global variable is suggested here: https://github.com/cucumber/cucumber/wiki/Hooks#running-a-before-hook-only-once)
In the step definition I simply called $rake
# step definition
Then(/^the daily digest task is run$/) do
$rake['collector:daily_digest'].execute
end
Any feedback is welcome.