I have a problem during testing my chef cookbooks using Kitchen.
I use Vagrant as a driver and Virtual Box as virtualization system. They running on Linux platform.
I have create a test recipe to reboot my machine.
This is the code:
#
# Author:: Alessandro Zucchelli
# Recipe:: test_recipe_required_reboot
#
reboot 'test if kitchen runs when a recipe leaves it to the post-install phase' do
action :request_reboot
reason 'Need to reboot when the run completes successfully.'
delay_mins 1
end
And I configure my .kitchen.yml in this mode:
driver:
name: vagrant
gui: true
boot_timeout: 1200
# log_level: debug
provisioner:
name: chef_zero
require_chef_omnibus: 12.11 # need the RFC 062 exit codes
retry_on_exit_code:
- 35 # 35 is the exit code signaling that the node is rebooting
max_retries: 3
multiple_converge: 3
client_rb:
exit_status: :enabled # Opt-in to the standardized exit codes
client_fork: false # Forked instances don't return the real exit code
#log_level: debug
platforms:
- name: mybox/win-7-professional-amd64-nocm
transport:
name: winrm
elevated: true
connection_timeout: 3600
max_wait_until_ready: 600
connection_retry_sleep: 300
I expect that by launching the recipe go looped restarting vm more than once.
Instead, after the first reboot I get:
Starting Chef Client, version 12.11.18
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>> Failed to complete #converge action: [execution expired] on ktest-mybox-win-7-professional-amd64-nocm
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration
I solved my problem!
I added this parameters (view code block) in kitchen.yml and now works correctly!
max_retries: 3
multiple_converge: 3
wait_for_retry: 600
My recipe now reboot 4 times my VMs!
Related
Initially I started looking at Configure New Relic Infrastructure using Chef to setup newrelic infra chef cookbook for my project using chef solo, after some research I found that the dependencies in the cookbook is no longer supported.
So I decided to write a custom resource in the recipe so I can utilize chef idempotency.
I have tried the following steps under Install for Ubuntu in my ubuntu box and verified new relic-infra installation:
Now I am trying write a chef resource like this:
Step1: Create a configuration file, and add your license key:
echo "license_key: YOUR_LICENSE_KEY" | sudo tee -a /etc/newrelic-infra.yml
Added this resource block in my recipe for Step1:
file '/etc/newrelic-infra.yml' do
content 'license_key: added_key_here'
mode '0755'
owner 'root'
group 'root'
end
Step2: Enable New Relic's GPG key:
curl https://download.newrelic.com/infrastructure_agent/gpg/newrelic-infra.gpg | sudo apt-key add -
Added this resource block in my recipe for Step2:
apt_repository 'newrelic_key' do
uri 'https://download.newrelic.com/infrastructure_agent/gpg'
trusted true
key 'https://download.newrelic.com/infrastructure_agent/gpg/newrelic-infra.gpg'
end
I verified this step in the local box by listing the keys using this command:
sudo apt-key list
Step3: Create the agent's apt repo using the command for your distribution version:
printf "deb [arch=amd64] https://download.newrelic.com/infrastructure_agent/linux/apt bionic main" | sudo tee -a /etc/apt/sources.list.d/newrelic-infra.list
Added this resource block in my recipe for Step3:
file '/etc/apt/sources.list.d/newrelic-infra.list' do
content 'deb [arch=amd64] https://download.newrelic.com/infrastructure_agent/linux/apt bionic main'
mode '0755'
owner 'root'
group 'root'
end
Step4: Update your apt cache and Run the install script:
sudo apt-get update
sudo apt-get install newrelic-infra -y
Added this resource block in my recipe for Step4:
apt_update
apt_package 'newrelic-infra'
Error:
But the install fails with the following error:
===============================================================================
default: Error executing action `update` on resource 'apt_update[newrelic-infra]'
default: ================================================================================
default:
default: Mixlib::ShellOut::ShellCommandFailed
default: ------------------------------------
default: execute[apt-get -q update] (/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.4.56/lib/chef/provider/apt_update.rb line 70) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '100'
default: ---- Begin output of ["apt-get", "-q", "update"] ----
default: STDOUT:
default:
default: STDERR: E: Malformed entry 1 in list file /etc/apt/sources.list.d/newrelic-infra.list (Component)
default:
default:
default: * apt_package[newrelic-infra] action install
default: * No candidate version available for newrelic-infra
default:
default:
default: ================================================================================
default:
default: Error executing action `install` on resource 'apt_package[newrelic-infra]'
default:
default: ================================================================================
default:
default:
default:
default:
default: Chef::Exceptions::Package
default: -------------------------
default: No candidate version available for newrelic-infra
default:
default: Resource Declaration:
default: ---------------------
default: # In /etc/chef/local-mode-cache/cache/cookbooks/repo/recipes/default.rb
default:
default: 38: apt_package 'newrelic-infra'
default: 39:
default:
default: Compiled Resource:
default: ------------------
default: # Declared in /etc/chef/local-mode-cache/cache/cookbooks/repo/recipes/default.rb:38:in `from_file'
default:
default: apt_package("newrelic-infra") do
default: package_name "newrelic-infra"
default: action [:install]
default: default_guard_interpreter :default
default:
default: declared_type :apt_package
default:
default:
default: cookbook_name "repo"
default: recipe_name "default"
default: end
default:
default: System Info:
default: ------------
default: chef_version=14.4.56
default: platform=ubuntu
default: platform_version=18.04
default: ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
default: program_name=/usr/bin/chef-solo
default: executable=/opt/chefdk/bin/chef-solo
default:
default:
default: Running handlers:
default: [2019-08-30T18:19:30+00:00] ERROR: Running exception handlers
default: Running handlers complete
default: [2019-08-30T18:19:30+00:00] ERROR: Exception handlers complete
default: Chef Client failed. 6 resources updated in 48 seconds
default: [2019-08-30T18:19:30+00:00] FATAL: Stacktrace dumped to /etc/chef/local-mode-cache/cache/chef-stacktrace.out
default: [2019-08-30T18:19:30+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
default: [2019-08-30T18:19:30+00:00] FATAL: Chef::Exceptions::Package: apt_package[newrelic-infra] (repo-deploy::default line 38) had an error: Chef::Exceptions::Package: No candidate version available for newrelic-infra
I ran my vagrant file and it works for every step successfully but fails in the final install step... What am I doing wrong? Any troubleshooting tips would be helpful. Thanks!
firstly, you can combine together steps 2 and 3 using apt_repository by utilizing the arch and distribution.
if you read apt_repository documentation, you can see that you can even drop the apt update in step 4
Adding a new repository will update the APT package cache immediately.
secondly, back to your question...
from looking at your logs, specifically
default: execute[apt-get -q update] (/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.4.56/lib/chef/provider/apt_update.rb line 70) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '100'
it appears that you do not run chef-client on your node, rather you are using chef-dk. make sure that you are running chef-client (or chef infra client nowadays) within the node that you like to converge.
you can run chef-client in higher log level to reveal more about the execution by specifying the log_level
The level of logging to be stored in a log file. Possible levels: auto (default), debug, info, warn, error, or fatal. Default value: warn (when a terminal is available) or info (when a terminal is not available).
i hope it will help you solving your issue
Thanks again #Mr for providing your input. If anyone is wondering about the custom resource for new relic infra, here you go:
file '/etc/newrelic-infra.yml' do
content 'license_key: added my license key here'
mode '0755'
owner 'root'
group 'root'
end
apt_repository 'newrelic-infra' do
uri 'https://download.newrelic.com/infrastructure_agent/linux/apt'
trusted true
arch 'amd64'
components ['main']
distribution 'bionic'
key 'https://download.newrelic.com/infrastructure_agent/gpg/newrelic-infra.gpg'
end
I run kitchen test, cant accept the license, answer yes does nothing
kitchen.yml
provisioner:
name: chef_zero
always_update_cookbooks: true
retry_on_exit_code:
- 35 # 35 is the exit code signaling that the node is rebooting
max_retries: 1
client_rb:
exit_status: :enabled # Opt-in to the standardized exit codes
client_fork: false # Forked instances don't return the real exit code
environment: _default
chef_license: accept
product_name: chef
chef-client: 14
Try to update from accept to accept-no-persist in the provisioner section per the github issue: https://github.com/test-kitchen/test-kitchen/issues/1553
provisioner:
name: chef_zero
always_update_cookbooks: true
log_level: info
chef_license: accept-no-persist
product_name: chef
product_version: 14
note (via tas50 comment on the github issue): Test Kitchen 1.x does not support chef licensing. The plan is to backport the work on the 2.x branch to the 1-stable branch and release a new 1.X release with the support.
I am trying to learn how to do local development of chef recipes. I am following this guide https://gist.github.com/smford22/f00f46471047422bd8a7
I am prefixing all the kitchen commands with chef exec because if I try to run kitchen directly, I get all sorts of ruby/gem errors.
When I run chef exec kitchen converge it gets stuck on installing the Chef Omnibus, hanging on "Trying wget..."
If I login to the VM and try to run curl and wget commands like curl https://google.com it indeed cannot access the internet.
chef exec kitchen -v
Test Kitchen version 1.23.2
chef -v
Chef Development Kit Version: 3.5.13
chef-client version: 14.7.17
delivery version: master (6862f27aba89109a9630f0b6c6798efec56b4efe)
berks version: 7.0.6
kitchen version: 1.23.2
inspec version: 3.0.52
.kitchen.yml:
---
driver:
name: vagrant
## The private_network feature lets you setup a private network on the VM guest
## via localhost on the host.
## see also: https://www.vagrantup.com/docs/networking/private_network.html
# network:
# - ["private_network", {ip: "33.33.33.33"}]
provisioner:
name: chef_zero
## The verifier section determines which test platform you want to use.
verifier:
name: inspec
format: doc
platforms:
- name: centos-6.7
suites:
- name: default
run_list:
- recipe[chef_httpd::default]
attributes:
Are you accessing the internet through proxy?
If yes, you need to configure the same for your Vagrant VMs using vagrant-proxyconf plugin.
Documentation: http://tmatilai.github.io/vagrant-proxyconf/
I am trying to run kitchen-converge/test using Vagrant (1.8.1) and Virtualbox (5.0), behind a proxy server (which has been configured using charles proxy).
I am either getting hung up while running kitchen test or getting connection refused while running kitchen converge.
I have posted my config and log files including the errors. I was using berkshelf,test-kitchen along with proxyconf as there is proxy server.
Could any one advise on what could be missing and causing this error to occur and how it can be resolved.
Config & Log Files:
kitchen.yml
driver:
name: vagrant
customize:
natdnshostresolver1: "on"
network:
- ["forwarded_port", { guest: 81, host: 8082 }]
- ["forwarded_port", { guest: 80, host: 8087 }]
provisioner:
chef_omnibus_url: http://www.chef.io/chef/install.sh
name: chef_zero
platforms:
- name: centos-6.7
suites:
- name: default
run_list:
- recipe[chef-boxtest::default]
attributes:
Vagrant File
if Vagrant.has_plugin?("vagrant-proxyconf")
config.proxy.http = "http://proxy.abc.com:8099"
config.proxy.https = "https://proxy.abc.com:8099"
config.proxy.no_proxy = "localhost,127.0.0.1"
end
kitchen-test
[SSH] Established
Vagrant instance <default-centos-67> created.
Finished creating <default-centos-67> (3m39.40s).
-----> Converging <default-centos-67>...
Preparing files for transfer
Preparing dna.json
Resolving cookbook dependencies with Berkshelf 4.3.1...
Removing non-cookbook files before transfer
Preparing validation.pem
Preparing client.rb
-----> Installing Chef Omnibus (install only if missing)
Downloading http://www.chef.io/chef/install.sh to file /tmp/install.sh
Trying wget...
kitchen-converge
[SSH] connection failed, retrying in 1 seconds (#<Errno::ECONNREFUSED: Connection refused - connect(2) for 127.0.0.1:2222>)
[SSH] connection failed, retrying in 1 seconds (#<Errno::ECONNREFUSED: Connection refused - connect(2) for 127.0.0.1:2222>)
$$$$$$ [SSH] connection failed, terminating (#<Errno::ECONNREFUSED: Connection refused - connect(2) for 127.0.0.1:2200>)
>>>>>> Converge failed on instance <default-ubuntu-1204>.
I have never used a Vagrantfile with kitchen, as the kitchen-vagrant driver generates the Vagrant file on the fly.
I would suggest that you remove your Vagrantfile and modify your .kitchen.yml file to include the proxy data you need.
I am using vagrant and chef to build a VM.
Host: OSX 10.9.5
audrey:ubuntu2 jluc$ chef -v
Chef Development Kit Version: 0.10.0
chef-client version: 12.5.1
berks version: 4.0.1
kitchen version: 1.4.2
audrey:ubuntu2 jluc$ vagrant -v
Vagrant 1.8.1
audrey:ubuntu2 jluc$ vagrant plugin list
vagrant-berkshelf (4.1.0)
vagrant-omnibus (1.4.1)
vagrant-share (1.1.5, system)
Virtualbox: 5, but same on 4 previously.
Guest: Ubuntu 14.04
Background: I am using alien to install Oracle rpm. However, you can't just point the client to a url or package somewhere, you need to download the rpms from Oracle, accept conditions and then make the files available to your client install. There are 2 rpms, one 30MB, one 640KB.
I am doing this by launching a http server on the host, on port 9003.
Problem:
The download/remote_file time, within the chef run, is anywhere from 5-20 minutes, while I can wget/curl them from within the guest in 5-15 seconds.
Details:
The following chef code deals with those files:
ip = node[:network][:default_gateway]
simplehttp_port=node[:basedjango][:port_host_fileserver]
fn_basic = "oracle-instantclient12.1-basiclite-12.1.0.2.0-1.x86_64.rpm"
execute 'download_basic' do
cwd "/tmp"
command "wget http://#{ip}:#{simplehttp_port}/#{fn_basic} "
not_if do ::File.exists?(fnp_basic) end
end
#I have also used this instead, just as slow...
remote_file fnp_basic do
uri = "http://#{ip}:#{simplehttp_port}/#{fn_basic}"
not_if do ::File.exists?(fnp_basic) end
mode '0755'
action :create
end
My problem is that I can vagrant ssh into the guest and wget or curl the file in anything from 5 to 15 seconds. From the chef cookbooks, I am looking at 5-20 minutes each time (there is a condition guard to avoid downloading it if it exists).
==> default: [2016-01-18T11:13:58-08:00] INFO: file[/etc/profile.d/ORACLE_HOME.sh] mode changed to 755
==> default:
==> default: - change mode from '' to '0755'
==> default:
==> default: - change owner from '' to 'root'
==> default:
==> default: - change group from '' to 'root'
==> default:
==> default: (up to date)
==> default: Recipe: basedjango::oracle
==> default:
==> default: * execute[download_basic] action run
!!! 11:13 to 11:28 => 15 minutes here.
==> default: [2016-01-18T11:28:04-08:00] INFO:
execute[download_basic] ran successfully
==> default:
==> default: - execute wget http://10.0.2.2:9003/oracle-instantclient12.1-basiclite-12.1.0.2.0-1.x86_64.rpm
==> default: * execute[install_basic] action run
And here is the same command, executed manually via vagrant ssh. About 2 seconds in this case.
vagrant#vagrant:/tmp$ wget http://10.0.2.2:9003/oracle-instantclient12.1-basiclite-12.1.0.2.0-1.x86_64.rpm
--2016-01-18 11:50:40-- http://10.0.2.2:9003/oracle-instantclient12.1-basiclite-12.1.0.2.0-1.x86_64.rpm
Connecting to 10.0.2.2:9003... connected.
HTTP request sent, awaiting response... 200 OK
Length: 30940809 (30M) [text/html]
Saving to: ‘oracle-instantclient12.1-basiclite-12.1.0.2.0-1.x86_64.rpm’
100%[====================================================================================================================================================================================>] 30,940,809 17.8MB/s in 1.7s
2016-01-18 11:50:41 (17.8 MB/s) - ‘oracle-instantclient12.1-basiclite-12.1.0.2.0-1.x86_64.rpm’ saved [30940809/30940809]
The remote_file resource behaves the same way, so at first I thought it was something wrong with it or the way I use it. The rpms are NOT being served from the Vagrant shared directory, btw.
Any ideas?
I don't know if you haven't already found a solution, but when I had that issue (even downloading from another server) I could 'fix' it by enabling debug output for chef. This magically increased the download speed.
Simply put
config.vm.provion :chef_solo |chef|
chef.log_level = 'debug'
[...]
end