Chef - create template with dynamic variable? - ruby

I'm having a bit of a challenge on a Chef recipe. I'm new to Chef, so please bear with me.
Step 1: My chef recipe installs Ruby Passenger, then compiles the Passenger nginx module along with Nginx.
# Install passenger and nginx module
bash "Install Passenger" do
code <<-EOF
source /usr/local/rvm/scripts/rvm
gem install passenger
EOF
user "root"
not_if { `gem list`.lines.grep(/^passenger \(.*\)/).count > 0 }
end
# Install passenger
# Note that we have to explicitly include the RVM script otherwise it won't setup the environment correctly
bash "Install passenger nginx module and nginx from source" do
code <<-EOF
source /usr/local/rvm/scripts/rvm
passenger-install-nginx-module --auto --prefix=/opt/nginx --auto-download
EOF
user "root"
not_if { File.directory? "/opt/nginx" }
end
Step 2: After that, I create the nginx config file using a template. This configuration requires the location of Passenger, which is dependent on step 1 completing.
template "/opt/nginx/conf/nginx.conf" do
source "nginx.conf.erb"
action :create
variables(
deploy_user: deploy_user,
passenger_root: `bash -c "source /usr/local/rvm/scripts/rvm; passenger-config --root"`.chomp,
passenger_ruby: `bash -c "source /usr/local/rvm/scripts/rvm; which ruby"`.chomp,
passenger: node[:passenger]
)
end
Problem: Chef appears to compile templates at th ebeginning of the run. So what ends up happening is that Step 2 is actually compiled before Step 1 is run. This means that the passenger_root variable is blank. It needs Step 1 to complete before being able to get the passenger_root, then run the template.
I tried wrapping the step 2 code in a ruby_block but that doesn't work: undefined methodtemplate' for Chef::Resource::RubyBlock`.
Not sure what to do here, or what is the best practice for Chef for something like this?
Thanks in advance,
Leonard

A cleaner and recommended way is to use Lazy Attribute Evaluation.
template "/opt/nginx/conf/nginx.conf" do
source "nginx.conf.erb"
action :create
variables lazy {
{
deploy_user: deploy_user,
passenger_root: `bash -c "source /usr/local/rvm/scripts/rvm; passenger-config --root"`.strip,
passenger_ruby: `bash -c "source /usr/local/rvm/scripts/rvm; which ruby"`.strip,
passenger: node[:passenger]
}
}
end
Also, I'd suggest using strip instead of chomp [thanks Draco].

As soon as you wrap your code in ruby_block you cannot use ordinary recipe resource declaration anymore. You have to write pure ruby code:
ruby_block "create /opt/nginx/conf/nginx.conf from template" do
block do
res = Chef::Resource::Template.new "/opt/nginx/conf/nginx.conf", run_context
res.source "nginx.conf.erb"
res.variables(
deploy_user: deploy_user,
passenger_root: `bash -c "source /usr/local/rvm/scripts/rvm; passenger-config --root"`.chomp,
passenger_ruby: `bash -c "source /usr/local/rvm/scripts/rvm; which ruby"`.chomp,
passenger: node[:passenger]
)
res.run_action :create
end
end
PS. And I guess you want to use strip instead of chomp to remove whitespace.

Yeah, Chef is a beast. I think part of the problem is there are a million ways to do the same things but there really is no documentation detailing the best way. What you probably want is to use Notifications, so that the block 1 runs first than notifies the block 2 to run. This means block 2 needs action :none so that it does not trigger until it gets notified.
I added the notify to your example in block 1 and added the action :none to block 2.
bash "Install Passenger" do
code <<-EOF
source /usr/local/rvm/scripts/rvm
gem install passenger
EOF
user "root"
not_if { `gem list`.lines.grep(/^passenger \(.*\)/).count > 0 }
notifies :run, 'bash[Install passenger nginx module and nginx from source]', :immediately
end
bash "Install passenger nginx module and nginx from source" do
code <<-EOF
source /usr/local/rvm/scripts/rvm
passenger-install-nginx-module --auto --prefix=/opt/nginx --auto-download
EOF
user "root"
action :none
end

Related

Chef use cookbook_file in ruby block

I have the following code to figure out where Java is located on the box. Java comes with our application and what Java version that is included with the application differs.
def app_java_home
if Dir.exist?("#{app_home}/jre-server/linux")
Dir.chdir("#{app_home}/jre-server/linux") do
Dir.glob('jdk*').select { |f| File.directory? f }[0]
end
end
end
Then, in my cookbook I have
aws_s3_file "#{app_download_path}/#{app_s3['archive_file']}" do
bucket app_s3['bucket']
remote_path app_s3['remote_path']
region aws_region
not_if { ::Dir.exists?(app_bin_dir) }
not_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
end
execute 'extract' do
user 'root'
command "unzip #{app_download_path}/#{app_s3['archive_file']} > /dev/null"
not_if { ::Dir.exists?("#{app_home}/ourapp") }
only_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
end
execute 'move' do
user 'root'
command "mv #{app_download_path}/ourapp/ #{app_install_path}"
not_if { ::Dir.exists?(app_home) }
end
cookbook_file "#{app_java_home}/jre/lib/security/local_policy.jar" do
source %W[#{app_release}/local_policy.jar default/local_policy.jar]
owner app_user_name
group app_group_name
mode 0755
end
cookbook_file "#{app_java_home}/jre/lib/security/US_export_policy.jar" do
source %W[#{app_release}/US_export_policy.jar default/US_export_policy.jar]
owner app_user_name
group app_group_name
mode 0755
end
However, the two cookbook_file resources fails because it can't find the directory:
No such file or directory # dir_chdir - /ourapp/jre-server/linux/
After a lot of googling, I've come to the conclusion that it's a .. "missmatch" (?) between compile time and run time of the recipes. Basically, if I understand it correctly, it tries to run the cookbook_file resource(s) first but fails. So never downloads, unpacks and installs the app artefact.
I've tried running app_java_home when the directory exists, and it does seem to work the way I want it..
I tried putting the cookbook_file resources in a ruby_block, but then I instead get:
undefined method `cookbook_file' for Chef::Resource::RubyBlock
The app_java_home .. function (?) used to look like this:
def app_java_home
"#{app_home}/jre-server/linux/#{jdk_version}"
end
Where jdk_version came from the databag. This worked fine, but we have a long standing bug/feature request in our system where it sometimes happens that "they" get the version they put in the databag wrong, causing all sorts of problems.. So they want a way to remove this dependency and instead "figure this out" dynamically.
Ruby and Chef isn't my forte, so I'm not sure what to try next. I have found references to Chef::Resources::CookbookFile (which, if I understand it, could/should be used inside ruby_blocks), but can't find any examples or documentation about it. The link on RubyDocs is broken.
Adding an answer here for a better explanation.
Any (Ruby) code that is not within any of the Chef resources, will run in Compile phase
All resource declarations will run in Convergence phase in the order they are defined
Thankfully, there is a way to make resources run in Compile phase if so required. Though IMHO it should be done sparingly and in exceptional cases.
As per your comment aws_s3_file and execute resources are the ones that unpack the app (and create the directory). In this case, it seems you want them to run in compile phase.
Prior to Chef client 16.0
Use the run_action option with the action that should be performed at the compile time. For example execute resource takes action :run:
# Note action ":nothing" and "run_action"
execute 'extract' do
user 'root'
command "unzip #{app_download_path}/#{app_s3['archive_file']} > /dev/null"
not_if { ::Dir.exists?("#{app_home}/ourapp") }
only_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
action :nothing
end.run_action(:run)
Chef client 16.0 onwards
We can add a common property to the resources. Example with execute resource:
# Note the extra property "compile_time"
execute 'extract' do
user 'root'
command "unzip #{app_download_path}/#{app_s3['archive_file']} > /dev/null"
not_if { ::Dir.exists?("#{app_home}/ourapp") }
only_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
compile_time true
end
And finally to answer the subject of the question:
Chef use cookbook_file in ruby block
This is not possible. Refer to the first point on the top. If we want Ruby code to run during converge (instead of compile), we put it within the ruby_block resource. So it can contain code like (for example):
ruby_block 'get directory' do
block do
def app_java_home
"#{app_home}/jre-server/linux/#{jdk_version}"
end
end
end
With the help of #seshadri_c, I finally managed to solve the problem! It took some doing, because I kept misunderstanding the suggestions etc.
So this is what I came up with (for posterity):
def jdk_version(required = true)
base_dir = "#{app_home}/jre-server/linux"
if Dir.exist?("#{base_dir}")
Dir.chdir("#{app_home}/jre-server/linux") do
Dir.glob("jdk*").each do |f|
if File.directory?(f)
return "#{f}"
end
end
end
end
end
def app_java_home
return "#{app_home}/jre-server/linux/#{jdk_version}"
end
Turns out I need to get just the version, individually, as well, so I rearranged the functions a bit. I'm sure it could be written much cleaner, but here the trick was to use return instead of puts/print! Well, I'm a programmer, but not a Ruby programmer so didn't know that was an option..
Then, in the cookbook, I added the .run_action() where needed. I didn't need them for the cookbook_file, which simplified things a bit:
aws_s3_file "#{app_download_path}/#{app_s3['archive_file']}" do
bucket app_s3['bucket']
remote_path app_s3['remote_path']
region aws_region
not_if { ::Dir.exists?(app_bin_dir) }
not_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
end.run_action(:create)
execute 'extract' do
user 'root'
command "unzip #{app_download_path}/#{app_s3['archive_file']} > /dev/null"
not_if { ::Dir.exists?("#{app_home}/app") }
only_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
end.run_action(:run)
execute 'move' do
user 'root'
command "mv #{app_download_path}/app/ #{app_install_path}"
not_if { ::Dir.exists?(app_home) }
end.run_action(:run)
# JCE Unlimited Strength Jurisdiction Policy Files
cookbook_file "#{app_java_home}/jre/lib/security/local_policy.jar" do
source %W[#{app_release}/local_policy.jar default/local_policy.jar]
owner app_user_name
group app_group_name
mode 0755
end
cookbook_file "#{app_java_home}/jre/lib/security/US_export_policy.jar" do
source %W[#{app_release}/US_export_policy.jar default/US_export_policy.jar]
owner app_user_name
group app_group_name
mode 0755
end
With all that, everything is running exactly when they're supposed to and everything seems to be working.

Aruba: Command "seedly-calculator" not found in PATH-variable

So, I am trying to run the test but I am getting an error says.
Aruba::LaunchError:Command "seedly-calculator.rb" not found in PATH-variable
-seedly-calculator
-bin
-src
-seedly-calculator.rb
I have tried to change the path in rake file but it doesn't work.
My seedly-calculator.rb file is in the root directory.
require "rspec/core/rake_task"
namespace :spec do
desc "Run the functional suite against the CLI"
RSpec::Core::RakeTask.new(:functional, [] => [:set_path])
task :set_path do
project_bin_dir = File.join(File.dirname(File.expand_path(__FILE__)), '..', 'bin')
ENV['PATH'] = project_bin_dir + ':'+ ENV['PATH']
end
end
it shows error like:
Failure/Error: let(:command) { run "seedly-calculator.rb" }
Aruba::LaunchError:
Command "seedly-calculator.rb" not found in PATH-variable "/Users/bilaltariq/Desktop/seedly-calculator/functional_spec/bin:/Users/bilaltariq/Desktop/seedly-calculator/functional_spec/exe:/Users/bilaltariq/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/bin:/Users/bilaltariq/Desktop/seedly-calculator/functional_spec/../bin:/Users/bilaltariq/.rbenv/versions/2.6.2/bin:/usr/local/Cellar/rbenv/1.1.1/libexec:/Users/bilaltariq/.rbenv/shims:/Users/bilaltariq/.asdf/shims:/Users/bilaltariq/.asdf/bin:/usr/local/bin:/Users/bilaltariq/.bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin".
I expect it to hit the file so i can write some test.
am i doing something wrong?
require 'spec_helper'
RSpec.describe 'Command Validation', type: :aruba do
let(:command) { run "seedly-calculator.rb" }
it "wrong/missing arguments" do
command.write("lookup\n")
stop_all_commands
expect(command.output).to end_with("Missing bank_name argument.\n")
end
end
seedly-calculator.rb:
#!/usr/bin/env ruby
# Complete bin/setup so that after it is
# run, ruby seedly-calculator.rb can be used to launch
# it.
# frozen_string_literal: true
require_relative './src/runner'
if !ARGV.length.zero?
input = ARGV
Runner.new.send('process_input', input)
else
puts "Arguments required!."
end
Update
To run a ruby script using run you need to make sure your ruby script is executable and contains a shebang so your system knows to run it with ruby. Here's example from this starter example
#!/usr/bin/env ruby
file = ARGV[0]
if file.nil? || file.empty?
abort "aruba-test-cli [file]: Filename is missing"
elsif !File.exist? file
abort "aruba-test-cli [file]: File does not exist"
end
puts File.read(file).chomp
So in your case you'll need to add this to the first line of your seedly-calculator.rb file
#!/usr/bin/env ruby
Then run this from command line to make it executable.
chmod +x #!/usr/bin/env ruby
I made a simple example forked off the one I reffed above. See this commit
Rspec convention is that it should match the same file structure of your project. It is not a good idea to set PATH manually.
Rake tasks are normally put in a tasks folder so you should have in project root a tasks folder
my_project/tasks/something.rake
Then you should have a spec folder that matches
my_project/spec/tasks/something_spec.rb
Then you should be able to get rid of task :set_path do end block and just run the spec without that.
You should also have a Gemfile to load your gems, run bundle install then invoke your test with
bundle exec rspec spec/tasks/sometask_spec.rb

How can I loop through bash routine in Chef?

I have a bash script in Chef that fetches the time through NTP protocol from 3 instances running NTP server. The code at present is
if not node.run_list.roles.include?("ntp_server")
bash "ntpdate" do
code <<-EOH
/usr/sbin/ntpdate -s 10.204.255.15 10.204.251.41 10.204.251.21
EOH
end
end
This has been working just fine. However, I am supposed to automate the task such as if one of the instances is replaced, there is no manual intervention required to update the IP in the code above.
To achieve that, I have been successfully able to fetch the instances running the ntp_server role.
ntp_servers = search(:node, 'role:ntp_server')
Having done that, I am unable to add those IP's to the bash subroutine in Chef shown above in the code.
Can someone let me know how am I supposed to achieve that?
You shouldn't use bash block and call ntpdate with each chef run. ntpd should take care of clock being in sync, Chef has cookbook for this.
You could move IP addresses to the node and use join in code.
...
code "/usr/sbin/ntpdate -s #{node["ntp_ipaddresses"].join(" ")}"
...
Please, use ntp cookbook.
I managed to solve what I had posted in the question. The way I did it was using a Template and then the bash script. The recipe code now looks
ntp_servers = search(:node, 'role:ntp_server')
if not node.run_list.roles.include?("ntp_server")
template "/usr/local/bin/ntpdate.sh" do
source "ntpdate.sh.erb"
owner "root"
group "root"
mode 0644
variables(
:ntp_servers => ntp_servers
)
end
bash "ntpdate" do
user "root"
code <<-EOH
bash /usr/local/bin/ntpdate.sh
EOH
end
end
Having done this, I created a template in chef with the following configuration
#!/bin/bash
/usr/sbin/ntpdate -s <% #ntp_servers.each do |ntp_server| -%> <%= ntp_server['ipaddress'] %> <% end -%>
This way I could not dynamically add the ip addresses of the servers belonging to role ntp_server

Chef Recipe Compile Error

Does anyone know why the following code results the error: undefined method 'tar' for "riak-1.4.2":String
remote_file "/vagrant/usr/src/#{node.default['riak']['version'].tar.gz}" do
source "#{node.default['riak']['url']}"
mode 0755
notifies :run, "bash[extract_riak]", :immediately
end
bash "extract_riak" do
code <<-EOH
# Following is the line which causes the error.
/bin/tar xzf /vagrant/usr/src/#{node.default['riak']['version']}.tar.gz -C /vagrant/usr/src/#{node.default['riak']['version']}
EOH
notifies :run, "bash[make_riak]", :immediately
end
This line is raising the error:
remote_file "/vagrant/usr/src/#{node.default['riak']['version'].tar.gz}"
The .tar.gz should be outside the brackets, like so:
remote_file "/vagrant/usr/src/#{node.default['riak']['version']}.tar.gz"
Everything between the brackets is executed as ruby code and the result takes it's place in the string. node.default['riak']['version'].tar.gz is a chain of function calls, including calling a non-existent tar and gz function at the end. These are part of the filename, and should go outside the brackets.
As a side note, you probably want to use node[:attribute] to get attributes, and only use node.default[:attribute] to set attributes.
I recommend the ark cookbook as better choice for handling archives.
The following example recipe:
include_recipe "ark"
ark "riak" do
url "http://s3.amazonaws.com/downloads.basho.com/riak/1.4/1.4.2/riak-1.4.2.tar.gz"
version "1.4.2"
end
will install riak under the "/usr/local/riak-1.4.2" directory.
Finally, there is a riak cookbook available as well, which reportedly will also install from source.
Instead of:
#{node.default['riak']['version']}.tar.gz
you want:
#{node.default['riak']['version'].tar.gz}

Ruby Guard ignore files

I would like to run the requirejs optimization script when any .js file is changed (except for built.js).
I read there is an ignore method. I have the following in my Gaurdfile (which is causing an infinite loop).
guard 'shell', :ignore => 'built.js' do
watch(/script\/(.*).js/) { `node r.js -o build.js` }
end
My question is: How do I configure my Guardfile to ignore the file built.js?
First, assuming you already have the guard-shell gem installed...
I think this gives you something to work from given what you are trying to do.
It will ignore the script/build.js file and trigger a shell command when any other .js file changes.
ignore /script\/build.js/
guard :shell do
watch /.*\.js/ do |m|
`yourcommandhere`
msg = "Processed #{m[0]}"
n msg, 'mycustomshellcommand'
"-> #{msg}"
end
end
See this link for Guardfile examples.
See this link for the syntax of guard-shell.

Resources