I've got a Vagrant setup in which I'm trying to use Chef-solo to generate an conf file which loops though defined variables to pass to the application. Everything is working except the loop and I'm not familiar enough with Ruby/Chef to spot the error.
I'm going to lay out the whole chain of events in case there is something along the way that is the problem, but the first portions of this process seem to work fine.
A config file is written in yaml and includes env variable definitions to be passed:
...
variables:
- DEBUG: 2
...
The config file is read in by the Vagrantfile into a ruby hash and used to create the Chef json nodes:
...
settings = YAML::load(File.read("config.yaml"))
# Provision The Virtual Machine Using Chef
config.vm.provision "chef_solo" do |chef|
chef.json = {
"mysql" => {"server_root_password" => "secret"},
"postgresql" => {"password" => {"postgres" => "secret"}},
"nginx" => {"pid" => "/run/nginx.pid"},
"php-fpm" => {"pid" => "/run/php5-fpm.pid"},
"databases" => settings["databases"] || [],
"sites" => settings["sites"] || [],
"variables" => settings["variables"] || []
}
...
A bunch of chef cookbooks are run (apt, php, nginx, mysql etc) and finally my custom cookbook which is whats giving me grief. The portion of the cookbook responsible for creating a the conf file is shown here:
# Configure All Of The Server Environment Variables
template "#{node['php-fpm']['pool_conf_dir']}/vars.conf" do
source "vars.erb"
owner "root"
group "root"
mode 0644
variables(
:vars => node['variables']
)
notifies :restart, "service[php-fpm]"
end
And the vars.erb is just a one-liner
<%= #vars.each {|key, value| puts "env[" + key + " = " + value } %>
So, when I run all this chef spits out an error about not being able to convert a hash to a string.
can't convert Chef::Node::immutableMash into String
So for some reason this is coming across as an immutableMash and the value of key ends up being the hash [{"DEBUG"=>2}] and value ends up a nil object, but I'm not sure why or how to correct it.
The hash is ending up as the value of key in your example because the YAML file declares DEBUG: 2 as a list member of variables. This translates to variables being an array with a single hash member.
Try changing the template code to this:
<%= #vars[0].each {|key, value| puts "env[" + key + " = " + value } %>
Or try changing the YAML to this and not changing the template code:
variables:
DEBUG: 2
Either change will get your template loop iterating over the hash that you are expecting.
Related
I have working Ruby code to query DNS details and create Puppet custom facts (puppet 5, Facter 3.11.6) however I am trying to modify it to create nested facts from the key/value pairs that the query obtains.
Code that works to set individual facts with the key name is:
require 'resolv'
Resolv::DNS::Config.default_config_hash.each do | key, value |
if !value.nil?
Facter.add("dns_#{key}") do
if value.is_a?(Array)
setcode { value.join(',') }
else
setcode { value }
end
end
end
end
which creates individual facts thus:
dns_nameserver => 192.168.1.1,192.168.1.2
dns_ndots => 1
dns_search => test.domain
My failed attempt so far to create a nested fact under the parent fact of 'DNS' is:
require 'resolv'
Facter.add("dns") do
value ={}
Resolv::DNS::Config.default_config_hash.each do | key, result |
if !result.nil?
if result.is_a?(Array)
setcode { value['#{key}'] = result.join(',') }
else
setcode { value['#{key}'] = result }
end
end
end
end
which gives a limited result of just:
dns => 1
Other code I have tried seems to put an array output into the string and multiple IPs are quoted inside square brackets over 2 lines instead of being output as per the first code block at top of page.
The fact structure I am TRYING to achieve (by modifying the top of page code) is:
dns => {
nameserver => 192.168.1.1,192.168.1.2,
ndots => 1,
search => test.domain,
}
Thanks in advance for any assistance.
I finally got this with the assistance from a poster who put some great code leads here, but unfortunately removed it soon afterward. Here is the code that works:
require 'resolv'
Facter.add(:networking_dns) do
setcode do
Resolv::DNS::Config.default_config_hash.each_with_object({}) do | (key, value), sub|
if !value.nil?
sub[key] = value
sub
end
end
end
end
Now for some explanatory notes (please feel free to correct me or offer any optimisations to this):
# the resolv gem is required
require 'resolv'
# create the parent fact (has no value of its own)
Facter.add(:networking_dns) do
# start building instructions in the fact
setcode do
# use the resolv gem to lookup values in /etc/resolv.conf and add .each to process all key/value pairs returned
# also add _with_object({}) and sub in the variables to set a blank value for sub. Saves doing it separately. Sub can be any name but denotes the declaration for the nested facts
Resolv::DNS::Config.default_config_hash.each_with_object({}) do | (key, value), sub|
# create facts only when the value is not nil
if !value.nil?
sub[key] = value
sub
# using a closing blank entry for a nested fact is critical or they won't create! Place this outside of the case statement to prevent blank values
end
end
end
end
# use the appropriate number of ends and indent for readability
Thanks to the person who posted their guidance here before removing it. I would like to upvote you if you post again.
Any tips on optimisation to the able solution are welcome, as I'm still grasping Ruby (spent hours on this!)
I'm trying to iterate through a hash defined in my Chef attributes file and write it to a config template:
default['disk'] = node['block_device'].select { |i,j| j['state'] == 'running' && i != 'cdrom' }.select { |r| puts "Disk #{r}"}
In my Chef template, I am calling the variable with <%= #disk %>, so all the work is being done in the attributes file variable.
The above attribute will show me the result I want when it is compiling the cookbook, but using the puts method will not write to the config template, and I come up with empty strings written instead (see below).
Compiling Cookbooks...
Disk sda
Disk sdb
Converging 7 resources
....
+ Disk "{}"
If I remove the puts method (should not need it to write to config template), then I get the entire ['block_device'] structure (instead of just the device name) as a Disk value written to config template instead.
I've also tried playing around with using the puts method within the config template as well, but got no where. How can I write a new line in my template per key value in the array during a chef-client run? I would like to get it written to the config template instead of STDOUT during compile??
Chef templates use Erb formatting do you would want to actually use that:
# recipe
template '/asdf' do
# ...
variables disks: node['block_device'].select { |i,j| j['state'] == 'running' && i != 'cdrom' }
end
# template
<%- #disk.each do |i, j| -%>
<%= i %>
<%- end -%>
In my chef environment I have a variable like this:
"repl_set_members": ["tmongo01", "tmongo02", "tmongo03"]
I need to create JSON to be sent to a Mongo instance and build the replica set.
I created a bash file from a template with:
template "/opt/create_repl_set.sh" do
source "create_repl_set.sh.erb"
owner 'root'
group 'root'
mode "0755"
variables(
:repl_set_name => node['mongodb']['shardname'],
:members => node['mongodb']['repl_set_members']
)
end
And in the bash template I would have something like this:
config = "{_id: '<%= #repl_set_name %>', members: [
<% #members.each_with_index do |name,number| %>
{_id: <%= "#{number}" %>, host: '<%= "#{name}" %>:27017'},
<% end %>
] }"
mongo localhost:27091 --eval "JSON.stringify(db.adminCommand({'replSetInitiate' : $config}))"
But the resulting JSON includes a last comma which I don't know how to get rid of.
Does anyone have a better idea?
A quick and dirty way to remove your last comma would be to use bash string manipulation and call your script as follow :
mongo localhost:27091 --eval "JSON.stringify(db.adminCommand({'replSetInitiate' : ${config/%,/}))"
Note that this will always delte the last comma in your JSON value, so it will only work if your resulting JSON always has an extra comma (i.e. even when your JSON is empty)
What you probably want is an execute resource an .to_json
mongo_cmd = {
'replSetInitiate' => {
'_id' => node['mongodb']['shardname'],
'members' => node['mongodb']['repl_set_members'].each_with_index.map { |name, number|
{'_id' => number, 'host' => "#{name}:27017"}
},
},
}
execute "mongo localhost:27091 --eval 'JSON.stringify(db.adminCommand(#{mongo_cmd.to_json}))'"
The code below gets a list of images from the public/images directory, creates an array of hashes and converts it to JSON then returns to the caller.
The code works perfectly on the local host - I get the list of image names as needed.
I then uploaded the code to my VPS and ran it on the same environment, running on thin, and nothing is returned at all. No matter what I change, either the path or method for getting filenames, like using glob instead of just Dir, nothing works that I tried.
Here is the code in the route I call from the client-side JavaScript using Ajax:
# get all images
get '/debug/posts/images/' do
puts '>> debug > posts > images > get'
all_images = Array.new
# build substitute prefix path
uri = URI(request.url)
prefix ='http://' + uri.host
if request.port
prefix += ':' + request.port.to_s
end
prefix += '/content/'
begin
content_type :json
# get list of images
pics = Dir['public/images/*']
pics.map { |pic|
# build hash for use with tinyMCE
pic.split('/')
pic_hash = {:title => File.basename(pic).to_s, :value => prefix + File.basename(pic).to_s}
all_images.push(pic_hash)
}
# convert to json
pic_json = JSON.generate(all_images)
body(pic_json)
rescue Sequel::Error => e
puts e.message
status(400).to_json
end
end
I get an array of values back running on the localhost:
[ {title: "bear-love.jpg"value: "/content/bear-love.jpg"}, {title: "bear-love2.jpg"value: "/content/bear-love2.jpg"}...]
I get an empty array from the VPS:
[]
Check that the server is being run as a user who has permission to read and execute public/images
verify that public/images exists
verify that public/images is where you think it is.
Turns out that it's easier to get the current path on the server and simply get a list of the image filenames I needed that way and just add my prefix path for use from the browser.
I have a Chef recipe for a multi-node web service, each node of which needs to get the hostname and IP of the other nodes, to put it into its own local configuration.
The code is shown below. The problem is that when the node.set[][] assignments are made in the ruby_block as shown, the values are empty when the template that relies upon them is created. If I want to create that template, I have to move all of the ruby_block code outside, and have it "loose" in the recipe. Which makes it harder to do unit-testing with Chefspec and the like.
Can any Chef guru set me straight? Is it just impossible to do node.set[] like this inside of a ruby_block? And if so, why doesn't it say so in the docs?
$cm = { :name => "web", :hostname => "" , :ip_addr => "" }
$ca = { :name => "data", :hostname => "" , :ip_addr => "" }
$cg = { :name => "gateway", :hostname => "" , :ip_addr => "" }
$component_list = [$cm, $ca, $cg]
ruby_block "get host addresses" do
block do
for cmpnt in $component_list
# do REST calls to external service to get cmpnt.hostname, ip_addr
# .......
node.set[cmpnt.name]['name'] = cmpnt.name
node.set[cmpnt.name]['host'] = cmpnt.hostname
node.set[cmpnt.name]['ip'] = cmpnt.ip_addr
end
end
end
template "/etc/app/configuration/config.xml" do
source "config.xml.erb"
variables( :dataHost => node['data']['host'],
:webHost => node['web']['host'],
:gatewayHost => node['gateway']['host'] )
action :create
end
I also added
subscribes :create, "ruby_block[get host addresses]", :immediately
to the template definition to ensure that the ruby_block ran before the template was created. This didn't make a difference.
I realize this is an old post, however for future reference, I just ran across this gist which gives a nice example of node variable assignments in the Compile vs. Converge phases. To adapt the gist to your example, you'll need to add code like the following to your ruby_block:
template_r = run_context.resource_collection.find(:template => "/etc/app/configuration/config.xml")
template_r.content node['data']['host']
template_r.content node['web']['host']
template_r.content node['gateway']['host']
For Chef 11, also see Lazy Attribute Evaluation.
The problem seems to be that attribute values inside your template resource definition get evaluated before actually invoking any resources.
I.e. the file is first executed as simple Ruby, compiling the resources, and only the the resource actions gets invoked. By that time, it is too late already.
I ran into the same problem when trying to encapsulate certain attribute manipulations into a resource. It simply does not work. Should anyone know a solution to this problem, I would appreciate it very much.
EDIT:
b = ruby_block...
...
end
b.run_action(:create)
Could possibly do the trick. It invokes the resource immediately.
The simplest answer to this is to not use chef attributes and not use ruby_block to do the work of talking to the REST API. The code can also be moved to a custom resource for better reuse:
unified_mode true
provides :my_resource
action :run do
cm = { :name => "web", :hostname => "" , :ip_addr => "" }
ca = { :name => "data", :hostname => "" , :ip_addr => "" }
cg = { :name => "gateway", :hostname => "" , :ip_addr => "" }
component_list = [cm, ca, cg]
hash = {}
for cmpnt in component_list
# do REST calls to external service to get cmpnt.hostname, ip_addr
# .......
hash[cmpnt.name] = {}
hash[cmpnt.name]['name'] = cmpnt.name
hash[cmpnt.name]['host'] = cmpnt.hostname
hash[cmpnt.name]['ip'] = cmpnt.ip_addr
end
template "/etc/app/configuration/config.xml" do
source "config.xml.erb"
variables( :dataHost => hash['data']['host'],
:webHost => hash['web']['host'],
:gatewayHost => hash['gateway']['host'] )
action :create
end
end
By using unified_mode and moving into a custom resource, it also makes it easier to use a node attribute without requiring the use of lazy {} or ruby_blocks. It also still allows chef configuration (like setting up resolv.conf or other network requirements before doing the REST calls) prior to calling this code while not having to think about compile/converge two pass issues in recipe context.
There is also no reason to use a resource like ruby_block to do pure ruby processing which does not change the system under management. In this case the ruby_block is hitting a REST service purely to collect data. That does not need to be placed into a Chef resource. It isn't clear from the question if that was being done because the questioner though it was a "best practice" (in this case it is not), or if it was being done to move execution to compile time in order to allow other chef resources that aren't part of the question to fire first (in which case using a custom resource is a much better solution than using a ruby_block).
It's been a while since this question, but in case someone is still looking for it, lazy evaluate is your friend:
template '/tmp/sql_file.sql' do
source "sql_file.sql.erb"
mode 0700
variables lazy {
# Create a new instance of MySQL library
mysql_lib = Acx::MySQL.new(
'127.0.0.1', 'root', node['mysql']['service']['pass']
)
password = node['mysql']['service']['support_admin']['ct_password']
# It returns the encrypted password after evaluate it, to
# be used in template variables
{ admin_password: mysql_lib.encrypted_password(password) }
}
end
https://docs.chef.io/resource_common.html#lazy-evaluation