Nested iteration in Puppet template [Ruby] - ruby

I started to learn Puppet this week and trying hard to implement user's keys to /etc/ssh/sudo_authorized_keys.
I have a dictionary of users with keys in sudo_users.yaml:
core::sudo_users_keys:
kate:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC3N..."
john:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC..."
marvin:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC3Nza..."
Then I create this in sudokeys.pp file:
class core::sudokeys {
file { "/etc/ssh/sudo_authorized_keys":
ensure => file,
mode => "0440",
content => template("core/sudo_authorized_keys.erb"),
}
As you can see, I want to implement template with iteration.
This is my current template:
<%- scope['core::sudo_users_keys'].each | user | -%>
{
<%- if user[1] -%>
<%- $user[1]['keys'].each do | key | -%>
<%= $key[1]['type'] $key[1]['key'] -%>
<%- end end -%>
}
<%- end -%>
I have the same dictionary with id_rsa keys to ssh and use interaction as below.
It works perfectly for ssh_authorized_key, but I can use it in this case by adding keys to /etc/ssh/sudo_authorized_keys. That's why I decided to use a template and only inject keys inside the sudo_authorized_keys file.
class core::sshkeys {
lookup("core::sudo_users_keys").each | $user | {
if $user[1] {
$user[1]["keys"].each | $key | {
ssh_authorized_key { "${user[0]}-${key[0]}":
ensure => present,
user => $user[0],
type => $key[1]["type"],
key => $key[1]["key"],
}
}
}
}
}
Puppet documentation doesn't include this kind of more complicated iterations and I feel like wandering in the fog.
Currently, I'm getting this error when deploying my template, but I feel like the way I prepare this will not work as I wanted.
Internal Server Error: org.jruby.exceptions.SyntaxError: (SyntaxError) /etc/puppetlabs/code/environments/test/modules/core/templates/sudo_authorized_keys.erb:2: syntax error, unexpected tSTRING_BEG
_erbout.<< " {\n".freeze
^
I will appreciate any suggestions about template construction. What should I change to make it work and extract only type and key values?

Your approach is workable, but there are multiple issues with your code, among them
the scope object in a template provides access to Puppet variables, not to Hiera data. Probably the easiest way to address that would be to
give your core::sudokeys class a class parameter to associate with the data ($userkeys, say),
change the Hiera key for the data so that it will be automatically bound to the new class parameter, and
in the template, access the data via the class parameter (which doesn't even require using the scope object in that case).
in the template, you seem to be assuming a different structure for your data than it actually has in Hiera. Your data is a hashes of hashes of hashes, but you are accessing parts of it as if there were arrays involved somewhere (user[1], key[1]). Also, if there were one or more arrays, then do be aware that Ruby array indexes start at 0, not at 1.
Scriptlet code in an ERB template is (only) Ruby. Your scriptlet code appears to mix Ruby and Puppet syntax. In particular, Ruby variable names are not prefixed with a $, and Ruby block parameters appear inside the block, not outside it.
<%= ... %> tags a for outputting the value of one Ruby expression each. You appear to be trying to emit multiple expressions with the same tag.
Also, it's worth noting that scriptlet code can span multiple lines. That can make it clearer. Additionally, you may need to pay attention to whitespace and newlines in your template to get the output you want. ERB has options to consume unwanted whitespace around scriptlet tags, if needed, but sometimes its at least as easy to just avoid putting in whitespace that you don't want. On the other hand, be sure not to suppress whitespace that you actually wanted.
So, here's one form that all of the above might take:
sudo_users.yaml
core::sudokeys::users_keys:
kate:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC3N..."
john:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC..."
marvin:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC3Nza..."
sudokeys.pp
class core::sudokeys($users_keys) {
file { "/etc/ssh/sudo_authorized_keys":
ensure => file,
mode => "0440",
content => template("core/sudo_authorized_keys.erb"),
}
}
sudo_authorized_keys.erb
<% #users_keys.each do | user |
if user.has_key?('keys') do
user['keys'].each do |key|
%><%= key['type'] %> <%= key['key'] %>
<% # don't consume preceding whitespace here
end
end
end
%>

I used John Bollinger answer, however template wasn't perfect in my case, because I was constantly getting errors like eg. syntax error, unexpected end-of-file _erbout.
The proper sudo_authorized_keys.erb file that worked for me is this one:
<%- #users_keys.each do | user, config | -%>
<%- if config.has_key?('keys') -%>
<%- config['keys'].each do | name, value | -%>
<%= value['type'] %> <%= value['key'] %> <%= user %>
<%- end -%>
<%- end -%>
<%- end -%>

Related

Chef/Ruby: Iterating through a Hash Array

I'm trying to iterate through a hash defined in my Chef attributes file and write it to a config template:
default['disk'] = node['block_device'].select { |i,j| j['state'] == 'running' && i != 'cdrom' }.select { |r| puts "Disk #{r}"}
In my Chef template, I am calling the variable with <%= #disk %>, so all the work is being done in the attributes file variable.
The above attribute will show me the result I want when it is compiling the cookbook, but using the puts method will not write to the config template, and I come up with empty strings written instead (see below).
Compiling Cookbooks...
Disk sda
Disk sdb
Converging 7 resources
....
+ Disk "{}"
If I remove the puts method (should not need it to write to config template), then I get the entire ['block_device'] structure (instead of just the device name) as a Disk value written to config template instead.
I've also tried playing around with using the puts method within the config template as well, but got no where. How can I write a new line in my template per key value in the array during a chef-client run? I would like to get it written to the config template instead of STDOUT during compile??
Chef templates use Erb formatting do you would want to actually use that:
# recipe
template '/asdf' do
# ...
variables disks: node['block_device'].select { |i,j| j['state'] == 'running' && i != 'cdrom' }
end
# template
<%- #disk.each do |i, j| -%>
<%= i %>
<%- end -%>

Puppet Templates Iterations and yaml files

I have the following yaml file in my data dir:
---
type:
- config_setting1:
foo: bar
- config_setting2:
foo: bar
My .erb template looks like this:
conf {
<% settings = YAML.load_file('/etc/puppetlabs/code/environments/example/data/conf.yaml') -%>
<% settings['type'].each do |val| -%>
<%= val %>
<% end -%>
}
When I run puppet on my agent machine I end up with this:
conf {
{"config_setting1"=>nil, "foo"=>"bar"}
{"config_setting2"=>nil, "foo"=>"bar"}
}
My end goal is to get the output to look like this:
conf {
config_setting1 {
foo: bar
}
config_setting2 {
foo: bar
}
}
I know I have some clean up to do on my template to actually get things to output that way, but I'm more focused on the how than the end result at the moment. As you can see I'm familiar with using the ['type'] on the end of the settings to navigate through the nested hash, and I realize I could create this structure pretty easily if I hard coded it but I want to understand how to use it iteratively. I've been attempting to follow the Puppet Documentation on iterations but their examples don't work even when you copy them verbatim... which makes things a little difficult. How can I call pull out a single piece of data in a nested yaml file like I have? Either just the key or just a specific value? I tried something like:
<% settings['type'].each do |val| -%>
<%= settings['val'] %>
<% end -%>
and multiple variations of this but I couldn't find the right syntax to get what I wanted. I've also tried having something along the lines of <% settings['type'].each do |index, value| -%> but I was unable to get any results I could use out of that either. If anyone could point me in the right direction I'd appreciate it. I'm open to being told that I'm going about this entirely the wrong way as well; if there is a better way to get at this data I'm all ears.
Another question that's less important, but still irks me - in my load_file I have the absolute path... is there a way to use relative?
Amazing how typing something out will answer your own question. I realized there was a pretty easy solution. If we take my template:
conf {
<% settings = YAML.load_file('/etc/puppetlabs/code/environments/example/data/conf.yaml') -%>
<% settings['type'].each do |val| -%>
<%= val %>
<% end -%>
}
and on line three replace <% settings['type'].each do |val| -%> with <% settings.keys.each do |val| -%> I'm able to get what I'm looking for. I'd still be interested if there is a better way to do this though, either how I'm loading via yaml or otherwise.

How to replace words inside template placeholders

I'm trying to write a regular expression to replace <%= Name %> with "Some Person".
I'm using a regex because I want to modify it so that I don't have to worry about the spaces between = and Name as well as the E in Name and the %>
I tried:
body = %q(
Hello <%= Name %>,
This is a test. hello test
some more stuff here
and here.
<%= Name %>
)
parsed_body = body.gsub(/\A<%= Name %>\Z/, "Some person")
puts parsed_body
When parsed_body is printed out, the string is unchanged. What is wrong with my regex?
In your Regex, you have added the \A and \z anchors. These ensure that your regex only matches, if the string only contains exactly <%= Name %> with nothing before or after.
To match the your pattern anywhere in the string, you can simply remove the anchors:
parsed_body = body.gsub(/<%= Name %>/, "Some person")
Just another option considering what I am assuming you are trying to accomplish
tag_pattern = /(?<open_tag><%=\s*)(?<key>(\w+))(?<close_tag>\s*%>)/
body = <<-B
Hello,
My name is <%= Name %> and I know some things. Just because I am a
<%= Occupation %> doesn't mean I know everything, but I sure can
<%=Duty%> just as well as anyone else.
Also please don't replace <%= This %>
Thank you,
<%= Name %>
B
dict = {"Name" => "engineersmnky", "Occupation" => "Monkey", "Duty" => "drive a train"}
body.gsub(tag_pattern) {|m| dict[$2] || m }
#=> Hello,
# My name is engineersmnky and I know some things. Just because I am a
# Monkey doesn't mean I know everything, but I sure can
# drive a train just as well as anyone else.
# Also please don't replace <%= This %>
#
# Thank you,
# engineersmnky
In this case I used a dictionary of the anticipated portions of the "erb" to be replaced and used the block style of String#gsub to handle the replacements where $2 is the named capture key. When there is not a matching key it just leaves the match untouched e.g. "Also please don't replace <%= This %>"
You could implement this with any pattern you choose but if you are going to use "erb" style lines maybe try leveraging erb other wise the same will work below:
tag_pattern = (?<open_tag>{!!\s*)(?<key>(\w+))(?<close_tag>\s*!\?})
body = <<-B Hello,
My name is {!! Name !?} and I know some things. Just because I am a
{!! Occupation !?} doesn't mean I know everything, but I sure can
{!!Duty !?} just as well as anyone else.
Thank you,
{!! Name !?}
B
As long as you define tag_pattern correctly the replacement is fairly simple. Rubular Example
It looks like you're trying to write your own template parser, which is asking for a lot more trouble that it's worth considering those already exist.
However, this is the basic idea for such a thing:
erb = <<EOT
Owner: <%= name %>
Location: <%= address %>
EOT
FIELD_DATA = {
name: 'Olive Oyl',
address: '5 Sweethaven Village'
}
FIELD_RE = Regexp.union(FIELD_DATA.keys.map(&:to_s)).source # => "name|address"
puts erb.gsub(/<%=\s+(#{FIELD_RE})\s+%>/) { |k|
k # => "<%= name %>", "<%= address %>"
k[/\s+(\S+)\s+/, 1] # => "name", "address"
FIELD_DATA[k[/\s+(\S+)\s+/, 1].to_sym] # => "Olive Oyl", "5 Sweethaven Village"
}
Which, when run, outputs:
Owner: Olive Oyl
Location: 5 Sweethaven Village
This works because gsub can take a regular expression and a block. For every match of the expression it passes in the match to the block, which is then used to return the actual value being substituted in.
If you have a lot of target values, rather than use Regexp.union, instead use the RegexpTrie gem. See "Is there an efficient way to perform hundreds of text substitutions in Ruby?" for more information.
Again, template parsers exist, they've been around a long time, they're very well tested, and they handle edge cases you haven't thought about, so don't write a new partially-implemented one, instead reuse an existing one.

Ruby accessing nested hash and erb template

I have the following hash that lists what services exist on what machine type and Chef code block. Some of the services have port numbers associated with them.
services_hash = {
"bugs_services" => ["bugs"],
"max_api_services" => ["max-api"],
"max_api_ports_max-api" => 84,
"max_data_services" => ["max-data-http"],
"max_data_ports_max-data-http" => 85,
"max_logger_services" => ["max-analytics", "max-logger"],
"max_logger_ports_max-analytics" => 83,
"max_logger_ports_max-logger" => 82
}
%w( max_api max_data max_logger ).each do |haproxy_chef_role|
template "/home/ubuntu/ansible/deploy/role_#{haproxy_chef_role}.yml" do
source 'haproxy_services.yml.erb'
owner 'ubuntu'
group 'ubuntu'
action :create
mode 0644
variables({
number_of_services: number_of_services,
services: services_hash["#{haproxy_chef_role}_services"],
ports: ???
haproxy_chef_role: haproxy_chef_role
})
end
end
I also have an erb template that resembles this.
<% #services.each do |service| -%>
<% if service.include?("max-logger") %>
shell: for i in {0..<%= #number_of_services %>}; do echo <%= service %>:<%= ports %>$(printf '%02d' $i); done
<% else %>
shell: echo <%= service %>:<%= ports %>00
<% end %>
<% end %>
How can I nest hashes such that a port number is associated to a given service and I can is callable from within the erb template?
I was thinking something like this where I have an array with strings representing the services, and then a hash mapping service to port. I think this will work, but I don't know how to properly extract out the data to fill in my erb template.
services_hash = {
"max_logger_services" => [
"max-logger",
"max-analytics",
{ "max-analytics" => 83, "max-logger" => 82] }
}
I was trying to do something like this earlier where I interpolate whatever I'm currently processing into another hash query, but it's not working well and seems like the wrong way to go about this.
ports: services_hash["#{haproxy_chef_role}_ports_#{services_hash["#{haproxy_chef_role}_services"]}"],
edit:
I've now got the line that returns the hash per Sebastian's answer below which returns the hash relating services to ports.
ports: services_hash['max_logger_services'].inject{|_k,v| v},
In the erb template now though, I don't quite understand how I am supposed to query the hash keys for values. Particularly that I don't seem to be able to nest variables together in an erb.
<% #services.each do |service| -%>
<% if service.include?("max-logger") %>
shell: for i in {0..<%= #number_of_services %>}; do echo disable server <%= service.gsub('max-', '') %>-backend/{{ ansible_fqdn }}:<%= #ports["<%= service %>"] %>$(printf '%02d' $i) | sudo socat stdio /run/admin.sock; done
✗ <% else %>
shell: echo disable server <%= service.gsub('max-', '') %>-backend/{{ ansible_fqdn }}:<%= #ports['<%= service -%>'] -%>00 | sud o socat stdio /run/admin.sock
<% end %>
<% end %>
This line right here seems to the be the problem. The hash is being passed to the erb but I'm not able to drop in the current service being processed into the hash to spit out a value. This is the same fundamental problem I was having that prompted me to ask this question: I can't seem to tell which service I am currently processing to find further data regarding that particular service.
<%= #ports["<%= service %>"] %>
If services_hash had this format:
services_hash = {
"max_logger_services" => [
"max-logger",
"max-analytics", {
"max-analytics" => 83,
"max-logger" => 82
}
]
}
Then you could access max-analytics and to max-logger just with:
hash = services_hash['max_logger_services'][2]
p hash['max-analytics']
# => 83
p hash['max-logger']
# => 82
If you don't know if the data will be always formatted the same way, at least you can "dig" until the deeper hash values:
hash = services_hash['max_logger_services'].inject{|_k,v| v}
# => {"max-analytics"=>83, "max-logger"=>82}

erb file with chef syntax

Trying to output the contents of
node['a'] = {:b "1" :c "2"}
by doing this:
a:
<% a = node['a'] %>
b: <% a[:b] %>
c: <% a[:c] %>
<% end %>
to generate this:
a:
b: 1
c: 2
However not entirely sure the correct syntax to do this being new to ruby, chef and erb.
Okay, so let's rewind a bit. The first thing is that you generally don't want to reference node attributes directly in templates. In some cases like attributes coming from Ohai it can be okay as a shorthand, but for important data I would also pass it in via the variables property like this:
template '/etc/whatever.conf' do
source 'whatever.conf.erb'
variables a: node['a']
end
With that in place we've now expose the data as a template variable. The second piece of improving this is to let Ruby do the heavy lifting of generating YAML. We can do this using the .to_yaml method in the template:
<%= #a.to_yaml %>
That should be all you need!

Resources