I have the following hash that lists what services exist on what machine type and Chef code block. Some of the services have port numbers associated with them.
services_hash = {
"bugs_services" => ["bugs"],
"max_api_services" => ["max-api"],
"max_api_ports_max-api" => 84,
"max_data_services" => ["max-data-http"],
"max_data_ports_max-data-http" => 85,
"max_logger_services" => ["max-analytics", "max-logger"],
"max_logger_ports_max-analytics" => 83,
"max_logger_ports_max-logger" => 82
}
%w( max_api max_data max_logger ).each do |haproxy_chef_role|
template "/home/ubuntu/ansible/deploy/role_#{haproxy_chef_role}.yml" do
source 'haproxy_services.yml.erb'
owner 'ubuntu'
group 'ubuntu'
action :create
mode 0644
variables({
number_of_services: number_of_services,
services: services_hash["#{haproxy_chef_role}_services"],
ports: ???
haproxy_chef_role: haproxy_chef_role
})
end
end
I also have an erb template that resembles this.
<% #services.each do |service| -%>
<% if service.include?("max-logger") %>
shell: for i in {0..<%= #number_of_services %>}; do echo <%= service %>:<%= ports %>$(printf '%02d' $i); done
<% else %>
shell: echo <%= service %>:<%= ports %>00
<% end %>
<% end %>
How can I nest hashes such that a port number is associated to a given service and I can is callable from within the erb template?
I was thinking something like this where I have an array with strings representing the services, and then a hash mapping service to port. I think this will work, but I don't know how to properly extract out the data to fill in my erb template.
services_hash = {
"max_logger_services" => [
"max-logger",
"max-analytics",
{ "max-analytics" => 83, "max-logger" => 82] }
}
I was trying to do something like this earlier where I interpolate whatever I'm currently processing into another hash query, but it's not working well and seems like the wrong way to go about this.
ports: services_hash["#{haproxy_chef_role}_ports_#{services_hash["#{haproxy_chef_role}_services"]}"],
edit:
I've now got the line that returns the hash per Sebastian's answer below which returns the hash relating services to ports.
ports: services_hash['max_logger_services'].inject{|_k,v| v},
In the erb template now though, I don't quite understand how I am supposed to query the hash keys for values. Particularly that I don't seem to be able to nest variables together in an erb.
<% #services.each do |service| -%>
<% if service.include?("max-logger") %>
shell: for i in {0..<%= #number_of_services %>}; do echo disable server <%= service.gsub('max-', '') %>-backend/{{ ansible_fqdn }}:<%= #ports["<%= service %>"] %>$(printf '%02d' $i) | sudo socat stdio /run/admin.sock; done
✗ <% else %>
shell: echo disable server <%= service.gsub('max-', '') %>-backend/{{ ansible_fqdn }}:<%= #ports['<%= service -%>'] -%>00 | sud o socat stdio /run/admin.sock
<% end %>
<% end %>
This line right here seems to the be the problem. The hash is being passed to the erb but I'm not able to drop in the current service being processed into the hash to spit out a value. This is the same fundamental problem I was having that prompted me to ask this question: I can't seem to tell which service I am currently processing to find further data regarding that particular service.
<%= #ports["<%= service %>"] %>
If services_hash had this format:
services_hash = {
"max_logger_services" => [
"max-logger",
"max-analytics", {
"max-analytics" => 83,
"max-logger" => 82
}
]
}
Then you could access max-analytics and to max-logger just with:
hash = services_hash['max_logger_services'][2]
p hash['max-analytics']
# => 83
p hash['max-logger']
# => 82
If you don't know if the data will be always formatted the same way, at least you can "dig" until the deeper hash values:
hash = services_hash['max_logger_services'].inject{|_k,v| v}
# => {"max-analytics"=>83, "max-logger"=>82}
Related
I started to learn Puppet this week and trying hard to implement user's keys to /etc/ssh/sudo_authorized_keys.
I have a dictionary of users with keys in sudo_users.yaml:
core::sudo_users_keys:
kate:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC3N..."
john:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC..."
marvin:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC3Nza..."
Then I create this in sudokeys.pp file:
class core::sudokeys {
file { "/etc/ssh/sudo_authorized_keys":
ensure => file,
mode => "0440",
content => template("core/sudo_authorized_keys.erb"),
}
As you can see, I want to implement template with iteration.
This is my current template:
<%- scope['core::sudo_users_keys'].each | user | -%>
{
<%- if user[1] -%>
<%- $user[1]['keys'].each do | key | -%>
<%= $key[1]['type'] $key[1]['key'] -%>
<%- end end -%>
}
<%- end -%>
I have the same dictionary with id_rsa keys to ssh and use interaction as below.
It works perfectly for ssh_authorized_key, but I can use it in this case by adding keys to /etc/ssh/sudo_authorized_keys. That's why I decided to use a template and only inject keys inside the sudo_authorized_keys file.
class core::sshkeys {
lookup("core::sudo_users_keys").each | $user | {
if $user[1] {
$user[1]["keys"].each | $key | {
ssh_authorized_key { "${user[0]}-${key[0]}":
ensure => present,
user => $user[0],
type => $key[1]["type"],
key => $key[1]["key"],
}
}
}
}
}
Puppet documentation doesn't include this kind of more complicated iterations and I feel like wandering in the fog.
Currently, I'm getting this error when deploying my template, but I feel like the way I prepare this will not work as I wanted.
Internal Server Error: org.jruby.exceptions.SyntaxError: (SyntaxError) /etc/puppetlabs/code/environments/test/modules/core/templates/sudo_authorized_keys.erb:2: syntax error, unexpected tSTRING_BEG
_erbout.<< " {\n".freeze
^
I will appreciate any suggestions about template construction. What should I change to make it work and extract only type and key values?
Your approach is workable, but there are multiple issues with your code, among them
the scope object in a template provides access to Puppet variables, not to Hiera data. Probably the easiest way to address that would be to
give your core::sudokeys class a class parameter to associate with the data ($userkeys, say),
change the Hiera key for the data so that it will be automatically bound to the new class parameter, and
in the template, access the data via the class parameter (which doesn't even require using the scope object in that case).
in the template, you seem to be assuming a different structure for your data than it actually has in Hiera. Your data is a hashes of hashes of hashes, but you are accessing parts of it as if there were arrays involved somewhere (user[1], key[1]). Also, if there were one or more arrays, then do be aware that Ruby array indexes start at 0, not at 1.
Scriptlet code in an ERB template is (only) Ruby. Your scriptlet code appears to mix Ruby and Puppet syntax. In particular, Ruby variable names are not prefixed with a $, and Ruby block parameters appear inside the block, not outside it.
<%= ... %> tags a for outputting the value of one Ruby expression each. You appear to be trying to emit multiple expressions with the same tag.
Also, it's worth noting that scriptlet code can span multiple lines. That can make it clearer. Additionally, you may need to pay attention to whitespace and newlines in your template to get the output you want. ERB has options to consume unwanted whitespace around scriptlet tags, if needed, but sometimes its at least as easy to just avoid putting in whitespace that you don't want. On the other hand, be sure not to suppress whitespace that you actually wanted.
So, here's one form that all of the above might take:
sudo_users.yaml
core::sudokeys::users_keys:
kate:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC3N..."
john:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC..."
marvin:
keys:
key1:
type: "ssh-ed25519"
key: "AAAAC3Nza..."
sudokeys.pp
class core::sudokeys($users_keys) {
file { "/etc/ssh/sudo_authorized_keys":
ensure => file,
mode => "0440",
content => template("core/sudo_authorized_keys.erb"),
}
}
sudo_authorized_keys.erb
<% #users_keys.each do | user |
if user.has_key?('keys') do
user['keys'].each do |key|
%><%= key['type'] %> <%= key['key'] %>
<% # don't consume preceding whitespace here
end
end
end
%>
I used John Bollinger answer, however template wasn't perfect in my case, because I was constantly getting errors like eg. syntax error, unexpected end-of-file _erbout.
The proper sudo_authorized_keys.erb file that worked for me is this one:
<%- #users_keys.each do | user, config | -%>
<%- if config.has_key?('keys') -%>
<%- config['keys'].each do | name, value | -%>
<%= value['type'] %> <%= value['key'] %> <%= user %>
<%- end -%>
<%- end -%>
<%- end -%>
I have the following yaml file in my data dir:
---
type:
- config_setting1:
foo: bar
- config_setting2:
foo: bar
My .erb template looks like this:
conf {
<% settings = YAML.load_file('/etc/puppetlabs/code/environments/example/data/conf.yaml') -%>
<% settings['type'].each do |val| -%>
<%= val %>
<% end -%>
}
When I run puppet on my agent machine I end up with this:
conf {
{"config_setting1"=>nil, "foo"=>"bar"}
{"config_setting2"=>nil, "foo"=>"bar"}
}
My end goal is to get the output to look like this:
conf {
config_setting1 {
foo: bar
}
config_setting2 {
foo: bar
}
}
I know I have some clean up to do on my template to actually get things to output that way, but I'm more focused on the how than the end result at the moment. As you can see I'm familiar with using the ['type'] on the end of the settings to navigate through the nested hash, and I realize I could create this structure pretty easily if I hard coded it but I want to understand how to use it iteratively. I've been attempting to follow the Puppet Documentation on iterations but their examples don't work even when you copy them verbatim... which makes things a little difficult. How can I call pull out a single piece of data in a nested yaml file like I have? Either just the key or just a specific value? I tried something like:
<% settings['type'].each do |val| -%>
<%= settings['val'] %>
<% end -%>
and multiple variations of this but I couldn't find the right syntax to get what I wanted. I've also tried having something along the lines of <% settings['type'].each do |index, value| -%> but I was unable to get any results I could use out of that either. If anyone could point me in the right direction I'd appreciate it. I'm open to being told that I'm going about this entirely the wrong way as well; if there is a better way to get at this data I'm all ears.
Another question that's less important, but still irks me - in my load_file I have the absolute path... is there a way to use relative?
Amazing how typing something out will answer your own question. I realized there was a pretty easy solution. If we take my template:
conf {
<% settings = YAML.load_file('/etc/puppetlabs/code/environments/example/data/conf.yaml') -%>
<% settings['type'].each do |val| -%>
<%= val %>
<% end -%>
}
and on line three replace <% settings['type'].each do |val| -%> with <% settings.keys.each do |val| -%> I'm able to get what I'm looking for. I'd still be interested if there is a better way to do this though, either how I'm loading via yaml or otherwise.
Trying to output the contents of
node['a'] = {:b "1" :c "2"}
by doing this:
a:
<% a = node['a'] %>
b: <% a[:b] %>
c: <% a[:c] %>
<% end %>
to generate this:
a:
b: 1
c: 2
However not entirely sure the correct syntax to do this being new to ruby, chef and erb.
Okay, so let's rewind a bit. The first thing is that you generally don't want to reference node attributes directly in templates. In some cases like attributes coming from Ohai it can be okay as a shorthand, but for important data I would also pass it in via the variables property like this:
template '/etc/whatever.conf' do
source 'whatever.conf.erb'
variables a: node['a']
end
With that in place we've now expose the data as a template variable. The second piece of improving this is to let Ruby do the heavy lifting of generating YAML. We can do this using the .to_yaml method in the template:
<%= #a.to_yaml %>
That should be all you need!
Optimally I'd want template to look something like this:
interfaces {
<%= name %> {
description "<%= description %>";
mtu <%= mtu %>;
}
}
However, I'd want lines not to be printed if the code evaluates to non-true value. It could be hack to ERB, where after \n it sets print_line = true and after evaluating code with b%> to false it sets print_line = false, and only print if print_line.
But it seems changing ERB for this isn't exactly trivial, it reads whole template, non-code parts are inserted as print "data", and result is single big string of ruby code which is evaluated as whole during #result. I would either need to eval b%> code during string-scanning and just insert print "data" if it returns true, or nothing at all or I'd need to rescan the code in #result to run b%> first.
It seems bit fruitless to even use template, if you end up writing all of it inside code blocks, like:
interfaces {
<%= name %> {
<% if description %>
description "<%= description %>";
<% end%>
<% if mtu %>
mtu <%= mtu%>;
<% end%>
}
}
or:
interfaces {
<%= name %> {
<%= "description \"#{description}\";" if description %>
<%= "mtu #{mtu};" if mtu %>
}
}
I need to support various different configuration formats, in some other configuration format it might be 'maximum-transfer-unit <%= mtu> bytes'. I wish to keep all platform-specific intelligence in the template and actual code in the template minimum. I have no problem adding platform agnostic complexity outside the templates.
Seems like relatively common use-case. Before I go NIHing my own template language or hacking ERB, are there perhaps some other template language better fit for my use-case. Or missingsomething else?
I've implemented the hack for <%b stuff %> where whole line is omitted if that evaluates to false. ERB was not designed for anything like this at all so it's very dirty, and it may be better for me to just write my own template language, unless someone can suggest maybe existing solution where I can cleanly implement something like.
For interested parties, hack is here http://p.ip.fi/-NeJ
Ended up re-inventing the wheel: https://github.com/ytti/malline
Helper methods
def description_helper(description)
"description \"#{description}\";" if description
end
def mtu_helper(mtu)
"mtu #{mtu};" if mtu
end
interfaces {
<%= name %> {
<%= description_helper %>
<%= mtu_helper %>
}
}
Decorator pattern
class MyObject
def description
'some description'
end
def mtu
nil
end
module ViewDecorator
def description
"description \"#{super}\";" if super
end
def mtu
"mtu #{super};" if super
end
end
end
o = MyObject.new
p o.description #=> "some description"
p o.mtu #=> nil
o.extend MyObject::ViewDecorator
p o.description #=> "description \"some description\";"
p o.mtu #=> nil
I'm writing an ERB template (for a Puppet module) that gets passed an Hash like this:
{"stuff" => {"foo"=>"aaa", "bar"=>"ccc"},
"other" => {"foo"=>"bbb", "bar"=>"ddd"}}
and I'm iterating over it in my templates producing rows of text:
<% #my_data.each_pair do |k, v| -%>
<%= k %> <%= v["foo"] %>:<%= v["bar"] %>
<% end -%>
Now I'd like to apply some mapping to the "foo" data with a second hash I'll pass to the template. In pseudocode:
mappings = {"aaa" => "something", "bbb" => "somethingelse"}
<% #my_data.each_pair do |k, v| -%>
<%= k %> <%= TRANSLATE_SOMEHOW(v["foo"], mappings) %>:<%= v["bar"] %>
<% end -%>
...in order to get "something" whenever the value was "aaa", and so on. I expect to get the original value if there is no corresponding key in the "mappings".
Doing that kind of thing in Puppet's language is probably possible (by extending it with some Ruby code) I think it is probably more appropriate in the ERB template, but I don't know how to do that and not knowing Ruby isn't helping me - tried google without much success.
I'm looking for code to achieve that in an ERB function or some pointers to relevant documentation for my RTFM pleasure.
EDIT:
for future readers, here's DigitalRoss' answer translated to my ERB example above:
<% #my_data.each_pair do |k, v| -%>
<%= k %> <%= mappings[v["foo"]] || v["foo"] %>:<%= v["bar"] %>
<% end -%>
With the erb stuff removed for clarity, this is what you want to do. (The p() function just prints its argument. You can try this in irb.)
#my_data.each do |k, v|
f, b = v['foo'], v['bar']
p(mappings[f] || f)
p(mappings[b] || b)
end