append multiple structured custom facts - ruby

I'm trying to get multiple structured custom facts to append to my root key (called rag here), but instead they always replace the current value.
Q1: Is this the expected behavior of Facter.add ?
So to get it to work i created two external facts, and on the custom fact i just read their results and append to the root with type => :aggregate
I do get the expected result, which is:
:~# facter -p rag
{
role => "win-mbox",
ambiente => "producao",
enabled_services => [
"auditd",
"lvm2-monitor",
"mdmonitor",
"rhsmcertd",
"rsyslog",
"sshd",
"syslog",
"sysstat",
]
}
I'm breaking it in multiple files for better maintenance (each script outputs 1 key), but i feel like there should be a better way.
Q2: Is there a better way? I mean, not having to use a custom fact just to read the external facts values and append.
Below, details about the code:
/opt/puppetlabs/puppet/cache/facts.d/rag_system_services.rb
#!/usr/bin/env ruby
require 'facter'
require 'json'
retorno = Hash.new { |h,k| h[k] = Hash.new(&h.default_proc) }
os_family = Facter.value(:osfamily)
if os_family == 'RedHat'
retorno[:rag_system_services] = `systemctl list-unit-files --no-legend --no-pager -t service --state=enabled`.scan(/(^.+?)\.service\s+enabled/i).flatten
else
retorno[:rag_system_services] = `rcconf --list | sort`.scan(/(^.+?)\s+on/i).flatten
end
puts JSON.pretty_generate(retorno)
/opt/puppetlabs/puppet/cache/facts.d/rag_role.rb
#!/usr/bin/env ruby
require 'facter'
require 'json'
retorno = Hash.new { |h,k| h[k] = Hash.new(&h.default_proc) }
fqdn = Facter.value(:fqdn)
retorno[:rag_role] = if fqdn.start_with? 'win-'
case fqdn
when /-aio/ then 'win-aio'
when /-ldap/ then 'win-ldap'
when /-logger/ then 'win-logger'
when /-mbox/ then 'win-mbox'
when /-mta/ then 'win-mta'
when /-proxy/ then 'win-proxy'
end
elsif fqdn.include? 'lnx-'
case fqdn
when /balancer/ then 'lnx-balancer'
when /database/ then 'lnx-database'
when /nfs/ then 'lnx-nfs'
when /server/ then 'lnx-server'
end
else
case fqdn
when /^dns-/ then 'dns'
when /^elastic-/ then 'elastic'
when /^pre-auth/ then 'pre-auth'
when /^puppetserver/ then 'puppetserver'
end
end
puts JSON.pretty_generate(retorno)
/opt/puppetlabs/puppet/cache/lib/facter/rag.rb
Facter.add(:rag, :type => :aggregate) do
chunk(:ambiente) do
rag = {}
rag['ambiente'] = (Facter.value(:fqdn).include? 'hom-') ? 'homologacao' : 'producao'
rag
end
chunk(:enabled_services) do
rag = {}
rag['enabled_services'] = Facter.value(:rag_system_services)
rag
end
chunk(:role) do
rag = {}
rag['role'] = Facter.value(:rag_role)
rag
end
end

Related

ruby-snmp: how to automatically convert response to its proper type?

In ruby I read some SNMP registers. Response is an array of objects.
Is there a nice way to convert each object to the proper type avoiding the case..when in the following code? It looks strange that it must be converted manually as the type is already known:
require 'snmp'
HOST = '127.0.0.1'.freeze
registers = ['sysContact.0', 'sysUpTime.0',
'upsIdentManufacturer.0', 'upsIdentModel.0', 'upsIdentName.0']
params_array = {}
SNMP::Manager.open(host: HOST) do |manager|
manager.load_module('UPS-MIB')
response = manager.get(registers)
response.each_varbind do |vb|
##################################
# change from here...
value = nil
case vb.value.asn1_type
when 'OCTET STRING' # <==========
value = vb.value
when 'INTEGER' # <==========
value = vb.value.to_i
when 'TimeTicks' # <==========
value = vb.value.to_s
else
puts "Type '#{vb.value.asn1_type}' not recognized!"
exit(1)
end
params_array[vb.name.to_s] = value
# ... to here
##################################
# with something like
# params_array[vb.name.to_s] = vb.value._to_its_proper_type_
end
end
pp params_array
Looking at the code in the gem repo, it doesn't look like like there is a method for this. I suppose you could try to monkey patch it, but not sure if it's worth the trouble.
If you don't like the switch syntax, you could just use a hash lookup like this:
require 'snmp'
HOST = '127.0.0.1'.freeze
TYPE_VALUES = {
'OCTET STRING' => :to_s,
'INTEGER' => :to_i,
'TimeTicks' => :to_s
}.freeze
registers = ['sysContact.0', 'sysUpTime.0',
'upsIdentManufacturer.0', 'upsIdentModel.0', 'upsIdentName.0']
params_array = {}
SNMP::Manager.open(host: HOST) do |manager|
manager.load_module('UPS-MIB')
response = manager.get(registers)
response.each_varbind do |vb|
if method = TYPE_VALUES[vb.value.ans1_type]
params_array[vb.name.to_s] = vb.value.send(method)
else
puts "Type '#{vb.value.asn1_type}' not recognized!"
exit(1)
end
end
end
pp params_array

How do I pass a hash to a custom function in puppet?

I have defined a custom function currently based on the very simple example here: https://docs.puppet.com/guides/custom_functions.html
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
File.open(filename, 'a') {|fd| fd.puts hash_to_be_transformed }
end
end
This kinda works. I can call it like this:
$my_hash = { key => "value1" , key2 => "value2" }
notify{ "new hash!! $my_hash" :}
transform_service_hash('/var/tmp/blah',$my_hash)
and the file displays:
mgt21 ~ # cat /var/tmp/blah
keyvalue1key2value2
But, if I try to access elements of the hash, nothing changes:
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
element1 = hash_to_be_transformed["key"]
File.open(filename, 'a') {|fd| fd.puts element1 }
end
end
The above block outputs the exact same data to /var/tmp/blah.
And, interestingly, if I remove the filename pass and define it statically in the module:
$my_hash = { key => "value1" , key2 => "value2" }
notify{ "new hash!! $my_hash. element1 is: $my_hash.key" :}
transform_service_hash($my_hash)
and
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
hash_to_be_transformed = args[0]
element1 = hash_to_be_transformed["key"]
File.open('/var/tmp/blah2', 'a') {|fd| fd.puts element1 }
end
end
I get the following error: "Error 400 on SERVER: can't convert Hash into String" with a line reference pointing to "transform_service_hash($my_hash)"
I am new to both puppet and ruby...so I'm unsure I am not passing the element properly, if I am not receiving it properly, or if it something that puppet cannot handle. Please note that I am using version 3.8 of puppet and 1.8.7 of ruby.
Thanks for any help. I've been banging my head against this, and google hasn't been forthcoming yet.
---Edit to clarify my goals (I also edited my code a bit for specificity): I am attempting to pass a hash into a custom ruby function within puppet. The "test" hash has two elements: one string and one array. It is defined as such:
$my_hash = { key => "value1" , key2 => ['array_value1', 'array_value2'] }
$my_display_element=$my_hash["key2"][0]
notify{ "new hash!! $my_hash. the first value of the array stored in element2 is: $my_display_element" :}
transform_service_hash('/var/tmp/blah',$my_hash)
The function appears like so:
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
element1 = args[1]["key"]
element2 = args[1]["key2"][0]
#element1 = hash_to_be_transformed["key"]
#element2 = hash_to_be_transformed["key2"][0]
File.open(filename, 'a') {|fd| fd.puts "hash_to_be_transformed: #{hash_to_be_transformed}\n" }
File.open(filename, 'a') {|fd| fd.puts "element1: #{element1}\n" }
File.open(filename, 'a') {|fd| fd.puts "element2: #{element2}\n" }
end
end
For now, I just want to be able to see that I am able to access elements within the passed hash like a hash. So I'd love for the output file to look like:
hash_to_be_transformed: keyvalue1key2array_value1array_value2
element1: value1
element2: array_value1
However, in the output file, I see:
mgt21 ~ # cat /var/tmp/blah
keyvalue1key2array_value1array_value2
Clearly, something is off here as my text is not being added and the full hash is just printed out just once and seemingly in string form.
I believe that this may be related to the error that I get when I don't pass in a file name (see above). I think that my hash is getting interpreted (or passed) as a string and, as such, I am unable to access the elements. Unfortunately, I still have been unable to verify this or figure out why it might be happening.
---Edit2 based on Matt's answer below.
I decided to simplify my code to isolate this "can't convert Hash into String error". I also made his suggested changes to remove the ambiguity from my key declarations.
$my_hash = { 'key' => "value1" , 'key2' => ['array_value1', 'array_value2'] }
$my_display_element=$my_hash["key2"][0]
notify{ "new hash!! $my_hash. the first value of the array stored in element2 is: $my_display_element" :}
transform_service_hash($my_hash)
and
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
hash_to_be_transformed = args[0]
element1 = args[0]['key']
element2 = args[0]['key2'][0]
File.open('/var/tmp/blah', 'a') {|fd| fd.puts "hash_to_be_transformed: #{hash_to_be_transformed}\n" }
File.open('/var/tmp/blah', 'a') {|fd| fd.puts "element1: #{element1}\n" }
File.open('/var/tmp/blah', 'a') {|fd| fd.puts "element2: #{element2}\n" }
end
end
But, I still end up with the same "Hash to String error". It is worth noting that I also tried simplifying my hash to:
$my_hash = { 'key' => "value1" , 'key2' => "value2" }
and I still get the "Hash to String error".
I quickly took your custom parser function and converted it into pure ruby like the following:
hash = { 'key' => 'value1', 'key2' => %w(array_value1 array_value2) }
def newfunction(filename, a_hash)
element1 = a_hash['key']
element2 = a_hash['key2'][0]
File.open(filename, 'a') do |fd|
fd.puts "hash_to_be_transformed: #{a_hash}"
fd.puts "element1: #{element1}"
fd.puts "element2: #{element2}"
end
end
newfunction('foo.txt', hash)
This results in the output text file like the following:
hash_to_be_transformed: {"key"=>"value1", "key2"=>["array_value1", "array_value2"]}
element1: value1
element2: array_value1
This seems to confirm my initial suspicion about what is going wrong here. Your hash in Puppet of:
$my_hash = { key => "value1" , key2 => ['array_value1', 'array_value2'] }
has keys of implicit/ambiguous types. In the ruby code I used to test, I explicitly established them as strings. This also correlates strongly with these lines in your code failing:
element1 = args[1]["key"]
element2 = args[1]["key2"][0]
and your error message of:
Error 400 on SERVER: can't convert Hash into String
because you are specifying in your ruby code that you expect the keys to be string. Changing your hash in Puppet to:
$my_hash = { 'key' => "value1" , 'key2' => "value2" }
should fix this.
On an unrelated note, I recommend the use of linters to help you learn these languages. Puppet-Lint, Rubocop, and Reek will all help point out suboptimal and messy parts of your code to help you learn the new languages.
On a related note, you may want to put something like this at the top of your custom parser function:
raise(Puppet::ParseError, 'newfunction expects two arguments') if args.length != 2
After much gnashing of teeth (and some very helpful pointers from #MattSchuchard), I realized that none of the changes to my function were going into effect. One needs to restart the puppetmaster service after each change to a custom function: docs.puppet.com/guides/custom_functions.html (appropriately under "Gotchas").
Once I started restarting this service after each change to the function, my hash was able to be parsed properly:
from the .pp file:
$filename = "/var/tmp/test"
$my_hash = { 'key' => "value1" , 'key2' => ["M\'lady\n*doffs cap*", 'array_value2'] }
transform_service_hash($filename, $my_hash)
from the ruby file:
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
array_val = hash_to_be_transformed['key2'][0]
File.open(filename, 'a') {|fd| fd.puts "#{array_val}\n" }
end
end
and output:
mgt21 tmp # cat test
M'lady
*doffs cap*

Dynamically check if a field in JSON is nil without using eval

Here's an extract of the code that I am using:
def retrieve(user_token, quote_id, check="quotes")
end_time = Time.now + 15
match = false
until Time.now > end_time || match
#response = http_request.get(quote_get_url(quote_id, user_token))
eval("match = !JSON.parse(#response.body)#{field(check)}.nil?")
end
match.eql?(false) ? nil : #response
end
private
def field (check)
hash = {"quotes" => '["quotes"][0]',
"transaction-items" => '["quotes"][0]["links"]["transactionItems"]'
}
hash[check]
end
I was informed that using eval in this manner is not good practice. Could anyone suggest a better way of dynamically checking the existence of a JSON node (field?). I want this to do:
psudo: match = !JSON.parse(#response.body) + dynamic-path + .nil?
Store paths as arrays of path elements (['quotes', 0]). With a little helper function you'll be able to avoid eval. It is, indeed, completely inappropriate here.
Something along these lines:
class Hash
def deep_get(path)
path.reduce(self) do |memo, path_element|
return unless memo
memo[path_element]
end
end
end
path = ['quotes', 0]
hash = JSON.parse(response.body)
match = !hash.deep_get(path).nil?

Obtain BigQuery table list in Ruby

I would like to use the Google bigquery gem (https://rubygems.org/gems/bigquery) to create an Array of table names. So far, this is what I have written:
require 'json'
bqRepsonse = bq.tables('myDataSet')
bqRepsonseCleaned = bqRepsonse.to_s.gsub("=>", ":")
data = JSON.parse(bqRepsonseCleaned)
tableListing = []
data["tableID"]["type"].each do |item|
case item["type"]
when 'TABLE'
bqTableList << item["tableId"]
else
end
end
If I print bqResponse, I get this result:
[{"kind"=>"bigquery#table",
"id"=>"curious-idea-532:dataset_test_4.TableA",
"tableReference"=>{"projectId"=>"curious-idea-532",
"datasetId"=>"dataset_test_4", "tableId"=>"TableA"}, "type"=>"TABLE"},
{"kind"=>"bigquery#table",
"id"=>"curious-idea-532:dataset_test_4.TableB",
"tableReference"=>{"projectId"=>"curious-idea-532",
"datasetId"=>"dataset_test_4", "tableId"=>"TableB"}, "type"=>"TABLE"},
{"kind"=>"bigquery#table",
"id"=>"curious-idea-532:dataset_test_4.TableC",
"tableReference"=>{"projectId"=>"curious-idea-532",
"datasetId"=>"dataset_test_4", "tableId"=>"TableC"}, "type"=>"TABLE"},
{"kind"=>"bigquery#table",
"id"=>"curious-idea-532:dataset_test_4.TableD",
"tableReference"=>{"projectId"=>"curious-idea-532",
"datasetId"=>"dataset_test_4", "tableId"=>"TableD"}, "type"=>"TABLE"}]
And running the code throws and error
`[]': no implicit conversion of String into Integer (TypeError)
Not sure where to correct this. My desired outcome is:
tableListing = ["TableA","TableB","TableC","TableD"]
Thanks in advance for your advice.
Try this:
require 'json'
string = '[{"kind": "bigquery#table", "id": "curious-idea-532:dataset_test_4.TableA", "tableReference" : {"projectId":"curious-idea-532", "datasetId":"dataset_test_4", "tableId":"TableA"}, "type":"TABLE"}, {"kind":"bigquery#table", "id":"curious-idea-532:dataset_test_4.TableB", "tableReference":{"projectId":"curious-idea-532", "datasetId":"dataset_test_4", "tableId":"TableB"}, "type":"TABLE"}, {"kind":"bigquery#table", "id":"curious-idea-532:dataset_test_4.TableC", "tableReference":{"projectId":"curious-idea-532", "datasetId":"dataset_test_4", "tableId":"TableC"}, "type":"TABLE"}, {"kind":"bigquery#table", "id":"curious-idea-532:dataset_test_4.TableD", "tableReference":{"projectId":"curious-idea-532", "datasetId":"dataset_test_4", "tableId":"TableD"}, "type":"TABLE"}]'
data = JSON.parse(string)
tableListing = []
# Here we are iterating over the data instead of its child element
data.each do |item|
case item["type"]
when 'TABLE'
tableListing << item["tableReference"]["tableId"]
else
end
end
puts tableListing

merging similar hashes in ruby?

I've tried and tried, but I can't make this less ugly/more ruby-like. It seems like there just must be a better way. Help me learn.
class Df
attr_accessor :thresh
attr_reader :dfo
def initialize
#dfo = []
#df = '/opt/TWWfsw/bin/gdf'
case RUBY_PLATFORM
when /hpux/i
#fstyp = 'vxfs'
when /solaris/i
# fix: need /tmp too
#fstyp = 'ufs'
when /linux/i
#df = '/bin/df'
#fstyp = 'ext3'
end
#dfo = parsedf
end
def parsedf
ldf = []
[" "," -i"] .each do |arg|
fields = %w{device size used avail capp mount}
fields = %w{device inodes inodesused inodesavail iusep mount} if arg == ' -i'
ldf.push %x{#{#df} -P -t #{#fstyp}#{arg}}.split(/\n/)[1..-1].collect{|line| Hash[*fields.zip(line.split).flatten]}
end
out = []
# surely there must be an easier way
ldf[0].each do |x|
ldf[1].select { |y|
if y['device'] == x['device']
out.push x.merge(y)
end
}
end
out
end
end
In my machine, your ldf array after the df calls yields the following:
irb(main):011:0> ldf
=> [[{"device"=>"/dev/sda5", "size"=>"49399372", "mount"=>"/", "avail"=>"22728988", "used"=>"24161036", "capp"=>"52%"}], [{"device"=>"/dev/sda5", "inodes"=>"3137536", "mount"=>"/", "iusep"=>"13%", "inodesavail"=>"2752040", "inodesused"=>"385496"}]]
The most flexible approach to merging such a structure is probably something along these lines:
irb(main):013:0> ldf.flatten.inject {|a,b| a.merge(b)}
=> {"device"=>"/dev/sda5", "inodes"=>"3137536", "size"=>"49399372", "mount"=>"/", "avail"=>"22728988", "inodesavail"=>"2752040", "iusep"=>"13%", "used"=>"24161036", "capp"=>"52%", "inodesused"=>"385496"}
Some ruby programmers frown on this use of inject, but I like it, so your mileage may vary.
As for helping making your code more ruby like, I suggest you talk to some experienced rubyist you might know over your code to help you rewriting it in a way that follows good style and best practices. Probably that would the preferable than to just have someone rewrite it for you here.
Best of Luck!
Didn't test the code, but here goes:
ARGUMENTS = {
" " => %w{size used avail capp mount},
" -i" => %w{inodes inodesused inodesavail iusep mount}
}
def parsedf
# Store resulting info in a hash:
device_info = Hash.new do |h, dev|
h[dev] = {} # Each value will be a empty hash by default
end
ARGUMENTS.each do |arg, fields|
%x{#{#df} -P -t #{#fstyp}#{arg}}.split(/\n/)[1..-1].each do |line|
device, *data = line.split
device_info[device].merge! Hash[fields.zip(data)]
end
end
device_info
end
Notes: returns something a bit different than what you had:
{ "/dev/sda5" => {"inodes" => "...", ...},
"other device" => {...}
}
Also, I'm assuming Ruby 1.8.7 or better for Hash[key_value_pairs], otherwise you can resort to the Hash[*key_value_pairs.flatten] form you had
Depending on your needs, you should consider switch the fields from string to symbols; they are the best type of keys.

Resources