Ruby, parsing YAML and outputting value - ruby

I'm pretty new to ruby and all the documentation on this subject has confused me a bit. So here goes.
I'm using inspec to test my infrastructure, but I want it to consume some variables from the YAML file used by ansible. This effectively means I can share vars from ansible code and use them in ruby.
The YAML file looks like this:
- name: Converge
hosts: all
vars:
elasticsearch_config:
cluster.name: "{{ elasticsearch_cluster_name }}"
node.name: "es-test-node"
path.data: "/var/lib/elasticsearch"
path.logs: "/var/log/elasticsearch"
elasticsearch_cluster_name: test
pre_tasks:
roles:
- elasticsearch
post_tasks:
At this point, I'm just playing around with ruby code to extract that, and have:
require 'yaml'
parsed = begin
YAML.load(File.open("../../playbook.yml"))
rescue ArgumentError => e
puts "Could not parse YAML: #{e.message}"
end
puts parsed
Which outputs the hash:
{"name"=>"Converge", "hosts"=>"all", "vars"=>{"elasticsearch_config"=>{"cluster.name"=>"{{ elasticsearch_cluster_name }}", "node.name"=>"es-test-node", "path.data"=>"/var/lib/elasticsearch", "path.logs"=>"/var/log/elasticsearch"}, "elasticsearch_cluster_name"=>"test"}, "pre_tasks"=>nil, "roles"=>["elasticsearch"], "post_tasks"=>nil}
So far so good. This all makes sense to me. Now, I would like to pull values out of this data and use them in the ruby code, referencing them by the keys. So, if I wanted to get the value of vars.elasticsearch_config.node.name, how would I go about doing this?

YAML.load reads the document into an array, so you must get the first element in your example:
loaded_yaml[0]["vars"]["elasticsearch_config"]["node.name"]
The reason for this is that the document you are parsing begins with a single dash, indicating a list item. Even though there is only one item in the list, Psych (thy YAML engine) is still placing it into an array representing a list. This is also why you got a no implicit conversion of String to Integer error. Note that the response you get is enclosed by square brackets:
=> [{"name"=>"Converge", "hosts"=>"all", "vars"=>{"elasticsearch_config"=>{"cluster.name"=>"{{ elasticsearch_cluster_name }}", "node.name"=>"es-test-node", "path.data"=>"/var/lib/elasticsearch", "path.logs"=>"/var/log/elasticsearch"}, "elasticsearch_cluster_name"=>"test"}, "pre_tasks"=>nil, "roles"=>["elasticsearch"], "post_tasks"=>nil}]

Related

Ruby 1.8 vs 2.3 handling YAML config arrays differently

I'm trying to upgrade a server that has ruby scripts developed by another person. I'm a perl/php developer and no little about ruby, just trying to get the scripts to work that was developed with Ruby 1.8 and the scripts seems to behave differently handling arrays in the newer version. The script was not matching iterated folders with a config file array with the folder names and I believe I've boiled it down to the way the YAML config file is converted to an array. I put together this simple script:
require 'rubygems'
require 'yaml'
config_filename = File.expand_path(File.dirname(__FILE__) + "/testruby.yml")
#config = YAML.load(File.open(config_filename))
puts #config
The YAML testruby.yml config file looks like this:
1_01:
name: Monday Show
suffix: showM
program_id: 123
segment: 1
dated: false
1_02:
name: Monday Show
suffix: showM
program_id: 123
segment: 2
dated: false
1_03:
name: Tuesday Show
suffix: showT
program_id: 124
segment: 1
dated: true
When I run this on the original server with Ruby 1.8, the result is:
1_03program_id124nameTuesday Showsegment1suffixshowTdatedtrue1_02program_id123nameMonday Showsegment2suffixshowMdatedfalse1_01program_id123nameMonday Showsegment1suffixshowMdatedfalse
But when ran on the new server with Ruby 2.3 I get a array:
{101=>{"name"=>"Monday Show", "suffix"=>"showM", "program_id"=>123, "segment"=>1, "dated"=>false}, 102=>{"name"=>"Monday Show", "suffix"=>"showM", "program_id"=>123, "segment"=>2, "dated"=>false}, 103=>{"name"=>"Tuesday Show", "suffix"=>"showT", "program_id"=>124, "segment"=>1, "dated"=>true}}
It even removes the underscore from the folder name key in the config file. For this reason, later in the script, calls to #config[1_01] does not match of course. Is there a way to get the array to build like version 1.9 so the rest of the script works as designed?
One more thing to note, not sure if it related to the issue. The require 'yaml' line was not present in the script, I added after receiving this error when ran:
testruby.rb:4:in `<main>': uninitialized constant YAML (NameError)
Well, it seems all I had to do was enclose the YAML keys in quotes and now the hash object includes the underscore in the keys and the rest of the script works!

Writing untagged YAML with Ruby Psych, then reading It

I have Ruby using Psych that writes YAML like this:
---
- !ruby/struct:Dhc::QueryEvent
time: 2018-01-02T07:45:18.470Z
hits: 2525
qtime: 13
server: search1
q: search term
searched_in:
- productA
- productB
- productC
Is there a way I can write the YAML without the "!ruby/struct:Dhc::QueryEvent" tag? I want the storage formatted so it is not dependent on my struct:Dhc::QueryEvent for reading. In other words, I want others to have the flexibility to read the YAML however they want. Currently, they get
undefined class/module Dhc:: (ArgumentError)
Then, when I'm reading the file myself, is there a configuration in Psych that allows me to read the untagged records into my Dhc::QueryEvent struct? I assume this would be something like
ar = []
Psych.parse_stream(yaml_stream) do |node|
ar << Dhc::QueryEvent.new(node.time, node.hits, etc...)
end
but maybe there is a shorter way?

mcollective inventory script

I created mCollective inventory script as below,
def formatting(users_ids)
YAML.load(File.open(users_ids))
end
inventory do
format "%s\t%s\t"
fields { [facts["hostname"], formatting(facts["users_ids"]) ] }
end
Here users_ids facter is in yaml format on the server. So when I do inventory for this facter I need to parse that yaml format to hash. But when I run with this script getting below error,
[root#mco-server]#
The inventory application failed to run, use -v for full error backtrace details: (eval):2:in `initialize': No such file or directory - ---
root: 0
test1: 503
testuser: 2033
[root#mco-server]#
Not sure if am missing something to get the output parsed. The strange thing is its not printing hostname also.
The facter output is below on the server from facts.yaml
users_ids: |-
---
root: 0
test1: 503
testuser: 2033
Any help would be much appreciated.
According to the error message, the argument you are passing to users_ids is not a valid filename.
def formatting(users_ids)
YAML.load(File.open(users_ids))
end
Somehow your code is passing --- as an argument to that method. This is likely due to the combination of your API calls to parse and load the yaml and the yaml file itself. Consider changing the API call to a cleaner:
def formatting(users_ids)
YAML.load_file(users_ids)
end
and I think you really want a hash in your yaml and not an array of key-value pairs with an element of ---, so your yaml should really be:
users_ids:
root: 0
test1: 503
testuser: 2033
which would also remove the --- which typically indicates the beginning of a yaml, and also seems to be what your code is erroring on in the way you are trying to load the yaml.

Conditional check in yaml file to show the proper content

How can I check if / else in yaml file.
like:
if %{attribute}
attributes:
shipping_comment: Shipping comment / Instructions
else
attributes:
shipping_date: Date
YAML is a data serialisation language, so it's not meant to contain if/else style executable statements: that's the responsibility of the programming language you're using.
A simple example in Ruby to determine which config string from a YAML file to output could be defining your YAML config file as follows:
data.yml
attributes:
shipping_comment: Shipping comment / Instructions
shipping_date: Date
Then, in your program, read the file in and run the conditional there:
shipping.rb
#!/usr/bin/env ruby
require 'yaml'
config = YAML.load_file('data.yml')
attribute = true # your attribute to check here
if attribute
puts config['attributes']['shipping_comment']
else
puts config['attributes']['shipping_date']
end
Out of the box .yaml files won't include any conditional logic, as Paul Fioravanti says:
YAML is a data serialisation language, so it's not meant to contain if/else style executable statements: that's the responsibility of the programming language you're using.
However, there are some cases, such as Infrastructure as Code where you might not have the luxury of Paul's solution. In these cases, most decent Infrastructure tools provide an inbuilt way of achieving conditional logic.
Since it appears that infra is not the area you're looking in, I won't go into detail on how to write each tools solution, but for anyone that end's up here like I did, docs such as these helped me and may prove useful for you:
Ansible
CloudFormation
This is a little late for the original poster, but may be of use to others: The Azure implementation of the YAML parser does support conditional checks. For example, the following YAML in an azure-pipeline.yml:
#azure-pipeline.yml
parameters:
- name: testCondition
displayName: 'Use experimental build process?'
type: boolean
default: true
steps:
- ${{ if eq(parameters.testCondition, true) }}:
- bash: echo 'true'
- ${{ if not(eq(parameters.testCondition, true)) }}:
- bash: echo 'false'
will evaluate the value of the testCondition parameter. When run, the bash task will echo "true", something like this:

How to write a file that is both valid ruby syntax and valid YAML syntax

In order to have only a single point of configuration for my app I need to make a YAML config file that is also valid ruby code. I.e. a mixed syntax file that can be parsed as YAML and parsed as ruby.
My application is a suite of processes managed by the god gem. I want to load a new group of maintained processes (watches) for each new configuration file.
God allows loading a new app.god (ruby) file with new watches defined, but I don't want an app.god and app.yml, just one file. Simplest might be to just have the app.god file and include the configuration within that, but I preferred the idea of a yml file that was also valid ruby code.
#I found this that might be helpful:
#This is a valid ruby and a valid YAML file
#Comments are the same in YAML and ruby
true ?true:
- <<YAML.to_i
# At this point in ruby it is the contents of a here doc (that will be
# converted to an integer and negated if true happens not to be true)
# In YAML it is a hash with the first entry having key "true ?true"
# representing a list containing the string "- <<YAML.to_i"
# If your YAML object should be a list not a hash you could remove the first line
any_valid_yaml: from here
a_list:
- or
- anything
- really
#Then mark the end of the YAML document with
---
#And YAML is done and ignores anything from here on
#Next terminate the ruby here document
YAML
#Now we're in ruby world
#this = "pure ruby"
def anything(ruby)
"here"
end

Resources