I have Ruby using Psych that writes YAML like this:
---
- !ruby/struct:Dhc::QueryEvent
time: 2018-01-02T07:45:18.470Z
hits: 2525
qtime: 13
server: search1
q: search term
searched_in:
- productA
- productB
- productC
Is there a way I can write the YAML without the "!ruby/struct:Dhc::QueryEvent" tag? I want the storage formatted so it is not dependent on my struct:Dhc::QueryEvent for reading. In other words, I want others to have the flexibility to read the YAML however they want. Currently, they get
undefined class/module Dhc:: (ArgumentError)
Then, when I'm reading the file myself, is there a configuration in Psych that allows me to read the untagged records into my Dhc::QueryEvent struct? I assume this would be something like
ar = []
Psych.parse_stream(yaml_stream) do |node|
ar << Dhc::QueryEvent.new(node.time, node.hits, etc...)
end
but maybe there is a shorter way?
Related
I would like to get the name of a key in a YAML configuration file like the following
The YAML file contains a collection of structs in structs, the organisations have a number represented as a symbol which is the key/class name. I need to retrieve this symbol
require 'yaml'
data = YAML.load(DATA)
data.organisations.each do |organisation|
organisation #<struct language="nl", name="myname">
# following line is what I need, I expect it to be :"121"
organisation.class #<Class:0x00000004fd4248>
end
__END__
--- !ruby/struct
organisations: !ruby/struct
:121: !ruby/struct
language: nl
name: organisationname
Can someone help me retrieving the name of the struct ?
I tried around a bit and found this:
data.organisations.members
=> [:"121"]
The way I found this (which is useful in other circumstances, too) is the following:
data.organisations.methods - Object.new.methods
And then I tried out every method that seemed reasonable.
I'm pretty new to ruby and all the documentation on this subject has confused me a bit. So here goes.
I'm using inspec to test my infrastructure, but I want it to consume some variables from the YAML file used by ansible. This effectively means I can share vars from ansible code and use them in ruby.
The YAML file looks like this:
- name: Converge
hosts: all
vars:
elasticsearch_config:
cluster.name: "{{ elasticsearch_cluster_name }}"
node.name: "es-test-node"
path.data: "/var/lib/elasticsearch"
path.logs: "/var/log/elasticsearch"
elasticsearch_cluster_name: test
pre_tasks:
roles:
- elasticsearch
post_tasks:
At this point, I'm just playing around with ruby code to extract that, and have:
require 'yaml'
parsed = begin
YAML.load(File.open("../../playbook.yml"))
rescue ArgumentError => e
puts "Could not parse YAML: #{e.message}"
end
puts parsed
Which outputs the hash:
{"name"=>"Converge", "hosts"=>"all", "vars"=>{"elasticsearch_config"=>{"cluster.name"=>"{{ elasticsearch_cluster_name }}", "node.name"=>"es-test-node", "path.data"=>"/var/lib/elasticsearch", "path.logs"=>"/var/log/elasticsearch"}, "elasticsearch_cluster_name"=>"test"}, "pre_tasks"=>nil, "roles"=>["elasticsearch"], "post_tasks"=>nil}
So far so good. This all makes sense to me. Now, I would like to pull values out of this data and use them in the ruby code, referencing them by the keys. So, if I wanted to get the value of vars.elasticsearch_config.node.name, how would I go about doing this?
YAML.load reads the document into an array, so you must get the first element in your example:
loaded_yaml[0]["vars"]["elasticsearch_config"]["node.name"]
The reason for this is that the document you are parsing begins with a single dash, indicating a list item. Even though there is only one item in the list, Psych (thy YAML engine) is still placing it into an array representing a list. This is also why you got a no implicit conversion of String to Integer error. Note that the response you get is enclosed by square brackets:
=> [{"name"=>"Converge", "hosts"=>"all", "vars"=>{"elasticsearch_config"=>{"cluster.name"=>"{{ elasticsearch_cluster_name }}", "node.name"=>"es-test-node", "path.data"=>"/var/lib/elasticsearch", "path.logs"=>"/var/log/elasticsearch"}, "elasticsearch_cluster_name"=>"test"}, "pre_tasks"=>nil, "roles"=>["elasticsearch"], "post_tasks"=>nil}]
In order to have only a single point of configuration for my app I need to make a YAML config file that is also valid ruby code. I.e. a mixed syntax file that can be parsed as YAML and parsed as ruby.
My application is a suite of processes managed by the god gem. I want to load a new group of maintained processes (watches) for each new configuration file.
God allows loading a new app.god (ruby) file with new watches defined, but I don't want an app.god and app.yml, just one file. Simplest might be to just have the app.god file and include the configuration within that, but I preferred the idea of a yml file that was also valid ruby code.
#I found this that might be helpful:
#This is a valid ruby and a valid YAML file
#Comments are the same in YAML and ruby
true ?true:
- <<YAML.to_i
# At this point in ruby it is the contents of a here doc (that will be
# converted to an integer and negated if true happens not to be true)
# In YAML it is a hash with the first entry having key "true ?true"
# representing a list containing the string "- <<YAML.to_i"
# If your YAML object should be a list not a hash you could remove the first line
any_valid_yaml: from here
a_list:
- or
- anything
- really
#Then mark the end of the YAML document with
---
#And YAML is done and ignores anything from here on
#Next terminate the ruby here document
YAML
#Now we're in ruby world
#this = "pure ruby"
def anything(ruby)
"here"
end
I want to replace properties in one file from those in another. (I am new to ruby, and read about Ruby and YAML. I have a Java background)
Eg.
File 1
server_ip_address=$[ip]
value_threshold=$[threshold]
system_name=$[sys_name]
File 2
ip=192.168.1.1
threshold=10
sys_name=foo
The ruby script should replace the $ values by their real values (I do not know if $[] is the format used in ruby. Also do Files 1 and 2 have to be YAML files, or erb files?) and produce File 1 as :
server_ip_address=192.168.1.1
value_threshold=10
system_name=foo
I searched the web for this, but could not express it in the right keywords to find a solution/pointer to a solution/reference material on google. How can this be done by a ruby script?
Thanks
If you can switch the formats, this should be as easy as:
require 'yaml'
variables = YAML.load(File.open('file2.yaml'))
template = File.read('file1.conf')
puts template.gsub(/\$\[(\w+)\]/) { variables[$1] }
Your template can stay as-is, but the substitution file would look like:
ip: 192.168.1.1
threshold: 10
sys_name: foo
This makes it easy to read in using the YAML library.
so I saved to disk some objects using the following code (this is Ruby 1.9.2 on Windows BTW):
open('1.txt', "wb") { |file|
file.write(YAML::dump( results))
}
Now I'm trying to get back that data, but get 'invalid byte sequence in UTF-8 (ArgumentError)'. I've tryed everything I could think of to save the data in different format, but no luck. For example
open('1.txt', 'rb'){|f| a1 = YAML::load(f.read)}
a1.each do |a|
JSON.generate(a)
end
results in:
C:/m/ruby-1.9.2-p0-i386-mingw32/lib/ruby/1.9.1/json/common.rb:212:in `match':
invalid byte sequence
in UTF-8 (ArgumentError)
from C:/m/ruby-1.9.2-p0-i386-mingw32/lib/ruby/1.9.1/json/common.rb:212:in `generate'
from C:/m/ruby-1.9.2-p0-i386-mingw32/lib/ruby/1.9.1/json/common.rb:212:in `generate'
from merge3.rb:31:in `block in <main>'
from merge3.rb:29:in `each'
from merge3.rb:29:in `<main>'
What can I do?
EDIT: from the file:
---
- !ruby/object:Product
name: HSF
- !ruby/object:Product
name: "almer\xA2n"
The 1st product works OK, but the 2nd gives the exception.
This is probably your encoding being wrong. You could try this:
Encoding.default_external = 'BINARY'
This should read in the file raw, not interpreted as UTF-8. You are presumably using some kind of ISO-8859-1 accent.
You need to read the file back in using the same encoding you used to write it out, obviously. Since you don't specify an encoding in either case, you will basically end up with an environment-dependent encoding outside of your control, which is why it is never a good idea to not specify an encoding.
The snippet you posted is clearly not valid UTF-8, so the fact that you get an exception is perfectly appropriate.
I'm not sure if this is what you're after, but currently your YAML file looks like:
---
- !ruby/object:Product
name: HSF
- !ruby/object:Product
name: "almer\xA2n"
If you remove the !ruby/object:Product from the array lines you'll get an array of hashes:
---
- name: HSF
- name: "almer\xA2n"
results in:
YAML::load_file('test.yaml') #=> [{"name"=>"HSF"}, {"name"=>"almer\xA2n"}]
If I print the second element's value when my terminal is set to Windows character sets I see the cent sign. So, if you're trying to regain access to the data all you have to do is a bit of manipulation of the data file.