I inherited a tool that is working correctly but when I try to extend it it just fails. Since I am new to ruby and yaml I dont really know what is the reason why this fails...
So I have a class config that looks like this
class Configuration
def self.[] key
##config[key]
end
def self.load name
##config = nil
io = File.open( File.dirname(__FILE__) + "/../../../config/config.yml" )
YAML::load_documents(io) { |doc| ##config = doc[name] }
raise "Could not locate a configuration named \"#{name}\"" unless ##config
end
def self.[]=key, value
##config[key] = value
end
end
end
raise "Please set the A environment variable" unless ENV['A']
Helpers::Configuration.load(ENV['A'])
raise "Please set the D environment variable" unless ENV['D']
Helpers::Configuration.load(ENV['D'])
raise "Please set the P environment variable" unless ENV['P']
Helpers::Configuration.load(ENV['P'])
So I had a first version with the environment variable A that worked fine, then when I want to integrate 2 more environment variables it fails (they are different key/value sets). I did debug it and it looks like when it reads the second key/value it removes the other ones (such as reading the 3rd removes the previous 2, so I end up with ##config with only the 3rd key/value par instead of all the values I need).
It is probably easy to fix this, any idea how?
Thanks!
EDIT:
The config file use to look like:
Test:
position_x: “56”
position_y: “56”
Now I want to make it like
“x56”:
position_x: “56”
“x15”:
position_x: “15”
“y56”:
position_y: “56”
“y15”:
position_y: “15”
My idea is that I set them separately and I don’t need to create all the combinations…
Each time you call load you delete the previous configuration (in the line ##config = nil). If you want the configuration to be a merger of all files you will want to merge the new configuration to the existing configuration rather than overriding it.
Something like this:
def self.load name
##config ||= {}
io = File.open( File.dirname(__FILE__) + "/../../../config/config.yml" )
YAML::load_documents(io) do |doc|
raise "Could not locate a configuration named \"#{name}\"" unless doc[name]
##config.merge!(doc[name])
end
end
Be aware that if the code was written as it has been because the method was called more than once, and the configuration is expected to reset between reads, you will need to explicitly reset the configuration now:
class Configuration
# ...
def reset_configuration
#config = {}
end
end
Helpers::Configuration.reset_configuration
raise "Please set the A environment variable" unless ENV['A']
Helpers::Configuration.load(ENV['A'])
raise "Please set the D environment variable" unless ENV['D']
Helpers::Configuration.load(ENV['D'])
raise "Please set the P environment variable" unless ENV['P']
Helpers::Configuration.load(ENV['P'])
I'd access the YAML using:
YAML::load_file(File.expand_path("../../../config/config.yml", File.dirname(__FILE__)))
expand_path cleans up the '..' chain and returns the cleaned-up version, relative to FILE. For instance:
foo = '/path/to/a/file'
File.expand_path("../config.yml", File.dirname(foo)) # => "/path/to/config.yml"
load_file reads and parses the entire file and returns it.
Related
I am trying to create a test for a FileProcessor that reads from a text file, passes it to another class and then writes output. I made a test file and am able to access but it feels bulky. I'm also going to need to test that it writes the output in a new file and I am not sure how to set this up. I've seen a lot of tutorials but they are be rails centric. My goal is to get rid of writing the path in the test and to clean up the generated output files after each test.
describe FileProcessor do
test_file = File.dirname(__FILE__) + '/fixtures/test_input.txt'
output_file = File.dirname(__FILE__) + '/fixtures/test_output.txt'
subject {FileProcessor.new(test_file, output_file)}
describe '#read_file' do
it 'reads a file' do
expect(subject.read_file).to eq('This is a test.')
end
end
def write_file(str)
File.open("#{output_file}", "w+") { |file| file.write(str) }
end
end
How about using StringIO:
require 'stringio'
class FileProcessor
def initialize(infile, outfile)
#infile = infile
#outfile = outfile
#content = nil
end
def read_file
#content ||= #infile.read
end
def write_file(text)
#outfile.write(text)
end
end
describe FileProcessor do
let(:outfile) { StringIO.new }
subject(:file_processor) do
infile = StringIO.new('This is a test')
FileProcessor.new(infile, outfile)
end
describe '#read_file' do
it "returns correct text" do
expect(file_processor.read_file).to eq("This is a test")
end
end
describe '#write_file' do
it "writes correct text" do
file_processor.write_file("Hello world")
outfile.rewind
expect(outfile.read).to eq("Hello world")
end
end
end
There's not a great way to avoid writing the path of your input file. You could move that into a helper method, but on the other hand having the path in the test has the benefit that someone else (or you six months from now) looking at the code will know immediately where the test data comes from.
As for the output file, the simplest solution is to use Ruby's built-in Tempfile class. Tempfile.new is like File.new, except that it automatically puts the file in /tmp (or wherever your OS's temporary file directory is) and gives it a unique name. This way you don't have to worry about cleaning it up, because the next time you run the test it'll use a file with a different name (and your OS will automatically delete the file). For example:
require 'tempfile'
describe FileProcessor do
let(:test_file_path) { File.dirname(__FILE__) + '/fixtures/test_input.txt' }
let(:output_file) { Tempfile.new('test_output.txt').path }
subject { FileProcessor.new(test_file_path, output_file.path) }
describe '#read_file' do
it 'reads a file' do
expect(subject.read_file).to eq('This is a test.')
end
end
end
Using let (instead of just assigning a local variable) ensures that each example will use its own unique output file. In RSpec you should almost always prefer let.
If you want to get really serious, you could instead use the FakeFS gem, which mocks all of Ruby's built-in file-related classes (File, Pathname, etc.) so you're never writing to your actual filesystem. Here's a quick tutorial on using FakeFS: http://www.bignerdranch.com/blog/fake-it/
The need here is to read a json file and to make the variables which is done from one class and use them with in another class. What I have so far is
helper.rb
class MAGEINSTALLER_Helper
#note nonrelated items removed
require 'fileutils'
#REFACTOR THIS LATER
def load_settings()
require 'json'
file = File.open("scripts/installer_settings.json", "rb")
contents = file.read
file.close
#note this should be changed for a better content check.. ie:valid json
#so it's a hack for now
if contents.length > 5
begin
parsed = JSON.parse(contents)
rescue SystemCallError
puts "must redo the settings file"
else
puts parsed['bs_mode']
parsed.each do |key, value|
puts "#{key}=>#{value}"
instance_variable_set("#" + key, value) #better way?
end
end
else
puts "must redo the settings file"
end
end
#a method to provide feedback simply
def download(from,to)
puts "completed download for #{from}\n"
end
end
Which is called in a file of Pre_start.rb
class Pre_start
#note nonrelated items removed
def initialize(params=nil)
puts 'World'
mi_h = MAGEINSTALLER_Helper.new
mi_h.load_settings()
bs_MAGEversion=instance_variable_get("#bs_MAGEversion") #doesn't seem to work
file="www/depo/newfile-#{bs_MAGEversion}.tar.gz"
if !File.exist?(file)
mi_h.download("http://www.dom.com/#{bs_MAGEversion}/file-#{bs_MAGEversion}.tar.gz",file)
else
puts "mage package exists"
end
end
end
the josn file is valid json and is a simple object (note there is more just showing the relevant)
{
"bs_mode":"lite",
"bs_MAGEversion":"1.8.0.0"
}
The reason I need to have a json settings file is that I will need to pull settings from a bash script and later a php script. This file is the common thread that is used to pass settings each share and need to match.
Right now I end up with an empty string for the value.
The instance_variable_setis creating the variable inside MAGEINSTALLER_Helper class. That's the reason why you can't access these variables.
You can refactor it into a module, like this:
require 'fileutils'
require 'json'
module MAGEINSTALLER_Helper
#note nonrelated items removed
#REFACTOR THIS LATER
def load_settings()
content = begin
JSON.load_file('scripts/installer_settings.json')
rescue
puts 'must redo the settings file'
{} # return an empty Hash object
end
parsed.each {|key, value| instance_variable_set("##{key}", value)}
end
#a method to provide feedback simply
def download(from,to)
puts "completed download for #{from}\n"
end
end
class PreStart
include MAGEINSTALLER_Helper
#note nonrelated items removed
def initialize(params=nil)
puts 'World'
load_settings # The method is available inside the class
file="www/depo/newfile-#{#bs_MAGEversion}.tar.gz"
if !File.exist?(file)
download("http://www.dom.com/#{#bs_MAGEversion}/file-#{#bs_MAGEversion}.tar.gz",file)
else
puts "mage package exists"
end
end
end
I refactored a little bit to more Rubish style.
On this line:
bs_MAGEversion=instance_variable_get("#bs_MAGEversion") #doesn't seem to work
instance_variable_get isn't retrieving from the mi_h Object, which is where your value is stored. The way you've used it, that line is equivalent to:
bs_MAGEversion=#bs_MAGEversion
Changing it to mi_h.instance_variable_get would work. It would also be painfully ugly ruby. But I sense that's not quite what you're after. If I read you correctly, you want this line:
mi_h.load_settings()
to populate #bs_MAGEversion and #bs_mode in your Pre_start object. Ruby doesn't quite work that way. The closest thing to what you're looking for here would probably be a mixin, as described here:
http://www.ruby-doc.org/docs/ProgrammingRuby/html/tut_modules.html
We do something similar to this all the time in code at work. The problem, and solution, is proper use of variables and scoping in the main level of your code. We use YAML, you're using JSON, but the idea is the same.
Typically we define a constant, like CONFIG, which we load the YAML into, in our main code, and which is then available in all the code we require. For you, using JSON instead:
require 'json'
require_relative 'helper'
CONFIG = JSON.load_file('path/to/json')
At this point CONFIG would be available to the top-level code and in "helper.rb" code.
As an alternate way of doing it, just load your JSON in either file. The load-time is negligible and it'll still be the same data.
Since the JSON data should be static for the run-time of the program, it's OK to use it in a CONSTANT. Storing it in an instance variable only makes sense if the data would vary from instance to instance of the code, which makes no sense when you're loading data from a JSON or YAML-type file.
Also, notice that I'm using a method from the JSON class. Don't go through the rigamarole you're using to try to copy the JSON into the instance variable.
Stripping your code down as an example:
require 'fileutils'
require 'json'
CONTENTS = JSON.load_file('scripts/installer_settings.json')
class MAGEINSTALLER_Helper
def download(from,to)
puts "completed download for #{from}\n"
end
end
class Pre_start
def initialize(params=nil)
file = "www/depo/newfile-#{ CONFIG['bs_MAGEversion'] }.tar.gz"
if !File.exist?(file)
mi_h.download("http://www.dom.com/#{ CONFIG['bs_MAGEversion'] }/file-#{ CONFIG['bs_MAGEversion'] }.tar.gz", file)
else
puts "mage package exists"
end
end
end
CONFIG can be initialized/loaded in either file, just do it from the top-level before you need to access the contents.
Remember, Ruby starts executing it at the top of the first file and reads downward. Code that is outside of def, class and module blocks gets executed as it's encountered, so the CONFIG initialization will happen as soon as Ruby sees that code. If that happens before you start calling your methods and creating instances of classes then your code will be happy.
What's the best way to do a little DRY within a chef recipe? I.e. just break out little bits of the Ruby code, so I'm not copying pasting it over and over again.
The following fails of course, with:
NoMethodError: undefined method `connect_root' for Chef::Resource::RubyBlock
I may have multiple ruby_blocks in one recipe, as they do different things and need to have different not_if blocks to be truley idempotent.
def connect_root(root_password)
m = Mysql.new("localhost", "root", root_password)
begin
yield m
ensure
m.close
end
end
ruby_block "set readonly" do
block do
connect_root node[:mysql][:server_root_password] do |connection|
command = 'SET GLOBAL read_only = ON'
Chef::Log.info "#{command}"
connection.query(command)
end
end
not_if do
ro = nil
connect_root node[:mysql][:server_root_password] do |connection|
connection.query("SELECT ##read_only as ro") {|r| r.each_hash {|h|
ro = h['ro']
} }
end
ro
end
end
As you already figured out, you cannot define functions in recipes. For that libraries are provided. You should create a file (e.g. mysql_helper.rb ) inside libraries folder in your cookbook with the following:
module MysqlHelper
def self.connect_root( root_password )
m = Mysql.new("localhost", "root", root_password)
begin
yield m
ensure
m.close
end
end
end
It must be a module, not a class. Notice also we define it as static (using self.method_name). Then you will be able to use functions defined in this module in your recipes using module name with method name:
MysqlHelper.connect_root node[:mysql][:server_root_password] do |connection|
[...]
end
For the record, I just created a library with the following. But that seems overkill for DRY within one file. I also couldn't figure out how to get any other namespace for the module to use, to work.
class Chef
class Resource
def connect_root(root_password)
...
Let me preface by stating I'm a "new" programmer - an IT guy trying his hand at his first "real" problem after working through various tutorials.
So - here is what I'm trying to do. I'm watching a directory for a .csv file - it will be in this format: 999999_888_filename.csv
I want to return each part of the "_" filename as a variable to pass on to another program/script for some other task. I have come up w/ the following code:
require 'rubygems'
require 'fssm'
class Watcher
def start
monitor = FSSM::Monitor.new(:directories => true)
monitor.path('/data/testing/uploads') do |path|
path.update do |base, relative, ftype|
output(relative)
end
path.create do |base, relative, ftype|
output(relative)
end
path.delete { |base, relative, ftype| puts "DELETED #{relative} (#{ftype})" }
end
monitor.run
end
def output(relative)
puts "#{relative} added"
values = relative.split('_',)
sitenum = values[0]
numrecs = values[1]
filename = values[2]
puts sitenum
end
end
My first "puts" gives me the full filename (it's just there to show me the script is working), and the second puts returns the 'sitenum'. I want to be able to access this "outside" of this output method. I have this file (named watcher.rb) in a libs/ folder and I have a second file in the project root called 'monitor.rb' which contains simply:
require './lib/watcher'
watcher = Watcher.new
watcher.start
And I can't figure out how to access my 'sitenum', 'numrecs' and 'filename' from this file. I'm not sure if it needs to be a variable, instance variable or what. I've played around w/ attr_accessible and other things, and nothing works. I decided to ask here since I've been spinning my wheels for a couple of things, and I'm starting to confuse myself by searching on my own.
Thanks in advance for any help or advice you may have.
At the top of the Watcher class, you're going to want to define three attr_accessor declarations, which give the behavior you want. (attr_reader if you're only reading, attr_writer if you're only writing, attr_accessor if both.)
class Watcher
attr_accessor :sitenum, :numrecs, :filename
...
# later on, use # for class variables
...
#sitenum = 5
...
end
Now you should have no problem with watcher.sitenum etc. Here's an example.
EDIT: Some typos.
In addition to Jordan Scales' answer, these variable should initialized
class Watcher
attr_accessor :sitenum, :numrecs, :filename
def initialize
#sitenum = 'default value'
#numrecs = 'default value'
#filename = 'default value'
end
...
end
Otherwise you'll get uninformative value nil
How this simple task can be done in Ruby?
I have some simple config file
=== config.rb
config = { 'var' => 'val' }
I want to load config file from some method, defined in main.rb file so that the local variables from config.rb became local vars of that method.
Something like this:
=== main.rb
Class App
def loader
load('config.rb') # or smth like that
p config['var'] # => "val"
end
end
I know that i can use global vars in config.rb and then undefine them when done, but i hope there's a ruby way )
The config file.
{ 'var' => 'val' }
Loading the config file
class App
def loader
config = eval(File.open(File.expand_path('~/config.rb')).read)
p config['var']
end
end
As others said, for configuration it's better to use YAML or JSON. To eval a file
binding.eval(File.open(File.expand_path('~/config.rb')).read, "config.rb")
binding.eval(File.read(File.expand_path('~/config.rb')), "config.rb")
This syntax would allow you to see filename in backtraces which is important. See api docs [1].
Updated eval command to avoid FD (file descriptor) leaks. I must have been sleeping or maybe should have been sleeping at that time of the night instead of writing on stackoverflow..
[1] http://www.ruby-doc.org/core-1.9.3/Binding.html
You certainly could hack out a solution using eval and File.read, but the fact this is hard should give you a signal that this is not a ruby-like way to solve the problem you have. Two alternative designs would be using yaml for your config api, or defining a simple dsl.
The YAML case is the easiest, you'd simply have something like this in main.rb:
Class App
def loader
config = YAML.load('config.yml')
p config['var'] # => "val"
end
end
and your config file would look like:
---
var: val
I do NOT recommend doing this except in a controlled environment.
Save a module to a file with a predetermined name that defines an initialize and run_it methods. For this example I used test.rb as the filename:
module Test
##classvar = 'Hello'
def initialize
#who = 'me'
end
def get_who
#who
end
def run_it
print "#{##classvar} #{get_who()}"
end
end
Then write a simple app to load and execute it:
require 'test'
class Foo
include Test
end
END {
Foo.new.run_it
}
# >> Hello me
Just because you can do something doesn't mean you should. I cannot think of a reason I'd do it in production and only show it here as a curiosity and proof-of-concept. Making this available to unknown people would be a good way to get your machine hacked because the code could do anything the owning account could do.
I just had to do a similar thing as I wanted to be able to load a "Ruby DLL" where it returns an anonymous class ( a factory for instances of things ) I created this which keeps track of items already loaded and allows the loaded file to return a value which can be anything - a totally anonymous Class, Module, data etc. It could be a module which you could then "include" in an object after it is loaded and it could could supply a host of "attributes" or methods. you could also add an "unload" item to clear it from the loaded hash and dereference any object it loaded.
module LoadableModule
##loadedByFile_ = {};
def self.load(fileName)
fileName = File.expand_path(fileName);
mod = ##loadedByFile_[fileName];
return mod if mod;
begin
Thread.current[:loadReturn] = nil;
Kernel.load(fileName);
mod = Thread.current[:loadReturn];
##loadedByFile_[fileName] = mod if(mod);
rescue => e
puts(e);
puts(e.backtrace);
mod = nil;
end
Thread.current[:loadReturn] = nil;
mod
end
def self.onLoaded(retVal)
Thread.current[:loadReturn] = retVal;
end
end
inside the loaded file:
LoadableModule.onLoaded("a value to return from the loaded file");