Hey this is my first post and I´m an complete ruby noob.
This is my existing source-code for my Dashing/Roo Project.
require 'roo-xls'
SCHEDULER.every '10m' do
file_path = "/home/numbers.xlsx"
def fetch_spreadsheet_data(path)
s = Roo::Excelx.new(path)
#This should be edited
send_event('Department1', { value:s.cell('C',5,s.sheets[0]) })
end
#Checker if file has been modified
module Handler
def file_modified
fetch_spreadsheet_data(path)
end
end
fetch_spreadsheet_data(file_path)
end
I want to add a few Departments (for example Department1, Factory2 ....)
For Department1 it should use: 'C',1,s.sheets[0]; For Factory2 it should use: 'C',2,s.sheets[0] and so on.
I want to save the names into an array and then iterate through it.
So how could I implement this logic?
thanks a lot
Related
I am writing a class that takes a CSV files, transforms it, and then writes the new data out.
module Transformer
class Base
def initialize(file)
#file = file
end
def original_data(&block)
opts = { headers: true }
CSV.open(file, 'rb', opts, &block)
end
def transformer
# complex manipulations here like modifying columns, picking only certain
# columns to put into new_data, etc but simplified to `+10` to keep
# example concise
-> { |row| new_data << row['some_header'] + 10 }
end
def transformed_data
self.original_data(self.transformer)
end
def write_new_data
CSV.open('new_file.csv', 'wb', opts) do |new_data|
transformed_data
end
end
end
end
What I'd like to be able to do is:
Look at the transformed data without writing it out (so I can test that it transforms the data correctly, and I don't need to write it to file right away: maybe I want to do more manipulation before writing it out)
Don't slurp all the file at once, so it works no matter the size of the original data
Have this as a base class with an empty transformer so that instances only need to implement their own transformers but the behavior for reading and writing is given by the base class.
But obviously the above doesn't work because I don't really have a reference to new_data in transformer.
How could I achieve this elegantly?
I can recommend one of two approaches, depending on your needs and personal taste.
I have intentionally distilled the code to just its bare minimum (without your wrapping class), for clarity.
1. Simple read-modify-write loop
Since you do not want to slurp the file, use CSV::Foreach. For example, for a quick debugging session, do:
CSV.foreach "source.csv", headers: true do |row|
row["name"] = row["name"].upcase
row["new column"] = "new value"
p row
end
And if you wish to write to file during that same iteration:
require 'csv'
csv_options = { headers: true }
# Open the target file for writing
CSV.open("target.csv", "wb") do |target|
# Add a header
target << %w[new header column names]
# Iterate over the source CSV rows
CSV.foreach "source.csv", **csv_options do |row|
# Mutate and add columns
row["name"] = row["name"].upcase
row["new column"] = "new value"
# Push the new row to the target file
target << row
end
end
2. Using CSV::Converters
There is a built in functionality that might be helpful - CSV::Converters - (see the :converters definition in the CSV::New documentation)
require 'csv'
# Register a converter in the options hash
csv_options = { headers: true, converters: [:stripper] }
# Define a converter
CSV::Converters[:stripper] = lambda do |value, field|
value ? value.to_s.strip : value
end
CSV.open("target.csv", "wb") do |target|
# same as above
CSV.foreach "source.csv", **csv_options do |row|
# same as above - input data will already be converted
# you can do additional things here if needed
end
end
3. Separate input and output from your converter classes
Based on your comment, and since you want to minimize I/O and iterations, perhaps extracting the read/write operations from the responsibility of the transformers might be of interest. Something like this.
require 'csv'
class NameCapitalizer
def self.call(row)
row["name"] = row["name"].upcase
end
end
class EmailRemover
def self.call(row)
row.delete 'email'
end
end
csv_options = { headers: true }
converters = [NameCapitalizer, EmailRemover]
CSV.open("target.csv", "wb") do |target|
CSV.foreach "source.csv", **csv_options do |row|
converters.each { |c| c.call row }
target << row
end
end
Note that the above code still does not handle the header, in case it was changed. You will probably have to reserve the last row (after all transformations) and prepend its #headers to the output CSV.
There are probably plenty other ways to do it, but the CSV class in Ruby does not have the cleanest interface, so I try to keep code that deals with it as simple as I can.
I'm experimenting with a Ruby script that will add data to a Neo4j database using REST API. (Here's the tutorial with all the code if interested.)
The script works if I include the hash data structure in the initialize method but I would like to move the data into a different file so I can make changes to it separately using a different script.
I'm relatively new to Ruby. If I copy the following data structure into a separate file, is there a simple way to read it from my existing script when I call #data? I've heard one could do something with YAML or JSON (not familiar with how either work). What's the easiest way to read a file and how could I go about coding that?
#I want to copy this data into a different file and read it with my script when I call #data.
{
nodes:[
{:label=>"Person", :title=>"title_here", :name=>"name_here"}
]
}
And here is part of my code, it should be enough for the purposes of this question.
class RGraph
def initialize
#url = 'http://localhost:7474/db/data/cypher'
#If I put this hash structure into a different file, how do I make #data read that file?
#data = {
nodes:[
{:label=>"Person", :title=>"title_here", :name=>"name_here"}
]
}
end
#more code here... not relevant to question
def create_nodes
# Scan file, find each node and create it in Neo4j
#data.each do |key,value|
if key == :nodes
#data[key].each do |node| # Cycle through each node
next unless node.has_key?(:label) # Make sure this node has a label
#WE have sufficient data to create a node
label = node[:label]
attr = Hash.new
node.each do |k,v| # Hunt for additional attributes
next if k == :label # Don't create an attribute for "label"
attr[k] = v
end
create_node(label,attr)
end
end
end
end
rGraph = RGraph.new
rGraph.create_nodes
end
Given that OP said in comments "I'm not against using either of those", let's do it in YAML (which preserves the Ruby object structure best). Save it:
#data = {
nodes:[
{:label=>"Person", :title=>"title_here", :name=>"name_here"}
]
}
require 'yaml'
File.write('config.yaml', YAML.dump(#data))
This will create config.yaml:
---
:nodes:
- :label: Person
:title: title_here
:name: name_here
If you read it in, you get exactly what you saved:
require 'yaml'
#data = YAML.load(File.read('config.yaml'))
puts #data.inspect
# => {:nodes=>[{:label=>"Person", :title=>"title_here", :name=>"name_here"}]}
def test_post_with_file filename = 'test01.xml'
File.open(filename) do |file|
response = #http_client.post(url, {'documents'=>file})
end
end
How do I modify the above method to handle a multi-file-array post/upload?
file_array = ['test01.xml', 'test02.xml']
You mean like this?
def test_post_with_file(file_array=[])
file_array.each do |filename|
File.open(filename) do |file|
response = #http_client.post(url, {'documents'=>file})
end
end
end
I was having the same problem and finally figured out how to do it:
def test_post_with_file(file_array)
form = file_array.map { |n| ['documents[]', File.open(n)] }
response = #http_client.post(#url, form)
end
You can see in the docs how to pass multiple values: http://rubydoc.info/gems/httpclient/HTTPClient#post_content-instance_method .
In the "body" row, I tried without success to use the 4th example. Somehow HttpClient just decides to apply .to_s to each hash in the array.
Then I tried the 2nd solution and it wouldn't work either because only the last value is kept by the server. But I discovered after some tinkering that the second solution works if the parameter name includes the square brackets to indicate there are mutiple values as an array.
Maybe this is a bug in Sinatra (that's what I'm using), maybe the handling of such data is implementation-dependent, maybe the HttpClient doc is outdated/wrong. Or a combination of these.
class Desktop
# grab the names of items on the desktop
desktop_content = Dir.entries("/Users/jorgeolivero/desktop")
def initialize(desktop_content)
#desktop_content = desktop_content
end
def initlialize(select)
arr.select { |x| x.include? "y"}
end
def initialize(desktop_files)
#desktop_files = desktop_files
end
# grabs the files from the desktop_content array
desktop_files = desktop_content.select { |c| c.include? '.'}
puts desktop_files
desktop_files.each { |files| File.rename('/Users/jorgeolivero/desktop/***feed array items here***', '/Users/jorgeolivero/desktop/folder/***feed array items here***') }
end
My guts tells me that this might look painful to some of you, so please know this is my first attempt at an app. So if you can remember that process, you understand how "stuck" I am right now.
I want to move all of the items in the "desktop_files" array into a folder with one command. That last piece of code shows how I am trying to do but I can't figure out how to feed each file name onto the end of each path (as denoted by the ***s).
Many thanks.
You can use the following code for copy all your files:
require 'fileutils'
FileUtils.mv Dir['/Users/jorgeolivero/desktop/*'], 'dest folder'
You don't need to create a class for it. You can run this code on irb or put in a single file.
PS: Ruby doesn't accept multiple initializers for a class.
I have gone over the documentation, and I can't find a specific way to go about this. I have already added some dynamic attributes to a model, and I would like to be able to iterate over all of them.
So, for a concrete example:
class Order
include Mongoid::Document
field :status, type: String, default: "pending"
end
And then I do the following:
Order.new(status: "processed", internal_id: "1111")
And later I want to come back and be able to get a list/array of all the dynamic attributes (in this case, "internal_id" is it).
I'm still digging, but I'd love to hear if anyone else has solved this already.
Just include something like this in your model:
module DynamicAttributeSupport
def self.included(base)
base.send :include, InstanceMethods
end
module InstanceMethods
def dynamic_attributes
attributes.keys - _protected_attributes[:default].to_a - fields.keys
end
def static_attributes
fields.keys - dynamic_attributes
end
end
end
and here is a spec to go with it:
require 'spec_helper'
describe "dynamic attributes" do
class DynamicAttributeModel
include Mongoid::Document
include DynamicAttributeSupport
field :defined_field, type: String
end
it "provides dynamic_attribute helper" do
d = DynamicAttributeModel.new(age: 45, defined_field: 'George')
d.dynamic_attributes.should == ['age']
end
it "has static attributes" do
d = DynamicAttributeModel.new(foo: 'bar')
d.static_attributes.should include('defined_field')
d.static_attributes.should_not include('foo')
end
it "allows creation with dynamic attributes" do
d = DynamicAttributeModel.create(age: 99, blood_type: 'A')
d = DynamicAttributeModel.find(d.id)
d.age.should == 99
d.blood_type.should == 'A'
d.dynamic_attributes.should == ['age', 'blood_type']
end
end
this will give you only the dynamic field names for a given record x:
dynamic_attribute_names = x.attributes.keys - x.fields.keys
if you use additional Mongoid features, you need to subtract the fields associated with those features:
e.g. for Mongoid::Versioning :
dynamic_attribute_names = (x.attributes.keys - x.fields.keys) - ['versions']
To get the key/value pairs for only the dynamic attributes:
make sure to clone the result of attributes(), otherwise you modify x !!
attr_hash = x.attributes.clone #### make sure to clone this, otherwise you modify x !!
dyn_attr_hash = attr_hash.delete_if{|k,v| ! dynamic_attribute_names.include?(k)}
or in one line:
x.attributes.clone.delete_if{|k,v| ! dynamic_attribute_names.include?(k)}
So, what I ended up doing is this. I'm not sure if it's the best way to go about it, but it seems to give me the results I'm looking for.
class Order
def dynamic_attributes
self.attributes.delete_if { |attribute|
self.fields.keys.member? attribute
}
end
end
Attributes appears to be a list of the actual attributes on the object, while fields appears to be a hash of the fields that were predefined. Couldn't exactly find that in the documentation, but I'm going with it for now unless someone else knows of a better way!
try .methods or .instance_variables
Not sure if I liked the clone approach, so I wrote one too. From this you could easily build a hash of the content too. This merely outputs it all the dynamic fields (flat structure)
(d.attributes.keys - d.fields.keys).each {|a| puts "#{a} = #{d[a]}"};
I wasn't able to get any of the above solutions to work (as I didn't want to have to add slabs and slabs of code to each model, and, for some reason, the attributes method does not exist on a model instance, for me. :/), so I decided to write my own helper to do this for me. Please note that this method includes both dynamic and predefined fields.
helpers/mongoid_attribute_helper.rb:
module MongoidAttributeHelper
def self.included(base)
base.extend(AttributeMethods)
end
module AttributeMethods
def get_all_attributes
map = %Q{
function() {
for(var key in this)
{
emit(key, null);
}
}
}
reduce = %Q{
function(key, value) {
return null;
}
}
hashedResults = self.map_reduce(map, reduce).out(inline: true) # Returns an array of Hashes (i.e. {"_id"=>"EmailAddress", "value"=>nil} )
# Build an array of just the "_id"s.
results = Array.new
hashedResults.each do |value|
results << value["_id"]
end
return results
end
end
end
models/user.rb:
class User
include Mongoid::Document
include MongoidAttributeHelper
...
end
Once I've added the aforementioned include (include MongoidAttributeHelper) to each model which I would like to use this method with, I can get a list of all fields using User.get_all_attributes.
Granted, this may not be the most efficient or elegant of methods, but it definitely works. :)