I use a pipeline in Logstash that have a config file of Ruby codes.
In this file, I scan all logs in that pipeline with word table.
I follow this structure as below:
def filter(event)
red = Redis.new(host: "127.0.0.1", port: 6379, db: 7)
start = 0
key_collection = []
index, keys = red.scan(start)
while index != "0"
start = index
keys.each do |key|
key_collection << key if key.include?('table')
end
index, keys = red.scan(start)
end
event.set(#is_valid, key_collection)
return [event]
end
Is this way correct?
The key_collection << key if key.include?('table') does not work. How can I add all keys that include 'table*' in an array?
Related
I have a Hash which needs to be converted in a String with escaped characters.
{name: "fakename"}
and should end up like this:
'name:\'fakename\'
I don't know how this type of string is called. Maybe there is an already existing method, which I simply don't know...
At the end I would do something like this:
name = {name: "fakename"}
metadata = {}
metadata['foo'] = 'bar'
"#{name} AND #{metadata}"
which ends up in that:
'name:\'fakename\' AND metadata[\'foo\']:\'bar\''
Context: This query a requirement to search Stripe API: https://stripe.com/docs/api/customers/search
If possible I would use Stripe's gem.
In case you can't use it, this piece of code extracted from the gem should help you encode the query parameters.
require 'cgi'
# Copied from here: https://github.com/stripe/stripe-ruby/blob/a06b1477e7c28f299222de454fa387e53bfd2c66/lib/stripe/util.rb
class Util
def self.flatten_params(params, parent_key = nil)
result = []
# do not sort the final output because arrays (and arrays of hashes
# especially) can be order sensitive, but do sort incoming parameters
params.each do |key, value|
calculated_key = parent_key ? "#{parent_key}[#{key}]" : key.to_s
if value.is_a?(Hash)
result += flatten_params(value, calculated_key)
elsif value.is_a?(Array)
result += flatten_params_array(value, calculated_key)
else
result << [calculated_key, value]
end
end
result
end
def self.flatten_params_array(value, calculated_key)
result = []
value.each_with_index do |elem, i|
if elem.is_a?(Hash)
result += flatten_params(elem, "#{calculated_key}[#{i}]")
elsif elem.is_a?(Array)
result += flatten_params_array(elem, calculated_key)
else
result << ["#{calculated_key}[#{i}]", elem]
end
end
result
end
def self.url_encode(key)
CGI.escape(key.to_s).
# Don't use strict form encoding by changing the square bracket control
# characters back to their literals. This is fine by the server, and
# makes these parameter strings easier to read.
gsub("%5B", "[").gsub("%5D", "]")
end
end
params = { name: 'fakename', metadata: { foo: 'bar' } }
Util.flatten_params(params).map { |k, v| "#{Util.url_encode(k)}=#{Util.url_encode(v)}" }.join("&")
I use it now with that string, which works... Quite straigt forward:
"email:\'#{email}\'"
email = "test#test.com"
key = "foo"
value = "bar"
["email:\'#{email}\'", "metadata[\'#{key}\']:\'#{value}\'"].join(" AND ")
=> "email:'test#test.com' AND metadata['foo']:'bar'"
which is accepted by Stripe API
PStore is a data-serializer in the Stdlib which is human-readable like JSON or YAML.
Everything has to be done in a "transaction", e.g.
storage = PStore.new("db")
storage.transaction do |db|
db[:key] = {}
db[:key][:sub_key] = "val"
return db[:key][:sub_key]
end
# ==> "val"
I want to be able to make a method like this:
def store(storage, list_of_keys, value)
storage.transaction do |db|
db[list_of_keys[0][list_of_keys[1]][list_of_keys[n]] = value
end
end
Conceptually I think currying would be useful here, but I've only done it in Javascript.
It doesn't seem to work to store the references to the intermediate values in variables like this:
storage.transaction do |db|
db_val_1 = db[keys[0]]
db_val_1 = value
end
I don't know exactly what you want. But my guess is you wanna store a value inside a nested hash with given list of keys. Since it's a nested has, I would use, well, a hash obviously. Here is a full working code for what I think you are trying to do:
require 'pstore'
require 'minitest/autorun'
storage = PStore.new("data_file.pstore")
def nest(key_list, value)
reverse_key_list = key_list.reverse
inner_most = {"#{reverse_key_list.shift}" => value}
reverse_key_list.reduce(inner_most) {|acc, current| {"#{current}" => acc }}
end
def nested_store(storage, key_list, value)
root = key_list.shift
data = nest(key_list, value)
storage.transaction { storage[root] = data }
end
describe "pstore test" do
before do
#key_list = %w(root sub1 sub2 sub3)
#val = "value"
end
it "should nest the list" do
nest(#key_list, #val).must_equal({"root" => {"sub1" => {"sub2" => {"sub3" => "value" }}}})
end
it "store the nested list" do
storage = PStore.new("data_file.pstore")
nested_store(storage, #key_list, #val)
storage.transaction do
storage["root"]["sub1"]["sub2"]["sub3"].must_equal "value"
end
end
end
Part of class KeyServer
#generated_keys = Hash.new
def generate_key
key = SecureRandom.urlsafe_base64
while(purged_keys.include?(key))
key = SecureRandom.urlsafe_base64
end
#add new key to hashes that maintain records
#generated_keys.merge!({key => Time.now})
#all_keys.merge!(#generated_keys) { |key, v1, v2| v1 }
return key
end
And I use the generated keys here: (I need a random pair to be selected and allotted to user)
def get_available_key
if(generated_keys.empty?)
return "404. No keys available"
else
new_key = #generated_keys.to_a.sample(1)
#generated_keys.delete(new_key[0][0].to_s)
#blocked_keys.merge!({new_key[0][0].to_s => Time.now})
end
end
This is how I use it in Sinatra
api = KeyServer.new
get '/block_key' do
api.get_available_key
end
I tried the solution mentioned in this question but when I run this as part of my Sinatra server I obtain an Internal Server Error: No implicit conversion from Array to String
How do I make this work? Any other method to obtain a random pair from a Hash would be welcome.
To get a random element from a Hash to return as a Hash you could simply patch Hash to do this like
class Hash
def sample(n)
Hash[to_a.sample(n)]
end
end
Then call like
h = {a: 1, b: 2, c: 3}
h.sample(1)
#=> {b: 2}
h.sample(2)
#=> {:b=>2, :a=>1}
Note: I used Hash::[] for compatibility purposes in Ruby 2.X you could use to_h instead.
Other than that I think there might be a few more issues with your code and it's return values.
If I were to refactor your code the sample code above would not be needed I would instead go with something like it would be something like
def get_available_key
if(generated_keys.empty?)
{"error" => "404. No keys available"}
else
new_key = #generated_keys.keys.sample(1)
#generated_keys.delete(new_key)
#blocked_keys.merge!({new_key => Time.now})[new_key]
end
end
This way it will always respond with a Hash object for handling purposes and it need not worry about multidimensional arrays at all.
I would also change the initial code to be more like this
def create_new_key
key = SecureRandom.urlsafe_base64
purged_keys.include?(key) ? create_new_key : key
end
def generate_key
key = create_new_key
#add new key to hashes that maintain records
#generated_keys.merge!({key => Time.now})
#all_keys.merge!(#generated_keys) { |key, v1, v2| v1 }
key
end
def add_to_key_chain(length)
#generated_keys ||= {}
length.times do
create_new_key
end
end
Although I don't know what the purged_keys method looks like.
hash.to_a.sample evaluates to a two-element array where the first element is some key and the second is the corresponding value.
When you call delete you should be using hash.delete(new_key[0]) instead of hash.delete(new_key[0][0].to_s).
What's the best way of converting a dot notation path (or even an array of strings) into a nested hash key-value? Ex: I need to convert 'foo.bar.baz' equal to 'qux' like this:
{
'foo' => {
'bar' => {
'baz' => 'qux'
}
}
}
I've done this in PHP, but I managed that by creating a key in the array and then setting a tmp variable to that array key's value by reference so any changes would also take place in the array.
Try this
f = "root/sub-1/sub-2/file"
f.split("/").reverse.inject{|a,n| {n=>a}} #=>{"root"=>{"sub-1"=>{"sub-2"=>"file"}}}
I'd probably use recursion. For example:
def hasherizer(arr, value)
if arr.empty?
value
else
{}.tap do |hash|
hash[arr.shift] = hasherizer(arr, value)
end
end
end
This results in:
> hasherizer 'foo.bar.baz'.split('.'), 'qux'
=> {"foo"=>{"bar"=>{"baz"=>"qux"}}}
I like this method below which operates on itself (or your own hash class). It'll create new hash keys or reuse/append to existing keys in a hash to add or update the value.
# set a new or existing nested key's value by a dotted-string key
def dotkey_set(dottedkey, value, deep_hash = self)
keys = dottedkey.to_s.split('.')
first = keys.first
if keys.length == 1
deep_hash[first] = value
else
# in the case that we are creating a hash from a dotted key, we'll assign a default
deep_hash[first] = (deep_hash[first] || {})
dotkey_set(keys.slice(1..-1).join('.'), value, deep_hash[first])
end
end
Usage:
hash = {}
hash.dotkey_set('how.are.you', 'good')
# => "good"
hash
# => {"how"=>{"are"=>{"you"=>"good"}}}
hash.dotkey_set('how.goes.it', 'fine')
# => "fine"
hash
# => {"how"=>{"are"=>{"you"=>"good"}, "goes"=>{"it"=>"fine"}}}
I did something similar when I wrote an HTTP server that had to move all the parameters passed in the request into a multiple value hash which might contain arrays or strings or hashes...
You can look at the code for the Plezi server and framework... although the code over there deals with values surrounded with []...
It could possibly be adjusted like so:
def add_param_to_hash param_name, param_value, target_hash = {}
begin
a = target_hash
p = param_name.split(/[\/\.]/)
val = param_value
# the following, somewhat complex line, runs through the existing (?) tree, making sure to preserve existing values and add values where needed.
p.each_index { |i| p[i].strip! ; n = p[i].match(/^[0-9]+$/) ? p[i].to_i : p[i].to_sym ; p[i+1] ? [ ( a[n] ||= ( p[i+1].empty? ? [] : {} ) ), ( a = a[n]) ] : ( a.is_a?(Hash) ? (a[n] ? (a[n].is_a?(Array) ? (a << val) : a[n] = [a[n], val] ) : (a[n] = val) ) : (a << val) ) }
rescue Exception => e
warn "(Silent): parameters parse error for #{param_name} ... maybe conflicts with a different set?"
target_hash[param_name] = param_value
end
end
This should preserve existing values while adding new values if they exist.
The long line looks something like this when broken down:
def add_param_to_hash param_name, param_value, target_hash = {}
begin
# a will hold the object to which we Add.
# As we walk the tree we change `a`. we start at the root...
a = target_hash
p = param_name.split(/[\/\.]/)
val = param_value
# the following, somewhat complex line, runs through the existing (?) tree, making sure to preserve existing values and add values where needed.
p.each_index do |i|
p[i].strip!
# converts the current key string to either numbers or symbols... you might want to replace this with: n=p[i]
n = p[i].match(/^[0-9]+$/) ? p[i].to_i : p[i].to_sym
if p[i+1]
a[n] ||= ( p[i+1].empty? ? [] : {} ) # is the new object we'll add to
a = a[n] # move to the next branch.
else
if a.is_a?(Hash)
if a[n]
if a[n].is_a?(Array)
a << val
else
a[n] = [a[n], val]
end
else
a[n] = val
end
else
a << val
end
end
end
rescue Exception => e
warn "(Silent): parameters parse error for #{param_name} ... maybe conflicts with a different set?"
target_hash[param_name] = param_value
end
end
Brrr... Looking at the code like this, I wonder what I was thinking...
Edit: The issue is being unable to get the quantity of arrays within the hash, so it can be, x = amount of arrays. so it can be used as function.each_index{|x| code }
Trying to use the index of the amount of rows as a way of repeating an action X amount of times depending on how much data is pulled from a CSV file.
Terminal issued
=> Can't convert symbol to integer (TypeError)
Complete error:
=> ~/home/tests/Product.rb:30:in '[]' can't convert symbol into integer (TypeError) from ~home/tests/Product.rub:30:in 'getNumbRel'
from test.rb:36:in '<main>'
the function is that is performing the action is:
def getNumRel
if defined? #releaseHashTable
return #releaseHashTable[:releasename].length
else
#releaseHashTable = readReleaseCSV()
return #releaseHashTable[:releasename].length
end
end
The csv data pull is just a hash of arrays, nothing snazzy.
def readReleaseCSV()
$log.info("Method "+"#{self.class.name}"+"."+"#{__method__}"+" has started")
$log.debug("reading product csv file")
# Create a Hash where the default is an empty Array
result = Array.new
csvPath = "#{File.dirname(__FILE__)}"+"/../../data/addingProdRelProjIterTestSuite/releaseCSVdata.csv"
CSV.foreach(csvPath, :headers => true, :header_converters => :symbol) do |row|
row.each do |column, value|
if "#{column}" == "prodid"
proHash = Hash.new { |h, k| h[k] = [ ] }
proHash['relid'] << row[:relid]
proHash['releasename'] << row[:releasename]
proHash['inheritcomponents'] << row[:inheritcomponents]
productId = Integer(value)
if result[productId] == nil
result[productId] = Array.new
end
result[productId][result[productId].length] = proHash
end
end
end
$log.info("Method "+"#{self.class.name}"+"."+"#{__method__}"+" has finished")
#productReleaseArr = result
end
Sorry, couldn't resist, cleaned up your method.
# empty brackets unnecessary, no uppercase in method names
def read_release_csv
# you don't need + here
$log.info("Method #{self.class.name}.#{__method__} has started")
$log.debug("reading product csv file")
# you're returning this array. It is not a hash. [] is preferred over Array.new
result = []
csvPath = "#{File.dirname(__FILE__)}/../../data/addingProdRelProjIterTestSuite/releaseCSVdata.csv"
CSV.foreach(csvPath, :headers => true, :header_converters => :symbol) do |row|
row.each do |column, value|
# to_s is preferred
if column.to_s == "prodid"
proHash = Hash.new { |h, k| h[k] = [ ] }
proHash['relid'] << row[:relid]
proHash['releasename'] << row[:releasename]
proHash['inheritcomponents'] << row[:inheritcomponents]
# to_i is preferred
productId = value.to_i
# this notation is preferred
result[productId] ||= []
# this is identical to what you did and more readable
result[productId] << proHash
end
end
end
$log.info("Method #{self.class.name}.#{__method__} has finished")
#productReleaseArr = result
end
You haven't given much to go on, but it appears that #releaseHashTable contains an Array, not a Hash.
Update: Based on the implementation you posted, you can see that productId is an integer and that the return value of readReleaseCSV() is an array.
In order to get the releasename you want, you have to do this:
#releaseHashTable[productId][n][:releasename]
where productId and n are integers. Either you'll have to specify them specifically, or (if you don't know n) you'll have to introduce a loop to collect all the releasenames for all the products of a particular productId.
This is what Mark Thomas meant:
> a = [1,2,3] # => [1, 2, 3]
> a[:sym]
TypeError: can't convert Symbol into Integer
# here starts the backstrace
from (irb):2:in `[]'
from (irb):2
An Array is only accessible by an index like so a[1] this fetches the second element from the array
Your return a an array and thats why your code fails:
#....
result = Array.new
#....
#productReleaseArr = result
# and then later on you call
#releaseHashTable = readReleaseCSV()
#releaseHashTable[:releasename] # which gives you TypeError: can't convert Symbol into Integer