I am trying to write e ruby script that query an elasticsearch database and build a report on it. I am using jbuilder to build the query string like this:
require 'elasticsearch'
require 'date'
require 'jbuilder'
client = Elasticsearch::Client.new log: true, host: 'x.x.x.x', request_timeout: 10
filter_conditions = {}
filter_conditions['must'] = []
filter_conditions['should'] = []
filter_conditions['must'] << Jbuilder.encode do |json|
json.term do
json._type 'httpry-log'
end
end
filter_conditions['must'] << Jbuilder.encode do |json|
json.range do
json.set! '#timestamp' do
_now = DateTime.now
json.gte (_now - 1.00/24).strftime('%Q').to_i
json.lte _now.strftime('%Q').to_i
json.format 'epoch_millis'
end
end
end
query = Jbuilder.encode do |json|
json.size 10
json.query do
json.bool do
json.must do
json.array!(filter_conditions['must'])
end
end
end
end
puts query
But here is the result I get for the query:
{"size":10,"query":{"bool":{"must":["{\"term\":{\"_type\":\"httpry-log\"}}","{\"range\":{\"#timestamp\":{\"gte\":1477919154057,\"lte\":1477922754057,\"format\":\"epoch_millis\"}}}"]}}}
How to get the unscaped version of the inner array inside the main json output?
Thanks in advance,
Assuming that the query returns the same object and key each time:
a = your_query_hash
# The following map! will parse your string array into an array in ruby
a[:query][:bool][:must].map! { |arr| JSON.parse(arr) }
If the query does not return the same object and key each time, I'd suggest writing a recursive method that parses each value.
Related
Simple parser which turned out to be much harder than i thought. I need a string parser to convert nested fields to ruby object. In my case api response will only return desired fields.
Given
Parser.parse "album{name, photo{name, picture, tags}}, post{id}"
Desired output or similar
{album: [:name, photo: [:name, :picture, :tags]], post: [:id]}
Any thoughts?
Wrote my own solution
module Parser
extend self
def parse str
parse_list(str).map do |i|
extract_item_fields i
end
end
def extract_item_fields item
field_name, fields_str = item.scan(/(.+?){(.+)}/).flatten
if field_name.nil?
item
else
fields = parse_list fields_str
result = fields.map { |field| extract_item_fields(field) }
{ field_name => result }
end
end
def parse_list list
return list if list.nil?
list.concat(',').scan(/([^,{}]+({.+?})?),/).map(&:first).map(&:strip)
end
end
str = 'album{name, photo{name, picture, tags}}, post{id}'
puts Parser.parse(str).inspect
# => [{"album"=>["name", {"photo"=>["name", "picture", "tags"]}]}, {"post"=>["id"]}]
Here's an extract of the code that I am using:
def retrieve(user_token, quote_id, check="quotes")
end_time = Time.now + 15
match = false
until Time.now > end_time || match
#response = http_request.get(quote_get_url(quote_id, user_token))
eval("match = !JSON.parse(#response.body)#{field(check)}.nil?")
end
match.eql?(false) ? nil : #response
end
private
def field (check)
hash = {"quotes" => '["quotes"][0]',
"transaction-items" => '["quotes"][0]["links"]["transactionItems"]'
}
hash[check]
end
I was informed that using eval in this manner is not good practice. Could anyone suggest a better way of dynamically checking the existence of a JSON node (field?). I want this to do:
psudo: match = !JSON.parse(#response.body) + dynamic-path + .nil?
Store paths as arrays of path elements (['quotes', 0]). With a little helper function you'll be able to avoid eval. It is, indeed, completely inappropriate here.
Something along these lines:
class Hash
def deep_get(path)
path.reduce(self) do |memo, path_element|
return unless memo
memo[path_element]
end
end
end
path = ['quotes', 0]
hash = JSON.parse(response.body)
match = !hash.deep_get(path).nil?
I have a query string that looks as follows:
http://localhost:3000/events?appointment_practices%5B10%5D=Injury&appointment_practices%5B18%5D=Immigration&appointment_practices%5B8%5D=Bankruptcy
appointment_practices is actually a hash I inserted into the query string during a redirect:
appointment_practices = practices.reduce({}) do |acc, practice|
acc[practice.id] = practice.class.name
acc
end
redirect_to events_path(appointment_practices: appointment_practices)
Now I want to parse that query string. When I tried to parse it with decode_www_form, it returns an array with a nil element:
[nil]
This is the code that is giving me the nil element:
#http_refer = #_env['HTTP_REFERER']
begin
uri = URI.parse #http_refer
practices = Hash[URI::decode_www_form(uri.query)].values_at('appointment_practices')
puts "practices: #{practices}"
rescue StandardError
end
I am trying to extract the hash. For example, in appointment_practices%5B10%5D=Injury, the id is 10 and the practice is Injury.
What other options do I have besides regex?
You can use Rack::Utils.parse_nested_query:
require 'uri'
require 'rack'
uri = URI.parse('http://localhost:3000/events?appointment_practices%5B10%5D=Injury&appointment_practices%5B18%5D=Immigration&appointment_practices%5B8%5D=Bankruptcy')
Rack::Utils.parse_nested_query(uri.query)
#=> {"appointment_practices"=>{"10"=>"Injury", "18"=>"Immigration", "8"=>"Bankruptcy"}}
I would like to use the Google bigquery gem (https://rubygems.org/gems/bigquery) to create an Array of table names. So far, this is what I have written:
require 'json'
bqRepsonse = bq.tables('myDataSet')
bqRepsonseCleaned = bqRepsonse.to_s.gsub("=>", ":")
data = JSON.parse(bqRepsonseCleaned)
tableListing = []
data["tableID"]["type"].each do |item|
case item["type"]
when 'TABLE'
bqTableList << item["tableId"]
else
end
end
If I print bqResponse, I get this result:
[{"kind"=>"bigquery#table",
"id"=>"curious-idea-532:dataset_test_4.TableA",
"tableReference"=>{"projectId"=>"curious-idea-532",
"datasetId"=>"dataset_test_4", "tableId"=>"TableA"}, "type"=>"TABLE"},
{"kind"=>"bigquery#table",
"id"=>"curious-idea-532:dataset_test_4.TableB",
"tableReference"=>{"projectId"=>"curious-idea-532",
"datasetId"=>"dataset_test_4", "tableId"=>"TableB"}, "type"=>"TABLE"},
{"kind"=>"bigquery#table",
"id"=>"curious-idea-532:dataset_test_4.TableC",
"tableReference"=>{"projectId"=>"curious-idea-532",
"datasetId"=>"dataset_test_4", "tableId"=>"TableC"}, "type"=>"TABLE"},
{"kind"=>"bigquery#table",
"id"=>"curious-idea-532:dataset_test_4.TableD",
"tableReference"=>{"projectId"=>"curious-idea-532",
"datasetId"=>"dataset_test_4", "tableId"=>"TableD"}, "type"=>"TABLE"}]
And running the code throws and error
`[]': no implicit conversion of String into Integer (TypeError)
Not sure where to correct this. My desired outcome is:
tableListing = ["TableA","TableB","TableC","TableD"]
Thanks in advance for your advice.
Try this:
require 'json'
string = '[{"kind": "bigquery#table", "id": "curious-idea-532:dataset_test_4.TableA", "tableReference" : {"projectId":"curious-idea-532", "datasetId":"dataset_test_4", "tableId":"TableA"}, "type":"TABLE"}, {"kind":"bigquery#table", "id":"curious-idea-532:dataset_test_4.TableB", "tableReference":{"projectId":"curious-idea-532", "datasetId":"dataset_test_4", "tableId":"TableB"}, "type":"TABLE"}, {"kind":"bigquery#table", "id":"curious-idea-532:dataset_test_4.TableC", "tableReference":{"projectId":"curious-idea-532", "datasetId":"dataset_test_4", "tableId":"TableC"}, "type":"TABLE"}, {"kind":"bigquery#table", "id":"curious-idea-532:dataset_test_4.TableD", "tableReference":{"projectId":"curious-idea-532", "datasetId":"dataset_test_4", "tableId":"TableD"}, "type":"TABLE"}]'
data = JSON.parse(string)
tableListing = []
# Here we are iterating over the data instead of its child element
data.each do |item|
case item["type"]
when 'TABLE'
tableListing << item["tableReference"]["tableId"]
else
end
end
puts tableListing
I implemeting async thread manager and I want to pass reference to thread, where it should save the results of his work. And then when all thread finished i will handle all results.
What I need is to know how to work with 'references'.
lets assume I have variable result (or hash[:result1]), I want to pass it to the thread like
def test_func
return 777;
end
def thread_start(result)
Thread.new do
result = test_func;
end
end
and I want is to get following result
result = 555
thread_start(result);
#thread_wait_logic_there
result == 777; #=> true
hash = {:res1 => 555};
thread_start(hash[:res1])
#thread_wait_logic_there
hash[:res1]==777 #=> true
what should I chnage in my code to make it work?
Ruby version is 1.9.3
You can pass entrire hash to function:
def test_func
return 777;
end
def thread_start(hash, key)
Thread.new do
hash[key] = test_func;
end
end
Then this will work:
hash = {:res1 => 555};
thread_start(hash, :res1)
hash[:res1]==777 #=> true
Also if you want to be sure that you getting result after computations finished you must wait for thread, like this:
hash = {:res1 => 555};
thread_start(hash, :res1).join
hash[:res1]==777 #=> true
Edit: Added key,join