I have a web service that returns a json in the following format:
[
{
"key": "linux.ubuntu.ip",
"value": "10.10.10.10"
},
{
"key": "linux.ubuntu.hostname",
"value": "stageubuntu"
}
]
I have a ruby code that makes a call to this service and gets the json. Deep in this code, there is a variable configure of type Hashie::Mash.
I want to achieve this:
configure.linux.ubuntu.ip = 10.10.10.10 [Hashie::Mash]
configure.linux.ubuntu.hostname = stageubuntu [Hashie::Mash]
Could anybody tell me if it is possible to achieve this (w.r.t to the json output that I have)? If so, what is the best method to do it?
To get a JSON string to a Hashie::Mash object you can simply do:
require 'json'
require 'hashie'
json_str = '{ "foo": "bar" }'
ruby_hash = JSON.parse(json_str)
Hashie::Mash.new(ruby_hash)
For this specific problem, though not ideal (but we all have our unique use cases) you're needing to parse to an array, then extract the 'key's into nested Hashie::Mash objects of some unknown depth.
require 'json'
require 'hashie'
json_str = <<-JSON
[
{
"key": "linux.ubuntu.ip",
"value": "10.10.10.10"
},
{
"key": "linux.ubuntu.hostname",
"value": "stageubuntu"
}
]
JSON
parsed_arr = JSON.parse(json_str)
#=> [{"key"=>"linux.ubuntu.ip", "value"=>"10.10.10.10"}, {"key"=>"linux.ubuntu.hostname", "value"=>"stageubuntu"}]
configure = parsed_arr.map do |parsed_hash|
method_chain = parsed_hash['key'].split('.')
init_value = Hashie::Mash.new(method_chain.pop => parsed_hash['value'])
method_chain.reverse.inject(init_value) do |ret_value, method_name|
Hashie::Mash.new(method_name => ret_value)
end
end.inject(:merge) # <-- hashie is allows you to perform a deep merge into a single object here.
# you can now do
configure.linux.ubuntu.ip = 10.10.10.10 [Hashie::Mash]
#=> "10.10.10.10"
configure.linux.ubuntu.hostname
#=> "stageubuntu"
Related
My scenario is based on Gmail API.
I've learned that email messages can have their message parts deeply or shallowly nested based upon varying factors, but mostly the presence of attachments.
I'm using the Google API Ruby Client gem, so I'm not working with JSON, I'm getting objects with all the same information, but I think the JSON representation makes it easier to understand my issue.
A simple message JSON response looks like this (one parts array with 2 hashes inside it):
{
"id": "175b418b1ff69896",
"snippet": "COVID-19: Resources to help your business manage through uncertainty 20 Liters 500 PEOPLE FOUND YOU ON GOOGLE Here are the top search queries used to find you: 20 liters used by 146 people volunteer",
"payload": {
"parts": [
{
"mimeType": "text/plain",
"body": {
"data": "Hey, you found the body of the email! I want this!"
}
},
{
"mimeType": "text/html",
"body": {
"data": "<div>I actually don't want this</div>"
}
}
]
}
}
The value I want is not that hard to get:
response.payload.parts.each do |part|
#body_data = part.body.data if part.mime_type == 'text/plain'
end
BUT The JSON response of a more complex email message with attachments looks something like this (now parts nests itself 3 levels deep):
{
"id": "175aee26de8209d2",
"snippet": "snippet text...",
"payload": {
"parts": [
{
"mimeType": "multipart/related",
"parts": [
{
"mimeType": "multipart/alternative",
"parts": [
{
"mimeType": "text/plain",
"body": {
"data": "hey, you found me! This is what I want!!"
}
},
{
"mimeType": "text/html",
"body": {
"data": "<div>I actually don't want this one.</div>"
}
}
]
},
{
"mimeType": "image/jpeg"
},
{
"mimeType": "image/png"
},
{
"mimeType": "image/png"
},
{
"mimeType": "image/jpeg"
},
{
"mimeType": "image/png"
},
{
"mimeType": "image/png"
}
]
},
{
"mimeType": "application/pdf"
}
]
}
}
And looking at a few other messages, the object can vary from 1 to 5 levels (maybe more) of parts
I need to loop over an unknown number of parts and then loop over an unknown number of nested parts and the repeat this again until I reach the bottom, hopefully finding the thing I want.
Here's my best attempt:
def trim_response(response)
# remove headers I don't care about
response.payload.headers.keep_if { |header| #valuable_headers.include? header.name }
# remove parts I don't care about
response.payload.parts.each do |part|
# parts can be nested within parts, within parts, within...
if part.mime_type == #valuable_mime_part && part.body.present?
#body_data = part.body.data
break
elsif part.parts.present?
# there are more layers down
find_body(part)
end
end
end
def find_body(part)
part.parts.each do |sub_part|
if sub_part.mime_type == #valuable_mime_part && sub_part.body.present?
#body_data = sub_part.body.data
break
elsif sub_part.parts.present?
# there are more layers down
######### THIS FEELS BAD!!! ###########
find_body(sub_part)
end
end
end
Yep, there's a method calling itself. I know, that's why I'm here.
This does work, I've tested it on a few dozen messages, but... there has to be a better, DRY-er way to do this.
How do I recursively loop and then move down a level and loop again in a DRY fashion when I don't know how deep the nesting goes?
No need to go through all this pain. Just keep diving in the parts dictionary until you find the first value where there is no parts anymore. At this moment you have the final parts in your parts variable.
Code:
reponse = {"id" => "175aee26de8209d2","snippet" => "snippet text...","payload" => {"parts" => [{"mimeType" => "multipart/related","parts" => [{"mimeType" => "multipart/alternative","parts" => [{"mimeType" => "text/plain","body" => {"data" => "hey, you found me! This is what I want!!"}},{"mimeType" => "text/html","body" => {"data" => "<div>I actually don't want this one.</div>"}}]},{"mimeType" => "image/jpeg"}]},{"mimeType" => "application/pdf"}]}}
parts = reponse["payload"]
parts = (parts["parts"].send("first") || parts["parts"]) while parts["parts"]
data = parts["body"]["data"]
puts data
Output:
hey, you found me! This is what I want!!
You can compute the desired result using recursion.
def find_it(h, top_key, k1, k2, k3)
return nil unless h.key?(top_key)
recurse(h[top_key], k1, k2, k3)
end
def recurse(h, k1, k2, k3)
return nil unless h.key?(k1)
h[k1].each do |g|
v = g.dig(k2,k3) || recurse(g, k1 , k2, k3)
return v unless v.nil?
end
nil
end
See Hash#dig.
Let h1 and h2 equal the two hashes given in the example1. Then:
find_it(h1, :payload, :parts, :body, :data)
#=> "Hey, you found the body of the email! I want this!"
find_it(h2, :payload, :parts, :body, :data)
#=> "hey, you found me! This is what I want!!"
1. The hash h[:payload][:parts].last #=> { "mimeType": "application/pdf" } appears to contain hidden characters that are causing a problem. I therefore removed that hash from h2.
I have a JSONB payload in my database. This payload is from a GraphQL query of the shopify_api.
For the shop_order below, I am trying to query for the name of the fourth order in the node.
shop_order = {"data":{"orders":{"edges":[{"node":{"id":"gid://shopify/Order/2228134674512","name":"#1001","createdAt":"2020-05-01T18:46:04Z","shippingAddress":{"address1":"1234 Long Avenue, 2N","address2":"","city":"Chicago","province":"Illinois","provinceCode":"IL","zip":"55555"}}},{"node":{"id":"gid://shopify/Order/2239643451472","name":"#1002","createdAt":"2020-05-05T14:40:36Z","shippingAddress":{"address1":"1234 Long Avenue","address2":"2N","city":"Chicago","province":"Illinois","provinceCode":"IL","zip":"55555"}}},{"node":{"id":"gid://shopify/Order/2239950323792","name":"#1003","createdAt":"2020-05-05T16:35:38Z","shippingAddress":{"address1":"1234 Long Avenue","address2":"2N","city":"Chicago","province":"Illinois","provinceCode":"IL","zip":"55555"}}},{"node":{"id":"gid://shopify/Order/2239959105616","name":"#1004","createdAt":"2020-05-05T16:38:27Z","shippingAddress":{"address1":"1234 Long Avenue","address2":"2N","city":"Chicago","province":"Illinois","provinceCode":"IL","zip":"55555"}}}]}},"casted_data":{},"errors":[]}
order = shop_order[:data][:orders][:edges][3]
puts order
response > {:node=>{:id=>"gid://shopify/Order/2239959105616", :name=>"#1004", :createdAt=>"2020-05-05T16:38:27Z", :shippingAddress=>{:address1=>"1234 Long Avenue", :address2=>"2N", :city=>"Chicago", :province=>"Illinois", :provinceCode=>"IL", :zip=>"55555"}}}
order_to_a = shop_order[:data][:orders][:edges][3].to_a
puts order_to_a
response > node
{:id=>"gid://shopify/Order/2239959105616", :name=>"#1004", :createdAt=>"2020-05-05T16:38:27Z", :shippingAddress=>{:address1=>"1234 Long Avenue", :address2=>"2N", :city=>"Chicago", :province=>"Illinois", :provinceCode=>"IL", :zip=>"55555"}}
How do I query and display a specific value from a key that is inside a node?
It's not entirely clear what your intent is, but your access of elements in a hash can be streamlined using dig:
shop_order = {
"data": {
"orders": {
"edges": [
{}, {}, {}, {
"node": {
"name": '#1004',
"shippingAddress": {
"zip": '55555'
}
}
}
]
}
}
}
Access data using:
order = shop_order.dig(:data, :orders, :edges)[3]
# => {:node=>{:name=>"#1004", :shippingAddress=>{:zip=>"55555"}}}
or:
order = shop_order.dig(:data, :orders, :edges, 3)
# => {:node=>{:name=>"#1004", :shippingAddress=>{:zip=>"55555"}}}
How do I query and display a specific value from a key that is inside a node?
Huh? If you want information inside order, do the same sort of thing:
order.dig(:node, :name) # => "#1004"
order.dig(:node, :shippingAddress, :zip) # => "55555"
or:
shop_order.dig(:data, :orders, :edges, 3, :node, :name) # => "#1004"
shop_order.dig(:data, :orders, :edges, 3, :node, :shippingAddress, :zip) # => "55555"
Many times when we're walking through a complex hash of arrays we point to the array in a variable and then work from that point. It's similar to putting your finger on a page in a recipe, so we can go back to it quickly. We do the same when parsing HTML/XML, parsed JSON and YAML, etc.
Is it possible to run a custom validator against each element of array in Grape? I know I can validate whole array with my validator, but I think error messages would be better if I use it for each element.
My parameters look like this:
"conditions": [
{
"field": "interests",
"operator": "any",
"value": ['cars', 'cats']
},
{
"field": "age",
"operator": "gt",
"value": 25
}
]
With requires :conditions, type: Array, valid_conditions: true the validator is run for the whole array. Is it best I can get?
This is totally possible, You can assert the value for specific keys in a response.
assert_equal some_obj[0].first[1], "interests"
Here is this same thing in irb
🇱🇷 Success weeds out the uncommitted ~ irb
2.2.3 :001 > a = [{:field=>"interests", :operator=>"any", :value=>["cars", "cats"]}]
=> [{:field=>"interests", :operator=>"any", :value=>["cars", "cats"]}]
2.2.3 :002 > a[0].first
=> [:field, "interests"]
2.2.3 :003 > a[0].first[1]
=> "interests"
2.2.3 :004 >
Yes it's possible, but you have to use a custom validator.
Here is an example
class Validator < Grape::Validations::Base
def validate_param!(attr_name, params)
unless params[attr_name].each { //your code here }
fail Grape::Exceptions::Validation, params: [#scope.full_name(attr_name)], message: 'your message'
end
end
end
You would then use it like this:
requires :conditions, type: Array, validator: true
I have a JSON array that looks like this:
response = {
"items"=>[
{
"tags"=>[
"random"
],
"timestamp"=>12345,
"storage"=>{
"url"=>"https://example.com/example",
"key"=>"mykeys"
},
"envelope"=>{
},
"log-level"=>"info",
"id"=>"random_id_test_1",
"campaigns"=>[
],
"user-variables"=>{
},
"flags"=>{
"is-test-mode"=>false
},
"message"=>{
"headers"=>{
"to"=>"random#example.com",
"message-id"=>"foobar#example.com",
"from"=>"noreply#example.com",
"subject"=>"new subject"
},
"attachments"=>[
],
"recipients"=>[
"result#example.com"
],
"size"=>4444
},
"event"=>"stored"
},
{
"tags"=>[
"flowerPower"
],
"timestamp"=>567890,
"storage"=>{
"url"=>"https://yahoo.com",
"key"=>"some_really_cool_keys_go_here"
},
"envelope"=>{
},
"log-level"=>"info",
"id"=>"some_really_cool_ids_go_here",
"campaigns"=>[
],
"user-variables"=>{
},
"flags"=>{
"is-test-mode"=>false
},
"message"=>{
"headers"=>{
"to"=>"another_great#example.com",
"message-id"=>"email_id#example.com",
"from"=>"from#example.com",
"subject"=>"email_looks_good"
},
"attachments"=>[
],
"recipients"=>[
"example#example.com"
],
"size"=>2222
},
"event"=>"stored"
}]
}
I am trying to obtain the "storage" "url" based on the "to" email.
How do I iterate through this array where x is just the element in the array
response['items'][x]["message"]["headers"]["to"]
Once I find the specific email that I need, it will stop and return the value of x which is the element number.
I was going to use that value for x and call response['items'][x]['storage']['url']
which will return the string for the URL.
I thought about doing this but there's gotta be a better way:
x = 0
user_email = another_great#example.com
while user_email != response['items'][x]["message"]["headers"]["to"] do
x+=1
value = x
puts value
end
target =
response['items'].detect do |i|
i['message']['headers']['to'] == 'another_great#example.com'
end
then
target['storage']['url']
This is another option by creating Hash with key of to's email. And on basis of it fetch required information like this:
email_hash = Hash.new
response["items"].each do |i|
email_hash[i["message"]["headers"]["to"]] = i
end
Now if you want to fetch "storage" "url" then simply do:
user_email = "another_great#example.com"
puts email_hash[user_email]["storage"]["url"] if email_hash[user_email]
#=> "https://yahoo.com"
You can use it as #Satoru suggested. As a suggestion, if you use case involves complex queries on json data (more complex than this), then you can store your data in mongodb, and can elegantly query anything.
I am trying to write a Jekyll extension that will embed comments from a Blogger blog.
I am able to fetch the comments feed as JSON, and process it enough to pull out the total number of comments. However, I have not figured out how to process each comment in the feed.
json_url = "http://www.blogger.com/feeds/8505008/593465383646513269/comments/default/?alt=json"
json_rep = Net::HTTP.get_response(json_url)
json_rep = JSON.parse(json_rep.body)
json_rep['feed']['openSearch$totalResults']['$t'] # => "4"
json_rep['feed']['entry'].class # => Array
json_rep['feed']['entry'].length
# => Liquid Exception: undefined method `length' for nil:NilClass in post
This is my first time writing any code in Ruby. What am I doing wrong?
Here are the relevant parts of the JSON I am trying to parse.
{
"feed": {
"openSearch$totalResults": {
"$t": "4"
},
"entry": [
{
"id": {
"$t": "tag:blogger.com,1999:blog-8505008.post-491866073982779922"
},
"published": {
"$t": "2013-01-08T15:23:47.322-04:00"
},
"content": {
"type": "html",
"$t": "Recently, my sister has updated it more than I have. \u00dcber-fail on my part. :p"
}
}
]
}
}
This is what you should look at doing:
require 'rubygems'
require 'json'
require 'net/http'
require 'net/https'
require 'uri'
url = "http://www.blogger.com/feeds/8505008/593465383646513269/comments/default/?alt=json"
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Get.new(uri.request_uri)
response = http.request(request)
json_rep = JSON.parse(response.body)
puts json_rep['feed']['openSearch$totalResults']['$t']
entries = json_rep['feed']['entry']
entries.each do |entry|
puts entry["id"]["$t"]
#add what ever code you like here
end
This outputs:
4
tag:blogger.com,1999:blog-8505008.post-491866073982779922
tag:blogger.com,1999:blog-8505008.post-4792479891671746788
tag:blogger.com,1999:blog-8505008.post-4766604955439002209
tag:blogger.com,1999:blog-8505008.post-5484003770204916000